Study finds polynomial convergence rate for Farey sequences linked to Riemann hypothesis.
problem Understanding convergence rates of maximum mean discrepancies for Farey sequences.
method Identifying positive-semidefinite kernels and their polynomial convergence rates.
result Polynomial convergence rate of maximum mean discrepancies of Farey sequences is equivalent to the Riemann hypothesis.
A new learning method uses data to learn from large model sets.
problem Learning with large sets of candidate models where uniform convergence is hard.
method Data-dependent learning that incorporates empirical data less reliant on prior assumptions.
result Demonstrates improved generalization in various learning assumptions.
Generates positive examples from noisy data streams.
problem Learning from noisy example streams in hypothesis classes.
method Extending results from previous studies to account for noise.
result Conditions for noisily generatable binary hypothesis classes.
Given a hypothesis space, the large volume principle by Vladimir Vapnik prioritizes equivalence classes according to their volume in the hypothesis space. The volume approximation has hitherto been successfully applied to binary learning problems. In this paper, we extend it naturally to a more general definition which…
Learnable multiclass hypothesis classes don't always have a sample compression scheme of fixed size.
problem The limitation of sample compression schemes for multiclass hypothesis classes.
method Analysis of DS dimension and sample compression schemes.
result Learnable multiclass hypothesis classes do not always have a sample compression scheme of fixed size.
We consider a problem of risk estimation for large-margin multi-class classifiers. We propose a novel risk bound for the multi-class classification problem. The bound involves the marginal distribution of the classifier and the Rademacher complexity of the hypothesis class. We prove that our bound is tight in the numbe…
Comparative learning combines realizable and agnostic settings for two hypothesis classes, reducing sample complexity.
problem Learning with two hypothesis classes in a more general setting than single hypothesis classes.
method Introduces comparative learning, defines mutual VC dimension and Littlestone dimension, and applies insights to multiaccuracy and multicalibration.
result Sample complexity of comparative learning is characterized by mutual VC dimension and Littlestone dimension.
We show that all groups in a very large class of Coxeter groups are locally quasiconvex and have uniform membership problem solvable in quadratic time. If a group in the class satisfies a further hypothesis it is subgroup separable and relevant homomorphisms are also calculable in quadratic time. The algorithm also dec…
We give in this paper bounds for the Morse indices of a large class of simple geodesics on a surface with a generic metric. To our knowledge these bounds are the first that use only the generic hypothesis on the metric.
New algorithm for online omniprediction with strong guarantees for continuous hypothesis classes.
problem Online adversarial learning with continuous hypothesis classes.
method Developed an oracle-efficient online multicalibration algorithm for infinite benchmark classes.
result First efficient online omnipredictor with strong guarantees for Lipschitz convex loss functions.
In this work, we give the first algorithms for tolerant testing of nontrivial classes in the active model: estimating the distance of a target function to a hypothesis class C with respect to some arbitrary distribution D, using only a small number of label queries to a polynomial-sized pool of unlabeled examples drawn…
New algorithm reduces sample complexity for multi-distribution learning.
problem Achieving data-efficient multi-distribution learning with robustness and fairness.
method Proposes a novel algorithm with sample complexity (d+k)/varepsilon^2 for Vapnik-Chervonenkis (VC) dimension d, matching lower bounds.
result Algorithm matches best-known lower bound and resolves open problems in COLT 2023.
New algorithms for multitask learning with long-term memory.
problem Learning from tasks partitioned into unknown segments with associated hypotheses.
method Online multitask learning algorithms exploiting segmentation and hypothesis association.
result Regret bounds and efficient algorithms for various hypothesis classes.
Framework for online hypothesis testing across various data types.
problem Testing various nonparametric hypotheses in data streams.
method Unified framework using operators on data distributions, leveraging ML models.
result Efficient, adaptive, and error-controlled sequential tests.
We consider the problem of reinforcement learning over episodes of a finite-horizon deterministic system and as a solution propose optimistic constraint propagation (OCP), an algorithm designed to synthesize efficient exploration and value function generalization. We establish that when the true value function lies wit…
SnapBoost uses random base hypothesis classes to improve gradient boosting performance.
problem Improving gradient boosting performance.
method Heterogeneous Newton Boosting Machine (HNBM) with variable base hypothesis classes.
result SnapBoost achieves better generalization loss than competing frameworks.
Analyzes the complexity of linear hypothesis sets using Rademacher complexity.
problem Understanding the complexity of linear hypothesis sets for various norms.
method Tight analysis of empirical Rademacher complexity for linear hypothesis classes with bounded weights.
result Improved bounds on Rademacher complexity for linear hypothesis sets, matching or improving existing results.
Estimates neural network errors for classification problems.
problem Binary and multi-class classification problems.
method Rademacher complexity estimates and direct approximation theorems.
result A priori error estimates for regularized loss functionals.
We provide a differentially private algorithm for hypothesis selection. Given samples from an unknown probability distribution P and a set of m probability distributions H, the goal is to output, in a ε-differentially private manner, a distribution from H whose total variation di…
A framework for hypothesis testing on attributed graphs using sampling.
problem Statistical testing on graph data, especially large attributed graphs.
method Sampling-based framework with PHASE and PHASEopt for accurate and efficient hypothesis testing.
result PHASE and PHASEopt improve accuracy and efficiency of hypothesis testing in attributed graphs.
Improved bounds on combining hypothesis classes for binary functions.
problem Understanding how to combine hypothesis classes for binary functions.
method Established upper bounds on Littlestone and threshold dimensions for combined classes.
result Upper bounds are nearly tight and give exponential improvements.
Multiple hypothesis testing is a core problem in statistical inference and arises in almost every scientific field. Given a set of null hypotheses H(n)=(H1,…,Hn), Benjamini and Hochberg introduced the false discovery rate (FDR), which is the expected proportion of false positives among rejected nu…
I study the behavior and the performance of the long-term forecasts issued by financial analysts with respect to the Extrapolation Hypothesis. That hypothesis states that investors, extrapolating from the firms' recent performances, are too optimistic about growth and large firms and too pessimistic about value and sma…
Logarithmic pruning simplifies lottery ticket hypothesis.
problem Finding efficient subnetworks in large neural networks.
method Logarithmic pruning approach to identify subnetworks.
result Randomly initialized subnetworks achieve comparable performance.
We study the question of learning an adversarially robust predictor. We show that any hypothesis class H with finite VC dimension is robustly PAC learnable with an improper learning rule. The requirement of being improper is necessary as we exhibit examples of hypothesis classes H with finite VC…
Improved financial market calibration reveals large excess volatility.
problem Large excess volatility in financial markets.
method Extended Chiarella model to handle long-term value drifts, calibrated on multiple asset classes.
result Large excess volatility (factor ≈ 4 for stock indices) and bimodal mispricing distribution.
Proposes a new approach to regression learning that addresses overfitting and underfitting.
problem Regression learning issues, including overfitting and underfitting.
method Introduces epsilon-Confidence Approximately Correct (epsilon CoAC) framework using Kullback Leibler divergence.
result Demonstrates improved learnability and accuracy compared to cross-validation.
The Weyl curvature hypothesis of Penrose attempts to explain the high homogeneity and isotropy, and the very low entropy of the early universe, by conjecturing the vanishing of the Weyl tensor at the Big-Bang singularity. In previous papers it has been proposed an equivalent form of Einstein's equation, which extends i…
We show that the disagreement coefficient of certain smooth hypothesis classes is O(m), where m is the dimension of the hypothesis space, thereby answering a question posed in \cite{friedman09}.
New algorithms for hypothesis testing in high-dimensional data are shown to be effective under various noisy conditions.
problem Testing high-dimensional probability measures under noisy conditions.
method Low coordinate degree functions (LCDF) using Efron-Stein decomposition.
result LCDF can effectively test high-dimensional probability measures under noisy channels, with efficacy depending on scalar Fisher information.
This work connects hardness of approximation and learning.
problem Hardness of approximation and learnability in machine learning.
method Shows a single hardness property implying both approximation and learning hardness.
result Obtains new results on hardness of approximation and learnability of specific functions.
New method quantifies inductive bias for machine learning tasks.
problem Quantifying the amount of inductive bias in machine learning models.
method Estimates inductive bias by modeling loss distribution of random hypotheses.
result Higher dimensional tasks require greater inductive bias.
The paper tests if LLMs' capabilities are executed by small subnetworks (circuits).
problem Understanding how LLMs execute their capabilities.
method Formalized criteria for circuits, developed hypothesis tests, applied to six circuits.
result Synthetic circuits align with idealized properties, while Transformer circuits vary in their alignment.
Develops GLRT for defending against adversarial attacks in hypothesis testing.
problem Adversarial attacks on machine learning models causing misclassification.
method Generalized likelihood ratio test applied to composite hypothesis testing problem.
result GLRT approach yields competitive robustness-accuracy tradeoff under various attacks.
This work characterizes when a hypothesis class can be k-list learned.
problem Characterizing when a hypothesis class can be k-list learned.
method Introducing the k-DS dimension and proving the equivalence of k-list learnability and the finiteness of the k-DS dimension.
result A hypothesis class is k-list learnable if and only if the k-DS dimension is finite.
StatLoRA uses statistical inference to allocate ranks in LoRA fine-tuning, improving performance.
problem Balancing efficiency, expressiveness, and generalization in LoRA rank allocation.
method Formulates LoRA rank allocation as a statistical hypothesis testing problem, using estimated p-values to determine component retention or pruning.
result StatLoRA achieves comparable or better performance than existing methods under matched rank budgets.
Unified framework for large-scale hypothesis testing with confounders.
problem Bias in large-scale hypothesis testing due to unmeasured confounders.
method Unified statistical estimation and inference framework that disentangles confounding effects and jointly estimates latent and primary effects.
result Effective Type-I error control and power in hypothesis testing.
Scientific discovery is limited by hypothesis redundancy, and hybrid methods can exploit non-local exploration.
problem Limitation of scientific discovery due to hypothesis redundancy.
method Hybrid discovery systems combining structured local search with LLM-generated non-local proposals.
result Hybrid methods can exploit non-local exploration when three geometric conditions co-occur.
End-to-end approach learns pseudo-distance for verifying example sets.
problem Verifying example sets from same or different classes.
method Jointly trains parametric pseudo-distance with encoder.
result End-to-end approach outperforms simple scorers and classifiers.
Optimal domain adaptation model using Fisher's Linear Discriminant.
problem Improving classification accuracy across different domains.
method Convex combination of source and target hypotheses, derived under 0-1 loss.
result Effective classifier can be computed without direct source task information.
We study the power of interactivity in local differential privacy. First, we focus on the difference between fully interactive and sequentially interactive protocols. Sequentially interactive protocols may query users adaptively in sequence, but they cannot return to previously queried users. The vast majority of exist…
The two-sample hypothesis testing problem is studied for the challenging scenario of high dimensional data sets with small sample sizes. We show that the two-sample hypothesis testing problem can be posed as a one-class set classification problem. In the set classification problem the goal is to classify a set of data …
Online learning is the process of answering a sequence of questions based on the correct answers to the previous questions. It is studied in many research areas such as game theory, information theory and machine learning. There are two main components of online learning framework. First, the learning algorithm also kn…
Gradient descent with large momentum finds flatter minima.
problem Understanding the effects of momentum in gradient descent.
method Empirical and theoretical analysis of gradient descent with large momentum.
result Large momentum leads to flatter minima than gradient descent.
Paper develops PAC verification for hypothesis classes and statistical algorithms.
problem Verifying machine learning models interactively.
method Develops interactive proof for PAC verification, proves lower bounds, and introduces a generalization.
result Improved protocol for verifying unions of intervals and statistical query algorithms.
Alternative hypothesis tests for class-conditional noise using local maximum likelihood.
problem Assessing label noise in supervised learning datasets.
method Proposes hypothesis tests based on local maximum likelihood estimation for nonparametric logistic regression.
result Shows improved applicability and flexibility of the proposed tests compared to parametric approaches.
Extends linear representation hypothesis to categorical and hierarchical concepts in LLMs.
problem Representing concepts without natural contrasts in large language models.
method Formalizes linear representation hypothesis for categorical and hierarchical concepts, proving relationships between concept hierarchy and representation geometry.
result Validated theoretical results on large language models, estimating representations for 900+ concepts.
Study robust online learning with adversarial perturbations.
problem Learning robust classifiers in the presence of adversarial perturbations.
method Formulated as an online learning problem, considered both realizable and agnostic learnability, defined new dimension controlling mistake/regret bounds.
result Showed new dimension controls mistake/regret bounds, generalized to multiclass hypothesis classes.