Selecting appropriate regularization coefficients is critical to performance with respect to regularized empirical risk minimization problems. Existing theoretical approaches attempt to determine the coefficients in order for regularized empirical objectives to be upper-bounds of true objectives, uniformly over a hypot…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “hypothesis space”
Study risk bounds for distributed ERM with general loss functions and hypothesis spaces.
problem Limited theoretical analysis for distributed ERM with general loss functions and hypothesis spaces.
method Derive tight risk bounds under assumptions on hypothesis space and loss function.
result Developed more general risk bound for distributed ERM without strong convexity restriction.
Finite resources limit false discovery rate control in structured hypothesis spaces.
problem Controlling false discovery rate in hypothesis testing with finite data and structured hypothesis spaces.
method Framework for exact FDR control and adaptive power maximization.
result Exact FDR control and adaptive power maximization.
The paper defines a hypothesis space for deep learning using DNNs.
problem Developing a mathematical framework for deep learning.
method Introducing a Banach space of functions of input variables based on DNNs, proving it's a RKBS, and establishing representer theorems for learning models.
result Solutions to learning problems can be expressed as finite sums of kernel expansions based on training data.
In this paper we address the problem of pool based active learning, and provide an algorithm, called UPAL, that works by minimizing the unbiased estimator of the risk of a hypothesis in a given hypothesis space. For the space of linear classifiers and the squared loss we show that UPAL is equivalent to an exponentially…
The effect of the choice of neural network depth and breadth on the size of its hypothesis spacecs.LG
We show that the number of unique function mappings in a neural network hypothesis space is inversely proportional to , where is the number of neurons in the hidden layer .
This paper introduces localized discrepancy theories for unsupervised domain adaptation.
problem Improving generalization bounds for unsupervised domain adaptation.
method Localized discrepancies defined on the hypothesis space after localization, leading to smaller and asymmetric values.
result Improved generalization bounds and sample complexity reduction.
The paper refutes the manifold hypothesis for image data and proposes the union of manifolds hypothesis.
problem The manifold hypothesis fails to capture the structure of image data.
method Empirical verification of the union of manifolds hypothesis on image datasets.
result Image data lies on a disconnected set with varying intrinsic dimensions.
We show that the disagreement coefficient of certain smooth hypothesis classes is , where is the dimension of the hypothesis space, thereby answering a question posed in \cite{friedman09}.
Human input has enabled autonomous systems to improve their capabilities and achieve complex behaviors that are otherwise challenging to generate automatically. Recent work focuses on how robots can use such input - like demonstrations or corrections - to learn intended objectives. These techniques assume that the huma…
New algorithms for multitask learning with long-term memory.
problem Learning from tasks partitioned into unknown segments with associated hypotheses.
method Online multitask learning algorithms exploiting segmentation and hypothesis association.
result Regret bounds and efficient algorithms for various hypothesis classes.
Scientific discovery is limited by hypothesis redundancy, and hybrid methods can exploit non-local exploration.
problem Limitation of scientific discovery due to hypothesis redundancy.
method Hybrid discovery systems combining structured local search with LLM-generated non-local proposals.
result Hybrid methods can exploit non-local exploration when three geometric conditions co-occur.
Generalization Bounds for Learning with Linear, Polygonal, Quadratic and Conic Side Knowledgestat.ML
In this paper, we consider a supervised learning setting where side knowledge is provided about the labels of unlabeled examples. The side knowledge has the effect of reducing the hypothesis space, leading to tighter generalization bounds, and thus possibly better generalization. We consider several types of side knowl…
Learning robot objective functions from human input has become increasingly important, but state-of-the-art techniques assume that the human's desired objective lies within the robot's hypothesis space. When this is not true, even methods that keep track of uncertainty over the objective fail because they reason about …
Controlling False Discovery in Arbitrarily Structured Hypothesis Spaces via Reproducing Kernelsstat.ME
New method controls false discoveries in structured hypothesis spaces.
problem Controlling false discoveries in large-scale, interconnected hypothesis spaces.
method Reproducing Kernel Hilbert Space (RKHS) optimization for structured FDR control.
result Unified framework for continuous domains, graphs, and hierarchies.
To reduce the label complexity in Agnostic Active Learning (A^2 algorithm), volume-splitting splits the hypothesis edges to reduce the Vapnik-Chervonenkis (VC) dimension in version space. However, the effectiveness of volume-splitting critically depends on the initial hypothesis and this problem is also known as target…
Paper relaxes assumptions for non-parametric estimation in pairwise learning.
problem Generalization performance of non-parametric estimation for pairwise learning.
method Significantly relaxes restrictive assumptions, constructs structured deep ReLU neural network, and designs targeted hypothesis space.
result Establishes a sharp oracle inequality for empirical minimizer with general hypothesis space for Lipschitz continuous pairwise losses.
Optimal rates for vector-valued regression on various norms.
problem Optimal rates for vector-valued ridge regression on continuous norms.
method Combining standard capacity assumptions with tensor product constructions of vector-valued interpolation spaces.
result Optimal rates for vector-valued ridge regression, independent of output space dimension.
Defines computable learning for binary classification over metric spaces.
problem Defines computable PAC learning for binary classification over computable metric spaces.
method Provides sufficient conditions for ERM learners to be computable and bounds the strong Weihrauch degree of an ERM learner.
result Gives a hypothesis class that does not admit any proper computable PAC learner with computable sample function.
A typical approach in estimating the learning rate of a regularized learning scheme is to bound the approximation error by the sum of the sampling error, the hypothesis error and the regularization error. Using a reproducing kernel space that satisfies the linear representer theorem brings the advantage of discarding t…
Kernel interpolation is inconsistent for norms with smoothness above a constant.
problem Inconsistency of kernel interpolation in reproducing kernel Hilbert spaces.
method Lower bounds for generalization error in Sobolev norms.
result Kernel interpolation is always inconsistent for norms with smoothness above a constant.
We give a topological condition for a generic sliced space to be globally hyperbolic, without any hypothesis on the lapse function, shift function and spatial metric.
Study shows diffusion models adapt to manifold hypothesis without dimensionality issues.
problem Empirical success of diffusion models in high-dimensional data.
method Developed a new framework connecting diffusion models to Gaussian Processes theory.
result Achieves rates independent of ambient dimension in terms of score learning and sampling complexity.
Develops bounds for deep learning risk via Hilbert coresets.
problem Risk estimation for complex deep learning models.
method Hilbert coreset approach for transductive risk bounds.
result Effective and meaningful bounds for deep neural networks.
Deep learning's anomalous generalization explained by standard frameworks.
problem Anomalous generalization in deep neural networks.
method Intuitive understanding and rigorous characterization using PAC-Bayes and countable hypothesis bounds.
result Deep learning's anomalous generalization can be explained by soft inductive biases.
On the inclusion of the quasiconformal Teichmüller space into the length-spectrum Teichmüller spacemath.GT
Given a surface of infinite topological type, there are several Teichmüller spaces associated with it, depending on the basepoint and on the point of view that one uses to compare different complex structures. This paper is about the comparison between the quasiconformal Teichmüller space and the length-spectrum Teichm…
This paper reviews the functional aspects of statistical learning theory. The main point under consideration is the nature of the hypothesis set when no prior information is available but data. Within this framework we first discuss about the hypothesis set: it is a vectorial space, it is a set of pointwise defined fun…
New approach combines geometric and probabilistic methods to estimate manifold dimension in high-dimensional data.
problem Estimating the dimension of manifolds in high-dimensional data.
method Combines a modified box-counting algorithm (geometric) and a new probabilistic method (nearest neighbor distance analysis).
result The combined method is robust, fast, and effective in estimating manifold dimension.
SnapBoost uses random base hypothesis classes to improve gradient boosting performance.
problem Improving gradient boosting performance.
method Heterogeneous Newton Boosting Machine (HNBM) with variable base hypothesis classes.
result SnapBoost achieves better generalization loss than competing frameworks.
We introduce a notion of algorithmic stability of learning algorithms---that we term \emph{argument stability}---that captures stability of the hypothesis output by the learning algorithm in the normed space of functions from which hypotheses are selected. The main result of the paper bounds the generalization error of…
Develops a new method for estimating models with conditional moment restrictions.
problem Estimating models with conditional moment restrictions, especially non-parametric instrumental variable regression.
method Introduces a min-max criterion function to solve a zero-sum game between modeler and adversary, analyzing estimation rates for various hypothesis spaces.
result Shows that with regularization and rich test function spaces, estimation rates scale with the critical radius of hypothesis and test function spaces.
We propose a new notion of `n-category with duals', which we call a Whitney n-category. There are two motivations. The first is that Baez and Dolan's Tangle Hypothesis is (almost) tautological when interpreted as a statement about Whitney categories. The second is that we can functorially construct `fundamental Whitney…
When analyzing empirical data, we often find that global linear models overestimate the number of parameters required. In such cases, we may ask whether the data lies on or near a manifold or a set of manifolds (a so-called multi-manifold) of lower dimension than the ambient space. This question can be phrased as a (mu…
Paper tackles efficient learning of non-convex hypotheses in metric spaces.
problem Efficiently find consistent hypotheses for non-convex hypotheses composed of possibly several disconnected regions.
method Proposes a general domain-independent algorithm for finding consistent weakly convex hypotheses and proves sufficient conditions for its efficiency.
result Shows that consistent hypothesis finding problem can be solved in polynomial time for a broad class of weakly convex hypotheses over metric spaces.
The study examines how bias affects hypothesis formation in neural networks.
problem Characterizing the impact of bias on hypothesis formation in neural networks.
method An automated data-driven projection pursuit neural network to extract and select features for binary classification.
result The refinement of a working hypothesis converges to a robust multivariate perception of data.
This paper proposes grid cells encode position via a conformal isometric embedding of 2D physical space.
problem Hexagonal grid firing patterns in grid cells.
method Learning a distance-preserving position embedding in neural space using a recurrent neural network.
result The conformal isometric embedding of 2D physical space into neural space explains hexagonal grid firing patterns.
A framework for hypothesis testing on attributed graphs using sampling.
problem Statistical testing on graph data, especially large attributed graphs.
method Sampling-based framework with PHASE and PHASEopt for accurate and efficient hypothesis testing.
result PHASE and PHASEopt improve accuracy and efficiency of hypothesis testing in attributed graphs.
Quantum machine learning classification depends on mutual informations between state and parameter spaces.
problem Generalization in quantum machine learning models.
method Link between quantum machine learning and quantum hypothesis testing, using mutual informations.
result Quantum classifier accuracy and generalization depend on mutual informations between state and parameter spaces.
Paper proposes efficient communication scheme for statistical learning.
problem Efficiently conveying a statistical hypothesis from a client to a server.
method Joint training and source coding scheme with KL divergence constraints.
result Guarantees small average empirical risk, generalization error, and communication cost.
Extends linear representation hypothesis to categorical and hierarchical concepts in LLMs.
problem Representing concepts without natural contrasts in large language models.
method Formalizes linear representation hypothesis for categorical and hierarchical concepts, proving relationships between concept hierarchy and representation geometry.
result Validated theoretical results on large language models, estimating representations for 900+ concepts.
New method quantifies inductive bias for machine learning tasks.
problem Quantifying the amount of inductive bias in machine learning models.
method Estimates inductive bias by modeling loss distribution of random hypotheses.
result Higher dimensional tasks require greater inductive bias.
The paper extends graph-based semi-supervised learning to infinite-dimensional Wasserstein space.
problem Graph-based semi-supervised learning in high-dimensional data.
method Laplace Learning in the Wasserstein space, proving variational convergence and characterizing the Laplace-Beltrami operator.
result Consistent classification performance in high-dimensional settings.
Method measures weight similarity in neural networks using normalization and statistical inference.
problem Quantifying weight similarity in non-convex neural networks.
method Chain normalization rule and hypothesis-training-testing statistical inference.
result Weights of identical neural networks converge to similar local solutions.
We consider the problem of diagnosis where a set of simple observations are used to infer a potentially complex hidden hypothesis. Finding the optimal subset of observations is intractable in general, thus we focus on the problem of active diagnosis, where the agent selects the next most-informative observation based o…
This paper proves the manifold hypothesis for lower embedding dimensions using osculating hyperspheres.
problem The dataset lies on a low-dimensional submanifold in high-dimensional space.
method Constructing osculating hyperspheres and applying surgery theory to embed the hypersurface.
result The manifold hypothesis holds for embedding dimensionalities up to .
This paper considers a multiple regression model and compares, under full model hypothesis, analytically as well as by simulation, the performance characteristics of some popular penalty estimators such as ridge regression, LASSO, adaptive LASSO, SCAD, and elastic net versus Least Squares Estimator, restricted estimato…
The two-sample hypothesis testing problem is studied for the challenging scenario of high dimensional data sets with small sample sizes. We show that the two-sample hypothesis testing problem can be posed as a one-class set classification problem. In the set classification problem the goal is to classify a set of data …
A result of M. Ledoux is that a complete Riemannian manifold with non negative Ricci curvature satisfying the Euclidean Sobolev inequality is the Euclidean space. We present a shortcut of the proof. We also give a refinement of a result of B-L. Chen et X-P. Zhu about locally conformally flat manifolds with non negative…