A form of generalisation error known as Off Training Set (OTS) error was recently introduced in [Wolpert, 1996b], along with a theorem showing that small training set error does not guarantee small OTS error, unless assumptions are made about the target function. Here it is shown that the applicability of this theorem …
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “small error”
How PC-based Methods Err: Towards Better Reporting of Assumption Violations and Small Sample Errorsstat.ML
The study analyzes and mitigates errors in PC-based causal discovery methods.
problem Errors in PC-based causal discovery methods can lead to incorrect graphs.
method The study introduces coherency scores to detect assumption violations and small sample errors in PC-based methods.
result The coherency scores can detect errors that other methods cannot, bridging between global and local error detection.
This work finds a point with small test error in polynomial time for mildly overparameterized neural nets.
problem Achieving small test error in mildly overparameterized neural networks.
method The work shows that the landscape of loss functions with explicit regularization has a property that all local minima and certain stationary points achieve small test error. It also proves the existence of polynomial time algorithms for finding such points in convolutional and fully connected neural nets.
result Polynomial time algorithms exist for finding points with small test error in mildly overparameterized neural nets.
Dynamic hedging of an European option under a general local volatility model with small linear transaction costs is studied. A continuous control version of Leland's strategy that asymptotically replicates the payoff is constructed. An associated central limit theorem of hedging error is proved. The asymptotic error va…
Langevin dynamics fails to produce accurate samples even with small score function errors.
problem Robustness of Langevin dynamics to score function errors.
method Analysis of Langevin dynamics and score function errors.
result Langevin dynamics produces a distribution far from the target distribution in TV distance even with small errors in the score function.
Novel approach for SEM in small samples with .
problem Small sample size and issues in factor-based SEM.
method Reformulates covariance structure into self-covariance and cross-covariance, defines a feasible set with relative error constraint.
result Improved stability and directional information in small-sample settings.
Robust Importance Sampling for Error Estimation in the Context of Optimal Bayesian Transfer Learningstat.ML
Paper introduces a new method for error estimation in classification tasks with limited data.
problem Challenges in designing accurate classifiers and evaluating their performance with limited training data.
method Introduces a novel Bayesian MMSE estimator for optimal Bayesian transfer learning (OBTL) using Monte Carlo importance sampling.
result Proposed OBTL error estimation scheme outperforms standard methods, especially in small-sample settings.
New learner achieves optimal agnostic error in small error regime.
problem Optimizing agnostic learning in the small error regime.
method Careful aggregations of ERM classifiers.
result Achieves error $c \cdot τ+ O \left(\sqrt{\frac{τ(d + \log(1 / δ))}{m}} + \frac{d + \log(1 / δ)}{m}
ight)$, matching lower bound when .
The pricing of options in exponential Levy models amounts to the computation of expectations of functionals of Levy processes. In many situations, Monte-Carlo methods are used. However, the simulation of a Levy process with infinite Levy measure generally requires either to truncate small jumps or to replace them by a …
Complexity Dependent Error Rates for Physics-informed Statistical Learning via the Small-ball Methodstat.ML
Paper develops error rates for physics-informed learning, comparing it to data-driven methods.
problem Understanding the trade-off between soft penalties and hard constraints in PISL.
method Develops complexity-dependent error rates using the small-ball method.
result Physics-informed estimators have comparable error rates to hard constrained methods, differing only by constants.
The skip-connections used in residual networks have become a standard architecture choice in deep learning due to the increased training stability and generalization performance with this architecture, although there has been limited theoretical understanding for this improvement. In this work, we analyze overparameter…
New findings show score matching's accuracy doesn't ensure numerical stability in diffusion sampling.
problem Numerical stability issues in diffusion sampling despite small forward-marginal error.
method Constructing a smooth score field with arbitrarily small forward-marginal error, showing nonexplosive behavior and moments of every order.
result Euler--Maruyama discretizations can converge in probability even when moments diverge, demonstrating failure of weak convergence.
Confidence measures for the generalization error are crucial when small training samples are used to construct classifiers. A common approach is to estimate the generalization error by resampling and then assume the resampled estimator follows a known distribution to form a confidence set [Kohavi 1995, Martin 1996,Yang…
ASGD outperforms SGD in overparameterized linear regression, especially in subspaces of small eigenvalues.
problem Generalization of ASGD for overparameterized linear regression.
method Established instance-dependent excess risk bound for ASGD in each eigen-subspace of the data covariance matrix.
result ASGD outperforms SGD in subspaces of small eigenvalues, exhibiting faster decay of bias error.
The paper gives a constructive method, based on greedy algorithms, that provides for the classes of functions with small mixed smoothness the best possible in the sense of order approximation error for the -term approximation with respect to the trigonometric system.
In this paper we define a small variation of the Taylor method and a formula for the global error of this new numerical method that allows us to keep track of the round-off error and does not require previous knowledge of the exact solution. As an application we provide a rigorous proof of the construction/existence of…
Passive investing can incur hidden costs due to market timing inefficiencies.
problem Hidden costs in passive investing due to market timing inefficiencies.
method Analysis of passive investing strategies, including gradual share acquisition and post-announcement trading.
result Post-announcement trading can earn significant profits, often exceeding 1%.
Gradient descent benefits from tangent kernel advantages under specific conditions.
problem Comparing gradient descent with tangent kernel methods in learning.
method Analysis of gradient descent and tangent kernel methods under different conditions.
result Gradient descent can achieve small error only if tangent kernel methods have a non-trivial advantage, but this advantage can be very small.
In this report, we derive a non-negative series expansion for the Jensen-Shannon divergence (JSD) between two probability distributions. This series expansion is shown to be useful for numerical calculations of the JSD, when the probability distributions are nearly equal, and for which, consequently, small numerical er…
Binary classification improves with a small fraction of corrupted labels.
problem Binary classification with corrupted labels.
method Established corruption as a form of regularization and computed upper bounds on estimation error.
result Corruption is beneficial only up to a small fraction of the total sample, scaling with the square root of the sample size.
Random feature model shows slow self-correction of generalization gap.
problem Slow deterioration of generalization error in random feature model.
method Examined the dynamic behavior of gradient descent in the model's resonance regime.
result Gradient descent exhibits a self-correction mechanism, reducing generalization gap over time.
We study the effects of approximate inference on the performance of Thompson sampling in the -armed bandit problems. Thompson sampling is a successful algorithm for online decision-making but requires posterior inference, which often must be approximated in practice. We show that even small constant inference error …
Fast algorithm recovers principal eigenvector from noisy matrices.
problem Recovering the first principal eigenvector from noisy positive semidefinite matrices.
method Cone projected power iteration algorithm.
result Achieves polynomial time complexity and small error for certain convex cones.
The CLT fails for LLM evaluations with small data, leading to underestimation of uncertainty.
problem Inaccurate uncertainty estimates in LLM evaluations with small datasets.
method Alternative frequentist and Bayesian methods for uncertainty quantification.
result CLT-based methods underestimate uncertainty in small data settings.
Paper proposes efficient communication scheme for statistical learning.
problem Efficiently conveying a statistical hypothesis from a client to a server.
method Joint training and source coding scheme with KL divergence constraints.
result Guarantees small average empirical risk, generalization error, and communication cost.
This research evaluates generalization measures in deep learning.
problem Understanding why deep learning models generalize well despite small training error.
method Empirical evaluation of generalization bounds and measures.
result Generalization measures should be evaluated using distributional robustness.
For binary classification we establish learning rates up to the order of for support vector machines (SVMs) with hinge loss and Gaussian RBF kernels. These rates are in terms of two assumptions on the considered distributions: Tsybakov's noise assumption to establish a small estimation error, and a new geometr…
Study evaluates uncertainty quantification for atomistic neural networks, revealing complex relationships between error and uncertainty.
problem Uncertainty quantification for predictions of atomistic neural networks.
method Modified PhysNet NN architecture, evaluated with various metrics, analyzed QM9 and tautomerization reaction databases.
result Error and uncertainty are not linearly related; redundancy and noise complicate predictions, especially for small changes.
Dimensionality reduction is a first step of many machine learning pipelines. Two popular approaches are principal component analysis, which projects onto a small number of well chosen but non-interpretable directions, and feature selection, which selects a small number of the original features. Feature selection can be…
Study on Transfer Elastic Net error bounds and grouping effect.
problem Estimation error and grouping effect in Transfer Elastic Net.
method Derives non-asymptotic error bound and examines grouping effect scenarios.
result Effective error bounds and grouping effect observed in Transfer Elastic Net.
Small initialization improves tensor recovery from noisy data.
problem Recovering low-tubal-rank tensors from noisy measurements.
method Factorized gradient descent with small initialization.
result Achieves nearly minimax optimal recovery error.
We propose to study the generalization error of a learned predictor in terms of that of a surrogate (potentially randomized) predictor that is coupled to and designed to trade empirical risk for control of generalization error. In the case where interpolates the data, it is interesting to con…
Identifies bilinear systems from a single trajectory with optimal sample complexity.
problem Learning bilinear systems from a single trajectory of states and inputs.
method Uses a mild marginal mean-square stability assumption and martingale small-ball condition.
result Sample complexity and statistical error rates are optimal.
Neural networks can approximate rectifiable measures with small error.
problem Approximating complex rectifiable measures using neural networks.
method Using ReLU neural networks to approximate (countably) m-rectifiable measures as push-forwards of the Lebesgue measure.
result The approximation error in terms of Wasserstein distance can be made arbitrarily small.
Paper analyzes Gibbs and Langevin Monte Carlo for interpolation regime, showing generalization from low errors.
problem Analyzing Gibbs and Langevin Monte Carlo in overparameterized interpolation regime.
method Data-dependent bounds and stability under approximation with Langevin Monte Carlo.
result Generalization is signaled by small training errors in noisy regime, with bounds stable under approximation.
We propose rectified factor networks (RFNs) to efficiently construct very sparse, non-linear, high-dimensional representations of the input. RFN models identify rare and small events in the input, have a low interference between code units, have a small reconstruction error, and explain the data covariance structure. R…
Gradient descent fails to learn simple neural networks efficiently.
problem Learning one-layer neural networks efficiently using gradient descent.
method Gradient descent and statistical query algorithms.
result Superpolynomial lower bounds for learning one-layer neural networks.
We design a new algorithm for the Euclidean -means problem that operates in the local model of differential privacy. Unlike in the non-private literature, differentially private algorithms for the -means objective incur both additive and multiplicative errors. Our algorithm significantly reduces the additive erro…
Study proposes a stopping criterion for active learning based on error stability.
problem Improving predictive performance in active learning by adaptively annotating samples.
method Proposes a stopping criterion based on error stability for Bayesian active learning.
result Demonstrates the proposed criterion stops active learning at the appropriate timing for various models and datasets.
Stochastic gradient descent updates parameters with summation gradient computed from a random data batch. This summation will lead to unbalanced training process if the data we obtained is unbalanced. To address this issue, this paper takes the error variance and error mean both into consideration. The adaptively adjus…
This work presents a technique for statistically modeling errors introduced by reduced-order models. The method employs Gaussian-process regression to construct a mapping from a small number of computationally inexpensive `error indicators' to a distribution over the true error. The variance of this distribution can be…
Suppose some classifiers are selected from a set of hypothesis classifiers to form an equally-weighted ensemble that selects a member classifier at random for each input example. Then the ensemble has an error bound consisting of the average error bound for the member classifiers, a term for selectivity that varies fro…
Ex ante forecast outcomes should be interpreted as counterfactuals (potential histories), with errors as the spread between outcomes. Reapplying measurements of uncertainty about the estimation errors of the estimation errors of an estimation leads to branching counterfactuals. Such recursions of epistemic uncertainty …
Hardness proof for agnostically learning halfspaces from worst-case lattice problems.
problem Agnostically learning halfspaces in the presence of noise.
method Reduction to worst-case lattice problems (GapSVP, SIVP).
result No efficient algorithm can achieve misclassification error better than 1/2 - γ under given hardness assumptions.
In this paper, we study the problem of sparse multiple kernel learning (MKL), where the goal is to efficiently learn a combination of a fixed small number of kernels from a large pool that could lead to a kernel classifier with a small prediction error. We develop an efficient algorithm based on the greedy coordinate d…
Geometric framework links clustering accuracy to structural recovery.
problem Understanding the trade-off between robustness and sensitivity in clustering.
method Develops a clustering condition number to compare within-cluster scale to the minimum loss increase required to move a point across a cluster boundary.
result Sharp phase transitions for exact recovery under different objectives, providing geometric principle for interpreting low objective values.
A new data-adaptive prior stabilizes kernel learning in operators.
problem Learning kernels in operators from data is ill-posed due to nonlocal dependence.
method Introduces a data-adaptive prior to stabilize the Bayesian posterior mean.
result The data-adaptive prior achieves a stable posterior with small noise limits.
The CUR matrix decomposition is an important extension of Nyström approximation to a general matrix. It approximates any data matrix in terms of a small number of its columns and rows. In this paper we propose a novel randomized CUR algorithm with an expected relative-error bound. The proposed algorithm has the advanta…