Paper tackles one-bit compressed sensing using PAC learning theory.
problem One-bit compressed sensing problem.
method Formulated as PAC learning problem, uses VC-dimension and PAC learning theory.
result Consistent algorithm can recover k k k -sparse vectors with O ( k lg ( n / k ) ) O(k \lg (n/k)) O ( k lg ( n / k )) measurements. The paper proposes a least squares method for binary compressive sampling with low intrinsic dimension signals.
problem Recovering signals from binary measurements with noise and sign flips.
method Least squares decoder for signals with low generative intrinsic dimension.
result The least squares decoder achieves a sharp estimation error of O ( k log ( L n ) m ) O(\sqrt{\frac{k\log (Ln)}{m}}) O ( m k l o g ( L n ) ) under certain conditions. FlipOut prunes neural networks by flipping weights' signs, achieving high sparsity.
problem Redundant weights in neural networks increase training time and resource usage.
method Uses sign flips during training to determine weight saliency for pruning.
result Competitive with existing methods, achieving state-of-the-art performance for high sparsity.
A new protocol evaluates small machine learning improvements conservatively.
problem Uncertainty in small gains reported in machine learning papers.
method Paired bootstrap protocol with BCa confidence intervals and sign-flip permutation tests.
result Conservative evaluation reduces over-claiming of small improvements.
New neural architectures invariant to sign flips and basis symmetries for graph representation learning.
problem Learning invariant graph representations from eigenvectors.
method SignNet and BasisNet neural architectures that are invariant to sign flips and basis symmetries.
result Proven to be universal, approximating any continuous function of eigenvectors with desired invariances.
Improves bit error tolerance in RRAM-based BNNs without overfitting.
problem Bit errors in RRAM-based BNNs reduce accuracy and overfit to training error rates.
method Proposes straight-through gradient approximation and a novel regularizer.
result Improves BNNs' robustness to bit errors without overfitting.
The resilience of low-degree Rademacher chaos is studied, providing probabilistic lower bounds.
problem Understanding how much a Rademacher chaos can withstand adversarial sign-flips without significant probability changes.
method Probabilistic lower-bound guarantees for the resilience of Rademacher chaos of arbitrary degree.
result Probabilistic lower-bound guarantees for the resilience of Rademacher chaos of arbitrary degree, especially meaningful for constant degree.
We present a simple hybrid dynamical model as a tool to investigate behavioral strategies based on trend following. The multiplicative symbolic dynamics are generated using a lognormal diffusion model for the at-the-money implied volatility term structure. Thus, are model exploits information from derivative markets to…
Paper assesses error estimates of Random Forests classification.
problem Quantitative assessment of Random Forests error estimates.
method Theoretical and empirical investigation of various error estimation methods.
result Random Forests' error estimates are closer to true error rate than average prediction error.
Study loop corrections in random feature models affecting training and test errors.
problem Analyzing loop corrections in random feature models to understand training and test errors.
method Statistical physics and effective field theory approach to study loop corrections.
result Derived loop corrections to training error, test error, and generalization gap.
Paper proposes diagnostics for error and variance estimation in randomized matrix computations.
problem Safe use of randomized matrix algorithms in applications.
method Leave-one-out error estimator and jackknife resampling method.
result Provides rapid diagnostics to assess quality of randomized matrix computations.
Unified framework for estimating random forest prediction errors.
problem Estimating prediction errors for random forests.
method Novel estimator of conditional prediction error distribution function.
result Proposed estimators enable competitive prediction intervals.
Analyzes the generalization and training errors of the random feature model over time.
problem Understanding the temporal behavior of generalization and training errors in deep learning.
method Uses Cauchy complex integral representations and random matrix methods based on linear pencils.
result Analytical solution of the full time-evolution path of generalization and training errors.
Random Fourier Features reduce kernel matrix reconstruction error without dimensionality dependence.
problem Error reduction in kernel matrix reconstruction for high-dimensional data.
method Random Fourier Features with theoretical error bounds.
result Error probability is independent of data dimensionality.
Random forest training yields confidence intervals for generalization error.
problem Computing accurate confidence intervals for random forest generalization error.
method Directly computes confidence intervals from training data without data splitting.
result Confidence intervals have good coverage and appropriate width.
Random features and KRR generalize similarly when N is large enough.
problem Understanding the generalization error of random features and KRR methods.
method Analyzing spectral conditions and hypercontractivity on kernel eigenfunctions.
result The test error of random features is larger than KRR when N is small, but they achieve the same error when N is large.
Unified bounds for random subset generalization error and improved SGD Langevin dynamics.
problem Generalization error bounds for random subsets and stochastic gradient Langevin dynamics.
method Unified framework based on Hellström and Durisi's work, extending bounds for Langevin dynamics.
result Unified and refined bounds for generalization error in stochastic gradient Langevin dynamics.
This paper solves quadratic systems with sparse or generative priors.
problem Recovering signals from quadratic systems with full-rank matrices.
method Thresholded Wirtinger flow (TWF) and projected gradient descent (PGD) algorithms.
result The proposed methods significantly outperform existing algorithms in signal recovery.
Study improves error bounds for sparse regression with heavy-tailed covariates.
problem Estimating sparse coefficients in linear regression with heavy-tailed covariates.
method Employed an ℓ 1 \ell_1 ℓ 1 -penalized Huber regression method. result Error bound identical to Gaussian case for L L L -subexponential covariates. Study shows exponential convergence in classification errors using random features and SGD.
problem Scalability issues in kernel methods for large datasets.
method Binary classification problem with random features and stochastic gradient descent.
result Exponential convergence rate of expected classification error achieved.
RQMC improves kernel-based learning by reducing deterministic error and offering computational advantages.
problem Improving kernel-based learning methods to reduce deterministic error and computational complexity.
method Randomized quasi-Monte Carlo (RQMC) methods applied to random feature approximations.
result RQMC methods improve deterministic approximation error bound from O P ( 1 / M ) O_P(1/\sqrt{M}) O P ( 1/ M ) to O ( 1 / M ) O(1/M) O ( 1/ M ) , matching QMC methods. This paper develops a bootstrap method to estimate errors in Random Fourier Features.
problem Inability to estimate the error of Random Fourier Features approximations.
method Develops a bootstrap approach to numerically estimate the errors of RFF approximations.
result Specific, flexible, and adaptive error estimates for RFF approximations.
Bootstrap method estimates randomized LS algorithm errors.
problem Lack of error estimation for randomized LS algorithms.
method Bootstrap method to compute a posteriori error estimates.
result The method provides numerical error assessment and work prediction.
We find a deterministic equivalent for random feature regression's test error, independent of feature map dimension.
problem Understanding the generalization performance of random feature ridge regression.
method We derive a deterministic equivalent for the test error of RFRR under a concentration property, showing it can be approximated by a closed-form expression dependent on feature map eigenvalues.
result Our approximation guarantee is non-asymptotic, multiplicative, and independent of the feature map dimension, providing a tight result for the smallest number of features achieving optimal minimax error rate.
A new method for approximating softmax and Gaussian kernels with reduced error.
problem Approximating softmax and Gaussian kernels with low error.
method Simplex Random Features (SimRFs) and SimRFs+.
result SimRFs provide the smallest MSE among weight-independent geometrically-coupled PRF mechanisms.
Improved analysis of a random forest model reduces prediction error.
problem Improving prediction accuracy in random forest models.
method Revisited a historical random forest model, analyzing feature selection and splits.
result Mean-squared prediction error rate is improved to O((n(\log n)^{(S-1)/2})^{-\frac{1}{S\log2+1}}).
New method corrects for random measurement error in causal discovery.
problem Measurement error invalidates causal discovery results.
method Upper bound for measurement error variance from covariance matrix, applied to constraint-based causal discovery.
result Corrected causal discovery results are more reproducible.
Paper analyzes error bounds for learning with vector-valued RF, improving existing analyses.
problem Learning with vector-valued random features in infinite-dimensional settings.
method Direct analysis of risk functional, avoiding random matrix theory.
result Strong consistency and minimax optimal convergence rates established.
New framework optimizes random forest parameters for stability and cost.
problem Optimizing random forest parameters for industrial applications.
method Bayesian optimization framework considering error, stability, and cost.
result Parameter settings that balance error, stability, and cost.
New framework reduces private mean estimation error with optimal efficiency.
problem Locally private mean estimation of high-dimensional vectors.
method ProjUnit framework: random projections, normalization, and optimal algorithm execution in lower dimensions.
result Optimal error up to a 1+o(1)-factor with computational efficiency and low communication complexity.
New gradient coding schemes reduce decoding error in both random and adversarial straggler settings.
problem Creating efficient approximate gradient coding schemes for distributed optimization.
method Introduced novel approximate gradient codes based on expander graphs, achieving optimal decoding coefficients.
result Achieved nearly optimal error in random setting and nearly half the error in adversarial setting compared to existing codes.
New ensemble SVM model reduces prediction error without choosing best kernel.
problem Reducing prediction error in regression problems.
method Bagged-weighted support vector regression model with random machines.
result Regression Random Machines achieve lower generalization error.
Short proof shows how ridge regression works with random data.
problem Understanding prediction error in ridge regression with random design.
method Combination of exchangeability arguments, matrix perturbation, and operator convexity.
result Elementary proof of prediction error without complex inequalities.
This work gives a simultaneous analysis of both the ordinary least squares estimator and the ridge regression estimator in the random design setting under mild assumptions on the covariate/response distributions. In particular, the analysis provides sharp results on the ``out-of-sample'' prediction error, as opposed to…
ESM-CNN uses error feedback to build a random CNN for time series forecasting.
problem Improving time series forecasting accuracy with CNNs.
method Incrementally adding random filters and neurons to adaptively construct a CNN.
result ESM-CNN outperforms state-of-the-art models in prediction accuracy and efficiency.
New PAC-Bayesian bounds improve random forest generalization error estimates.
problem Improving rigorous upper bounds on random forest generalization error.
method PAC-Bayesian approaches to derive bounds, exploiting out-of-bag samples, and analyzing dependencies between ensemble members.
result PAC-Bayesian bounds can be superior and often reasonably tight, but performance varies with correlation levels and validation set usage.
Analyzes deep neural networks training errors with SGD and random init.
problem Lack of rigorous understanding of deep learning algorithms.
method Mathematical analysis of deep learning with SGD and random init.
result First full error analysis for deep learning with SGD and random init.
Random feature method approximates operators with theoretical guarantees and reduced computation.
problem Approximating operators between infinite dimensional Banach spaces using machine learning.
method Random feature operator learning method with theoretical guarantees and error bounds.
result The random feature method can achieve similar or better test errors than kernel-based methods and neural networks with significantly reduced training times.
The study proves Gaussian universality of deep random features learning.
problem Understanding the test error in deep random features learning.
method Proving Gaussian universality of test error in ridge regression and arbitrary convex losses.
result Sharp asymptotic formula for test error in ridge regression setting.
Bayes-optimal learning of deep random networks with Gaussian weights is studied.
problem Learning a target function corresponding to a deep, extensive-width, non-linear neural network with random Gaussian weights.
method Closed-form expressions for Bayes-optimal test error, ridge regression, kernel and random features regression are computed.
result Optimally regularized ridge regression and kernel regression achieve Bayes-optimal performances, while logistic loss yields a near-optimal test error for classification.
Study shows gap between uniform convergence and test error in random feature models.
problem Understanding the gap between uniform convergence and test error in random feature models.
method Analytical expressions for uniform convergence over norm balls, interpolators, and minimum norm interpolator risk derived and proved.
result Uniform convergence over interpolators still gives a non-trivial bound of test error even when classical uniform convergence is vacuous.
The paper analyzes the generalization error of random features regression, revealing a double descent curve.
problem Analyzing the generalization error of random features regression.
method Performing ridge regression on N random features of the form σ(wa^T x), where wa are random weights.
result The test error follows a double descent curve, with a global minimum above the interpolation threshold.
LoCoV reduces portfolio optimization errors from sample covariance matrices.
problem Large errors in sample covariance matrix for optimal portfolio weights.
method LoCoV (low dimension covariance voting) algorithm to reduce these errors.
result LoCoV outperforms classical methods in portfolio optimization experiments.
New algorithm LSTD( λ λ λ )-RP uses random projections and eligibility traces for efficient reinforcement learning.
problem Policy evaluation in high-dimensional feature spaces with linear function approximation.
method Proposes LSTD( λ λ λ )-RP algorithm combining random projections and eligibility traces. result Demonstrates improved performance and better error bounds compared to prior methods.
One-bit quantization improves inference speed for Random Features models.
problem Efficient inference on resource-constrained devices.
method Analysis of one-bit quantization in Random Features model.
result Asymptotically, quantizing weights except the last incurs no loss in generalization error.
Optimal AFs minimize RFR test error and sensitivity.
problem Finding optimal AFs for RFR to minimize test error and sensitivity.
method Closed-form solution for AFs minimizing test error and sensitivity under different functional parsimony.
result Optimal AFs can be linear, saturated linear, or Hermite polynomial expressions.
RQMC improves QMC by providing practical error bounds for financial applications.
problem Lack of practical error estimates in QMC methods.
method Combines Sobol LDS with randomized scrambling methods.
result RQMC outperforms standard QMC in convergence rates and provides error bounds.
Develops a bootstrap method to estimate error in randomized matrix multiplication.
problem Estimating the accuracy of randomized matrix multiplication methods.
method Bootstrap method for directly estimating accuracy as a function of reduced dimension.
result Demonstrates the effectiveness of the bootstrap method through both theoretical and empirical results.