The abstract warns against flawed empirical research in machine learning.
problem Flawed empirical research in machine learning leading to unreliable results.
method Call for more awareness of experimental knowledge plurality and epistemic limitations.
result Current empirical machine learning research should be exploratory, not confirmatory.
A new method approximates expected empirical loss for stochastic deep learning tasks.
problem Determining optimal step sizes for stochastic gradient descent in deep learning.
method Applying one-dimensional function fitting to noisy losses of vertical cross sections to approximate expected empirical loss.
result The method leads to a robust and straightforward optimization method that performs well across datasets and architectures.
A new model of learning corrects for chance to improve learning outcomes.
problem The importance of chance-corrected measures in learning.
method Developed two models: Informatron and AdaBook, based on empirical psychological results.
result Chance correction facilitates learning, as shown by computational results.
New learning algorithm for real analytic functions without gradient descent.
problem Learning real analytic functions without gradient descent.
method Taylor approximation and sampling data distribution.
result Nonuniform learning result for real analytic functions.
A new DP algorithm for weighted ERM protects sensitive data in predictive models.
problem Protecting sensitive personal information in predictive models trained via ERM.
method Proposes the first differentially private algorithm for weighted ERM with formal privacy guarantees.
result Demonstrates strong DP guarantees while maintaining robust performance in real-world data.
Paper proposes E3BM for robust few-shot learning with few examples.
problem Few-shot learning with limited data leads to poor model performance.
method Meta-learn ensemble of epoch-wise empirical Bayes models (E3BM).
result Top performance achieved using epoch-dependent transductive hyperprior learner.
The paper improves semi-supervised learning using f-divergences and α-Rényi divergences.
problem Improving semi-supervised learning with noisy pseudo-labels.
method Inspired by f-divergences and α-Rényi divergences, the paper develops new empirical risk functions and regularization techniques. result The new methods show better performance than traditional self-training methods, especially in noisy pseudo-label scenarios.
A new SGD framework reduces empirical risk by favoring higher loss observations.
problem Minimizing empirical risk in machine learning problems.
method Develops a biased gradient estimator for stochastic optimization.
result Minimizes an ordered modification of the empirical average loss.
The paper examines the tilted empirical risk's generalization and robustness under negative tilt.
problem The generalization error of machine learning algorithms under negative tilt.
method Uniform and information-theoretic bounds on the tilted generalization error under negative tilt.
result The tilted empirical risk's generalization error has a convergence rate of \(O(n^{-ε/(1+ε)})\).
The paper provides bounds for the empirical angular measure and applies them to improve statistical learning in extreme regions.
problem Estimating the angular measure in high-dimensional data with different distributions.
method Established bounds for the maximal deviations of the empirical angular measure from the true measure, using rank transformation and analyzing the most extreme observations.
result The bounds provide performance guarantees for statistical learning procedures in extreme regions, such as binary classification and anomaly detection.
DOC3 learns from contradictions to improve deep one class classification.
problem Deep one class classification problems.
method Formalizes learning from contradictions for one class large-margin loss, proposes DOC3 algorithm.
result DOC3 incurs lower generalization error compared to traditional inductive learning.
Spectral regularization improves learning over combinatorial spaces with limited data.
problem Learning pseudo-Boolean functions with scarce labeled data.
method Regularizing the spectral representation of learned functions using the L_1 norm.
result Regularization allows for data-frugal learning and achieves statistically optimal generalization performance.
This paper improves random feature sampling using empirical leverage scores.
problem Optimizing the number of features for kernel approximation and supervised learning.
method Uses empirical leverage scores to optimize feature sampling.
result Empirical sampling of random features using leverage scores outperforms vanilla Monte Carlo sampling.
Study evaluates off-policy reinforcement learning methods.
problem Evaluating reinforcement learning policies without direct access to the behavior policy.
method Experimental benchmarking suite with diverse design parameters.
result Guidelines for using off-policy evaluation methods in practice.
Improved multi-group learning with group-realizable concepts.
problem Enhancing multi-group learning efficiency.
method Empirical risk minimization over group-realizable concepts.
result Improved sample complexity in group-realizable settings.
Empirical moment matrix reveals properties of point clouds.
problem Uncovering properties of point clouds, especially those with singular support.
method Combining statistics, real algebraic geometry, and approximation theory.
result The empirical moment matrix provides insights into data analysis.
We information-theoretically reformulate two measures of capacity from statistical learning theory: empirical VC-entropy and empirical Rademacher complexity. We show these capacity measures count the number of hypotheses about a dataset that a learning algorithm falsifies when it finds the classifier in its repertoire …
Paper proposes an unbiased classifier from triplet comparison data.
problem Learning a classifier from triplet comparison data.
method Empirical risk minimization framework with an unbiased estimator.
result The proposed method achieves better performance than baseline methods.
Max-diversity improves distributed learning performance.
problem Improving risk performance in distributed learning.
method Max-diversity distributed learning algorithm (MDD).
result MDD outperforms existing methods with tighter risk bounds.
The paper proves deep learning can be robust with certain loss functions.
problem The robustness of deep learning models under flawed data.
method Empirical-risk minimization with unbounded, Lipschitz-continuous loss functions.
result These loss functions provide efficient prediction under minimal data assumptions.
Proposes a new method to learn meta-priors from data.
problem Improving learning systems with domain knowledge and controlling parameter learning rates.
method Hierarchical Empirical Bayes approach to decouple learning rates of features.
result Meta-prior learning improves performance and convergence time in various applications.
The optimal approach is to theorize after examining data, not before.
problem Optimal sequencing of theory and empirical analysis for economic questions.
method Formalized a Bayesian model to trade off Darwinian and Statistical Learning.
result Post hoc theorizing is typically optimal in modern economics.
Unified framework for N-tuples learning improves weakly supervised tasks.
problem Reducing annotation burden in supervised learning.
method Empirical risk minimization framework integrating pointwise unlabeled data.
result Framework improves generalization across various N-tuples learning tasks.
Adaptive Bayesian method improves deep learning performance and robustness.
problem Ensuring sparsity in deep learning models to improve performance and robustness.
method Adaptive empirical Bayesian method with self-adaptive spike-and-slab priors and stochastic gradient MCMC optimization.
result The method achieves state-of-the-art performance on MNIST and Fashion MNIST and compression performance on CIFAR10.
New method estimates Schrödinger bridge potentials via empirical risk minimization.
problem Estimating Schrödinger bridge potentials from samples.
method Rewriting Schrödinger system as a fixed-point equation and estimating the potential via empirical risk minimization.
result Uniform concentration of empirical risk around population counterpart under sub-Gaussian assumptions.
New algorithm closes empirical gap in PFSGD performance.
problem Empirical performance gap between tuned SGD and PFSGD.
method Parameter-free algorithm based on Coin-Betting ODE updates.
result New algorithm outperforms tuned baselines and matches optimal performance.
Prove non-asymptotic bounds for minimal risk in statistical learning
problem Estimating minimal risk in statistical learning
method Using concentration inequalities
result Non-asymptotic bounds for minimal risk
Unified kernel density and empirical Bayes for unsupervised learning.
problem Addressing problems in unsupervised learning with geometric interpretation.
method Parametrizing the energy function with a neural network and combining kernel density estimation and empirical Bayes.
result Emergence of rich 'creative memories' as attractors in associative memory framework.
Sharp-MAML improves MAML by reducing saddle points in few-shot learning.
problem Challenges in optimizing MAML due to complex loss landscape.
method Sharpness-aware minimization applied to MAML.
result Sharp-MAML and its variant outperform plain MAML on few-shot learning tasks.
Paper finds optimal mini-batch size for SGD to speed up learning.
problem Optimizing mini-batch size for faster SGD convergence.
method Empirical inverse law and theoretical bound on mini-batch SGD training.
result An accurate model for predicting training time and identifying implications for algorithm and hardware.
We consider distributed convex optimization problems originated from sample average approximation of stochastic optimization, or empirical risk minimization in machine learning. We assume that each machine in the distributed computing system has access to a local empirical loss function, constructed with i.i.d. data sa…
Review of latest DRL algorithms with theoretical and practical insights.
problem Challenges in reinforcement learning and deep learning.
method Theoretical justification and empirical analysis of DRL algorithms.
result Empirical properties and practical limitations of DRL algorithms are discussed.
Paper tackles constrained learning with non-convex losses, overcoming challenges with new approach.
problem Challenges in learning with non-convex losses and statistical constraints.
method Learning in the empirical dual domain, bounding empirical duality gap.
result Established a constrained counterpart to classical learning theory.
New method for PU learning with instance-dependent propensity scores.
problem Learning from positive and unlabeled data with instance-dependent labeling.
method Empirical risk minimization of joint risk function, alternating optimization of posterior probability and propensity score.
result The method achieves comparable or better performance than state-of-the-art methods.
A new learning method uses data to learn from large model sets.
problem Learning with large sets of candidate models where uniform convergence is hard.
method Data-dependent learning that incorporates empirical data less reliant on prior assumptions.
result Demonstrates improved generalization in various learning assumptions.
Study landscape of non-convex empirical risk with degenerate population risk.
problem Degenerate non-convex population risk in machine learning problems.
method Analyze population risk first, then connect to empirical risk landscape.
result Established correspondence between empirical and population risk critical points.
Empirical study on hyperparameters affecting RL generalization.
problem Understanding how hyperparameters affect RL generalization across different domains.
method Empirical analysis of hyperparameters including architecture, regularization, and RL-dependent variables.
result Complex and interdependent relationships between hyperparameters and generalization performance.
Transformers can learn Markov processes with constant depth, surprising results.
problem Understanding how transformers learn context in Markov processes.
method Empirical study and theoretical analysis of attention-based transformers on Markov data.
result Transformers with constant depth can achieve low test loss on Markov sequences, matching empirical and theoretical findings.
A new method estimates the learning coefficient using empirical loss.
problem Estimating the learning coefficient in complex models.
method Numerical estimation using Empirical Loss.
result Our method outperforms existing techniques in terms of bias and variance.
Study finds LLMs hallucinate in finance tasks, needing research.
problem Hallucination in LLMs in finance.
method Empirical investigation of four methods to mitigate hallucination.
result LLMs hallucinate in financial tasks.
Backward exploration reduces sample complexity in policy evaluation.
problem Empirical policy evaluation in reinforcement learning.
method Backward exploration algorithms from high-cost states.
result Reduced average-case sample complexity to O(logS). Corrects sample selection bias in empirical risk minimization using importance sampling.
problem Statistical learning with biased training data.
method Weighted empirical risk minimization using importance sampling.
result Generalization capacity preserved with estimated importance weights.
Meta-learning framework improves model performance on few-shot classification tasks.
problem Improving model performance on few-shot classification tasks.
method Empirical Bayes formulation with synthetic gradients for transductive meta-learning.
result Meta-learning framework outperforms previous state-of-the-art methods on benchmarks.
Empirical risk minimizers can be non-monotonic in learning curves.
problem Understanding the behavior of learning curves for empirical risk minimizers.
method Introducing risk monotonicity and analyzing its implications for various learners.
result Standard learners that minimize empirical risk can be non-monotonic regardless of training sample size.
Machine learning creates non-factor covariance matrices for risk models.
problem Creating robust risk models for financial portfolios.
method Developed an explicit algorithm and source code for machine learning risk models.
result Machine learning models outperform traditional risk models in empirical backtests.
Improves deep transfer learning by preventing performance degradation.
problem Deep transfer learning can degrade performance when using inappropriate pre-trained weights.
method Proposes a novel strategy to compute new descent directions that preserve regularization effects.
result DTNH strategy improves performance of deep transfer learning tasks by 0.1%--7%.
FedDANE adapts DANE for federated learning, but underperforms compared to existing methods.
problem Federated learning's practical constraints and device heterogeneity.
method Adapted DANE for federated learning, providing convergence guarantees for convex and non-convex functions.
result Empirically, FedDANE underperforms compared to FedAvg and FedProx.
Currently, machine learning plays an important role in the lives and individual activities of numerous people. Accordingly, it has become necessary to design machine learning algorithms to ensure that discrimination, biased views, or unfair treatment do not result from decision making or predictions made via machine le…