Paper introduces a method to control early classification accuracy gaps.
problem Maintaining accuracy in early classification without full input processing.
method Statistical framework for a calibrated stopping rule.
result Reduces up to 94% of timesteps while controlling accuracy gaps.
Extracurricular learning closes the accuracy gap in knowledge distillation.
problem Accuracy gap between teacher and student models after knowledge distillation.
method Modeling student and teacher output distributions, sampling from an extended data distribution, and matching over this set.
result Extracurricular learning reduces the accuracy gap by 46% to 68%.
New framework tightens certified robustness gaps in machine learning models.
problem Persistent gap between theoretical certified robustness and empirical accuracy.
method Leverages Lipschitz continuity and novel confidence intervals.
result Improves robust accuracy, compressing the gap between theory and practice.
KZImputer improves time series data quality with adaptive imputation for short to medium-sized gaps.
problem Missing data in time series analysis.
method Adaptive imputation method for univariate time series with tailored strategies for different gap positions.
result KZImputer achieves strong performance, especially for high missingness rates and high-sparsity regimes.
Optimal privacy-preserving algorithm for solving saddle point problems.
problem Solving convex-concave stochastic saddle point problems under differential privacy constraints.
method Recursive regularization technique repurposed for saddle point problems, achieving strong gap rate of O(1/√n + √d/nε).
result Achieves nearly optimal strong gap rate of O(1/√n + √d/nε) with gradient complexity O(min{n^2ε^(1.5)/√d, n^(3/2)}).
New method uses model's generalization gap to predict membership inference attacks.
problem Predicting membership inference attacks on machine learning models.
method Uses the model's generalization gap as a metric to estimate the vulnerability to membership inference attacks.
result Demonstrates that the model's generalization gap provides an upper bound on MIA security.
We introduce a novel class of localized atomic environment representations, based upon the Coulomb matrix. By combining these functions with the Gaussian approximation potential approach, we present LC-GAP, a new system for generating atomic potentials through machine learning (ML). Tests on the QM7, QM7b and GDB9 biom…
We consider the weakly supervised binary classification problem where the labels are randomly flipped with probability 1−α. Although there exist numerous algorithms for this problem, it remains theoretically unexplored how the statistical accuracies and computational efficiency of these algorithms depend on the degr…
New theorem improves spectral gap for sampling from mixture distributions.
problem Sampling from multimodal distributions with simulated tempering.
method Introduced a decomposition theorem for the restricted spectral gap of simulated tempering.
result Lower bound on the restricted spectral gap for mixture distributions.
We demonstrate, theoretically and empirically, that adversarial robustness can significantly benefit from semisupervised learning. Theoretically, we revisit the simple Gaussian model of Schmidt et al. that shows a sample complexity gap between standard and robust classification. We prove that unlabeled data bridges thi…
FinMaster benchmarks LLMs in financial tasks, revealing gaps in reasoning.
problem Challenges in financial tasks, including labor-intensive processes and low error tolerance.
method Developed a comprehensive financial benchmark (FinMaster) with three modules: FinSim, FinSuite, and FinEval.
result LLMs struggle with complex financial reasoning, showing significant accuracy drops.
WR-CP reduces prediction set size and coverage gap under distribution shift.
problem Guaranteed coverage under distribution shift not achievable with i.i.d. assumption.
method Wasserstein distance, probability measure pushforwards, importance weighting, regularized representation learning.
result Reduces coverage gap to 3.2% across different confidence levels.
Dropout, a simple and effective way to train deep neural networks, has led to a number of impressive empirical successes and spawned many recent theoretical investigations. However, the gap between dropout's training and inference phases, introduced due to tractability considerations, has largely remained under-appreci…
BITS for GAPS uses Bayesian methods to improve surrogate model accuracy in complex systems.
problem Improving surrogate model accuracy in complex physical systems with uncertainty.
method Bayesian Information-Theoretic Sampling for hierarchical Gaussian Process Surrogates.
result Increased expected information gain and predictive accuracy by targeting high-uncertainty regions.
This work analyzes how multi-agent reinforcement learning can bridge the gap to reality in distributed multi-robot systems.
problem Collaborative learning in distributed multi-robot systems with varying sensors and actuators.
method Simulation-based analysis using PPO and Bullet physics engine, considering different types of perturbations.
result PPO's robustness is affected by the presence of different types of perturbations and the number of agents experiencing them.
Crowdsourcing platforms provide marketplaces where task requesters can pay to get labels on their data. Such markets have emerged recently as popular venues for collecting annotations that are crucial in training machine learning models in various applications. However, as jobs are tedious and payments are low, errors …
Adversarial training leads to large generalization gap, decomposed into bias and variance.
problem Understanding the large generalization gap in adversarially trained models.
method Bias-Variance decomposition of test risk as a function of adversarial perturbation radius.
result Bias increases monotonically with adversarial perturbation radius and is dominant in test risk.
A necessary characteristic for the deployment of deep learning models in real world applications is resistance to small adversarial perturbations while maintaining accuracy on non-malicious inputs. While robust training provides models that exhibit better adversarial accuracy than standard models, there is still a sign…
Cloud computing is becoming increasingly popular as a platform for distributed training of deep neural networks. Synchronous stochastic gradient descent (SSGD) suffers from substantial slowdowns due to stragglers if the environment is non-dedicated, as is common in cloud computing. Asynchronous SGD (ASGD) methods are i…
CRC improves multivariate forecasting accuracy without risking performance degradation.
problem Systematic errors and lack of guarantees in multivariate forecasters.
method CRC uses a causality-inspired encoder and hybrid corrector with a safety mechanism.
result CRC consistently improves accuracy and ensures high non-degradation rates.
Minimalistic unsupervised learning with sparse manifold transform achieves SOTA performance.
problem Achieving state-of-the-art unsupervised learning performance without complex engineering.
method Sparse manifold transform, leveraging sparse coding, manifold learning, and slow feature analysis.
result 99.3% KNN top-1 accuracy on MNIST, 81.1% on CIFAR-10, and 53.2% on CIFAR-100.
New methods show robustness and accuracy can coexist.
problem Inevitability of robustness-accuracy tradeoff in deep learning.
method Prove robustness and accuracy achievable through locally Lipschitz functions; explore combining dropout with robust training methods.
result Achieving robustness and accuracy requires methods imposing local Lipschitzness and deep learning generalization techniques.
Recent advances in stochastic gradient variational inference have made it possible to perform variational Bayesian inference with posterior approximations containing auxiliary random variables. This enables us to explore a new synthesis of variational inference and Monte Carlo methods where we incorporate one or more s…
Researchers study fairness-accuracy tradeoffs in predictive models for multiple groups.
problem Understanding the tradeoff between fairness and accuracy in models serving multiple demographic groups.
method Characterizing the fairness-accuracy (FA) Pareto frontier, approximating it from limited data, and bounding the worst-case gap.
result Derivation of worst-case-optimal estimators and uniform finite-sample bounds for the entire FA frontier.
Motivated by the gap between theoretical optimal approximation rates of deep neural networks (DNNs) and the accuracy realized in practice, we seek to improve the training of DNNs. The adoption of an adaptive basis viewpoint of DNNs leads to novel initializations and a hybrid least squares/gradient descent optimizer. We…
Foundation models improve wage gap decomposition by capturing omitted career history factors.
problem Estimating wage disparities using incomplete career history data.
method Fine-tuning foundation models to mitigate omitted variable bias and estimate wage gaps.
result Foundation models can decompose gender wage gaps more accurately than traditional econometric methods.
Data augmentation has been widely applied as an effective methodology to improve generalization in particular when training deep neural networks. Recently, researchers proposed a few intensive data augmentation techniques, which indeed improved accuracy, yet we notice that these methods augment data have also caused a …
Generative models improve adversarial robustness by adding synthetic data.
problem Improving robustness in machine learning models trained on limited data.
method Using synthetic data generated from a large dataset to augment the original training set.
result Generative models can significantly reduce the robust-accuracy gap compared to models trained with additional real data.
A new topology improves decentralized learning efficiency and accuracy.
problem Finding efficient decentralized learning topologies with fast consensus and low maximum degree.
method Proposed the Base-(k+1) Graph topology for decentralized learning. result The Base-(k+1) Graph enables faster convergence and better communication efficiency than the exponential graph. Sparse attention model reduces long-context inference time with exponential accuracy guarantees.
problem Efficiently processing long-context queries in large language models.
method Formalizes attention as a projection onto key vectors, analyzes entropic relaxation, and introduces Vashista Sparse Attention.
result Sparse attention concentrates on a constant-size active face, leading to exponential decay of inactive tokens' mass and linear scaling of active face error.
As shown in recent research, deep neural networks can perfectly fit randomly labeled data, but with very poor accuracy on held out data. This phenomenon indicates that loss functions such as cross-entropy are not a reliable indicator of generalization. This leads to the crucial question of how generalization gap should…
Machine learning models are often susceptible to adversarial perturbations of their inputs. Even small perturbations can cause state-of-the-art classifiers with high "standard" accuracy to produce an incorrect prediction with high confidence. To better understand this phenomenon, we study adversarially robust learning …
New method corrects quantization errors in LLMs using low-rank matrices.
problem Correcting quantization errors in large language models.
method Introducing low-rank weight matrices to correct quantized activations in LLMs.
result Reduces accuracy gap with original model by more than 50% using low-rank matrices.
Improved random forest proximities capture data geometry.
problem Inaccurate random forest proximities do not reflect learned data geometry.
method Introduce RF-GAP: Geometry- and Accuracy-Preserving proximities.
result RF-GAP improves geometric representation in tasks like data imputation.
We develop a dual-control method for approximating investment strategies in incomplete environments that emerge from the presence of trading constraints. Convex duality enables the approximate technology to generate lower and upper bounds on the optimal value function. The mechanism rests on closed-form expressions per…
Improves spline quality and accuracy in computational microscopy.
problem Detecting slender, overlapping structures in microscopy images.
method Differentiable rendering approach for spline refinement.
result Achieves high reliability and sub-pixel accuracy.
JTT improves model worst-group accuracy without group annotations.
problem Low worst-group accuracy in standard ERM models with spurious correlations.
method Two-stage approach: first ERM, then upweight misclassified examples.
result JTT closes 75% of the gap in worst-group accuracy compared to group DRO.
This paper analyzes the conflict between Hamming loss and subset accuracy in multi-label classification.
problem The conflict between Hamming loss and subset accuracy in multi-label classification.
method The paper analyzes the learning guarantees of algorithms optimizing Hamming loss and subset accuracy, providing theoretical bounds and experimental support.
result Optimizing Hamming loss with its surrogate loss can lead to good performance on subset accuracy in small label spaces, contrary to theoretical expectations.
Differential privacy (DP) is a popular mechanism for training machine learning models with bounded leakage about the presence of specific points in the training data. The cost of differential privacy is a reduction in the model's accuracy. We demonstrate that in the neural networks trained using differentially private …
This paper analyzes the trade-off between accuracy and communication in personalized federated learning.
problem The accuracy-communication trade-off in personalized federated learning.
method The paper provides a quantitative characterization of the personalization degree on the trade-off, establishing minimax optimality.
result The paper offers theoretical insights for choosing the personalization degree and validates the results on synthetic and real-world datasets.
Paper analyzes complexity of solving nonconvex-strongly-concave problems.
problem Finding approximate stationary points of nonconvex-strongly-concave minimax problems.
method Introduces a generic acceleration scheme to solve crafted subproblems.
result Algorithm nearly matches lower complexity bounds in general setting.
It has been demonstrated that deep neural networks are prone to noisy examples particular adversarial samples during inference process. The gap between robust deep learning systems in real world applications and vulnerable neural networks is still large. Current adversarial training strategies improve the robustness ag…
Unified approach optimizes neural network training for various metrics.
problem Training and evaluation of neural network binary classifiers often use different metrics.
method Combines differentiable approximation and probabilistic soft sets.
result Effective in optimizing for metrics like F1-Score across various domains.
DARTS is a popular algorithm for neural architecture search (NAS). Despite its great advantage in search efficiency, DARTS often suffers weak stability, which reflects in the large variation among individual trials as well as the sensitivity to the hyper-parameters of the search process. This paper owes such instabilit…
Adaptive object detection method synthesizes target domain images from source domain images.
problem Large domain gap between source and target domains in object detection.
method Cross-domain CutMix with adversarial learning.
result Higher accuracy in different domain settings compared to conventional methods.
New classifier combines locally linear kernels for fast and accurate non-linear classification.
problem Developing a fast and accurate non-linear classifier.
method Combines locally linear classifiers using a ℓ1 Multiple Kernel Learning (MKL) problem with scalable MKL training for streaming kernels. result The resulting classifier achieves high accuracy with fast inference time.
Hinge-FM2I fills missing data in time series with high accuracy.
problem Handling missing data in univariate time series.
method Inspired by door hinges, Hinge-FM2I imputes missing data using FM2I and selects the best imputed gap.
result Hinge-FM2I significantly outperforms other methods in sMAPE scores.
We improve private training accuracy with learning rate schedules and matrix factorizations.
problem Private training with learning rate schedules and correlated noise.
method General upper and lower bounds for learning rate schedules, memory-efficient constructions, and schedule-aware factorizations.
result Schedule-aware factorizations improve accuracy in private training.