OTS error shows small training error doesn't guarantee small test error.
problem The relationship between training and test errors is unclear.
method An analysis of the conditions under which small training set error guarantees small OTS error.
result The theorem is limited to models with distinct training and test distributions.
Off-policy reinforcement learning aims to leverage experience collected from prior policies for sample-efficient learning. However, in practice, commonly used off-policy approximate dynamic programming methods based on Q-learning and actor-critic methods are highly sensitive to the data distribution, and can make only …
SCORE resolves the robustness vs accuracy trade-off by redefining robust error.
problem The inherent trade-off between robustness and accuracy in adversarial training.
method SCORE defines local equivariance as the ideal robust behavior, leading to a new robust error metric.
result SCORE reconciles robustness and accuracy, improving model performance on RobustBench.
Asynchronous SGD can speed up training with a trade-off of gradient staleness.
problem Asynchronous SGD suffers from gradient staleness, affecting convergence error.
method Theoretical analysis of the error-runtime trade-off considering random straggling delays.
result A method of gradually varying synchronicity in distributed SGD is proposed and demonstrated.
Study shows exponential error reduction in multiclass classification without bias-variance trade-off.
problem Multiclass classification with margin conditions.
method Analysis of classification error under hard-margin conditions.
result Exponential decrease in classification error without bias-variance trade-off.
The paper analyzes CycleGAN's error components for unpaired data generation.
problem Analyzing approximation and estimation errors in CycleGAN for unpaired data.
method Decomposes risk into approximation and estimation errors, analyzing each separately and considering their trade-offs.
result Theoretical insights into CycleGAN's performance through error analysis.
Greedy training of recursive partitioning estimators faces a computational barrier when the true function doesn't satisfy a specific property.
problem Computational inefficiency of greedy training for recursive partitioning estimators.
method Analysis of greedy training for sparse regression functions over binary features.
result Greedy training requires exponential samples when the true function doesn't satisfy a specific property (MSP), but only logarithmic samples when it does.
Differentially private GANs improve image privacy without significant quality loss.
problem Anonymizing image data sets while maintaining image quality.
method Training GANs with differential privacy on MNIST, analyzing privacy-utility trade-offs and explaining optimization methods.
result An increasing privacy budget adds little to generated image quality, revealing a saturated training regime.
This paper studies the problem of error-runtime trade-off, typically encountered in decentralized training based on stochastic gradient descent (SGD) using a given network. While a denser (sparser) network topology results in faster (slower) error convergence in terms of iterations, it incurs more (less) communication …
In this paper we study the stability and its trade-off with optimization error for stochastic gradient descent (SGD) algorithms in the pairwise learning setting. Pairwise learning refers to a learning task which involves a loss function depending on pairs of instances among which notable examples are bipartite ranking,…
This paper studies trade-offs in private prediction methods.
problem Leakage of training data information in machine learning predictions.
method Private training and private prediction methods with trade-offs.
result Private training methods outperform private prediction methods in various settings.
We consider the problem of off-policy evaluation in Markov decision processes. Off-policy evaluation is the task of evaluating the expected return of one policy with data generated by a different, behavior policy. Importance sampling is a technique for off-policy evaluation that re-weights off-policy returns to account…
New method finds coresets for faster machine learning.
problem Improving training speed and understanding results.
method Iterative optimization using multi-objective evolutionary algorithms.
result Results deliver lower error and better generalization.
Stochastic gradient descent updates parameters with summation gradient computed from a random data batch. This summation will lead to unbalanced training process if the data we obtained is unbalanced. To address this issue, this paper takes the error variance and error mean both into consideration. The adaptively adjus…
Efficiently approximates fairness-accuracy trade-offs for diverse datasets.
problem Inherent trade-off between fairness and accuracy in machine learning models.
method You-Only-Train-Once (YOTO) framework for computationally efficient approximation.
result Robust methodology for auditing model fairness with statistical guarantees.
Paper tackles non-Markovian control problems with new learning methods.
problem Non-Markovian stochastic control problems with unknown parameters.
method Off-model training and importance sampling for deep neural network approximation.
result Quantitative error bounds for adaptive learning under model uncertainty.
Stochastic RNNs classify biological neural network paths with robust error bounds.
problem Classifying biological neural network paths.
method Modelled as a continuous-time stochastic recurrent neural network (RNN) with identity activation function, analysed in the robust regime.
result Generalisation error bound holds with high probability, showing the empirical risk minimiser is the best-in-class hypothesis.
Improves ASR accuracy for domain mismatch using machine translation.
problem Domain mismatch in ASR systems leads to suboptimal results.
method Machine translation to map out-of-domain ASR errors to in-domain terms.
result 7% absolute improvement in word error rate, 4 point BLEU score improvement.
New method for distributional off-policy evaluation using Bellman residual minimization.
problem Learning return distribution from offline data generated by a different policy.
method Energy Bellman Residual Minimizer (EBRM) method.
result Established finite-sample error bound for EBRM estimator.
Distributed Stochastic Gradient Descent (SGD) when run in a synchronous manner, suffers from delays in waiting for the slowest learners (stragglers). Asynchronous methods can alleviate stragglers, but cause gradient staleness that can adversely affect convergence. In this work we present a novel theoretical characteriz…
Unified framework for error quantification in off-policy evaluation with distributional shift.
problem Establishing high-confidence CI for target policy value from offline data.
method Unified error analysis quantifying misspecification and sampling errors.
result Achieves tightest possible CI and robustness against distributional shifts.
New insights into bias and variance in over-parameterized models.
problem Understanding bias and variance in over-parameterized models.
method Analytic expressions derived from statistical physics for two minimal models.
result Over-parameterized models can overfit even in noiseless conditions.
Improves bit error tolerance in RRAM-based BNNs without overfitting.
problem Bit errors in RRAM-based BNNs reduce accuracy and overfit to training error rates.
method Proposes straight-through gradient approximation and a novel regularizer.
result Improves BNNs' robustness to bit errors without overfitting.
Study tests whether trade-off functions are above or below benchmarks using finite samples.
problem Testing trade-off functions between unknown distributions.
method Identifies a condition for nontrivial testing, constructs a test with error guarantees, and inverts the test for confidence bands.
result Finite-sample testing is possible under specific structural assumptions about rejection regions.
Adapts GRPO for off-policy RL, improving reward.
problem Improving training stability and efficiency in RL.
method Adapts GRPO to off-policy setting, uses clipped surrogate objectives.
result Off-policy GRPO outperforms on-policy GRPO in empirical tests.
When learning from a batch of logged bandit feedback, the discrepancy between the policy to be learned and the off-policy training data imposes statistical and computational challenges. Unlike classical supervised learning and online learning settings, in batch contextual bandit learning, one only has access to a colle…
Sparse modeling improves portfolio optimization by reducing errors in complex market systems.
problem Errors in multivariate modeling of markets and economy.
method L0-norm sparse elliptical modeling to reduce oversimplification, and study likelihood in- and out-of-sample for different parameter lengths.
result Sparse models lead to better portfolio performance, higher out-of-sample likelihood, and lower volatility.
This work provides theoretical and empirical evidence that invariance-inducing regularizers can increase predictive accuracy for worst-case spatial transformations (spatial robustness). Evaluated on these adversarially transformed examples, we demonstrate that adding regularization on top of standard or adversarial tra…
Deep neural networks can generalize by reducing high-frequency noise over time, not always following a monotonic learning bias.
problem Understanding the learning dynamics and generalization of over-parameterized DNNs.
method Experimental analysis of deep double descent, focusing on the spectral bias of DNNs.
result The high-frequency components of DNNs diminish over training, leading to a second descent in test error.
This work studies the problem of batch off-policy evaluation for Reinforcement Learning in partially observable environments. Off-policy evaluation under partial observability is inherently prone to bias, with risk of arbitrarily large errors. We define the problem of off-policy evaluation for Partially Observable Mark…
Optimal defenses protect FL models from gradient reconstruction attacks.
problem Gradient reconstruction attacks compromise FL models by recovering original data from shared gradients.
method Derive a theoretical lower bound of reconstruction error, customize noise and pruning defenses, and achieve optimal trade-off between leakage and utility.
result Our methods outperform Gradient Noise and Pruning in protecting training data and maintaining model utility.
This paper quantifies privacy-robustness and generalization-robustness trade-offs in adversarial training.
problem Privacy and generalization issues in adversarial training.
method Defines robustified intensity and empirical robustified intensity to measure robustness, proving differential privacy and generalization bounds.
result Proves adversarial training is (ε,δ)-differentially private and provides generalization bounds. Large-scale machine learning training, in particular distributed stochastic gradient descent, needs to be robust to inherent system variability such as node straggling and random communication delays. This work considers a distributed training framework where each worker node is allowed to perform local model updates a…
New methods for off-policy evaluation in reinforcement learning.
problem Improving off-policy evaluation in reinforcement learning.
method Developed MWL and MQL estimators for importance weights and value functions.
result MWL and MQL provide a unified view of algorithms in reinforcement learning.
Paper analyzes gradient descent with noisy data copies for linear regression, showing regularization and acceleration effects.
problem Improving generalization in machine learning through data augmentation with noise.
method Gradient descent with on-line noisy copies for linear regression analysis.
result Training with on-line noisy copies is equivalent to ridge regularization with a specific regularization parameter.
Boosting algorithms produce a classifier by iteratively combining base hypotheses. It has been observed experimentally that the generalization error keeps improving even after achieving zero training error. One popular explanation attributes this to improvements in margins. A common goal in a long line of research, is …
New algorithm reduces bias in off-policy reinforcement learning.
problem Challenges in designing off-policy reinforcement learning algorithms.
method Doubly robust off-policy actor-critic (DR-Off-PAC) with a single timescale structure.
result Establishes the first overall sample complexity analysis for a single time-scale off-policy AC algorithm.
Simplifies RL training with fewer techniques, reducing bias and instability.
problem Training instabilities and high sample complexity in RL.
method Introduced a simple deterministic policy gradient, used propensity estimation, and delayed policy updates.
result Improved performance and reduced sample complexity through these techniques.
Unified coverage analysis for linear off-policy evaluation in reinforcement learning.
problem Lack of a unified understanding of coverage parameters in linear off-policy evaluation.
method Developed a novel finite-sample analysis for LSTDQ algorithm, introducing feature-dynamics coverage.
result Unified understanding of coverage parameters in linear off-policy evaluation.
New algorithm for sequential off-policy learning improves performance over batch methods.
problem Training policies from logged interaction data in a sequential setting.
method Combines Logarithmic Smoothing with online PAC-Bayesian tools.
result Improves performance and accelerates convergence in sequential off-policy learning.
The intrinsic error tolerance of neural network (NN) makes approximate computing a promising technique to improve the energy efficiency of NN inference. Conventional approximate computing focuses on balancing the efficiency-accuracy trade-off for existing pre-trained networks, which can lead to suboptimal solutions. In…
Flexible deep learning models for dynamic accuracy and speed trade-offs.
problem Dynamic accuracy and speed trade-offs in real-world applications.
method Training deep neural networks with a new method allowing flexible numerical precision during inference.
result Achieved comparable accuracy to dedicated models trained at the same precision with dynamic precision settings.
The overall performance or expected excess risk of an iterative machine learning algorithm can be decomposed into training error and generalization error. While the former is controlled by its convergence analysis, the latter can be tightly handled by algorithmic stability. The machine learning community has a rich his…
Paper improves DNN accelerator robustness against bit errors with energy savings.
problem Bit errors in quantized DNN weights reduce energy efficiency.
method Combines robust fixed-point quantization, weight clipping, and random bit error training.
result Significantly improves robustness against random bit errors with high energy savings.
We develop an off-policy actor-critic algorithm for learning an optimal policy from a training set composed of data from multiple individuals. This algorithm is developed with a view towards its use in mobile health.
Sparse coding in learned dictionaries has been established as a successful approach for signal denoising, source separation and solving inverse problems in general. A dictionary learning method adapts an initial dictionary to a particular signal class by iteratively computing an approximate factorization of a training …
This paper studies optimal approximation factors in misspecified off-policy RL, identifying key factors under various settings.
problem Understanding optimal approximation factors in misspecified off-policy value function estimation.
method Examined various settings including weighted L2-norm, L∞ norm, state aliasing, and state coverage. result Established optimal asymptotic approximation factors for different norms and identified two instance-dependent factors for L2(μ) norm. We analyze Gibbs-based transfer learning algorithms using information theory.
problem Understanding the generalization error of transfer learning.
method Information-theoretic analysis focusing on α-weighted-ERM and two-stage-ERM. result Exact characterization of generalization behavior using conditional symmetrized KL information.