FetchSGD reduces communication in federated learning with sketching.
problem Communication bottlenecks and convergence issues in federated learning.
method FetchSGD uses Count Sketch to compress and merge model updates efficiently.
result FetchSGD achieves high compression rates and good convergence without sparse client participation.
Bias correction needed after deep learning regression training.
problem Systematic error accumulation in deep learning regression models.
method Adjust bias of the machine learning model post-training.
result Bias correction efficiently solves error accumulation.
New data accumulation prevents model collapse in generative models.
problem Model collapse in generative models trained on their own outputs.
method Empirical study of language models, diffusion models, and variational autoencoders; analytically tractable framework for linear models.
result Accumulating synthetic data alongside real data avoids model collapse, preventing performance degradation.
Improved statistical computation through efficient matrix sampling.
problem Reducing computational cost in large-scale statistical methods.
method Accumulative sub-sampling method to improve statistical efficiency.
result Effective matrix size control improves computational efficiency.
Proposes reducing random error in stochastic optimization by variance regularization.
problem Random error accumulation in stochastic optimization algorithms.
method Regularizes learning-rate based on mini-batch variances.
result Speeds up convergence and stabilizes stochastic optimization.
VCoTTA uses variational Bayesian methods to adapt models under continuous domain shifts.
problem Error accumulation in continual test-time adaptation.
method VCoTTA employs variational Bayesian techniques to update a Bayesian Neural Network (BNN) during testing, combining priors from source and teacher models.
result VCoTTA effectively mitigates error accumulation in CTTA, as shown by experimental results on three datasets.
Off-policy reinforcement learning aims to leverage experience collected from prior policies for sample-efficient learning. However, in practice, commonly used off-policy approximate dynamic programming methods based on Q-learning and actor-critic methods are highly sensitive to the data distribution, and can make only …
The paper analyzes the error accumulation in a compositional score-based algorithm for SBI.
problem How to effectively combine multiple observations to improve parameter inference.
method Study of the GAUSS algorithm's compositional score and its mean squared error.
result Established an upper bound on the mean squared error of the compositional score.
Lifelong learning can be viewed as a continuous transfer learning procedure over consecutive tasks, where learning a given task depends on accumulated knowledge --- the so-called knowledge base. Most published work on lifelong learning makes a batch processing of each task, implying that a data collection step is requi…
In many large-scale machine learning applications, data are accumulated with time, and thus, an appropriate model should be able to update in an online paradigm. Moreover, as the whole data volume is unknown when constructing the model, it is desired to scan each data item only once with a storage independent with the …
Proposes a non-autoregressive Transformer for time series forecasting.
problem Autoregressive errors and spatial-temporal dependencies in time series forecasting.
method Introduces a Non-Autoregressive Transformer with a learned temporal influence map.
result Demonstrates state-of-the-art performance on time series forecasting datasets.
In this paper, we consider a best action identification problem in the stochastic linear bandit setup with a fixed confident constraint. In the considered best action identification problem, instead of minimizing the accumulative regret as done in existing works, the learner aims to obtain an accurate estimate of the u…
Training-free method improves text-to-image generation quality.
problem Error accumulation in simultaneous token updates of masked diffusion models.
method Training-free self-correction framework exploiting inductive biases.
result Significantly improved generation quality on text-to-image tasks.
Model compression has been widely adopted to obtain light-weighted deep neural networks. Most prevalent methods, however, require fine-tuning with sufficient training data to ensure accuracy, which could be challenged by privacy and security issues. As a compromise between privacy and performance, in this paper we inve…
The paper analyzes how synthetic data training degrades diffusion models, providing bounds and characterizing different drift regimes.
problem The degradation of performance in diffusion models trained on synthetic data.
method Theoretical analysis of score-based diffusion models, focusing on the accumulated divergence between generated and target distributions.
result Upper and lower bounds on the accumulated divergence, providing the first lower bound for diffusion models.
Analyzes error sources in global feature effect estimation methods.
problem Unexplored error sources in global feature effect estimation methods.
method Systematic, estimator-level analysis of bias and variance.
result Holdout data is theoretically cleanest, but estimation variance depends on sample size and model characteristics.
MQF2 forecasts multivariate quantiles globally.
problem Forecasting multi-horizon dependencies with error accumulation.
method Multivariate quantile function using input-convex neural networks.
result MQF2 avoids quantile crossing and captures time dependency. Efforts to reduce the numerical precision of computations in deep learning training have yielded systems that aggressively quantize weights and activations, yet employ wide high-precision accumulators for partial sums in inner-product operations to preserve the quality of convergence. The absence of any framework to an…
Paper introduces a method to control early classification accuracy gaps.
problem Maintaining accuracy in early classification without full input processing.
method Statistical framework for a calibrated stopping rule.
result Reduces up to 94% of timesteps while controlling accuracy gaps.
The paper improves self-training in semi-supervised learning by selecting more robust pseudo-labeled data.
problem Improving the reliability of pseudo-labeled data selection in self-training for semi-supervised learning.
method Proposes a multi-objective utility function to select pseudo-labeled data that maximizes reliability, considering model selection, accumulation of errors, and covariate shift uncertainties.
result Robustness towards model choice can lead to substantial accuracy gains in self-training.
For a given cusped 3-manifold M admitting an ideal triangulation, we describe a method to rigorously prove that either M or a filling of M admits a complete hyperbolic structure via verified computer calculations. Central to our method are an implementation of interval arithmetic and Krawczyk's Test. These techni…
A method to improve sequential learning by keeping past data errors in check.
problem Challenges in sequential learning with Gaussian processes due to accumulating errors.
method Memory-based dual sparse variational Gaussian processes.
result Improves accuracy in inference and learning for various applications.
New method uses randomized sparse neural networks to solve time-dependent PDEs more accurately and efficiently.
problem Numerical challenges in training neural networks sequentially in time to solve time-dependent PDEs.
method Introduces Neural Galerkin schemes that update randomized sparse subsets of network parameters at each time step.
result Up to two orders of magnitude more accurate and two orders of magnitude faster than dense update schemes.
Accumulated stock returns exhibit tempered skew t-distribution.
problem Analyzing the distribution of stock returns over multiple days.
method Employing a tempered skew t-distribution model.
result Tempered skew t-distribution fits the distribution of accumulated stock returns well.
Generative diffusion models improve financial LOB simulation and forecasting.
problem High noise and complexity in financial LOB data makes deep generative models ineffective.
method Convert LOB data to images, apply diffusion models with inpainting for long-term sequence generation.
result Our method achieves state-of-the-art performance on LOB-Bench, improving coherence over local details.
Proposes QEP to mitigate quantization error propagation in layer-wise post-training quantization.
problem Growth of quantization errors across layers degrades performance, especially in low-bit regimes.
method Quantization Error Propagation (QEP) framework that explicitly propagates and compensates for quantization errors.
result QEP-enhanced layer-wise PTQ achieves substantially higher accuracy, especially in low-bit regimes.
Latent Block-Diffusion Temporal Point Processes (LBDTPP) is a semi-autoregressive framework for generating asynchronous event sequences.
problem Generating asynchronous event sequences
method Latent Block-Diffusion Temporal Point Processes
result Outperforms state-of-the-art TPP baselines in both unconditional and conditional generation tasks
A new method for pricing derivatives using self-exciting dynamics and finite-difference transforms.
problem Pricing derivatives with accumulated marks using a self-exciting marked point process.
method Derive discounted pricing equation as a PIDE, transform to one-dimensional PIDEs, use Laplace/Fourier transform, approximate jump term, solve using finite difference scheme.
result Efficiently price derivatives with accumulated marks using a novel finite-difference and transform approach.
This paper analyzes how errors accumulate in PCA's deflation method.
problem Error accumulation in PCA's deflation method.
method Mathematical analysis of inexact Hotelling's deflation method in two scenarios.
result Characterization of error propagation in PCA's deflation method.
Paper analyzes Langevin dynamics for multimodal Gaussian mixtures, controlling errors across dimensions.
problem Challenges in obtaining stable diffusion-based samplers in high- and infinite-dimensional settings.
method Study of preconditioned Annealed Langevin Dynamics (ALD) for Gaussian mixtures, focusing on Euler-Maruyama (EM) and exponential-integrator schemes.
result Proves dimension-uniform KL bounds for the exponential-integrator scheme, allowing arbitrarily small divergence with dimension.
Unfolding paths in Outer space accumulate on a simplex, not converge.
problem Understanding accumulation points in Outer space.
method Constructing an unfolding path in Outer space.
result Unfolding paths accumulate on a 1-simplex, not converge.
Paper explores low-precision SGLD for neural networks, reducing costs without sacrificing performance.
problem Infeasibility of low-precision sampling in large-scale scenarios.
method Developed low-precision SGLD with quantization function and full-precision gradient accumulators.
result Low-precision SGLD achieves comparable performance to full-precision SGLD with only 8 bits.
Unified framework for high-dimensional online learning with non-divergent error bounds and adaptive gains.
problem Divergence of error bounds in high-dimensional online learning as data batches increase.
method Asynchronous decomposition framework with summary statistics and dynamic regularization.
result Non-divergent error bounds and adaptive gains in sparse online optimization.
Novel method for nonlinear data assimilation using Langevin sampling.
problem Nonlinear data assimilation challenges in Bayesian filtering.
method Score-based sequential Langevin sampling (SSLS) with dynamic models and annealing.
result Asymptotic stability and error bounds for local posterior sampling.
Wealth inequality is an important matter for economic theory and policy. Ongoing debates have been discussing recent rise in wealth inequality in connection with recent development of active financial markets around the world. Existing literature on wealth distribution connects the origins of wealth inequality with a v…
Neural CDEs correct errors in learned time-series models for better forecasting.
problem Error accumulation in multi-step forecasts of learned time-series models.
method Predictor-Corrector framework with a neural controlled differential equation.
result The proposed framework consistently improves forecasting performance across various models.
Study of historic stock returns distributions, highlighting asymmetry and outliers.
problem Understanding the asymmetry in accumulated gains and losses in stock returns over time.
method Analyzing decades-long historic distributions of S&P500 returns, comparing gains and losses, using statistical U-tests and fitting log-log scale linearly.
result The mean of de-trended distributions increases linearly with the number of days of accumulation, and the overall skew is negative, indicating heavier tails of losses.
In this paper, we propose a novel policy iteration method, called dynamic policy programming (DPP), to estimate the optimal policy in the infinite-horizon Markov decision processes. We prove the finite-iteration and asymptotic l\infty-norm performance-loss bounds for DPP in the presence of approximation/estimation erro…
Markowitz (1952, 1959) laid down the ground-breaking work on the mean-variance analysis. Under his framework, the theoretical optimal allocation vector can be very different from the estimated one for large portfolios due to the intrinsic difficulty of estimating a vast covariance matrix and return vector. This can res…
In this paper, finite type domains with hyperbolic orbit accumulation points are studied. We prove, in case of C2, it has to be a (global) pseudoconvex domain, after an assumption of boundary regularity. Moreover, one of the applications will realize the classification of domains within this class, precisel…
This paper presents a numerical model to solve the problem of cash accumulation strategies for products with an unknown future price, like assets. Stock prices are modeled by a discretized Wiener Process, and by the means of ordinary integrals this Wiener Process will be exactly matched at a preset terminal time. Three…
Several statistical and machine learning methods are proposed to estimate the type and intensity of physical load and accumulated fatigue . They are based on the statistical analysis of accumulated and moving window data subsets with construction of a kurtosis-skewness diagram. This approach was applied to the data gat…
Full-batch GD achieves generalization close to any stationary point with fewer assumptions.
problem Generalization and excess risk bounds for smooth losses, including non-Lipschitz and nonconvex cases.
method Path-dependent analysis of GD's generalization error, focusing on optimization error and stability.
result Generalization error is tightly bound in terms of optimization error and iteration count, bypassing common assumptions.
Paper presents a method to reduce prediction variance of DNNs for unknown systems.
problem Uncertainty in DNN predictions due to high variance.
method Ensemble averaging of multiple DNN models trained independently.
result Reduction in variance of DNN predictions, improving reliability.
Analyzes multi-day stock returns, showing linear volatility and mean dependence.
problem Linear dependence of volatility and mean in accumulated stock returns.
method Modified Jones-Faddy skew t-distribution analysis.
result Linear dependence of volatility and mean on the number of days of accumulation.
An open question in systems and computational neuroscience is how neural circuits accumulate evidence towards a decision. Fitting models of decision-making theory to neural activity helps answer this question, but current approaches limit the number of these models that we can fit to neural data. Here we propose a unif…
There is widespread sentiment that it is not possible to effectively utilize fast gradient methods (e.g. Nesterov's acceleration, conjugate gradient, heavy ball) for the purposes of stochastic optimization due to their instability and error accumulation, a notion made precise in d'Aspremont 2008 and Devolder, Glineur, …
Sequential hypothesis testing is a desirable decision making strategy in any time sensitive scenario. Compared with fixed sample-size testing, sequential testing is capable of achieving identical probability of error requirements using less samples in average. For a binary detection problem, it is well known that for k…