RNA accelerates CNNs for image recognition.
problem Improving the optimization process of CNNs for image recognition.
method Regularized Nonlinear Acceleration (RNA) applied to neural networks.
result RNA improves the optimization process of CNNs slightly.
New RNA scheme accelerates gradient methods online and improves convergence.
problem Improving convergence rates of gradient methods.
method Adapting Regularized Nonlinear Acceleration (RNA) to handle faster multistep algorithms.
result Optimal complexity bounds and asymptotically optimal rates for convex minimization problems.
Proposes a new AFT model for nonlinear survival data.
problem Limited ability of classical AFT models to represent nonlinear relationships and handle complex covariate structures.
method Structured nonparametric extension using Kolmogorov--Arnold representations and unified censoring-adjusted losses.
result Method captures nonlinear effects and recovers linear structure when appropriate.
Improved Anderson acceleration speeds up nonlinear optimization.
problem Optimizing nonlinear functions efficiently.
method Combining Anderson acceleration with Chebyshev polynomials.
result Achieves optimal convergence rate for nonlinear problems.
Paper accelerates nonlinear mapping in online systems with lower time complexity.
problem Speeding up nonlinear mapping in online systems.
method Integrates an acceleration module into Dendrite Net (DD) to reduce time complexity.
result DD with AC has lower time complexity while maintaining nonlinear mapping and system identification properties.
New algorithms accelerate solving nonlinear matrix decomposition with ReLU.
problem Nonlinear matrix decomposition with ReLU function.
method Two new algorithms: A-NMD and 3B-NMD, with adaptive extrapolation and block parametrization.
result Effective algorithms accelerate solving ReLU-NMD problems.
Paper introduces XBART for nonlinear regression, outperforming XGBoost.
problem Nonlinear regression problems, especially in speed and accuracy.
method Combines Bayesian modeling and recursive partitioning for efficient, accurate predictions.
result XBART provides faster and more accurate predictions than XGBoost.
Unified framework for understanding and optimizing training acceleration.
problem Challenges in optimizing training with regularization and acceleration techniques.
method Explains how AdaGrad, RMSProp, and Adam accelerate training, and derives a generalization for L 1 L_1 L 1 -regularization. result Derives a unified mathematical framework for understanding and optimizing training acceleration.
NDI enables high-quality QSM without parameter tuning.
problem Quantitative Susceptibility Mapping (QSM) with regularization tuning issues.
method Nonlinear Dipole Inversion (NDI) using a physics-based forward model and a Variational Network (VN).
result NDI achieves high-quality QSM from as few as 2-direction data.
Accelerates sampling from Gibbs distributions using ARWP method.
problem Sampling from Gibbs distributions efficiently.
method ARWP method, combining Nesterov acceleration and regularized Wasserstein proximal.
result ARWP exhibits higher contraction rate and faster tail exploration.
A dynamical system on the total space of the fibre bundle of second order accelerations, T 2 M T^2M T 2 M , is defined as a third order vector field S S S on T 2 M T^2M T 2 M , called semispray, which is mapped by the second order tangent structure into one of the Liouville vector field. For a regular Lagrangian of second order we prove that …
Accelerates deep RL algorithms with regularized Anderson acceleration.
problem Slow convergence and sample inefficiency in model-free, off-policy deep RL.
method Proposes a general acceleration method for deep RL by extending regularized Anderson acceleration.
result Improves both learning speed and final performance of deep RL algorithms.
New method accelerates convergence for entropy-regularized reinforcement learning problems.
problem Slow convergence of standard first-order methods for entropy-regularized Markov decision processes.
method Introduce a quadratically convexified primal-dual formulation and a new interpolating metric to accelerate convergence.
result Global convergence and exponential convergence rate for the new method.
Parallelizes feedforward computation using nonlinear equation solving.
problem Sequential nature of feedforward computation limits parallelization.
method Frame feedforward computation as solving nonlinear equations; use Jacobi or Gauss-Seidel methods for parallel updates.
result Accelerates feedforward computation with reduced parallelizable iterations.
Dropout improves neural networks by accelerating gradient flow.
problem Understanding why dropout works and improving neural network performance.
method Proposed an optimization technique to push input towards saturation area of activation functions.
result Gradient acceleration in activation function (GAAF) improves image classification performance.
Study accelerates gradient methods in machine learning, revealing risk and stability connections.
problem Understanding the statistical risk of accelerated gradient methods in machine learning.
method Continuous-time analysis of Nesterov's accelerated gradient method and Polyak's heavy ball method for least squares regression.
result Connections between early stopping, stability, and curvature of loss function are revealed.
Acceleration in Hilbert spaces reduces computations but not accuracy.
problem Improving learning accuracy with fewer computations.
method Analysis of Nesterov acceleration and heavy-ball methods in Hilbert spaces.
result Acceleration can reduce computations but not improve accuracy with respect to gradient descent.
Accelerates coordinate descent methods for machine learning problems.
problem Slowness of coordinate descent methods in machine learning.
method Extrapolation-based accelerated coordinate descent.
result Significant speed-up in practice compared to existing methods.
NO approximates non-Markovian BSDEs with polynomial scaling in 1/ε.
problem Complexity of NO approximations for structured families of BSDEs.
method Identifying structured families of non-Markovian BSDEs, informing NO's inductive bias.
result Polynomial scaling in 1/ε for NO approximations of BSDE solution operators.
DRAG decreases regularization to accelerate semi-discrete OT convergence.
problem Mitigating bias in semi-discrete OT problems with entropic regularization.
method DRAG: Decreasing Regularization Averaged Gradient, a stochastic gradient descent algorithm.
result DRAG achieves unbiased O ( 1 / t ) \mathcal{O}(1/t) O ( 1/ t ) sample and iteration complexity for OT cost and potential estimation, and O ( 1 / t ) \mathcal{O}(1/\sqrt{t}) O ( 1/ t ) rate for OT map. Note on advancements in nonlinear elliptic equations' regularity theory.
problem Nonlinear elliptic equations and their regularity.
method De Giorgi-Nash-Moser theory, Krylov-Safonov theory, Evans-Safonov theory.
result Contributions to Hilbert's 19th problem and fully nonlinear equations.
New algorithms accelerate MAP inference in Markov fields with faster convergence.
problem Finding the most likely configuration in discrete-valued Markov random fields.
method Entropy-regularized linear programming with accelerated gradient methods.
result Accelerated algorithms find optimal solutions faster, especially when the LP is tight.
Unified framework for accelerating DNNs on resource-limited platforms.
problem Accelerating DNN execution on resource-limited platforms.
method Block-based pruning framework with reweighted regularization.
result First universal framework for both CNNs and RNNs with real-time acceleration and no accuracy compromise.
New algorithms optimize constrained problems faster, avoiding full set optimization.
problem Optimizing constrained problems efficiently and quickly.
method Designing accelerated first-order algorithms that avoid full set optimization.
result Proved convergence to stationary points in nonconvex settings and accelerated rates in convex settings.
GrokAlign aligns Jacobians to accelerate grokking in deep networks.
problem Accelerating the training dynamics of deep networks to avoid delayed generalisation and robustness.
method Aligning the Jacobians of a deep network with the training data to ensure grokking under a low-rank assumption.
result GrokAlign regularizes Jacobians to induce grokking sooner than conventional methods.
ControlBurn selects few features from tree ensembles for better model interpretability.
problem Improving model interpretability in machine learning models.
method Sparse tree ensembles with lasso optimization.
result ControlBurn selects feature-sparse subsets for better model interpretability.
The paper applies momentum to CR Newton's method for nonconvex optimization, improving convergence.
problem Improving convergence in nonconvex optimization problems.
method Applying momentum to cubic regularized Newton's method.
result CR with momentum achieves the best convergence rate to second-order stationary points.
Paper discovers structural dynamics equations from only acceleration data.
problem Discovering equations from only acceleration measurements in structural dynamics.
method Library-based approach with Approximate Bayesian Computation (ABC) prioritizing parsimonious models.
result Efficacy demonstrated in four structural dynamics examples, including linear and nonlinear systems.
Improved first-order algorithm for entropy regularized OT with faster convergence.
problem Solving entropy regularized optimal transport efficiently.
method Accelerated primal-dual stochastic mirror descent algorithm with variance reduction.
result Improved rate from O ~ ( n 2.5 / ε ) \widetilde{O}({n^{2.5}}/ε) O ( n 2.5 / ε ) to O ~ ( n 2 / ε ) \widetilde{O}({n^2}/ε) O ( n 2 / ε ) . Large stepsizes can accelerate gradient descent for logistic regression.
problem Optimizing logistic regression with large stepsizes.
method Gradient descent with large stepsize for ℓ 2 \ell_2 ℓ 2 -regularized logistic regression. result Large stepsizes can achieve O ~ ( κ ) \widetilde{\mathcal{O}}(\sqrtκ) O ( κ ) convergence, improving over O ~ ( κ ) \widetilde{\mathcal{O}}(\sqrtκ) O ( κ ) from classical theory. In [19], a general, inexact, efficient proximal quasi-Newton algorithm for composite optimization problems has been proposed and a sublinear global convergence rate has been established. In this paper, we analyze the convergence properties of this method, both in the exact and inexact setting, in the case when the obje…
The paper examines the regularity of solutions to a nonlinear sigma model with gravitino in higher dimensions.
problem The study focuses on the regularity of weak solutions to a nonlinear sigma model with gravitino fields in higher dimensions.
method The authors derive the Euler--Lagrange equations and consider the regularity of weak solutions in Sobolev spaces. They show smoothness under smallness assumptions for certain Morrey norms and partial regularity for stationary solutions with higher integrability of the vector spinor.
result The paper demonstrates that weak solutions are smooth under certain conditions and shows partial regularity for stationary solutions in higher dimensions.
Autoencoders discover and accelerate molecular dynamics simulations.
problem Efficient sampling of macromolecular folding landscapes with high free energy barriers.
method Employing auto-associative artificial neural networks to learn nonlinear collective variables (CVs) that are explicit and differentiable functions of atomic coordinates.
result Substantial speedups in exploration of configurational space and discovery of data-driven CVs.
Sparsity helps reduce the computational complexity of deep neural networks by skipping zeros. Taking advantage of sparsity is listed as a high priority in next generation DNN accelerators such as TPU. The structure of sparsity, i.e., the granularity of pruning, affects the efficiency of hardware accelerator design as w…
A novel BMC model with nonconvex regularizers and accelerated proximal algorithm for binary matrix completion.
problem Recovering a binary matrix from partial observed positive elements.
method Proposes a novel BMC model with nonconvex regularizers and accelerates proximal algorithm for solving the nonconvex optimization problem.
result The proposed model and algorithm outperform other methods in both synthetic and real-world data sets.
DynNet models dynamic responses of linear and nonlinear systems with fewer variables and higher accuracy.
problem Predicting dynamic responses of linear and nonlinear systems.
method Physics-based recurrent neural network with optimized architecture and training techniques.
result Higher accuracy and fewer trainable variables compared to existing models.
Deep neural networks improve AFT model for non-linear predictors.
problem Nonlinearity in predictors of AFT models.
method Apply DNNs to fit AFT models using Gehan-type loss and sub-sampling.
result DeepR-AFT outperforms parametric and semiparametric models.
SRV learns slow molecular modes from simulations.
problem Discovering slow collective motions in molecular dynamics.
method State-free reversible VAMPnets (SRV) for nonlinear CV approximation.
result SRVs capture slow dynamics in complex systems.
Improved convergence of fixed-point methods using windowed Anderson acceleration.
problem Improving convergence of fixed-point methods for symmetric operators.
method Windowed Anderson acceleration for symmetric fixed-point iterations.
result Windowed Anderson acceleration improves convergence over standard fixed-point methods.
New method speeds up solving L0-regularized least-squares problems.
problem Solving L0-regularized least-squares problems efficiently.
method Safe peeling for Branch-and-Bound algorithm.
result Significant gains in solving time and node exploration.
We accelerate CNF by reducing ODE truncation errors with polynomial regularization.
problem High computation cost of CNF due to large truncation errors in solving ODEs.
method Add polynomial regularization to approximate ODE trajectories with polynomial functions.
result 42.3% to 71.3% reduction of NFE on density estimation, 19.3% to 32.1% on variational auto-encoder.
Paper proposes an algorithm to solve complex minimax problems efficiently.
problem Stochastic nonconvex-concave minimax problems in various fields.
method Accelerated first-order regularized momentum descent ascent algorithm (FORMDA).
result Achieves best-known complexity bound of i l d e O ( ε − 6.5 ) ilde{\mathcal{O}}(\varepsilon ^{-6.5}) i l d e O ( ε − 6.5 ) for single-loop algorithms. For a n n n -dimensional spin manifold M M M with a fixed spin structure and a spinor bundle Σ M ΣM Σ M , we prove an ε ε ε -regularity theorem for weak solutions to the nonlinear Dirac equation of cubic nonlinearity. This, in particular, answers a regularity question raised by Chen-Jost-Wang when n = 2 n=2 n = 2 .
New Hermite approximations accelerate convergence with adaptive coordinate transformations.
problem Accelerating convergence of spectral approximations for Hermite expansions.
method Using normalizing flows for adaptive coordinate transformations and deriving error estimates.
result Error estimates for Hermite expansions under adaptive coordinate transformations.
Enhances RSCNs with hybrid regularization for nonlinear dynamics.
problem Modeling nonlinear dynamic systems with uncertainties.
method Recurrent stochastic configuration networks with hybrid regularization.
result The method outperforms other models in nonlinear system identification and industrial tasks.
In this paper we study the geometrical structures on the cotangent bundle using the notions of adapted tangent structure and regular vector fields. We prove that the dynamical covariant derivative on T ∗ M T^{*}M T ∗ M fix a nonlinear connection for a given J \mathcal{J} J -regular vector field. Using the Legendre transformation in…
We improve GANs by enforcing gradient constraints for MMD critics.
problem Stabilizing and accelerating training of GANs.
method Propose a new gradient-based regularization method for MMD GANs.
result New loss function stabilizes and accelerates GAN training.
The paper connects dynamical systems to ADMM for solving nonsmooth constrained problems.
problem Solving nonsmooth and constrained optimization problems.
method Developed differential inclusions for accelerated ADMM variants and analyzed their convergence rates.
result Derived rates of convergence for the dynamical systems under different settings, showing tradeoffs between damping strategies.