Improved stability for large-scale Bayesian sampling.
problem Reducing instability in Langevin dynamics for large datasets.
method Introducing a modified CCAdL thermostat with a scaling and squaring method and a truncated Taylor series approximation.
result Significantly improved numerical stability and accuracy over existing methods.
By using numerical simulation, we confirm that Takayasu--Sato--Takayasu (TST) model which leads Pareto's law satisfies the detailed balance under Gibrat's law. In the simulation, we take an exponential tent-shaped function as the growth rate distribution. We also numerically confirm the reflection law equivalent to the…
The paper studies scaling laws for associative memory mechanisms.
problem Understanding and optimizing learning and memorization processes.
method High-dimensional matrices of outer products of embeddings, relating to transformer models. Derived scaling laws with sample and parameter sizes. Extensive numerical experiments.
result Precise scaling laws and statistical efficiency of estimators.
This paper tackles ranking-based performance normalization for optimization algorithms.
problem Ranking optimization algorithms across diverse numerical scales disrupts performance comparisons.
method Introduces absolute ranking and a sampling-based computational method to address numerical scale variation.
result Provides a more robust framework for assessing performance across multiple algorithms and problems.
Novel neural network solves PDEs with multi-scale resolution.
problem Solving time-dependent PDEs with varying spatial and temporal scales.
method Multi-scale message passing neural network with temporal and spatial gating modules.
result Outperforms baselines on PDEs with diverse scales.
Adaptive loss scaling speeds up and improves deep learning training.
problem Numerical underflow in mixed precision training.
method Adaptive loss scaling that automatically computes layer-wise loss scale values during training.
result Adaptive loss scaling leads to shorter convergence time and improved accuracy.
This work studies scaling laws for low-precision training in high-dimensional linear regression.
problem Optimizing trade-off between model quality and training costs in high-dimensional linear regression.
method Theoretical study of scaling laws for low-precision training within a high-dimensional sketched linear regression framework, analyzing multiplicative and additive quantization.
result Multiplicative quantization maintains full-precision model size, while additive quantization reduces effective model size.
Deep learning upscales geologic models efficiently.
problem Upscaling large-scale geologic models for efficient simulation.
method Theory-guided convolutional neural network (TgCNN) trained to approximate hydraulic conductivity relationships.
result Deep learning method achieves equivalent upscaling accuracy to numerical methods but with significantly improved efficiency.
Extracts coarse-grained PDEs from microscopic simulations.
problem Discovering effective PDEs for macro-scale processes from microscopic data.
method Combining neural networks with equation-free numerics and data-driven approaches.
result Efficiently discovers macro-scale PDEs from microscopic simulations.
Paper proposes MWDE for estimating finite location-scale mixtures.
problem Estimating finite location-scale mixtures using MLE is problematic.
method Investigates minimum Wasserstein distance estimators (MWDE).
result MWDE is consistent and provides a numerical solution.
Efficient kernel methods for large datasets using GPU acceleration.
problem Handling large-scale nonparametric learning problems efficiently.
method Preconditioned gradient solver, GPU acceleration, parallelization, out-of-core linear algebra, numerical precision optimization.
result Dramatic speedups on datasets with billions of points, maintaining state-of-the-art performance.
Proposes MscaleDNN for solving high-dimensional PDEs efficiently.
problem Solving high-dimensional PDEs efficiently.
method Radial scaling in frequency domain and compact support activation functions.
result Increased power in multi-scale resolution and high frequency capturing.
Boosting is a learning scheme that combines weak prediction rules to produce a strong composite estimator, with the underlying intuition that one can obtain accurate prediction rules by combining "rough" ones. Although boosting is proved to be consistent and overfitting-resistant, its numerical convergence rate is rela…
The effectiveness of Convolutional Neural Networks (CNNs) has been substantially attributed to their built-in property of translation equivariance. However, CNNs do not have embedded mechanisms to handle other types of transformations. In this work, we pay attention to scale changes, which regularly appear in various t…
Spike sorting is a fundamental preprocessing step in neuroscience that is central to access simultaneous but distinct neuronal activities and therefore to better understand the animal or even human brain. But numerical complexity limits studies that require processing large scale datasets in terms of number of electrod…
Celeste is a procedure for inferring astronomical catalogs that attains state-of-the-art scientific results. To date, Celeste has been scaled to at most hundreds of megabytes of astronomical images: Bayesian posterior inference is notoriously demanding computationally. In this paper, we report on a scalable, parallel v…
Classical multidimensional scaling is an important dimension reduction technique. Yet few theoretical results characterizing its statistical performance exist. This paper provides a theoretical framework for analyzing the quality of embedded samples produced by classical multidimensional scaling. This lays the foundati…
We provide a numerically robust and fast method capable of exploiting the local geometry when solving large-scale stochastic optimisation problems. Our key innovation is an auxiliary variable construction coupled with an inverse Hessian approximation computed using a receding history of iterates and gradients. It is th…
New method scales features for better clustering.
problem Irregular features disrupt classification.
method Spectral clustering with modified feature scales.
result Outperforms existing methods in experiments.
Generative models improve for multiscale scientific data with new noise and interpolation techniques.
problem Numerical challenges in generating high-fidelity samples for multiscale scientific data.
method Design of noise distributions and interpolation schedules in function space to ensure Lipschitz regularity and finite noise roughness.
result Scale-adaptive noise and interpolation schedules improve numerical efficiency and fidelity of generated samples.
A new method solves large-scale sparse group square-root Lasso problems efficiently.
problem Large-scale linearly constrained sparse group square-root Lasso problems.
method Dual semismooth Newton based augmented Lagrangian method (ALM).
result The proposed method efficiently solves the problem with numerical experiments demonstrating its effectiveness.
Deep neural networks provide meaningful uncertainty estimates for large-scale simulations.
problem Uncertainty estimates for deep neural network predictions from large-scale simulations.
method General variational inference approach to calibrate Bayesian uncertainties.
result Calibrated Bayesian uncertainties preserved physics-correlations in predicted quantities.
In this paper, we perform registration of noisy curves. We provide an appropriate model in estimating the rotation and scaling parameters to adjust a set of curves through a M-estimation procedure. We prove the consistency and the asymptotic normality of our estimators. Numerical simulation and a real life aeronautic e…
Grokking occurs at numerical stability edge, requiring regularization to prevent.
problem Delayed generalization in deep learning models.
method Identified Softmax Collapse (SC) as the cause of grokking without regularization.
result Mitigating SC enables grokking without regularization.
Proposes a more robust rating scale for banks.
problem Inconsistent rating scale validation leading to higher capital requirements.
method Develops a new rating scale that is statistically distinguishable and robust.
result Reduces the calibration probability of default, saving capital requirements.
Develops active learning for scale-bridging simulations.
problem Quantitative predictions in nanoporous media and inertial confinement fusion.
method Active learning approach to optimize fine-scale simulations for coarse-scale hydrodynamics.
result Optimizes use of fine-scale simulations for coarse-scale predictions.
Three methods for tuning HMC diagonal scale matrices compared.
problem Improving Hamiltonian Monte Carlo efficiency with diagonal scale matrices.
method Three approaches: ISG, median crossing frequency, and estimated marginal standard deviations.
result ISG method leads to more efficient sampling in many cases.
Models predict order book recovery from aggressive trading follows a simple t^1/3 scaling.
problem Understanding order book dynamics in financial markets.
method Developed macroscopic models and solved numerically and asymptotically.
result Order book recovery follows a t^1/3 scaling law.
Scalable solver reduces PDE uncertainty with active learning.
problem High computational cost in solving PDEs.
method Stochastic dual descent and clustering-based active learning.
result Solver scales to large number of collocation points.
Dynamic Vocabulary Pruning stabilizes LLM training by removing low-probability tokens.
problem Training Large Language Models (LLMs) with Reinforcement Learning (RL) causes numerical divergence between inference and training.
method Dynamic Vocabulary Pruning (DVP) constrains the RL objective to a safe vocabulary that excludes low-probability tokens.
result DVP stabilizes training by reducing systematic bias introduced by the extreme tail of the token distribution.
This work explores variably scaled kernels to improve non-stationary Gaussian processes.
problem Limited ability of stationary kernels to represent heterogeneous correlation structures.
method Introduces variably scaled kernels to modify correlation structures explicitly.
result Improved reconstruction accuracy and better uncertainty estimates for non-stationary data.
Machine learning enhances ocean studies by analyzing sparse, multi-scale data.
problem Sparse, multi-scale ocean data limits traditional methods.
method Machine learning techniques applied to ocean observations, theory, and numerical simulations.
result ML can advance theoretical oceanographic exploration and numerical simulations.
New phases identified in neural scaling laws with compute limits.
problem Understanding neural scaling laws under compute constraints.
method Solved neural scaling model with stochastic gradient descent, derived loss curves, analyzed model-parameter-count phases.
result Identified 4 phases (+3 subphases) in data-complexity/target-complexity phase-plane, derived exponents.
Paper solves robust multi-dimensional scaling with accelerated projections.
problem Localize point locations from noisy pairwise distances.
method Alternating projections with tangent space acceleration.
result Linear convergence of reconstructed points to original points.
Multidimensional scaling is an important dimension reduction tool in statistics and machine learning. Yet few theoretical results characterizing its statistical performance exist, not to mention any in high dimensions. By considering a unified framework that includes low, moderate and high dimensions, we study multidim…
The optimal dividend problem by De Finetti (1957) has been recently generalized to the spectrally negative Lévy model where the implementation of optimal strategies draws upon the computation of scale functions and their derivatives. This paper proposes a phase-type fitting approximation of the optimal strategy. We con…
Developing stable and scalable probabilistic ODE solvers for stiff and high-dimensional problems.
problem Stiff and high-dimensional ODEs
method Matrix-free update step and iterative re-linearization
result Improved stability and scalability
We propose a novel framework to investigate lead-lag relationships between two financial assets. Our framework bridges a gap between continuous-time modeling based on Brownian motion and the existing wavelet methods for lead-lag analysis based on discrete-time models and enables us to analyze the multi-scale structure …
SALSA efficiently approximates leverage scores for big data, improving ARMA model fitting.
problem Efficiently approximating leverage scores for large matrices.
method Sequential approximate leverage-score algorithm (SALSA) using randomized numerical linear algebra.
result SALSA approximates leverage scores within (1+O(ε)) with high probability. Gaussian equivalence fails for simple polynomial embeddings in quadratic scaling RF models.
problem Failure of Gaussian equivalence in polynomial feature embeddings under quadratic scaling.
method Introduced Conditional Gaussian Equivalent (CGE) model to capture non-Gaussian behavior.
result Correct asymptotics derived for training and test errors in CGE model.
Encoding the scale information explicitly into the representation learned by a convolutional neural network (CNN) is beneficial for many computer vision tasks especially when dealing with multiscale inputs. We study, in this paper, a scaling-translation-equivariant (ST-equivariant) CNN with joint convolutions across th…
In this paper, we propose and study a Nyström based approach to efficient large scale kernel principal component analysis (PCA). The latter is a natural nonlinear extension of classical PCA based on considering a nonlinear feature map or the corresponding kernel. Like other kernel approaches, kernel PCA enjoys good mat…
The miltifractal properties and scaling behaviour of the exchange rate variations of the Iranian rial against the US dollar from a daily perspective is numerically investigated. For this purpose the multifractal detrended fluctuation analysis (MF-DFA) is used. Through multifractal analysis, the scaling exponents, gener…
Scaling feature values is an important step in numerous machine learning tasks. Different features can have different value ranges and some form of a feature scaling is often required in order to learn an accurate classifier. However, feature scaling is conducted as a preprocessing task prior to learning. This is probl…
Based on the new type of random walk process called the Potentials of Unbalanced Complex Kinetics (PUCK) model, we theoretically show that the price diffusion in large scales is amplified 2/(2 + b) times, where b is the coefficient of quadratic term of the potential. In short time scales the price diffusion depends on …
Stochastic Volatility in Mean models with heavy-tailed distributions using Hidden Markov Models
problem Accurate inference for Stochastic Volatility in Mean models with heavy-tailed distributions
method Numerically stable estimation procedure and parallel computing
result Significant reduction in computational times
Unified quadrature framework for large-scale kernel machines.
problem Efficiently approximating kernel functions for large-scale machine learning.
method Deterministic and randomized interpolatory rules for numerical integration of kernel functions.
result The proposed method reduces the number of nodes needed for accurate kernel approximation.
Develops statistical confidence sets for multidimensional scaling.
problem Statistical uncertainty in multidimensional scaling of noisy data.
method Formal statistical framework, distributional convergence results, uniform confidence sets, bootstrap procedures.
result Construction of reliable confidence sets for latent configurations in multidimensional scaling.