SCI-PI solves scale invariant problems efficiently.
problem Solving scale invariant problems in optimization.
method Introduces SCI-PI and proves its convergence.
result SCI-PI achieves local linear convergence.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,051 papers · 148 categories
Trend · papers per month
48 results for “scale invariant problems”
Critical points of scale-invariant curvature energies in 4D are analytic.
problem Analyzing critical points of curvature energies in 4D manifolds.
method Applying Noether's theorem to identify conservation laws and lower order elliptic system of PDEs, then using integrability by compensation and interpolation theory.
result Critical points of scale-invariant curvature energies in 4D are analytic.
New tool improves scalability of data-driven invariant inference.
problem Scaling data-driven invariant inference to programs with many variables.
method Developed oasis tool to improve scalability.
result Outperforms state-of-the-art systems on benchmarks.
SAM improves deep learning tasks by promoting balancedness, reducing outlier impact.
problem Improving generalization in deep learning tasks, especially with scale-invariant problems.
method Introduces balancedness as a new concept to depict global behaviors of SAM, focusing on the difference between squared norms of two variables.
result SAM promotes balancedness and is data-responsive, outperforming SGD in outlier scenarios.
DeepHoyer: Learning Sparser Neural Network with Differentiable Scale-Invariant Sparsity Measurescs.LG
DeepHoyer introduces differentiable, scale-invariant sparsity measures for neural networks.
problem Efficiently sparsifying neural networks with scale-invariant sparsity measures.
method Developed DeepHoyer, a set of differentiable, scale-invariant sparsity-inducing regularizers based on the Hoyer measure.
result DeepHoyer produces sparser neural networks than previous methods, maintaining similar accuracy.
Formula derived for a magnetic line invariant.
problem Solving MHD problems with two-scale mean fields.
method Combinatorial definition of invariant for three-component links.
result Proven formula for invariant verified for simple cases.
New bounds on self-normalized martingales improve online linear regression performance.
problem Improving regret bounds in online linear regression.
method Characterizing scale-invariant bounds on self-normalized martingales.
result For , doubly-uniform regret is possible; for , sublinear doubly-uniform regret is impossible.
New samplers improve MCMC efficiency in high dimensions.
problem Efficient sampling in high-dimensional problems.
method Affine invariant ensemble samplers, including derivative-free and derivative-based HMC.
result Affine invariant ensemble HMC outperforms standard HMC in high dimensions.
The radial Newton problem: nonlinear dynamics of minimal resistance in central fieldsphysics.flu-dyn
The paper studies minimal resistance dynamics in radial fields, finding unique solutions for incompressible flows.
problem Nonlinear dynamics of minimal resistance in radial fields.
method Analysis of two non-equilibrium scenarios: scale-invariant free expansion and incompressible source flow.
result Incompressible flow acts as a structural regularizer, admitting unique, smooth, and strictly concave solutions.
Generalized Translation and Scale Invariant Online Algorithm for Adversarial Multi-Armed Banditscs.LG
New algorithm handles bandit problems under translations and scales.
problem Adversarial multi-armed bandit problems with arbitrary translations and scales.
method Innovative online algorithm invariant to translations and scales, using universal prediction.
result Second-order regret bounds, unaffected by affine transformations of losses.
It is well known that neural networks with rectified linear units (ReLU) activation functions are positively scale-invariant. Conventional algorithms like stochastic gradient descent optimize the neural networks in the vector space of weights, which is, however, not positively scale-invariant. This mismatch may lead to…
A new approach to group fairness treats it as a bargaining problem.
problem Fairness in deploying predictors across subpopulations.
method Interpreting fairness as a bargaining problem and proposing relative improvement.
result Relative improvement provides axiomatic justification and finite-sample convergence guarantees.
A Generalized Online Algorithm for Translation and Scale Invariant Prediction with Expert Advicecs.LG
Generalized algorithm for translation and scale-invariant prediction.
problem Sequential prediction with expert advice, focusing on translation and scale invariance.
method Designing a generalized online algorithm using the universal prediction perspective to compete against a generic class of expert selection strategies.
result No preliminary knowledge of loss sequences is required; performance bounds are stable under arbitrary scalings and translations.
New algorithms predict without tuning for varying feature scales.
problem Learning linear models online with unknown feature scales.
method Adaptive scale-invariant online algorithms without tuning parameters.
result Achieves regret bounds matching OGD with optimal tuning, comparable runtime.
New approach improves classification guarantees by focusing on direction rather than regression risk.
problem Improving classification guarantees in binary classification problems.
method Establishing a geometric distinction between classification and regression, leveraging scale invariance.
result Improved guarantees for classification risk compared to regression risk.
Characterizes conformal Killing tensors and their Killing scales.
problem Characterizing conformal Killing tensors and their Killing scales.
method Differential prolongation using conformally invariant tractor calculus.
result Provides an invariant characterisation of Einstein Killing scales.
We study scale invariant but not necessarily conformal invariant deformations of non-relativistic conformal field theories from the dual gravity viewpoint. We present the corresponding metric that solves the Einstein equation coupled with a massive vector field. We find that, within the class of metric we study, when w…
Introduces bounded scale measure and generalizes property A.
problem Defining property A for large scale spaces with bounded geometry.
method Introduces bounded scale measure, shows its coarse invariance, and generalizes property A.
result Definition of property A for large scale spaces with bounded scale measure is a coarse invariant.
Study scaling limits of utility indifference prices in discretized Bachelier model.
problem Analyzing utility indifference prices for path-dependent European options in a discretized Bachelier model.
method Purely probabilistic approach, including duality argument, optimal drift control problem, martingale techniques, and strong invariance principles.
result Obtained a scaling limit for utility indifference prices as the number of trading times increases.
Scales attention for long contexts in LLMs.
problem Development of attention mechanisms for long context inference.
method Scale-invariant total attention and sparsity conditions, with a position-dependent transformation of logits.
result Scale-invariant attention scheme improves validation loss and long-context retrieval.
Fast generalization error bound of deep learning without scale invariance of activation functionsstat.ML
Paper derives a fast learning rate for deep neural networks without scale invariant activation functions.
problem Analyzing the impact of non-scale invariant activation functions on deep learning performance.
method Using Suzuki (2018) framework, derived a tight generalization error bound for deep neural networks with non-scale invariant activations.
result Without scale invariance of activation functions, deep learning can still achieve a fast learning rate.
When solving data analysis problems it is important to integrate prior knowledge and/or structural invariances. This paper contributes by a novel framework for incorporating algebraic invariance structure into kernels. In particular, we show that algebraic properties such as sign symmetries in data, phase independence,…
We study higher form Proca equations on Einstein manifolds with boundary data along conformal infinity. We solve these Laplace-type boundary problems formally, and to all orders, by constructing an operator which projects arbitrary forms to solutions. We also develop a product formula for solving these asymptotic probl…
A new algorithm solves nonnegative least squares faster with nonnegative data.
problem Nonnegative least squares problems with nonnegative data.
method Primal-dual perspective accelerated algorithm with adaptive restart.
result Oracle complexity independent of matrix constants, solvable to multiplicative error.
We consider a variant of online convex optimization in which both the instances (input vectors) and the comparator (weight vector) are unconstrained. We exploit a natural scale invariance symmetry in our unconstrained setting: the predictions of the optimal comparator are invariant under any linear transformation of th…
New learning dynamics achieve fast convergence in games without needing to know utility scales.
problem Fast convergence guarantees in learning games require prior knowledge of utility scales.
method Developed scale-free and scale-invariant learning dynamics using optimistic follow-the-regularized-leader with adaptive learning rates and clipping techniques.
result Achieved fast convergence rates to Nash and correlated equilibria without prior utility scale knowledge.
Three training regimes found for scale-invariant neural networks on the sphere.
problem Training scale-invariant neural networks on the sphere with varying effective learning rate.
method Investigated three regimes of training: convergence, chaotic equilibrium, and divergence.
result Discovered three distinct training regimes with unique characteristics.
We introduce a new weight-decay scaling rule to maintain sublayer gains across different widths in modern scale-invariant architectures.
problem In modern scale-invariant architectures, training quickly enters a steady state where normalization layers create backward scale sensitivity, degrading learning-rate transfer.
method We introduce a weight-decay scaling rule for AdamW that preserves sublayer gain across widths by equalizing the effective learning rate.
result Our empirical weight-decay scaling rule approximately keeps sublayer gains width invariant, enabling zero-shot transfer of learning rate and weight decay.
Proves uniqueness of Ricci flow with scaling invariant estimates.
problem Proving uniqueness of Ricci flow with scaling invariant curvature bound.
method Solving Ricci-harmonic map heat flow in unbounded curvature background.
result Complete Ricci flow starting from uniformly non-collapsed, non-negatively curved manifold is unique in dimension three.
Improved SVMs learn from few samples with composition and multiple scales.
problem Learning with small sample sizes.
method Transformation-invariant SVMs with composition and locality at multiple scales.
result Kernels based on maximum similarity are positive definite and yield superior accuracy.
Adam performs better with equal momentum parameters, revealing a gradient scale invariance principle.
problem Why Adam performs better with .
method Formalized gradient scale invariance and proved it for Adam with equal and .
result Adam becomes gradient scale invariant of first order if and only if .
Scale-invariant Truncated Lévy Processcond-mat.stat-mech
We develop a scale-invariant truncated Lévy (STL) process to describe physical systems characterized by correlated stochastic variables. The STL process exhibits Lévy stability for the probability density, and hence shows scaling properties (as observed in empirical data); it has the advantage that all moments are fini…
The study examines stationary surfaces with boundaries and their properties.
problem Investigating stationary surfaces with boundaries and their critical points.
method A generalized bending energy functional is considered, and the first variation is computed. Boundary-value problems are examined, and a characterization of free-boundary surfaces is given.
result Characterization of free-boundary surfaces with rotational symmetry for scaling-invariant functionals.
This paper argues that a class of Riemannian metrics, called warped metrics, plays a fundamental role in statistical problems involving location-scale models. The paper reports three new results : i) the Rao-Fisher metric of any location-scale model is a warped metric, provided that this model satisfies a natural invar…
AdamP optimizes momentum-based optimizers for scale-invariant weights, improving model performance.
problem Premature decay of effective step sizes in momentum-based optimizers for scale-invariant weights.
method Proposes SGDP and AdamP to eliminate the radial component at each optimizer step, preserving convergence properties.
result Uniform gains across multiple benchmarks, improving model performance.
Proves conditions for Willmore surfaces to have finite ends or finite total curvature.
problem Conditions for Willmore surfaces to have finite ends or finite total curvature.
method Analyzes scale-invariant second fundamental form near infinity.
result Proves conditions for Willmore surfaces to have finite ends or finite total curvature.
Neural networks can approximate positive homogeneous functions, especially with multiple hidden layers.
problem Approximating positive homogeneous functions with neural networks.
method Using scale-invariant ReLU networks with multiple hidden layers.
result Approximation of positive homogeneous functions is possible with neural networks, especially with two hidden layers.
The paper classifies left-invariant pseudo-Riemannian metrics on specific Lie groups.
problem Classifying left-invariant pseudo-Riemannian metrics on Lie groups.
method Analyzing left-invariant metrics on specific Lie groups with .
result A complete classification of left-invariant pseudo-Riemannian metrics for Lie groups of dimension .
New algorithm uses GNNs to optimize rewards in graph-structured data.
problem Optimizing rewards in molecule design with graph-structured data.
method Embedding permutation invariance into GNNs and using GNTK for regret bounds.
result First GNN confidence bound and phased-elimination algorithm with sublinear regret.
Wavelet scattering spectra model non-Gaussian time-series, proving scale invariance for self-similar processes.
problem Modeling non-Gaussian time-series with stationary increments.
method Complex wavelet transform for scale variations, joint correlation matrix for scale dependencies, second wavelet transform for diagonalization, maximum entropy models conditioned by scattering spectra coefficients.
result Scattering spectra of self-similar processes are scale invariant, allowing statistical testing and generation of new time-series.
ASAM: Adaptive Sharpness-Aware Minimization for Scale-Invariant Learning of Deep Neural Networkscs.LG
ASAM improves deep neural network generalization by adapting sharpness to scale.
problem Fixed-radius sharpness measure is sensitive to parameter scaling, weakening its connection to generalization.
method Introduces adaptive sharpness, a scale-invariant measure, and proposes ASAM for deep learning.
result ASAM significantly improves model generalization performance across various datasets.
Investigates energy minimizers and critical points of scale-invariant tangent-point energies for knots.
problem Finding and characterizing minimizers and critical points of scale-invariant tangent-point energies for closed curves.
method Develops convergence and regularity theories based on fractional Sobolev spaces and new energy functionals.
result Minimizing sequences converge to locally critical embeddings in all but finitely many points, and locally critical embeddings are regular.
Paper introduces new Gromov-type distances for comparing Gaussian mixture models.
problem Comparing distributions across different metric spaces using Gromov-Wasserstein distances.
method Incorporates invariance properties into MW2, introducing MGW2 and EW2.
result MGW2 and EW2 are efficient for estimating distances between GMMs in practical applications.
The paper analyzes geometric densities and compression radii for knot types.
problem Optimizing geometric quantities associated with knot types.
method Develops a factorization framework for scale-covariant size functionals.
result Different minimizing sequences for density, compression, packing, and ropelength problems.
The paper develops a regularity theory for O'hara knot energies, focusing on Möbius energy.
problem Developing a regularity theory for extremal knots of scale invariant knot energies defined by J. O'hara.
method Reinterpreting O'hara knot energies as a nonlinear, nonlocal -energy acting on the unit tangent of the knot parametrization, drawing a connection to the theory of (fractional) harmonic maps into spheres.
result Proves regularity for minimizers and critical knots of the scale-invariant O'hara knot energies.
Study shows how near crushing singularities, Kasner-like regions can exist.
problem Understanding spatial volume densities near crushing singularities.
method Relates existence of Kasner-like regions to asymptotics of spatial volume densities under scale-invariant curvature bounds.
result Kasner-like regions can exist near crushing singularities under certain curvature conditions.
Transforms improve CNNs' invariance to image transformations.
problem Current CNN models lack robustness to spatial transformations.
method Randomly transform feature maps during training to learn invariant representations.
result Significant improvements on benchmark tasks, including image recognition and retrieval.
A new sliced IGW distance for Gromov-Wasserstein alignment.
problem Scalability issues in Gromov-Wasserstein alignment for high-dimensional problems.
method Proposed a sliced IGW distance with rotational invariance.
result Natural rotational invariance of the sliced IGW distance.