arXiv research

Power iteration has been generalized to solve many interesting problems in machine learning and statistics. Despite its striking success, theoretical understanding of when and how such an algorithm enjoys good convergence property is limited. In this work, we introduce a new class of optimization problems called scale …

Critical points of scale-invariant curvature energies in 4D are analytic.

SAM improves deep learning tasks by promoting balancedness, reducing outlier impact.

New bounds on self-normalized martingales improve online linear regression performance.

The paper studies minimal resistance dynamics in radial fields, finding unique solutions for incompressible flows.

Scales attention for long contexts in LLMs.

Three training regimes found for scale-invariant neural networks on the sphere.

New approach improves classification guarantees by focusing on direction rather than regression risk.

In theoretical analysis of deep learning, discovering which features of deep learning lead to good performance is an important task. In this paper, using the framework for analyzing the generalization error developed in Suzuki (2018), we derive a fast learning rate for deep neural networks with more general activation …

Proves uniqueness of Ricci flow with scaling invariant estimates.

We consider a variant of online convex optimization in which both the instances (input vectors) and the comparator (weight vector) are unconstrained. We exploit a natural scale invariance symmetry in our unconstrained setting: the predictions of the optimal comparator are invariant under any linear transformation of th…

AdamP optimizes momentum-based optimizers for scale-invariant weights, improving model performance.

Proves conditions for Willmore surfaces to have finite ends or finite total curvature.

In seeking for sparse and efficient neural network models, many previous works investigated on enforcing L1 or L0 regularizers to encourage weight sparsity during training. The L0 regularizer measures the parameter sparsity directly and is invariant to the scaling of parameter values, but it cannot provide useful gradi…

A new approach to group fairness treats it as a bargaining problem.

Investigates energy minimizers and critical points of scale-invariant tangent-point energies for knots.

We study scale invariant but not necessarily conformal invariant deformations of non-relativistic conformal field theories from the dual gravity viewpoint. We present the corresponding metric that solves the Einstein equation coupled with a massive vector field. We find that, within the class of metric we study, when w…

Study shows how near crushing singularities, Kasner-like regions can exist.

New learning dynamics achieve fast convergence in games without needing to know utility scales.

Adam performs better with equal momentum parameters, revealing a gradient scale invariance principle.

This article investigates stationary surfaces with boundaries, which arise as the critical points of functionals dependent on curvature. Precisely, a generalized "bending energy" functional $\mathcal{W}$ is considered which involves a Lagrangian that is symmetric in the principal curvatures. The first variation of $\ma…

It is well known that neural networks with rectified linear units (ReLU) activation functions are positively scale-invariant. Conventional algorithms like stochastic gradient descent optimize the neural networks in the vector space of weights, which is, however, not positively scale-invariant. This mismatch may lead to…

Generalized algorithm for translation and scale-invariant prediction.

Neural networks can approximate positive homogeneous functions, especially with multiple hidden layers.

We develop a scale-invariant truncated Lévy (STL) process to describe physical systems characterized by correlated stochastic variables. The STL process exhibits Lévy stability for the probability density, and hence shows scaling properties (as observed in empirical data); it has the advantage that all moments are fini…

ASAM improves deep neural network generalization by adapting sharpness to scale.

Unified framework for scale-invariant representation learning using MAPCA.

New method trains deep networks robustly without adaptive methods.

Proves long-time Ricci flow existence and topological rigidity for pinched integral curvature manifolds.

We develop a regularity theory for extremal knots of scale invariant knot energies defined by J. O'hara in 1991. This class contains as a special case the Möbius energy. For the Möbius energy, due to the celebrated work of Freedman, He, and Wang, we have a relatively good understanding. Their approch is crucially based…

A new algorithm solves nonnegative least squares faster with nonnegative data.

Wavelet scattering spectra model non-Gaussian time-series, proving scale invariance for self-similar processes.

A challenging problem in physics concerns the possibility of forecasting rare but extreme phenomena such as large earthquakes, financial market crashes, and material rupture. A promising line of research involves the early detection of precursory log-periodic oscillations to help forecast extreme events in collective p…

Proves planarity and convexity for ancient solutions of mean curvature flow.

NOTEARS fails to identify true causal relationships from data.

The results of R^2 dynamical random surface model (2-dimensional quantum gravity with a $R^2$ term) are applied to explain the personal income distribution. A scale invariance exists if there is not the $R^2$ term in the action. The R^2 term provides a typical scale and breaks the scale invariance explicitly in the low…

We explore the impact of learning paradigms on training deep neural networks for the Travelling Salesman Problem. We design controlled experiments to train supervised learning (SL) and reinforcement learning (RL) models on fixed graph sizes up to 100 nodes, and evaluate them on variable sized graphs up to 500 nodes. Be…

We propose a new high dimensional semiparametric principal component analysis (PCA) method, named Copula Component Analysis (COCA). The semiparametric model assumes that, after unspecified marginally monotone transformations, the distributions are multivariate Gaussian. COCA improves upon PCA and sparse PCA in three as…

The statistical properties of the multipliers of the absolute returns are investigated using one-minute high-frequency data of financial time series. The multiplier distribution is found to be independent of the box size $s$ when $s$ is larger than some crossover scale, providing direct evidence of the existence of sca…

We develop an entropic framework to model the dynamics of stocks and European Options. Entropic inference is an inductive inference framework equipped with proper tools to handle situations where incomplete information is available. The objective of the paper is to lay down an alternative framework for modeling dynamic…

Reduces symplectic Hamiltonian systems to contact systems, realizing Poincaré's dream.

SAM improves generalization in overparameterized models, but its behavior in tensorized models is less understood.

Deep learning models are vulnerable to adversarial examples crafted by applying human-imperceptible perturbations on benign inputs. However, under the black-box setting, most existing adversaries often have a poor transferability to attack other defense models. In this work, from the perspective of regarding the advers…

Study Ricci-Deturck flow from rough metrics, proving short-time existence.

Scale invariance, collective behaviours and structural reorganization are crucial for portfolio management (portfolio composition, hedging, alternative definition of risk, etc.). This lack of any characteristic scale and such elaborated behaviours find their origin in the theory of complex systems. There are several me…

We consider inverse curvature flows in hyperbolic space with starshaped initial hypersurface, driven by positive powers of a homogeneous curvature function. The solutions exist for all time and, after rescaling, converge to a sphere.

A new method treats all variables equally in fitting data.

Using the monotonicity formulas of Colding and Minicozzi, we prove that on any complete, non-parabolic Riemannian manifold $(M^3, g)$ with non-negative Ricci curvature, the asymptotic weighted scaling invariant integral of scalar curvature has an explicit bound in form of asymptotic volume ratio.