arXiv research

Unified framework for nonconvex matrix completion with linearly parameterized factors.

This paper improves reinforcement learning efficiency for large-scale MDPs.

Policy gradient converges linearly with Hadamard parameterization in tabular settings.

We show that there is a family of pseudo-Anosov braids independently parameterized by the braid index and the (canonical) length whose smallest conjugacy invariant sets grow exponentially in the braid index and linearly in the length and conclude that the conjugacy problem remains exponential in the braid index under t…

The paper axiomatizes strong emergence in parameterized field theories and proves existence theorems.

New methods solve tensor-on-tensor regression with unknown rank, revealing benefits of over-parameterization.

Gradient descent converges linearly for neural networks with specific conditions.

PrecGD restores linear convergence in over-parameterized nonconvex matrix factorization.

We study the generalization properties of stochastic gradient methods for learning with convex loss functions and linearly parameterized functions. We show that, in the absence of penalizations or constraints, the stability and approximation properties of the algorithm can be controlled by tuning either the step-size o…

Global results are proved about the way in which Boyland's forcing partial order organizes a set of braid types: those of periodic orbits of Smale's horseshoe map for which the associated train track is a star. This is a special case of a conjecture introduced in a previous paper, which claims that forcing organizes al…

FedAvg converges linearly to global minimum in federated learning with partial participation.

Gradient descent with small initialization solves matrix completion without regularization.

Motivated by models of human decision making proposed to explain commonly observed deviations from conventional expected value preferences, we formulate two stochastic multi-armed bandit problems with distorted probabilities on the reward distributions: the classic $K$-armed bandit and the linearly parameterized bandit…

We propose randomized least-squares value iteration (RLSVI) -- a new reinforcement learning algorithm designed to explore and generalize efficiently via linearly parameterized value functions. We explain why versions of least-squares value iteration that use Boltzmann or epsilon-greedy exploration can be highly ineffic…

Geometrically classifies maps from R^0|2 to any manifold, unifying theories.

Entropy-regularized NPG methods converge linearly in discounted MDPs.

Improves conditions for mode connectivity in deep neural networks.

We seek to improve the data efficiency of neural networks and present novel implementations of parameterized piece-wise polynomial activation functions. The parameters are the y-coordinates of n+1 Chebyshev nodes per hidden unit and Lagrangian interpolation between the nodes produces the polynomial on [-1, 1]. We show …

The optimization of multilayer neural networks typically leads to a solution with zero training error, yet the landscape can exhibit spurious local minima and the minima can be disconnected. In this paper, we shed light on this phenomenon: we show that the combination of stochastic gradient descent (SGD) and over-param…

New method calibrates neural network predictions for better reliability.

Paper explores how unsupervised learning can be understood through linear algebra concepts.

A recent breakthrough in deep learning theory shows that the training of over-parameterized deep neural networks can be characterized by a kernel function called \textit{neural tangent kernel} (NTK). However, it is known that this type of results does not perfectly match the practice, as NTK-based analysis requires the…

The paper analyzes the maximum margin algorithm's performance on noisy data.

Study shows how diffusion models learn on low-dimensional manifolds.

In this paper, we theoretically prove that gradient descent can find a global minimum of non-convex optimization of all layers for nonlinear deep neural networks of sizes commonly encountered in practice. The theory developed in this paper only requires the practical degrees of over-parameterization unlike previous the…

We reveal a model rank that predicts successful recovery of target functions at overparameterization.

Study on singular points of translation surfaces under linearly dependent conditions.

For a variety of regularized optimization problems in machine learning, algorithms computing the entire solution path have been developed recently. Most of these methods are quadratic programs that are parameterized by a single parameter, as for example the Support Vector Machine (SVM). Solution path algorithms do not …

New invariants from divisibility of Lee classes for slice-torus.

Study on symmetries in wide neural networks' dynamics without bias.

Paper studies how few pretraining tasks are needed for a linear model to solve new tasks.

Entropy-regularized NPG converges linearly with linear function approximation.

We study the linear contextual bandit problem with finite action sets. When the problem dimension is $d$, the time horizon is $T$, and there are $n \leq 2^{d/2}$ candidate actions per time period, we (1) show that the minimax expected regret is $Ω(\sqrt{dT (\log T) (\log n)})$ for every algorithm, and (2) introduce a V…

Small initialization improves tensor recovery from noisy data.

We consider stochastic second-order methods for minimizing smooth and strongly-convex functions under an interpolation condition satisfied by over-parameterized models. Under this condition, we show that the regularized subsampled Newton method (R-SSN) achieves global linear convergence with an adaptive step-size and a…

State-of-the-art computer codes for simulating real physical systems are often characterized by a vast number of input parameters. Performing uncertainty quantification (UQ) tasks with Monte Carlo (MC) methods is almost always infeasible because of the need to perform hundreds of thousands or even millions of forward m…

The paper introduces various canonical parameterizations for 2D-curved shapes.

There are currently two parameterizations used to derive fixed kernels corresponding to infinite width neural networks, the NTK (Neural Tangent Kernel) parameterization and the naive standard parameterization. However, the extrapolation of both of these parameterizations to infinite width is problematic. The standard p…

Importance-weighted risk minimization is a key ingredient in many machine learning algorithms for causal inference, domain adaptation, class imbalance, and off-policy reinforcement learning. While the effect of importance weighting is well-characterized for low-capacity misspecified models, little is known about how it…

Supervised learning frequently boils down to determining hidden and bright parameters in a parameterized hypothesis space based on finite input-output samples. The hidden parameters determine the attributions of hidden predictors or the nonlinear mechanism of an estimator, while the bright parameters characterize how h…

New parameterization for $(1,1)$-knots simplifies their study.

We present a reduction from reinforcement learning (RL) to no-regret online learning based on the saddle-point formulation of RL, by which "any" online algorithm with sublinear regret can generate policies with provable performance guarantees. This new perspective decouples the RL problem into two parts: regret minimiz…

If a closed 3-manifold M supports a closed, nonsingular, irrational 1-form which linearly deforms into contact forms, then M supports a K-contact form. On the 3-torus, a closed nonsingular 1-form deforms linearly into contact forms if and only if it is a fibration 1-form. on any other 2-torus bundle over the circle, ev…

In this paper, we presented a novel semi-supervised one-class classification algorithm which assumes that class is linearly separable from other elements. We proved theoretically that class is linearly separable if and only if it is maximal by probability within the sets with the same mean. Furthermore, we presented an…

Stochastic parameterizations account for uncertainty in the representation of unresolved sub-grid processes by sampling from the distribution of possible sub-grid forcings. Some existing stochastic parameterizations utilize data-driven approaches to characterize uncertainty, but these approaches require significant str…

Unified approach for federated learning using MM optimization.

The current paper discusses some new results about conformal polynomic surface parameterizations. A new theorem is proved: Given a conformal polynomic surface parameterization of any degree it must be harmonic on each component. As a first geometrical application, every surface that admits a conformal polynomic paramet…

The paper explores linearly free graphs and their embeddings into 3D space.