Canonical correlation analysis (CCA) is a fundamental statistical tool for exploring the correlation structure between two sets of random variables. In this paper, motivated by recent success of applying CCA to learn low dimensional representations of high dimensional objects, we propose to quantify the estimation loss…
A new method learns proper multiclass losses and probabilities.
problem Learning proper multiclass losses for complex classification tasks.
method Extends monotonicity to multiclass problems using convex functions.
result Consistently outperforms natural multiclass baseline on up to 1,000 class datasets.
Paper proposes fitting loss functions to data using source functions from information geometry.
problem Choosing appropriate loss functions for machine learning models.
method Introduces source functions from information geometry to fit loss functions to the domain at hand.
result Significant improvements over state-of-the-art methods in model training.
Unified framework for non-Euclidean CPD under scalable stochastic mirror descent.
problem Handling non-Euclidean losses in tensor decomposition.
method Tensor fiber sampling strategy-based stochastic mirror descent.
result Global convergence to a stationary point under reasonable conditions.
This paper rethinks confidence calibration under covariate shifts.
problem Calibration methods struggle with covariate shifts and unstable importance weighting.
method Derives Expectation consistency condition and proposes Expectation consistency loss (ECL).
result ECL loss is compatible with various types of calibration and has the same sample complexity as ECE.
In this paper, we present some theoretical work to explain why simple gradient descent methods are so successful in solving non-convex optimization problems in learning large-scale neural networks (NN). After introducing a mathematical tool called canonical space, we have proved that the objective functions in learning…
We analyze bias-variance of margin losses.
problem Understanding model overfitting/underfitting.
method Bias-variance decomposition for strictly convex margin losses.
result Expected risk decomposes into central model risk and data variation.
A new GAN model α-GAN with tunable loss function addresses gradient vanishing and mode collapse issues.
problem Addressing vanishing gradients and mode collapse in GANs.
method Introduced a tunable GAN α-GAN using a supervised α-loss function. result Holistic understanding of α-GAN related to Arimoto divergence and convergence properties. Deep nets trained with MSE loss exhibit Neural Collapse, collapsing features and classifiers to class means.
problem Understanding Neural Collapse in MSE-trained deep nets.
method Developed a new MSE loss decomposition and introduced the central path concept.
result Exact dynamics of Neural Collapse along the central path can be predicted.
A new method merges neural networks using CCA to improve model performance.
problem Improving model accuracy through ensembling while reducing computational and storage costs.
method CCA Merge, a new model merging algorithm based on Canonical Correlation Analysis.
result CCA Merge leads to better model performance than past methods, especially in merging more than two models.
Proposes ℓ0-CCA for sparse CCA with improved representation learning.
problem CCA models break with too many variables, and sparsity is beneficial.
method Sparse CCA with stochastic gates and ℓ0-regularization. result Improves representation learning by gating nuisance variables.
Paper tackles tensor decomposition for unaligned observations using RKHS and novel loss functions.
problem Tackles tensor decomposition for unaligned observations.
method Uses functions in RKHS to represent mode with unaligned observations, introduces versatile loss function, proposes optimization algorithm and stochastic gradient method.
result Demonstrates improved tensor decomposition efficiency and effectiveness with synthetic and real data.
Tensor decomposition is a well-known tool for multiway data analysis. This work proposes using stochastic gradients for efficient generalized canonical polyadic (GCP) tensor decomposition of large-scale tensors. GCP tensor decomposition is a recently proposed version of tensor decomposition that allows for a variety of…
The study of a machine learning problem is in many ways is difficult to separate from the study of the loss function being used. One avenue of inquiry has been to look at these loss functions in terms of their properties as scoring rules via the proper-composite representation, in which predictions are mapped to probab…
Compact Kahler-Einstein manifolds converge to semi-log canonical models.
problem Compactness of Kahler-Einstein manifolds of negative scalar curvature.
method Gromov-Hausdorff convergence and Weil-Petersson metric extension.
result Convergence to a finite union of complete Kahler-Einstein metric spaces.
A new method for learning manifolds efficiently using canonical basis functions.
problem Learning manifolds in high-dimensional data with efficient and distinct latent dimensions.
method Proposes a novel optimization objective to enforce a transformation matrix with a few prominent and non-degenerate basis functions.
result Demonstrates that minimizing the off-diagonal manifold metric elements ℓ1-norm results in a more efficient latent space representation. We analyse finite-time singularities of the Teichmüller harmonic map flow -- a natural gradient flow of the harmonic map energy -- and find a canonical way of flowing beyond them in order to construct global solutions in full generality. Moreover, we prove a no-loss-of-topology result at finite time, which completes th…
Develops a new framework to analyze gradient flow regimes and derive explicit solutions.
problem Analyzing scaling regimes and deriving explicit analytic solutions for gradient flow in large learning problems.
method Formal power series expansion of the loss evolution with coefficients encoded by diagrams.
result Reveals different learning phases and obtains explicit solutions in some cases.
Uniform AMMs control loss in prediction markets.
problem Controlling loss in prediction markets.
method Loss-versus-rebalancing (LVR) framework and uniform AMMs.
result Uniform AMMs achieve proportional LVR to pool value.
Uniform AMMs control loss in prediction markets.
problem Controlling loss in prediction markets.
method Loss-versus-rebalancing (LVR) framework and uniform AMMs.
result Uniform AMMs achieve proportional LVR to pool value.
We consider the insurance company as a physical system which is immersed in its environment (the financial market). The insurer company interacts with the market by exchanging the money through the payments for loss claims and receiving the premium. Here in the equilibrium state we obtain the premium by using the canon…
In the canonical framework, we propose an alternative approach for the multifractal analysis based on the detrending moving average method (MF-DMA). We define a canonical measure such that the multifractal mass exponent τ(q) is related to the partition function and the multifractal spectrum f(α) can be directly det…
D-GCCA improves multi-view data analysis by separating common and distinctive components.
problem Analyzing multi-view high-dimensional data with latent factors.
method Decomposes each view's data matrix into common and distinctive sources with orthogonality constraints.
result Consistent estimators with good performance and efficient computation.
A new concept of confidence in learning is defined and analyzed.
problem Understanding and quantifying trust in learning processes.
method Formal axioms, continuum measures, vector fields, loss functions.
result Confidence can be represented and optimized in learning.
Develops SymGCP for tensor decompositions with general symmetry.
problem Handling symmetry in tensor decompositions for better model accuracy.
method Introduces SymGCP, a generalized CP decomposition that accounts for any subset of tensor modes' symmetry.
result SymGCP enables efficient and scalable tensor decomposition with improved model robustness and accuracy.
We establish scale-invariant Strichartz estimates for the Schrödinger flow on any compact Lie group equipped with canonical rational metrics. In particular, full Strichartz estimates without loss for some non-rectangular tori are given. The highlights of this paper include estimates for some Weyl type sums defined on r…
Adaptive market maker curves minimize arbitrage losses in DeFi.
problem Asset trading prices in AMMs trail behind centralized exchanges, causing LP losses.
method Adapts market maker bonding curves to trader behavior using a differential equation derived from the Glosten-Milgrom model.
result Optimal adaptive curves minimize arbitrage losses while remaining competitive.
Many unsupervised kernel methods rely on the estimation of the kernel covariance operator (kernel CO) or kernel cross-covariance operator (kernel CCO). Both kernel CO and kernel CCO are sensitive to contaminated data, even when bounded positive definite kernels are used. To the best of our knowledge, there are few well…
Optimizes cryptocurrency exchanges' risk management by reducing positions based on leverage.
problem Managing risk in cryptocurrency futures exchanges during large price moves.
method Formulates ADL as an optimization problem to minimize risk of loss, using a water-filling rule to equalize leverage.
result The optimal ADL policy minimizes maximum leverage among participants, providing a transparent and implementable benchmark.
We show that the Kahler-Ricci flow on an algebraic manifold of positive Kodaira dimension and semi-ample canonical line bundle converges to a unique canonical metric on its canonical model. It is also shown that there exists a canonical measure of analytic Zariski decomposition on an algebraic manifold of positive Koda…
In this article we construct a canonical Kähler-Einstein current on a LC (log canonical) pairs of log general type as the limit of a sequence of canonical Kähler-Einstein currents on KLT(Kawamata log terminal) pairs of log general type. We call the volume form associated with the canonical Kähler-Einstein current the c…
Develops a new tensor classification method for high-dimensional data.
problem Efficient learning algorithms exploiting tensorial structure in high-dimensional multi-way arrays.
method Tensor Train Multi-way Multi-level Kernel (TT-MMK) combining Canonical Polyadic decomposition, Dual Structure-preserving Support Vector Machine, and Tensor Train approximation.
result The TT-MMK method provides higher prediction accuracy and is more reliable computationally compared to other techniques.
New Hamiltonian Monte Carlo method for non-canonical dynamics.
problem Incompatibility of canonical symplectic structure with non-canonical dynamics.
method Developed a framework for Hamiltonian Monte Carlo using non-canonical symplectic structures with implicit integration.
result Non-canonical Hamiltonian Monte Carlo provides sampling advantages.
Derives metrics for DeFi vaults, addressing credit risk.
problem Credit risk in DeFi lending vaults.
method Three-level decomposition of vault risk; six structural features identified.
result Estimation architecture for credit risk metrics.
The balance property is crucial for insurance pricing, ensuring total actuarial price equals loss. Maximum likelihood GLMs fulfill it, but Lindholm-Wüthrich suggests three methods, with constrained GLM being superior.
problem Ensuring the balance property in insurance pricing models
method Using constrained GLM fitting
result Constrained GLM fitting is superior to the two previously discussed balance correction methods
A new federated learning algorithm improves on existing methods by exploiting data smoothness.
problem Federated learning optimization with smooth loss functions.
method Federated Low Rank Gradient Descent (FedLRGD) algorithm.
result FedLRGD outperforms Federated Averaging (FedAve) in federated oracle complexity under certain conditions.
The paper finds canonical triangulations for specific 3-manifolds.
problem Finding canonical decompositions for cusped hyperbolic 3-manifolds.
method Showed local convexity at every face of the geometric triangulation.
result Found canonical triangulations for Dehn fillings of the Borromean rings link complement and related manifolds.
In representation learning and non-linear dimension reduction, there is a huge interest to learn the 'disentangled' latent variables, where each sub-coordinate almost uniquely controls a facet of the observed data. While many regularization approaches have been proposed on variational autoencoders, heuristic tuning is …
The paper defines and studies canonical parameters on surfaces in 4D space.
problem Understanding surfaces in 4D space without minimal points.
method Defining and proving existence of canonical principal parameters.
result Surfaces in 4D space are uniquely determined by four functions satisfying partial differential equations.
Counterexample disproves log canonical Beauville--Bogomolov decomposition.
problem Disproving the log canonical Beauville--Bogomolov decomposition.
method Constructing a specific log canonical, K-trivial variety with non-birational fibers.
result Provides a counterexample to the Beauville--Bogomolov decomposition in the log canonical setting.
New insights into CE dynamics reveal how Hadamard initialization simplifies softmax.
problem Understanding the dynamics of cross-entropy training loss in deep learning.
method Analyzing a two-layer linear neural network with standard-basis vectors as inputs.
result Gradient flow on cross-entropy converges to neural collapse geometry, proving global convergence.
For a submanifold M in a Euclidean space, the tangential component x^T of the position vector field x of M is the most natural vector field tangent to the Euclidean submanifold, called the canonical vector field of M. In this article, first we prove that the canonical vector field of every Euclidean submanifold is alwa…
Canonical correlation analysis was proposed by Hotelling [6] and it measures linear relationship between two multidimensional variables. In high dimensional setting, the classical canonical correlation analysis breaks down. We propose a sparse canonical correlation analysis by adding l1 constraints on the canonical vec…
Exponential family extensions of principal component analysis (EPCA) have received a considerable amount of attention in recent years, demonstrating the growing need for basic modeling tools that do not assume the squared loss or Gaussian distribution. We extend the EPCA model toolbox by presenting the first exponentia…
Paper classifies compact symmetric triads using double Satake diagrams and canonical forms.
problem Classifying compact symmetric triads.
method Introducing double Satake diagrams and canonical forms, proving their existence and properties.
result Existence and properties of canonical forms for compact simple symmetric triads.
The paper classifies solitons on specific Lie groups.
problem Classifying solitons on three-dimensional Lorentzian Lie groups.
method Computing Wanas tensor and defining algebraic Wanas solitons.
result Classification of algebraic Wanas solitons on specific Lie groups.
Geometric framework explains and controls implicit bias in machine learning.
problem Understanding and controlling the selection of solutions in overparameterized models.
method Developed a theoretical and constructive framework based on geometric corrections induced by gradient noise and continuous symmetries of the loss.
result Computed the induced bias across various architectures and enabled inverse design to shape the bias.
Paper proposes a method to efficiently cluster stretched mixtures.
problem Clustering stretched elliptical mixtures using standard methods like PCA and k-means fails.
method Proposes a non-convex program to transform data into a one-dimensional point cloud.
result Efficient first-order algorithm achieves near-optimal statistical precision.