Extends LOSS invariant naturality to positive contact surgeries.
problem Naturality of LOSS invariant under positive contact surgeries.
method Proves naturality of LOSS invariant for Legendrian knots under positive integer contact surgeries.
result Contact invariant of +n surgery depends only on classical invariants of S.
We show that the EH class and the LOSS invariant of Legendrian knots in contact 3-manifolds are functorial under regular Lagrangian concordances in Weinstein cobordisms. This gives computable obstructions to the existence of regular Lagrangian concordances.
Monge SAM improves deep learning by making sharpness-aware minimization invariant to reparametrizations.
problem Non-invariance of sharpness-aware minimization (SAM) to reparametrizations.
method Introduces Monge SAM, a reparametrization-invariant version of SAM using a Riemannian metric.
result Monge SAM enhances robustness and generalization compared to previous methods.
Generalized algorithm for translation and scale-invariant prediction.
problem Sequential prediction with expert advice, focusing on translation and scale invariance.
method Designing a generalized online algorithm using the universal prediction perspective to compete against a generic class of expert selection strategies.
result No preliminary knowledge of loss sequences is required; performance bounds are stable under arbitrary scalings and translations.
Using established principles from Statistics and Information Theory, we show that invariance to nuisance factors in a deep neural network is equivalent to information minimality of the learned representation, and that stacking layers and injecting noise during training naturally bias the network towards learning invari…
New algorithm handles bandit problems under translations and scales.
problem Adversarial multi-armed bandit problems with arbitrary translations and scales.
method Innovative online algorithm invariant to translations and scales, using universal prediction.
result Second-order regret bounds, unaffected by affine transformations of losses.
Paper develops upper-bounds for target general loss in multiple source DA and DG settings.
problem Complexity and trade-offs in multiple source domain adaptation and domain generalization.
method Defines two types of domain-invariant representations and studies their pros, cons, and trade-offs.
result Developed upper-bounds for target general loss offer insights into domain-invariant representations.
We propose a generalization of the classical notion of the V@Rλ that takes into account not only the probability of the losses, but the balance between such probability and the amount of the loss. This is obtained by defining a new class of law invariant risk measures based on an appropriate family of acceptance set…
Consider a transverse knot which is the binding of an open book for the ambient contact manifold. In this paper, we show that the transverse invariants defined by Lisca, Ozsvath, Stipsicz, and Szabo (LOSS) are nonvanishing for such transverse knots. This is true regardless of whether or not the ambient contact structur…
Most of the banks' operational risk internal models are based on loss pooling in risk and business line categories. The parameters and outputs of operational risk models are sensitive to the pooling of the data and the choice of the risk classification. In a simple model, we establish the link between the number of ris…
DiMS sampler explores neural network loss minima via dissipative dynamics.
problem Sampling reparameterization invariant solutions in neural networks.
method Dynamical system based on kinetic energy with dissipative friction.
result DiMS sampler samples exactly from minimum level sets.
Three training regimes found for scale-invariant neural networks on the sphere.
problem Training scale-invariant neural networks on the sphere with varying effective learning rate.
method Investigated three regimes of training: convergence, chaotic equilibrium, and divergence.
result Discovered three distinct training regimes with unique characteristics.
Neural networks' weights don't converge to stationary points but training loss stabilizes.
problem The disconnect between theoretical analyses and neural network training practice.
method An invariant measure perspective inspired by ergodic theory of dynamical systems.
result The distribution of weights converges to an approximate invariant measure, explaining loss stabilization.
Study on scalar curvature minimizability loss and saddle point solutions.
problem Loss of minimizability in prescribing scalar curvature.
method Variational approach, focusing on saddle point solutions.
result Existence of saddle point solutions despite minimizability loss.
The paper proposes a method to create domain-invariant representations using Wasserstein distance.
problem Domain shifts in training data affect machine learning model performance across different domains.
method The method combines classification/regression losses with a GAN-type discriminator to minimize the Wasserstein distance between domains.
result The approach produces the highest minimum classification accuracy and most invariant representation across domains.
Vector embedding is a foundational building block of many deep learning models, especially in natural language processing. In this paper, we present a theoretical framework for understanding the effect of dimensionality on vector embeddings. We observe that the distributional hypothesis, a governing principle of statis…
The minimization of loss functions is the heart and soul of Machine Learning. In this paper, we propose an off-the-shelf optimization approach that can minimize virtually any non-differentiable and non-decomposable loss function (e.g. Miss-classification Rate, AUC, F1, Jaccard Index, Mathew Correlation Coefficient, etc…
Combinatorial method computes Legendrian knot invariant.
problem Computing the Heegaard Floer contact invariant for Legendrian knots.
method Combining Plamenevskaya's combinatorial description with Heegaard Floer theory.
result Hat version of LOSS invariant can be computed combinatorially.
LMC loss barrier decreases to zero with large network width.
problem Understanding the LMC phenomenon in neural networks.
method Fine-grained analysis of LMC for two-layer ReLU networks.
result LMC loss barrier decreases to zero at a rate of O(m^-1/2) for large network width.
Invariant polynomials improve machine learning performance.
problem Improving machine learning algorithms using invariant polynomials.
method Developed and implemented Lorentz- and permutation-invariant polynomial generators in neural networks.
result Reduction in loss on training and validation data with Hironaka decompositions.
Proposes IIB for domain generalization, overcoming failure modes of IRM.
problem Domain generalization with nonlinear classifiers and pseudo-invariant features.
method Invariant Information Bottleneck (IIB) using mutual information and variational formulation.
result Significantly outperforms IRM on synthetic datasets and real-world benchmarks.
We propose a permutation-invariant loss function designed for the neural networks reconstructing a set of elements without considering the order within its vector representation. Unlike popular approaches for encoding and decoding a set, our work does not rely on a carefully engineered network topology nor by any addit…
We consider the problem of estimating a low-rank matrix from a noisy observed matrix. Previous work has shown that the optimal method depends crucially on the choice of loss function. In this paper, we use a family of weighted loss functions, which arise naturally for problems such as submatrix denoising, denoising wit…
Framework adds invariance to pretrained networks without fine-tuning.
problem Adding invariance to pretrained networks without altering original behavior.
method Post-training augmentation invariance framework with Markov-Wasserstein minimization and Wasserstein correlation maximization losses.
result Adapter networks improve classification accuracy on rotated and noisy images.
We prove the equivalence of the invariants EH(L) and LOSS-(L) for oriented Legendrian knots L in the 3-sphere equipped with the standard contact structure, partially extending a previous result by Stipsicz and Vertesi. In the course of the proof we relate the sutured Floer homology groups associated with a knot complem…
Discover conservation laws from trajectories using a neural network.
problem Finding invariants and conservation laws from large-scale data without prior knowledge.
method ConservNet, a neural network trained with noise-variance loss to discover hidden invariants in grouped multi-dimensional observables.
result Successfully discovers underlying invariants from simulated and real-world systems.
We improve generative models for heavy-tailed multivariate data using an invariant statistical loss.
problem Traditional generative models struggle with heavy-tailed and multivariate data, leading to unstable training and mode dropping.
method We extend the invariant statistical loss method to handle heavy-tailed and multivariate data using a Pareto-ISL generator trained with input noise from a generalised Pareto distribution.
result Pareto-ISL accurately models the tails of heavy-tailed distributions while capturing central characteristics.
The gain-loss ratio is known to enjoy very good properties from a normative point of view. As a confirmation, we show that the best market gain-loss ratio in the presence of a random endowment is an acceptability index and we provide its dual representation for general probability spaces. However, the gain-loss ratio w…
ASAM improves deep neural network generalization by adapting sharpness to scale.
problem Fixed-radius sharpness measure is sensitive to parameter scaling, weakening its connection to generalization.
method Introduces adaptive sharpness, a scale-invariant measure, and proposes ASAM for deep learning.
result ASAM significantly improves model generalization performance across various datasets.
A new AMM design reduces impermanent loss and retains more liquidity.
problem Inefficiencies in conventional AMM designs lead to liquidity loss and user engagement issues in DEXs.
method Proposes a dual-mechanism framework: a power-law invariant BMM and dynamic rebate system.
result Reduces impermanent loss by 36% and retains 3.98x more liquidity during price volatility.
HistNetQ improves quantification tasks by optimizing loss functions and eliminating label requirements.
problem Quantification of class prevalence in bags of examples.
method Permutation-invariant Histograms and deep neural networks.
result HistNetQ outperforms other quantification methods and optimizes custom loss functions.
We prove that the LOSS and GRID invariants of Legendrian links in knot Floer homology behave in certain functorial ways with respect to decomposable Lagrangian cobordisms in the symplectization of the standard contact structure on R3. Our results give new, computable, and effective obstructions to the exist…
Learn invariances in neural networks by optimizing over augmentation parameters.
problem Lack of knowledge about present invariances and their extent in data.
method Parameterize a distribution over augmentations and optimize network parameters and augmentation parameters simultaneously.
result Recover correct set and extent of invariances on various tasks from training data alone.
New method improves transfer and robustness of supervised contrastive learning.
problem Class collapse in supervised contrastive learning leads to poor representation quality.
method Adding a weighted class-conditional InfoNCE loss and a class-conditional autoencoder.
result Improves transfer and robustness on 5 standard datasets and 3 worst-group robustness datasets.
SGD tends to favor simpler subnetworks, improving generalization.
problem SGD's tendency to favor simpler subnetworks over complex ones.
method Identifying invariant sets and analyzing SGD's behavior around them.
result SGD collapses networks to simpler subnetworks, improving generalization.
Fisher loss improves deep domain adaptation by learning discriminative within-class compact and between-class separable representations.
problem Improving deep domain adaptation performance by learning discriminative representations.
method Proposes a Fisher loss to learn discriminative representations that are within-class compact and between-class separable.
result Noticeable improvements in deep domain adaptation performance, e.g., 6.67% absolute improvement in mean accuracy on the Office-Home dataset.
We use the contact invariant defined in [2] to construct a new invariant of Legendrian knots in Kronheimer and Mrowka's monopole knot homology theory (KHM), following a prescription of Stipsicz and Vértesi. Our Legendrian invariant improves upon an earlier Legendrian invariant in KHM defined by the second author in sev…
IRM fails to capture natural invariances on simple problems.
problem IRM fails to capture natural invariances on simple problems.
method IRM formulation and practical linear variant.
result IRM can lead to worse generalization than unconstrained ERM.
New method trains generative models without discriminators, improving stability and accuracy.
problem Training implicit generative models with adversarial discriminators leads to instability and mode-dropping.
method Invariant statistical loss function, avoiding discriminators.
result Successfully trains generative models for various complex distributions without mode-dropping.
Proposes a non-adversarial method for distribution matching.
problem Stability and optimization challenges in adversarial matching methods.
method Non-adversarial VAE-based matching method with alignment upper bounds.
result Demonstrates applicability of non-adversarial matching methods without modifying original architectures.
Fine-grained analysis of gradient descent with momentum provides modified loss equations.
problem Understanding the dynamics of gradient descent with momentum.
method Fine-grained analysis and derivation of modified loss equations.
result Global approximation bounds and continuous modified equations for HB.
New surgeries on knots preserve contact structures.
problem Understanding how surgeries on Legendrian knots affect their contact structures.
method Analyzing surgeries on specific types of knots (twist and two-bridge knots) and proving distinct contact structures for certain surgeries.
result Negative rational surgeries on certain Legendrian knots yield distinct contact 3-manifolds.
We present an attention-based ranking framework for learning to order sentences given a paragraph. Our framework is built on a bidirectional sentence encoder and a self-attention based transformer network to obtain an input order invariant representation of paragraphs. Moreover, it allows seamless training using a vari…
ScoreStop uses gradient tests to stop gradient boosting early.
problem Overfitting in gradient boosted decision trees.
method ScoreStop uses a functional score test based on gradients to stop boosting.
result ScoreStop is competitive with loss-based early stopping methods.
Unified framework for risk evaluation under uncertainty.
problem Risk assessment under multiple economic scenarios.
method Axiomatic framework for generalized risk measures.
result Characterization of worst-case, coherent, and robust risk measures.
While stochastic gradient descent (SGD) and variants have been surprisingly successful for training deep nets, several aspects of the optimization dynamics and generalization are still not well understood. In this paper, we present new empirical observations and theoretical results on both the optimization dynamics and…
Scales attention for long contexts in LLMs.
problem Development of attention mechanisms for long context inference.
method Scale-invariant total attention and sparsity conditions, with a position-dependent transformation of logits.
result Scale-invariant attention scheme improves validation loss and long-context retrieval.
Intervention loss stabilizes GAN training and reduces mode collapse.
problem Stabilizing GAN training and mitigating mode collapse.
method Intervention loss regularization term introduced into GAN objective.
result Improved GAN training stability and reduced mode collapse.