Detects singularities in complex data to improve machine learning models.
problem Real-world data often contains non-manifold structures (singularities) that can mislead machine learning models.
method Develops a topological framework to quantify local intrinsic dimension and Euclidicity score for multiple scales.
result Identifies singularities and captures local geometric complexity in image data.
We introduce thermodynamic response functions for singular Bayesian models.
problem Singular Bayesian models violate regular asymptotics due to non-identifiability and degenerate Fisher geometry.
method Posterior tempering induces thermodynamic response functions, linking WAIC, WBIC, and singular fluctuation.
result WAIC, WBIC, and singular fluctuation are unified within a thermodynamic response framework.
Study shows LLC correlates with neural network compressibility.
problem Evaluating limits of neural network compression.
method Extended minimum description length principle using singular learning theory.
result Complexity estimates based on LLC are linearly correlated with compressibility.
Bayesian models' singular fluctuation is shown to be akin to specific heat, influencing model complexity and generalization.
problem Understanding the thermodynamic interpretation of singular fluctuation in Bayesian models.
method Showed singular fluctuation as the curvature of Bayesian free energy and variance of log-likelihood observable under a Gibbs posterior.
result Singular fluctuation is the statistical analogue of specific heat, controlling model complexity and generalization.
Deterministic bounds for tensor singular values and vectors, differing from matrix cases.
problem Spectral learning of higher-order orthogonally decomposable tensors.
method Deterministic perturbation bounds for singular values and vectors of orthogonally decomposable tensors.
result Perturbation affects each essential singular value/vector in isolation, independent of multiplicity and distance from other singular values.
Paper tackles singularity detection in PDEs using data-driven self-supervised learning.
problem Detecting singularities in PDE solutions for efficient numerical methods.
method Data-driven self-supervised learning framework with filtering tasks.
result Proposes filtering methods for raw unlabeled data to improve singularity detection.
Fast and accurate methods for low-rank learning problems.
problem Partial singular value decomposition and numerical rank estimation of huge matrices.
method Krylov subspaces and Ritz vectors for fast and accurate solutions.
result Advantages over traditional methods in accuracy and speed.
LS improves model selection for singular statistical models.
problem Challenges in model selection for singular statistical models.
method Integrates empirical loss from WAIC and sBIC penalty term.
result Enhanced utility for model selection without regularity constraints.
Examines learning efficiency in neural networks and related models.
problem Analyzing efficiency in deep learning models with singular learning coefficients.
method Examined learning coefficients in neural networks and three-layer neural networks with ReLU units.
result Extended results to include Softmax function, providing a broader understanding of learning efficiency.
New method for analyzing learning dynamics in singular models.
problem Challenges in analyzing learning of singular models with no one-to-one parameter space.
method Relative reparameterization technique to extract regular sub-models.
result Demonstrated differences in convergence behavior due to algorithmic and intrinsic aspects.
A new method learns meaningful distances between samples using optimal transport.
problem Learning meaningful distances between samples in datasets without labeled data.
method Computes OT distances between samples and features using singular vectors of a function mapping ground metrics to OT distances.
result Wasserstein Singular Vectors provide a scalable solution for unsupervised ground metric learning.
STEEL tackles batch RL with singularity, improving policy optimization.
problem Existing RL methods assume absolutely continuous data, but STEEL handles non-overlapping regions.
method Proposes STEEL algorithm using maximum mean discrepancy and distributionally robust optimization.
result First finite-sample regret guarantee for batch RL with singularity.
HADES detects data singularities quickly and accurately.
problem Detecting singularities in data efficiently.
method Kernel goodness-of-fit test based on differential geometry and optimal transport theory.
result Correctly detects singularities with high probability.
Quantum statistical models with singularities are studied for state estimation and model selection.
problem Understanding statistical properties of quantum singular models.
method Classical singular learning theory extended to quantum state estimation and model selection using algebraic geometrical methods.
result Asymptotically unbiased estimator (QWAIC) for quantum generalization loss constructed.
Advances variational Bayesian neural networks using singular learning theory.
problem Discrepancies between predictive performance and variational objective in BNNs.
method Corrected asymptotic form of singular posterior distributions to inform variational family design.
result Improvements in variational free energy and generalization error with proposed normalizing flow.
PAC-Bayes bounds for Gibbs posteriors derived via singular learning theory.
problem Generalization bounds for overparameterized models with data-dependent priors.
method Explicit non-asymptotic PAC-Bayes bounds using singular learning theory.
result Explicit posterior-averaged risk bounds for overparameterized models.
Study evaluates thresholds for removing noise from DNN weights using random matrix theory.
problem Removing noise from deep neural network weights for better approximation.
method Model weights as signal + noise, use random matrix theory to estimate thresholds, evaluate using cosine similarity.
result Proposed threshold estimation method improves approximation quality.
Lectures on surface evolution through singularities.
problem Analyzing the mean curvature flow of surfaces and their singularities.
method Analysis of neck and conical singularities, using monotonicity formulas, epsilon-regularity, weak solutions, and blowup techniques.
result Unique evolution through neck singularities, nonuniqueness through conical singularities.
We analyze the local Rademacher complexity of empirical risk minimization (ERM)-based multi-label learning algorithms, and in doing so propose a new algorithm for multi-label learning. Rather than using the trace norm to regularize the multi-label predictor, we instead minimize the tail sum of the singular values of th…
The paper derives an equation linking WAIC and WBIC for singular models.
problem In singular models, conventional criteria fail due to likelihood and posterior breakdown.
method Theoretical derivation linking WAIC and WBIC.
result An asymptotic equation linking WAIC and WBIC for singular models.
Introduces LLC, a new complexity measure for DNNs based on SLT.
problem Lack of effective complexity measures for DNNs.
method Uses Singular Learning Theory to define LLC and proposes scalable estimator.
result Empirical evidence shows LLC provides valuable insights into DNN complexity.
The paper tackles optimal stopping problems using reinforcement learning and singular control.
problem Continuous-time and state-space optimal stopping problems.
method Formulated as a singular control problem with randomized stopping times and penalized cumulative residual entropy.
result Identified unique optimal exploratory strategy through dynamic programming.
A new geometric concept, the dead direction, bridges singular learning theory and information geometry.
problem The gap between singular learning theory and information geometry.
method Introducing the dead direction, a unit vector along degenerating Fisher metric, and showing its KL order can be recovered.
result The KL order of the dead direction can be recovered as the decay rate of the directional Fisher curvature, providing a handle on singular geometry.
A low rank matrix X has been contaminated by uniformly distributed noise, missing values, outliers and corrupt entries. Reconstruction of X from the singular values and singular vectors of the contaminated matrix Y is a key problem in machine learning, computer vision and data science. In this paper we show that common…
The study uses a ReLU network to discern geometric structure in data via the Data Information Matrix.
problem Understanding the geometric structure of real data in high-dimensional spaces.
method Employing a ReLU neural network trained as a classifier and the Data Information Matrix (DIM) to discern a singular foliation structure.
result The singular points of the foliation are measure zero, and a local regular foliation exists almost everywhere.
It is well known that the initialization of weights in deep neural networks can have a dramatic impact on learning speed. For example, ensuring the mean squared singular value of a network's input-output Jacobian is O(1) is essential for avoiding the exponential vanishing or explosion of gradients. The stronger condi…
New method cleans cross-covariance matrices for better financial forecasting.
problem Asymptotically optimal cross-covariance cleaners fail in real-world, time-varying markets.
method Physics-informed neural network that learns from empirical singular values.
result Trained model outperforms analytical cleaners in out-of-sample cross-covariance prediction.
SLT explains neural network success by closing theory-practice gap.
problem Failure of classical inference and learning theory in modern neural networks.
method Physics-inspired Singular Learning Theory (SLT) applied to neural networks.
result SLT recovers known and novel scaling laws for neural network phase transitions.
Study on inventory management under uncertainty using smooth ambiguity preference.
problem Managing inventory under Knightian uncertainty with smooth ambiguity preference.
method Demonstrates continuous-time smooth ambiguity as the infinitesimal limit of Kalman-Bucy filtering with recursive robust utility. Solves forward-backward stochastic differential equations with quadratic growth to determine cost function. Derives value function and optimal control policy using variational inequalities and viscosity solutions. Transforms problem into two-dimensional singular control.
result Ambiguity drives decision-makers to act earlier, reducing the continuation region.
New method learns operators with geometric singularities from few samples.
problem Learning operators with geometric singularities from limited data.
method Double fibration transforms and cross-attention architectures.
result Operators can be learned superalgebraically from few samples.
New nonconvex regularizer speeds up low-rank matrix completion.
problem Low-rank matrix completion with good theoretical and empirical performance.
method Proposes a new nonconvex regularizer with adaptive shrinkage, scalable, and fast optimization.
result Proposed method achieves state-of-the-art recovery performance and is the fastest.
Optimizes bounds for threefold singularity volumes.
problem Bounding local volumes of threefold singularities.
method Analyzes Gorenstein canonical non-hypersurface threefold singularities.
result Establishes optimal upper bound for local volumes.
Derives a primal-dual MLSVD formulation for multilinear data.
problem Efficiently decompose multilinear data for signal analysis and deep learning.
method Kernelizable primal-dual formulation of MLSVD.
result Derives a new MLSVD formulation with computational advantages.
A new geometric structure for singular models is introduced.
problem Degenerate metrics in dually flat structures.
method Introducing quasi-Hessian manifolds with degenerate metrics and symmetric cubic tensors.
result Established Amari-Nagaoka's extended Pythagorean and projection theorems for singular models.
Study describes singularities of height functions on specific singular surfaces.
problem Analyzing singularities of height functions on singular surfaces.
method Using geometric language and blowing-ups, investigate singularities of height functions and dual surfaces.
result Characterized singularities of height functions and dual surfaces on specific singular surfaces.
The paper extends affine connection results to singular warped and twisted products.
problem Generalizing affine connections to singular warped and twisted products.
method Study of singular multiply warped products and singular twisted products with semi-symmetric metric and non-metric connections, discussing Koszul forms and curvature.
result Theoretical results on curvature and Koszul forms for singular multiply warped and twisted products.
New singularity concept in GR: volume singularities.
problem Understanding spacetime singularities in General Relativity.
method Introducing a new type of singularity: volume singularities.
result Volume singularities are hidden by event horizons.
The study shows stability of neckpinch singularities in mean curvature flows.
problem Stability of neckpinch singularities in mean curvature flows.
method Analysis of mean curvature flow and perturbations.
result Stability of neckpinch singularities in mean curvature flows.
Study describes singularities of distance squared functions on singular surfaces.
problem Characterizing singularities of distance squared functions on singular surfaces.
method Using smooth map-germs Sk, Bk, Ck, and F4 singularities, the study describes singularities via blowing-ups. result Characterization of singularities of wave-fronts and caustics of singular surfaces.
Paper proposes a new optimization framework for learning eigenfunctions of operators.
problem Computing eigenvalue decomposition of high-dimensional operators.
method Operator SVD with Neural Networks via Nested Low-Rank Approximation.
result Proposed method efficiently learns top-L singular values and functions in the correct order.
In this paper, we define the set of singular grid diagrams SG which provides a unified description for singular links, singular Legendrian links, singular transverse links, and singular braids. We also classify the complete set of all equivalence relations on SG which induce the bijection onto e…
Proves positive mass theorem for AF spin manifolds with conical singularities.
problem Proving the positive mass theorem for singular metrics on AF manifolds.
method Analyzes AF spin manifolds with isolated conical singularities, allowing topological singularities.
result Proves the positive mass theorem for AF spin manifolds with conical singularities.
The study examines singularities and geometric properties of surfaces derived from frontals with specific singular points.
problem Characterizing and understanding the singularities and geometric properties of surfaces formed by the singular loci of normal congruences of frontals with pure-frontal singular points.
method Characterizations of singularities in terms of geometric invariants of the initial frontal are provided for the normal ruled surface. Relations between certain singularities of focal surfaces and geometric properties of the frontal are also explored.
result Behavior of Gaussian curvature of focal surfaces of frontals with a 5/2-cuspidal edge is considered. New framework for higher-order singular-value derivatives of rectangular matrices.
problem Challenging to derive higher-order Fréchet derivatives of singular values in real rectangular matrices.
method Using Kato's analytic perturbation theory for self-adjoint operators and embedding rectangular matrices into block self-adjoint operators.
result Closed-form expressions for the n-th order spectral variations of singular values. Rectifies flat singular points of area-minimizing currents with singularity degree > 1.
problem Rectifying flat singular points of area-minimizing currents with singularity degree > 1.
method Subdividing singular points based on singularity degree and proving rectifiability of points with singularity degree > 1.
result The set of points with singularity degree > 1 is (m-2)-rectifiable.
Proposes a regularization method for unsupervised domain adaptation that aligns predictions with target data's top singular vectors.
problem Domain adaptation challenges in high joint error scenarios.
method Regularizes classifier to align with unsupervised target data guided by label alignment property (LAP).
result The method improves performance in MNIST-USPS domain adaptation and cross-lingual sentiment analysis.
Paper generalizes a theorem for real analytic singularities.
problem No specific problem stated; focuses on generalization.
method Generalization of a theorem for complex singularities.
result Generalized Join theorem for real analytic singularities.
The paper studies singularities of pedal curves of hyperbolic frontals.
problem Investigating singularities of pedal curves of spacelike frontals in hyperbolic 2-space.
method Analyzing singularities of pedal curves based on dual curve germs and pedal point locations.
result The singularities of pedal curves depend on the singularities of the first hyperbolic Legendrian curvature germ and the pedal point for non-singular dual curve germs. For singular dual curve germs, additional dependence on both Legendrian curvature germs is observed.