Magnitude is not continuous but may be stable for most finite metric spaces.
problem Stability of magnitude invariant in finite metric spaces.
method Investigates the continuity properties of magnitude with respect to Gromov-Hausdorff topology.
result Magnitude is nowhere continuous but may be generically continuous.
Novel metric space magnitude and weighting vectors improve machine learning tasks.
problem Improving machine learning algorithms using novel metric space concepts.
method Metric space magnitude and weighting vectors for better machine learning.
result The weighting vector effectively detects boundaries and improves classic machine learning tasks.
Defines magnitude for length spaces with measures, agreeing with finite spaces' magnitude.
problem Defining magnitude for non-finite metric spaces with measures.
method Integrals over geodesics, using counting and weight measures.
result Magnitude agrees with finite spaces' magnitude and volume under specific conditions.
Hepworth, Willerton, Leinster and Shulman introduced the magnitude homology groups for enriched categories, in particular, for metric spaces. The purpose of this paper is to describe the magnitude homology group of a metric space in terms of order complexes of posets. In a metric space, an interval (the set of points b…
New measures quantify diversity of latent representations using metric space magnitude.
problem Evaluating the diversity of latent representations in machine learning models.
method Developed magnitude-based measures for latent representations, stable under data perturbations.
result Demonstrated superior performance across various domains and tasks.
Magnitude is a real-valued invariant of metric spaces, analogous to the Euler characteristic of topological spaces and the cardinality of sets. The definition of magnitude is a special case of a general categorical definition that clarifies the analogies between various cardinality-like invariants in mathematics. Altho…
This research quantifies neural networks using magnitude, a topological invariant.
problem Understanding the generalization capabilities of neural networks.
method Using a novel topological invariant called magnitude to study neural network representations.
result Magnitude dimension is theoretically connected to generalisation error and can predict it.
Magnitude of Euclidean domains predicts Willmore energy in odd dimensions.
problem Magnitude function of compact domains in odd dimensions.
method Asymptotic expansion of magnitude function at infinity.
result Magnitude function determines Willmore energy of boundary in odd dimensions.
In this paper we define the magnitude of metric spaces using measures rather than finite subsets as had been done previously and show that this agrees with earlier work with Leinster in arXiv:0908.1582. An explicit formula for the magnitude of an n-sphere with its intrinsic metric is given. For an arbitrary homogeneous…
Magnitude study on manifolds using fractional Laplacian.
problem Magnitude invariant of compact metric spaces via fractional Laplacian.
method Semiclassical analysis of nonlocal boundary value problem related to fractional Laplacian.
result Asymptotic expansion of magnitude in terms of curvature invariants.
This paper introduces new invariants for time series analysis.
problem Analyzing the diversity and invariants of time series data.
method Introduces new invariants derived from the continuity of magnitude and maximum diversity.
result Demonstrates improved performance in machine learning experiments with real-world data.
Magnitude-based features capture interactions between different entities in multispecies spatial data.
problem Capturing interactions between different entities in multispecies spatial data.
method Developing magnitude-based features for multispecies spatial data.
result Identifies distinct neighbourhood types and spatial heterogeneity.
Study on earthquake metric on Teichmüller space, proving properties and new completions.
problem Understanding the earthquake metric on Teichmüller space.
method Proofs of properties, new completions, and interpretation of the metric.
result Coincidence of various completions for the earthquake metric.
Our work connects parameter magnitudes and Hessian eigenspaces in deep neural nets.
problem Understanding the relationship between parameter magnitudes and Hessian curvature in deep learning models.
method Developed a matrix-free algorithm based on sketched SVDs to measure similarity between parameter masks and Hessian eigenspaces.
result Top Hessian eigenvectors tend to be concentrated around larger parameters, indicating a connection between parameter magnitudes and loss curvature.
Magnitude of manifolds linked to Riesz energies and beta functions.
problem Magnitude invariant and its geometric significance.
method Relating magnitude invariant to Brylinski's beta function and pseudodifferential analysis.
result Precise relation between magnitude invariant and beta function for closed manifolds.
At critical coupling, the interactions of Ginzburg-Landau vortices are determined by the metric on the moduli space of static solutions. The asymptotic form of the metric for two well separated vortices is shown here to be expressible in terms of a Bessel function. A straightforward extension gives the metric for N vor…
Study small eigenvalues of Riemann surfaces degenerating with Kähler metrics.
problem Determining small eigenvalues of the Laplacian on degenerating Riemann surfaces.
method Combining heat kernel estimates and Quillen metrics to compute asymptotic behavior of eigenvalues.
result Explicit calculation of small eigenvalues as a function of the parameter.
Metric learning makes it plausible to learn distances for complex distributions of data from labeled data. However, to date, most metric learning methods are based on a single Mahalanobis metric, which cannot handle heterogeneous data well. Those that learn multiple metrics throughout the space have demonstrated superi…
Magnitude of geometric shapes studied for smooth manifolds, revealing spectral geometry insights.
problem Understanding the geometric significance of Leinster's magnitude for smooth manifolds.
method Investigation of magnitude function for various distance functions, including submanifolds and Riemannian manifolds, with asymptotic analysis in the limit.
result Magnitude function is well-defined and meromorphically continued for large distances, revealing volume, surface area, and curvature integrals.
New method uses weighting vectors for efficient boundary and outlier detection.
problem Boundary and outlier detection in machine learning.
method Recast metric space magnitude as weighting vector, solve kernelized SVM, apply nearest neighbor methods.
result Weighting vector can be efficiently approximated in linear time, outperforming state-of-the-art techniques.
We identify a set of "energy" functionals on the space of metrics in a given Kaehler class on a Calabi-Yau manifold, which are bounded below and minimized uniquely on the Ricci-flat metric in that class. Using these functionals, we recast the problem of numerically solving the Einstein equation as an optimization probl…
A new metric learning scheme for structured data combining graph and feature-space information.
problem Learning a metric from structured data while respecting metric constraints.
method Training metric-constrained linear combinations of dissimilarity matrices, applying graph-based optimization under constraints.
result Our approach can reduce computational complexity by one order of magnitude for some cases.
The Earth Mover's Distance (EMD) is a state-of-the art metric for comparing discrete probability distributions, but its high distinguishability comes at a high cost in computational complexity. Even though linear-complexity approximation algorithms have been proposed to improve its scalability, these algorithms are eit…
A new model for point processes without intensity function trade-offs.
problem Inefficiency and trade-offs in existing point process models.
method Point Set Diffusion, a diffusion-based latent variable model.
result Achieves state-of-the-art performance in point process generation.
Magnitude homology reveals that graphs can have torsion subgroups.
problem Understanding torsion in magnitude homology of graphs.
method Analysis of magnitude homology defined by Hepworth and Willerton.
result Torsion of any prime order can appear in graphs' magnitude homology.
Accelerated magnetic resonance (MR) scan acquisition with compressed sensing (CS) and parallel imaging is a powerful method to reduce MR imaging scan time. However, many reconstruction algorithms have high computational costs. To address this, we investigate deep residual learning networks to remove aliasing artifacts …
Metrics specifying distances between data points can be learned in a discriminative manner or from generative models. In this paper, we show how to unify generative and discriminative learning of metrics via a kernel learning framework. Specifically, we learn local metrics optimized from parametric generative models. T…
New Lipschitz bound for ReLU networks resists weight rescaling.
problem Lack of robustness guarantees for ReLU networks under weight perturbations.
method Rescaling-invariant Lipschitz bound based on path-metrics.
result The new bound applies to various ReLU-DAG architectures and resists neuron-wise rescalings.
We propose regression networks for the problem of few-shot classification, where a classifier must generalize to new classes not seen in the training set, given only a small number of examples of each class. In high dimensional embedding spaces the direction of data generally contains richer information than magnitude.…
CRBMs improve financial regime detection with PCD and free energy analysis.
problem Detecting systemic risk regimes in financial time series.
method Extended RBM to CRBM with autoregressive conditioning and PCD. Decomposed free energy into magnitude and correlation components.
result CRBM's free energy metric distinguishes between magnitude shocks and market regimes.
Machine learning speeds up finding Calabi-Yau metrics.
problem Finding numerical Calabi-Yau metrics efficiently.
method Combining curve fitting and machine learning to approximate Ricci-flat metrics.
result Machine learning can predict Calabi-Yau metrics with minimal training data.
Efficient audio synthesis is an inherently difficult machine learning task, as human perception is sensitive to both global structure and fine-scale waveform coherence. Autoregressive models, such as WaveNet, model local structure at the expense of global latent structure and slow iterative sampling, while Generative A…
We revisit the task of learning a Euclidean metric from data. We approach this problem from first principles and formulate it as a surprisingly simple optimization problem. Indeed, our formulation even admits a closed form solution. This solution possesses several very attractive properties: (i) an innate geometric app…
This paper proposes an inexpensive way to learn an effective dissimilarity function to be used for k-nearest neighbor (k-NN) classification. Unlike Mahalanobis metric learning methods that map both query (unlabeled) objects and labeled objects to new coordinates by a single transformation, our method learns a trans…
Cosine similarity can force points to grow in magnitude, causing convergence issues.
problem Cosine similarity loss can lead to convergence issues in deep learning.
method Analyzing under-explored settings and proposing cut-initialization.
result Cosine similarity optimization forces points to grow in magnitude, leading to convergence issues.
Neural nets approximate Ricci flat metrics for Calabi-Yau manifolds.
problem Lack of analytical Ricci flat metrics for Calabi-Yau threefolds.
method Employed neural network approximations for several Calabi-Yau manifolds of dimensions two and three.
result Measures of Ricci flatness improved by three orders of magnitude after training.
Lookahead pruning extends single-layer optimization to multi-layer, outperforming magnitude-based pruning.
problem Pruning neural networks to reduce computational cost and memory usage.
method Developed a multi-layer optimization approach extending the single-layer optimization of magnitude-based pruning.
result Consistently outperforms magnitude-based pruning on various networks, especially in high sparsity.
Are expansions and recessions more likely to end as their magnitude increases? In this paper we apply parametric hazard models to investigate this issue in a sample of 16 countries from 1881 to 2000. For the total sample we find evidence of positive magnitude dependence for recessions, while for expansions we are not a…
Neural networks can learn distance metrics affecting model performance.
problem Understanding how neural networks learn and represent data.
method Experiments with six MNIST architectures, constrained to learn either distance or intensity representations.
result Distance-based learning affects model performance, validating the geometric framework.
A new method models financial returns by separating sign and magnitude, improving forecasting accuracy.
problem Capturing nonlinear predictability in financial return dynamics.
method Decomposes returns into sign and magnitude components, using a joint distribution model.
result Significantly outperforms traditional linear models in forecasting U.S. stock market returns.
To address the limitations of existing magnitude-based pruning algorithms in cases where model weights or activations are of large and similar magnitude, we propose a novel perspective to discover parameter redundancy among channels and accelerate deep CNNs via channel pruning. Precisely, we argue that channels reveali…
Metrics assess uncertainty structure and distribution for regression models.
problem Quantifying uncertainty in high-dimensional and nonlinear regression tasks.
method Two bounded comparison metrics for uncertainty structure and distribution.
result DNNs and DNOs provide encouraging uncertainty metric values in high dimensions.
Develops methods to measure and set function-space learning rates in neural networks.
problem Measuring and optimizing changes in neural network output functions.
method Efficient methods to measure and set function-space learning rates, requiring minimal computational overhead.
result Demonstrates FLeRM (Function-space Learning Rate Matching) for hyperparameter transfer across model scales.
We find the minimal value of the length in de Sitter space of closed space-like curves with non-vanishing non-space-like geodesic curvature vector. These curves are in correspondence with closed almost-regular canal surfaces, and their length is a natural magnitude in conformal geometry. As an application, we get a low…
SAMBA improves safe reinforcement learning with active exploration metrics.
problem Safe reinforcement learning in dynamic systems.
method Combines probabilistic modelling, information theory, and statistics. Uses novel metrics for out-of-sample Gaussian process evaluation.
result Orders of magnitude reduction in samples and violations compared to state-of-the-art methods.
Previous studies indicate that nonlinear properties of Gaussian time series with long-range correlations, ui, can be detected and quantified by studying the correlations in the magnitude series ∣ui∣, i.e., the ``volatility''. However, the origin for this empirical observation still remains unclear, and the exact …
Bayesian approach improves AdaLoRA's performance and efficiency.
problem Improving the efficiency and performance of adaptive low-rank adaptation.
method Utilized Bayesian metrics and the Improved Variational Online Newton (IVON) optimizer for adaptive parameter budget allocation.
result Bayesian counterpart outperforms sensitivity-based importance metric and is faster than AdaLoRA.
A new pruning criterion reduces model size and improves performance.
problem Overparameterized neural networks are computationally and memory intensive, leading to overfitting.
method Introduces a magnitude and uncertainty (M&U) pruning criterion inspired by statistical Wald test.
result Our M&U pruning criterion leads to more compressed models with less loss in predictive power.