Paper connects sampling and labeling biases in large-output spaces.
problem Efficient training in large-output spaces with label imbalance.
method Unified approach to address sampling and labeling biases.
result Different negative sampling schemes trade-off performance on dominant and rare labels.
We consider the problem of retrieving the most relevant labels for a given input when the size of the output space is very large. Retrieval methods are modeled as set-valued classifiers which output a small set of classes for each input, and a mistake is made if the label is not in the output set. Despite its practical…
Predicting structured outputs can be computationally onerous due to the combinatorially large output spaces. In this paper, we focus on reducing the prediction time of a trained black-box structured classifier without losing accuracy. To do so, we train a speedup classifier that learns to mimic a black-box classifier u…
Efficiently samples sequences without replacement for machine learning models.
problem Generating diverse outputs from sequential models without duplicates.
method Incremental sampling procedure for randomized programs, including neural models.
result Efficacy and flexibility of incremental sampling for large output spaces.
A new method quantizes output space for multi-target regression.
problem Predicting multiple continuous targets using shared predictors.
method MRQ method that quantizes output space to model dependencies and scale.
result MRQ achieves high scalability and competitive accuracy.
Scalable method bounds Lipschitz constant of generative models.
problem Bounding the Lipschitz constant of generative models.
method Layerwise convex approximations using zonotopes.
result Efficient and tight bounds on generative models.
A new method for federated learning with only positive labels.
problem Learning with only positive labels leads to poor classifier performance.
method Federated Averaging with Spreadout (FedAwS) framework.
result FedAwS can almost match the performance of conventional learning with negative labels.
Novel risk bound for structured prediction tackles non-i.i.d. data.
problem Structured prediction challenges due to non-factorizable target objects.
method PAC-Bayesian risk bound with explicit structure distillation.
result Generalization rate scales with data size and structure.
Several recently proposed stochastic optimization methods that have been successfully used in training deep networks such as RMSProp, Adam, Adadelta, Nadam are based on using gradient updates scaled by square roots of exponential moving averages of squared past gradients. In many applications, e.g. learning with large …
Calculation of the log-normalizer is a major computational obstacle in applications of log-linear models with large output spaces. The problem of fast normalizer computation has therefore attracted significant attention in the theoretical and applied machine learning literature. In this paper, we analyze a recently pro…
This paper tackles multi-modal label disentanglement in partition-based XMC.
problem Existing partition-based XMC methods create mutually exclusive clusters, which is sub-optimal for multi-modal labels.
method Formulates label assignment as an optimization problem to maximize precision rates, creating flexible and overlapped label clusters.
result Successfully disentangles multi-modal labels, leading to state-of-the-art results on XMC benchmarks.
Many structured prediction problems (particularly in vision and language domains) are ambiguous, with multiple outputs being correct for an input - e.g. there are many ways of describing an image, multiple ways of translating a sentence; however, exhaustively annotating the applicability of all possible outputs is intr…
Sig-PCA integrates model outputs and observations to correct model biases.
problem Improving model accuracy and reliability by correcting biases and numerical approximations.
method Sig-PCA framework that combines summary statistics from model outputs with localized observations via a neural network.
result Corrects model outputs to align closely with observational data, preserving essential statistical information.
Unified approach to structured prediction combining entropy regularization and neuro-symbolic logic.
problem Structured prediction challenges due to large output spaces and insufficient labeled data.
method Neuro-symbolic entropy regularization loss that restricts entropy regularization to valid structures.
result Models predict more accurately and are more likely to be valid.
Structured learning is appropriate when predicting structured outputs such as trees, graphs, or sequences. Most prior work requires the training set to consist of complete trees, graphs or sequences. Specifying such detailed ground truth can be tedious or infeasible for large outputs. Our main contribution is a large m…
We consider the extreme multi-label text classification (XMC) problem: given an input text, return the most relevant labels from a large label collection. For example, the input text could be a product description on Amazon.com and the labels could be product categories. XMC is an important yet challenging problem in t…
The Softmax function on top of a final linear layer is the de facto method to output probability distributions in neural networks. In many applications such as language models or text generation, this model has to produce distributions over large output vocabularies. Recently, this has been shown to have limited repres…
For linear classifiers, the relationship between (normalized) output margin and generalization is captured in a clear and simple bound -- a large output margin implies good generalization. Unfortunately, for deep models, this relationship is less clear: existing analyses of the output margin give complicated bounds whi…
Recent state-of-the-art video generation systems employ Generative Adversarial Networks (GANs) or Variational Autoencoders (VAEs) to produce novel videos. However, VAE models typically produce blurry outputs when faced with sub-optimal conditioning of the input, and GANs are known to be unstable for large output sizes.…
Gradient descent converges linearly for neural networks with specific conditions.
problem Optimizing neural networks with fixed width and depth.
method Local Polyak-Lojasiewicz criterion for gradient flow and descent.
result Gradient descent converges to zero-loss solutions under certain conditions.
One LN layer stabilizes neural network extrapolation.
problem Understanding neural networks' extrapolation behavior.
method Applied NTK theory to infinitely-wide networks.
result Adding one LN layer stabilizes network outputs.
New framework trains large SciML models solving PDEs in reasonable time.
problem Training large SciML models solving PDEs is challenging and time-consuming.
method Data parallel distributed deep learning framework with optimized methods.
result Neural PDE solvers can be viably trained for practical applications.
Despite being the standard loss function to train multi-class neural networks, the log-softmax has two potential limitations. First, it involves computations that scale linearly with the number of output classes, which can restrict the size of problems we are able to tackle with current hardware. Second, it remains unc…
Paper proposes a new ML approach to estimate g-vulnerability without estimating conditional probabilities.
problem Estimating information leakage in black-box systems with large output domains.
method Developed a novel ML-based approach to estimate g-vulnerability without estimating conditional probabilities.
result The proposed method outperforms frequentist approach when dealing with large output domains.
Study of tangent spaces in diffeological spaces under Lie group actions.
problem Understanding tangent spaces in generalized spaces.
method Generalized tangent space construction and isomorphism proof.
result Internal tangent space isomorphic to stratified tangent space.
(1,1) non-L-space knots are foliar in 3D space.
problem Proving (1,1) non-L-space knots are foliar.
method Analyzing (1,1) non-L-space knots in S3 and lens spaces. result (1,1) non-L-space knots are persistently foliar.
Universal spaces for finite topological spaces simplify shape descriptions.
problem Describing shape properties of compact metric spaces.
method Inverse limits of finite spaces and Alexandroff extensions.
result Universal spaces simplify shape descriptions of compact metric spaces.
The paper extends Stone duality to topological convexity spaces.
problem Understanding the relationship between topological convexity spaces and sup-lattices.
method Extending Stone duality to topological convexity spaces using preconvexity spaces.
result An adjunction between topological convexity spaces and sup-lattices.
No Einstein hypersurfaces found in Damek-Ricci spaces.
problem Existence of Einstein hypersurfaces in symmetric spaces.
method Analyzing properties of Damek-Ricci spaces and proving no Einstein hypersurface exists.
result No Einstein hypersurfaces in Damek-Ricci spaces.
The distance function ϱ(p,q) (or d(p,q)) of a distance space (general metric space) is not differentiable in general. We investigate such distance spaces over Rn, whose distance functions are differentiable like in case of Finsler spaces. These spaces have several good properties, yet they are no F…
New kernels defined for various spaces, including measures.
problem Defining kernels on non-standard spaces like measures.
method Integrally strictly positive definite and characteristic kernels on Hilbert, Banach, and metric spaces.
result Explicit classes of kernels on Lp spaces and sets of measures. Metric spaces uniquely split into Hilbert and non-line-split parts.
problem Understanding the structure of metric spaces.
method Proved unique decomposition into Hilbert and non-line-split parts.
result Metric spaces have a unique decomposition into a Hilbert space and a non-line-split part.
In a paper (math.DG/0403528) we obtained explicit examples of Moishezon twistor spaces of some compact self-dual four-manifolds admitting a non-trivial Killing field, and also determined their moduli space. In this note we investigate minitwistor spaces associated to these twistor spaces. We determine their structure, …
Study geometry of tetrahedra in complex hyperbolic space and Hilbert spaces.
problem Understanding geometric relationships between complex hyperbolic spaces and Hilbert spaces.
method Use a complex analog of the cosine of a vertex angle as a novel technical tool.
result Describe possible triangular faces of tetrahedra in hyperbolic space and three-dimensional subspaces in Hilbert spaces with Pick kernels.
New curvature positivity helps classify spherical spaces and complex projective spaces.
problem Classifying spherical space forms and complex projective spaces.
method Introducing a new positivity notion for curvature.
result Characterizations for spherical space forms and complex projective spaces.
Study of complete space-like self-expanders in Minkovski space.
problem Characterize complete space-like self-expanders in Minkovski space.
method Use of maximum principle of Omori-Yau type to prove rigidity theorems.
result Classification of 2-dimensional complete space-like self-expanders with constant squared norm of the second fundamental form.
The abstract discusses the linear and smooth structures of mapping spaces.
problem The structure of mapping spaces in differential geometry.
method Proving diffeomorphisms and fibre bundle properties.
result Path spaces and base point preserving mapping spaces are Fréchet spaces.
Study on convergence of transformed metric spaces as dimensions grow.
problem Conditions for convergence of transformed metric spaces.
method Clarifying conditions for convergence of transformed spaces from original sequence and vice versa.
result Spheres and projective spaces converge to Gaussian space and its quotient as dimensions increase.
Characterizes when almost smooth spaces become RCD spaces.
problem Understanding conditions for almost smooth spaces to be RCD spaces.
method Characterizations via local volume doubling and Poincaré inequality.
result Characterizes Einstein 4-orbifolds.
Stability of Wasserstein spaces under various convergence types.
problem Stability and finiteness of Wasserstein spaces over singular and non-singular spaces.
method Gromov--Hausdorff convergence and equivariant Gromov--Hausdorff convergence.
result Analogue of Perelman's stability theorem on Wasserstein spaces.
New infinite families of flat spaces found from symmetric spaces.
problem Finding new flat homogeneous spaces.
method Starting from compact symmetric spaces, constructing infinite families of compact homogeneous spaces with invariant Bismut connections.
result Infinite families of compact homogeneous spaces with vanishing Ricci tensor.
New quasi space forms solve Thurston's geometrical space form problem.
problem Solving Thurston's geometrical space form problem.
method Introducing quasi space forms as non-real space forms with specific geometric properties.
result Quasi space forms offer a metrical, local geometrical solution to Thurston's problem.
Classifies compact spaces by shape, finite spaces by weak homotopy.
problem Classifying compact Hausdorff spaces and finite topological spaces.
method Constructs a category that classifies spaces by shape and weak homotopy.
result Classifies compact spaces by shape, finite spaces by weak homotopy.
Introduces new types of homogeneous spaces and their properties.
problem Defining and understanding new types of homogeneous spaces.
method Introducing and analyzing (strongly) (Θ-)discrete homogeneous spaces. result Discovers relationships between new and existing homogeneous space types.
This work describes compactifications of metric spaces and vector spaces using asymmetric norms.
problem Compactifying metric spaces and vector spaces using asymmetric norms.
method Nonstandard methods, ultrapowers of the spaces at hand.
result Polyhedral compactifications of vector spaces with stratified structure.
In this paper, we present a unified study of the moduli space of tropical curves and Outer space which we link via period maps to the moduli space of tropical abelian varieties and the space of positive definite quadratic forms. Our work is a first step towards exhibiting Outer space and the space of positive definite …
Survey Bernstein-type theorems for graphical surfaces in Euclidean and Lorentz-Minkowski spaces.
problem Proving theorems for minimal and constant mean curvature graphs in Euclidean and Lorentz-Minkowski spaces.
method Explains several proofs and provides mean curvature estimates for graphs in Euclidean and Lorentz-Minkowski spaces.
result Bernstein-type theorems for constant mean curvature graphs in Euclidean 3-space and space-like graphs in Lorentz-Minkowski 3-space.
CAT(0) spaces with small volume growth are homeomorphic to Euclidean space.
problem Characterizing CAT(0) spaces with specific volume growth properties.
method Analyzing asymptotic topological regularity and volume growth.
result CAT(0) spaces with small volume growth are homeomorphic to Euclidean space.