Random projections enhance neural networks by reducing dimensions and speeding up training.
problem Training and expressive power of neural networks with high-dimensional inputs.
method Random projections to embed sparse vectors or low-dimensional manifolds into a smaller space, reducing the number of parameters and speeding up training.
result The number of neurons required for approximating a function depends on sparsity or manifold dimension, not the input vector dimension.
UAPCA projects uncertain data to low dimensions using GMMs.
problem Uncertain multidimensional data not well described by normal distributions.
method Model data with Gaussian mixture models, derive UAPCA projection from general formulation.
result Low-dimensional projections better represent multidimensional distributions.
Paper presents a novel method for pruning redundant instances in low-dimensional data.
problem Pruning redundant or irrelevant instances in low-dimensional data for pattern recognition.
method Supervised projection to high-dimensional space, ranking and selection of quality instances.
result Significant improvements in recognition performance over other local descriptors.
Projective DP-SGD reduces privacy error by identifying low-dimensional gradient subspaces.
problem Differentially private SGD's error rate scales with model's dimensionality, problematic for over-parameterized models.
method Projective DP-SGD, projecting noisy gradients to a low-dimensional subspace identified from a public dataset.
result The method reduces the dependence on model dimensionality, improving accuracy in high privacy regimes.
Manifold learning has been successfully applied to a variety of medical imaging problems. Its use in real-time applications requires fast projection onto the low-dimensional space. To this end, out-of-sample extensions are applied by constructing an interpolation function that maps from the input space to the low-dimen…
The local linear embedding algorithm (LLE) is a non-linear dimension-reducing technique, widely used due to its computational simplicity and intuitive approach. LLE first linearly reconstructs each input point from its nearest neighbors and then preserves these neighborhood relations in the low-dimensional embedding. W…
Proposes a method to learn optimal neighbors and projection matrix in low-dimensional space.
problem Difficulty in precisely measuring similarity and selecting optimal neighbors in high-dimensional space.
method Models similarity and neighbors as variables, optimizing a unified objective function with nonnegative and sum-to-one constraints.
result Optimal similarity and projection matrix learned simultaneously, with adaptive regularization parameter.
Develops optimal low-dimensional approximations to high-dimensional SDEs.
problem Approximating solutions to high-dimensional SDEs in a low-dimensional space.
method Introduces Ito-vector and Ito-jet projections for optimal approximation.
result Optimal projection filters yield better approximations than Stratonovich projection.
This work proposes a method to price American basket options using a Markovian projection.
problem Pricing American basket options in high dimensions is computationally expensive.
method Use a stopping rule based on a low-dimensional Markovian projection of the basket's dynamics.
result Approximate the optimal early-exercise boundary in a lower-dimensional space, providing bounds for the option price.
Deep networks can efficiently represent low-dimensional manifolds.
problem Representing data on low-dimensional manifolds in high-dimensional spaces.
method Deep neural networks, specifically the first two layers, can embed points on monotonic chains and more general manifolds with minimal error.
result Deep networks can embed points on low-dimensional manifolds with an almost optimal number of parameters and low error.
Extends dimension reduction to data-driven settings without gradients.
problem Gradient-based dimension reduction limitations in data-driven settings.
method Score ratio matching framework, tailored parameterization, regularization, eigenvalue deflation.
result Outperforms standard score-matching for problems with low-dimensional structure.
Study minimax rates for cost-sensitive classification on manifolds using approximate nearest neighbours.
problem Minimizing classification error on manifolds embedded in high-dimensional spaces.
method Approximate nearest neighbour algorithm in a randomly projected low-dimensional space.
result Minimax rates for cost-sensitive classification on manifolds are achieved by the approximate nearest neighbour algorithm.
Improved neural networks for relational reasoning by projecting high-dimensional data to low-dimensional manifolds.
problem Out-of-distribution generalization in complex relational reasoning tasks.
method Neuroscience-inspired inductive-biased module projecting high-dimensional object representations to low-dimensional manifolds.
result Significantly better out-of-distribution generalization performance on relational reasoning tasks.
We prove that any real Lie group of dimension \leq 5 admits a left invariant flat projective structure. We also prove that a real Lie group L of dimension \leq 5 admits a left invariant flat affine structure if and only if the Lie algebra of L is not perfect.
SO-MPCA learns tensor features with relaxed orthogonality.
problem Difficult to enforce orthogonality in multilinear PCA for tensors.
method Semi-Orthogonal Multilinear PCA (SO-MPCA) with relaxed start (RS).
result SO-MPCA-RS outperforms other methods on face and gait data.
Sharp-SSL uses random projections to identify important variables for semi-supervised learning.
problem High-dimensional semi-supervised learning problems.
method Careful aggregation of low-dimensional results from many axis-aligned random projections.
result Sharp-SSL algorithm can recover signal coordinates with high probability.
We give explicit formulae for fringe lengths of the Calegari-Walker Ziggurats -- i.e. graphs of extremal rotation numbers associated to positive words in free groups. These formulae reveal (partial) integral projective self-similarity in ziggurat fringes, which are low-dimensional projections of characteristic polyhedr…
RandNE efficiently embeds billion-scale networks with random projection.
problem Handling billion-scale networks in network embedding.
method Iterative Gaussian random projection to reduce high-order proximities.
result RandNE achieves efficiency and efficacy in network reconstruction, link prediction, and node classification.
Develops accelerated methods for optimization using low-dimensional projected-gradient information.
problem Optimization with low-dimensional projected-gradient information and Nesterov acceleration.
method Randomized-subspace Nesterov accelerated gradient methods for smooth convex and strongly convex optimization.
result Established accelerated oracle-complexity guarantees and unified basis for comparing sketch families.
Product models of low dimensional experts are a powerful way to avoid the curse of dimensionality. We present the ``under-complete product of experts' (UPoE), where each expert models a one dimensional projection of the data. The UPoE is fully tractable and may be interpreted as a parametric probabilistic model for pro…
Deep networks can adapt to intrinsic dimensionality beyond domain constraints.
problem Approximating functions on low-dimensional manifolds with high-dimensional data.
method Two-layer compositions with ReLU activation, using dimensionality reducing feature maps.
result Near optimal approximation rates depend on the complexity of the dimensionality reducing map, not the ambient dimension.
Estimates covariance matrix from low-dimensional compressive measurements.
problem Estimating covariance matrix from limited data.
method Unbiased estimator using i.i.d. zero-mean entries with finite moments.
result Accurate estimation of covariance matrix on real-world data.
We initiate the study of holomorphically convex groups: groups that can be realized as fundamental groups of smooth complex projective varieties with holomorphically convex universal covers. If G is a holomorphically convex group of cohomological dimension two, we show that G is isomorphic to the fundamental group …
Improved SRHT for linear SVM classification with higher accuracy.
problem Inefficient random projection methods for high-dimensional data.
method Importance sampling and deterministic top-r sampling for effective low-dimensional embedding. result Higher classification accuracy on real-life datasets.
Adversarial robustness in multi-index models is as easy as standard learning.
problem Adversarial robustness in high-dimensional multi-index models.
method Proves that hidden directions of multi-index models offer a Bayes optimal low-dimensional projection for robustness against ℓ2-bounded adversarial perturbations. result Adversarially robust learning is as easy as standard learning, requiring no additional samples.
Enhances speech recognition by modeling acoustic space as low-dimensional subspaces.
problem Improving accuracy in continuous speech recognition with deep neural networks.
method Dictionary learning and sparse coding of training posteriors to project test posteriors into a low-dimensional space.
result 15.4% relative reduction in word error rate (WER) in clean and noisy conditions.
Improved neural network training in low-dimensional random bases.
problem Inefficient optimization in large-scale neural networks.
method Re-draw random subspace at each training step, apply independent projections to different network parts.
result Significantly better optimization performance and efficiency.
FlowSDR learns a low-dimensional projection preserving the response's conditional distribution.
problem Learning a low-dimensional projection that captures the response's conditional distribution.
method FlowSDR uses conditional log-likelihood maximization with monotone rational-quadratic spline flows to learn the projection and conditional density.
result FlowSDR outperforms existing SDR methods in various simulation settings and a face-age prediction task.
Framework uses low-dimensional maps to solve high-dimensional Bayesian inference problems.
problem High-dimensional Bayesian inference problems.
method Structure-exploiting lazy maps and flows, focusing on low-dimensional subspace.
result Weak convergence of generated distributions to the posterior.
SMAVE optimizes SDR by projecting onto a low-dimensional subspace on a Riemannian manifold.
problem High-dimensional regression challenges due to the curse of dimensionality.
method SMAVE combines nearest-neighbor localization and Riemannian stochastic gradient ascent.
result SMAVE achieves almost-sure convergence and matches RMAVE's synthetic subspace recovery rate.
Paper projects GP basis functions using tensor networks to reduce complexity.
problem Efficiently approximating Gaussian process regression with a large number of basis functions.
method Develops a method using tensor networks to approximate GP regression with an exponential number of basis functions without exponential computational complexity.
result Shows efficient GP regression on an 18-dimensional benchmark data set.
Autoencoders learn compressed representations via mutual information maximization.
problem Learning efficient compressed representations of high-dimensional data.
method Proposes Uncertainty Autoencoders that treat latent representations as noisy projections and optimize mutual information.
result 32% improvement in statistical compressed sensing of high-dimensional datasets.
LaRP framework improves object classification using random projections.
problem Efficiently approximating nonlinear kernels in high-dimensional spaces.
method Separates linear kernels and nonlinearity using a layered random projection approach.
result Notable improvement in object classification performance.
Agents collaborate to reduce regret in a multi-agent linear bandit problem with side information.
problem Reducing regret in a multi-agent stochastic linear bandit with side information.
method A decentralized algorithm where agents communicate subspace indices and each plays a projected LinUCB on the corresponding low-dimensional subspace.
result Per-agent finite-time regret is much smaller when agents communicate compared to non-communicating case.
Paper develops efficient algorithms to estimate channel subspace from low-dimensional projections.
problem Estimating channel subspace information from limited projections in JSDM.
method Develops novel algorithms requiring sampling only O(2√M) elements for p-dim beamformer.
result Estimators return a p-dim beamformer with performance comparable to full knowledge.
A new DDR framework learns low-dimensional data representations using dynamical systems.
problem Learning efficient low-dimensional data representations.
method DDR framework based on nonlinear dynamical systems, using linear combinations of functions and regularization.
result DDR method outperforms other methods on synthetic and real datasets.
This article surveys our ongoing project about the relationship between invariants extending the classical Rohlin invariant of homology spheres and those coming from 4-dimensional (Yang-Mills) gauge theory. The main conjecture towards which this project is directed is that the Rohlin invariant and the gauge theoretic i…
We apply the concept of castling transform of prehomogeneous vector spaces to produce new examples of minimal homogeneous Lagrangian submanifolds in the complex projective space. Furthermore we verify the Hamiltonian stability of a low dimensional example that can be obtained in this way.
We show that any effective isometric torus action of maximal rank on a compact Riemannian manifold with positive (sectional) curvature and maximal symmetry rank, that is, on a positively curved sphere, lens space, complex or real projective space, is equivariantaly diffeomorphic to a linear action. We show that a compa…
DUAL-LOCO uses random projections for efficient distributed statistical estimation.
problem Efficiently estimating statistical models in distributed systems.
method DUAL-LOCO employs low-dimensional random projections to approximate feature dependencies across workers.
result DUAL-LOCO achieves bounded approximation error with error bounds dependent weakly on the number of workers.
Optimizes optimal transport distances using low-dimensional embeddings.
problem High computational cost of optimal transport distances in high dimensions.
method Approximate OT distances using 1-Lipschitz maps in a lower-dimensional space.
result Efficiently approximates optimal transport distances with lower computational cost.
Develops a two-sample test using projected Wasserstein distance to handle high-dimensional data.
problem Testing whether two high-dimensional samples come from the same distribution.
method Optimal projection to find a low-dimensional linear mapping that maximizes the Wasserstein distance between projected probability distributions.
result Characterizes the convergence rate of the projected Wasserstein distance and presents practical algorithms.
The paper studies how norms of random vectors are preserved by random projections.
problem Understanding how random matrix affects norms of random vectors.
method Proved the distribution of the norm of random vector is preserved by random projection.
result Random matrix preserves the distribution of the norm of random vectors with i.i.d. entries.
Develops a new algorithm for high-dimensional contextual data in bandit problems.
problem Challenges of high-dimensional contextual data in traditional linear bandits.
method Contextual Bandits with Random Projection (CBRAP) algorithm.
result Provides a linear upper regret bound associated with reduced dimensions.
This paper tackles high-dimensional Bayesian optimization by projecting a manifold into a lower space.
problem High-dimensional optimization of expensive functions with limited labeled data.
method Random linear projection of a manifold embedded in high-dimensional space, combined with semi-supervised learning of the manifold's geometry.
result Our approach outperforms existing high-dimensional BO methods in various synthetic and real-world applications.
SILBO optimizes high-dimensional Bayesian optimization using semi-supervised embedding learning.
problem Bayesian optimization struggles with high-dimensional search spaces.
method SILBO uses semi-supervised dimension reduction to find a low-dimensional space for iterative optimization.
result SILBO outperforms existing methods on high-dimensional Bayesian optimization tasks.
A new method for SVGD reduces variance in high dimensions.
problem High-dimensional variance in SVGD.
method Grassmann Stein Variational Gradient Descent (GSVGD) projects onto arbitrary subspaces and uses coupled Grassmann-valued diffusion.
result GSVGD explores high-dimensional problems with intrinsic low-dimensional structure efficiently.
SqueezeFit reduces high-dimensional data to lower dimensions while preserving label distances.
problem Label-aware dimensionality reduction in high-dimensional spaces.
method Semidefinite programming relaxation of nearest neighbor classification.
result Provable recovery of a planted projection operator from labeled data.