Unified neural network for linear and nonlinear dimension reduction.
problem Efficiently perform linear and nonlinear sufficient dimension reduction.
method Belted and Ensembled Neural Network (BENN) framework.
result Unified framework for both linear and nonlinear dimension reduction.
Develops a new method for nonlinear dimension reduction using random features.
problem Statistical challenges in generalizing Gaussian process-based latent variable models to non-Gaussian data.
method Random feature latent variable models (RFLVMs) that approximate nonlinear relationships with linear functions of random features.
result RFLVMs produce comparable results to state-of-the-art methods on various data types.
New neural network method simplifies high-dimensional data.
problem Scalability issues in nonlinear sufficient dimension reduction.
method Stochastic neural network with adaptive gradient algorithm.
result Proposed method outperforms existing methods on large-scale data.
A new DDR framework learns low-dimensional data representations using dynamical systems.
problem Learning efficient low-dimensional data representations.
method DDR framework based on nonlinear dynamical systems, using linear combinations of functions and regularization.
result DDR method outperforms other methods on synthetic and real datasets.
A new deep neural network tackles nonlinear functional regression with improved dimensionality reduction.
problem Nonlinear functional regression in infinite-dimensional functional data analysis.
method Functional deep neural network with adaptive kernel embedding and projection steps.
result Explicit rates of approximating nonlinear smooth functionals are derived, and the network is shown to be effective in both simulated and real datasets.
New methods for functional data analysis improve manifold methods for continuous data.
problem Challenges in evaluating embeddings for functional data.
method Transfer manifold methods from tabular and image data to functional data, define a theoretical framework, and propose nuanced evaluation strategies.
result Manifold methods can be successfully applied to functional data, but careful evaluation is needed.
The paper shows how to recover true node positions from a graph or similarity matrix.
problem Recovering true distances and positions from a graph or similarity matrix.
method Two steps: matrix factorisation followed by nonlinear dimension reduction.
result Nonlinear dimension reduction can recover latent positions close to a manifold where geodesic distance is encoded.
GenSDR tackles SDR by leveraging generative models to fully recover lower-dimensional structures.
problem Challenges in identifying low-dimensional sufficient structures in nonlinear SDR.
method Proposes GenSDR, a method that uses modern generative models to fully recover information in the central σ-field.
result Establishes consistency of GenSDR estimator for sample-level data and extends its applicability to non-Euclidean responses.
Two methods preserve tensor structure for reduced dimensionality in tensor regression.
problem Reducing dimensionality of tensor predictors for improved interpretation and accuracy.
method Developed two tensor dimension reduction methods using Tucker and CP decompositions.
result Substantial improvement in accuracy over existing methods in simulations and applications.
New method for reducing dimensions of distributional data.
problem Nonlinear sufficient dimension reduction for distribution-on-distribution regression.
method Building universal kernels on metric spaces to characterize conditional independence.
result Method outperforms competing methods in synthetic and real data applications.
Proposes a stratified sampling method for high-dimensional models using neural active manifolds.
problem Uncertainty propagation in computationally expensive models with many inputs.
method Neural active manifolds for nonlinear dimensionality reduction, followed by stratification in the reduced space.
result Effective variance reduction in high-dimensional models using stratified sampling.
WeldNet reduces complex dynamics to simpler, manageable segments.
problem Complex, high-dimensional time-dependent datasets from physical processes are costly to simulate.
method Windowed Encoders for Learning Dynamics, splitting time domain into windows for nonlinear dimension reduction and propagator training.
result WeldNet captures nonlinear latent structures and dynamics, outperforming existing methods.
GD-VAEs learn dynamics from observations using geometric and topological information.
problem Learning parsimonious representations of nonlinear dynamics from observations.
method Develops data-driven methods incorporating geometric and topological information using Variational Autoencoders (VAEs).
result GD-VAEs provide methods for learning reduced dimensional representations of nonlinear dynamics.
Reduces function approximation dimensions from high to low with sparse data.
problem Function approximation from sparse data.
method Nonlinear Level Set Learning (NLL) with geometric information.
result Reduces input dimension to theoretical lower bound with minor accuracy loss.
Modified relative universality for unbiasedness and consistency in dimension reduction.
problem Gap in proof of unbiasedness and Fisher consistency in relative universality.
method Modified definition of relative universality using ǫ-measurability.
result Established unbiasedness and Fisher consistency rigorously.
Sensor data analysis plays a key role in health assessment of critical equipment. Such data are multivariate and exhibit nonlinear relationships. This paper describes how one can exploit nonlinear dimension reduction techniques, such as the t-distributed stochastic neighbor embedding (t-SNE) and kernel principal compon…
Proposes an online method for high-dimensional streaming data.
problem Increasing variable dimensions with sample size in online kernel sliced inverse regression.
method Introduces approximate linear dependence condition and dictionary variable sets to address the problem. Transforms into online generalized eigen-decomposition problem and uses stochastic optimization for updates.
result Achieves close performance to batch processing kernel sliced inverse regression.
New method for nonlinear SDR of complex non-Euclidean data.
problem Nonlinear SDR for complex non-Euclidean random objects.
method Fréchet Cumulative Covariance (FCCov) and neural networks.
result Robust and unbiased nonlinear SDR for complex data.
Variable selection and dimension reduction are two commonly adopted approaches for high-dimensional data analysis, but have traditionally been treated separately. Here we propose an integrated approach, called sparse gradient learning (SGL), for variable selection and dimension reduction via learning the gradients of t…
R package psvmSDR simplifies SDR computation for machine learning.
problem Sufficient dimension reduction in machine learning.
method Principal machine (PM) generalized from PSVM.
result Efficient computation of SDR estimators in various scenarios.
GDMaps reduces high-dimensional data to lower dimensions for better classification.
problem High-dimensional data classification and representation.
method Grassmannian Diffusion Maps technique for nonlinear dimensionality reduction.
result GDMaps effectively identifies intrinsic subspace structures in high-dimensional data.
Centroid-Encoder reduces high-dimensional data for better visualization.
problem Visualizing high-dimensional data efficiently and accurately.
method Centroid-Encoder integrates label information to keep similar objects close in reduced space.
result Centroid-Encoder outperforms other techniques in visualizing high-dimensional data.
New method deflates manifolds to visualize high-dimensional data.
problem Failure of nonlinear dimensionality reduction methods on simple manifolds.
method Iterative deflation of differential operators using single-coordinate estimates.
result Empirically, recovers novel embeddings on real-world and synthetic datasets.
AEALT uses autoencoders to reduce text embedding dimensions for improved efficiency.
problem High dimensionality of text embeddings hinders downstream tasks.
method Factor-augmented supervised learning with autoencoders.
result AEALT outperforms conventional deep-learning approaches.
MMbeddings reduces categorical embeddings by treating them as latent effects, significantly decreasing parameters and mitigating overfitting.
problem Large cardinalities in categorical embeddings lead to high parameter counts and overfitting.
method MMbeddings treats embeddings as latent random effects in a variational autoencoder framework, reducing parameter count and mitigating overfitting.
result MMbeddings consistently outperforms traditional embeddings across various tasks, demonstrating its potential in machine learning applications.
Mathematical analysis of SNE and t-SNE for dimension reduction.
problem Optimal mapping of high-dimensional data to low dimensions.
method Gradient flow of relative entropy to minimize the distance between points.
result The diameter of the evolving sets remains bounded for SNE but may blow up for t-SNE.
We consider forecasting a single time series using a large number of predictors in the presence of a possible nonlinear forecast function. Assuming that the predictors affect the response through the latent factors, we propose to first conduct factor analysis and then apply sufficient dimension reduction on the estimat…
Bayesian optimization (BO) has been broadly applied to computational expensive problems, but it is still challenging to extend BO to high dimensions. Existing works are usually under strict assumption of an additive or a linear embedding structure for objective functions. This paper directly introduces a supervised dim…
Estimates expected information gain using density approximations and dimension reduction.
problem Estimating expected information gain in nonlinear and non-Gaussian settings.
method Flexible transport-based schemes for EIG estimation, optimal sample allocation, and gradient-based upper bounds on mutual information.
result Optimal sample allocation and dimension reduction schemes improve EIG estimation accuracy and convergence rate.
We introduce a novel data-driven order reduction method for nonlinear control systems, drawing on recent progress in machine learning and statistical dimensionality reduction. The method rests on the assumption that the nonlinear system behaves linearly when lifted into a high (or infinite) dimensional feature space wh…
Paper introduces a nonparametric functional graphical model for random functions.
problem Estimating probabilistic conditional independence in functional graphical models.
method Functional sufficient dimension reduction to relax Gaussian or copula Gaussian assumptions.
result Enhances estimation accuracy and retains probabilistic conditional independence.
Review and compare model order reduction methods for process engineering.
problem Creating computationally efficient yet accurate models for real-time applications.
method Nonlinear model order reduction methods, including general-purpose and tailored approaches for chemical processes.
result Comparison of eight model order reduction methods applied to an air separation process model.
Novel autoencoder method approximates Koopman operator in low dimensions.
problem Challenges in approximating finite Koopman operators using data-driven methods.
method Mori-Zwanzig autoencoder (MZ-AE) for robust Koopman operator approximation.
result Improved predictive capability and robust long-term statistical performance.
SRCA reduces high-dimensional data to lower dimensions while preserving geometric structures.
problem High-dimensional datasets with underlying geometric structures.
method Spherical Rotation Component Analysis (SRCA) incorporating geometric loss functions.
result SRCA provides a low-rank spherical representation of data with general theoretic guarantees.
Multidimensional scaling is an important dimension reduction tool in statistics and machine learning. Yet few theoretical results characterizing its statistical performance exist, not to mention any in high dimensions. By considering a unified framework that includes low, moderate and high dimensions, we study multidim…
We introduce a data-driven order reduction method for nonlinear control systems, drawing on recent progress in machine learning and statistical dimensionality reduction. The method rests on the assumption that the nonlinear system behaves linearly when lifted into a high (or infinite) dimensional feature space where ba…
Nonlinear dimensionality reduction methods are a popular tool for data scientists and researchers to visualize complex, high dimensional data. However, while these methods continue to improve and grow in number, it is often difficult to evaluate the quality of a visualization due to a variety of factors such as lack of…
MEDAL converts manifold embeddings into models for rigorous validation.
problem Challenges in validating manifold embeddings without held-out validation.
method Develops MEDAL framework that distills embeddings into autoencoder models.
result Enables rigorous validation of manifold embeddings and hyperparameters.
Reduces IB problem to a simpler, lower-dimensional problem.
problem Information bottleneck problem in high-dimensional spaces.
method Identifies sufficient statistic that factors conditional distribution, reducing IB to a lower-dimensional problem.
result Preserves full IB curve and optimal representations, making IB tractable.
MSRL learns a representation maximizing mutual info with response variables.
problem Learning sufficient representations for complex, multi-dimensional data.
method Variational mutual information, deep neural networks, generalized Dudley's inequality.
result MSRL achieves consistent and accurate representation learning.
We discuss smooth nonlinear control systems with symmetry. For a free and proper action of the symmetry group, the reduction of symmetry gives rise to a reduced smooth nonlinear control system. If the action of the symmetry group is only proper, the reduced nonlinear control system need not be smooth. Using the smooth …
Uniform proof for ultradifferentiability in various classes and dimensions.
problem Generalizing ultradifferentiability conditions to multidimensional cases and infinite dimensional spaces.
method Uniform proof approach that works in all cases and dimensions, including infinite dimensional Banach spaces and convenient vector spaces.
result Characterization of ultradifferentiability for general analytic germs and functions.
Unified framework for fair representation learning in machine learning.
problem Ensuring fairness in machine learning models, especially when biased data representations lead to unfair predictions.
method Integrates nonlinear sufficient dimension reduction with deep learning to construct fair and informative representations, introducing a penalty term to enforce conditional independence between sensitive attributes and learned representations.
result Achieves a superior balance between fairness and utility, significantly outperforming state-of-the-art baselines on various data structures.
Dimensionality reduction (DR) is often used as a preprocessing step in classification, but usually one first fixes the DR mapping, possibly using label information, and then learns a classifier (a filter approach). Best performance would be obtained by optimizing the classification error jointly over DR mapping and cla…
IKD uses eigen-decomposition for nonlinear dimensionality reduction.
problem Lack of sophisticated and nonlinear dimensionality reduction methods.
method Inverse Kernel Decomposition (IKD) based on eigen-decomposition of sample covariance matrix.
result IKD achieves comparable performance to optimization-based methods with faster running speeds.
CAG method predicts nonlinear solid mechanics responses in real-time with high accuracy and efficiency.
problem Real-time prediction of nonlinear solid mechanics responses.
method Clustering adaptive Gaussian process regression (CAG) method.
result Offers predictions within a second with high precision using only 20 samples.
In this era of data deluge, many signal processing and machine learning tasks are faced with high-dimensional datasets, including images, videos, as well as time series generated from social, commercial and brain network interactions. Their efficient processing calls for dimensionality reduction techniques capable of p…
Develops a nonparametric graphical model for conditional independence.
problem Evaluation of conditional independence without distributional assumptions.
method Nonlinear sufficient dimension reduction techniques applied to a nonparametric graphical model.
result Method outperforms existing methods in non-Gaussian settings and high-dimensional data.