Anomaly detection for high-dimensional data using large deviations principle.
problem Challenges in anomaly detection for high-dimensional data.
method Large Deviations Anomaly Detection (LAD) algorithm.
result Outperforms state-of-the-art methods on high-dimensional data sets.
New method interpolates high-dimensional scattered data using kernel theory.
problem Scattered data in high-dimensional spaces defy traditional distributional assumptions.
method Kernel interpolation framework based on integral operator theory.
result Spectra of kernel matrices predict performance of interpolation methods.
Geometric framework detects outliers in high-dimensional data.
problem Detecting outliers in high-dimensional data.
method Geometric framework exploiting manifold structure.
result Significant improvement in outlier detection in high-dimensional data.
HD-BWDM improves clustering validation in high-dimensional data.
problem Determining the right number of clusters in high-dimensional data.
method HD-BWDM integrates random projection, PCA, trimmed clustering, and medoid-based distances.
result HD-BWDM remains stable and interpretable under high-dimensional projections and contamination.
Study improves Hayashi-Yoshida estimator for high-dimensional stock covolatility.
problem Inconsistent performance of Hayashi-Yoshida estimator in high dimensions.
method Analyzed the limiting spectral distribution of the Hayashi-Yoshida estimator.
result Established the connection between the estimator's spectrum and the true covariance matrix in high dimensions.
Review of privacy-preserving linear models for high-dimensional data.
problem Overfitting and data memorization in high-dimensional linear models.
method Comprehensive comparison of optimization techniques for differentially private high-dimensional linear models.
result Coordinate-optimized algorithms perform best in empirical tests.
New method estimates treatment effects from high dimensional data.
problem Estimating treatment effects from high dimensional data with confounders.
method Generative modeling approach to backdoor adjustment in variational inference.
result Empirically, estimates interventional likelihood in high dimensional settings.
rags2ridges simplifies graphical modeling of high-dimensional data.
problem Graphical modeling of high-dimensional precision matrices.
method Modular framework for extraction, visualization, and analysis of Gaussian graphical models.
result Provides a one-stop-shop for graphical modeling of high-dimensional precision matrices.
Proposes GPLFR for predicting high-dimensional outputs with few data.
problem Predicting high-dimensional outputs from limited data.
method GPLFR combines Gaussian process and linear-Gaussian decoding for high-dimensional prediction.
result GPLFR outperforms existing methods in predicting high-dimensional outputs.
A new method improves graph-based learning for high-dimensional data.
problem Inconsistent high-dimensional learning efficiency of semi-supervised graph regularization.
method Introducing a novel regularization approach involving centering operation.
result Empirical results show improved performance over spectral clustering.
This paper addresses measurement errors in high-dimensional compositional data using a log-contrast model calibration approach.
problem Measurement errors in high-dimensional regression models involving compositional covariates.
method Calibration approach for the linear log-contrast model under lenient sparsity conditions.
result Established asymptotic normality of the estimator for inference.
Novel Bayesian method for high-dimensional count data prediction.
problem Count data in high-dimensional settings requires feature selection.
method Pseudo-Bayesian framework with scaled Student prior and exponential weights.
result Strong performance compared to Lasso in various settings.
GTSNE improves data visualization for high-dimensional data.
problem Visualizing high-dimensional data points in a 2D map.
method GTSNE is a variation of t-SNE that captures both local and macro structures.
result GTSNE produces better visualizations of high-dimensional data compared to other methods.
The paper studies how more data affects prediction risk in high-dimensional models.
problem The impact of increasing data on prediction risk in high-dimensional models.
method Derives central limit theorem and provides finite-sample distribution and confidence interval for prediction risk.
result Demonstrates 'more data hurt' phenomenon in high-dimensional least squares estimation.
Proposes a new method for causal inference in high-dimensional complex data.
problem Challenges in making causal inference with high-dimensional, nonlinear data.
method Combines deep learning techniques like sparse deep learning and stochastic neural networks.
result Outperforms existing methods in numerical studies.
This study evaluates clustering algorithms on high-dimensional data.
problem Comparing clustering algorithms on high-dimensional datasets.
method Evaluation of K-means, DBSCAN, and Spectral Clustering using PCA, t-SNE, UMAP, and multiple metrics.
result UMAP preprocessing improves clustering quality across all algorithms, with Spectral Clustering excelling.
Nonsingular estimation of high dimensional covariance matrices is an important step in many statistical procedures like classification, clustering, variable selection an future extraction. After a review of the essential background material, this paper introduces a technique we call slicing for obtaining a nonsingular …
High dimensional data analysis is known to be as a challenging problem. In this article, we give a theoretical analysis of high dimensional classification of Gaussian data which relies on a geometrical analysis of the error measure. It links a problem of classification with a problem of nonparametric regression. We giv…
In general, the clustering problem is NP-hard, and global optimality cannot be established for non-trivial instances. For high-dimensional data, distance-based methods for clustering or classification face an additional difficulty, the unreliability of distances in very high-dimensional spaces. We propose a distance-ba…
The paper analyzes PLS-SVD in high-dimensional data integration, revealing its strengths and limitations.
problem Understanding the behavior of PLS-SVD in high-dimensional data integration.
method Analysis using random matrix theory and singular value decomposition.
result PLS-SVD exhibits counter-intuitive or limiting behavior in certain regimes and outperforms PCA when detecting common latent subspace.
A new method for analyzing high-dimensional time-series data using deep neural networks.
problem Challenges in modeling high-dimensional time-series data with explicit state and observation processes.
method Deep Direct Discriminative Decoders (D4) for high-dimensional observation processes.
result D4 outperforms traditional SSMs and RNNs in various time-series data applications.
Paper proposes data quality measures for large-scale high-dimensional data.
problem Lack of practical data quality measures for large-scale high-dimensional data.
method Proposes two data quality measures: class separability and in-class variability. Efficient algorithms based on random projections and bootstrapping are provided.
result Efficient algorithms for computing data quality measures on large-scale high-dimensional data.
Generative algorithms learn high-dimensional data efficiently and generate new samples.
problem Learning from scarce high-dimensional data.
method Lipschitz-regularized gradient flows and particle-based algorithms.
result Correctly transports gene expression data points with high dimensionality.
Reproduces IVFS for high-dimensional data structure preservation.
problem Preserving high-dimensional data structure in unsupervised feature selection.
method Inspired by random subset method, IVFS maintains data similarity through topological structure.
result IVFS outperforms SPEC and MCFS on most datasets.
Efficient Bayesian LMM framework for high-dimensional longitudinal data.
problem Scalability and dependence in high-dimensional longitudinal data.
method Partitioned empirical Bayes ECM algorithm for scalable MAP estimation.
result Identification of genes and clinical factors associated with a lupus biomarker.
Modern techniques simplify complex high-dimensional data.
problem Complex, high-dimensional data.
method Unsupervised dimension reduction techniques.
result Simplified representation of high-dimensional data.
For high dimensional data, some of the standard statistical techniques do not work well. So modification or further development of statistical methods are necessary. In this paper, we explore these modifications. We start with the important problem of estimating high dimensional covariance matrix. Then we explore some …
SPPCSO addresses multicollinearity in high-dimensional data, improving model stability and predictive accuracy.
problem Multicollinearity in high-dimensional data leads to unstable estimation and reduced predictive accuracy.
method SPPCSO integrates principal component regression and L1 regularization to adaptively adjust shrinkage factors.
result SPPCSO achieves stable and reliable estimation in high-noise settings, distinguishing signal variables from noise.
New robust method for high-dimensional data analysis in imaging studies.
problem Analyzing high-dimensional data with complex dependence and outliers.
method Robust high-dimensional regression with coefficient thresholding and Huber loss.
result Statistical consistency and computational convergence under high-dimensional settings.
Bayesian tree ensemble model for estimating treatment effects in high-dimensional survival data.
problem Estimating heterogeneous treatment effects in censored survival data with many covariates.
method Developed a Bayesian tree ensemble model with a horseshoe prior for adaptive shrinkage.
result Accurately estimates treatment effects in high-dimensional covariate spaces and non-linear functions.
Density Estimation is one of the central areas of statistics whose purpose is to estimate the probability density function underlying the observed data. It serves as a building block for many tasks in statistical inference, visualization, and machine learning. Density Estimation is widely adopted in the domain of unsup…
Tensor Regression tackles high-dimensional data analysis.
problem Challenges in traditional data representation methods for high-dimensional data.
method Systematic study and analysis of tensor-based regression models.
result Provides solutions for specific regression tasks with multiway data.
Paper proposes new density estimators for high-dimensional data.
problem Prohibitive computational cost and slow convergence rate in high-dimensional density estimation.
method Adaptive hyperbolic cross density estimators in mixed smooth Sobolev spaces.
result Proposed estimators do not suffer curse of dimensionality under Integral Probability Metrics.
FSGD uses latent factors to scale SGD for high-dimensional learning.
problem Scalable optimization in high-dimensional machine learning.
method Factor-Augmented SGD (FSGD) that operates on streaming data.
result Established theoretical framework for latent factor estimation error in SGD.
Overview of high-dimensional time series regression methods.
problem Estimation and inference with high-dimensional time series data.
method Limit theory for high-dimensional dependent data, asymptotic theory for time series regression, statistical learning methods.
result Main limit theory results and asymptotic theory for high-dimensional time series regression.
A new method reduces high-dimensional data's impact on CWMs using TSNE.
problem High-dimensional data hampers CWMs' accuracy and speed.
method TSNE for dimensionality reduction, parsimonious technique, expectation maximization.
result TSNE enhances CWMs' performance in high-dimensional space.
Proposes Causal-Batle for estimating treatment effects in small high-dimensional datasets.
problem Estimating treatment effects with small high-dimensional datasets.
method Adopts transfer learning techniques for causal inference.
result Improves treatment effect estimates in small high-dimensional datasets.
A framework infers feature importance with uncertainties for high-dimensional data.
problem Estimating feature importance in high-dimensional data with uncertainty.
method Shapley value based framework, sub-SAGE, bootstrapping.
result Uncertainties in feature importance can be estimated from bootstrapping.
Prevalidated ridge regression simplifies logistic regression for high-dimensional data.
problem Efficient probabilistic classification in high-dimensional data with logistic regression.
method Developed a prevalidated ridge regression model that matches logistic regression's performance but is more computationally efficient.
result Prevalidated ridge regression achieves similar classification error and log-loss to logistic regression for high-dimensional data.
Paper proposes sparse classification method for high-dimensional data.
problem Sparse classification in high-dimensional data with positive-confidence samples.
method Developed a novel sparse-penalization framework using L1, SCAD, and MCP penalties for convex and non-convex shrinkage.
result Proved near minimax-optimal sparse recovery rates under Restricted Strong Convexity condition.
New methods integrate nonlinear, sparse, and multi-view aspects for high-dimensional data analysis.
problem Integrating nonlinear dependence, sparsity, and multi-view data in high-dimensional datasets.
method Proposes HSIC-SGCCA, SA-KGCCA, and TS-KGCCA methods for multi-view high-dimensional data analysis.
result HSIC-SGCCA outperforms competing methods in multi-view variable selection.
A neural network model tackles high-dimensional data with latent structures.
problem Modeling high-dimensional data with latent low-dimensional structures.
method Integrates PCA and Soft PCA layers into neural network architecture for factor modeling and non-linear transformations.
result Demonstrates improved performance in forecasting and nowcasting with real-world data.
Classification is an important topic in statistics and machine learning with great potential in many real applications. In this paper, we investigate two popular large margin classification methods, Support Vector Machine (SVM) and Distance Weighted Discrimination (DWD), under two contexts: the high-dimensional, low-sa…
Mapper and Ball Mapper tools for complex data analysis.
problem Exploring and visualizing high-dimensional data and scalar functions.
method Combining Mapper and Ball Mapper, adding new features for encoding structure and symmetries.
result A new hybrid algorithm, Mapper on Ball Mapper, for comparing high-dimensional data descriptors.
A new method identifies critical transitions in high-dimensional data.
problem Challenges in identifying critical transitions in high-dimensional time-series data.
method Spatial-temporal Principal Component Analysis (stPCA)
result Identifies tipping points before critical transitions reliably.
High-dimensional data poses unique challenges in outlier detection process. Most of the existing algorithms fail to properly address the issues stemming from a large number of features. In particular, outlier detection algorithms perform poorly on data set of small size with a large number of features. In this paper, w…
A new method computes high-dimensional optimal transport using flow neural networks.
problem Computing optimal transport for high-dimensional data.
method Optimizing a flow model to minimize transport cost between two arbitrary distributions.
result Trained optimal transport flow enables downstream tasks like DRE and domain adaptation.
Skeleton clustering detects clusters in high-dimensional data without needing prototypes.
problem Detecting clusters in high-dimensional data with irregular shapes.
method Skeleton clustering combines prototype methods, density-based clustering, and hierarchical clustering using surrogate density measures.
result Skeleton clustering reliably detects clusters in multivariate and high-dimensional data.