arXiv research

Random feature matrices' singular values concentrate near their full expectation in high dimensions.

t-SNE loses important features in data visualization.

Develops a new method for nonlinear dimension reduction using random features.

PSMM method optimizes matrix sufficient dimension reduction.

Study reduces dimensions for $k$-means clustering for better accuracy.

A method to automatically choose feature dimensions in linear attention for better approximation quality.

The study analyzes decision trees on real and categorical features, deriving bounds on their VC dimension and proposing improved pruning algorithms.

A new tree-based model for varying coefficients using CGBM.

We find a deterministic equivalent for random feature regression's test error, independent of feature map dimension.

Paper proposes an unsupervised feature selection algorithm with stability guarantees.

Estimates mean dimension of neural networks to reveal interaction effects.

A new tensor-based layer reduces neural network dimensions without losing important features.

Scientific explanation often requires inferring maximally predictive features from a given data set. Unfortunately, the collection of minimal maximally predictive features for most stochastic processes is uncountably infinite. In such cases, one compromises and instead seeks nearly maximally predictive features. Here, …

Introduces greedy feature selection for classifier-dependent feature ranking.

This paper deals with a new filter algorithm for selecting the smallest subset of features carrying all the information content of a data set (i.e. for removing redundant features). It is an advanced version of the fractal dimension reduction technique, and it relies on the recently introduced Morisita estimator of Int…

Supervised linear feature extraction can be achieved by fitting a reduced rank multivariate model. This paper studies rank penalized and rank constrained vector generalized linear models. From the perspective of thresholding rules, we build a framework for fitting singular value penalized models and use it for feature …

Max-margin classifiers' behavior is studied in high dimensions with non-Gaussian features.

In this paper, a novel learning paradigm is presented to automatically identify groups of informative and correlated features from very high dimensions. Specifically, we explicitly incorporate correlation measures as constraints and then propose an efficient embedded feature selection method using recently developed cu…

Recommendation problems with large numbers of discrete items, such as products, webpages, or videos, are ubiquitous in the technology industry. Deep neural networks are being increasingly used for these recommendation problems. These models use embeddings to represent discrete items as continuous vectors, and the vocab…

Dimensionality reduction (DR) on the manifold includes effective methods which project the data from an implicit relational space onto a vectorial space. Regardless of the achievements in this area, these algorithms suffer from the lack of interpretation of the projection dimensions. Therefore, it is often difficult to…

In this paper, we propose a test, called Flagged-1-Bit (F1B) test, to study the intrinsic capability of recurrent neural networks in sequence learning. Four different recurrent network models are studied both analytically and experimentally using this test. Our results suggest that in general there exists a conflict be…

Unsupervised dimension selection is an important problem that seeks to reduce dimensionality of data, while preserving the most useful characteristics. While dimensionality reduction is commonly utilized to construct low-dimensional embeddings, they produce feature spaces that are hard to interpret. Further, in applica…

Feature selection is an important challenge in machine learning. It plays a crucial role in the explainability of machine-driven decisions that are rapidly permeating throughout modern society. Unfortunately, the explosion in the size and dimensionality of real-world datasets poses a severe challenge to standard featur…

Scalable feature selection improves GBDT model training speed.

The asymptotic dimension theory was founded by Gromov in the early 90s. In this paper we give a survey of its recent history where we emphasize two of its features: an analogy with the dimension theory of compact metric spaces and applications to the theory of discrete groups.

TrIM improves gradient-based dimension reduction and regression.

We consider the problem of clustering data points in high dimensions, i.e. when the number of data points may be much smaller than the number of dimensions. Specifically, we consider a Gaussian mixture model (GMM) with non-spherical Gaussian components, where the clusters are distinguished by only a few relevant dimens…

New bounds for high-dimensional sparse linear bandits, balancing information and regret.

Semi-supervised learning improves classification in high dimensions.

Kernel methods are powerful and flexible approach to solve many problems in machine learning. Due to the pairwise evaluations in kernel methods, the complexity of kernel computation grows as the data size increases; thus the applicability of kernel methods is limited for large scale datasets. Random Fourier Features (R…

Novel approach uses neural networks to enhance CI testing for feature selection.

Data acquisition, storage and management have been improved, while the key factors of many phenomena are not well known. Consequently, irrelevant and redundant features artificially increase the size of datasets, which complicates learning tasks, such as regression. To address this problem, feature selection methods ha…

We prove that many features of Thurston's Dehn surgery theory for hyperbolic 3-manifolds generalize to Einstein metrics in any dimension. In particular, this gives large, infinite families of new Einstein metrics on compact manifolds.

The paper compares Bayesian uncertainty to MAP estimator in random features regression.

Paper proposes a tensor data model for incomplete imaging data.

Generalization in nonlinear least squares can be studied via algorithmic stability and effective dimension.

High-dimensional data in many machine learning applications leads to computational and analytical complexities. Feature selection provides an effective way for solving these problems by removing irrelevant and redundant features, thus reducing model complexity and improving accuracy and generalization capability of the…

Adversarial robustness in multi-index models is as easy as standard learning.

Study shows how diffusion models learn on low-dimensional manifolds.

We develop an approach for feature elimination in statistical learning with kernel machines, based on recursive elimination of features.We present theoretical properties of this method and show that it is uniformly consistent in finding the correct feature space under certain generalized assumptions.We present four cas…

Generative adversarial networks benefit from optimal input dimension and adaptive generator architecture.

Improved uniform convergence bound with fat-shattering dimension reduces sample complexity gap.

Learning a distribution conditional on a set of discrete-valued features is a commonly encountered task. This becomes more challenging with a high-dimensional feature set when there is the possibility of interaction between the features. In addition, many frequently applied techniques consider only prediction of the me…

Efficient learning of minimax risk classifiers in high dimensions.

This paper proposes a multi-view learning framework for dimension reduction.

DIVI clusters noisy high-dimensional data with stable feature gating.

Single Index Models (SIMs) are simple yet flexible semi-parametric models for classification and regression. Response variables are modeled as a nonlinear, monotonic function of a linear combination of features. Estimation in this context requires learning both the feature weights, and the nonlinear function. While met…

New methods interpret clustering outcomes without altering data structure.