arXiv research

New method for interpreting non-linear models using forward marginal effects.

We tackle linear bandits with partially observable features, achieving sublinear regret.

We simplify complex regression coefficients using linearization and feature comparison.

Study proposes Local Linear Encoding for better feature discretization.

New method distinguishes feature relevance in non-linear contexts.

This paper explores a new form of the linear bandit problem in which the algorithm receives the usual stochastic rewards as well as stochastic feedback about which features are relevant to the rewards, the latter feedback being the novel aspect. The focus of this paper is the development of new theory and algorithms fo…

Analyzes neural networks using linear models to understand their behavior.

Pruning method removes less important features in linear models.

Gradient descent with growing learning rate enables learning non-linear features in neural networks.

This paper describes the autofeat Python library, which provides scikit-learn style linear regression and classification models with automated feature engineering and selection capabilities. Complex non-linear machine learning models, such as neural networks, are in practice often difficult to train and even harder to …

RFM reduces feature space for linear models, improving sparse recovery.

LLT transforms time series features based on linear laws.

In this paper we consider a problem of searching a space of predictive models for a given training data set. We propose an iterative procedure for deriving a sequence of improving models and a corresponding sequence of sets of non-linear features on the original input space. After a finite number of iterations N, the n…

Two new algorithms reduce feature space while preserving non-linear relationships.

In this paper, we propose and study random maxout features, which are constructed by first projecting the input data onto sets of randomly generated vectors with Gaussian elements, and then outputing the maximum projection value for each set. We show that the resulting random feature map, when used in conjunction with …

TSRGA scales multivariate linear regression for feature-distributed data.

A new estimator learns sparse linear models with context-dependent coefficients.

Study agnostic feature-based dynamic pricing models with linear policies and noisy valuations.

New algorithm disentangles latent features without strict assumptions.

OOMP selects features online for sparse linear regression.

Complex problems may require sophisticated, non-linear learning methods such as kernel machines or deep neural networks to achieve state of the art prediction accuracies. However, high prediction accuracies are not the only objective to consider when solving problems using machine learning. Instead, particular scientif…

Study online linear regression with paid noise reduction.

This paper concerns a method of selecting a subset of features for a sequential logit model. Tanaka and Nakagawa (2014) proposed a mixed integer quadratic optimization formulation for solving the problem based on a quadratic approximation of the logistic loss function. However, since there is a significant gap between …

ERM performs well in feature learning with minimal feature maps.

Paper analyzes TD($λ$) convergence rates for arbitrary features.

Supervised linear feature extraction can be achieved by fitting a reduced rank multivariate model. This paper studies rank penalized and rank constrained vector generalized linear models. From the perspective of thresholding rules, we build a framework for fitting singular value penalized models and use it for feature …

This paper makes kernels interpretable for wide feature matrices.

Fast classification for sparse models, even with correlated features.

Analyzes bias-variance in overparameterized linear models using random features.

Removing spurious features can hurt model accuracy and disproportionately affect different groups.

The generalized partially linear additive model (GPLAM) is a flexible and interpretable approach to building predictive models. It combines features in an additive manner, allowing each to have either a linear or nonlinear effect on the response. However, the choice of which features to treat as linear or nonlinear is …

A method to automatically choose feature dimensions in linear attention for better approximation quality.

This paper deals with the problem of large-scale linear supervised learning in settings where a large number of continuous features are available. We propose to combine the well-known trick of one-hot encoding of continuous features with a new penalization called \emph{binarsity}. In each group of binary features comin…

Feature selection problems have been extensively studied for linear estimation, for instance, Lasso, but less emphasis has been placed on feature selection for non-linear functions. In this study, we propose a method for feature selection in high-dimensional non-linear function estimation problems. The new procedure is…

Selecting important features in non-linear or kernel spaces is a difficult challenge in both classification and regression problems. When many of the features are irrelevant, kernel methods such as the support vector machine and kernel ridge regression can sometimes perform poorly. We propose weighting the features wit…

LIT-LVM improves linear predictors by estimating interaction terms with latent vectors.

The paper improves prediction and testing for signals from a linear combination of translated features with Gaussian noise.

Unified derivation of high-dimensional linear models using stochastic gradient descent.

For various applications, the relations between the dependent and independent variables are highly nonlinear. Consequently, for large scale complex problems, neural networks and regression trees are commonly preferred over linear models such as Lasso. This work proposes learning the feature nonlinearities by binning fe…

Theory and method for reducing prediction variance in noisy feature-subsampled ridge ensembles.

We propose a novel algorithm for greedy forward feature selection for regularized least-squares (RLS) regression and classification, also known as the least-squares support vector machine or ridge regression. The algorithm, which we call greedy RLS, starts from the empty feature set, and on each iteration adds the feat…

LUNA improves linear attention for long sequences without sacrificing accuracy.

The goal of supervised feature selection is to find a subset of input features that are responsible for predicting output values. The least absolute shrinkage and selection operator (Lasso) allows computationally efficient feature selection based on linear dependency between input features and output values. In this pa…

This work interprets SFA through variational inference, relaxing linearity constraints.

High-dimensional data in many areas such as computer vision and machine learning tasks brings in computational and analytical difficulty. Feature selection which selects a subset from observed features is a widely used approach for improving performance and effectiveness of machine learning models with high-dimensional…

Distributed sensors compress and send features to a fusion center for linear regression.

A framework to compare atomistic descriptors and their transformations.

Much work has been done recently to make neural networks more interpretable, and one obvious approach is to arrange for the network to use only a subset of the available features. In linear models, Lasso (or $\ell_1$-regularized) regression assigns zero weights to the most irrelevant or redundant features, and is widel…