Develops an efficient method for real-time data analysis and visualization.
problem Challenges of analyzing high-dimensional data.
method Incremental non-linear manifold approximation using GMRA framework.
result Accurately represents non-linear manifolds with small initial samples.
Review of privacy-preserving linear models for high-dimensional data.
problem Overfitting and data memorization in high-dimensional linear models.
method Comprehensive comparison of optimization techniques for differentially private high-dimensional linear models.
result Coordinate-optimized algorithms perform best in empirical tests.
New methods combine low and high-fidelity data for accurate surrogate modeling.
problem Challenges in surrogate modeling for high-dimensional outputs with limited training data.
method Projection-based multifidelity linear regression methods integrating low-fidelity and high-fidelity data.
result Multifidelity methods achieve up to 12% improvement in median accuracy compared to single-fidelity methods.
This paper investigates how data augmentation improves linear separation of manifold data.
problem Understanding how data augmentation enhances linear separation of manifold data.
method Investigates the conditions under which self-supervised representations can linearly separate multi-manifold data.
result Self-supervised learning can linearly separate manifolds with a smaller distance than unsupervised learning.
Develops estimators for near-optimal linear regression under distribution shift.
problem Linear regression under distribution shift with scarce target domain data.
method Minimax linear risk estimators covering various transfer learning settings.
result Achieves near-optimal risk for linear regression problems under distribution shift.
Proposes a method for differentially private linear regression and synthetic data generation.
problem Lack of valid inference and synthetic data generation methods for small-scale datasets in privacy-aware settings.
method Gaussian differentially private linear regression with bias-corrected estimator and SDG procedure.
result Improves accuracy and provides valid confidence intervals for downstream tasks.
We identify linear models from nonlinear systems with initialization constraints.
problem Identifying linear models from nonlinear systems with initialization constraints.
method Multiple trajectories-based deterministic data acquisition algorithm followed by regularized least squares.
result We provide a finite sample error bound on the learned linearized dynamics.
Paper proves linear convergence of SCMS algorithm for directional data.
problem Identifying density ridges in directional data.
method Generalized SCMS algorithm to directional data, derived from SCGA with adaptive step size.
result Linear convergence of the proposed directional SCMS algorithm.
We learn linear models from nonlinear systems using multiple trajectories and regularization.
problem Identifying linear models from data when the underlying dynamics are nonlinear.
method Multiple trajectories data acquisition followed by regularized least squares.
result Learn linearized dynamics with arbitrarily small error given enough samples.
DDGroup identifies subgroups with uniform linear relationships.
problem Heterogeneous effects of covariates in linear models.
method Data-driven method to identify subgroups with uniform linear relationships.
result DDGroup can discover subgroups with improved performance.
Synthetic data can amplify privacy in linear regression models.
problem Understanding how synthetic data can enhance privacy in linear regression models.
method Investigated through the linear regression framework, analyzing synthetic data generated from random inputs and controlled inputs.
result Releasing a limited number of synthetic data points amplifies privacy beyond the model's inherent guarantees when inputs are random, but not when inputs are controlled by an adversary.
We simplify complex regression coefficients using linearization and feature comparison.
problem Interpreting high-dimensional regression coefficients from nonlinear responses.
method Developed a linearization method to derive feature coefficients and compare them with regression coefficients.
result Shows how regression coefficients relate to linearized feature coefficients and how they change under regularization.
Paper introduces data-dependent SSP for private linear and logistic regression.
problem Private linear and logistic regression with better performance.
method Data-dependent sufficient statistic perturbation (SSP) for linear and logistic regression.
result Data-dependent SSP outperforms state-of-the-art methods for linear and logistic regression.
The network Lasso (nLasso) has been proposed recently as an efficient learning algorithm for massive networked data sets (big data over networks). It extends the well-known least absolute shrinkage and selection operator (Lasso) from learning sparse (generalized) linear models to network models. Efficient implementatio…
Improved algorithm for conditional linear regression with heterogeneous covariances.
problem Identifying a linear predictor for a fraction of data with varying covariances.
method Polynomial time algorithm using Disjunctive Normal Form (DNF) to identify a condition and linear predictor.
result Removed requirement for similar covariances in each condition term, improving algorithm applicability.
Linear classifiers in product space forms improve scRNA-seq data classification.
problem Linear classification in products of Euclidean, spherical, and hyperbolic spaces.
method Novel formulations of linear classifiers on Riemannian manifolds, proving expressive power, and formalizing perceptron and SVM classifiers.
result Linear classifiers in product space forms have the same expressive power as in Euclidean space of the same dimension.
diproperm tests differences in HDLSS data with binary classifiers.
problem Testing differences in HDLSS data with binary classifiers.
method DiProPerm test for binary linear classifiers.
result Validates the DiProPerm test on real-world data.
SSL framework identifies non-linear systems without labeled data.
problem System identification in non-linear environments without labeled data.
method Dynamics contrastive learning framework.
result SSL can identify non-linear dynamics in latent space.
New insights into optimization and generalization for linear models.
problem Understanding the implicit regularization of optimization methods for linear models.
method Investigating the norms minimized by interpolating solutions and using projections to move between solutions.
result Proving that for over-parameterized linear classification, projections onto the data-span enable the use of under-parameterized techniques.
BELIEF framework interprets GLMs using binary linear models.
problem Understanding and interpreting generalized linear models (GLMs) with binary outcomes.
method Developed a framework called binary expansion linear effect (BELIEF) to interpret GLMs through transparent linear models.
result BELIEF framework reveals perfect predictors in complete separation scenarios.
Estimates the upper bound of linear regions in spheres centered at specific data points in ReLU neural networks.
problem Bounding the number of linear regions in specific areas of neural networks using ReLU activations.
method Developed a method to estimate the upper bound of linear regions in any sphere within the input space of a ReLU neural network.
result The boundaries of linear regions move away from training data points during training, and spheres centered at these points contain more regions than arbitrary points.
Bayesian approach learns linear operators from noisy data.
problem Learning linear operators from noisy data in infinite-dimensional spaces.
method Bayesian approach with Gaussian priors.
result Establishes posterior contraction rates and generalization error guarantees.
Optimal data split ratio is sqrt(p):1 for linear regression.
problem Lack of clear guidance on optimal training/testing data split ratio.
method Showed that optimal ratio is sqrt(p):1 for linear regression.
result Optimal ratio for training/testing split is sqrt(p):1.
Matrix completion aims to predict missing elements in a partially observed data matrix which in typical applications, such as collaborative filtering, is large and extremely sparsely observed. A standard solution is matrix factorization, which predicts unobserved entries as linear combinations of latent variables. We g…
Unified strategy for efficient data compression and model estimation.
problem Limited interactive exploration and data interaction in linear model development and deployment.
method Conditionally sufficient statistics for optimal data compression and estimation of linear models.
result Linear models can be estimated from compressed data without loss of parameters or covariances.
Paper investigates optimal interpolation methods in linear regression.
problem Understanding when interpolating methods generalize well in linear regression.
method Investigates optimal response-linear interpolators using functions linear in the response variable.
result Provides a closed-form expression for the optimal interpolator and shows it can be derived as the limit of gradient descent.
The importance of interpretability of machine learning models has been increasing due to emerging enterprise predictive analytics, threat of data privacy, accountability of artificial intelligence in society, and so on. Piecewise linear models have been actively studied to achieve both accuracy and interpretability. Th…
New algorithm combines new and historical data with different input dimensions for linear regression.
problem Combining new and historical data with different input dimensions for improved accuracy.
method Proposes a transfer learning algorithm with rigorous theoretical robustness analysis.
result Achieves state-of-the-art performance on 9 real-life datasets.
Study linear contextual bandits with confounded offline data, improving regret bounds.
problem Linear contextual bandits with confounded offline data.
method Construct a linear bandit algorithm that utilizes projected information.
result Proved regret bounds that improve current bounds by a factor related to visible dimensionality.
New bounds for transfer learning in linear models, improving generalization.
problem Understanding when auxiliary data helps in improving generalization in linear models.
method Derivation of exact error bounds and optimal task weights for linear regression and linear neural networks.
result First non-vacuous sufficient conditions for beneficial auxiliary learning in linear neural networks.
A new method for LDA using randomized Kaczmarz improves accuracy for large datasets.
problem Efficiently performing LDA on large datasets.
method Randomized Kaczmarz method applied to linear discriminant analysis.
result The method achieves comparable accuracy to full data LDA.
Theoretical comparison of three invariance approaches in deep linear networks.
problem Understanding invariance in deep linear networks.
method Data augmentation, regularization, and hard-wiring approaches.
result Regularization introduces additional critical points, but they remain saddles except for the global optimum.
Proposes a method to calibrate data for more accurate linear correlation testing.
problem Inaccurate Pearson's correlation coefficient due to sample size and data non-normality.
method Predictive data calibration using machine learning to condition data on expected linear relationship.
result Calibrated Pearson's correlation coefficient yields a calibrated p-value and r estimate for posterior probability interpretation.
Develops a Riemannian archetypal analysis for interpretable non-linear data.
problem Limited performance of classical archetypal analysis on non-linear data.
method Riemannian geometry for data-driven pullback, geodesic convex combinations, convex relaxation followed by non-convex refinement.
result Combines interpretability of classical archetypal analysis with expressive power of modern non-linear models.
Causality-aware methods outperform linear residualization in confounding adjustment for anticausal prediction.
problem Adjusting for confounding in anticausal prediction tasks.
method Causality-aware counterfactual confounding adjustment.
result Causality-aware methods asymptotically outperform linear residualization in predictive performance.
This paper studies the addition of linear constraints to the Support Vector Regression (SVR) when the kernel is linear. Adding those constraints into the problem allows to add prior knowledge on the estimator obtained, such as finding probability vector or monotone data. We propose a generalization of the Sequential Mi…
Linear classification has been widely used in many high-dimensional applications like text classification. To perform linear classification for large-scale tasks, we often need to design distributed learning methods on a cluster of multiple machines. In this paper, we propose a new distributed learning method, called f…
TSRGA scales multivariate linear regression for feature-distributed data.
problem Multivariate linear regression for feature-distributed data with high dimensions and many computing nodes.
method Two-stage relaxed greedy algorithm (TSRGA) for multivariate linear regression.
result TSRGA is highly scalable and can yield low-rank coefficient estimates.
Adam optimizes linear classifiers with separable data.
problem Understanding Adam's implicit bias in linear logistic regression.
method Study of Adam's behavior on linearly separable data.
result Adam converges to a linear classifier with maximum ℓ∞-margin. Neural networks improve geospatial data analysis by relaxing linearity assumptions.
problem Traditional geospatial analysis assumes linear models, limiting flexibility.
method Embedding neural networks within traditional geostatistical models for non-linear mean functions.
result NN-GLS algorithm provides consistent and scalable predictions for irregular spatial data.
Flexible DNN for survival data, avoiding proportional hazards assumption.
problem Survival analysis with complex interactions and non-proportional hazards.
method Partially linear DNN model with a flexible nonparametric component.
result FLEXI-Haz achieves optimal convergence rates and asymptotic efficiency.
Linear dimensionality reduction methods are a cornerstone of analyzing high dimensional data, due to their simple geometric interpretations and typically attractive computational properties. These methods capture many data features of interest, such as covariance, dynamical structure, correlation between data sets, inp…
The MEM method uses data-driven priors for linear inverse problems, proving convergence and estimating differences.
problem Linear inverse problems with approximate priors.
method Maximum Entropy on the Mean (MEM) method with data-driven priors.
result Empirical mean convergence and estimates for prior differences based on epigraphical distance.
Estimates causal effects in Gaussian Linear SCMs with finite data.
problem Estimating causal effects from observational data with latent confounders.
method Centralized Gaussian Linear SCMs (CGL-SCMs) and EM-based estimation algorithm.
result Learned CGL-SCM parameters accurately recover causal distributions from finite observational samples.
Study reveals how manifold geometry impacts linear regression solutions.
problem Impact of manifold geometry on linear regression solutions.
method Linear regression applied to manifold-structured data, focusing on extrinsic geometry.
result Linear regression does not have a unique solution on flat manifolds.
PILOT is a fast algorithm for linear model trees that outperforms existing methods.
problem Fitting linear model trees to large datasets efficiently and accurately.
method Greedy training with L2 boosting and model selection rule. result PILOT outperforms standard decision trees and other linear model trees on various datasets.
New neural network models for functional data.
problem Handling non-linear functional data.
method Functional Direct Neural Network (FDNN) and Functional Basis Neural Network (FBNN) with gradient-based optimization.
result Demonstrated effectiveness in complex functional models.
The paper shows how data and algorithm interactions affect overparameterized linear regression generalization.
problem Understanding generalization in overparameterized linear regression.
method Introducing data-algorithm compatibility and performing data-dependent trajectory analysis with gradient descent.
result Early stopping iterates lead to better generalization than last-iterate analysis, with weaker restrictions.