The paper tackles identifying a small segment of a population with sparse linear regression.
problem Identifying a small segment of a population with a sparse linear regression fit.
method Algorithms for joint identification of a significant segment of a population with a sparse linear regression fit under the sup norm, using k-DNF conditions and s-sparse regression fits.
result Preliminary algorithms and challenges for future work in non-sparse regression and expected error.
Kernel Multigrid accelerates Back-fitting for additive Gaussian Processes.
problem Slow convergence of Back-fitting in training additive Gaussian Processes.
method Kernel Packets (KP) and Sparse Gaussian Process Regression (GPR) to enhance Back-fitting.
result Kernel Multigrid reduces the required iterations to O(logn). Standard sparse pseudo-input approximations to the Gaussian process (GP) cannot handle complex functions well. Sparse spectrum alternatives attempt to answer this but are known to over-fit. We suggest the use of variational inference for the sparse spectrum approximation to avoid both issues. We model the covariance fu…
Efficiently finds sparse solutions to max-plus equations for convex regression.
problem Finding sparse solutions to max-plus equations for convex multivariate regression.
method Polynomial-time algorithm for sparse approximate solutions.
result Optimal piecewise-linear fitting with minimum number of regions.
A new distributed algorithm for fitting sparse additive models with feature division and decorrelation.
problem Fitting high-dimensional sparse additive models efficiently and accurately.
method Divide, decorrelate, and conquer approach.
result Effective and efficient recovery of sparsity patterns and statistical inference for each component.
New method fits sparse Markov models to categorical time series using convex clustering.
problem Exponentially growing parameters in higher-order Markov chains.
method Convex clustering and regularization for parsimonious modeling.
result Theoretical consistency and finite sample performance demonstrated.
SAEs struggle with curved activation manifolds, revealing layer-dependent scaling laws.
problem Sparse autoencoders' reconstruction error varies across layers, not fitting existing scaling laws.
method Cross-layer study of 844 SAE checkpoints, fitting and regressing on manifold geometry.
result Manifold geometry predicts layer-dependent width exponents in SAEs, with transferable coefficients.
The paper tackles sparse model fitting in distributed machine learning with graph-structured data.
problem Sparse model fitting across a distributed collection of heterogeneous data sets.
method Basis Pursuit Denoising with a total variation penalty, using ADMM for distributed methods.
result Recovery is successful with fewer samples than solving problems independently, or using methods with large overlap in signal supports.
New rules reduce SLOPE model fitting time by screening out irrelevant variables.
problem Expensive tuning of regularization parameter in penalized regression models.
method Strong screening rules for group-based SLOPE models.
result Significant acceleration of fitting process for Group SLOPE and sparse-group SLOPE.
New estimators improve sparse semiparametric additive modeling.
problem Sparse semiparametric additive modeling with structured sparsity.
method Combines group subset selection with shrinkage for nonconvex optimization.
result New estimators outperform alternatives in synthetic and real-world data.
A new framework DECO for distributed sparse regression reduces model dimensionality and improves accuracy.
problem Sparse regression challenges in high-dimensional datasets.
method DECO framework for feature space partitioning, decorrelating features, and distributed computation.
result DECO achieves consistent variable selection and parameter estimation with nearly optimal convergence rate.
Fitting high-dimensional data involves a delicate tradeoff between faithful representation and the use of sparse models. Too often, sparsity assumptions on the fitted model are too restrictive to provide a faithful representation of the observed data. In this paper, we present a novel framework incorporating sparsity i…
Many algorithms have been proposed for fitting network models with communities, but most of them do not scale well to large networks, and often fail on sparse networks. Here we propose a new fast pseudo-likelihood method for fitting the stochastic block model for networks, as well as a variant that allows for an arbitr…
Proposes an optimization framework for sparse robust subspace estimation.
problem Sparse robust one-dimensional subspace estimation.
method l1-norm regularization, linear relaxation, simple ratios, sorting techniques.
result Achieves global optimality for sparse robust subspace with polynomial time efficiency.
We introduce a new algorithm, called adaptive sparse backfitting algorithm, for solving high dimensional Sparse Additive Model (SpAM) utilizing symmetric, non-negative definite smoothers. Unlike the previous sparse backfitting algorithm, our method is essentially a block coordinate descent algorithm that guarantees to …
Sparse feature selection improves batch RL efficiency.
problem High-dimensional batch RL with many features.
method Sparse linear function approximation, Lasso, group Lasso, fitted Q-evaluation, fitted Q-iteration.
result Sparse feature selection makes batch RL more sample efficient.
Sparse linear models improve neural network debuggability.
problem Improving neural network interpretability and debugging.
method Using sparse linear models over learned deep feature representations.
result The approach leads to more debuggable and accurate neural networks.
New sparse GP model learns compositional kernels efficiently.
problem Learning accurate Gaussian Process models with complex kernel structures.
method MultiSVGP model with Horseshoe prior for kernel selection.
result Our model provides better fit and faster computation for large-scale data.
The power of sparse signal modeling with learned over-complete dictionaries has been demonstrated in a variety of applications and fields, from signal processing to statistical inference and machine learning. However, the statistical properties of these models, such as under-fitting or over-fitting given sets of data, …
New method combines spectral and sparse methods for Gaussian processes.
problem Efficiently fitting Gaussian processes to large datasets.
method Orthogonally decoupled variational Fourier features.
result Competitive performance on synthetic and real-world data.
A new estimator learns sparse linear models with context-dependent coefficients.
problem Sparse linear models lack flexibility compared to deep neural networks for handling feature groups.
method Contextual lasso estimator using a deep neural network with lasso regularization.
result Learned models can be sparser than standard lasso without sacrificing predictive power.
Gaussian Processes improve data interpolation from diverse experiments.
problem Interpolation of sparse and inconsistent datasets from various experiments.
method Used Gaussian Processes (GP) for data interpolation, including uncertainty quantification.
result GPs successfully interpolate data and quantify uncertainties, demonstrating consistency across different sources.
WamOL uses PINNs to efficiently calibrate IVS from sparse data.
problem Calibrating time-dependent IVS from sparse market data.
method Physics-Informed Neural Networks (PINNs) with adaptive reweighting.
result WamOL outperforms in calibrating intraday IVS from uneven data.
There has been a lot of work fitting Ising models to multivariate binary data in order to understand the conditional dependency relationships between the variables. However, additional covariates are frequently recorded together with the binary data, and may influence the dependence relationships. Motivated by such a d…
The paper studies efficient Hessian fitting methods for stochastic optimization.
problem Efficient Hessian fitting for stochastic optimization.
method Preconditioned Stochastic Gradient Descent (PSGD) method and Lie groups.
result Hessian fitting problem is strongly convex in certain Lie groups.
Regularized MLE for MoE models tackles high-dimensional heterogeneous data.
problem Fitting and feature selection in Mixtures-of-Experts models for high-dimensional data.
method Proposes a regularized maximum likelihood estimation approach with hybrid EM/MM algorithms.
result Automatic recovery of sparse solutions without thresholding and matrix inversion.
New method estimates multivariate Gaussian fields using sparse precision matrix.
problem Estimating covariance matrices for large multivariate Gaussian fields.
method Sparse Precision Matrix Selection (SPS) algorithm for multivariate GRFs.
result Theoretical rates of convergence for estimated covariance and parameters validated.
Sparse matrices simplify computation of GP variances and likelihoods.
problem Efficient computation of posterior variance and log-likelihood for additive Matérn GPs.
method Represented posterior mean, variance, log-likelihood, and gradient using sparse matrices.
result Efficient computation of posterior mean, variance, log-likelihood, and gradient in O(nlogn) time. New method estimates travel times more accurately with fewer components.
problem Dynamic number of travel time modes and skewed distributions.
method Gamma mixture densities with sparse estimation and recursive algorithm.
result Improved travel time estimation with fewer components and better fitting accuracy.
Sparse-input neural networks handle high-dimensional data with fewer features.
problem Neural networks struggle with high-dimensional data where input features exceed observations.
method Sparse group lasso penalty on first-layer input weights.
result Sparse-input neural networks achieve better performance than existing methods in high-dimensional data with complex interactions.
The paper offers streamlined algorithms for fitting complex linear mixed models.
problem Linear mixed models with crossed random effects in large dimensions.
method Mean field variational Bayes algorithms with various relaxations and storage strategies.
result Different inference strategies have varying trade-offs between accuracy and computational demands.
The study reveals decision trees' limitations in fitting data from additive models, proving a generalization lower bound.
problem Understanding the generalization performance of decision trees on additive models.
method Analyzing decision tree algorithms with sparse additive models, proving generalization lower bounds.
result Generalization lower bounds for decision trees on sparse additive models are much worse than minimax rates.
New method for efficient ERG fitting on large graphs.
problem Fitting non-trivial ERGs on large graphs.
method Fast matrix block-approximation techniques for dyadic independence.
result Models can generate networks with similar properties to observed networks.
Paper proposes SKCCA for sparse kernel CCA, improving sparsity and reducing overfitting.
problem Lack of sparsity in kernel CCA solutions.
method Introduces SKCCA using ℓ1-regularization to penalize the dual vectors for sparsity. result Demonstrates improved sparsity and reduced overfitting in kernel CCA.
New method uses reinforcement learning to sample from complex data structures efficiently.
problem Constructing reliable samples from high-dimensional polytopes for goodness-of-fit tests.
method Markov decision process and reinforcement learning for sampling.
result Demonstrated scalable tools from linear algebra for theoretical guarantees in non-linear algebra context.
New methods improve deep learning models for sparse, high-dimensional data.
problem Underfitting in inference networks for sparse, high-dimensional data.
method Iterative optimization inspired by stochastic variational inference and improvements in sparse data representation.
result State-of-the-art results on text-count dataset and excellent recommendation results.
New algorithms find conditions and linear rules with high probability and loss.
problem Finding conditions and rules with high probability and loss in conditional sparse regression.
method Efficient algorithms for identifying conditions and rules with optimal probability and loss.
result Achieved algorithms that nearly match the probability of the ideal condition and improve the approximation to the target loss.
We study the problem of variable selection in convex nonparametric regression. Under the assumption that the true regression function is convex and sparse, we develop a screening procedure to select a subset of variables that contains the relevant variables. Our approach is a two-stage quadratic programming method that…
The paper tackles reward-relevance in offline RL with sparse decision dynamics.
problem Offline reinforcement learning with sparse decision dynamics and estimation sparsity.
method Reward-filtered least-squares policy evaluation using thresholded lasso.
result The method provides theoretical guarantees with sample complexity dependent on sparse component size.
New method detects nonlinear causality in multivariate time series data.
problem Detecting nonlinear causal relationships in multidimensional time series.
method Sparse additive models (SpAMs) with B-spline bases and group-lasso optimization.
result The method can accurately estimate nonlinear causal relationships in β-mixing time series.
New method discovers causal relationships in sparse linear data.
problem Discovering cause-effect relationships in sparse linear data.
method Uses structural matrix to reconstruct data and identify causal structures without independence tests.
result Outperforms existing methods in sparse causal structure recovery.
Gaussian graphical models (GGM) have been widely used in many high-dimensional applications ranging from biological and financial data to recommender systems. Sparsity in GGM plays a central role both statistically and computationally. Unfortunately, real-world data often does not fit well to sparse graphical models. I…
SRF learns sparse rule models by screening out features efficiently.
problem Learning optimal sparse rule models is computationally intractable due to the large number of possible rules.
method SRF uses meta safe screening (mSS) to efficiently screen out multiple features, improving the learning of sparse rule models.
result SRF provides a general framework for fitting sparse rule models and can handle group regularization.
Proposes a method to emulate sparse priors using L1 regularization without complex transformations.
problem Sparse priors in under-determined estimation problems.
method Parameter transform to emulate sparse priors under L2 regularization.
result L1 regularization can be achieved with a remapping of parameters under normal priors.
Iterative methods for fitting a Gaussian Random Field (GRF) model via maximum likelihood (ML) estimation requires solving a nonconvex optimization problem. The problem is aggravated for anisotropic GRFs where the number of covariance function parameters increases with the dimension. Even evaluation of the likelihood fu…
RGAM builds more accurate models by preferring linear features over non-linear ones.
problem Building accurate models when linearity assumption is poor.
method Multi-stage algorithm guided by the principle of preferring linear features.
result RGAM can fit sparse generalized additive models at scale for various data types.
ANN learner finds sparse needles in nonlinear haystacks with high probability.
problem Finding sparse features in nonlinear data.
method Generalized LASSO penalty with stochastic gradient descent, warm-start algorithm.
result Phase transition in ANN learner's ability to find needles, better than other learners.
Jointly learns feature and sample relevancies for robust sparse recovery.
problem Sparse recovery sensitivity to data contaminants like outliers or misspecified noise.
method Jointly learns feature and sample relevancies via marginal likelihood optimization.
result Consistent sparse and robust prediction models across diverse tasks.