Analyzes generalization error in generalized linear models, explaining double descent phenomenon.
problem Understanding generalization of machine learning models in high dimensions.
method Develops a framework to characterize asymptotic generalization error for generalized linear models.
result Rigorously explains the double descent phenomenon in generalized linear models.
RGAM builds more accurate models by preferring linear features over non-linear ones.
problem Building accurate models when linearity assumption is poor.
method Multi-stage algorithm guided by the principle of preferring linear features.
result RGAM can fit sparse generalized additive models at scale for various data types.
Paper introduces structured sparsity estimators for Generalized Linear Models.
problem Estimating structured sparsity in GLMs with debiased estimators.
method Extends Stucky and van de Geer's results to GLMs with structured sparsity.
result Proves oracle inequalities for structured sparsity estimators in GLMs.
StarNet trains deep models without gradients using linear equations.
problem Training deep generative models with gradients.
method Solving determined systems of linear equations.
result Least-square bounds for latent codes and model parameters.
BELIEF framework interprets GLMs using binary linear models.
problem Understanding and interpreting generalized linear models (GLMs) with binary outcomes.
method Developed a framework called binary expansion linear effect (BELIEF) to interpret GLMs through transparent linear models.
result BELIEF framework reveals perfect predictors in complete separation scenarios.
Proposes LARS and LASSO methods for generalized linear models.
problem Sparse estimation for generalized linear models.
method Transforms statistical model manifold into tangent space, applies LARS and LASSO algorithms.
result Efficient and performs well, similar to l 1 l_1 l 1 -regularized maximum likelihood estimation. NGSLL combines DNN accuracy with linear model interpretability.
problem Combining high accuracy of DNNs with interpretability of linear models.
method Neural generators of sparse local linear models (NGSLL) using DNNs to approximate non-linear functions.
result Effective in real-world datasets, achieving high predictive performance and interpretability.
This paper aims to review the methodology behind the generalized linear models which are used in analyzing the actuarial situations instead of the ordinary multiple linear regression. We introduce how to assess the adequacy of the model which includes comparing nested models using the deviance and the scaled deviance. …
The paper proposes a method to optimize rule-based models for better accuracy and interpretability.
problem Developing rule-based models for regression and classification with better accuracy and interpretability.
method Column generation to optimize over an exponentially large space of rules, using integer programming or a heuristic.
result The proposed methods achieve better accuracy-complexity trade-offs than existing rule ensemble algorithms.
Holistic GLMs add constraints for better model quality.
problem Improving classical linear regression models.
method Sparsity-inducing, sign-coherence, and linear constraints.
result Holistic GLMs reliably solve GLMs for various responses.
Paper learns dynamic generator models for video sequences.
problem Modeling spatial-temporal processes like dynamic textures and actions.
method Alternating back-propagation through time algorithm to learn latent state vectors and generator model.
result Trains realistic models for dynamic textures and actions.
This paper tackles efficient federated learning for generalized linear bandits.
problem Limited communication efficiency restricts existing federated learning solutions to linear models.
method Proposes a communication-efficient solution framework using online and offline regression.
result Proves sub-linear regret and communication cost for generalized linear bandits.
Extends linear MDP to handle nonlinear rewards.
problem Restrictive linear MDP assumption limits real-world applicability.
method Proposes Generalized Linear MDP (GLMDP) with GLMs for rewards.
result Develops offline RL algorithms achieving suboptimality guarantees.
LOL method simplifies forming linear combinations of latent variables.
problem Lack of general-purpose methods for manipulating latent variables.
method Latent Optimal Linear combinations (LOL) method.
result LOL simplifies creation of expressive low-dimensional representations.
Auto-Associative models cover a large class of methods used in data analysis. In this paper, we describe the generals properties of these models when the projection component is linear and we propose and test an easy to implement Probabilistic Semi-Linear Auto- Associative model in a Gaussian setting. We show it is a g…
Two randomized algorithms improve regret bounds for generalized linear bandits.
problem Improving regret bounds for generalized linear bandits.
method Two randomized algorithms: GLM-TSL and GLM-FPL.
result Upper bounds of O ( d n log K ) O(d \sqrt{n \log K}) O ( d n log K ) on regret for both algorithms. New insights into optimization and generalization for linear models.
problem Understanding the implicit regularization of optimization methods for linear models.
method Investigating the norms minimized by interpolating solutions and using projections to move between solutions.
result Proving that for over-parameterized linear classification, projections onto the data-span enable the use of under-parameterized techniques.
New linear flows using exponential of linear transformations improve generative models.
problem Improving generative models in machine learning.
method Developed convolution exponentials and generalized Sylvester Flows using the exponential of linear transformations.
result Convolution exponentials and Convolutional Sylvester Flows outperform other models in log-likelihood.
New method estimates GGLM parameters, overcoming non-convexity.
problem Estimating parameters in GGLM with dependencies.
method Monotone operator-based variational inequality method.
result Guarantees for parameter recovery in GLM and GGLM.
Develops estimators for near-optimal linear regression under distribution shift.
problem Linear regression under distribution shift with scarce target domain data.
method Minimax linear risk estimators covering various transfer learning settings.
result Achieves near-optimal risk for linear regression problems under distribution shift.
Python library automates feature engineering and selection for linear models.
problem Difficulties in training and explaining complex machine learning models.
method Automated feature engineering and selection for linear models.
result Improves prediction accuracy of linear models while retaining interpretability.
Synthetic data can amplify privacy in linear regression models.
problem Understanding how synthetic data can enhance privacy in linear regression models.
method Investigated through the linear regression framework, analyzing synthetic data generated from random inputs and controlled inputs.
result Releasing a limited number of synthetic data points amplifies privacy beyond the model's inherent guarantees when inputs are random, but not when inputs are controlled by an adversary.
Paper analyzes agnostic learning of mixed linear regression without generative models.
problem Learning mixed linear regression without assuming stochastic generation.
method Expectation Maximization (EM) and Alternating Minimization (AM) algorithms.
result AM and EM algorithms converge to population loss minimizers under standard conditions.
Estimates GLMs robustly against label corruptions.
problem Learning GLMs under adversarial label corruptions.
method Iterative trimmed maximum likelihood estimator.
result Achieves minimax near-optimal risk.
Optimizes explanations for linear models to enhance trust and interpretability.
problem Lack of well-defined interpretability for linear models and ad hoc approaches.
method Optimization framework using coordinate updates on coefficients.
result Derives a family of interpretability metrics for linear models.
Paper improves PAC-Bayesian bounds for linear regression.
problem Improving PAC-Bayesian error bounds for linear regression.
method Proposes a tighter error bound for linear regression that converges to the generalization loss with a well-chosen temperature parameter. Also applies to non-i.i.d. training data.
result Error bound applies to certain time series generated by dynamical models.
New insights into continual learning for deep models, showing convergence issues but local linear solutions.
problem Challenges in continual learning for homogeneous deep models.
method Sequential projections onto task margin sets, leveraging nonconvex projection theory.
result Local linear convergence under certain conditions for homogeneous deep networks.
New models learn stable latent clusters without side info.
problem Stability of non-linear ICA representations without side information.
method Deep generative models with latent clusterings, compared to standard VAEs and auxiliary labeled models.
result Deep generative models with latent clusterings are as stable as models with side information.
Linear models can grok without understanding, improving generalization.
problem Understanding the phenomenon of grokking in linear models.
method Analytical and numerical derivation of training and generalization dynamics in linear networks.
result Grokking can occur in linear networks without reaching understanding, and its timing depends on various parameters.
New algorithm minimizes regret in stochastic linear bandits with perturbed history.
problem Minimizing cumulative regret in stochastic linear bandits.
method Perturbed-history exploration in a linear bandit (LinPHE) algorithm.
result Achieves a O ( d n ) O(d \sqrt{n}) O ( d n ) gap-free bound on cumulative regret. This paper improves Bayesian neural nets by using local linearization.
problem Underfitting in Bayesian neural networks.
method Local linearization of Bayesian neural networks to create a generalized linear model (GLM) for predictions.
result The GLM predictive resolves common underfitting problems of the Laplace approximation.
Region-specific linear models are widely used in practical applications because of their non-linear but highly interpretable model representations. One of the key challenges in their use is non-convexity in simultaneous optimization of regions and region-specific models. This paper proposes novel convex region-specific…
Automates model selection for GLMs using optimization.
problem Automating model selection for generalized linear models.
method Mixed-integer conic optimization for feature subset selection and model constraints.
result Optimization of AIC and BIC criteria with multicollinearity constraints.
This paper analyzes generalization for linear models with spiked covariance structures.
problem Understanding the generalization performance of linear models with spiked covariance structures.
method Derives the generalization error for two simple models with spiked covariances using random matrix theory.
result The eigenvector and eigenvalue corresponding to the spike significantly influence the generalization error.
We develop a one-dimensional notion of affine processes under parameter uncertainty, which we call non-linear affine processes. This is done as follows: given a set of parameters for the process, we construct a corresponding non-linear expectation on the path space of continuous processes. By a general dynamic programm…
Efficient algorithms recover two sparse models from a mix of linear queries.
problem Recovering two sparse models from a mix of linear queries.
method Efficient algorithms for query complexity problem.
result Improved query complexity for model recovery.
PatternLocal improves XAI for non-linear models by suppressing suppressor variables.
problem Suppressor variables cause false-positive feature attributions in non-linear models.
method PatternLocal uses locally linear surrogate models and transforms weights into a generative representation.
result PatternLocal reduces false-positive attributions and provides more reliable explanations.
The abstract reviews models for analyzing count data.
problem Challenges in analyzing count data with standard methods.
method Review of generalized linear models and multinomial models.
result Fundamental connections between multinomial and count models.
Contextual bandits are widely used in Internet services from news recommendation to advertising, and to Web search. Generalized linear models (logistical regression in particular) have demonstrated stronger performance than linear models in many applications where rewards are binary. However, most theoretical analyses …
New deep learning model interprets tabular data with variable selection and explainability.
problem Deep learning models lack interpretability and variable selection.
method Proposes a new network architecture that combines deep learning with generalized linear models.
result The model provides superior predictive power and interpretable results.
Noise improves model quality in non-linear neural networks during decentralized training.
problem Improving generalization of locally trained neural networks.
method Injecting noise into the weights of neural networks during decentralized training.
result Noise injection improves model quality for non-linear neural networks, but not for linear models.
Linear attention in Transformers can be interpreted as dynamic VAR models.
problem Misalignment between Transformers and autoregressive forecasting objectives.
method Interpreting linear attention as VAR, rearranging MLP, attention, and flow.
result SAMoVAR improves performance, interpretability, and efficiency.
Linear trends in classifier accuracy observed under distribution shift.
problem Understanding why classifier accuracies show linear trends under distribution shift.
method Assumed model similarity and verified empirically.
result Linear trend in classifier accuracy occurs unless distribution shift is large.
Study identifies parameters in causal models with latent confounding.
problem Parameter identification in linear non-Gaussian causal models with latent confounding.
method Graphical criterion for necessary and sufficient identifiability of direct causal effects, with polynomial-time algorithm.
result Developed a graphical criterion for identifying direct causal effects in latent variable models with arbitrary non-linear confounding.
New algorithm for identifying best drug arm in generalized linear bandits.
problem Identifying the best drug arm in drug design with minimal trials.
method Proposes an algorithm for best-arm identification in generalized linear bandits, providing theoretical guarantees and simulations.
result First algorithm for best-arm identification in generalized linear bandits with theoretical guarantees.
Paper proposes linear transformers for efficient in-context learning without context length limitations.
problem Quadratic complexity of softmax transformers limits data processing speed.
method Investigates linear transformers under domain generalization, showing they learn mappings from context distributions to response functions.
result Linear transformers achieve in-context learning with a linear complexity in context length, offering a dimension-independent convergence rate.
Kernel methods and MLPs perform similarly to linear models in high dimensions.
problem Understanding the performance of kernel methods and MLPs in high-dimensional settings.
method Analysis of kernel methods and MLPs in a high-dimensional regime with proportional asymptotics.
result Linear models are optimal in high-dimensional settings when data is generated by kernel models with nonlinear relationships.
We analyzed optimism in linear and kernel regression models.
problem Understanding predictive complexity in regression models.
method Derived closed-form asymptotic optimism for linear and kernel regression models.
result Scaled optimism is a useful measure for model complexity.