Analyzes generalization error in generalized linear models, explaining double descent phenomenon.
problem Understanding generalization of machine learning models in high dimensions.
method Develops a framework to characterize asymptotic generalization error for generalized linear models.
result Rigorously explains the double descent phenomenon in generalized linear models.
New minimal hypersurfaces and cones found through linear superposition.
problem Understanding minimal surfaces in Euclidean space.
method Linear superposition principle applied to minimal hypersurfaces and cones.
result New minimal hypersurfaces and cones discovered.
Generative LLE modifies LLE to generate stochastic embeddings.
problem Nonlinear dimensionality reduction and manifold learning.
method Generative LLE modifies LLE by using stochastic linear reconstruction.
result Generative LLE can generate various LLE embeddings stochastically.
Let F_n denote the free group generated by n letters. The purpose of this article is to show that Hol(F_2), the holomorph of the free group on two generators, is linear. Consequently, any split group extension of F_2 by a linear group H is linear. This result gives a large linear subgroup of Aut(F_3). A second applicat…
Study one-dimensional topological theories with linear generating functions.
problem Understanding one-dimensional topological theories with defects.
method Construct bases of hom spaces for decorated unoriented one-dimensional cobordisms.
result Gram determinant and linear generating functions constructed.
New algorithm for identifying best drug arm in generalized linear bandits.
problem Identifying the best drug arm in drug design with minimal trials.
method Proposes an algorithm for best-arm identification in generalized linear bandits, providing theoretical guarantees and simulations.
result First algorithm for best-arm identification in generalized linear bandits with theoretical guarantees.
Linear F-manifolds are studied with connections and dual spaces.
problem Understanding linear F-manifolds and their dual spaces.
method Developed systematic treatment and defined duality using connections.
result Defined compatibility conditions between linear F-manifolds and generalized tangent bundle.
New bounds for nearly-linear networks without training.
problem Generalization of neural networks close to linearity.
method Perturbation of linear networks to derive bounds.
result First non-vacuous bounds for neural nets.
Paper introduces structured sparsity estimators for Generalized Linear Models.
problem Estimating structured sparsity in GLMs with debiased estimators.
method Extends Stucky and van de Geer's results to GLMs with structured sparsity.
result Proves oracle inequalities for structured sparsity estimators in GLMs.
The paper identifies generators of linear SDEs with noise types.
problem Identifying the generator of linear SDEs from their solution distribution.
method Deriving sufficient and necessary conditions for additive noise, and sufficient conditions for multiplicative noise.
result Generic conditions for identifying the generator of linear SDEs with both types of noise.
Investigates linearity of group amalgams and new examples of non-linear groups.
problem Linearity of group amalgams and examples of non-linear groups.
method Investigates linearity of amalgams of subgroups of algebraic groups.
result Establishes linearity of certain 'doubles' of linear groups and finds new non-linear examples.
Extends linear MDP to handle nonlinear rewards.
problem Restrictive linear MDP assumption limits real-world applicability.
method Proposes Generalized Linear MDP (GLMDP) with GLMs for rewards.
result Develops offline RL algorithms achieving suboptimality guarantees.
Study on RCD(0,N) spaces with small linear diameter growth.
problem Understanding structure properties of RCD(0,N) spaces.
method Analyzing the (revised) fundamental group of RCD(0,N) spaces.
result Proved that the revised fundamental group is finitely generated for RCD(0,N) spaces with small linear diameter growth.
New linear flows using exponential of linear transformations improve generative models.
problem Improving generative models in machine learning.
method Developed convolution exponentials and generalized Sylvester Flows using the exponential of linear transformations.
result Convolution exponentials and Convolutional Sylvester Flows outperform other models in log-likelihood.
Neural networks with DAGs show linearity as width increases.
problem Understanding linearity in neural networks with arbitrary DAG structures.
method Analyzing the transition to linearity in networks with arbitrary DAGs, characterizing width by minimum in-degree.
result General neural networks with DAGs exhibit linearity as width approaches infinity.
RGAM builds more accurate models by preferring linear features over non-linear ones.
problem Building accurate models when linearity assumption is poor.
method Multi-stage algorithm guided by the principle of preferring linear features.
result RGAM can fit sparse generalized additive models at scale for various data types.
New algorithm reduces regret in generalized linear contextual bandits.
problem Optimizing rewards in binary reward contexts with feature vectors.
method Upper confidence bound algorithm for generalized linear models.
result Achieves optimal regret of i l d e O ( d T ) ilde{O}(\sqrt{dT}) i l d e O ( d T ) . StarNet trains deep models without gradients using linear equations.
problem Training deep generative models with gradients.
method Solving determined systems of linear equations.
result Least-square bounds for latent codes and model parameters.
In recent work on both generative and discriminative score to log-likelihood-ratio calibration, it was shown that linear transforms give good accuracy only for a limited range of operating points. Moreover, these methods required tailoring of the calibration training objective functions in order to target the desired r…
Paper analyzes agnostic learning of mixed linear regression without generative models.
problem Learning mixed linear regression without assuming stochastic generation.
method Expectation Maximization (EM) and Alternating Minimization (AM) algorithms.
result AM and EM algorithms converge to population loss minimizers under standard conditions.
Generative classifiers' properties are linked to linear constraints.
problem Understanding the Markov property in generative classifiers.
method Characterization of discrimination functions using linear constraints and a second order finite difference operator.
result Discrimination functions of undirected Markov network classifiers are characterized by sets of linear constraints.
Develops estimators for near-optimal linear regression under distribution shift.
problem Linear regression under distribution shift with scarce target domain data.
method Minimax linear risk estimators covering various transfer learning settings.
result Achieves near-optimal risk for linear regression problems under distribution shift.
New insights into optimization and generalization for linear models.
problem Understanding the implicit regularization of optimization methods for linear models.
method Investigating the norms minimized by interpolating solutions and using projections to move between solutions.
result Proving that for over-parameterized linear classification, projections onto the data-span enable the use of under-parameterized techniques.
New bounds for transfer learning in linear models, improving generalization.
problem Understanding when auxiliary data helps in improving generalization in linear models.
method Derivation of exact error bounds and optimal task weights for linear regression and linear neural networks.
result First non-vacuous sufficient conditions for beneficial auxiliary learning in linear neural networks.
Two randomized algorithms improve regret bounds for generalized linear bandits.
problem Improving regret bounds for generalized linear bandits.
method Two randomized algorithms: GLM-TSL and GLM-FPL.
result Upper bounds of O ( d n log K ) O(d \sqrt{n \log K}) O ( d n log K ) on regret for both algorithms. Extends tangent functor to microformal morphisms, creating non-linear pullbacks for forms and cohomology.
problem Generalizing smooth maps to microformal morphisms for new types of mappings.
method Introduces microformal morphisms and shows how they act on functions and forms via non-linear pullbacks.
result Non-linear pullbacks of forms respect de Rham differentials and induce transformations of cohomology.
NGSLL combines DNN accuracy with linear model interpretability.
problem Combining high accuracy of DNNs with interpretability of linear models.
method Neural generators of sparse local linear models (NGSLL) using DNNs to approximate non-linear functions.
result Effective in real-world datasets, achieving high predictive performance and interpretability.
Study on neural scaling laws for solving linear systems in-context.
problem Theoretical guarantees for solving linear systems using a linear transformer architecture.
method Neural scaling laws and task diversity for in-domain and out-of-domain generalization.
result Novel notion of task diversity for necessary and sufficient condition of generalization under task shifts.
Study Loday algebroids, prove splitting theorem, and linearize problems.
problem Splitting and linearization of Loday algebroids.
method Local splitting-type results, Euler-like derivations.
result Established a general linearization principle.
Paper proposes linear transformers for efficient in-context learning without context length limitations.
problem Quadratic complexity of softmax transformers limits data processing speed.
method Investigates linear transformers under domain generalization, showing they learn mappings from context distributions to response functions.
result Linear transformers achieve in-context learning with a linear complexity in context length, offering a dimension-independent convergence rate.
The linear transports along paths in vector bundles introduced in Ref. [1] are applied to the special case of tensor bundles over a given differentiable manifold. Links with the transports along paths generated by derivations of tensor algebras are investigated. A possible generalization of the theory of geodesics is p…
BELIEF framework interprets GLMs using binary linear models.
problem Understanding and interpreting generalized linear models (GLMs) with binary outcomes.
method Developed a framework called binary expansion linear effect (BELIEF) to interpret GLMs through transparent linear models.
result BELIEF framework reveals perfect predictors in complete separation scenarios.
The paper characterizes compatible linear connections on 3D Finsler manifolds.
problem Characterizing compatible linear connections on Finsler manifolds of dimension three.
method Intrinsic method to characterize compatible linear connections, focusing on indicatrices and Euclidean symmetries.
result If a compatible linear connection is not unique, indicatrices must be Euclidean surfaces of revolution.
New algorithm reduces reinforcement learning regret to sqrt(d^3T).
problem Efficient reinforcement learning with generalized linear function approximation.
method Optimistic closure assumption for generalized linear functions.
result Proved regret bound of O(sqrt(d^3T)).
The paper reviews and applies GLMs in actuarial analysis.
problem Analyzing actuarial situations using generalized linear models.
method Assess model adequacy using deviance and Akaike information criterion.
result Best chosen GLM model accurately predicts claims in an automobile portfolio.
Linear groups without infinite order unipotents have good properties.
problem Properties of linear groups without unipotent elements of infinite order.
method Analyzing the structure and properties of linear groups without unipotent elements of infinite order.
result Linear groups without unipotent elements of infinite order have good properties, including centralisers virtually splitting and finitely generated abelian subgroups being undistorted.
We construct analogues of FI-modules where the role of the symmetric group is played by the general linear groups and the symplectic groups over finite rings and prove basic structural properties such as Noetherianity. Applications include a proof of the Lannes--Schwartz Artinian conjecture in the generic representatio…
Linear models can grok without understanding, improving generalization.
problem Understanding the phenomenon of grokking in linear models.
method Analytical and numerical derivation of training and generalization dynamics in linear networks.
result Grokking can occur in linear networks without reaching understanding, and its timing depends on various parameters.
Study shows a linear quadratic regulator's imitation learning converges globally.
problem Global convergence of imitation learning for linear quadratic regulators.
method Analyzed alternating gradient algorithm and established Q-linear rate of convergence.
result Established a unique saddle point for globally optimal policy and reward function.
LOL method simplifies forming linear combinations of latent variables.
problem Lack of general-purpose methods for manipulating latent variables.
method Latent Optimal Linear combinations (LOL) method.
result LOL simplifies creation of expressive low-dimensional representations.
This paper tackles efficient federated learning for generalized linear bandits.
problem Limited communication efficiency restricts existing federated learning solutions to linear models.
method Proposes a communication-efficient solution framework using online and offline regression.
result Proves sub-linear regret and communication cost for generalized linear bandits.
The paper proposes a method to optimize rule-based models for better accuracy and interpretability.
problem Developing rule-based models for regression and classification with better accuracy and interpretability.
method Column generation to optimize over an exponentially large space of rules, using integer programming or a heuristic.
result The proposed methods achieve better accuracy-complexity trade-offs than existing rule ensemble algorithms.
We prove that the semistability growth of hyperbolic groups is linear, which implies that hyperbolic groups which are sci (simply connected at infinity) have linear sci growth. Based on the linearity of the end-depth of finitely presented groups we show that the linear sci is preserved under amalgamated products over f…
Proposes LARS and LASSO methods for generalized linear models.
problem Sparse estimation for generalized linear models.
method Transforms statistical model manifold into tangent space, applies LARS and LASSO algorithms.
result Efficient and performs well, similar to l 1 l_1 l 1 -regularized maximum likelihood estimation. The paper provides a non-asymptotic error bound for linear system identification under nonlinear policies.
problem System identification for linear systems with nonlinear and/or time-varying policies under i.i.d. random excitation noises.
method Least square estimation with non-asymptotic error bound for bounded state and action trajectories.
result The error bound is consistent with linear policies and generalizes existing guarantees.
Proposes a method for differentially private linear regression and synthetic data generation.
problem Lack of valid inference and synthetic data generation methods for small-scale datasets in privacy-aware settings.
method Gaussian differentially private linear regression with bias-corrected estimator and SDG procedure.
result Improves accuracy and provides valid confidence intervals for downstream tasks.
New algorithm minimizes regret in stochastic linear bandits with perturbed history.
problem Minimizing cumulative regret in stochastic linear bandits.
method Perturbed-history exploration in a linear bandit (LinPHE) algorithm.
result Achieves a O ( d n ) O(d \sqrt{n}) O ( d n ) gap-free bound on cumulative regret. We show that classical Wilczynski--Se-ashi invariants of linear systems of ordinary differential equations are generalized in a natural way to contact invariants of non-linear ODEs. We explore geometric structures associated with equations that have vanishing generalized Wilczynski invariants and establish relationship…