This article introduces the concepts around Online Bandit Linear Optimization and explores an efficient setup called SCRiBLe (Self-Concordant Regularization in Bandit Learning) created by Abernethy et. al.\cite{abernethy}. The SCRiBLe setup and algorithm yield a regret bound and polynomial run time comple…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “linear setup”
Lin et al. (Reports, 7 September 2018, p. 1004) reported a remarkable proposal that employs a passive, strictly linear optical setup to perform pattern classifications. But interpreting the multilayer diffractive setup as a deep neural network and advocating it as an all-optical deep learning framework are not well jus…
Unpaired multi-domain causal representation learning is possible with sufficient conditions.
problem Learning shared causal representation from unpaired data across domains.
method Identify sufficient conditions for joint distribution and shared causal graph recovery.
result Practical method to recover shared latent causal graph from marginal distributions.
We provide a microfoundation for linear price impact models in a stationary market.
problem Deriving linear price impact models in a stationary market with asymmetric information.
method Deriving linear price impact models as the equilibrium of an agent-based system.
result The model shows compatibility with universal price diffusion at small times and non-universal mean-reversion at larger times.
Linear models can grok without understanding, improving generalization.
problem Understanding the phenomenon of grokking in linear models.
method Analytical and numerical derivation of training and generalization dynamics in linear networks.
result Grokking can occur in linear networks without reaching understanding, and its timing depends on various parameters.
New algorithm for traffic routing in congested conditions.
problem Optimal routing in congested traffic networks.
method Congested Bandits model, UCB algorithm, iterative least squares planner.
result No-regret algorithms for congestion-aware routing.
R package for online forecasting in various fields.
problem Frequent updates of forecasts in online settings.
method Generalized setup for time-adaptive fitting of dynamical and non-linear models.
result Effective use of forecasts as model inputs in operational settings.
A new bandit strategy improves learning speed at the cost of accuracy.
problem Learning speed vs. accuracy trade-off in multi-armed bandits.
method Weighted Thompson Sampling (WTS) for multi-armed bandits with Gaussian feedback.
result Upper bound matches log(T) lower bound, solving fundamental problem.
Linear models can overfit without harming OOD generalization under certain conditions.
problem Understanding how overparameterized linear models generalize to out-of-distribution data.
method Analyzing overparameterized linear models under covariate shift, providing guarantees for OOD generalization.
result Benign overfitting occurs in standard ridge regression under OOD conditions, with specific structural conditions on target covariance.
We analyze the necessary number of samples for sparse vector recovery in a noisy linear prediction setup. This model includes problems such as linear regression and classification. We focus on structured graph models. In particular, we prove that sufficient number of samples for the weighted graph model proposed by Heg…
We examine a fundamental problem that models various active sampling setups, such as network tomography. We analyze sampling of a multivariate normal distribution with an unknown expectation that needs to be estimated: in our setup it is possible to sample the distribution from a given set of linear functionals, and th…
Contrastive learning helps linear models understand document topics.
problem Document classification with limited labeled data.
method Contrastive learning applied to document topic modeling.
result Linear models can recover topic posterior information from contrastive learning representations.
Gradient EM converges exponentially to optimal solution in agnostic mixtures.
problem Fitting parametric functions to given data points without a generative model.
method Gradient EM algorithm for agnostic mixtures of arbitrary parametric functions.
result Gradient EM converges exponentially to population loss minimizers with high probability.
Proves wave equation solutions in Kerr-de Sitter spacetime have specific asymptotic expansions.
problem Analyzing solutions to wave equations in Kerr-de Sitter spacetime.
method Developed a Fredholm setup for quasinormal modes and analyzed trapping of lightlike geodesics.
result Proves asymptotic expansions of wave equation solutions up to a decay order.
Overparameterized linear model shows strong classification but weak regression, susceptible to adversarial perturbations.
problem Adversarial vulnerability of overparameterized linear models.
method Lifted Fourier feature map, analyzing overparameterized linear ensemble.
result Spatial localization leads to adversarial vulnerability in an intermediate classification regime.
Non-Euclidean Differentially Private Stochastic Convex Optimization: Optimal Rates in Linear Timecs.LG
New algorithm achieves optimal privacy and efficiency in non-Euclidean convex optimization.
problem Optimizing convex functions while maintaining privacy in non-Euclidean settings.
method Developed a linear-time algorithm for -setups, leveraging geometric properties.
result Optimal excess risk achieved in linear time for .
Study optimal execution in financial markets with constraints.
problem Optimal execution with non-negative constraints in a linear price impact model.
method Purely probabilistic approach via non-linear ODE.
result Complete characterization of value and optimal control.
Supervised linear feature extraction can be achieved by fitting a reduced rank multivariate model. This paper studies rank penalized and rank constrained vector generalized linear models. From the perspective of thresholding rules, we build a framework for fitting singular value penalized models and use it for feature …
Paper uses SGD for solving linear inverse problems, improving empirical performance.
problem Solving statistical inverse problems in science and engineering.
method Stochastic Gradient Descent (SGD) for linear inverse problems, with smoothing techniques.
result Consistency and finite sample bounds for excess risk demonstrated.
In a regression setup with deterministic design, we study the pure aggregation problem and introduce a natural extension from the Gaussian distribution to distributions in the exponential family. While this extension bears strong connections with generalized linear models, it does not require identifiability of the par…
We develop the fundamental theorem of asset pricing in a probability-free infinite-dimensional setup. We replace the usual assumption of a prior probability by a certain continuity property in the state variable. Probabilities enter then endogenously as full support martingale measures (instead of equivalent martingale…
Improved bounds for unbounded losses using transductive priors.
problem Sequential regression and classification with unbounded losses.
method Exponential weights algorithm with transductive priors.
result Statistical bounds independent of design vectors and optimal solution norm.
The non-abelian Hodge correspondence identifies complex variations of Hodge structures with certain Higgs bundles. In this work we analyze this relationship, and some of its ramifications, when the variations of Hodge structures are determined by a (complete) one-dimensional family of compact Calabi-Yau manifolds. This…
Simple probabilistic solution for optimal liquidation with linear price impact.
problem Maximizing expected terminal wealth in a setup with quadratic transaction costs.
method Provided a simple probabilistic solution to the problem.
result Simple and probabilistic form of the solution not previously published.
It has been shown that injecting noise into the neural network weights during the training process leads to a better generalization of the resulting model. Noise injection in the distributed setup is a straightforward technique and it represents a promising approach to improve the locally trained models. We investigate…
Linear Mixed Models (LMMs) are important tools in statistical genetics. When used for feature selection, they allow to find a sparse set of genetic traits that best predict a continuous phenotype of interest, while simultaneously correcting for various confounding factors such as age, ethnicity and population structure…
Paper analyzes robustness of MDPDE under INH setups.
problem Global reliability and breakdown behavior of MDPDE under INH.
method Asymptotic breakdown point analysis of MDPDE.
result Derives a theoretical lower bound for the asymptotic breakdown point.
Paper studies how few pretraining tasks are needed for a linear model to solve new tasks.
problem How many pretraining tasks are needed for a linear model to solve new tasks?
method Pretrained a linear attention model for linear regression with a Gaussian prior.
result Effective pretraining requires a small number of independent tasks, and the model closely matches Bayes optimal.
MO-GP models fill gaps in biophysical data with across-domain info transfer.
problem Gap filling of biophysical parameters LAI and fAPAR over rice areas.
method Multi-output Gaussian Processes (MO-GP) based on Linear Model of Coregionalization (LMC).
result MO-GP models successfully predict biophysical variables even in high missing data regimes.
New algorithm recovers model coefficients and supports from noisy data.
problem Simultaneous estimation and support recovery in linear models with Gaussian noise.
method Projection-based algorithm for STG regularized minimization problem, proving convergence and support recovery guarantees.
result New algorithm outperforms existing methods in support recovery for various data setups.
We prove that the wrapped Fukaya category of any -dimensional Weinstein manifold (or, more generally, Weinstein sector) is generated by the unstable manifolds of the index critical points of its Liouville vector field. Our proof is geometric in nature, relying on a surgery formula for Floer cohomology and t…
Improved rates for continual learning using SGD and last-iterate analysis.
problem Forgetting in overparameterized models after fitting multiple tasks.
method Developed novel SGD upper bounds for continual linear models and analyzed their performance.
result Established universal forgetting rates for continual learning.
New algorithms combat adversarial attacks in stochastic linear bandits.
problem Adversarial attacks on stochastic linear bandit rewards.
method Two variants of Robust Phased Elimination algorithms, one knowing and one not.
result Near-optimal regret in non-corrupted case and additive terms dependent on .
Study finds unique radial solutions on manifolds using differential geometry and analysis.
problem Existence and uniqueness of solutions for semi-linear equations on manifolds.
method Combining differential geometry and analysis, transforming problems into equivalent ones over a submanifold of dimension one.
result Established the existence and uniqueness of constant solutions through orbits of a group action.
Broadens Jourdain and Martini's method to non-linear stochastic processes.
problem Applying pricing methods to non-linear stochastic processes.
method Analyzes from probabilistic and analytic viewpoints, extending Jourdain and Martini's method.
result Broadens applicability of pricing methods to non-linear frameworks.
Study on private algorithms for saddle point and variational inequalities, improving efficiency and applicability.
problem Private algorithms for solving saddle point and variational inequalities under differential privacy constraints.
method Developed a recursive regularization algorithm for both Euclidean and non-Euclidean setups, providing bounds on strong SP-gap and VI-gap.
result Achieved nearly optimal rates for strong SP-gap and VI-gap under -differential privacy, applicable to various setups.
The paper identifies causal effects in latent variable models using higher-order cumulants.
problem Challenges in identifying causal effects in latent variable models with latent confounders.
method Using higher-order cumulants, the paper addresses two challenging setups: a single proxy variable and underspecified instrumental variables.
result Causal effects are identifiable with a single proxy or instrument.
Improved reinforcement learning algorithm with linear approximation for unknown dynamics.
problem Reinforcement learning with adversarial changing cost functions and bandit feedback.
method Combines mirror-descent and least squares policy evaluation in an auxiliary MDP.
result Obtains an regret bound, significantly improving over previous methods.
We study contextual linear bandit problems under feature uncertainty, where the features are noisy and have missing entries. To address the challenges posed by this noise, we analyze Bayesian oracles given the observed noisy features. Our Bayesian analysis reveals that the optimal hypothesis can significantly deviate f…
Reinforcement learning is a promising approach to developing hard-to-engineer adaptive solutions for complex and diverse robotic tasks. However, learning with real-world robots is often unreliable and difficult, which resulted in their low adoption in reinforcement learning research. This difficulty is worsened by the …
This thesis formalizes metric selection for machine learning applications.
problem Selecting the right performance metric for machine learning models.
method Metric Elicitation framework using pairwise preference feedback.
result Novel strategies for eliciting various classification metrics robust to noise.
We introduce a new model for online ranking in which the click probability factors into an examination and attractiveness function and the attractiveness function is a linear function of a feature vector and an unknown parameter. Only relatively mild assumptions are made on the examination function. A novel algorithm f…
The SPS method constructs confidence regions for true parameters with optimal sample complexity.
problem Constructing exact, non-asymptotic confidence regions for true system parameters.
method Sign-Perturbed Sums (SPS) method, generalized to various types of problems.
result High probability upper bounds for SPS confidence regions show optimal shrinkage rate.
Paper generalizes strategic classification framework and introduces SVC for PAC-learning.
problem Strategic manipulation of testing data to fool classifiers.
method Unified framework for strategic classification, strategic VC-dimension (SVC).
result Characterizes the learnability and computational tractability of linear classifiers.
Study online learner attacks by manipulating labels, revealing critical thresholds.
problem Adversarial attacks on online learners, especially label manipulation.
method Control-theoretical perspective, theoretical analysis, empirical study.
result Discontinuous transition in learner's accuracy occurs at a critical threshold.
Unsupervised nonparametric detection of unknown objects in noisy images based on percolation theorymath.ST
We develop an unsupervised, nonparametric, and scalable statistical learning method for detection of unknown objects in noisy images. The method uses results from percolation theory and random graph theory. We present an algorithm that allows to detect objects of unknown shapes and sizes in the presence of nonparametri…
SVRG and its variants are among the state of art optimization algorithms for large scale machine learning problems. It is well known that SVRG converges linearly when the objective function is strongly convex. However this setup can be restrictive, and does not include several important formulations such as Lasso, grou…
New sample complexity bound for ERM in stochastic convex optimization.
problem How many data points are needed for ERM to perform well in stochastic convex optimization?
method Developed a new upper bound for sample complexity, proving it is .
result Sample complexity bound of settles the question of ERMs in stochastic convex optimization.