New findings on kernel regression in the quadratic regime, improving understanding of machine learning models.
problem Understanding kernel ridge regression in the quadratic asymptotic regime.
method Extended study of kernel regression to the quadratic regime, establishing approximation bounds and spectral distributions.
result Broad class of inner-product kernels exhibit behavior similar to a quadratic kernel, with precise asymptotic training and test errors characterized.
In this paper, we consider a discrete time economy where we assume that the short term interest rate follows a quadratic term structure of a regime switching asset process. The possible non-linear structure and the fact that the interest rate can have different economic or financial trends justify the interest of Regim…
We study the supervised learning problem under either of the following two models: (1) Feature vectors xi are d-dimensional Gaussians and responses are yi=f∗(xi) for f∗ an unknown quadratic function; (2) Feature vectors xi are distributed as a mixture of two $…
Study how generalization scales with model size and data in quadratic neural networks.
problem Understanding how generalization scales with model size and data in quadratic neural networks.
method Analyzed ℓ2-regularized empirical test error minimization in a quadratic two-layer network with finite-sample setting and structured data. result Revealed a phase diagram with distinct scaling regimes as the number of parameters varies, showing data-dependent power laws controlled by spectral structure of the target.
We introduce and solve a new type of quadratic backward stochastic differential equation systems defined in an infinite time horizon, called \emph{ergodic BSDE systems}. Such systems arise naturally as candidate solutions to characterize forward performance processes and their associated optimal trading strategies in a…
Gaussian equivalence fails for simple polynomial embeddings in quadratic scaling RF models.
problem Failure of Gaussian equivalence in polynomial feature embeddings under quadratic scaling.
method Introduced Conditional Gaussian Equivalent (CGE) model to capture non-Gaussian behavior.
result Correct asymptotics derived for training and test errors in CGE model.
Quadratic models explain neural network behavior during training.
problem Understanding neural network dynamics during training with large learning rates.
method Developed and tested Neural Quadratic Models.
result Neural Quadratic Models exhibit the 'catapult phase' similar to neural networks.
Study on SGD dynamics and scaling laws for training quadratic neural networks in high dimensions.
problem Optimizing and understanding the training dynamics of quadratic neural networks in high-dimensional settings.
method Sharp analysis of SGD dynamics, combining matrix Riccati differential equations and matrix monotonicity arguments.
result Derivation of scaling laws for prediction risk, highlighting power-law dependencies on optimization time, sample size, and model width.
Paper solves complex game theory problems with new equations.
problem Zero-sum stochastic games with non-Markovian switching.
method New multidimensional SRE and BSDE solutions.
result Existence and uniqueness of SRE solutions.
New model captures asymmetric rough volatility with Zumbach effect.
problem Capturing asymmetric rough volatility and Zumbach effect.
method Proposes a bivariate QHawkes process to model asymmetric buying and selling actions.
result Derives a super-rough-Heston model preserving the Zumbach effect.
SRRM improves recursive transport surrogates in the small-discrepancy regime.
problem Insufficient understanding of recursive partitioning methods' statistical behavior and resolution in the small-discrepancy regime.
method Introduced Selective Recursive Rank Matching (SRRM) to improve the resolution of Recursive Rank Matching (RRM).
result SRRM yields a higher-fidelity practical surrogate for the Wasserstein distance at moderate additional computational cost.
A new algorithm for solving constrained convex optimization problems efficiently.
problem Constrained convex optimization problems requiring high accuracy solutions.
method Second-Order Conditional Gradient Sliding (SOCGS) algorithm, using projection-free methods to solve quadratic subproblems inexactly.
result Converges quadratically in primal gap after a finite number of linearly convergent iterations.
Study optimal portfolios in a non-Markovian regime-switching model with random time horizon.
problem Optimal portfolio selection in a market with non-Markovian regime-switching and random time horizon.
method Formulated as a constrained stochastic linear-quadratic optimal control problem, derived closed-form expressions for optimal portfolios and efficient frontier.
result Closed-form expressions for optimal portfolios and efficient frontier derived under non-Markovian regime-switching and random time horizon.
MARCD uses generative scenarios to improve portfolio decisions during regime shifts.
problem Improving portfolio decisions under regime shifts and drawdowns.
method MARCD employs a Gaussian HMM for regime inference, a diffusion generator for scenario production, and a CVaR allocator with tail-weighted and crisis-aware components.
result MARCD reduces maximum drawdowns by 34% compared to baseline methods over 2020-2025.
We reconsider the problem of optimal trading in the presence of linear and quadratic costs, for arbitrary linear costs but in the limit where quadratic costs are small. Using matched asymptotic expansion techniques, we find that the trading speed vanishes inside a band that is narrower than in the absence of quadratic …
New algorithm achieves almost exact graph matching in almost quadratic time.
problem Graph matching under correlated Erdős-Rényi models.
method Rank-based graph matching using local tree correlation tests.
result Achieves almost exact recovery in almost quadratic time complexity.
The paper extends risk measures to two-step approximations and studies log-concave distributions.
problem Extending classical risk measures to two-step approximations.
method Optimization problem for determining optimal regime thresholds and values for log-concave distributions.
result Conditions for the uniqueness of regime changing in log-concave distributions.
New bootstraps improve speed and accuracy for graph count functionals.
problem Efficiently counting subgraphs in large graphs.
method Developed two types of multiplier bootstraps: a fast, approximate linear one and a quadratic one for denser graphs.
result Both bootstraps provide valid inference and higher-order accuracy under different graph sparsity conditions.
New findings show GD converges to a linear interpolator even with quadratic loss function under certain conditions.
problem Understanding convergence of Gradient Descent with quadratic loss functions.
method Parameterized linear regression with quadratic loss function, empirical and theoretical analysis.
result Gradient Descent converges to a linear interpolator even with quadratic loss function under the Edge of Stability regime.
New bounds on optimal transport regularization show faster convergence rates than previously known.
problem Understanding the localization rate of Quadratically Regularized Optimal Transport (QOT) optimizers.
method Established lower bounds and derived mean-squared deviation controls for QOT optimizers.
result Lower bound of support concentration rate εd+21 in directed Hausdorff distance. CRBMs improve financial regime detection with PCD and free energy analysis.
problem Detecting systemic risk regimes in financial time series.
method Extended RBM to CRBM with autoregressive conditioning and PCD. Decomposed free energy into magnitude and correlation components.
result CRBM's free energy metric distinguishes between magnitude shocks and market regimes.
Study on neural network dynamics in high dimensions with quadratic activation.
problem Understanding training dynamics in overparameterized neural networks.
method Derivation of gradient flow equations and analysis under l2-regularization.
result Characterization of estimator performance and spectral properties in the high-dimensional limit.
In this paper, we consider a risk-based optimal investment problem of an insurer in a regime-switching jump diffusion model with noisy memory. Using the model uncertainty modeling, we formulate the investment problem as a zero-sum, stochastic differential delay game between the insurer and the market, with a convex ris…
The paper solves a complex control problem with stochastic elements and switching conditions.
problem Non-homogeneous stochastic LQ control with regime switching and random coefficients.
method Explicit optimal control and value obtained through two systems of backward stochastic differential equations (BSDEs). Existence and uniqueness of solutions proved using BMO martingales and contraction mapping method.
result Explicit optimal state feedback control and optimal value derived for the problem.
Study optimal consumption and investment strategies with constraints in a market with random coefficients.
problem Optimal consumption and investment strategies with constraints in a regime switching market with random coefficients.
method Explicit optimal strategies provided via solutions to new BSDE systems.
result Solving new BSDEs to find optimal values and strategies.
Study uncovers scaling laws and spectral properties of shallow neural networks.
problem Understanding scaling laws and spectral properties of shallow neural networks.
method Leveraging connections with matrix compressed sensing and LASSO, derived a phase diagram for excess risk.
result Uncovered crossovers between scaling regimes and plateau behaviors, validated empirical observations.
Develops a control framework for systemic risk under uncertainty.
problem Systemic risk under model uncertainty.
method Linear-quadratic mean-field control framework with viscosity solutions and verification theorems.
result Explicit feedback controls derived from a coupled Riccati system, preserving analytical tractability.
Study the properties of SGD in non-vanishing learning rate regime.
problem Understanding the noise and fluctuation in SGD with finite learning rates.
method Derive exact solvable results for discrete-time SGD in quadratic loss functions.
result Fluctuation caused by discrete-time dynamics is larger than continuous-time theory predicts.
Bayes-optimal learning of a neural network with quadratic activations is achieved with GAMP-RIE.
problem Learning a neural network with quadratic activations from quadratic samples.
method Combining approximate message passing with rotationally invariant matrix denoising.
result Derives a closed-form expression for Bayes-optimal test error.
Study optimal liquidation strategies with infinite horizon and regime switching.
problem Optimal liquidation with semimartingale strategies in a stochastic environment.
method Characterization of value function and optimal strategy via BSDEs with infinite horizon.
result Existence and uniqueness of optimal control problem solutions.
Theoretical analysis explains why models generalize after overfitting in modular addition.
problem Understanding why models generalize after overfitting in modular addition.
method Theoretical analysis and gradient descent behavior of two-layer quadratic networks and Transformers.
result Two-layer quadratic networks and simple Transformers generalize well after initially overfitting, indicating grokking.
Study on neural networks with quadratic activation functions, focusing on optimization and generalization.
problem Understanding the dynamics and generalization of neural networks with quadratic activation in the over-parametrized regime.
method Teacher-student scenario, empirical loss landscape analysis, gradient descent dynamics, numerical experiments.
result Conditions for the neural network to recover the teacher and achieve small generalization error.
New study shows faster convergence of SGD and Kaczmarz methods.
problem Improving convergence rates of iterative linear system solvers.
method Last-iterate convergence analysis of SGD with greedy step size over smooth quadratics.
result The t-th iterate attains an O(1/t3/4) convergence rate. Study on adversarial robustness in neural networks across initialization and training phases.
problem Understanding adversarial robustness in neural networks during different learning stages.
method Analyzes adversarial robustness in various scenarios of over-parameterized networks with quadratic targets and infinite samples.
result Robustness can worsen when test error improves, and vice versa, revealing new tradeoffs.
New optimization method helps models generalize better after achieving near-perfect training performance.
problem Models can achieve near-perfect training performance but fail to generalize well to unseen examples.
method GROKtimizer combines rapid convergence to interpolation with post-interpolation norm minimization using Critically Damped Momentum.
result GROKtimizer provides a quadratic speedup over classical gradient descent, offering a natural solution for selecting low-norm interpolating solutions.
A TTA framework improves forecasting accuracy in non-stationary time series.
problem Improving forecasting accuracy in non-stationary time series.
method Normalization-based test-time adaptation for causal timeseries forecasting and direction classification.
result Normalization-based TTA improves forecasting error in synthetic gradual drift and can even hurt in aggressive norm-only adaptation in financial markets.
Study resolvent convergence for random matrices with general covariance profiles.
problem Analyzing resolvent convergence for random matrices with non-identically distributed columns.
method Using moments of quadratic forms and deterministic equivalents, the study provides bounds on the trace of matrix products.
result The trace of matrix products is close to the trace of a deterministic equivalent, controlled by matrix norms.
One-pass SGD dynamics in overparameterized quadratic networks show slow escape from poor solutions.
problem Slow escape from poor generalization solutions in overparameterized neural networks.
method Analysis of one-pass SGD dynamics using ordinary differential equations for overlap matrices.
result Overparameterization only modestly accelerates escape from poor solutions.
Private optimization faster on interpolation problems with quadratic growth.
problem Private optimization in interpolation problems.
method Adaptive algorithm with improved sample complexity.
result Exponential improvement in private sample complexity for quadratic growth.
Recent theoretical work has established connections between over-parametrized neural networks and linearized models governed by he Neural Tangent Kernels (NTKs). NTK theory leads to concrete convergence and generalization results, yet the empirical performance of neural networks are observed to exceed their linearized …
The study provides precise asymptotic theory for in-context learning by Transformers.
problem Understanding the sample complexity, pretraining task diversity, and context length for successful in-context learning.
method An exactly solvable model of linear regression task by linear attention, deriving sharp asymptotics.
result Double-descent learning curve with increasing pretraining examples, phase transition between low and high task diversity regimes.
Combines dynamic programming and neural networks for optimal portfolio execution in regime-switching markets.
problem Optimal execution in a market with multiple regimes and non-linear impact costs.
method Four-step numerical framework: approximated orthogonal portfolios, dynamic program for schedule, neural network optimization.
result Neural network optimized strategy outperforms traditional methods in both CRRA and mean-variance objectives.
This paper investigates the finite horizon risk-sensitive portfolio optimization in a regime-switching credit market with physical and information-induced default contagion. It is assumed that the underlying regime-switching process has countable states and is unobservable. The stochastic control problem is formulated …
New method accelerates energetic variational inference using particle dynamics.
problem Efficiently solving variational inference problems with reduced computational cost.
method Particle-based variational inference with implicit scheme, inspired by energy quadratization and operator splitting.
result Significantly reduces computational cost compared to existing methods.
Inverse problems correspond to a certain type of optimization problems formulated over appropriate input distributions. Recently, there has been a growing interest in understanding the computational hardness of these optimization problems, not only in the worst case, but in an average-complexity sense under this same i…
RFRBoost uses random features to boost deep residual neural networks, improving performance and computational efficiency.
problem Improving performance of deep residual neural networks (RFNNs) while preserving convex optimization benefits.
method Random Feature Representation Boosting (RFRBoost) using boosting theory and random features at each layer.
result RFRBoost significantly outperforms RFNNs and end-to-end trained MLP ResNets in small- to medium-scale tabular datasets.
We study the problem of minimizing the average of a very large number of smooth functions, which is of key importance in training supervised learning models. One of the most celebrated methods in this context is the SAGA algorithm. Despite years of research on the topic, a general-purpose version of SAGA---one that wou…
This paper studies pricing derivatives in an age-dependent semi-Markov modulated market. We consider a financial market where the asset price dynamics follow a regime switching geometric Brownian motion model in which the coefficients depend on finitely many age-dependent semi-Markov processes. We further allow the vol…