arXiv research

More data can actually hurt linear regression performance in certain conditions.

Study shows double and triple descent in unsupervised autoencoders, improving performance in various tasks.

Optimal regularization can prevent the double descent phenomenon in learning models.

New algorithm borrows future randomness to stabilize model-free control.

New method improves regression estimates, reducing bias.

DoubleML implements machine learning for causal inference in R.

New method extracts joint and individual signals from multi-view data.

We show that a variety of modern deep learning tasks exhibit a "double-descent" phenomenon where, as we increase model size, performance first gets worse and then gets better. Moreover, we show that double descent occurs not just as a function of model size, but also as a function of the number of training epochs. We u…

Least squares regression shows unexpected double descent in under-parameterized models.

The paper investigates how calibrating propensity scores improves DML estimates of average treatment effects.

Double descent refers to the phase transition that is exhibited by the generalization error of unregularized learning models when varying the ratio between the number of parameters and the number of training samples. The recent success of highly over-parameterized machine learning models such as deep neural networks ha…

The paper develops methods to estimate treatment effects in sample selection models.

Paper proposes ADC framework to reduce ViT SL training communication overhead.

Study on double descent behavior in two-layer neural networks for binary classification.

Reinforcement learning studies how to balance exploration and exploitation in real-world systems, optimizing interactions with the world while simultaneously learning how the world operates. One general class of algorithms for such learning is the multi-armed bandit setting. Randomized probability matching, based upon …

The "double descent" risk curve was proposed to qualitatively describe the out-of-sample prediction accuracy of variably-parameterized machine learning models. This article provides a precise mathematical analysis for the shape of this curve in two simple data models with the least squares/least norm predictor. Specifi…

Double descent in portfolio optimization shows improved performance with complexity, then declines, due to overfitting.

Improves treatment effect estimates using coordinated deep learning.

In this paper, we propose a Double Thompson Sampling (D-TS) algorithm for dueling bandit problems. As indicated by its name, D-TS selects both the first and the second candidates according to Thompson Sampling. Specifically, D-TS maintains a posterior distribution for the preference matrix, and chooses the pair of arms…

Paper analyzes finite-time convergence of double Q-learning.

Proposes AAA for efficient association estimation with confounders.

Study on RF regression with SGD shows double descent phenomenon.

A theorem for debiasing machine learning with finite sample guarantees.

Paper proposes FPG algorithm for unbiased off-policy PG estimation.

We address the problem of multi-class classification in the case where the number of classes is very large. We propose a double sampling strategy on top of a multi-class to binary reduction strategy, which transforms the original multi-class problem into a binary classification problem over pairs of examples. The aim o…

Histogram binning method proven with guarantees without splitting data.

Hybrid regularization avoids double descent in random feature models.

A new method for causal inference in high-dimensional data using machine learning.

The paper uses double machine learning to estimate dynamic treatment effects robustly.

Sparse coding is a crucial subroutine in algorithms for various signal processing, deep learning, and other machine learning applications. The central goal is to learn an overcomplete dictionary that can sparsely represent a given input dataset. However, a key challenge is that storage, transmission, and processing of …

New method learns operators with geometric singularities from few samples.

Modern ML methods show unexpected behaviors that contradict classical statistics.

The aim of this paper is to establish two fundamental measure-metric properties of particular random geometric graphs. We consider $\varepsilon$-neighborhood graphs whose vertices are drawn independently and identically distributed from a common distribution defined on a regular submanifold of $\mathbb{R}^K$. We show t…

Two new algorithms improve Q* approximation in batch RL with linear error propagation.

New findings challenge the traditional U-shaped curve of model complexity and error, revealing a second descent in error as model size increases.

This work reveals how label noise can cause a final ascent in neural network performance curves.

Unified framework for causal inference under sample selection.

A Klein surface is a surface with a dianalytic structure. A double of a Klein surface $X$ is a Klein surface $Y$ such that there is a degree two morphism (of Klein surfaces) $Y\rightarrow X$. There are many doubles of a given Klein surface and among them the so-called natural doubles which are: the complex double, the …

Changing kernel bandwidth during training improves kernel regression performance.

Characterizes RFF regression in large $n,p,N$ setting, providing precise learning phases and double descent curve.

Improves robustness of propensity score estimators in challenging settings.

Develops a test for conditional local independence of counting processes.

We define a general notion of abstract double Lie algebroid. We show (1) that the double Lie algebroid of a double Lie groupoid is a double Lie algebroid in this sense; (2) that the double cotangent constructed from Lie algebroid structures on a vector bundle A and its dual A* is a double Lie algebroid if and only if (…

The word `double' was used by Ehresmann to mean `an object X in the category of all X'. Double categories, double groupoids and double vector bundles are instances, but the notion of Lie algebroid cannot readily be doubled in the Ehresmann sense, since a Lie algebroid bracket cannot be defined diagrammatically. In this…

We propose a generalized double Pareto prior for Bayesian shrinkage estimation and inferences in linear models. The prior can be obtained via a scale mixture of Laplace or normal distributions, forming a bridge between the Laplace and Normal-Jeffreys' priors. While it has a spike at zero like the Laplace density, it al…

We define double principal bundles (DPBs), for which the frame bundle of a double vector bundle, double Lie groups and double homogeneous spaces are basic examples. It is shown that a double vector bundle can be realized as the associated bundle of its frame bundle. Also dual structures, gauge transformations and conne…

Generalizes Hecke algebra for double torus, linking to skein algebra.

Introduces Poisson double algebroids and their relation to Lie 2-bialgebras.