arXiv research

This paper introduces a new transfer learning method for regression.

A simple 2-layer linear network outperforms neural networks in learning sparse targets.

New bounds show limitations of sample-wise information-theoretic generalization.

This paper proposes a fast and accurate method for sparse regression in the presence of missing data. The underlying statistical model encapsulates the low-dimensional structure of the incomplete data matrix and the sparsity of the regression coefficients, and the proposed algorithm jointly learns the low-dimensional s…

Study on limits of LLM-based multi-agent planning reliability.

Bayesian method recovers causal structure in SEMs with equal error variances.

The goal of subspace learning is to find a $k$-dimensional subspace of $\mathbb{R}^d$, such that the expected squared distance between instance vectors and the subspace is as small as possible. In this paper we study subspace learning in a partial information setting, in which the learner can only observe $r \le d$ att…

MIC improves VAR order selection accuracy.

Discrete time hedging in a complete diffusion market is considered. The hedge portfolio is rebalanced when the absolute difference between delta of the hedge portfolio and the derivative contract reaches a threshold level. The rate of convergence of the expected squared hedging error as the threshold level approaches z…

Given a Gaussian Markov random field, we consider the problem of selecting a subset of variables to observe which minimizes the total expected squared prediction error of the unobserved variables. We first show that finding an exact solution is NP-hard even for a restricted class of Gaussian Markov random fields, calle…

Our study analyzes how neural network initialization affects privacy and utility in overparameterized models.

In linear regression we wish to estimate the optimum linear least squares predictor for a distribution over $d$-dimensional input points and real-valued responses, based on a small sample. Under standard random design analysis, where the sample is drawn i.i.d. from the input distribution, the least squares solution for…

Efficient method for choosing data points in machine learning models.

We analyze the errors arising from discrete readjustment of the hedging portfolio when hedging options in exponential Levy models, and establish the rate at which the expected squared error goes to zero when the readjustment frequency increases. We compare the quadratic hedging strategy with the common market practice …

We propose a flexible framework for hedging a contingent claim by holding static positions in vanilla European calls, puts, bonds, and forwards. A model-free expression is derived for the optimal static hedging strategy that minimizes the expected squared hedging error subject to a cost constraint. The optimal hedge in…

We present a general-purpose method to train Markov chain Monte Carlo kernels, parameterized by deep neural networks, that converge and mix quickly to their target distribution. Our method generalizes Hamiltonian Monte Carlo and is trained to maximize expected squared jumped distance, a proxy for mixing speed. We demon…

In a financial market model, we consider the variance-optimal semi-static hedging of a given contingent claim, a generalization of the classic variance-optimal hedging. To obtain a tractable formula for the expected squared hedging error and the optimal hedging strategy, we use a Fourier approach in a general multidime…

Gradient descent learns a single neuron without knowing the relationship between inputs and labels.

Optimal preconditioning improves Langevin sampling efficiency.

We address the new problem of estimating a piece-wise constant signal with the purpose of detecting its change points and the levels of clusters. Our approach is to model it as a nonparametric penalized least square model selection on a family of models indexed over the collection of partitions of the design points and…

K-means clustering improved for robustness to outliers and distribution shifts.

Study improves understanding of non-differentiable penalties in high-dimensional settings.

This paper improves active learning for Gaussian process regression to handle distributional uncertainty.

Paper proposes an unbiased optimization method for Bayesian experimental design.

This study optimizes model averaging for personalized collaborative learning.

Adaptive HMC improves sampling efficiency by optimizing mass matrix.

We introduce a distributionally robust minimium mean square error estimation model with a Wasserstein ambiguity set to recover an unknown signal from a noisy observation. The proposed model can be viewed as a zero-sum game between a statistician choosing an estimator -- that is, a measurable function of the observation…

ChEES-HMC improves SMC samplers' efficiency and speed.

Optimizes target value in stochastic black box functions.

We study the problem of out-of-sample risk estimation in the high dimensional regime where both the sample size $n$ and number of features $p$ are large, and $n/p$ can be less than one. Extensive empirical evidence confirms the accuracy of leave-one-out cross validation (LO) for out-of-sample risk estimation. Yet, a un…

Stein showed that the multivariate sample mean is outperformed by "shrinking" to a constant target vector. Ledoit and Wolf extended this approach to the sample covariance matrix and proposed a multiple of the identity as shrinkage target. In a general framework, independent of a specific estimator, we extend the shrink…

New method tunes SMC samplers efficiently without high costs.

Recent research has documented a significant rise in the volatility (e.g., expected squared change) of individual incomes in the U.S. since the 1970s. Existing measures of this trend abstract from individual heterogeneity, effectively estimating an increase in average volatility. We decompose this increase in average v…

A new method calculates intrinsic effective sample size for manifold-valued data.

RLMH improves adaptive MCMC by optimizing contrastive divergence reward.

This work introduces a new sampling method to approximate an optimal design problem in ridge regression.

Study optimizes online learning for vector-valued data regression.

Optimizes Metropolis-Hastings algorithms for efficient sampling in high dimensions.

We consider a model of selective prediction, where the prediction algorithm is given a data sequence in an online fashion and asked to predict a pre-specified statistic of the upcoming data points. The algorithm is allowed to choose when to make the prediction as well as the length of the prediction window, possibly de…

Paper estimates EOT maps for non-compactly supported measures with subGaussian target.

We propose a randomized nonmonotone block proximal gradient (RNBPG) method for minimizing the sum of a smooth (possibly nonconvex) function and a block-separable (possibly nonconvex nonsmooth) function. At each iteration, this method randomly picks a block according to any prescribed probability distribution and solves…

Develops variance-reduced methods for solving generalized equations.

This book introduces linear models and their theories rigorously.

New variance-reduction methods solve stochastic composite inclusions.

Introduces Fitzpatrick losses, tighter than Fenchel-Young losses.

We study losses for binary classification and class probability estimation and extend the understanding of them from margin losses to general composite losses which are the composition of a proper loss with a link function. We characterise when margin losses can be proper composite losses, explicitly show how to determ…

We present the Tamed Cross Entropy (TCE) loss function, a robust derivative of the standard Cross Entropy (CE) loss used in deep learning for classification tasks. However, unlike other robust losses, the TCE loss is designed to exhibit the same training properties than the CE loss in noiseless scenarios. Therefore, th…

Unified surrogate loss framework for multi-label learning with strong consistency guarantees.