CACTI improves tabular data imputation by leveraging missingness patterns and contextual information.
problem Tabular data imputation with improved accuracy and robustness.
method Masked autoencoding approach with median truncated copy masking and contextual information.
result Average R2 gain of 7.8% over the next best method across various datasets and missingness conditions. Unified framework for mean testing under truncation bias.
problem High-dimensional mean testing under arbitrary truncation.
method Characterizes fundamental limits and develops a simple second-order test.
result Unified framework connects finite-moment, sub-Gaussian, and median-regular structural regimes.
Proposes a faster Transformer decoding method by truncating target-side self-attention windows.
problem Efficiency in Transformer decoding with minimal BLEU score loss.
method N-gram assumption to truncate target-side self-attention windows.
result N-gram masked self-attention model maintains BLEU score for N values from 4 to 8. Recent work has demonstrated the effectiveness of gradient descent for directly recovering the factors of low-rank matrices from random linear measurements in a globally convergent manner when initialized properly. However, the performance of existing algorithms is highly sensitive in the presence of outliers that may …
New method for estimating median and mean with high probability privacy.
problem Estimating median and mean with differential privacy.
method Propose, Test, Release (PTR) mechanism with concentration inequalities.
result First sub-Gaussian high probability bounds for differentially private median and mean estimation.
This paper investigates the phase retrieval problem, which aims to recover a signal from the magnitudes of its linear measurements. We develop statistically and computationally efficient algorithms for the situation when the measurements are corrupted by sparse outliers that can take arbitrary values. We propose a nove…
Study robust linear regression without distributional assumptions for heavy-tailed responses.
problem Linear regression with heavy-tailed responses and no distributional assumptions.
method Combining truncated least squares, median-of-means, and aggregation theory to construct a non-linear estimator.
result Achieves excess risk of order d/n with optimal sub-exponential tail. New algorithms for stochastic linear bandits with heavy-tailed payoffs achieve nearly optimal regret.
problem Stochastic linear bandits with heavy-tailed payoffs.
method Median of means and dynamic truncation.
result Sublinear regret bound of O(d21T1+ε1) for ε∈(0,1]. Transformers exhibit abrupt learning in matrix completion tasks.
problem Understanding abrupt learning in Transformers for matrix completion.
method Formulated matrix completion as MLM task, trained BERT model, analyzed model components.
result Sudden drop in loss despite no changes in training procedure or hyper-parameters.
We develop a unified approach for classification and regression support vector machines for data subject to right censoring. We provide finite sample bounds on the generalization error of the algorithm, prove risk consistency for a wide class of probability measures, and study the associated learning rates. We apply th…
Ancient solutions to curve shortening flow are constructed and analyzed.
problem Constructing ancient solutions to curve shortening flow.
method Analyzing the rotating Yin-Yang soliton and Grim Reaper translating soliton to approximate the solution.
result An ancient solution to planar curve shortening is constructed and analyzed.
Paper proposes robust methods for estimating optimal treatment rules with censored survival data.
problem Estimating optimal treatment rules for censored survival data.
method Developed two robust criteria and a sampling-based difference-of-convex algorithm for learning optimal treatment rules.
result Proposed methods show improved performance compared to existing methods in simulations and real data.
Recurrent neural networks (RNNs) are important class of architectures among neural networks useful for language modeling and sequential prediction. However, optimizing RNNs is known to be harder compared to feed-forward neural networks. A number of techniques have been proposed in literature to address this problem. In…
The stochastic multi-armed bandit problem is well understood when the reward distributions are sub-Gaussian. In this paper we examine the bandit problem under the weaker assumption that the distributions have moments of order 1+ε, for some ε∈(0,1]. Surprisingly, moments of order 2 (i.e., finite variance) are suffi…
We construct compactifications for median spaces with compact intervals, generalising Roller boundaries of CAT(0) cube complexes. Examples of median spaces with compact intervals include all finite rank median spaces and all proper median spaces of infinite rank. Our methods also work for general median algebra…
Study compares LRMC algorithms under dependent sampling in various applications.
problem Recovering missing entries in partially observed low-rank matrices with dependent sampling.
method Various LRMC algorithms tested under dependent sampling in different contexts.
result Performance differences among LRMC algorithms under dependent sampling.
PMI-Masking improves MLM pretraining by masking correlated spans efficiently.
problem Uniform token masking leads to inefficient and suboptimal performance in MLMs.
method PMI-Masking uses Pointwise Mutual Information to mask n-grams with high collocation.
result PMI-Masking reaches half the training time and improves performance.
New concept of coarse medians for higher rank symmetric spaces.
problem Understanding medians in higher rank symmetric spaces.
method Introducing coarse r-median spaces and proving their existence. result Existence of coarse higher medians on divisible and quasi-homogeneous convex domains.
Unique median structures found in hyperbolic spaces.
problem Uniqueness of median structures in hyperbolic spaces.
method Analyzing product of hyperbolic spaces and properties of relative hyperbolicity.
result Non-hyperbolic pants graphs can have unique median structures.
Study on median algebra structures on Euclidean spaces and manifolds with local CAT(0) cubulation.
problem Understanding median algebra structures on Euclidean spaces and manifolds.
method Showed local CAT(0) cubulation for median structures on ER homology manifolds.
result Median structures on ER homology manifolds have a local CAT(0) cubulation structure.
In this paper, we consider the problem of linear regression with heavy-tailed distributions. Different from previous studies that use the squared loss to measure the performance, we choose the absolute loss, which is capable of estimating the conditional median. To address the challenge that both the input and output c…
We prove a version of the Tits alternative for groups acting on complete, finite rank median spaces. This shows that group actions on finite rank median spaces are much more restricted than actions on general median spaces. Along the way, we extend to median spaces the Caprace-Sageev machinery and part of Hagen's theor…
We show that uniform lattices of isometries of products of real hyperbolic spaces act properly discontinuously and cocompactly on a median space. For lattices in products of at least two factors, this is the strongest degree of compatibility possible with the median geometry. Our theorem is also relevant for potential …
Convex cores found for group actions on median spaces.
problem Understanding group actions on median spaces without metric or topology.
method Introduced convex cores for actions on finite-rank median algebras.
result Actions on median spaces have nonempty convex cores.
We introduce and begin to explore the mean and median of finite sets of shapes represented as integral currents. The median can be computed efficiently in practice, and we focus most of our theoretical and computational attention on medians. We consider questions on the existence and regularity of medians. While the me…
New algorithm reduces heavy-tailed linear bandits' computational cost.
problem Stochastic linear bandits with heavy-tailed noise.
method One-pass online mirror descent with adaptive Huber regression.
result Near-optimal regret bound with reduced computational cost.
This paper is a short summary of our recent work on the medians and means of probability measures in Riemannian manifolds. Firstly, the existence and uniqueness results of local medians are given. In order to compute medians in practical cases, we propose a subgradient algorithm and prove its convergence. After that, F…
To improve the off-sample generalization of classical procedures minimizing the empirical risk under potentially heavy-tailed data, new robust learning algorithms have been proposed in recent years, with generalized median-of-means strategies being particularly salient. These procedures enjoy performance guarantees in …
ST-MTM models complex time series by decomposing and masking seasonal and trend components.
problem Forecasting complex time series with intricate temporal variations.
method Seasonal-Trend Decomposition with Masking and Contrastive Learning.
result ST-MTM achieves superior forecasting performance compared to existing methods.
A method for estimating the median of gradients in stochastic optimization.
problem Robust gradient estimation in stochastic optimization for various applications.
method Stochastic Proximal Point Method for median gradient estimation.
result The proposed method can converge even under heavy-tailed, state-dependent noise.
We study model-agnostic copies of machine learning classifiers. We develop the theory behind the problem of copying, highlighting its differences with that of learning, and propose a framework to copy the functionality of any classifier using no prior knowledge of its parameters or training data distribution. We identi…
This paper introduces online algorithms to estimate robust geometric median in large data streams.
problem Detecting outliers in large data sets using robust statistical measures.
method Online stochastic Newton methods for estimating the geometric median.
result Rates of convergence for online estimation of the geometric median.
The study finds that maximizing median returns is the only viable strategy in portfolio selection.
problem Difficulties in studying optimal portfolio strategies due to discontinuity and time inconsistency in maximizing median and quantile returns.
method Used intra-personal equilibrium approach to analyze portfolio selection under median and quantile maximization.
result Median maximization is the only viable strategy, with no investment in risky assets for other quantiles.
Expands MLM by masking token positions, improving performance and convergence.
problem Improving language model performance and convergence.
method Masking token positions along with [MASK] tokens, using a fully connected classifier stage.
result Shows .3% improvement and 50% faster convergence for BERT Base with position masking.
New graph properties inherited by Frechet mean and median.
problem Characterizing the average of graph-valued samples.
method Analysis of Frechet mean and median graphs.
result Edge density is hereditary in Frechet mean and median graphs.
SMART training improves mask-predict translations.
problem Closing the performance gap between semi-autoregressive and autoregressive models.
method SMART training method for conditional masked language models.
result SMART-trained models produce higher-quality translations.
Tukey median performance analyzed under TV corruptions.
problem Performance analysis of Tukey median under TV corruptions.
method Analysis of Tukey median and projection algorithm under TV corruptions.
result Breakdown point reduced to 1/4 under TV corruptions, compared to 1/3 under Huber's model.
The consistency of Fréchet medians is proved for probability measures in proper metric spaces. In the context of Riemannian manifolds, assuming that the probability measure has more than a half mass lying in a convex ball and verifies some concentration conditions, the positions of its Fréchet medians are estimated. It…
New method reduces diffusion model function evaluations for discrete data.
problem High computational burden in generating samples from masked diffusion models.
method Modified causal attention mask and speculative sampling mechanism for non-factorized predictions.
result Achieved ~2x reduction in required network forward passes.
Proposes a proportional masking strategy for better tabular data imputation.
problem Heterogeneity of tabular data disrupts uniform random masking in MAEs.
method Computes missingness statistics, generates proportional masks, uses MLP token mixing.
result Proportional masking preserves missingness distribution, improves imputation performance.
New subspace prototype flag median improves clustering on noisy data.
problem Finding robust prototypes for datasets of images and videos.
method Proposes flag median and introduces FlagIRLS algorithm for its calculation.
result Flag median is robust to outliers and improves cluster purity.
In high dimensions, the mean and geometric median are nearly identical.
problem Understanding the relationship between mean and geometric median in high-dimensional spaces.
method Analytical derivation and simulation of the distance between mean and geometric median.
result The distance between mean and geometric median vanishes with dimensionality in high dimensions.
Improved median of means estimator with tighter bounds.
problem Improving the efficiency and reliability of median of means estimator.
method Modification of the median of means estimator with sub-Gaussian deviation bounds.
result Achieves nearly optimal constants under minimal assumptions.
Empirical median performs well in estimating location with varying scales.
problem Estimating location with varying scales in data.
method Analysis of empirical median as an estimator.
result Matching upper and lower bounds on estimation error.
This article is devoted to the problem of predicting the value taken by a random permutation Σ, describing the preferences of an individual over a set of numbered items {1,…,n} say, based on the observation of an input/explanatory r.v. X e.g. characteristics of the individual), when error is measured…
New estimator for symmetric kernel expectations, robust to missing data.
problem Efficient estimation of symmetric kernel expectations with missing data.
method Median-of-Incomplete-U-Statistics (MIU) estimator.
result Established finite-sample concentration rate for MIU.
New analysis reveals masked self-supervised learning's effectiveness in extracting data structure.
problem Analyzing masked self-supervised learning in high-dimensional data.
method Developed precise high-dimensional analysis of masked modeling objectives.
result Identified phase transitions and structured regimes for masked self-supervised learning.
Median-of-means sampling outperforms mean-of-means for large sample sizes in numerical integration.
problem Improving numerical integration accuracy in high dimensions.
method Median-of-means sampling compared to mean-of-means using RQMC methods.
result Median-of-means sampling is superior for large sample sizes, while mean-of-means is better for smaller sample sizes.