arXiv research

New algorithms for GLMs with oblivious noise, identifying solutions even when half the data is corrupted.

Linear regression models contaminated by Gaussian noise (inlier) and possibly unbounded sparse outliers are common in many signal processing applications. Sparse recovery inspired robust regression (SRIRR) techniques are shown to deliver high quality estimation performance in such regression models. Unfortunately, most…

New families of translation surfaces with multiple oblivious points discovered.

Orthogonal matching pursuit (OMP) is a widely used algorithm for recovering sparse high dimensional vectors in linear regression models. The optimal performance of OMP requires \textit{a priori} knowledge of either the sparsity of regression vector or noise statistics. Both these statistics are rarely known \textit{a p…

Study non-oblivious adversarial bandits with delayed feedback and propose algorithms with improved regret bounds.

Develops efficient estimators for PCA and sparse regression in the presence of oblivious outliers.

Classical multi-armed bandit problems use the expected value of an arm as a metric to evaluate its goodness. However, the expected value is a risk-neutral metric. In many applications like finance, one is interested in balancing the expected return of an arm (or portfolio) with the risk associated with that return. In …

New study shows data-oblivious attacks can outperform data-aware ones.

The paper tackles pure exploration in multi-armed bandits with low rank structure using oblivious sampling.

New algorithm recovers sparse signals robustly against Gaussian noise and adaptive adversaries.

Recovering the support of sparse vectors in underdetermined linear regression models, \textit{aka}, compressive sensing is important in many signal processing applications. High SNR consistency (HSC), i.e., the ability of a support recovery technique to correctly identify the support with increasing signal to noise rat…

The paper develops adaptive confidence intervals for Efron's Gaussian two-groups model with unknown contamination.

DOFEN improves DNN performance on tabular data benchmarks.

Optimal subspace embedding with near-optimal sparsity for high-dimensional data.

This work introduces oblivious fairness definitions for image generation.

New insights into variable selection with different model assumptions.

A new data-oblivious sketch for logistic regression reduces data size while maintaining approximation accuracy.

Paper improves performance guarantees for Rademacher projections.

Study improves privacy-preserving online prediction from experts with speed-ups.

New algorithm achieves both static and dynamic regret optimally against an oblivious adversary for deterministic losses.

We investigate the problem of algorithmic fairness in the case where sensitive and non-sensitive features are available and one aims to generate new, `oblivious', features that closely approximate the non-sensitive features, and are only minimally dependent on the sensitive ones. We study this question in the context o…

We study the problem of robust linear regression with response variable corruptions. We consider the oblivious adversary model, where the adversary corrupts a fraction of the responses in complete ignorance of the data. We provide a nearly linear time estimator which consistently estimates the true regression vector, e…

This paper highlights new opportunities for designing large-scale machine learning systems as a consequence of blurring traditional boundaries that have allowed algorithm designers and application-level practitioners to stay -- for the most part -- oblivious to the details of the underlying hardware-level implementatio…

Nowadays, deep neural networks (DNNs) have become the main instrument for machine learning tasks within a wide range of domains, including vision, NLP, and speech. Meanwhile, in an important case of heterogenous tabular data, the advantage of DNNs over shallow counterparts remains questionable. In particular, there is …

New algorithms recover signals robustly against outliers and heavy-tailed noise.

Orthogonal matching pursuit (OMP) and orthogonal least squares (OLS) are widely used for sparse signal reconstruction in under-determined linear regression problems. The performance of these compressed sensing (CS) algorithms depends crucially on the \textit{a priori} knowledge of either the sparsity of the signal ($k_…

Study shows efficient algorithms for noiseless linear regression require quadratic sample complexity in contamination rate.

A new method for faster optimization of noisy functions.

New algorithms optimize without knowing problem parameters.

Improved SGD for robust linear and ReLU regression with adversarial corruptions.

New robust regression method works with fewer data points than previous methods.

Random projections (RP) are a popular tool for reducing dimensionality while preserving local geometry. In many applications the data set to be projected is given to us in advance, yet the current RP techniques do not make use of information about the data. In this paper, we provide a computationally light way to extra…

Sparse OSEs achieve optimal embedding dimension of O(d).

Deeper models have a more favorable optimization landscape, making them more robust to noise.

Stochastic gradient descent on l1 loss converges to true parameter in online robust regression.

Study the tradeoffs of bandit feedback in multiclass classification.

New algorithm mitigates bias in subset selection with noisy protected attributes.

A new model shows fairness mechanisms can improve selection utility even without implicit bias.

Slow running or straggler tasks can significantly reduce computation speed in distributed computation. Recently, coding-theory-inspired approaches have been applied to mitigate the effect of straggling, through embedding redundancy in certain linear computational steps of the optimization algorithm, thus completing the…

In this paper new general modewise Johnson-Lindenstrauss (JL) subspace embeddings are proposed that are both considerably faster to generate and easier to store than traditional JL embeddings when working with extremely large vectors and/or tensors. Corresponding embedding results are then proven for two different type…

We study the problem of maximizing a monotone set function subject to a cardinality constraint $k$ in the setting where some number of elements $τ$ is deleted from the returned set. The focus of this work is on the worst-case adversarial setting. While there exist constant-factor guarantees when the function is submodu…

Bandit algorithms struggle with consistent performance and robustness.

Stochastic games provide a framework for interactions among multiple agents and enable a myriad of applications. In these games, agents decide on actions simultaneously, the state of every agent moves to the next state, and each agent receives a reward. However, finding an equilibrium (if exists) in this game is often …

This paper introduces TNTK to study infinite soft tree ensembles.

New algorithm speeds up polynomial kernel approximations.

We give a fast oblivious L2-embedding of $A\in \mathbb{R}^{n x d}$ to $B\in \mathbb{R}^{r x d}$ satisfying $(1-\varepsilon)\|A x\|_2^2 \le \|B x\|_2^2 <= (1+\varepsilon) \|Ax\|_2^2.$ Our embedding dimension $r$ equals $d$, a constant independent of the distortion $\varepsilon$. We use as a black-box any L2-embedding $Π…

Improved upper bound for online calibrated forecasting of binary sequences.

Discovering and characterizing the large-scale topological features in empirical networks are crucial steps in understanding how complex systems function. However, most existing methods used to obtain the modular structure of networks suffer from serious problems, such as being oblivious to the statistical evidence sup…