arXiv research

New families of translation surfaces with multiple oblivious points discovered.

New study shows data-oblivious attacks can outperform data-aware ones.

A new data-oblivious sketch for logistic regression reduces data size while maintaining approximation accuracy.

Study non-oblivious adversarial bandits with delayed feedback and propose algorithms with improved regret bounds.

We investigate the problem of algorithmic fairness in the case where sensitive and non-sensitive features are available and one aims to generate new, `oblivious', features that closely approximate the non-sensitive features, and are only minimally dependent on the sensitive ones. We study this question in the context o…

Classical multi-armed bandit problems use the expected value of an arm as a metric to evaluate its goodness. However, the expected value is a risk-neutral metric. In many applications like finance, one is interested in balancing the expected return of an arm (or portfolio) with the risk associated with that return. In …

The paper tackles pure exploration in multi-armed bandits with low rank structure using oblivious sampling.

DOFEN improves DNN performance on tabular data benchmarks.

Optimal subspace embedding with near-optimal sparsity for high-dimensional data.

This work introduces oblivious fairness definitions for image generation.

New algorithms for GLMs with oblivious noise, identifying solutions even when half the data is corrupted.

New insights into variable selection with different model assumptions.

Linear regression models contaminated by Gaussian noise (inlier) and possibly unbounded sparse outliers are common in many signal processing applications. Sparse recovery inspired robust regression (SRIRR) techniques are shown to deliver high quality estimation performance in such regression models. Unfortunately, most…

Paper improves performance guarantees for Rademacher projections.

Study improves privacy-preserving online prediction from experts with speed-ups.

New algorithm achieves both static and dynamic regret optimally against an oblivious adversary for deterministic losses.

Nowadays, deep neural networks (DNNs) have become the main instrument for machine learning tasks within a wide range of domains, including vision, NLP, and speech. Meanwhile, in an important case of heterogenous tabular data, the advantage of DNNs over shallow counterparts remains questionable. In particular, there is …

Study shows efficient algorithms for noiseless linear regression require quadratic sample complexity in contamination rate.

New algorithms optimize without knowing problem parameters.

Random projections (RP) are a popular tool for reducing dimensionality while preserving local geometry. In many applications the data set to be projected is given to us in advance, yet the current RP techniques do not make use of information about the data. In this paper, we provide a computationally light way to extra…

Sparse OSEs achieve optimal embedding dimension of O(d).

Let X be a data matrix of rank ρ, whose rows represent n points in d-dimensional space. The linear support vector machine constructs a hyperplane separator that maximizes the 1-norm soft margin. We develop a new oblivious dimension reduction technique which is precomputed and can be applied to any input matrix X. We pr…

The paper develops adaptive confidence intervals for Efron's Gaussian two-groups model with unknown contamination.

Study the tradeoffs of bandit feedback in multiclass classification.

Slow running or straggler tasks can significantly reduce computation speed in distributed computation. Recently, coding-theory-inspired approaches have been applied to mitigate the effect of straggling, through embedding redundancy in certain linear computational steps of the optimization algorithm, thus completing the…

In this paper new general modewise Johnson-Lindenstrauss (JL) subspace embeddings are proposed that are both considerably faster to generate and easier to store than traditional JL embeddings when working with extremely large vectors and/or tensors. Corresponding embedding results are then proven for two different type…

We study the problem of maximizing a monotone set function subject to a cardinality constraint $k$ in the setting where some number of elements $τ$ is deleted from the returned set. The focus of this work is on the worst-case adversarial setting. While there exist constant-factor guarantees when the function is submodu…

Bandit algorithms struggle with consistent performance and robustness.

Stochastic games provide a framework for interactions among multiple agents and enable a myriad of applications. In these games, agents decide on actions simultaneously, the state of every agent moves to the next state, and each agent receives a reward. However, finding an equilibrium (if exists) in this game is often …

This paper introduces TNTK to study infinite soft tree ensembles.

New algorithm speeds up polynomial kernel approximations.

We give a fast oblivious L2-embedding of $A\in \mathbb{R}^{n x d}$ to $B\in \mathbb{R}^{r x d}$ satisfying $(1-\varepsilon)\|A x\|_2^2 \le \|B x\|_2^2 <= (1+\varepsilon) \|Ax\|_2^2.$ Our embedding dimension $r$ equals $d$, a constant independent of the distortion $\varepsilon$. We use as a black-box any L2-embedding $Π…

New robust regression method works with fewer data points than previous methods.

Improved upper bound for online calibrated forecasting of binary sequences.

There is an increasing body of evidence suggesting that exact nearest neighbour search in high-dimensional spaces is affected by the curse of dimensionality at a fundamental level. Does it necessarily mean that the same is true for k nearest neighbours based learning algorithms such as the k-NN classifier? We analyse t…

Develops efficient estimators for PCA and sparse regression in the presence of oblivious outliers.

New method adapts to unknown mixing time in stochastic optimization.

New algorithms improve online prediction from experts with privacy constraints.

The paper describes an application of Aggregating Algorithm to the problem of regression. It generalizes earlier results concerned with plain linear regression to kernel techniques and presents an on-line algorithm which performs nearly as well as any oblivious kernel predictor. The paper contains the derivation of an …

This work improves tensor decomposition methods, especially for large datasets.

Paper details Hilbert-curve for high-performance data mining.

Extended Q-learning stability and convergence analysis.

We study the problem of robust linear regression with response variable corruptions. We consider the oblivious adversary model, where the adversary corrupts a fraction of the responses in complete ignorance of the data. We provide a nearly linear time estimator which consistently estimates the true regression vector, e…

New DP algorithms with margin guarantees for various hypothesis sets.

Simplified proof for dimension reduction of polygonal curves.

We survey distributed deep learning models for training or inference without accessing raw data from clients. These methods aim to protect confidential patterns in data while still allowing servers to train models. The distributed deep learning methods of federated learning, split learning and large batch stochastic gr…

Convolutional Neural Networks (CNN) are being actively explored for safety-critical applications such as autonomous vehicles and aerospace, where it is essential to ensure the reliability of inference results in the presence of possible memory faults. Traditional methods such as error correction codes (ECC) and Triple …

In this work we investigate the reasons why Batch Normalization (BN) improves the generalization performance of deep networks. We argue that one major reason, distinguishing it from data-independent normalization methods, is randomness of batch statistics. This randomness appears in the parameters rather than in activa…