arXiv research

New features generated from kernel methods are minimally dependent on sensitive features.

New study shows data-oblivious attacks can outperform data-aware ones.

New families of translation surfaces with multiple oblivious points discovered.

Study non-oblivious adversarial bandits with delayed feedback and propose algorithms with improved regret bounds.

Classical multi-armed bandit problems use the expected value of an arm as a metric to evaluate its goodness. However, the expected value is a risk-neutral metric. In many applications like finance, one is interested in balancing the expected return of an arm (or portfolio) with the risk associated with that return. In …

The paper tackles pure exploration in multi-armed bandits with low rank structure using oblivious sampling.

DOFEN improves DNN performance on tabular data benchmarks.

Optimal subspace embedding with near-optimal sparsity for high-dimensional data.

This work introduces oblivious fairness definitions for image generation.

New algorithms for GLMs with oblivious noise, identifying solutions even when half the data is corrupted.

New insights into variable selection with different model assumptions.

A new data-oblivious sketch for logistic regression reduces data size while maintaining approximation accuracy.

Paper improves performance guarantees for Rademacher projections.

Study improves privacy-preserving online prediction from experts with speed-ups.

New algorithm achieves both static and dynamic regret optimally against an oblivious adversary for deterministic losses.

Nowadays, deep neural networks (DNNs) have become the main instrument for machine learning tasks within a wide range of domains, including vision, NLP, and speech. Meanwhile, in an important case of heterogenous tabular data, the advantage of DNNs over shallow counterparts remains questionable. In particular, there is …

Study shows efficient algorithms for noiseless linear regression require quadratic sample complexity in contamination rate.

New algorithms optimize without knowing problem parameters.

Let X be a data matrix of rank ρ, whose rows represent n points in d-dimensional space. The linear support vector machine constructs a hyperplane separator that maximizes the 1-norm soft margin. We develop a new oblivious dimension reduction technique which is precomputed and can be applied to any input matrix X. We pr…

Random projections (RP) are a popular tool for reducing dimensionality while preserving local geometry. In many applications the data set to be projected is given to us in advance, yet the current RP techniques do not make use of information about the data. In this paper, we provide a computationally light way to extra…

Sparse OSEs achieve optimal embedding dimension of O(d).

Study the tradeoffs of bandit feedback in multiclass classification.

Slow running or straggler tasks can significantly reduce computation speed in distributed computation. Recently, coding-theory-inspired approaches have been applied to mitigate the effect of straggling, through embedding redundancy in certain linear computational steps of the optimization algorithm, thus completing the…

In this paper new general modewise Johnson-Lindenstrauss (JL) subspace embeddings are proposed that are both considerably faster to generate and easier to store than traditional JL embeddings when working with extremely large vectors and/or tensors. Corresponding embedding results are then proven for two different type…

We study the problem of maximizing a monotone set function subject to a cardinality constraint $k$ in the setting where some number of elements $τ$ is deleted from the returned set. The focus of this work is on the worst-case adversarial setting. While there exist constant-factor guarantees when the function is submodu…

Bandit algorithms struggle with consistent performance and robustness.

Stochastic games provide a framework for interactions among multiple agents and enable a myriad of applications. In these games, agents decide on actions simultaneously, the state of every agent moves to the next state, and each agent receives a reward. However, finding an equilibrium (if exists) in this game is often …

Linear regression models contaminated by Gaussian noise (inlier) and possibly unbounded sparse outliers are common in many signal processing applications. Sparse recovery inspired robust regression (SRIRR) techniques are shown to deliver high quality estimation performance in such regression models. Unfortunately, most…

This paper introduces TNTK to study infinite soft tree ensembles.

New algorithm speeds up polynomial kernel approximations.

We give a fast oblivious L2-embedding of $A\in \mathbb{R}^{n x d}$ to $B\in \mathbb{R}^{r x d}$ satisfying $(1-\varepsilon)\|A x\|_2^2 \le \|B x\|_2^2 <= (1+\varepsilon) \|Ax\|_2^2.$ Our embedding dimension $r$ equals $d$, a constant independent of the distortion $\varepsilon$. We use as a black-box any L2-embedding $Π…

Deep neural networks (DNNs) are powerful nonlinear architectures that are known to be robust to random perturbations of the input. However, these models are vulnerable to adversarial perturbations--small input changes crafted explicitly to fool the model. In this paper, we ask whether a DNN can distinguish adversarial …

Improved upper bound for online calibrated forecasting of binary sequences.

Develops efficient estimators for PCA and sparse regression in the presence of oblivious outliers.

New algorithms improve online prediction from experts with privacy constraints.

The paper describes an application of Aggregating Algorithm to the problem of regression. It generalizes earlier results concerned with plain linear regression to kernel techniques and presents an on-line algorithm which performs nearly as well as any oblivious kernel predictor. The paper contains the derivation of an …

This work improves tensor decomposition methods, especially for large datasets.

Paper details Hilbert-curve for high-performance data mining.

Extended Q-learning stability and convergence analysis.

Tutorial on information bottleneck problems with connections to coding and learning.

A new algorithm learns graph embeddings considering directionality, improving multiple tasks.

We survey distributed deep learning models for training or inference without accessing raw data from clients. These methods aim to protect confidential patterns in data while still allowing servers to train models. The distributed deep learning methods of federated learning, split learning and large batch stochastic gr…

Convolutional Neural Networks (CNN) are being actively explored for safety-critical applications such as autonomous vehicles and aerospace, where it is essential to ensure the reliability of inference results in the presence of possible memory faults. Traditional methods such as error correction codes (ECC) and Triple …

Paper introduces a new IS scheme for estimating distribution tails of complex models.

In this work we investigate the reasons why Batch Normalization (BN) improves the generalization performance of deep networks. We argue that one major reason, distinguishing it from data-independent normalization methods, is randomness of batch statistics. This randomness appears in the parameters rather than in activa…

STAD improves travel time estimation by learning from real traffic data.

New robust regression method works with fewer data points than previous methods.

New algorithm recovers sparse signals robustly against Gaussian noise and adaptive adversaries.