arXiv research

Paper identifies and solves a 'scrambled translation' issue in UNMT models.

Paper proves privacy guarantees for shuffled and online PNSGD, reducing noise over time.

The paper analyzes the variance of different shuffling methods in stochastic gradient descent.

Optimizer memory affects learning rate sensitivity in shuffle order, impacting fine-tuning noise.

Balls-and-Bins sampling improves DP-SGD privacy and utility.

GROVER improves word representations by gradually adding random noises during training.

Study phase transitions in shuffled regression problems.

Paper tackles sparse recovery with shuffled labels, establishing statistical and computational limits.

Federated learning promises to make machine learning feasible on distributed, private datasets by implementing gradient descent using secure aggregation methods. The idea is to compute a global weight update without revealing the contributions of individual users. Current practical protocols for secure aggregation work…

Unified framework for word embedding models using noise examples.

The paper proposes an efficient estimator for linear regression with shuffled labels.

The paper explores how low-degree polynomials can detect shuffled linear regression models.

Develops shuffling gradient-based methods for nonconvex-concave minimax optimization.

SMG combines shuffling and momentum for non-convex optimization.

A new algorithm improves convergence rates for convex optimization problems.

A new Gaussian mechanism for differential privacy in the shuffle model is introduced.

Improves privacy amplification by shuffling for differential privacy.

When using stochastic gradient descent to solve large-scale machine learning problems, a common practice of data processing is to shuffle the training data, partition the data across multiple machines if needed, and then perform several epochs of training on the re-shuffled (either locally or globally) data. The above …

Paper proposes a method to handle linear regression with partially shuffled data.

New shuffling methods improve convergence without Lipschitz smoothness.

This work studies differential privacy in the context of the recently proposed shuffle model. Unlike in the local model, where the server collecting privatized data from users can track back an input to a specific user, in the shuffle model users submit their privatized inputs to a server anonymously. This setup yields…

Simpler, faster algorithm for uniformity testing in the shuffle model.

A new method resolves permutation issues in shuffled linear regression for large-scale applications.

BUDS balances privacy and utility by shuffling data, achieving strong privacy with minimal loss.

New convergence rates for shuffling gradient methods without strong convexity.

Optimal SGD rates achieved with shuffling, covering non-convex and convex cases.

The roles played by learning and memorization represent an important topic in deep learning research. Recent work on this subject has shown that the optimization behavior of DNNs trained on shuffled labels is qualitatively different from DNNs trained with real labels. Here, we propose a novel permutation approach that …

Unified analysis for shuffling-type gradient methods in optimization.

New defense method inspired by encryption improves visual classification accuracy.

New convergence bounds for shuffling-based SGD methods in distributed learning.

The so-called level crossing analysis has been used to investigate the empirical data set. But there is a lack of interpretation for what is reflected by the level crossing results. The fractional Gaussian noise as a well-defined stochastic series could be a suitable benchmark to make the level crossing findings more s…

Proposes methods to recover labels from shuffled networks using graph averages.

An exciting new development in differential privacy is the shuffled model, in which an anonymous channel enables non-interactive, differentially private protocols with error much smaller than what is possible in the local model, while relying on weaker trust assumptions than in the central model. In this paper, we stud…

New method to rank metrics on non-shuffled traffic.

Improved shuffling technique amplifies privacy guarantees for anonymous data contributions.

New method amplifies privacy in decentralized learning without centralized communication.

Improved shuffling gradient methods converge faster for nonsmooth convex optimization.

Proposes a text perturbation method using a Mahalanobis metric to balance privacy and utility.

Enhanced financial reward with shuffled feature CNN-DRL.

New neural model processes 2D data with long-range dependencies efficiently.

We study the performance of stochastic gradient descent (SGD) on smooth and strongly-convex finite-sum optimization problems. In contrast to the majority of existing theoretical works, which assume that individual functions are sampled with replacement, we focus here on popular but poorly-understood heuristics, which i…

The multivariate linear regression model with shuffled data and additive Gaussian noise arises in various correspondence estimation and matching problems. Focusing on the denoising aspect of this problem, we provide a characterization the minimax error rate that is sharp up to logarithmic factors. We also analyze the p…

In recent work, Cheu et al. (Eurocrypt 2019) proposed a protocol for $n$-party real summation in the shuffle model of differential privacy with $O_{ε, δ}(1)$ error and $Θ(ε\sqrt{n})$ one-bit messages per party. In contrast, every local model protocol for real summation must incur error $Ω(1/\sqrt{n})$, and there exist …

Power of network tests degrades when vertices are misaligned.

Random shuffle method boosts HF dataset size 10-21 times.

We consider the problem of inference in a linear regression model in which the relative ordering of the input features and output labels is not known. Such datasets naturally arise from experiments in which the samples are shuffled or permuted during the protocol. In this work, we propose a framework that treats the un…

The paper presents a method to preserve privacy in text analysis using calibrated noise.

Sampling without replacement speeds up optimization in minimax problems.