arXiv research

Develops shuffling gradient-based methods for nonconvex-concave minimax optimization.

A new algorithm improves convergence rates for convex optimization problems.

SMG combines shuffling and momentum for non-convex optimization.

New convergence rates for shuffling gradient methods without strong convexity.

New shuffling methods improve convergence without Lipschitz smoothness.

Unified analysis for shuffling-type gradient methods in optimization.

Improved shuffling gradient methods converge faster for nonsmooth convex optimization.

When using stochastic gradient descent to solve large-scale machine learning problems, a common practice of data processing is to shuffle the training data, partition the data across multiple machines if needed, and then perform several epochs of training on the re-shuffled (either locally or globally) data. The above …

The paper analyzes the variance of different shuffling methods in stochastic gradient descent.

Sampling without replacement speeds up optimization in minimax problems.

New convergence bounds for shuffling-based SGD methods in distributed learning.

A novel RNN model with shuffled hidden states.

We study the performance of stochastic gradient descent (SGD) on smooth and strongly-convex finite-sum optimization problems. In contrast to the majority of existing theoretical works, which assume that individual functions are sampled with replacement, we focus here on popular but poorly-understood heuristics, which i…

Optimal SGD rates achieved with shuffling, covering non-convex and convex cases.

Random Reshuffling outperforms Stochastic Gradient Descent in smooth convex optimization.

Unified analysis of asynchronous-SGD algorithms for distributed learning.

Federated learning promises to make machine learning feasible on distributed, private datasets by implementing gradient descent using secure aggregation methods. The idea is to compute a global weight update without revealing the contributions of individual users. Current practical protocols for secure aggregation work…

RR algorithm improves convergence rate without strong convexity assumptions.

A new method resolves permutation issues in shuffled linear regression for large-scale applications.

Improved shuffling technique amplifies privacy guarantees for anonymous data contributions.

New defense method inspired by encryption improves visual classification accuracy.

This paper improves privacy bounds for DP algorithms using $f$-DP.

This paper improves convergence guarantees for SGD algorithms in non-convex smooth functions.

Paper proves privacy guarantees for shuffled and online PNSGD, reducing noise over time.

This work studies differential privacy in the context of the recently proposed shuffle model. Unlike in the local model, where the server collecting privatized data from users can track back an input to a specific user, in the shuffle model users submit their privatized inputs to a server anonymously. This setup yields…

New method improves FL efficiency by shuffling data, balancing privacy and accuracy.

ShuffleNet is a state-of-the-art light weight convolutional neural network architecture. Its basic operations include group, channel-wise convolution and channel shuffling. However, channel shuffling is manually designed empirically. Mathematically, shuffling is a multiplication by a permutation matrix. In this paper, …

A new Gaussian mechanism for differential privacy in the shuffle model is introduced.

Improves privacy amplification by shuffling for differential privacy.

Random shuffle method boosts HF dataset size 10-21 times.

Enhanced financial reward with shuffled feature CNN-DRL.

Proposes methods to recover labels from shuffled networks using graph averages.

Simpler, faster algorithm for uniformity testing in the shuffle model.

New method amplifies privacy in decentralized learning without centralized communication.

BUDS balances privacy and utility by shuffling data, achieving strong privacy with minimal loss.

New method uses exponential family priors to handle shuffled data problems.

Optimizer memory affects learning rate sensitivity in shuffle order, impacting fine-tuning noise.

Proof of injection from double shuffle to Kashiwara-Vergne Lie algebra.

Most deep learning models are based on deep neural networks with multiple layers between input and output. The parameters defining these layers are initialized using random values and are "learned" from data, typically using stochastic gradient descent based algorithms. These algorithms rely on data being randomly shuf…

Balls-and-Bins sampling improves DP-SGD privacy and utility.

An exciting new development in differential privacy is the shuffled model, in which an anonymous channel enables non-interactive, differentially private protocols with error much smaller than what is possible in the local model, while relying on weaker trust assumptions than in the central model. In this paper, we stud…

FLAME improves privacy in federated learning without trusted parties.

Paper improves privacy bounds for shuffle model using novel numerical techniques.

New neural model processes 2D data with long-range dependencies efficiently.

Ranking metrics are a family of metrics largely used to evaluate recommender systems. However they typically suffer from the fact the reward is affected by the order in which recommended items are displayed to the user. A classical way to overcome this position bias is to uniformly shuffle a proportion of the recommend…

In recent work, Cheu et al. (Eurocrypt 2019) proposed a protocol for $n$-party real summation in the shuffle model of differential privacy with $O_{ε, δ}(1)$ error and $Θ(ε\sqrt{n})$ one-bit messages per party. In contrast, every local model protocol for real summation must incur error $Ω(1/\sqrt{n})$, and there exist …

Power of network tests degrades when vertices are misaligned.

We consider the problem of inference in a linear regression model in which the relative ordering of the input features and output labels is not known. Such datasets naturally arise from experiments in which the samples are shuffled or permuted during the protocol. In this work, we propose a framework that treats the un…