A novel RNN model with shuffled hidden states.
problem Improving recurrent neural network performance.
method Permuting hidden states and applying learned functions.
result Competitive results compared to literature baselines.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “Hidden State Shuffling”
New method shows hidden state can significantly improve differential privacy in SGD.
problem Differential privacy in SGD with hidden state.
method Proves converging privacy bounds for hidden state SGD, using privacy amplification techniques.
result Privacy bound converges exponentially fast and is smaller than composition bounds.
Improves privacy amplification by shuffling for differential privacy.
problem Enhancing privacy guarantees in systems with anonymous data contributions.
method Theoretical and numerical analysis of Rényi differential privacy parameters and privacy amplification by shuffling.
result First asymptotically optimal analysis of Rényi differential privacy parameters for shuffled outputs.
Develops shuffling gradient-based methods for nonconvex-concave minimax optimization.
problem Nonconvex-concave minimax optimization problems.
method Two shuffling gradient-based algorithms for nonconvex-linear and nonconvex-strongly concave settings.
result Achieves state-of-the-art oracle complexity in nonconvex optimization and best-known complexity bounds for nonconvex-strongly concave setting.
SMG combines shuffling and momentum for non-convex optimization.
problem Non-convex finite-sum optimization problems.
method Shuffling Gradient-based method with momentum.
result Established state-of-the-art convergence rates for SMG.
Paper proves privacy guarantees for shuffled and online PNSGD, reducing noise over time.
problem Privacy amplification in shuffled and online PNSGD settings.
method Iterative analysis of PNSGD with hidden updates, proving privacy guarantees for shuffled and online settings.
result Privacy guarantees for shuffled and online PNSGD with reduced noise over time.
A new Gaussian mechanism for differential privacy in the shuffle model is introduced.
problem Improving differential privacy in distributed learning environments.
method Characterization and upper-bounding of Rényi differential privacy (RDP) for the shuffle Gaussian mechanism.
result The shuffle Gaussian mechanism provides improved privacy guarantees compared to existing methods.
A new algorithm improves convergence rates for convex optimization problems.
problem Convex optimization problems with finite-sum structure.
method Nesterov Accelerated Shuffling Gradient (NASG) integrating Nesterov's acceleration with different shuffling schemes.
result Improved convergence rate of O(1/T) for unified shuffling schemes.
When using stochastic gradient descent to solve large-scale machine learning problems, a common practice of data processing is to shuffle the training data, partition the data across multiple machines if needed, and then perform several epochs of training on the re-shuffled (either locally or globally) data. The above …
New shuffling methods improve convergence without Lipschitz smoothness.
problem Lack of convergence guarantees for shuffling methods under non-Lipschitz conditions.
method Revisit shuffling methods, prove convergence under general bounded variance condition.
result Matched current best-known convergence rates without Lipschitz smoothness.
This work studies differential privacy in the context of the recently proposed shuffle model. Unlike in the local model, where the server collecting privatized data from users can track back an input to a specific user, in the shuffle model users submit their privatized inputs to a server anonymously. This setup yields…
Simpler, faster algorithm for uniformity testing in the shuffle model.
problem Testing uniformity of data in the shuffle model with privacy constraints.
method Simplified analysis and use of privacy amplification via shuffling.
result An algorithm with the same guarantees but simpler and more streamlined.
A new method resolves permutation issues in shuffled linear regression for large-scale applications.
problem Estimating latent features through linear transformation with unknown permutations.
method Spectral matching method to align spectral components of measurement and feature covariances.
result Achieves accurate estimates in shuffled LS and LASSO settings with sufficient samples.
ShuffleNet is a state-of-the-art light weight convolutional neural network architecture. Its basic operations include group, channel-wise convolution and channel shuffling. However, channel shuffling is manually designed empirically. Mathematically, shuffling is a multiplication by a permutation matrix. In this paper, …
BUDS balances privacy and utility by shuffling data, achieving strong privacy with minimal loss.
problem Balancing privacy and utility in crowd-sourced statistical databases.
method One-hot encoding, iterative shuffling, loss estimation, risk minimization.
result Achieves for privacy, maintaining a privacy bound of .
New convergence rates for shuffling gradient methods without strong convexity.
problem Theoretical gap between shuffling gradient methods' empirical success and established convergence rates.
method Proved last-iterate convergence rates for shuffling gradient methods using function value gap.
result First last-iterate convergence rates for shuffling gradient methods without strong convexity.
Optimal SGD rates achieved with shuffling, covering non-convex and convex cases.
problem Optimizing finite-sum optimization problems with shuffling strategies.
method RandomShuffle and SingleShuffle algorithms for SGD, analyzing convergence rates.
result Minimax optimal convergence rates established, generalizing to non-convex costs.
Unified analysis for shuffling-type gradient methods in optimization.
problem Optimization of finite-sum problems using shuffling strategies.
method Unified convergence analysis for various shuffling methods.
result Improved convergence rates for nonconvex problems and matching rates for convex problems.
Optimizer memory affects learning rate sensitivity in shuffle order, impacting fine-tuning noise.
problem Optimizer memory affects the learning rate sensitivity in shuffle order, leading to fine-tuning noise.
method Isolated the mechanism of fixed-clock optimizer memory affecting the learning rate sensitivity in shuffle order, deriving a fit-free way to size the noise.
result Fixed-clock optimizers like AdamW produce a larger first-order noise channel compared to memoryless optimizers, affecting fine-tuning comparisons.
New convergence bounds for shuffling-based SGD methods in distributed learning.
problem Analyzing the performance of shuffling-based variants of SGD in distributed learning.
method Study of minibatch and local Random Reshuffling methods, proving convergence bounds and lower bounds.
result Shuffling-based variants converge faster than with-replacement sampling methods, and the bounds are tight.
Proposes methods to recover labels from shuffled networks using graph averages.
problem Recovering labels from a shuffled network using graph averages.
method Cluster networks into classes, then match the new graph to cluster-averages, minimizing the graph matching objective function.
result Higher fidelity matching performance when clustering networks into different classes.
An exciting new development in differential privacy is the shuffled model, in which an anonymous channel enables non-interactive, differentially private protocols with error much smaller than what is possible in the local model, while relying on weaker trust assumptions than in the central model. In this paper, we stud…
Hiding Among the Clones: A Simple and Nearly Optimal Analysis of Privacy Amplification by Shufflingcs.LG
Improved shuffling technique amplifies privacy guarantees for anonymous data contributions.
problem Enhancing privacy in systems where data is contributed anonymously.
method Developed a new approach to random shuffling that amplifies differential privacy guarantees.
result Achieved asymptotically optimal privacy amplification with nearly optimal dependence in ε.
Improved Last-Iterate Convergence of Shuffling Gradient Methods for Nonsmooth Convex Optimizationmath.OC
Improved shuffling gradient methods converge faster for nonsmooth convex optimization.
problem Improving convergence rates for nonsmooth convex optimization problems.
method Analysis of shuffling gradient methods, focusing on Random Reshuffle and Single Shuffle strategies.
result Shuffling gradient methods, particularly Random Reshuffle and Single Shuffle, converge faster than Proximal Gradient Descent for nonsmooth convex optimization.
Enhanced financial reward with shuffled feature CNN-DRL.
problem Improving reward in financial data using CNN-DRL.
method Applying shuffled features to financial data for CNN-DRL.
result Substantial enhancement in reward attainment.
New neural model processes 2D data with long-range dependencies efficiently.
problem Limited receptive field of convolutions for complex 2D tasks.
method Proposes Matrix Shuffle-Exchange network with layers and complexity.
result Exceeds convolutional and graph neural network baselines in long-range dependency modeling.
We study the performance of stochastic gradient descent (SGD) on smooth and strongly-convex finite-sum optimization problems. In contrast to the majority of existing theoretical works, which assume that individual functions are sampled with replacement, we focus here on popular but poorly-understood heuristics, which i…
In recent work, Cheu et al. (Eurocrypt 2019) proposed a protocol for -party real summation in the shuffle model of differential privacy with error and one-bit messages per party. In contrast, every local model protocol for real summation must incur error , and there exist …
Federated learning promises to make machine learning feasible on distributed, private datasets by implementing gradient descent using secure aggregation methods. The idea is to compute a global weight update without revealing the contributions of individual users. Current practical protocols for secure aggregation work…
Power of network tests degrades when vertices are misaligned.
problem Power loss in network hypothesis testing due to vertex shuffling.
method Theoretical analysis and simulations of Frobenius norm differences in random dot product and stochastic block models.
result Shuffling vertices can significantly reduce the power of network tests.
Random shuffle method boosts HF dataset size 10-21 times.
problem Narrow dataset cardinality in HF clinical studies.
method Random shuffle method to enhance dataset size.
result Enhanced HF dataset size 10-21 times.
WASH trains ensembles with shuffled weights to improve accuracy and reduce communication.
problem Training ensembles for weight averaging leads to models converging to different loss basins.
method WASH randomly shuffles a small percentage of weights during training to keep models within the same basin.
result WASH achieves state-of-the-art image classification accuracy with lower communication costs.
We consider the problem of inference in a linear regression model in which the relative ordering of the input features and output labels is not known. Such datasets naturally arise from experiments in which the samples are shuffled or permuted during the protocol. In this work, we propose a framework that treats the un…
Sampling without replacement speeds up optimization in minimax problems.
problem Optimizing minimax problems with faster convergence rates.
method Analysis of gradient descent ascent and proximal point method with two sampling strategies.
result Sampling without replacement leads to faster convergence rates in minimax optimization.
The paper analyzes the variance of different shuffling methods in stochastic gradient descent.
problem Understanding the variance of different shuffling methods in stochastic gradient descent.
method Power spectral density analysis to study the noise sequences of stochastic gradients.
result The stationary variances of iterates decrease in the order of SGD, SGD-RR, and SGD-SO.
Brunnian braids and the inclusion from double shuffle Lie algebra to Kashiwara-Vergne Lie algebramath.QA
Proof of injection from double shuffle to Kashiwara-Vergne Lie algebra.
problem Injecting double shuffle Lie algebra into Kashiwara-Vergne Lie algebra.
method Inclusion of brunnian braids group on different genus 0 surfaces, using lower central series of brunnian Lie algebras, and explicit links between maps.
result Injection of double shuffle Lie algebra into symmetric Kashiwara-Vergne Lie algebra.
Meta-learning improves by adding more data to tasks.
problem Overfitting to meta-training tasks impairs generalization to novel tasks.
method Task augmentation methods (MetaMix and Channel Shuffle) are proposed to improve generalization.
result MetaMix and Channel Shuffle outperform state-of-the-art results across multiple datasets.
Regularization for Shuffled Data Problems via Exponential Family Priors on the Permutation Groupstat.ML
New method uses exponential family priors to handle shuffled data problems.
problem Handling mismatch errors in record linkage of two data files.
method Flexible exponential family prior on the permutation group for regularization.
result The proposed method outperforms competing methods in synthetic and real data.
Balls-and-Bins sampling improves DP-SGD privacy and utility.
problem Improving privacy and utility in DP-SGD implementations.
method Introducing Balls-and-Bins sampling as an alternative to shuffling in DP-SGD.
result Balls-and-Bins sampling achieves utility comparable to shuffling while offering better privacy amplification.
Is it possible to perform linear regression on datasets whose labels are shuffled with respect to the inputs? We explore this question by proposing several estimators that recover the weights of a noisy linear model from labels that are shuffled by an unknown permutation. We show that the analog of the classical least-…
Markov Observation Modelsstat.ML
Expands Hidden Markov Model to include Markov chain observations.
problem Handling Markov chain observations in Hidden Markov Models.
method Developed Expectation-Maximization algorithm and Viterbi algorithm analogs.
result Estimates transition probabilities for hidden states and observations.
This paper develops a path-first theory using signatures and jump lifts for self-exiting processes.
problem Developing a universal coordinate system for various types of paths and processes.
method Using signatures, jump lifts, and expected signatures, the paper presents a geometricity framework with algebraic properties and obstructions.
result The framework links various mathematical concepts and offers four main contributions to understanding and modeling self-exiting processes.
New defense method inspired by encryption improves visual classification accuracy.
problem Conventional defenses reduce accuracy and are defeated by obfuscated gradients.
method Block-wise pixel shuffling with secret key for training and test images.
result Achieves high accuracy (91.55%) on clean images and (89.66%) on adversarial examples.
Detects anomalies in multiple processes using hidden Markov models.
problem Detecting an anomalous process among many with hidden states.
method Sequential search strategy using ADHM algorithm.
result ADHM algorithm effectively leverages temporal correlations.
Random Reshuffling outperforms Stochastic Gradient Descent in smooth convex optimization.
problem Theoretical limitations of Random Reshuffling in smooth convex optimization.
method Random Reshuffling (RR) as a variant of Shuffling Stochastic Gradient Descent (Shuffling SGD).
result Random Reshuffling (RR) dominates Stochastic Gradient Descent (SGD) in smooth convex optimization under any reasonable stepsize after any finite number of epochs.
The paper introduces FMCI and hybrid decoding for hidden Markov models.
problem Computing distributions and decoding hidden state sequences in HMMs.
method Finite Markov chain imbedding (FMCI) and hybrid decoding.
result Hybrid decoding improves performance over traditional methods.
Paper improves privacy bounds for shuffle model using novel numerical techniques.
problem Improving privacy guarantees in the shuffle model of differential privacy.
method Develops and evaluates numerical techniques for tighter -differential privacy bounds.
result Accurately evaluates privacy loss distribution for adaptive compositions of shufflers.
This paper tackles hidden state inference for HMMs using particle filtering.
problem Inference for hidden states under HMMs is challenging due to unavailable true labels.
method Adaptive conformal inference framework using particle filtering.
result The framework produces prediction sets with specific aggregated coverage levels.