FedLog reduces communication in federated learning by sharing data summaries.
problem Significant communication overhead in federated learning with large model parameters.
method Shares minimal sufficient statistics via Bayesian inference and differential privacy.
result High learning accuracy with low communication overhead.
New protocols implement logical gates on encoded qubits with minimal overhead.
problem Efficiently performing universal logical gates on encoded qubits with minimal overhead.
method Using topological codes associated to hyperbolic surfaces, we introduce protocols to implement Dehn twists through constant depth unitary circuits.
result Demonstrated the possibility of applying universal logical gate sets on encoded qubits through constant depth unitary circuits and with constant space overhead.
Direct convolution eliminates memory overhead and improves performance.
problem Memory overhead and suboptimal performance in convolution layers.
method Implementing direct convolution without additional memory overhead.
result Performance improvement between 10% to 400% on various architectures.
Deep learning reduces training overhead in massive MIMO systems.
problem Reducing training overhead in massive MIMO systems.
method Use of deep learning (NNs) to improve CSI acquisition and feedback processes.
result Significant improvements in performance and reduced complexity.
We consider the problem of learning a binary classifier from n n n different data sources, among which at most an η η η fraction are adversarial. The overhead is defined as the ratio between the sample complexity of learning in this setting and that of learning the same hypothesis class on a single data distribution. We pr…
We study the collaborative PAC learning problem recently proposed in Blum et al.~\cite{BHPQ17}, in which we have k k k players and they want to learn a target function collaboratively, such that the learned function approximates the target function well on all players' distributions simultaneously. The quality of the col…
New method reduces parameter overhead for Bayesian neural networks.
problem High parameter overhead and difficulty of implementation in variational Bayesian neural networks.
method Constructs a general variational family for ensemble-based Bayesian neural networks that works well with batch normalization layers.
result Improves predictive accuracy and achieves almost perfect calibration on a ResNet-18 trained with ImageNet.
SRG improves optimization efficiency with reduced memory and computation overhead.
problem Limited applicability of variance-reduced optimization algorithms.
method SRG (stochastic reweighted gradient) using importance sampling.
result SRG outperforms SGD and provably improves convergence in strongly-convex cases.
SDN optimizes flow entries to reduce control plane overhead.
problem Efficiently manage flow entries in SDN switches with limited TCAM capacity.
method Proposes reinforcement learning algorithms to optimize flow entries.
result Achieves up to 60% reduction in long-term control plane overhead.
Federated LIDAR aided beam selection reduces mmWave beam search overhead.
problem Efficient link configuration in mmWave communication systems with reduced beam search overhead.
method Federated learning of LIDAR data to train a shared neural network for beam selection.
result Proposed method significantly outperforms previous works in performance and complexity.
New method estimates uncertainty without sampling for neural networks.
problem Estimating epistemic uncertainty in neural networks for safety-critical applications.
method Approximated Variance Propagation without sampling.
result Significantly reduces computational overhead compared to sampling-based methods.
Turbo-Aggregate reduces secure aggregation time from quadratic to nearly linear.
problem Quadratic overhead in secure model aggregation for federated learning.
method Multi-group circular strategy, additive secret sharing, and coding techniques.
result Achieves O ( N log N ) O(N\log{N}) O ( N log N ) overhead, compared to O ( N 2 ) O(N^2) O ( N 2 ) , for up to 50% user dropout. Local AdaAlter reduces communication in SGD with adaptive learning rates.
problem Communication overhead in distributed training.
method Novel SGD variant with adaptive learning rates and reduced communication.
result Empirically reduces communication overhead by up to 30%.
Deep learning predicts downlink channel from uplink data, reducing signaling overhead.
problem Large signaling overhead for full DL CSI in FDD MIMO.
method Deep learning-based channel extrapolation (prediction).
result Deep learning can infer DL CSI from UL CSI without additional overhead.
Efficient FPGA virtualization for deep learning reduces user isolation and overhead.
problem Poor isolation and heavy re-compilation overhead in FPGA-based DNN accelerators.
method Two-level instruction dispatch module, multi-core hardware resources pool, tiling-based instruction frame package, two-stage static-dynamic compilation.
result 1.07-1.69x and 1.88-3.12x throughput improvement over previous designs.
Paper proposes ADC framework to reduce ViT SL training communication overhead.
problem Reducing communication overhead in ViT SL training.
method Two parallel compression strategies: class-agnostic merging and token discarding.
result Significantly reduces communication overhead without sacrificing accuracy.
Improved SGD reduces communication overhead by sparsifying gradients.
problem Communication overhead in distributed optimization.
method Top-k sparsification and error compensation.
result Communication can be reduced by a factor of the problem dimension.
SOTERIA optimizes neural networks for secure inference with minimal overhead.
problem Protecting user privacy in ML-as-a-service models with low overhead.
method Neural architecture search with dual objectives of accuracy and cryptographic efficiency.
result SOTERIA constructs efficient models for secure inference.
ISAAC Newton uses input-based curvature for efficient training.
problem Efficient training in small-batch stochastic regimes.
method ISAAC Newton conditions gradients using selected second-order information based on input.
result Effective training even in small-batch stochastic regimes, competitive to first-order and second-order methods.
HCBM improves deep learning explainability by non-linear concept aggregation.
problem Lack of explainable and accurate predictions in deep learning for high-stake decisions.
method Introduce Hoeffding Concept Bottleneck Models (HCBM) using Hoeffding functional decomposition of gradient-boosted trees for non-linear and sparse concept aggregation.
result HCBM outperforms standard linear CBM and is robust to interconcept leakage.
Generative Adapter adapts LMs with a single forward pass, reducing inference overhead.
problem Efficient adaptation of large language models for new contexts.
method Generative Adapter directly maps new contexts to low-rank LM adapters via self-supervised learning.
result Significant reduction in inference overhead with no need for fine-tuning.
Paper uses ML to predict insulator flashover risk.
problem Predicting flashover risk of aging insulators.
method Supervised ML with XGBoost, using LC and voltage features.
result Model accurately estimates insulator flashover probability.
Meta-learning reduces training overhead for communication systems.
problem Inefficiency in machine learning due to frequent retraining when system configuration changes.
method Meta-learning selects a suitable inductive bias from related tasks, reducing training data and time requirements.
result Meta-learning can reduce training overhead for communication systems.
Information-theoretic Bayesian optimisation techniques have demonstrated state-of-the-art performance in tackling important global optimisation problems. However, current information-theoretic approaches require many approximations in implementation, introduce often-prohibitive computational overhead and limit the choi…
Paper introduces zero-space memory protection for CNNs without ECC overhead.
problem Ensuring reliability of CNNs in safety-critical applications.
method In-place zero-space ECC with weight distribution-oriented training.
result First known zero-space cost memory protection for CNNs.
Paper introduces a new adaptive gradient method with gradient compression for distributed training.
problem Communication overhead in distributed machine learning systems.
method Adaptive gradient method with gradient compression, scalable system BytePS-Compress.
result Convergence rate of O ( 1 / T ) \mathcal{O}(1/\sqrt{T}) O ( 1/ T ) for non-convex problems. Top-k sparsification reduces deep learning communication costs.
problem Reducing communication overhead in distributed deep learning.
method Extensive experiments and theoretical analysis of Top-k sparsification.
result A tighter bound for Top-k operator derived, improving scaling efficiency.
Ensuring differential privacy of models learned from sensitive user data is an important goal that has been studied extensively in recent years. It is now known that for some basic learning problems, especially those involving high-dimensional data, producing an accurate private model requires much more data than learn…
Distributed model training suffers from communication overheads due to frequent gradient updates transmitted between compute nodes. To mitigate these overheads, several studies propose the use of sparsified stochastic gradients. We argue that these are facets of a general sparsification method that can operate on any p…
PruneFL reduces FL training time on edge devices by pruning model size.
problem Limited computation and communication resources on edge devices in FL.
method Adaptive and distributed parameter pruning during FL process.
result Pruned model converges to similar accuracy as original model with reduced training time.
Sitatapatra blocks adversarial samples across different neural networks.
problem Adversarial samples can trick multiple neural networks trained on the same task.
method Sitatapatra diversifies neural networks using a cryptographic key and detects adversarial samples.
result Sitatapatra can trace back adversarial samples to the device that created them.
TorchGAN simplifies GAN training and evaluation with a flexible framework.
problem Training and evaluating GANs can be complex and time-consuming.
method Modular design, extensibility, built-in support for popular models and metrics.
result TorchGAN achieves near-zero overhead compared to vanilla PyTorch.
TG-PSM morphs encrypted traffic to reduce traffic analysis accuracy.
problem Traffic analysis on encrypted traffic reveals sensitive information.
method Clusters websites, finds target sites, morphs traffic using greedy algorithm.
result TG-PSM reduces attacker's accuracy significantly with low overhead.
New method reduces memory usage for adaptive optimization.
problem Significant memory overhead in adaptive optimizers like Adagrad and Adam.
method Memory-efficient adaptive optimization method that retains per-parameter adaptivity.
result Significantly larger models and batch sizes possible with up to 2-fold speedups.
A new FL algorithm reduces communication overhead by selectively updating model parameters.
problem Data heterogeneity and communication overhead in federated learning.
method Uses age of information metric to selectively update model parameters and group clients with similar data.
result Our method can expedite training and surpass other communication-efficient strategies in efficiency.
CBC makes CNNs robust against adversarial attacks with minimal computational overhead.
problem Making CNNs robust against adversarial attacks without increasing computational complexity.
method CBC uses a stacked encoder-convolutional model where an auto-encoder encodes the input image, and the latent representation is used for classification.
result CBC is more robust to adversarial examples and has significantly lower computational complexity.
Paper proposes neural network for efficient MIMO channel estimation and pilot reduction.
problem High overhead from pilot transmission in wideband MIMO systems.
method Neural network architecture for frequency-aware pilot design and channel estimation, with pruning technique.
result Neural network outperforms linear minimum mean square error (LMMSE) estimation.
QPyTorch simplifies low-precision training simulations.
problem Empirical evaluation of low-precision training algorithms.
method Native PyTorch framework with efficient fused-kernel approach.
result Supports various low-precision configurations and rounding options.
Meta-Neighborhoods adapts predictions based on input neighborhoods.
problem Adaptive prediction based on input neighborhoods for AI.
method Semi-parametric method with induced neighborhoods and meta-learning.
result Meta-Neighborhoods more accurately represents predictive distributions.
Distributed implementations of mini-batch stochastic gradient descent (SGD) suffer from communication overheads, attributed to the high frequency of gradient updates inherent in small-batch training. Training with large batches can reduce these overheads; however, large batches can affect the convergence properties and…
A new method for efficient online federated learning reduces communication overhead.
problem Real-world limitations in online federated learning, such as heterogeneous client participation and communication delays.
method Proposes a communication-efficient asynchronous online federated learning (PAO-Fed) strategy.
result Achieves the same convergence properties as online federated stochastic gradient while reducing communication overhead by 98 percent.
Clapping reduces memory usage in distributed optimization by reusing data samples.
problem Significant communication overhead and impractical memory overhead in pipeline-parallel distributed optimization.
method Lazy sampling strategy to reuse data samples across steps, supporting convergence without unbiased gradient assumptions.
result Clapping achieves convergence in few-epoch or online training regimes without sample-size memory overhead.
METRO predicts reactions using minimal templates, reducing computational overhead and achieving state-of-the-art results.
problem Predicting possible reaction substrates for complex molecules from simpler precursors.
method METRO (Molecule-Edit Templates for RetrOsynthesis) uses minimal templates to predict reactions efficiently and accurately.
result METRO achieves state-of-the-art results on standard benchmarks, reducing computational overhead.
New method reduces communication in deep learning training.
problem Communication overhead in distributed deep learning training.
method Random-block sparsification to reduce gradients communicated.
result Performance close to standard SGD with reduced communication.
One of the most significant bottleneck in training large scale machine learning models on parameter server (PS) is the communication overhead, because it needs to frequently exchange the model gradients between the workers and servers during the training iterations. Gradient quantization has been proposed as an effecti…
TCR improves DNN robustness to noisy labels with minimal overhead.
problem Training on noisy labeled datasets degrades DNN generalization.
method TCR combines original labels and previous epoch predictions for regularization.
result TCR consistently enhances DNN robustness to label noise.
New method gives high confidence bounds for stochastic convex optimization with minimal overhead.
problem Rare high probability guarantees in stochastic convex optimization.
method ProxBoost algorithm combining robust distance estimation and proximal point method.
result Wide class of stochastic optimization algorithms can achieve high confidence bounds with logarithmic and polylogarithmic overhead.
BayesAdapter turns pre-trained NNs into reliable BNNs with minimal overhead.
problem Scalability, accessibility, and reliability of Bayesian neural networks.
method Bayesian fine-tuning of pre-trained deterministic NNs to variational BNNs.
result BayesAdapter produces more reliable posteriors with less training overhead.