Paper proposes efficient weight updates for edge nodes with minimal communication.
problem Inefficient and resource-intensive full weight updates for edge nodes.
method Deep partial updating, selecting a subset of weights to update.
result Achieves similar performance with fewer weight updates.
On-device federated learning updates edge models by exchanging trained results.
problem Limited training data at edge devices due to model drift.
method OS-ELM for sequential training and autoencoder for anomaly detection, combined with federated learning.
result The proposed approach produces a merged model as accurately as traditional methods with lower costs.
Study on network-valued processes with asynchronous updates, proving consistency in community and changepoint estimation.
problem Understanding the behavior of network-valued stochastic processes with asynchronous updates.
method Analysis of concentration properties of aggregated adjacency and Laplacian matrices for lazy network-valued stochastic processes.
result Demonstrates consistency of estimators in community and changepoint estimation problems.
New algorithms update dynamic graph regression faster than existing methods.
problem Efficiently updating linear regression solutions for dynamic graphs.
method Subsampled randomized Hadamard transform and CountSketch.
result First sublinear update time randomized algorithms for dynamic graph regression.
This paper studies communication efficiency in federated learning by optimizing the sum-rate-distortion function for indirect multiterminal source coding.
problem Indirect multiterminal source coding in federated learning where edge devices send noisy gradients to the server.
method Analyzes the rate region for the quadratic vector Gaussian CEO problem under unbiased estimator and derives an explicit formula for the sum-rate-distortion function.
result Derives an explicit formula for the sum-rate-distortion function in the special case of identical gradients over edge devices.
Improved molecular property prediction using updated neural message passing.
problem Predicting properties of molecules and materials accurately.
method Extended neural message passing model with edge update network.
result Superior prediction of formation energies and other properties on multiple datasets.
Edge devices learn a global model collaboratively over wireless channels.
problem Learning a global model from edge devices with imperfect channel state information.
method Proposed analog aggregation scheme, receive beamforming at PS, and convergence analysis.
result Performance improvement with more PS antennas, even with imperfect CSI.
Hierarchical FL reduces latency in HCNs by sharing model updates.
problem Latency and privacy issues in federated learning across heterogeneous cellular networks.
method Hierarchical federated learning, gradient sparsification, periodic averaging.
result Significant reduction in communication latency without compromising model accuracy.
The success of deep learning models is heavily tied to the use of massive amount of labeled data and excessively long training time. With the emergence of intelligent edge applications that use these models, the critical challenge is to obtain the same inference capability on a resource-constrained device while providi…
This paper presents a method to summarize directed graphs while preserving edge information.
problem Summarizing directed graphs while maintaining edge directionality.
method A model based on minimizing reconstruction error with non-negative constraints, related to Max-Cut criterion, using multiplicative update algorithms.
result The proposed method identifies compressed nodes and directed compressed relations, providing a more accurate representation of directed graphs.
IGNN improves GNNs by maximizing edge-state transform mutual information.
problem Optimizing GNNs for better relational information.
method Variational information maximization to learn optimal transform parameters.
result IGNN achieves state-of-the-art performance on molecular graph tasks.
Random projections of labels enable feedforward training of deep neural networks without feedback.
problem Biological plausibility and low-cost adaptive smart sensors constrained by backpropagation.
method Fixed random projections of targets for feedforward training of hidden layers.
result DRTP algorithm provides a tradeoff between accuracy and computational cost suitable for edge computing.
Paper studies efficient regression for dynamic graphs.
problem Regression over dynamic graphs.
method Update-efficient matrix embedding, O(nm) time complexity for updates.
result Exact optimal solution can be updated efficiently in dynamic graphs.
A new graph neural network (NBA-GNN) avoids revisiting nodes to improve accuracy.
problem Redundancy in graph neural network updates causes over-squashing and inaccurate recognition.
method Proposes non-backtracking graph neural networks (NBA-GNN) that update messages without revisiting nodes.
result The NBA-GNN alleviates over-squashing and improves performance on graph benchmarks.
Improved prediction accuracy in matrix factorization using graph-based priors.
problem Graph side-information may not align with latent-feature relations in matrix completion.
method Identify and remove 'contested' edges using graphical lasso approximation, maintaining linear scalability.
result Improved prediction accuracy with fewer graph edges, demonstrating the often inaccurate nature of graph side-information.
Deep Graphs learn graph algorithms adaptively.
problem Efficiently learning graph algorithms for various tasks.
method Adaptive iterative vertex updates based on graph algorithms.
result Achieves excellent accuracy on graph labeling and regression tasks.
Enhances GNNs by improving input data quality from topology and labels.
problem Poor quality of graph data limits GNN performance.
method Improves graph data quality using model outputs for better semi-supervised node classification.
result SEG consistently improves GNN performance across various datasets.
AMS improves video inference on edge devices by adapting a small model with online knowledge distillation.
problem High computation cost of Deep Neural Networks for real-time video inference on edge devices.
method AMS uses a remote server to continually train and adapt a small model on edge devices, using online knowledge distillation from a large model.
result 0.4--17.8 percent mean Intersection-over-Union improvement in video semantic segmentation.
Efficiently updates vertex representations for dynamic graphs using random walks.
problem Updating vertex representations for dynamic graphs without re-generating them on each update.
method Proposes algorithms that extend random walk-based methods to dynamic graphs, considering the extent and rate of changes.
result Achieves competitive results to state-of-the-art methods while being computationally efficient.
GD converges in unstable regimes, even with oscillatory behavior.
problem Understanding convergence of GD in unstable regimes.
method Analysis of two-step gradient updates.
result Characterization of local conditions for convergence.
Gradient descent forces neural network eigenvalues to a specific threshold.
problem Understanding why gradient descent drives eigenvalues to a specific threshold.
method Introduced edge coupling, a functional on consecutive iterate pairs, to explain the trajectory towards the eigenvalue threshold.
result Gradient descent forces the Hessian eigenvalue to the threshold 2 / η 2/η 2/ η from arbitrary initialization. Local adaptive methods in FL can accelerate convergence but introduce bias, which is corrected.
problem The effect of using adaptive optimization methods for local updates in federated learning.
method Proposed correction techniques to overcome the bias introduced by local adaptive methods.
result Correction techniques can achieve faster convergence and higher test accuracy than baseline methods.
We tackle robust influence maximization in social networks with hyperparametric edge probabilities.
problem Maximizing worst-case influence in social networks with hyperparametric edge probabilities.
method Proposed a model with NP-hard proper robust optimization, using sampling and multiplicative weight updates.
result Empirically validated method outperforms state-of-the-art robust influence maximization techniques.
Real-time anomaly detection for edge streams using MIDAS and MIDAS-F.
problem Detecting unusual behavior in dynamic graph edge streams.
method MIDAS-F focuses on microcluster anomalies, incorporating anomalies into internal states to reduce poisoning effect.
result MIDAS-F outperforms MIDAS in accuracy, with up to 62% higher ROC-AUC.
Federated learning on edge devices achieves high accuracy with minimal data exchange.
problem Training deep neural networks on edge devices while maintaining user privacy.
method Training CNN, LSTM, and MLP on MNIST data using federated learning on edge devices (Raspberry Pi4s). Experimentally tested on IID and non-IID samples.
result Up to 85% test accuracy achieved with 2 minutes of training time and <10 MB data exchange per device.
The study analyzes sharpness dynamics in neural networks, revealing mechanisms and conditions.
problem Understanding sharpness in neural network training.
method Fixed point analysis and edge of stability analysis in a simplified 2-layer linear network.
result Reveals mechanisms behind sharpness trends, conditions for edge of stability, and a period-doubling route to chaos.
End-to-end graph-based SSL learns all graph factors dynamically.
problem Learning quality of graph in SSL is crucial but difficult.
method Proposes an end-to-end approach to optimize all graph factors.
result Demonstrates effectiveness on benchmark datasets.
Federated learning improves by unbiased gradient aggregation and controllable meta updating.
problem Gradient biases and inconsistency between target and optimization objectives in federated averaging.
method Unbiased gradient aggregation with keep-trace gradient descent and gradient evaluation strategy, controllable meta updating with small data samples.
result Faster convergence and higher accuracy with different network architectures in various FL settings.
Recent advances in Quantum Topology assign q q q -series to knots in at least three different ways. The q q q -series are given by generalized Nahm sums (i.e., special q q q -hypergeometric sums) and have unknown modular and asymptotic properties. We give an efficient method to compute those q q q -series that come from planar gra…
This article improves communication efficiency in distributed ML over wireless networks.
problem Achieving high ML inference accuracy at scale with zero communication latency.
method Optimizing communication payload types, techniques, scheduling, and ML architectures.
result Communication-efficient and distributed learning frameworks are presented.
DGNN models dynamic graphs by capturing sequential edge information.
problem Lack of models for dynamic graphs with evolving structure.
method Proposes DGNN, a dynamic graph neural network framework.
result DGNN effectively models dynamic graphs and improves performance.
A new energy-efficient pruning method for federated learning.
problem Energy inefficiency in gradient sparsification for federated learning.
method Formalized energy-constrained projection problem and proposed Cost-Weighted Magnitude Pruning (CWMP).
result CWMP optimally balances performance and energy efficiency in federated learning.
SGD updates align with a low-rank subspace but do not lead to further loss reduction.
problem Understanding the training dynamics of deep neural networks, particularly the role of the dominant subspace.
method Exploring whether neural networks can be trained within the dominant subspace of the loss Hessian.
result SGD updates, when projected onto the dominant subspace, do not decrease the training loss further, suggesting spurious alignment.
A novel approach for federated learning over-the-air computation to reduce latency and improve privacy.
problem Low-latency and privacy issues in edge machine learning for intelligent devices.
method Over-the-air computation and sparse-low-rank optimization for efficient global model aggregation.
result Efficient global model aggregation with low latency and improved privacy.
We introduce a new neural network model, together with a tractable and monotone online learning algorithm. Our model describes feed-forward networks for classification, with one output node for each class. The only nonlinear operation is rectification using a ReLU function with a bias. However, there is a rectifier on …
A novel method for learning Bayesian network structures from decentralized data, balancing privacy and efficiency.
problem Privacy and communication costs in learning Bayesian network structures from decentralized data.
method Fed-Sparse-BNSL, combining differential privacy with greedy updates targeting only a few relevant edges per participant.
result Achieves utility close to non-private baselines while offering stronger privacy and communication efficiency.
Emerging technologies and applications including Internet of Things (IoT), social networking, and crowd-sourcing generate large amounts of data at the network edge. Machine learning models are often built from the collected data, to enable the detection, classification, and prediction of future events. Due to bandwidth…
FedPAQ improves federated learning efficiency by averaging and quantizing updates.
problem Communication bottlenecks and scalability issues in federated learning.
method Periodic averaging, partial device participation, and quantized message-passing.
result FedPAQ achieves near-optimal theoretical guarantees and demonstrates communication-computation tradeoffs.
LoRA fine-tuning creates intruder dimensions that can cause forgetting, and a new law predicts when this happens.
problem Predicting when LoRA fine-tuning creates intruder dimensions that can cause catastrophic forgetting.
method Derived a per-layer critical update strength s ∗ s^\ast s ∗ and an exact secular-equation characterization of the updated spectrum. result The law localizes the empirical threshold within a factor of two on 82% of layers and separates intruder-bearing from intruder-free layers at deployment.
Two algorithms learn Gaussian graphical models from Glauber dynamics trajectories, achieving optimal performance.
problem Learning Gaussian graphical models from a single trajectory of a dependent stochastic process.
method Two algorithms based on dueling-neighborhood search and local statistics built from the update sequence of Glauber dynamics.
result Achieve κ − 2 κ^{-2} κ − 2 dependence of the information-theoretic lower bounds, mixing-free and signal-optimal. A major challenge in computed tomography (CT) is to reduce X-ray dose to a low or even ultra-low level while maintaining the high quality of reconstructed images. We propose a new method for CT reconstruction that combines penalized weighted-least squares reconstruction (PWLS) with regularization based on a sparsifying…
Algorithm learns Sherrington-Kirkpatrick model parameters at low temperatures.
problem Learning parameters of random graphical models at low temperatures.
method Multiplicative-weight update algorithm for polynomial time learning.
result Algorithm learns SK model parameters at β ≤ log n β\leq \sqrt{\log n} β ≤ log n . Paper develops a federated learning method to protect privacy without sacrificing model utility.
problem Privacy leakage in federated learning due to information exchange between edge devices and server.
method Combines local gradient perturbation, secure aggregation, and zCDP for privacy protection.
result Demonstrates superior trade-off between privacy and model utility through extensive experiments.
This paper evaluates and compares gradient leakage attacks in federated learning.
problem Gradient leakage attacks compromise client privacy in federated learning.
method Formal and experimental analysis of gradient leakage attacks, evaluation of attack effectiveness and cost.
result Gradient leakage attacks can reconstruct private local training data from shared parameter updates.
New algorithm updates eigenvectors of evolving graphs efficiently.
problem Updating eigenvectors of dynamic graphs.
method Subspace projection based on Rayleigh-Ritz projections.
result Strong performance in eigenvector approximation and downstream tasks.
Fast feature selection for SHM using canonical correlation.
problem Feature selection for structural health monitoring.
method Greedy search of sum of squared canonical correlation coefficients.
result Extremely fast feature selection with good performance.
Enhances neural architecture search efficiency and prevents performance collapse.
problem Improving memory efficiency and preventing performance collapse in neural architecture search.
method Employing continuous relaxation strategy and gradient-based optimization for over-parameterized BCNN construction, introducing Confident Learning Rate and partial channel connections.
result NAS-v2 delivers state-of-the-art search efficiency on CIFAR-10 and ImageNet.
Coded Federated Learning speeds up model convergence by preemptively computing on parity data.
problem Federated learning's convergence is slow on heterogeneous platforms due to stragglers.
method Develops CFL scheme where clients generate parity data and share it once, allowing the server to compute redundantly.
result CFL allows global model to converge nearly four times faster than uncoded federated learning.