Paper analyzes D-SGD convergence with heterogeneous data and proposes topology learning.
problem Efficiently dealing with data heterogeneity in decentralized learning.
method Revisits D-SGD analysis, introduces neighborhood heterogeneity, and proposes topology learning.
result Formulates topology learning as a tractable optimization problem and demonstrates its effectiveness.
New algorithms for clustering and synthetic data generation of heterogeneous tabular datasets.
problem Clustering and generating synthetic data from heterogeneous tabular datasets with hidden cluster structure.
method Developed MMM and MMMsynth algorithms for clustering and synthetic data generation.
result MMMsynth algorithm outperforms other literature tabular-data generators and approaches real data performance.
Algorithm uncovers treatment effect heterogeneity in educational RD designs.
problem Discovering sources of treatment effect heterogeneity in regression discontinuity designs.
method Causal supervised machine learning algorithm to build a 'regression discontinuity tree'.
result Algorithm uncovers various sources of heterogeneity in the impact of attending a better secondary school.
Proposes a new method for evaluating and constructing hierarchical topic models.
problem Evaluation and construction of hierarchical topic models.
method Represent HTM as layers and edges, introduce quality measures, and develop a heterogeneous algorithm.
result The proposed heterogeneous algorithm significantly outperforms baseline approaches.
EM algorithm speeds up convergence in federated learning with heterogenous data.
problem Understanding convergence rates of federated learning algorithms under data heterogeneity.
method Characterized convergence rate of EM algorithm for FMLR model under various regimes.
result EM algorithm converges to ground truth with SNR ≥ √K in all regimes.
The paper proposes a mixture model with segmentation for heterogeneous functional data.
problem Heterogeneity in time and population for functional data.
method Mixture model with segmentation of time, maximum likelihood estimator, EM algorithm with dynamic programming.
result The method is consistent and identifiable, and illustrated on simulated and real datasets.
A distributed SGD method for heterogeneous networks with hubs and workers.
problem Learning in heterogeneous multi-level networks with worker heterogeneity and varying communication.
method Multi-Level Local SGD: distributed SGD with hub-and-spoke paradigm and hub averaging.
result The method converges with error dependent on worker heterogeneity, hub network topology, and iterations.
This paper evaluates CFL algorithms for handling data heterogeneity in federated learning.
problem Handling data heterogeneity among clients in federated learning.
method Comparative evaluation of two state-of-the-art CFL algorithms with a proposed taxonomy of data heterogeneities.
result Analysis of CFL performance across different heterogeneity scenarios using extrinsic clustering metrics.
New algorithms improve Gaussian mixture model estimation in DFL with heterogeneous data.
problem Bias in EM algorithm for Gaussian mixtures in DFL with heterogeneous data.
method MNEM and semi-MNEM algorithms integrating historical and partially labeled data.
result MNEM and semi-MNEM achieve asymptotic efficiency and improved convergence.
A new algorithm optimizes local objectives in federated learning with heterogeneous clients.
problem Optimizing local objectives in federated learning with heterogeneous client data.
method Proposes PF-PNE algorithm with double elimination strategy.
result PF-PNE algorithm optimizes local objectives with arbitrary heterogeneity and protects client data confidentiality.
New algorithm estimates treatment effects for more efficient contextual bandits.
problem Contextual bandits struggle with action-independent reward redundancies.
method Reduces contextual bandits to heterogeneous treatment effect estimation.
result Heterogeneous treatment effect estimation leads to more efficient model estimation.
Study quantifies impacts of heterogeneity in FL on smartphone data.
problem Heterogeneity in FL devices causes performance degradation.
method Collected 136k smartphone data, built heterogeneity-aware FL platform, conducted extensive experiments.
result Heterogeneity causes up to 9.2% accuracy drop and 2.32x training time increase.
Study dynamic matching in heterogeneous networks using ODE model.
problem Dynamic matching in heterogeneous networks with compatibility restrictions.
method Introduced an ODE model to evaluate matching algorithms.
result Demonstrated trade-off between quick matching and optimal matching.
The thesis clarifies when local updates outperform centralized methods in heterogeneous data environments.
problem Understanding when local updates are more effective than centralized or mini-batch methods in distributed optimization.
method Fine-grained consensus-error-based analysis framework, focusing on bounded second-order heterogeneity and third-order smoothness.
result Local updates outperform centralized or mini-batch methods under realistic models of data heterogeneity.
New bucketing scheme improves Byzantine robustness for heterogeneous data.
problem Byzantine attacks on federated learning with heterogeneous data.
method Bucketing scheme to adapt robust algorithms to non-iid data.
result Bucketing scheme ensures convergence against Byzantine attacks.
Develops statistical inference for ML-discovered heterogeneous treatment effects.
problem ML algorithms may fail to accurately ascertain heterogeneous treatment effects in practical settings.
method Neyman's repeated sampling framework, dividing sample into groups, estimating average treatment effects, constructing confidence intervals.
result Valid methodology for estimating and testing heterogeneous treatment effects without relying on ML algorithm properties.
Framework assesses variable importance for heterogeneous treatment effects.
problem High-risk domains need reliable methods to assess treatment effect heterogeneity.
method Inferential framework based on Shapley values and semiparametric theory.
result Valid inference on variable importance for heterogeneous treatment effects.
Paper improves Bayesian inference in federated learning with new algorithm VR-FALD*.
problem Bayesian inference in federated learning with communication bottlenecks and statistical heterogeneity.
method Federated Averaging Langevin Dynamics (FALD) and VR-FALD*.
result VR-FALD* corrects client drift due to statistical heterogeneity, improving convergence.
New method quantifies variable importance in causal forests for treatment effect heterogeneity.
problem Lack of understanding how input variables affect treatment effect heterogeneity in causal forests.
method Developed a new importance variable algorithm for causal forests based on the drop and relearn principle.
result Shows how to handle forest retraining without a confounding variable and introduces a corrective term for confounders.
New algorithm learns from distributed, heterogeneous data without shuffling.
problem Efficiently learning from distributed, heterogeneous data in exascale simulations.
method Block-random gradient descent algorithm for distributed, heterogeneous data.
result Algorithm enables in situ learning without pre-shuffling data.
PerPCA separates unique and shared features from heterogeneous data.
problem Extracting shared and unique features from data collected from different sources with varying trends.
method Personalized PCA (PerPCA) uses orthogonal global and local principal components to encode both unique and shared features.
result PerPCA can identify and recover both unique and shared features under mild conditions.
Regularized MLE for MoE models tackles high-dimensional heterogeneous data.
problem Fitting and feature selection in Mixtures-of-Experts models for high-dimensional data.
method Proposes a regularized maximum likelihood estimation approach with hybrid EM/MM algorithms.
result Automatic recovery of sparse solutions without thresholding and matrix inversion.
Matrix decomposition is a popular and fundamental approach in machine learning and data mining. It has been successfully applied into various fields. Most matrix decomposition methods focus on decomposing a data matrix from one single source. However, it is common that data are from different sources with heterogeneous…
Two algorithms improve Federated RL in diverse environments.
problem Collaborative learning in environments with varying dynamics.
method Proposed two federated RL algorithms, QAvg and PAvg, and a personalization heuristic.
result Achieved better performance and generalization in diverse environments.
New algorithm tackles heterogeneous curvature in online convex optimization.
problem Adversarial bandit convex optimization with varying curvature.
method Developed an adaptive algorithm that learns curvature on the fly.
result Achieves optimal regret bounds even with heterogeneous curvature.
This study analyzes signSGD and medianSGD for heterogeneous data and proposes a noise correction mechanism.
problem The convergence of signSGD and medianSGD is non-convergent in distributed settings with heterogeneous data.
method The study analyzes signSGD and medianSGD for heterogeneous data and proposes a noise correction mechanism to overcome the convergence gap.
result The proposed noise correction mechanism provably closes the gap between mean and median of the gradients, leading to global convergence to stationary solutions.
Paper analyzes minimax risks of personalized federated learning algorithms.
problem Statistical heterogeneity among clients in federated learning.
method Minimax analysis of FedAvg and local training approaches.
result Threshold for optimality between FedAvg and local training depends on data heterogeneity.
A federated learning framework using superquantile aggregation for robust performance across heterogeneous data.
problem Robust predictive performance across clients with heterogeneous data.
method Superquantile-based learning objective and stochastic training algorithm with differential privacy.
result Proves finite time convergence guarantees and demonstrates competitive performance with tail statistics improvement.
Research shows collective learning across diverse environments is hard due to privacy and security concerns.
problem Privacy, security, and equity concerns restrict information sharing in diverse AI environments.
method Characterized learning algorithms as choice correspondences, provided minimum requirements for rational learning algorithms.
result The only rational learning algorithm in heterogeneous environments is unilaterally learning from a single environment without information sharing.
Proposes HeteroJIVE for joint subspace estimation in multi-view data with statistical and structural heterogeneity.
problem Joint subspace estimation in multi-view data with varying statistical and structural heterogeneity.
method HeteroJIVE: A weighted two-stage spectral algorithm addressing statistical and structural heterogeneity.
result HeteroJIVE achieves the O ( K − 1 / 2 ) O(K^{-1/2}) O ( K − 1/2 ) rate without iterative refinement, validating the oracle-optimal weighting scheme. Proposes a new method for medical diagnosis using network-based representation learning.
problem Improving medical diagnosis accuracy through better data representation.
method Heterogeneous network-based model and modified metapath2vec algorithm for learning latent node representations.
result Significant performance boost in symptom/disease classification and disease prediction tasks.
New methods handle both data and network heterogeneity in federated learning.
problem Challenges in federated learning due to data and network heterogeneity.
method Two novel client selection schemes that minimize theoretical runtime to convergence.
result Our methods are at least competitive to and up to 20 times better than existing baselines.
HGT models heterogeneous graphs with dedicated node and edge representations.
problem Infeasibility of existing GNNs for heterogeneous graphs.
method Node- and edge-type dependent parameters, relative temporal encoding, heterogeneous mini-batch graph sampling.
result HGT outperforms state-of-the-art GNNs by 9%--21% on various downstream tasks.
Paper introduces TEP to better model treatment effect heterogeneity.
problem Personalised decision making requires evidence of treatment suitability.
method Designs TEP to represent treatment effect heterogeneity, uses local causal structure to show important variables, derives formula for unbiased CATE estimation.
result Proposed method models treatment effect heterogeneity better than existing methods.
Robust Federated Learning tackles statistical and computational challenges in heterogeneous data and Byzantine machines.
problem Statistical and computational challenges in Federated Learning with heterogeneous data and Byzantine machines.
method Proposed a general statistical model for heterogeneous Federated Learning, solved the problem optimally, and proved statistical guarantees for outlier-robust clustering.
result Our algorithm matches the lower bound on estimation error and outperforms non-Byzantine-robust algorithms significantly.
A new method for embedding heterogeneous networks using spacey random walks.
problem Stationarity issues in meta-path guided random walks for HIN embedding.
method Heterogeneous personalized spacey random walk.
result Substantial improvement over existing network embedding algorithms.
A new algorithm for decentralized learning in heterogeneous networks reduces sub-optimality over time.
problem Learning in decentralized heterogeneous networks with local data streams and nonlinear constraints.
method Functional variant of stochastic primal-dual method with greedy subspace projection.
result The HALK algorithm achieves O ( T ) \mathcal{O}(\sqrt{T}) O ( T ) sub-optimality reduction and constraint satisfaction. A new method detects communities in multi-relational networks.
problem Detect communities in heterogeneous multi-relational networks.
method Message passing based algorithm on a hierarchical structure of homogeneous networks.
result Effectiveness of the proposed method confirmed on synthetic and real-world networks.
Bayesian machine learning algorithm for causal effects with imperfect compliance.
problem Heterogeneous causal effects in imperfect compliance scenarios.
method Bayesian Causal Forest with Instrumental Variable (BCF-IV) methodology.
result BCF-IV outperforms other techniques in discovering and estimating heterogeneous causal effects.
Adaptive sampling improves convergence in heterogeneous distributed optimization.
problem Poor performance of classical SGD and SVRG in heterogeneous distributed settings.
method Adaptive sampling of machines with an adaptive estimate of local Lipschitz constants.
result Significantly accelerates convergence rate from maximum to average Lipschitz constant.
New bounds for LDP with heterogeneous privacy levels guaranteeing high probability of accuracy.
problem Statistical estimation under LDP with users having varying privacy levels.
method Developed finite sample upper bounds in ℓ_2-norm with high probability, complemented by lower bounds.
result Optimal guarantees for heterogeneous LDP in terms of probability and constants.
Paper analyzes \FedAvg's convergence and introduces a new algorithm to reduce bias.
problem Analyzing convergence and bias in Federated Averaging.
method Markov property, first-order bias expansion, Richardson-Romberg extrapolation.
result Bias in \FedAvg can be decomposed into noise and client heterogeneity components.
A novel decentralized algorithm improves minimax optimization in federated learning.
problem Minimax optimization in federated learning with data heterogeneity.
method Decentralized Gradient Tracking (K-GT-Minimax) for nonconvex-strongly-concave optimization.
result Demonstrates superior convergence rate for NC-SC minimax optimization.
A post-hoc framework improves model performance by calibrating different feature spaces.
problem Improving AUC performance on binary classification tasks for overconfident models.
method Identifies heterogeneous partitions of the feature space and applies post-hoc calibration techniques to each partition.
result Theoretical optimality of the framework for any model, demonstrated on deep neural networks.
Generative models unify heterogeneous data for multimodal fusion.
problem Learning effective representations of mixed numerical and categorical data.
method Bayesian approach with exponential family distributions and Laplace-Bernstein approximation.
result Generative models enable fusion of multimodal data for various machine learning tasks.
Unified spectral clustering for sparse networks with heterogeneous degrees.
problem Efficiently detecting communities in sparse networks with varying degrees.
method Developed a parametrized regularized Laplacian matrix for spectral clustering.
result Improved parametrization accounts for network heterogeneity and community hardness.
Optimizes antenna settings in heterogeneous cellular networks using RL.
problem Complex interactions between cells cause optimization challenges.
method Two-step approach: offline multi-agent mean field RL, online single-agent deep RL.
result Approaches multi-agent RL performance with fewer trials.
New algorithm resists Byzantine attacks in distributed SGD for heterogeneous data.
problem Byzantine attacks in distributed SGD for heterogeneous data.
method Polynomial-time outlier-filtering for robust mean estimation, new matrix concentration result.
result Tolerates up to 25% Byzantine workers and achieves optimal convergence rates.