This paper tackles objective inconsistency in federated optimization with heterogeneous clients.
problem Objective inconsistency due to heterogeneity in clients' datasets and computation speeds.
method General framework for analyzing federated heterogeneous optimization algorithms, including FedAvg and FedProx, and proposing FedNova.
result FedNova eliminates objective inconsistency while preserving fast error convergence.
UNTIE learns representations of coupled categorical data.
problem Challenges in learning from unlabeled categorical data with complex couplings.
method UNTIE approach for unsupervised representation learning of heterogeneous couplings.
result UNTIE significantly improves categorical data representations on 25 diverse datasets.
DFFL tackles federated learning with heterogeneous objectives and constraints.
problem Federated learning with clients having different objectives and feasible regions.
method Derived heterogeneity bounds for cost-vector distances and support-function/shape-distance terms. Lifted pointwise bounds to local-versus-federated excess-risk comparison.
result Federation is beneficial when the statistical advantage of pooling exceeds a client-specific heterogeneity penalty.
Rescaled ASGD optimizes distributed learning under heterogeneous data.
problem Vanilla ASGD biases towards a frequency-weighted average of local objectives.
method Rescale worker stepsizes by their computation times.
result Rescaled ASGD converges to the correct global objective in fixed-computation model.
A new algorithm optimizes local objectives in federated learning with heterogeneous clients.
problem Optimizing local objectives in federated learning with heterogeneous client data.
method Proposes PF-PNE algorithm with double elimination strategy.
result PF-PNE algorithm optimizes local objectives with arbitrary heterogeneity and protects client data confidentiality.
CAP adapts optimization to class attributes for better fairness.
problem Heterogeneities across classes impede classification performance.
method CAP generates class-specific learning strategies based on attributes.
result CAP improves over naive approach and is competitive with prior art.
New method stabilizes private LASSO for high-dimensional data with diverse covariate scales.
problem Privacy constraints and heterogeneity in covariate scales degrade LASSO stability and accuracy.
method Gram-based anisotropic objective perturbation to counteract covariate structure.
result Significantly improves convergence and statistical efficiency of private LASSO estimators.
Minibatch SGD outperforms Local SGD in heterogeneous distributed learning.
problem Optimizing a combined convex objective with stochastic gradient estimates from different machines.
method Analysis of Minibatch SGD and Local SGD in a heterogeneous distributed setting.
result Minibatch SGD dominates Local SGD in the heterogeneous distributed setting.
Improved Local SGD convergence for general convex objectives with bounded second-order heterogeneity.
problem Understanding when and why Local SGD outperforms alternatives in distributed optimization.
method Established improved convergence guarantees for Local SGD on general convex objectives under bounded second-order heterogeneity.
result Upper bounds for Local SGD are nearly tight, providing a sharper convergence theory.
Collective classification has been intensively studied due to its impact in many important applications, such as web mining, bioinformatics and citation analysis. Collective classification approaches exploit the dependencies of a group of linked objects whose class labels are correlated and need to be predicted simulta…
Upper bounds on utility for managing heterogeneous collectivised funds.
problem Managing pension funds with diverse investor preferences and mortality.
method Axiomatic approach to define optimal management strategies.
result Asymptotically optimal strategies for maximizing investor utility.
A distributed SGD method for heterogeneous networks with hubs and workers.
problem Learning in heterogeneous multi-level networks with worker heterogeneity and varying communication.
method Multi-Level Local SGD: distributed SGD with hub-and-spoke paradigm and hub averaging.
result The method converges with error dependent on worker heterogeneity, hub network topology, and iterations.
The rising use of deep learning and other big-data algorithms has led to an increasing demand for hardware platforms that are computationally powerful, yet energy-efficient. Due to the amount of data parallelism in these algorithms, high-performance 3D manycore platforms that incorporate both CPUs and GPUs present a pr…
ie-HGCN addresses HIN challenges by efficiently learning node representations.
problem Lack of flexibility in exploring meta-paths and high computational complexity in HIN GCN methods.
method Hierarchical aggregation architecture that automatically extracts useful meta-paths and reduces computational cost.
result ie-HGCN outperforms state-of-the-art methods on real network datasets.
Latent feature modeling allows capturing the latent structure responsible for generating the observed properties of a set of objects. It is often used to make predictions either for new values of interest or missing information in the original data, as well as to perform data exploratory analysis. However, although the…
The problem of content search through comparisons has recently received considerable attention. In short, a user searching for a target object navigates through a database in the following manner: the user is asked to select the object most similar to her target from a small list of objects. A new object list is then p…
A new method detects communities in multi-relational networks.
problem Detect communities in heterogeneous multi-relational networks.
method Message passing based algorithm on a hierarchical structure of homogeneous networks.
result Effectiveness of the proposed method confirmed on synthetic and real-world networks.
Study on consistency of ML methods for moving objects in non-stationary environments.
problem Consistency of machine learning methods for moving objects in non-stationary environments.
method Least squares, ridge regression, and ℓs-penalized least squares methods under non-stationary spatial-temporal sampling. result Consistency and asymptotic normality of the estimates under weak conditions.
New model detects communities in networks with signed, continuous weights.
problem Detect communities in networks with signed, continuous weights.
method Heterogeneous Block Covariance Model (HBCM) with variational EM algorithm.
result Provable consistent estimates of group memberships.
Proposes KMvDA for object recognition from multi-view data.
problem Recognizing objects from different views, even when views are heterogeneous.
method Introduces kernel multi-view discriminant analysis (KMvDA) and uses random Fourier features (RFF) for large-scale learning.
result KMvDA and RFF approximation improve object recognition from multi-view data.
User-based attribute information, such as age and gender, is usually considered as user privacy information. It is difficult for enterprises to obtain user-based privacy attribute information. However, user-based privacy attribute information has a wide range of applications in personalized services, user behavior anal…
Federated learning calibrates insurance indices from renewable energy producers' data.
problem Calibrating parametric insurance indices under heterogeneous renewable energy production losses.
method Federated learning framework using Tweedie GLMs and distributed optimization.
result Federated learning recovers comparable index coefficients under moderate heterogeneity.
Paper tackles federated learning with personalised bandit algorithms.
problem Optimizing local and global objectives in a heterogeneous environment.
method Surrogate objective function combining client preferences and global knowledge; phase-based elimination algorithm.
result Achieves sublinear regret with logarithmic communication overhead.
A new federated algorithm reduces regret in X-armed bandit problems.
problem Collaborative optimization of heterogeneous local objectives.
method Fed-PNE algorithm using hierarchical partitioning and weak smoothness.
result Achieves sublinear cumulative regret with minimal communication.
New method for predicting paths of unpredictable objects with high confidence.
problem Need for dependable uncertainty estimates in motion planning with diverse unpredictable objects.
method Blend online conformal prediction, multiple time series techniques, and heteroscedasticity addressing.
result Simultaneous forecasting bands that cover entire paths with high probability.
Tree ensemble method tackles multi-objective constrained optimization in energy systems.
problem Complex, multi-objective, and constrained optimization problems in energy systems.
method Data-driven tree ensemble approach for black-box problems with heterogeneous variable spaces.
result Competitive performance and sampling efficiency compared to state-of-the-art tools.
LATTE tackles heterogeneous network embedding challenges with layer-stacked attention.
problem Aggregating higher-order indirect relations in heterogeneous networks.
method Layer-stacked ATTention Embedding (LATTE) that decomposes meta relations at each layer.
result LATTE achieves state-of-the-art performance on benchmark datasets.
CyBeR-0 optimizes federated learning with Byzantine resilience and reduced communication costs.
problem Byzantine attacks and communication inefficiency in federated learning.
method Transformed robust aggregation for zero-order optimization under client heterogeneity.
result CyBeR-0 achieves stable performance with minimal communication costs and reduced memory usage.
PANIS learns PDE surrogates for heterogeneous materials without solving the PDE.
problem Learning surrogates for parametrized PDEs in heterogeneous media.
method Physics-aware neural implicit solvers combining probabilistic learning and physics-informed discretization.
result Learned surrogates for effective solutions in heterogeneous materials without solving the reference problem.
This paper introduces a general Bayesian non- parametric latent feature model suitable to per- form automatic exploratory analysis of heterogeneous datasets, where the attributes describing each object can be either discrete, continuous or mixed variables. The proposed model presents several important properties. First…
A federated learning framework using superquantile aggregation for robust performance across heterogeneous data.
problem Robust predictive performance across clients with heterogeneous data.
method Superquantile-based learning objective and stochastic training algorithm with differential privacy.
result Proves finite time convergence guarantees and demonstrates competitive performance with tail statistics improvement.
Network representation learning (NRL) has been widely used to help analyze large-scale networks through mapping original networks into a low-dimensional vector space. However, existing NRL methods ignore the impact of properties of relations on the object relevance in heterogeneous information networks (HINs). To tackl…
This paper evaluates CFL algorithms for handling data heterogeneity in federated learning.
problem Handling data heterogeneity among clients in federated learning.
method Comparative evaluation of two state-of-the-art CFL algorithms with a proposed taxonomy of data heterogeneities.
result Analysis of CFL performance across different heterogeneity scenarios using extrinsic clustering metrics.
A federated minimax framework for heterogeneous clients.
problem Training with edge devices having different datasets and capabilities.
method Proposes a federated minimax optimization framework with normalized updates.
result Improves convergence and communication complexity for nonconvex functions.
Calculating similarities between objects defined by many heterogeneous data modalities is an important challenge in many multimedia applications. We use a multi-modal topic model as a basis for defining such a similarity between objects. We propose to compare the resulting similarities from different model realizations…
PRISM integrates diverse rewards in MORL, improving sample efficiency and Pareto coverage.
problem Heterogeneous MORL where dense objectives dominate, leading to poor sample efficiency.
method PRISM uses reflectional symmetry and ReSymNet to reconcile temporal-frequency mismatches and accelerate exploration.
result PRISM consistently outperforms sparse-reward baselines and oracles, achieving significant Pareto gains.
Scaff-PD improves fairness and robustness in federated learning with reduced communication.
problem Improving fairness and robustness in federated learning with limited communication.
method Scaff-PD uses a family of distributionally robust objectives and an accelerated primal dual algorithm with bias-corrected steps.
result Scaff-PD achieves significant gains in communication efficiency and convergence speed while maintaining fairness and robustness.
Develops wcPCA for better low-rank approximations in heterogeneous domains.
problem Worst-case performance of PCA in domains with distributional shifts.
method Unified framework (wcPCA) for worst-case optimization, applied to norm-minPCA and norm-maxregret.
result Empirical and theoretical worst-case optimality for low-rank approximations.
PCL tackles collaborative learning for diverse agents, reducing sample complexity.
problem Balancing collaborative speedup with personalization for heterogeneous agents.
method AffPCL, with bias and importance correction mechanisms.
result AffPCL reduces sample complexity by a factor of max{n−1,δ}, where n is the number of agents and δ∈[0,1] measures heterogeneity. Bayesian optimization improves with transfer learning for aircraft design.
problem Cold start problem in Bayesian optimization for aircraft design.
method Ensemble of surrogate models using transfer learning in a constrained Bayesian optimization framework.
result Significant improvement in convergence and prediction accuracy.
The Bayesian approach to feature extraction, known as factor analysis (FA), has been widely studied in machine learning to obtain a latent representation of the data. An adequate selection of the probabilities and priors of these bayesian models allows the model to better adapt to the data nature (i.e. heterogeneity, s…
Flexible estimation of heterogeneous treatment effects lies at the heart of many statistical challenges, such as personalized medicine and optimal resource allocation. In this paper, we develop a general class of two-step algorithms for heterogeneous treatment effect estimation in observational studies. We first estima…
We propose a federated learning framework to handle heterogeneous client devices which do not conform to the population data distribution. The approach hinges upon a parameterized superquantile-based objective, where the parameter ranges over levels of conformity. We present an optimization algorithm and establish its …
The paper tackles robust policy learning from multiple data sources.
problem Learning a policy that generalizes across diverse settings from multiple heterogeneous data sources.
method Proposes a minimax regret optimization objective and a policy learning algorithm combining doubly robust offline policy evaluation and no-regret learning.
result Achieves minimal worst-case mixture regret up to a moderated vanishing rate of the total data across all sources.
SUOD accelerates OD for large, diverse models.
problem Training and scoring new samples with many unsupervised, heterogeneous OD models.
method Data reduction, model approximation, and taskload optimization.
result SUOD accelerates OD for over 20 benchmark datasets and a real-world case.
A new method estimates treatment effects across multiple studies considering differences.
problem Estimating treatment effects across multiple studies with varying conditions.
method The multi-study R-learner framework that accounts for between-study heterogeneity.
result The multi-study R-learner is more efficient and normal than existing methods in the presence of heterogeneity.
ARCO-BO optimizes multi-agent design under heterogeneity, improving efficiency and performance.
problem Heterogeneous multi-agent optimization challenges in resource use and information sharing.
method ARCO-BO integrates a consensus mechanism, budget-aware sampling, and partial input sharing for heterogeneous design spaces.
result ARCO-BO outperforms independent and collaborative BO methods in complex multi-agent settings.
Improved FDAM algorithms for heterogeneous data with constant communication complexity.
problem Maximizing AUC for imbalanced data classification in federated learning.
method Solving non-convex strongly-concave min-max formulation in a distributed fashion.
result Communication complexity is a constant, independent of number of machines and accuracy level.