Study of supervised learning from multiple non-independent sequences.
problem Efficient learning from many non-independent sequences.
method Generalizes conditions for efficient learning from independent examples and single auto-correlated sequences.
result Error rate changes from Θ(n/mT) to Ω(n2/m2T) as the number of trajectories increases. Derives derivatives and geometric framework for functions with non-independent variables.
problem Characterizing functions with non-independent variables in probabilistic models.
method Derives actual and dependent partial derivatives, dependent Jacobian matrix, and tensor metric.
result Derives gradient, Hessian, and Taylor expansion for functions with non-independent variables.
New framework extends ICA for non-independent variables, identifying pairwise mean independence.
problem Non-independent variables complicating ICA recovery.
method Algebraic recovery algorithm based on least-squares optimization over the orthogonal group.
result Pairwise mean independence is identifiable, robust to independence constraints.
Paper develops federated GLMM algorithms for analyzing hierarchical data.
problem Analyzing hierarchical data with non-independent observations in a federated setting.
method Developed two federated GLMM algorithms using Laplace and Gaussian Hermite approximations.
result Federated GLMM can handle hierarchical data and achieve comparable or superior performance.
New definition of disentanglement for non-independent factors of variation.
problem Current disentanglement definitions assume independent factors of variation, limiting their applicability.
method Definition based on information theory, related to Information Bottleneck Method, proposed measurement method.
result Proposed method correctly measures disentanglement with non-independent factors of variation.
Improved PAC-Bayesian bounds by considering example difficulty.
problem Improving generalization bounds in machine learning.
method Introducing a modified excess risk that leverages example difficulty to reduce variance and tighten PAC-Bayesian bounds.
result Tighter PAC-Bayesian generalization bounds for machine learning models.
MMD test detects adversarial attacks by addressing kernel limitations and non-independence issues.
problem MMD test's failure to detect adversarial attacks.
method Replaced Gaussian kernel with deep kernel, maximized test power, and used wild bootstrap for non-independence.
result MMD test is aware of adversarial attacks.
Conditional Random Fields (CRFs) are undirected graphical models, a special case of which correspond to conditionally-trained finite state machines. A key advantage of these models is their great flexibility to include a wide array of overlapping, multi-granularity, non-independent features of the input. In face of thi…
We analyze a method to produce pairs of non independent Poisson processes M(t),N(t) from positively correlated, self-decomposable, exponential renewals. In particular the present paper provides the family of copulas pairing the renewals, along with the closed form for the joint distribution pm,n(s,t) of the pair…
We develop the necessary theory in computational algebraic geometry to place Bayesian networks into the realm of algebraic statistics. We present an algebra{statistics dictionary focused on statistical modeling. In particular, we link the notion of effiective dimension of a Bayesian network with the notion of algebraic…
In this paper we consider portmanteau tests for testing the adequacy of multiplicative seasonal autoregressive moving-average (SARMA) models under the assumption that the errors are uncorrelated but not necessarily independent.We relax the standard independence assumption on the error term in order to extend the range …
We provide a classification of graphical models according to their representation as subfamilies of exponential families. Undirected graphical models with no hidden variables are linear exponential families (LEFs), directed acyclic graphical models and chain graphs with no hidden variables, including Bayesian networks …
Client adaptation improves federated learning performance with non-IID data.
problem Improving model performance in federated learning with non-identically and non-independently distributed data.
method Simulates heterogeneous clients to learn client-specific conditioning using a conditional gated activation unit.
result Client adaptation enhances model performance across balanced and imbalanced data sets from audio and image domains.
We introduce a new method for estimating the parameter of the bivariate Clayton copulas within the framework of Algorithmic Inference. The method consists of a variant of the standard boot-strapping procedure for inferring random parameters, which we expressly devise to bypass the two pitfalls of this specific instance…
Exponential inequalities are main tools in machine learning theory. To prove exponential inequalities for non i.i.d random variables allows to extend many learning techniques to these variables. Indeed, much work has been done both on inequalities and learning theory for time series, in the past 15 years. However, for …
Study introduces new Bernstein inequalities for dependent data in Hilbert spaces.
problem Learning from non-independent and non-identically distributed data.
method Data-dependent Bernstein inequalities tailored for vector-valued processes in Hilbert space.
result Achieved novel risk bounds for covariance operator estimation and operator learning.
This work considers the problem of modified portmanteau tests for testing the adequacy of FARIMA models under the assumption that the errors are uncorrelated but not necessarily independent (i.e. weak FARIMA). We first study the joint distribution of the least squares estimator and the noise empirical autocovariances. …
CRF model improves protein secondary structure prediction.
problem Improving secondary structure prediction of proteins.
method Applied Conditional Random Fields (CRF) to protein classification.
result CRF model leads to extremely accurate protein secondary structure predictions.
FedGAN trains GANs across distributed data sources with reduced communication.
problem Training GANs across non-independent data sources with privacy and communication constraints.
method FedGAN uses local generators and discriminators synced via an intermediary, proving convergence under standard assumptions.
result FedGAN converges and performs similarly to general distributed GANs with reduced communication complexity.
In this article, I will present a paradox whose purpose is to draw your attention to an important topic in finance, concerning the non-independence of the financial returns (non-ergodic hypothesis). In this paradox, we have two people sitting at a table separated by a black sheet so that they cannot see each other and …
Graph Posterior Network improves uncertainty estimation for node classification in interdependent graphs.
problem Uncertainty quantification for non-independent node-level predictions in graphs.
method Derives axioms for expected predictive uncertainty, proposes Graph Posterior Network (GPN) which performs Bayesian posterior updates.
result GPN outperforms existing approaches for uncertainty estimation in semi-supervised node classification.
Semi-Federated Learning clusters clients for efficient model training.
problem Efficiency and data distribution challenges in Federated Learning.
method Local clustering and in-cluster training with a sequential training manner.
result Semi-Federated Learning reduces communication costs and improves robustness to Non-IID data.
Survey classifies Clustered Federated Learning into three types of approaches.
problem Non-independent and identically distributed (non-IID) data in Federated Learning.
method Systematic review of CFL literature, principled taxonomy.
result Core CFL and Metadata-based approaches have distinct focuses.
A new algorithm improves federated learning by combining knowledge distillation and weighted combination loss.
problem Non-IID client data in federated learning leads to model drift and poor generalization.
method pFedKD-WCL integrates knowledge distillation with bi-level optimization to address non-IID challenges.
result pFedKD-WCL outperforms state-of-the-art algorithms in accuracy and convergence speed.
This paper proposes a new D2D data sharing approach to improve distributed machine learning training speed.
problem Straggler dilemma in distributed edge learning.
method Proposes a D2D data sharing approach to balance computation loads and optimize radio resource allocation.
result Significantly reduces training delay and enhances training accuracy in non-i.i.d. data environments.
Study finds non-IID data causes FL performance issues.
problem Reduced performance in federated learning due to non-IID data.
method Investigated from IID to non-IID settings, categorized methods into two strategies.
result Inconsistencies in client loss landscapes are the primary cause of performance degradation.
We develop fast spectral algorithms for tensor decomposition that match the robustness guarantees of the best known polynomial-time algorithms for this problem based on the sum-of-squares (SOS) semidefinite programming hierarchy. Our algorithms can decompose a 4-tensor with n-dimensional orthonormal components in the…
This paper tackles federated incremental learning with dynamic memory allocation for improved model performance in non-IID data.
problem Catastrophic forgetting in federated healthcare systems with non-IID data.
method Dynamic memory allocation strategy based on data replay mechanism.
result Significant performance improvements in medical image datasets compared to baseline models.
Study evaluates scikit-learn regularization frameworks for machine learning models.
problem Choosing the right regularization framework for applied machine learning models.
method Empirical evaluation of four canonical frameworks (Ridge, Lasso, ElasticNet, Post-Lasso OLS) across 134,400 simulations.
result Lasso recall is highly fragile under multicollinearity; at high condition numbers (kappa) and low SNR, Lasso recall collapses to 0.18 while ElasticNet maintains 0.93.
Unified Bayesian framework for clustered federated learning improves model performance.
problem Handling non-IID client data in federated learning.
method A unified Bayesian framework for clustered federated learning that associates clients to clusters and proposes practical algorithms for data associations.
result The proposed framework increases model performance by circumventing the need for unique client-cluster associations.
Proposes inference for DNNs in GNRMs, addressing non-independence issues.
problem Inference for DNN-estimated means in GNRMs under non-independence.
method Develops a DNN estimator and ESM for variance estimation and confidence intervals.
result Demonstrates feasibility of inference under GNRMs with ESM.
Fed-TDA augments federated tabular data to improve performance and privacy.
problem Non-IID data challenges in federated learning.
method Synthesizes tabular data using simple statistics for augmentation.
result Fed-TDA improves test performance and communication efficiency.
New algorithm reduces reinforcement learning regret for linear MDPs with unknown transitions.
problem Adversarial linear mixture MDPs with bandit feedback and unknown transition.
method Proposes a new algorithm with a least square estimator and self-normalized concentration.
result Achieves improved regret bound with high probability.
We propose a novel adaptive kernel based regression method for complex-valued signals: the generalized complex-valued kernel least-mean-square (gCKLMS). We borrow from the new results on widely linear reproducing kernel Hilbert space (WL-RKHS) for nonlinear regression and complex-valued signals, recently proposed by th…
Early neural network training reveals important sub-networks and weight distributions.
problem Understanding the early phases of neural network training.
method Extensive measurements and quantitative probing of weight distribution and dataset reliance.
result Deep networks are not robust to reinitializing with random weights while maintaining signs, and weight distributions are highly non-independent.
Clust-PSI-PFL uses PSI to improve accuracy and fairness in federated learning.
problem Non-IID data biases federated learning performance.
method Clust-PSI-PFL uses clustering and PSI to form homogeneous groups of clients.
result Clust-PSI-PFL delivers up to 18% higher global accuracy and improves client fairness.
New method addresses class imbalance in federated learning.
problem Class imbalance in federated learning training data.
method Proposes a monitoring scheme to infer training data composition and a new loss function, Ratio Loss, to mitigate imbalance.
result Demonstrates effectiveness in mitigating class imbalance in federated learning, outperforming previous methods.
This study assesses the impact of non-IID data in federated learning, revealing significant performance drops.
problem The impact of non-IID data on federated learning model performance.
method Empirical analysis using Hellinger Distance to measure distribution differences, benchmarking four strategies for handling non-IID data.
result Significant performance drops occur at specific HD thresholds, especially for extreme non-IIDness.
The Riemannian barycentre is one of the most widely used statistical descriptors for probability distributions on Riemannian manifolds. At present, existing algorithms are able to compute the Riemannian barycentre of a probability distribution, only if i.i.d. samples of this distribution are readily available. However,…
Analyzes new economic paradigm for non-independent consumer choices.
problem Non-independent consumer choices due to firm supply and consumer information.
method Develops a new mathematical framework for economic systems.
result New paradigm for economic system description is necessary.
A novel sequence-to-sequence model predicts missing sensor data.
problem Missing sensor data in sequences.
method Formulated a novel sequence-to-sequence model using forward and backward RNNs.
result The model produces the lowest errors in 12% more cases than the current state-of-the-art.
This paper proposes a cooperative mechanism for mitigating the performance degradation due to non-independent-and-identically-distributed (non-IID) data in collaborative machine learning (ML), namely federated learning (FL), which trains an ML model using the rich data and computational resources of mobile clients with…
Study proposes BFEL framework for privacy-preserving FL in personalized healthcare.
problem Privacy and security concerns in traditional cloud-centric ML, especially in wearable devices.
method Develops a blockchain-enhanced federated edge learning (BFEL) framework based on FedCurv, incorporating fisher information matrix and public key encryption.
result Significant reduction in communication cost and high efficiency for federated training on non-iid and heterogeneous data.
Classifies intrinsically linked tournaments by their score sequences.
problem Classifying intrinsically linked tournaments using their score sequences.
method Examining the score sequences of tournaments and identifying linkless sequences.
result The vast majority of score sequences for 8-vertex tournaments are linkless.
Enhances sequence memory capacity in neural networks.
problem Limited sequence capacity in Hopfield-like neural networks.
method Introducing a nonlinear interaction term and a generalized pseudoinverse rule.
result Significantly increased sequence capacity with novel scaling laws.
Deep generative models have been successfully used to learn representations for high-dimensional discrete spaces by representing discrete objects as sequences and employing powerful sequence-based deep models. Unfortunately, these sequence-based models often produce invalid sequences: sequences which do not represent a…
Equivalent condition found for q-holonomic sequences.
problem Characterizing q-holonomic sequences.
method Elimination property for subsets of variables.
result Proved equivalence of q-holonomicity and elimination property.
The abstract discusses a spectral sequence for Lie algebroids.
problem The abstract tackles the spectral sequence of Lie algebroids.
method The abstract presents a spectral sequence for Lie algebroids, generalizing classical constructions.
result The spectral sequence converges to Lie algebroid cohomology for wide Lie subalgebroids and to formal Lie algebroid cohomology for Lie subalgebroids over proper submanifolds.