MaxCOSD algorithm tackles non-i.i.d. demands and stateful dynamics in online inventory control.
problem Managing inventory with non-i.i.d. demands and stateful dynamics.
method MaxCOSD, an online algorithm with provable guarantees for non-degeneracy assumptions.
result MaxCOSD achieves optimal performance for non-i.i.d. demands and stateful dynamics.
New bandit algorithm for non-i.i.d. noise, improving standard rates.
problem Linear stochastic bandit with non-i.i.d. observation noise.
method Developed new confidence sequences and an algorithm based on optimism in uncertainty.
result Regret bounds for the new algorithm, showing recovery of standard rates up to a factor of the mixing time.
This work develops a non-parametric test for relational independence in non-i.i.d. data.
problem Testing independence in relational systems where data samples are not i.i.d.
method Kernel mean embedding for relational variables, consistent non-parametric scalable kernel test.
result Empirically validated effectiveness compared to state-of-the-art tests.
Paper analyzes CRL generalization under non-i.i.d. settings, providing bounds for practical data reuse.
problem Limited theoretical understanding of CRL generalization under non-i.i.d. data conditions.
method Inspired by U-statistics, derives generalization bounds for CRL under non-i.i.d. settings.
result Required number of samples scales logarithmically with class covering number.
We study the task of learning from non-i.i.d. data. In particular, we aim at learning predictors that minimize the conditional risk for a stochastic process, i.e. the expected loss of the predictor on the next point conditioned on the set of training samples observed so far. For non-i.i.d. data, the training set contai…
A universal framework for constructing confidence sets using sequential likelihood mixing.
problem Constructing reliable confidence sets for realizable likelihood functions.
method Sequential likelihood mixing, integrating Bayesian inference and regret inequalities.
result Establishes fundamental connections and provable coverage guarantees for various inference techniques.
Study improves KRR for non-i.i.d. data, with applications in denoising.
problem Kernel regression in structured non-i.i.d. settings.
method Developed a blockwise decomposition method for dependent data, deriving excess risk bounds.
result Established generalization guarantees for KRR in non-i.i.d. settings.
Unified analysis of Federated Averaging and Nesterov FedAvg for linear speedup.
problem Understanding convergence of FL algorithms under non-i.i.d. data and partial participation.
method Systematic study of convergence guarantees for FedAvg and Nesterov FedAvg under different conditions.
result Unified analysis of linear speedup for FedAvg and Nesterov FedAvg in various settings.
New algorithm achieves linear speedup in non-i.i.d. federated bilevel learning.
problem Linear speedup in convergence for non-i.i.d. datasets in federated bilevel optimization.
method Proposes FedMBO with a novel client sampling scheme for non-i.i.d. datasets.
result Achieves a convergence rate of O(1/√(nK) + 1/K + √n/K³/²).
A method to robustly federate learning with non-i.i.d. data and Byzantine workers.
problem Byzantine workers sending malicious messages in federated learning with non-i.i.d. data.
method Resampling strategy to reduce inner and outer variation, stochastic average gradient, robust geometric median aggregation.
result The method reaches a neighborhood of the optimal solution at a linear convergence rate and learning error depends on the number of Byzantine workers.
We present an new sequential Monte Carlo sampler for coalescent based Bayesian hierarchical clustering. Our model is appropriate for modeling non-i.i.d. data and offers a substantial reduction of computational cost when compared to the original sampler without resorting to approximations. We also propose a quadratic co…
FedIV uses federated GMM for IV analysis in non-i.i.d. data.
problem Efficient IV analysis in non-i.i.d. federated data.
method Federated GMM via FedGDA algorithm.
result Federated solution consistently estimates local moment conditions.
Novel risk bound for structured prediction tackles non-i.i.d. data.
problem Structured prediction challenges due to non-factorizable target objects.
method PAC-Bayesian risk bound with explicit structure distillation.
result Generalization rate scales with data size and structure.
We present a generative model that is defined on finite sets of exchangeable, potentially high dimensional, data. As the architecture is an extension of RealNVPs, it inherits all its favorable properties, such as being invertible and allowing for exact log-likelihood evaluation. We show that this architecture is able t…
Temporal Difference Learning analysis under non-i.i.d. data and nonlinear approximation.
problem Finite-sample behavior of TD(0) under non-i.i.d. data and nonlinear approximation.
method High-probability, finite-sample analysis of vanilla TD(0) on polynomially mixing Markov data, assuming Holder continuity and bounded generalized gradients.
result Bounds on the convergence rate of TD(0) with high probability, matching known i.i.d. rates and holding even with nonstationary initialization.
New federated learning framework reduces model complexity and improves performance.
problem Slow convergence in traditional federated learning due to non-i.i.d. data.
method Clients train personalized local models, server trains shared model, addressing heterogeneity.
result Substantial performance gains over baselines, robust to non-i.i.d. data.
New algorithms for regression with adversarial responses on various metric spaces.
problem Regression with adversarial responses under non-i.i.d. sequences.
method Proves universal consistency for a wide range of non-stationary processes.
result Achieves universal consistency for a broader class of sequences than stationary processes.
The paper improves conformal prediction by analyzing the beta law of conditional coverage.
problem Improving finite-sample marginal coverage guarantees for non-i.i.d. data.
method The method uses Wasserstein distances to quantify deviations from the beta law of conditional coverage.
result The framework provides direct bounds on marginal coverage gaps and bad-calibration probabilities.
FedNAS automates federated learning by searching for better architectures.
problem Non-I.I.D. data makes predefined model architectures suboptimal.
method Federated Neural Architecture Search (FedNAS) for collaborative architecture optimization.
result FedNAS searches for better architectures that outperform predefined models.
Novel approach to universal online learning for bounded losses, closing open problems.
problem Characterizing processes for universal online learning under non-i.i.d. conditions.
method Characterization of processes admitting strong and weak universal learning, introduction of optimistically universal learning rule.
result Introduction of a novel 1NN algorithm that is optimistically universal for bounded losses.
New theory explains consistency of kernel methods with non-i.i.d. data.
problem Consistency of kernel methods under non-i.i.d. data.
method Empirical weak convergence (EWC) as a general assumption.
result Established consistency of SVMs, kernel mean embeddings, and CKMEs with EWC data.
Reduces bounded loss learning to binary classification.
problem Universal consistency of non-i.i.d. processes with bounded loss.
method Constructive reduction to binary classification.
result Any bounded loss output setting can be reduced to binary classification.
Paper optimizes FL communication efficiency with stochastic optimization.
problem Intermittent connectivity and non-i.i.d. data in FL.
method Convergence analysis of non-convex loss functions, stochastic optimization for client selection and power allocation.
result Significant reduction in communication time compared to random participation.
In this work, we develop a new approach to generative density estimation for exchangeable, non-i.i.d. data. The proposed framework, FlowScan, combines invertible flow transformations with a sorted scan to flexibly model the data while preserving exchangeability. Unlike most existing methods, FlowScan exploits the intra…
Generative data augmentation boosts learning performance in various tasks.
problem Theoretical understanding of generative data augmentation's effect.
method Established a stability bound for non-i.i.d. settings, analyzed Gaussian mixture models and generative adversarial nets.
result Generative data augmentation can improve learning guarantees, especially in small train sets.
New method improves representation learning from non-i.i.d. and non-isotropic data.
problem Improving representation learning from non-i.i.d. and non-isotropic data.
method Introducing a new adaptation of alternating minimization-descent scheme to reduce noise scaling.
result Established linear convergence to optimal representation with noise scaling down with total source data size.
Combining different models is a widely used paradigm in machine learning applications. While the most common approach is to form an ensemble of models and average their individual predictions, this approach is often rendered infeasible by given resource constraints in terms of memory and computation, which grow linearl…
Paper extends SMM to weakly convex and multi-convex surrogates for non-convex optimization.
problem Non-convex optimization with weakly convex or multi-convex surrogates.
method Stochastic majorization-minimization with proximal regularization or block-minimization.
result Convergence rates for empirical and expected losses under non-i.i.d. data.
Unified framework for representation and causal structure learning using exchangeable data.
problem Identifying latent representations or causal structures in non-i.i.d. data.
method Identifiable Exchangeable Mechanisms (IEM) framework for representation and structure learning.
result New insights and identifiability results for causal structure and representation learning.
Estimates stationary mass and frequency from non-i.i.d. data.
problem Estimating stationary mass and frequency from non-i.i.d. data.
method Combines plug-in estimator with WingIt modification for exponentially α-mixing processes. result Universal consistency in n for total variation distance estimation. New algorithms avoid non-monotonic risk curves in statistical learning.
problem Non-monotonic behavior of risk curves in statistical learning.
method Derive risk-monotonic algorithms under weak assumptions.
result Risk monotonicity does not necessarily lead to worse excess risk rates.
Develops a method to continuously audit black-box conditional quantile forecasts.
problem Continuous monitoring of black-box forecasts under changing data streams and regimes.
method Distribution-free and game-theoretic testing framework for non-i.i.d. losses.
result Derives finite-time detection guarantees for miscalibrated forecasts based on features.
We tackle the problem of Federated Learning in the non i.i.d. case, in which local models drift apart, inhibiting learning. Building on an analogy with Lifelong Learning, we adapt a solution for catastrophic forgetting to Federated Learning. We add a penalty term to the loss function, compelling all local models to con…
The paper improves Bayesian optimization by calibrating uncertainty estimates.
problem Improper uncertainty estimates in Bayesian optimization when data is non-stationary.
method Proposes online learning algorithms to maintain calibration on non-i.i.d. data and integrates them into Bayesian optimization.
result Calibrated Bayesian optimization converges to better optima in fewer steps.
New Gaussian min-max theorem extends classical results to non-i.i.d. Gaussian matrices.
problem Extending classical Gaussian min-max theorems to non-i.i.d. Gaussian matrices.
method Identifying a new pair of Gaussian processes that satisfy comparison inequalities.
result New Gaussian min-max and convex Gaussian min-max theorems with applications in multi-source Gaussian regression and binary classification.
Robotic vision is a field where continual learning can play a significant role. An embodied agent operating in a complex environment subject to frequent and unpredictable changes is required to learn and adapt continuously. In the context of object recognition, for example, a robot should be able to learn (without forg…
We present a distributionally robust formulation of a stochastic optimization problem for non-i.i.d vector autoregressive data. We use the Wasserstein distance to define robustness in the space of distributions and we show, using duality theory, that the problem is equivalent to a finite convex-concave saddle point pro…
Study improves off-policy evaluation from non-i.i.d. bandit samples.
problem Improving off-policy evaluation from non-independent bandit samples.
method Constructing an estimator from a standardized martingale difference sequence.
result Proposed estimator performs better than existing methods.
We consider practical data characteristics underlying federated learning, where unbalanced and non-i.i.d. data from clients have a block-cyclic structure: each cycle contains several blocks, and each client's training data follow block-specific and non-i.i.d. distributions. Such a data structure would introduce client …
FedBE aggregates local models into a robust global model via Bayesian inference.
problem Challenges in aggregating non-i.i.d. local models into a global model in federated learning.
method FedBE uses Bayesian inference to sample and combine higher-quality global models from local models.
result FedBE leads to more robust aggregation of local models into a global model, especially when data is non-i.i.d.
K-fold cross-validation (CV) with squared error loss is widely used for evaluating predictive models, especially when strong distributional assumptions cannot be taken. However, CV with squared error loss is not free from distributional assumptions, in particular in cases involving non-i.i.d. data. This paper analyzes …
FA-LD algorithm improves uncertainty quantification and mean predictions in federated learning.
problem Uncertainty quantification and mean predictions in federated learning with distributed clients.
method FA-LD algorithm for strongly log-concave distributions with non-i.i.d data, considering general models.
result The FA-LD algorithm provides theoretical guarantees for convergence and optimal noise injection.
The paper improves Q-learning by incorporating pessimism for better sample efficiency.
problem Improving sample efficiency in asynchronous Q-learning with non-i.i.d. data.
method Developed an algorithmic framework that incorporates the principle of pessimism into asynchronous Q-learning, penalizing infrequently-visited state-action pairs based on suitable lower confidence bounds (LCBs).
result Achieved near-optimal sample complexity, providing theoretical support for the use of pessimism in non-i.i.d. data.
The paper proposes a method to monitor deep learning predictions for retraining, reducing costs.
problem Reducing computational costs in deep learning by detecting when predictions are no longer valid.
method Sequential monitoring of network predictions based on projected second moments monitoring.
result The proposed method can drastically reduce computational costs in deep learning.
Paper optimizes experimental design for estimating treatment effect.
problem Estimating treatment effect with heterogeneous subjects and treatments.
method Adaptive experimental design incorporating bandit learning.
result Demonstrates optimality of proposed adaptive experiment framework.
SARSA is an on-policy algorithm to learn a Markov decision process policy in reinforcement learning. We investigate the SARSA algorithm with linear function approximation under the non-i.i.d.\ data, where a single sample trajectory is available. With a Lipschitz continuous policy improvement operator that is smooth eno…
Online Platt Scaling adapts to varying data distributions.
problem Adapting Platt scaling to non-i.i.d. settings with distribution drift.
method Combines Platt scaling with online logistic regression and calibeating.
result OPS+calibeating method is guaranteed to be calibrated for adversarial outcomes.
We present a novel statistical inference framework for convex empirical risk minimization, using approximate stochastic Newton steps. The proposed algorithm is based on the notion of finite differences and allows the approximation of a Hessian-vector product from first-order information. In theory, our method efficient…