arXiv research

New class of heavy-tailed distributions shows weighted averages dominate individual variables.

The paper analyzes ridge regression with random features for non-identically distributed data.

Study ridge regression for non-identically distributed data with varying variances.

This note displays an interesting phenomenon for percentiles of independent but non-identical random variables. Let $X_1,\cdots,X_n$ be independent random variables obeying non-identical continuous distributions and $X^{(1)}\geq \cdots\geq X^{(n)}$ be the corresponding order statistics. For any $p\in(0,1)$, we investig…

Federated Learning enables visual models to be trained in a privacy-preserving way using real-world data from mobile devices. Given their distributed nature, the statistics of the data across these devices is likely to differ significantly. In this work, we look at the effect such non-identical data distributions has o…

The paper provides guarantees for learning nonlinear representations from multiple non-identically distributed data sources.

Undirected graphs are often used to describe high dimensional distributions. Under sparsity conditions, the graph can be estimated using $\ell_1$ penalization methods. However, current methods assume that the data are independent and identically distributed. If the distribution, and hence the graph, evolves over time t…

This paper tackles computational bottlenecks in federated learning on mobile devices.

The study examines property testing and estimation under non-identically distributed samples, finding necessary and sufficient sample complexities.

Federated Learning is a distributed learning paradigm with two key challenges that differentiate it from traditional distributed optimization: (1) significant variability in terms of the systems characteristics on each device in the network (systems heterogeneity), and (2) non-identically distributed data across the ne…

To accelerate the training of machine learning models, distributed stochastic gradient descent (SGD) and its variants have been widely adopted, which apply multiple workers in parallel to speed up training. Among them, Local SGD has gained much attention due to its lower communication cost. Nevertheless, when the data …

Study large deviations in life insurance portfolios without identical distributions.

We strengthen the results of \cite{A1}, consequently, we improve the claims of \cite{A2} obtaining the best possible results. Namely, we prove that if a subgroup $Γ$ of $\mathrm{Diff}_{+}(I)$ contains a free semigroup on two generators then $Γ$ is not $C_0$-discrete. Using this, we extend the Hölder's Theorem in $\math…

Novel framework for data sharing and coordinated exploration in concurrent RL with non-identical environments.

Macbeath gave a formula for the number of fixed points for each non-identity element of a cyclic group of automorphisms of a compact Riemann surface in terms of the universal covering transformation group of the cyclic group. We observe that this formula generalizes to determine the fixed-point set of each non-identity…

New robust discriminant analysis for non-Gaussian data.

A new robust and flexible classification method for non-Gaussian data.

Client adaptation improves federated learning performance with non-IID data.

In this paper, we study and partially classify those Riemannian man-ifolds carrying a non-identically vanishing function f whose Hessian is minus f times the Ricci-tensor of the manifold.

Study robust estimation under varying corruption probabilities in data.

We consider the task of meta-analysis in high-dimensional settings in which the data sources are similar but non-identical. To borrow strength across such heterogeneous datasets, we introduce a global parameter that emphasizes interpretability and statistical efficiency in the presence of heterogeneity. We also propose…

We prove that if Γis subgroup of Diff_{+}^{1+ε}(I) and N is a natural number such that every non-identity element of Γhas at most N fixed points then Γis solvable. If in addition Γis a subgroup of Diff_{+}^{2}(I) then we can claim that Γis metaabelian.

In this paper we investigate the feasibility of using synthetic data to augment face datasets. In particular, we propose a novel generative adversarial network (GAN) that can disentangle identity-related attributes from non-identity-related attributes. This is done by training an embedding network that maps discrete id…

Study noisy rewards in online decision-making with unknown distributions.

Study introduces new Bernstein inequalities for dependent data in Hilbert spaces.

New neural network captures spatial correlations in wind speed predictions.

FedSmart optimizes federated learning models for non-IID data.

Electronic medical records (EMRs) supports the development of machine learning algorithms for predicting disease incidence, patient response to treatment, and other healthcare events. But insofar most algorithms have been centralized, taking little account of the decentralized, non-identically independently distributed…

In [13], it is proved that any subgroup of $\mathrm{Diff}_{+}^{ω}(I)$ (the group of orientation preserving analytic diffeomorphisms of the interval) is either metaabelian or does not satisfy a law. A stronger question is asked whether or not the Girth Alternative holds for subgroups of $\mathrm{Diff}_{+}^{ω}(I)$. In th…

This work addresses a new problem that learns generative adversarial networks (GANs) from multiple data collections that are each i) owned separately by different clients and ii) drawn from a non-identical distribution that comprises different classes. Given such non-iid data as input, we aim to learn a distribution in…

In this paper, we propose a distributed algorithm for stochastic smooth, non-convex optimization. We assume a worker-server architecture where $N$ nodes, each having $n$ (potentially infinite) number of samples, collaborate with the help of a central server to perform the optimization task. The global objective is to m…

Federated learning framework improves model generalization and privacy.

LP-FT improves personalized model training in FL by balancing generalization and personalization.

We consider the problem of estimating the common mean of independently sampled data, where samples are drawn in a possibly non-identical manner from symmetric, unimodal distributions with a common mean. This generalizes the setting of Gaussian mixture modeling, since the number of distinct mixture components may diverg…

Value-at-Risk can be superadditive for sufficiently heavy-tailed losses.

The portfolio optimization problem in which the variances of the return rates of assets are not identical is analyzed in this paper using the methodology of statistical mechanical informatics, specifically, replica analysis. We define two characteristic quantities of an optimal portfolio, namely, minimal investment ris…

Investigates VaR behavior for sums of one-sided random variables, showing impossibilities and conditions for super-additivity.

We give effective proofs of residual finiteness and conjugacy separability for finitely generated nilpotent groups. In particular, we give precise asymptotic bounds for a function introduced by Bou-Rabee that measures how large the quotients that are need to separate non-identity elements of bounded length from the ide…

New Gaussian min-max theorem extends classical results to non-i.i.d. Gaussian matrices.

FedNAS automates federated learning by searching for better architectures.

We consider the Dolbeault operator of $K^{1/2}$ -- the square root of the canonical line bundle which determines the spin structure of a compact Hermitian spin surface (M,g,J). We prove that the Dolbeault cohomology groups of $K^{1/2}$ vanish if the scalar curvature of g is non-negative and non-identically zero. Moreov…

Study resolvent convergence for random matrices with general covariance profiles.

Unified analysis for decentralized SGD across various topologies and updates.

WassFFed addresses fairness in Federated Learning by ensuring consistency between local and global models.

In many machine learning problems, labeled training data is limited but unlabeled data is ample. Some of these problems have instances that can be factored into multiple views, each of which is nearly sufficent in determining the correct labels. In this paper we present a new algorithm for probabilistic multi-view lear…

The emerging paradigm of federated learning strives to enable collaborative training of machine learning models on the network edge without centrally aggregating raw data and hence, improving data privacy. This sharply deviates from traditional machine learning and necessitates the design of algorithms robust to variou…

We prove that any smooth action of $\mathbb Z^{m-1}, m\ge 3$ on an $m$-dimensional manifold that preserves a measure such that all non-identity elements of the suspension have positive entropy is essentially algebraic, i.e. isomorphic up to a finite permutation to an affine action on the torus or its factor by $\pm\Id$…

Recently, the technique of local updates is a powerful tool in centralized settings to improve communication efficiency via periodical communication. For decentralized settings, it is still unclear how to efficiently combine local updates and decentralized communication. In this work, we propose an algorithm named as L…