arXiv research

The paper analyzes ridge regression with random features for non-identically distributed data.

Study ridge regression for non-identically distributed data with varying variances.

Federated Learning enables visual models to be trained in a privacy-preserving way using real-world data from mobile devices. Given their distributed nature, the statistics of the data across these devices is likely to differ significantly. In this work, we look at the effect such non-identical data distributions has o…

Undirected graphs are often used to describe high dimensional distributions. Under sparsity conditions, the graph can be estimated using $\ell_1$ penalization methods. However, current methods assume that the data are independent and identically distributed. If the distribution, and hence the graph, evolves over time t…

New class of heavy-tailed distributions shows weighted averages dominate individual variables.

The paper provides guarantees for learning nonlinear representations from multiple non-identically distributed data sources.

This paper tackles computational bottlenecks in federated learning on mobile devices.

This note displays an interesting phenomenon for percentiles of independent but non-identical random variables. Let $X_1,\cdots,X_n$ be independent random variables obeying non-identical continuous distributions and $X^{(1)}\geq \cdots\geq X^{(n)}$ be the corresponding order statistics. For any $p\in(0,1)$, we investig…

Federated Learning is a distributed learning paradigm with two key challenges that differentiate it from traditional distributed optimization: (1) significant variability in terms of the systems characteristics on each device in the network (systems heterogeneity), and (2) non-identically distributed data across the ne…

The study examines property testing and estimation under non-identically distributed samples, finding necessary and sufficient sample complexities.

To accelerate the training of machine learning models, distributed stochastic gradient descent (SGD) and its variants have been widely adopted, which apply multiple workers in parallel to speed up training. Among them, Local SGD has gained much attention due to its lower communication cost. Nevertheless, when the data …

New robust discriminant analysis for non-Gaussian data.

A new robust and flexible classification method for non-Gaussian data.

Client adaptation improves federated learning performance with non-IID data.

Novel framework for data sharing and coordinated exploration in concurrent RL with non-identical environments.

We consider the task of meta-analysis in high-dimensional settings in which the data sources are similar but non-identical. To borrow strength across such heterogeneous datasets, we introduce a global parameter that emphasizes interpretability and statistical efficiency in the presence of heterogeneity. We also propose…

Study large deviations in life insurance portfolios without identical distributions.

Study introduces new Bernstein inequalities for dependent data in Hilbert spaces.

Study robust estimation under varying corruption probabilities in data.

In this paper we investigate the feasibility of using synthetic data to augment face datasets. In particular, we propose a novel generative adversarial network (GAN) that can disentangle identity-related attributes from non-identity-related attributes. This is done by training an embedding network that maps discrete id…

FedSmart optimizes federated learning models for non-IID data.

We strengthen the results of \cite{A1}, consequently, we improve the claims of \cite{A2} obtaining the best possible results. Namely, we prove that if a subgroup $Γ$ of $\mathrm{Diff}_{+}(I)$ contains a free semigroup on two generators then $Γ$ is not $C_0$-discrete. Using this, we extend the Hölder's Theorem in $\math…

Macbeath gave a formula for the number of fixed points for each non-identity element of a cyclic group of automorphisms of a compact Riemann surface in terms of the universal covering transformation group of the cyclic group. We observe that this formula generalizes to determine the fixed-point set of each non-identity…

Electronic medical records (EMRs) supports the development of machine learning algorithms for predicting disease incidence, patient response to treatment, and other healthcare events. But insofar most algorithms have been centralized, taking little account of the decentralized, non-identically independently distributed…

In this paper, we study and partially classify those Riemannian man-ifolds carrying a non-identically vanishing function f whose Hessian is minus f times the Ricci-tensor of the manifold.

This work addresses a new problem that learns generative adversarial networks (GANs) from multiple data collections that are each i) owned separately by different clients and ii) drawn from a non-identical distribution that comprises different classes. Given such non-iid data as input, we aim to learn a distribution in…

We consider the problem of estimating the common mean of independently sampled data, where samples are drawn in a possibly non-identical manner from symmetric, unimodal distributions with a common mean. This generalizes the setting of Gaussian mixture modeling, since the number of distinct mixture components may diverg…

New neural network captures spatial correlations in wind speed predictions.

Federated learning framework improves model generalization and privacy.

We prove that if Γis subgroup of Diff_{+}^{1+ε}(I) and N is a natural number such that every non-identity element of Γhas at most N fixed points then Γis solvable. If in addition Γis a subgroup of Diff_{+}^{2}(I) then we can claim that Γis metaabelian.

FedNAS automates federated learning by searching for better architectures.

The emerging paradigm of federated learning strives to enable collaborative training of machine learning models on the network edge without centrally aggregating raw data and hence, improving data privacy. This sharply deviates from traditional machine learning and necessitates the design of algorithms robust to variou…

LP-FT improves personalized model training in FL by balancing generalization and personalization.

In this paper, we propose a distributed algorithm for stochastic smooth, non-convex optimization. We assume a worker-server architecture where $N$ nodes, each having $n$ (potentially infinite) number of samples, collaborate with the help of a central server to perform the optimization task. The global objective is to m…

WassFFed addresses fairness in Federated Learning by ensuring consistency between local and global models.

Unified analysis for decentralized SGD across various topologies and updates.

Study noisy rewards in online decision-making with unknown distributions.

In many machine learning problems, labeled training data is limited but unlabeled data is ample. Some of these problems have instances that can be factored into multiple views, each of which is nearly sufficent in determining the correct labels. In this paper we present a new algorithm for probabilistic multi-view lear…

In [13], it is proved that any subgroup of $\mathrm{Diff}_{+}^{ω}(I)$ (the group of orientation preserving analytic diffeomorphisms of the interval) is either metaabelian or does not satisfy a law. A stronger question is asked whether or not the Girth Alternative holds for subgroups of $\mathrm{Diff}_{+}^{ω}(I)$. In th…

Improved algorithm for partial recovery of tree-structured graphs with noisy data.

Recently, the technique of local updates is a powerful tool in centralized settings to improve communication efficiency via periodical communication. For decentralized settings, it is still unclear how to efficiently combine local updates and decentralized communication. In this work, we propose an algorithm named as L…

Value-at-Risk can be superadditive for sufficiently heavy-tailed losses.

Proves equations for high-dimensional gradient-based methods from Gaussian data.

In the era of big data, reducing data dimensionality is critical in many areas of science. Widely used Principal Component Analysis (PCA) addresses this problem by computing a low dimensional data embedding that maximally explain variance of the data. However, PCA has two major weaknesses. Firstly, it only considers li…

The portfolio optimization problem in which the variances of the return rates of assets are not identical is analyzed in this paper using the methodology of statistical mechanical informatics, specifically, replica analysis. We define two characteristic quantities of an optimal portfolio, namely, minimal investment ris…

Investigates VaR behavior for sums of one-sided random variables, showing impossibilities and conditions for super-additivity.

We give effective proofs of residual finiteness and conjugacy separability for finitely generated nilpotent groups. In particular, we give precise asymptotic bounds for a function introduced by Bou-Rabee that measures how large the quotients that are need to separate non-identity elements of bounded length from the ide…

This monograph deals with adaptive supervised classification, using tools borrowed from statistical mechanics and information theory, stemming from the PACBayesian approach pioneered by David McAllester and applied to a conception of statistical learning theory forged by Vladimir Vapnik. Using convex analysis on the se…