arXiv research

Study finds a non-locally contractible $r$-convex set.

MpFL models clients as strategic players to reach equilibrium with less communication.

Paper improves a method for fast global and local convergence in optimization.

Communication on heterogeneous edge networks is a fundamental bottleneck in Federated Learning (FL), restricting both model capacity and user participation. To address this issue, we introduce two novel strategies to reduce communication costs: (1) the use of lossy compression on the global model sent server-to-client;…

Deep learning model improves X-ray disease detection accuracy in Thai patients.

Eikonal-Constrained QRL improves goal-reaching in reinforcement learning.

Improved bounds for function approximation in nonlinear sets.

In important applications involving multi-task networks with multiple objectives, agents in the network need to decide between these multiple objectives and reach an agreement about which single objective to follow for the network. In this work we propose a distributed decision-making algorithm. The agents are assumed …

Paper shows faster convergence to local-minimizers in over-parametrized models under interpolation-like conditions.

We prove that the torsion of any closed space curve which bounds a simply connected locally convex surface vanishes at least 4 times. This answers a question of Rosenberg related to a problem of Yau on characterizing the boundary of positively curved disks in Euclidean space. Furthermore, our result generalizes the 4 v…

An algorithmic limit of compressed sensing or related variable-selection problems is analytically evaluated when a design matrix is given by an overcomplete random matrix. The replica method from statistical mechanics is employed to derive the result. The analysis is conducted through evaluation of the entropy, an expo…

In this paper we describe the local Ricci and Bianchi identities for an h-normal N-linear connection DΓ(N) on the dual 1-jet space J^{1*}(T,M). To reach this aim, we firstly give the expressions of the local distinguished (d-) adapted components of torsion and curvature tensors produced by DΓ(N), and then we analyze th…

Adaptor 'E' extends gradient-based optimizers to explore loss landscapes, improving generalization.

Deep Neural Networks (DNNs) excel on many complex perceptual tasks but it has proven notoriously difficult to understand how they reach their decisions. We here introduce a high-performance DNN architecture on ImageNet whose decisions are considerably easier to explain. Our model, a simple variant of the ResNet-50 arch…

Polyak step size GD reaches final radius of convergence after log iterations.

Study on sets with positive reach in Euclidean and Riemannian spaces.

Study calculates reach and curvature of a specific geometric variety.

Computes bounds on reach and r-convexity from point cloud data.

Communication bottleneck has been identified as a significant issue in distributed optimization of large-scale learning models. Recently, several approaches to mitigate this problem have been proposed, including different forms of gradient compression or computing local models and mixing them iteratively. In this paper…

Study improves understanding of submanifold reach in Riemannian geometry.

Training deep neural networks with the error backpropagation algorithm is considered implausible from a biological perspective. Numerous recent publications suggest elaborate models for biologically plausible variants of deep learning, typically defining success as reaching around 98% test accuracy on the MNIST data se…

Paper estimates manifold reach using convexity defect function.

Paper analyzes FedAvg and FedProx, showing they don't reach global optima and may not generalize well.

We give new characterisations of sets of positive reach and show that a closed hypersurface has positive reach if and only if it is of class $C^{1,1}$. These results are then used to prove new alternating Steiner formulæ for hypersurfaces of positive reach. Furthermore, it will turn out that every hypersurface that sat…

Paper proposes a method to learn goal-reaching behaviors from scratch using imitation learning.

Novel approach combines local and global brain changes for AD prediction.

We determine the optimal strategy for investing in a Black-Scholes market in order to maximize the probability that wealth at death meets a bequest goal $b$, a type of goal-seeking problem, as pioneered by Dubins and Savage (1965, 1976). The individual consumes at a constant rate $c$, so the level of wealth required fo…

Study local exploration on dynamic graphs with time-varying edges.

Various problems in manifold estimation make use of a quantity called the reach, denoted by $τ\_M$, which is a measure of the regularity of the manifold. This paper is the first investigation into the problem of how to estimate the reach. First, we study the geometry of the reach through an approximation perspective. W…

It has been shown that injecting noise into the neural network weights during the training process leads to a better generalization of the resulting model. Noise injection in the distributed setup is a straightforward technique and it represents a promising approach to improve the locally trained models. We investigate…

Given a sample $Y$ from an unknown manifold $X$ embedded in Euclidean space, it is possible to recover the homology groups of $X$ by building a Vietoris--Rips or Čech simplicial complex on top of the vertex set $Y$. However, these simplicial complexes need not inherit the metric structure of the manifold, in particular…

Recent breakthroughs in computer vision make use of large deep neural networks, utilizing the substantial speedup offered by GPUs. For applications running on limited hardware, however, high precision real-time processing can still be a challenge. One approach to solving this problem is training networks with binary or…

Could a gradient aggregation rule (GAR) for distributed machine learning be both robust and fast? This paper answers by the affirmative through multi-Bulyan. Given $n$ workers, $f$ of which are arbitrary malicious (Byzantine) and $m=n-f$ are not, we prove that multi-Bulyan can ensure a strong form of Byzantine resilien…

Optimization of very expensive black-box functions requires utilization of maximum information gathered by the process of optimization. Model Guided Sampling Optimization (MGSO) forms a more robust alternative to Jones' Gaussian-process-based EGO algorithm. Instead of EGO's maximizing expected improvement, the MGSO use…

Diffusion reach probability between two nodes on a network is defined as the probability of a cascade originating from one node reaching to another node. An infinite number of cascades would enable calculation of true diffusion reach probabilities between any two nodes. However, there exists only a finite number of cas…

New method for private learning with public features improves convergence rates.

Federated learning uses worst-case optimization to handle uncertain local data impacts.

The paper extends submanifold reach to less regular $C^{1,α}$ classes.

Recently similarity graphs became the leading paradigm for efficient nearest neighbor search, outperforming traditional tree-based and LSH-based methods. Similarity graphs perform the search via greedy routing: a query traverses the graph and in each vertex moves to the adjacent vertex that is the closest to this query…

The L1 loss landscape of neural nets near local minima behaves differently, revealing exponential decay and increased vertex density.

New curvature measures characterize non-convex Wulff shapes in normed spaces.

Guaranteed reachable set for unknown nonlinear systems on manifolds.

Study shows spheres in high dimensions have maximum volume if they are smooth and have a specific reach.

New insights on pruning deep networks by preserving function locality.

We establish a quantitative lower bound on the reach of flat norm minimizers for boundaries in $\mathbb{R}^2$.

Bayesian framework mixes imperfect models for improved predictions.

Local update methods' performance depends on learning rates, affecting convergence rates and alignment with true loss.

We propose a new \cu{class-optimal} algorithm for the distributed computation of Wasserstein Barycenters over networks. Assuming that each node in a graph has a probability distribution, we prove that every node can reach the barycenter of all distributions held in the network by using local interactions compliant with…