arXiv research

STAG injects noise into graph neural networks to improve performance.

Unified approach for federated learning using MM optimization.

LASG improves communication efficiency in distributed learning.

We describe the sample paths of the stochastic field $F = F_t(v,x,q)$ of aggregate utilities parameterized by Pareto weights $v$ and total cash amounts $x$ and stocks' quantities $q$ in an economy. We also describe the sample paths of the stochastic field $G = G_t(u,y,q)$, which is conjugate to $F$ with respect to the …

This paper addresses the problem of rank aggregation, which aims to find a consensus ranking among multiple ranking inputs. Traditional rank aggregation methods are deterministic, and can be categorized into explicit and implicit methods depending on whether rank information is explicitly or implicitly utilized. Surpri…

This paper applies AMP theory to improve learning tasks.

We consider the problem of learning convex aggregation of models, that is as good as the best convex aggregation, for the binary classification problem. Working in the stream based active learning setting, where the active learner has to make a decision on-the-fly, if it wants to query for the label of the point curren…

Paper tackles learning win-win solutions in aggregation systems.

We propose a novel robust aggregation rule for distributed synchronous Stochastic Gradient Descent~(SGD) under a general Byzantine failure model. The attackers can arbitrarily manipulate the data transferred between the servers and the workers in the parameter server~(PS) architecture. We prove the Byzantine resilience…

Study online learning in MDPs with aggregate bandit feedback, achieving low regret in both stochastic and adversarial settings.

We study a variant of the stochastic $K$-armed bandit problem, which we call "bandits with delayed, aggregated anonymous feedback". In this problem, when the player pulls an arm, a reward is generated, however it is not immediately observed. Instead, at the end of each round the player observes only the sum of a number…

New approach for prudent risk evaluation using model aggregation.

A new stochastic method handles ensemble creation with cost constraints.

New method improves training stochastic neural networks with tighter guarantees.

SDP approach recovers communities in multilayer hypergraphs from aggregated similarity matrices.

Paper addresses privacy and robustness in stochastic linear bandits.

MEVA aggregates model predictions to improve accuracy without needing model details.

We propose three new robust aggregation rules for distributed synchronous Stochastic Gradient Descent~(SGD) under a general Byzantine failure model. The attackers can arbitrarily manipulate the data transferred between the servers and the workers in the parameter server~(PS) architecture. We prove the Byzantine resilie…

Secure Aggregation protocols allow a collection of mutually distrust parties, each holding a private value, to collaboratively compute the sum of those values without revealing the values themselves. We consider training a deep neural network in the Federated Learning model, using distributed stochastic gradient descen…

Improved defense against data poisoning attacks by aggregating smaller subsets.

We consider a large, homogeneous portfolio of life or disability annuity policies. The policies are assumed to be independent conditional on an external stochastic process representing the economic-demographic environment. Using a conditional law of large numbers, we establish the connection between claims reserving an…

Modeling price formation with interacting Hawkes processes leading to stochastic volatility with leverage.

Robust algorithm for distributed optimization resistant to Byzantine failures.

Paper uses stochastic algorithms to estimate systemic risk measures.

We study a new aggregation operator for gradients coming from a mini-batch for stochastic gradient (SG) methods that allows a significant speed-up in the case of sparse optimization problems. We call this method AdaBatch and it only requires a few lines of code change compared to regular mini-batch SGD algorithms. We p…

We consider Feller mean-reverting square-root diffusion, which has been applied to model a wide variety of processes with linearly state-dependent diffusion, such as stochastic volatility and interest rates in finance, and neuronal and populations dynamics in natural sciences. We focus on the statistical mixing (or sup…

Value aggregation is a general framework for solving imitation learning problems. Based on the idea of data aggregation, it generates a policy sequence by iteratively interleaving policy optimization and evaluation in an online learning setting. While the existence of a good policy in the policy sequence can be guarant…

Two novel algorithms improve distributed machine learning in the presence of Byzantine adversaries.

The hierarchical structure of production planning has the advantage of assigning different decision variables to their respective time horizons and therefore ensures their manageability. However, the restrictive structure of this top-down approach implying that upper level decisions are the constraints for lower level …

Method learns dynamics from aggregate data without full trajectories.

More frequent model updates in FL increase generalization error.

GRAIN: Group Aggregation via Min-Norm Objective

The present paper develops a novel aggregated gradient approach for distributed machine learning that adaptively compresses the gradient communication. The key idea is to first quantize the computed gradients, and then skip less informative quantized gradient communications by reusing outdated gradients. Quantizing and…

Recently, Convolution Neural Networks (CNNs) obtained huge success in numerous vision tasks. In particular, DenseNets have demonstrated that feature reuse via dense skip connections can effectively alleviate the difficulty of training very deep networks and that reusing features generated by the initial layers in all s…

Stochastic simulation techniques employed for the analysis of portfolios of insurance/reinsurance risk, often referred to as `Aggregate Risk Analysis', can benefit from exploiting state-of-the-art high-performance computing platforms. In this paper, parallel methods to speed-up aggregate risk analysis for supporting re…

Byrd-SAGA reduces variance to robustify SGD against Byzantine attacks.

Develops a method to efficiently compute Wasserstein barycenters with variational distributions.

Deep learning models have shown state-of-the-art performance in many inverse reconstruction problems. However, it is not well understood what properties of the latent representation may improve the generalization ability of the network. Furthermore, limited models have been presented for inverse reconstructions over ti…

Proposes a method to improve Byzantine-robustness in compressed federated learning.

Study on network-valued processes with asynchronous updates, proving consistency in community and changepoint estimation.

Model analyzes RFQ markets using stochastic control to optimize dealer performance and inventory.

Study online monotone density estimation with expert aggregation and log-optimal calibration.

We introduce a new recursive aggregation procedure called Bernstein Online Aggregation (BOA). The exponential weights include an accuracy term and a second order term that is a proxy of the quadratic variation as in Hazan and Kale (2010). This second term stabilizes the procedure that is optimal in different senses. We…

A key factor in developing high performing machine learning models is the availability of sufficiently large datasets. This work is motivated by applications arising in Software as a Service (SaaS) companies where there exist numerous similar yet disjoint datasets from multiple client companies. To overcome the challen…

A method to robustly federate learning with non-i.i.d. data and Byzantine workers.

New class of heavy-tailed distributions shows weighted averages dominate individual variables.

Noisy labeled data is more a norm than a rarity for crowd sourced contents. It is effective to distill noise and infer correct labels through aggregation results from crowd workers. To ensure the time relevance and overcome slow responses of workers, online label aggregation is increasingly requested, calling for solut…

A general graph-structured neural network architecture operates on graphs through two core components: (1) complex enough message functions; (2) a fixed information aggregation process. In this paper, we present the Policy Message Passing algorithm, which takes a probabilistic perspective and reformulates the whole inf…