A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
This work analyzes generalization in federated learning using information theory.
problem Generalization performance in federated learning is less explored compared to centralized learning.
method The work applies an information-theoretic analysis via the conditional mutual information (CMI) framework to study federated learning's two-level generalization.
result The work derives multiple CMI-based bounds, including hypothesis-based CMI bounds and fast-rate evaluated CMI bounds, which improve convergence rates for specific model aggregation strategies and structured loss functions.
Conditional Mutual Information (CMI) is a measure of conditional dependence between random variables X and Y, given another random variable Z. It can be used to quantify conditional dependence among variables in many data-driven inference problems such as graphical models, causal learning, feature selection and time-se…
Mutual Information (MI) is often used for feature selection when developing classifier models. Estimating the MI for a subset of features is often intractable. We demonstrate, that under the assumptions of conditional independence, MI between a subset of features can be expressed as the Conditional Mutual Information (…
Due to the advantage of achieving a better performance under weak regularization, elastic net has attracted wide attention in statistics, machine learning, bioinformatics, and other fields. In particular, a variation of the elastic net, adaptive elastic net (AEN), integrates the adaptive grouping effect. In this paper,…
Testing for conditional independence is a core aspect of constraint-based causal discovery. Although commonly used tests are perfect in theory, they often fail to reject independence in practice, especially when conditioning on multiple variables. We focus on discrete data and propose a new test based on the notion of …
This paper surveys a few aspects of the global theory of wave equations. This material is structured around the contents of a minicourse given by the second author during the CMI/ETH Summer School on evolution equations during the Summer of 2008.
Canonical Correlation Analysis (CCA) is a classic technique for multi-view data analysis. To overcome the deficiency of linear correlation in practical multi-view learning tasks, various CCA variants were proposed to capture nonlinear dependency. However, it is non-trivial to have an in-principle understanding of these…
Feature selection aims to select the smallest feature subset that yields the minimum generalization error. In the rich literature in feature selection, information theory-based approaches seek a subset of features such that the mutual information between the selected features and the class labels is maximized. Despite …
Develops a new framework for analyzing sequential decision-making problems using information theory.
problem Lack of information-theoretic generalization bounds for sequential decision-making problems.
method Introduces a sequential supersample framework that separates learner filtration from proof-side enlargement, controlling the generalization gap by sequential CMI.
result Establishes a sequential CMI that controls the generalization gap in sequential decision-making problems.
We describe an end-to-end real-time S&P futures trading system. Inner-shell stochastic nonlinear dynamic models are developed, and Canonical Momenta Indicators (CMI) are derived from a fitted Lagrangian used by outer-shell trading models dependent on these indicators. Recursive and adaptive optimization using Adaptive …