Improves SGD convergence by online linear regression of gradients in multiple directions.
problem SGD's rough approximations of gradients lead to suboptimal plateaus and saddle points.
method Online linear regression of noisy gradients using PCA to estimate second-order behavior, and gradient descent outside the subspace.
result Improves convergence by avoiding suboptimal plateaus and saddles, leading to better final values.
Survey on statistical learning theory for control, focusing on linear systems.
problem Applying machine learning techniques to control systems, especially linear ones.
method Adapting tools from modern high-dimensional statistics and learning theory.
result Recent advances in statistical learning theory for control, particularly for linear systems.
The study finds a time lag effect in FDI-GDP correlations, with significant statistical significance.
problem The relationship between FDI and GDP growth is not immediate.
method Time-dependent Pearson correlation coefficient matrix analysis of 43 countries' data from 1970-2015.
result The correlation between FDI and GDP growth is time-lagged, evolving from positive to negative as inequality-adjusted human development index increases.
Data repetition improves SGD's learning of high-dimensional functions.
problem Learning pertinent features in multi-index models with high-dimensional noisy data.
method Investigation of two-layer shallow neural networks trained with gradient-based algorithms, focusing on data repetition.
result Data repetition significantly improves the computational efficiency of SGD, learning all directions with at most O(dlogd) steps. The paper introduces a new method for interpreting model predictions by considering both direct and indirect effects.
problem Interpreting model predictions to understand the causes of decisions.
method Proposes a new approach to quantify feature relevance by combining different types of interpretations and measures.
result Integrates various types of interpretations and measures to provide meaningful insights into model predictions.
New research shows LLMs can't be explained by statistical generalization alone.
problem Understanding why large language models (LLMs) perform well despite statistical generalization limitations.
method Examined the non-identifiability of AR probabilistic models and their implications for LLMs.
result Non-identifiability of LLMs leads to different behaviors and requires a separate theoretical explanation.
The paper addresses the relevance problem in statistical inference.
problem The relevance problem in statistical inference from large-scale data.
method Not specified in the abstract, likely involves statistical methods and analysis of large-scale data.
result The relevance problem is a long-neglected topic in statistical inference.
Federated learning trains models on remote devices without sharing data.
problem Training models on remote devices with limited data.
method Develops new methods for distributed optimization and privacy.
result New challenges and approaches for large-scale machine learning.
This thesis improves practical reinforcement learning methods with robustness, scalability, and efficiency.
problem Improving reinforcement learning methods for practical applications.
method Analyzes and develops robust, scalable, and efficient reinforcement learning algorithms.
result Proves the efficiency and robustness of new RL methods.
This paper provides a construction of a quantum statistical mechanical system associated to knots in the 3-sphere and cyclic branched coverings of the 3-sphere, which is an analog, in the sense of arithmetic topology, of the Bost-Connes system, with knots replacing primes, and cyclic branched coverings of the 3-sphere …
The agent-based model of stock price dynamics on a directed evolving complex network is suggested and studied by direct simulation. The stationary regime is maintained as a result of the balance between the extremal dynamics, adaptivity of strategic variables and reconnection rules. The inherent structure of node agent…
The intermarket analysis, in particular the lead-lag relationship, plays an important role within financial markets. Therefore a mathematical approach to be able to find interrelations between the price development of two different financial underlyings is developed in this paper. Computing the differences of the relat…
Neural networks speed up statistical inference.
problem Efficient statistical inference for complex models.
method Neural networks for learning complex mappings.
result Amortized inference speeds up inference processes.
DEDACT breaks down feature importance into direct and associative components.
problem Lack of clear distinction between direct and associative feature importance.
method DEDACT framework to decompose direct and associative importance measures.
result Provides insight into sources of prediction-relevant information and feature pathways.
Robustly detects and attributes climate change impacts under interventions.
problem Detect and attribute climate change impacts from observations robustly.
method Supervised learning with anchor regression for robust predictions under interventions.
result CO2 forcing can be robustly predicted from temperature patterns under strong solar forcing interventions.
New method distinguishes feature relevance in non-linear contexts.
problem Finding relevant features with preserved redundancies.
method Random forest models and statistical methods.
result Distinguishes strong from weak feature relevance in non-linear problems.
Framework for interpreting ML models to reveal properties of real-world phenomena.
problem Lack of direct interpretability in modern ML models hinders scientific understanding.
method Developed 'property descriptors' grounded in statistical learning theory.
result Property descriptors can reveal relevant properties of joint probability distributions.
The paper develops a method to identify conditionally relevant features with statistical guarantees.
problem Identifying features that are relevant given the values of other features.
method A generalization of the knockoff procedure that controls a generalized FDR for conditional feature selection.
result The method provides a statistical guarantee for conditional feature selection.
Sparse Polyak improves high-dimensional statistical estimation.
problem High-dimensional statistical estimation problems with growing problem dimension.
method Sparse Polyak modifies Polyak's adaptive step size to estimate restricted Lipschitz smoothness.
result Sparse Polyak achieves optimal statistical precision with fewer iterations.
The paper defines and analyzes IC using high-dimensional directional statistics.
problem Defining and analyzing the Information Coefficient (IC) in high-dimensional settings.
method High-dimensional directional statistics, closed-form expressions, optimization, simulation, empirical analysis.
result Explicit results of the projected normal distribution and IC's nature.
The relevance of data quantifies learning efficiency.
problem Understanding the statistical nature of high-dimensional, sparse data.
method Defining relevance as information content, and using it to define ideal limits of samples and learning machines.
result Maximally informative samples and optimal learning machines exhibit critical features like power-law frequency distributions and anomalously large susceptibility.
We show how to learn low-dimensional representations (embeddings) of patient visits from the corresponding electronic health record (EHR) where International Classification of Diseases (ICD) diagnosis codes are removed. We expect that these embeddings will be useful for the construction of predictive statistical models…
Mamba efficiently learns low-dimensional targets in-context via feature extraction.
problem Learning low-dimensional targets in context for computational efficiency.
method Test-time feature learning of a single-index model using Mamba's pretrained linear-time sequence model.
result Mamba achieves efficient in-context learning of low-dimensional targets via feature extraction.
A new depth function improves multivariate data analysis by considering variability directions.
problem Developing a depth function that respects quantile properties and is affine-invariant.
method Integrating rank-weighted depth with affine-invariance and covariance matrices.
result The AI-IRW depth function provides accurate quantile estimates and is robust to data variability.
We introduce a new factor model for log volatilities that performs dimensionality reduction and considers contributions globally through the market, and locally through cluster structure and their interactions. We do not assume a-priori the number of clusters in the data, instead using the Directed Bubble Hierarchical …
This paper summarizes closed-form relations for SE(3) maps and their derivatives.
problem Closed-form expressions for SE(3) maps and their derivatives are scattered in the literature.
method Summarizes and provides proofs for relevant closed-form relations of the exponential and Cayley map on SE(3).
result Provides an implicit generalized-alpha scheme for rigid/flexible multibody systems using the Cayley map.
Bayesian neural network improves feature selection and prediction.
problem Improving feature selection and prediction accuracy in neural networks.
method BNN-ARD with l2-norm feature importance measure.
result Improves variable selection and predictive performance on real-world data.
Embeds directed graphs into statistical manifolds for better geodesic preservation.
problem Preserving global geodesic information in directed graphs.
method Global minimization of pairwise relative entropy and graph geodesics.
result Our embedding outperforms existing models in various evaluation metrics.
Statistical methods remain relevant for ODE inverse problems, especially with sparse data.
problem The relevance of statistical methods in the era of deep learning for ODE inverse problems.
method Employed physics-informed neural networks (PINN) and manifold-constrained Gaussian process inference (MAGI) to compare statistical and deep learning approaches.
result Statistically principled methods outperform deep learning models in tasks like parameter inference and trajectory reconstruction.
Direct policy gradient methods for reinforcement learning and continuous control problems are a popular approach for a variety of reasons: 1) they are easy to implement without explicit knowledge of the underlying model 2) they are an "end-to-end" approach, directly optimizing the performance metric of interest 3) they…
Proposes RCVs for explaining deep neural network predictions in medical images.
problem Need for explainable predictions in medical applications.
method Uses continuous concept measures as RCVs in neural network activation space.
result Nuclei texture is a relevant concept in breast cancer grading.
Dimensional reduction of high dimensional data can be achieved by keeping only the relevant eigenmodes after principal component analysis. However, differentiating relevant eigenmodes from the random noise eigenmodes is problematic. A new method based on the random matrix theory and a statistical goodness-of-fit test i…
Data coarse graining improves model performance by filtering out less relevant features.
problem Lossy data transformations lose information but can improve model generalization.
method Data coarse graining schemes that systematically discard features based on relevance to the learning task.
result A 'high-pass' scheme helps models generalize better by filtering out less relevant features.
This paper examines sources of uncertainty in machine learning from a statistical perspective.
problem Quantifying uncertainty in supervised machine learning models.
method A conceptual, basic science approach examining aleatoric and epistemic uncertainty.
result Sources of uncertainty are diverse and cannot always be decomposed into aleatoric and epistemic.
The paper studies kernel smoothing and mean shift for directional data, deriving convergence rates and mode estimation.
problem Statistical and computational problems of kernel smoothing for directional data.
method Generalization of mean shift to directional data, derivation of convergence rates, and investigation of mode estimation.
result Statistical convergence rates of directional KDE and its derivatives, ascending property of directional mean shift, and mode estimation.
Algorithm detects unmeasured confounding in observational data.
problem Estimating treatment effects in observational studies with untestable conditions.
method Two-stage procedure that detects dependencies between causal mechanisms.
result Algorithm efficiently detects confounding on simulated and semi-synthetic data.
Paper proposes a method to compare vector fields across surfaces, useful for analyzing brain folding patterns.
problem Comparing vector fields across surfaces of different geometries is challenging.
method The paper introduces a framework to transport vector fields onto a common space using differential geometry.
result The proposed framework enables the computation of statistics on vector fields, demonstrating its effectiveness in analyzing brain folding patterns.
We consider the problem of variable selection in high-dimensional statistical models where the goal is to report a set of variables, out of many predictors X1,…,Xp, that are relevant to a response of interest. For linear high-dimensional model, where the number of parameters exceeds the number of samples $(p…
Elliptical Attention improves transformer performance by focusing on contextually relevant features.
problem Transformer models suffer from representation collapse and are vulnerable to contaminated samples.
method Uses Mahalanobis distance to define hyper-ellipsoidal neighborhoods for attention weights.
result Elliptical Attention reduces representation collapse and enhances model robustness.
DKMD is a fast signed statistic for comparing univariate distributions.
problem Comparing univariate distributions, especially preserving directionality.
method DKMD integrates kernel mean embeddings against an odd weighting function.
result DKMD preserves directionality and is robust to outliers.
Paper proposes a method to make learned models focus on task-relevant information.
problem Mismatch between model objective and downstream task objective.
method Direct prediction towards task-relevant information, self-supervised.
result Model more effectively models relevant parts of the scene conditioned on the goal.
Review of mean-field methods for neural network inference.
problem Understanding neural network learning from a theoretical perspective.
method Mean-field methods, high-temperature expansions, replica method, message passing algorithms.
result Equivalences and complementarities of mean-field methods.
Recently, a technique called Layer-wise Relevance Propagation (LRP) was shown to deliver insightful explanations in the form of input space relevances for understanding feed-forward neural network classification decisions. In the present work, we extend the usage of LRP to recurrent neural networks. We propose a specif…
Paper develops online statistical inference methods for stochastic optimization using Kiefer-Wolfowitz algorithms.
problem Online statistical inference of model parameters in stochastic optimization problems.
method Kiefer-Wolfowitz algorithm with random search directions, asymptotic distribution analysis.
result Developed valid confidence intervals for online statistical inference.
Feature selection with high-dimensional data and a very small proportion of relevant features poses a severe challenge to standard statistical methods. We have developed a new approach (HARVEST) that is straightforward to apply, albeit somewhat computer-intensive. This algorithm can be used to pre-screen a large number…
Develops a statistical model for SOFR term structure in incomplete markets.
problem Incomplete liquidity and completeness in SOFR derivatives market.
method Statistical model incorporating macroeconomic factors and jumps in SOFR rates.
result Model is well-suited for risk management and derivatives pricing.
Task-agnostic data valuation without validation requirements.
problem Valuing data without specific task assumptions.
method Estimating data diversity and relevance through queries without raw data.
result Estimates capture the diversity and relevance of seller's data for the buyer.
Improves relevancy of black-box anomaly detectors with user feedback.
problem Users often ignore many detected anomalies, requiring a method to identify and prioritize relevant ones.
method Uses user feedback to adjust anomaly selection process based on identified anomaly types.
result Significant improvements in precision and recall over various anomaly detectors.