This study shows neural nets can approximate Turing machines with meaningful statistical properties.
problem Theoretical limitations in approximating Turing machines with neural networks.
method Formal definition of statistically meaningful approximation, analysis of boolean circuits and Turing machines using neural nets.
result Transformers can statistically meaningfully approximate Turing machines with polynomial sample complexity.
Paper learns meaningful state and action representations from MDP trajectories.
problem Learning good state and action representations from MDP trajectories.
method Tensor decomposition, kernelization, importance sampling, low-Tucker-rank approximation.
result The learned state/action abstractions provide accurate approximations to latent block structures.
New method improves uncertainty quantification in latent variable models.
problem Uncertainty quantification in latent variable models with SGLD-Gibbs.
method Statistical scaling limit theory for SGLD-Gibbs, proposing hyperparameter tuning.
result Explicit guidance on hyperparameter tuning for SGLD-Gibbs ensures meaningful uncertainty quantification.
We show that training a deep network using batch normalization is equivalent to approximate inference in Bayesian models. We further demonstrate that this finding allows us to make meaningful estimates of the model uncertainty using conventional architectures, without modifications to the network or the training proced…
Paper proves MDS NP-hard and provides a PTAS.
problem Theoretical limitations of MDS objective function.
method Proves NP-hardness and provides a PTAS approximation algorithm.
result Minimizing Kamada-Kawai objective is NP-hard.
A new method for unlearning trained models without needing the original data.
problem Lack of access to original training data for privacy-preserving unlearning.
method Uses a surrogate dataset to approximate statistical properties and calibrates noise based on statistical distance.
result Effective unlearning of trained models with strong privacy guarantees, even without access to the original data.
New test identifies specific biological parameters for personalized CVD detection.
problem Ineffectual pathology tests fail to consider platelet activation and inter-individual variability.
method Stochastic platelet deposition model and approximate Bayesian computation with discriminative summary statistics.
result Inferred parameters help identify specific biological parameters for personalized CVD detection.
The paper explores intersectional fairness in machine learning, proving bounds on it.
problem Intersectional fairness in machine learning, especially when multiple protected attributes are involved.
method Statistical analysis and bounds on intersectional fairness, leveraging marginal fairness.
result Theoretical bounds on intersectional fairness can be computed from marginal fairness and other statistical quantities.
Sampling from various kinds of distributions is an issue of paramount importance in statistics since it is often the key ingredient for constructing estimators, test procedures or confidence intervals. In many situations, the exact sampling from a given distribution is impossible or computationally expensive and, there…
A new method learns meaningful distances between samples using optimal transport.
problem Learning meaningful distances between samples in datasets without labeled data.
method Computes OT distances between samples and features using singular vectors of a function mapping ground metrics to OT distances.
result Wasserstein Singular Vectors provide a scalable solution for unsupervised ground metric learning.
New algorithms estimate matrix leverage scores using rank revealing and randomization.
problem Estimating leverage scores for matrices of arbitrary rank.
method Combining rank revealing methods with randomized dimensionality reduction.
result Effective estimators for leverage scores, even in rank deficient cases.
Local decision boundary approximation improves model explanations for complex models.
problem Challenges in explaining complex, opaque machine learning models.
method Train a variational autoencoder to learn a latent space and map it to meaningful attributes. Use these attributes to approximate the local decision boundary and explain model predictions.
result Can recover latent attributes that determine class decisions in a new benchmark data set.
Kernel tests assess equivalence between distributions without assuming specific moments.
problem Traditional goodness-of-fit tests fail to detect meaningful distributional differences.
method Proposes kernel-based tests using kernel Stein discrepancy and Maximum Mean Discrepancy.
result Tests assess the absence of meaningful distributional differences under controlled error rates.
Paper presents a method to summarize HMC samples for neural networks, providing meaningful uncertainty estimates.
problem Lack of interpretable summary statistics for HMC samples in neural networks due to permutation symmetry.
method Introducing a transpositions metric to quantify permutations and using rebasin method to summarize HMC samples.
result Compact representation of HMC samples provides meaningful uncertainty estimates for each weight in a neural network.
We propose a new family of fairness definitions for classification problems that combine some of the best properties of both statistical and individual notions of fairness. We posit not only a distribution over individuals, but also a distribution over (or collection of) classification tasks. We then ask that standard …
In safety-critical applications a probabilistic model is usually required to be calibrated, i.e., to capture the uncertainty of its predictions accurately. In multi-class classification, calibration of the most confident predictions only is often not sufficient. We propose and study calibration measures for multi-class…
The Delta method is a classical procedure for quantifying epistemic uncertainty in statistical models, but its direct application to deep neural networks is prevented by the large number of parameters P. We propose a low cost variant of the Delta method applicable to L2-regularized deep neural networks based on th…
Latent variable time-series models are among the most heavily used tools from machine learning and applied statistics. These models have the advantage of learning latent structure both from noisy observations and from the temporal ordering in the data, where it is assumed that meaningful correlation structure exists ac…
We introduce new families of Integral Probability Metrics (IPM) for training Generative Adversarial Networks (GAN). Our IPMs are based on matching statistics of distributions embedded in a finite dimensional feature space. Mean and covariance feature matching IPMs allow for stable training of GANs, which we will call M…
Spectral denoising recovers meaningful network structure from noisy financial correlations.
problem Noise in empirical correlation matrices from financial returns obscures genuine interactions.
method Spectral decomposition to separate structured and random components.
result Structured networks derived from 10-16 eigenmodes exhibit stronger core-periphery organization and scale-free degree distributions.
New KQEs improve probability metrics without mean function constraints.
problem Improving probability metrics without relying on mean function representations.
method Kernel quantile embeddings (KQEs) to construct new distances.
result KQEs offer a competitive alternative to MMD with near-linear cost.
Bayesian methods solve complex nonlinear PDEs efficiently.
problem Solving nonlinear PDEs with high computational cost.
method Bayesian inference with approximate likelihood based on discretization.
result Probabilistic uncertainty quantification for PDE solutions is feasible.
We deconstruct the performance of GANs into three components: 1. Formulation: we propose a perturbation view of the population target of GANs. Building on this interpretation, we show that GANs can be viewed as a generalization of the robust statistics framework, and propose a novel GAN architecture, termed as Cascade …
A new DL framework preserves geometric structures for causal predictions.
problem Designing deep learning models for geometrically structured data.
method Introduces a universal causal geometric DL framework.
result DL models can approximate any regular map between metric spaces.
Wasserstein GANs fail to approximate Wasserstein distance, leading to their success.
problem Approximating Wasserstein distance in deep generative models.
method Analysis of differences between theoretical setup and training reality.
result Wasserstein GANs' success is due to their failure to approximate Wasserstein distance.
Non-symmetric rectangular correlation matrices occur in many problems in economics. We test the method of extracting statistically meaningful correlations between input and output variables of large dimensionality and build a toy model for artificially included correlations in large random time series.The results are t…
A test for comparing networks using stochastic block models.
problem Determining if two network datasets come from the same model.
method Adopting stochastic block models, the study introduces an efficient algorithm to match estimated network parameters and develops a powerful test.
result The test is consistent and asymptotically follows a chi-squared distribution.
New framework for interpretable firm characteristics factors.
problem Creating statistically efficient and economically interpretable factors from firm characteristics.
method Grouping related characteristics and deriving one factor per group, combining economic intuition with data-driven clustering.
result Parsimonious, transparent factors outperform benchmarks in out-of-sample tests.
The authors argue against the classification of forecasting methods as machine learning or statistical.
problem The classification of forecasting methods as machine learning or statistical limits insights into their appropriateness and effectiveness.
method Alternative characteristics of forecasting methods are proposed to draw meaningful conclusions.
result The distinction between machine learning and statistical forecasting methods is not fundamental.
This report concerns the problem of dimensionality reduction through information geometric methods on statistical manifolds. While there has been considerable work recently presented regarding dimensionality reduction for the purposes of learning tasks such as classification, clustering, and visualization, these method…
Bayesian framework proves thresholds for multi-graph alignment feasibility.
problem Determining when multi-graph alignment is statistically possible.
method Developed a Bayesian estimation framework over metric spaces.
result Identified thresholds for Gaussian and sparse Erdős-Rényi models.
Label switching is a phenomenon arising in mixture model posterior inference that prevents one from meaningfully assessing posterior statistics using standard Monte Carlo procedures. This issue arises due to invariance of the posterior under actions of a group; for example, permuting the ordering of mixture components …
Machine learning predicts Bitcoin returns but trading performance drops with costs.
problem Trading Bitcoin predictions with transaction costs.
method XGBoost, LSTM, iTransformer models evaluated in walk-forward protocol; cost-aware execution filter implemented.
result Cost-aware execution filter restores profitability; XGBoost strategy outperforms.
The paper assesses fairness in risk score models, focusing on epistemic value.
problem Fairness of risk score models in communicating uncertainty.
method Identified key fairness desiderata, developed metrics for quantitative assessment, and applied methodology in two case studies.
result Introduced a novel calibration error metric for meaningful comparisons between groups of different sizes.
Intelligent behaviour in the real-world requires the ability to acquire new knowledge from an ongoing sequence of experiences while preserving and reusing past knowledge. We propose a novel algorithm for unsupervised representation learning from piece-wise stationary visual data: Variational Autoencoder with Shared Emb…
Study shows limitations and possibilities of learning quantum circuit output distributions.
problem Learnability of output distributions of local quantum circuits.
method Investigated within two oracle models: statistical query model and direct sample access model.
result Output distributions of super-logarithmic depth Clifford circuits are not efficiently learnable in the statistical query model.
Policy evaluation with linear function approximation is an important problem in reinforcement learning. When facing high-dimensional feature spaces, such a problem becomes extremely hard considering the computation efficiency and quality of approximations. We propose a new algorithm, LSTD(λ)-RP, which leverages rando…
Nyström KPCA balances computational efficiency and statistical accuracy.
problem Computational burden in large sample situations for kernel methods.
method Theoretical analysis of Nyström approximate kernel principal component analysis (KPCA).
result Nyström approximate KPCA matches statistical performance of non-approximate KPCA while being computationally beneficial.
Machine learning models are vulnerable to adversarial inputs that induce seemingly unjustifiable errors. As automated classifiers are increasingly used in industrial control systems and machinery, these adversarial errors could grow to be a serious problem. Despite numerous studies over the past few years, the field of…
New approach to meaningful and robust algorithmic recourse.
problem Ineffective and unmeaningful algorithmic recourse explanations.
method Meaningful Algorithmic Recourse (MAR) and Effective Algorithmic Recourse (EAR).
result Proposes new constraints for algorithmic recourse that improve both prediction and target.
Improved texture synthesis using wavelet-based statistics with rectifier non-linearity.
problem Improving texture synthesis quality using wavelet representations.
method Proposes a family of statistics based on non-linear wavelet representations with a generalized rectifier non-linearity.
result Significantly improves visual quality of texture synthesis compared to classical wavelet-based models.
Paper proposes a statistical test for feature selection pipelines using selective inference.
problem Assessing the significance of feature selection pipelines in data analysis.
method Selective inference technique applied to feature selection pipelines composed of various algorithms.
result The proposed statistical test controls false positive feature selection probabilities.
Study proposes a statistical testing framework for evaluating clustering pipelines.
problem Quantifying the statistical reliability of clustering results from data analysis pipelines.
method Selective inference-based statistical testing framework for clustering pipelines.
result The proposed test controls the type I error rate and is effective in validating clustering results.
ElbowSig assesses clustering structure at multiple scales.
problem Selecting optimal number of clusters in unsupervised learning.
method Formalizes elbow heuristic with a normalized discrete curvature statistic.
result Validates multiscale clustering structure over various resolutions.
Kernel methods are powerful learning methodologies that allow to perform non-linear data analysis. Despite their popularity, they suffer from poor scalability in big data scenarios. Various approximation methods, including random feature approximation, have been proposed to alleviate the problem. However, the statistic…
Approximate Bayesian Computation (ABC) methods are used to approximate posterior distributions in models with unknown or computationally intractable likelihoods. Both the accuracy and computational efficiency of ABC depend on the choice of summary statistic, but outside of special cases where the optimal summary statis…
Modern neural networks tend to be overconfident on unseen, noisy or incorrectly labelled data and do not produce meaningful uncertainty measures. Bayesian deep learning aims to address this shortcoming with variational approximations (such as Bayes by Backprop or Multiplicative Normalising Flows). However, current appr…
The paper develops new algorithms for KL-divergence NMF, proving convergence and performance.
problem Improving NMF for nonnegative data with KL divergence.
method Collect and analyze properties of KL objective function, propose and test new algorithms.
result Guaranteed non-increasing objective function for one proposed algorithm, global convergence.