New exact tests detect changepoints in binary and count data, especially when normal approximations fail.
problem Detecting changepoints in multichannel binary and count data.
method Exact tests combining two-sample conditional tests with multiplicity correction.
result Exact tests are much more powerful than asymptotic tests in various settings.
We investigate multiple testing and variable selection using the Least Angle Regression (LARS) algorithm in high dimensions under the assumption of Gaussian noise. LARS is known to produce a piecewise affine solution path with change points referred to as the knots of the LARS path. The key to our results is an express…
New test detects sparse alternatives in Gaussian random fields.
problem Detecting sparse alternatives in Gaussian random fields.
method Ad-hoc Kac Rice formula for second maximum distribution, exact spacing test.
result Exact t-spacing test for high power in detecting sparse alternatives. New algorithm achieves almost exact graph matching in almost quadratic time.
problem Graph matching under correlated Erdős-Rényi models.
method Rank-based graph matching using local tree correlation tests.
result Achieves almost exact recovery in almost quadratic time complexity.
New tests for binary classification regression functions without distribution assumptions.
problem Testing regression functions in binary classification without distributional assumptions.
method Conditional kernel mean embeddings and resampling-based framework.
result Distribution-free hypothesis tests with exact type I error control.
Cheap permutation tests speed up distribution testing without sacrificing accuracy.
problem Efficiently testing distribution differences and independence.
method Group datapoints into bins and permute only these bins, using stored sufficient statistics.
result Cheap permutation tests maintain the accuracy and optimality of standard tests but are significantly faster.
A new permutation method improves two-sample testing power.
problem Two-sample testing with improved power and validity.
method Structured block-restricted cross-swaps.
result Block-restricted permutations achieve higher power than full permutations.
This study proposes the segmentation procedure of univariate time series based on Fisher's exact test. We show that an adequate change point can be detected as the minimum value of p-value. It is shown that the proposed procedure can detect change points for an artificial time series. We apply the proposed method to fi…
Recent findings in multi-agent deep learning systems point towards the emergence of compositional languages. These claims are often made without exact analysis or testing of the language. In this work, we analyze the emergent language resulting from two different cooperative multi-agent game with more exact measures fo…
New framework for robust hypothesis testing using Sinkhorn uncertainty sets.
problem Non-convex robust hypothesis testing problem.
method Exact mixed-integer exponential conic reformulation and convex approximation.
result Satisfactory testing performance and computational efficiency.
DD algorithm tracks test error from train error without validation data.
problem Systematic generalization gap between train and test errors in modern model training.
method Decoupled descent (DD) algorithm that cancels data reuse biases via approximate message passing.
result DD algorithm rigorously demonstrates zero-cost validation and 100% data utilization.
We analyze how an observer synchronizes to the internal state of a finite-state information source, using the epsilon-machine causal representation. Here, we treat the case of exact synchronization, when it is possible for the observer to synchronize completely after a finite number of observations. The more difficult …
The paper improves confidence intervals for test error using cross-validation.
problem Improving confidence intervals for test error in machine learning.
method Develops central limit theorems and consistent estimators for cross-validation.
result Provides asymptotically-exact confidence intervals and hypothesis tests.
We develop a framework for post model selection inference, via marginal screening, in linear regression. At the core of this framework is a result that characterizes the exact distribution of linear functions of the response y, conditional on the model being selected (``condition on selection" framework). This allows…
Accurate goodness-of-fit tests for the extreme tails of empirical distributions is a very important issue, relevant in many contexts, including geophysics, insurance, and finance. We have derived exact asymptotic results for a generalization of the large-sample Kolmogorov-Smirnov test, well suited to testing these extr…
In this paper, we propose a convergent parallel best-response algorithm with the exact line search for the nondifferentiable nonconvex sparsity-regularized rank minimization problem. On the one hand, it exhibits a faster convergence than subgradient algorithms and block coordinate descent algorithms. On the other hand,…
Exact second-order optimization for deep learning reduces computational cost and improves performance.
problem Inadequate use of second-order optimization methods in deep learning due to high computational cost and non-convexity.
method Developed an exact stochastic second-order Newton method that addresses the non-convexity issue and provides an expression for the stochastic Hessian.
result Exact second-order Newton direction formula and its application in deep learning datasets.
Study tests whether trade-off functions are above or below benchmarks using finite samples.
problem Testing trade-off functions between unknown distributions.
method Identifies a condition for nontrivial testing, constructs a test with error guarantees, and inverts the test for confidence bands.
result Finite-sample testing is possible under specific structural assumptions about rejection regions.
A new metric, Weighted Regret, unifies FDR and power evaluation in online multiple testing.
problem The asymmetric costs of false positives and false negatives in automated pipelines.
method Introducing Weighted Regret and Decoupled-OMT (DOMT) to unify FDR and power evaluation.
result DOMT achieves an order-optimal sublinear mitigation of threshold depletion in bursty environments.
Study shows gap between uniform convergence and test error in random feature models.
problem Understanding the gap between uniform convergence and test error in random feature models.
method Analytical expressions for uniform convergence over norm balls, interpolators, and minimum norm interpolator risk derived and proved.
result Uniform convergence over interpolators still gives a non-trivial bound of test error even when classical uniform convergence is vacuous.
We present an extension of the Kolmogorov-Smirnov (KS) two-sample test, which can be more sensitive to differences in the tails. Our test statistic is an integral probability metric (IPM) defined over a higher-order total variation ball, recovering the original KS test as its simplest case. We give an exact representer…
Paper uses Shapley values to identify key confounders in product funnel data.
problem Identifying important confounders in product funnel data.
method Applies Shapley values for scalable coarsened exact matching.
result Shapley values provide robust importance-ranking for confounders.
MFVI can overestimate predictive variance compared to the exact posterior
problem MFVI underestimates posterior variance
method Analyzing conjugate Bayesian Linear Regression
result MFVI can overestimate predictive variance compared to the exact posterior
New simulation method simplifies Heston model with Poisson conditioning for better accuracy and efficiency.
problem Computational expense in exact simulation schemes for Heston model.
method Proposes a new exact simulation scheme without modified Bessel function evaluations, leveraging conditional integrated variance simplification.
result Good performance in terms of accuracy, efficiency, and reliability compared to existing methods.
Boosts A/B test precision using auxiliary data from historical users.
problem Small sample sizes and imprecise estimates in A/B tests.
method Coupling design-based causal estimation with machine-learning models of historical user data.
result Effect estimates using auxiliary data are roughly equivalent to increasing sample size by 20%, or up to 50-80% in some cases.
New insights into how high-dimensional models handle covariate shifts.
problem Covariate shift in high-dimensional random feature regression.
method Exact high-dimensional asymptotics of random feature regression under covariate shift.
result Overparameterized models exhibit enhanced robustness to covariate shift.
Distance-based tests, also called "energy statistics", are leading methods for two-sample and independence tests from the statistics community. Kernel-based tests, developed from "kernel mean embeddings", are leading methods for two-sample and independence tests from the machine learning community. A fixed-point transf…
Locally-verifiable conditions ensure exactness of spline discrete de Rham complex.
problem Ensuring cohomological equivalence of spline discrete complex to continuous de Rham complex.
method Theoretical analysis and locally-verifiable sufficient conditions for exactness.
result Locally-verifiable conditions guarantee exactness of hierarchical B-spline discrete de Rham complex.
A machine learning configuration refers to a combination of preprocessor, learner, and hyperparameters. Given a set of configurations and a large dataset randomly split into training and testing set, we study how to efficiently select the best configuration with approximately the highest testing accuracy when trained f…
The study analyzes group testing algorithms for identifying defective items with high confidence.
problem Identifying defective items from a population using group testing with high confidence.
method Formulated as a function learning problem using the PAC framework, analyzed three algorithms: column matching, combinatorial basis pursuit, and definite defectives.
result Derived bounds on the number of tests needed for approximate set identification, comparing with existing bounds and simulating performance.
CSD improves goodness-of-fit testing for higher-order dependence.
problem Insensitivity of standard KSDs to higher-order dependence features like tail dependence.
method Introduces Copula-Stein Discrepancy (CSD) that targets dependence geometry directly on copula density.
result CSD is sensitive to differences in tail dependence coefficients and metrizes weak convergence of copula distributions.
Study on test risk dynamics in learning theory with stochastic gradient flow.
problem Understanding test risk in stochastic gradient flow dynamics.
method Path integral formulation for small learning rates, explicit computation for weak features.
result Explicit corrections due to stochastic term in dynamics, good agreement with simulations.
Estimates and tests treatment effects on entire outcome distributions.
problem Treatment effects on entire outcome distributions, not just averages.
method Proposes a novel estimand and doubly robust estimator, develops a test.
result First test with provably valid type 1 error guarantees in this setting.
Tests for classifier independence without ground truth labels.
problem Validation of classifier independence without ground truth labels.
method Exact solution for independent binary classifiers using algebraic geometry.
result Self-consistent test for classifier independence without ground truth labels.
The Restricted Boltzmann Machines (RBM) can be used either as classifiers or as generative models. The quality of the generative RBM is measured through the average log-likelihood on test data. Due to the high computational complexity of evaluating the partition function, exact calculation of test log-likelihood is ver…
Weighted log-rank tests are arguably the most widely used tests by practitioners for the two-sample problem in the context of right-censored data. Many approaches have been considered to make weighted log-rank tests more robust against a broader family of alternatives, among them, considering linear combinations of wei…
We establish an integral test describing the exact cut-off between recurrence and transience for normally reflected Brownian motion in certain unbounded domains in a class of warped product manifolds. Besides extending a previous result by R. Pinsky, who treated the case in which the ambient space is flat, our result r…
Statistical test rejects market efficiency using entropy from price returns.
problem Determining market efficiency using information theory.
method Symbolic representation of price returns, Shannon entropy, and statistical test.
result Rejects market efficiency hypothesis for various datasets.
Sequential Kernel-based Conditional Independence Testing via Adaptive Betting
problem Testing conditional independence
method Testing-by-betting on an adaptively optimized Kernel Conditional Independence statistic
result Significantly reduces Type I error inflation while preserving high power
Given two sets of independent samples from unknown distributions P and Q, a two-sample test decides whether to reject the null hypothesis that P=Q. Recent attention has focused on kernel two-sample tests as the test statistics are easy to compute, converge fast, and have low bias with their finite sample estimate…
Decentralized Gaussian processes for multi-agent systems.
problem Scalable and flexible learning solutions for multi-agent systems.
method Asymptotically exact decentralized solution to Gaussian processes, with online Bayesian model averaging for hyperparameter selection.
result Asymptotically exact decentralized Gaussian process approximation and online Bayesian model averaging.
Machine learning boosts RCT efficiency by controlling type I error and improving statistical power.
problem Improving statistical efficiency in RCTs with complex covariate adjustments.
method Machine learning-assisted adjustment under Rosenbaum's framework for exact tests.
result The proposed method robustly controls type I error and significantly boosts statistical efficiency.
Develops a framework to test excessive influence of small data subsets.
problem Identifying when small data subsets significantly impact model conclusions.
method Formalizes the concept of most influential sets, deriving influence formulas and extreme value distributions.
result Allows rigorous hypothesis testing for excessive influence, resolving contested findings.
This paper studies the geometry of minimum-volume confidence sets for multinomial parameters.
problem Determining if minimum-volume confidence sets for multinomial outcomes are disjoint.
method Enumerating and covering the continuous regions of the exact p-value function to study the geometry of minimum-volume confidence sets.
result The geometry of minimum-volume confidence sets for multinomial parameters is studied, providing insights into their structure and properties.
We consider clustering based on significance tests for Gaussian Mixture Models (GMMs). Our starting point is the SigClust method developed by Liu et al. (2008), which introduces a test based on the k-means objective (with k = 2) to decide whether the data should be split into two clusters. When applied recursively, thi…
Paper certifies intersection of minimum-volume confidence sets for multinomial outcomes.
problem Certifying intersection of minimum-volume confidence sets for multinomial outcomes.
method Exploits likelihood ordering to induce halfspace constraints, enabling adaptive geometric partitioning and computable bounds on p-values.
result Efficient and provably sound algorithm for certifying intersection, disjointness, or indeterminate result.
We characterize the asymptotic performance of nonparametric one- and two-sample testing. The exponential decay rate or error exponent of the type-II error probability is used as the asymptotic performance metric, and an optimal test achieves the maximum rate subject to a constant level constraint on the type-I error pr…
TDS provides exact samples for conditional distributions in diffusion models.
problem Lack of exact sampling methods for diffusion models.
method Sequential Monte Carlo (SMC) algorithm with twisting technique.
result TDS offers more accurate approximations with fewer particles compared to heuristics.