We improve adversarial robustness calibration analysis for broader hypothesis sets.
problem Improving calibration for adversarial robustness in machine learning.
method A finer definition of calibration for adversarial robustness.
result Our results cover most common hypothesis sets in machine learning.
Study on calibration and consistency of adversarial surrogate losses.
problem Designing robust classifiers with theoretical guarantees.
method Extensive analysis of H-calibration and H-consistency of adversarial surrogate losses.
result Some convex loss functions and supremum-based convex losses are not H-calibrated for important hypothesis sets.
New tests for binary classification regression functions without distribution assumptions.
problem Testing regression functions in binary classification without distributional assumptions.
method Conditional kernel mean embeddings and resampling-based framework.
result Distribution-free hypothesis tests with exact type I error control.
Suppose some classifiers are selected from a set of hypothesis classifiers to form an equally-weighted ensemble that selects a member classifier at random for each input example. Then the ensemble has an error bound consisting of the average error bound for the member classifiers, a term for selectivity that varies fro…
Paper tackles efficient learning of non-convex hypotheses in metric spaces.
problem Efficiently find consistent hypotheses for non-convex hypotheses composed of possibly several disconnected regions.
method Proposes a general domain-independent algorithm for finding consistent weakly convex hypotheses and proves sufficient conditions for its efficiency.
result Shows that consistent hypothesis finding problem can be solved in polynomial time for a broad class of weakly convex hypotheses over metric spaces.
Study of loss functions for learning to defer, proving consistency.
problem Learning to defer in machine learning.
method Introduced a family of surrogate losses parameterized by Ψ and proved their consistency. result Proved realizable H-consistency and Bayes-consistency of specific surrogate losses. We investigate whether fractal markets hypothesis and its focus on liquidity and invest- ment horizons give reasonable predictions about dynamics of the financial markets during the turbulences such as the Global Financial Crisis of late 2000s. Compared to the mainstream efficient markets hypothesis, fractal markets hy…
New approach to nonuniform learnability using measure theory.
problem Nonuniform learnability of hypotheses with varying sample sizes.
method Measure theoretic approach to redefine nonuniform learnability, introducing a new algorithm (Generalize Measure Learnability).
result Achieved statistical consistency in learning countable hypothesis classes.
In this paper we address the problem of pool based active learning, and provide an algorithm, called UPAL, that works by minimizing the unbiased estimator of the risk of a hypothesis in a given hypothesis space. For the space of linear classifiers and the squared loss we show that UPAL is equivalent to an exponentially…
Enhanced consistency bounds derived for classification under a new noise condition.
problem Enhanced consistency bounds for classification under a new noise condition.
method Model Margin Noise (MM noise) assumption, derived enhanced H-consistency bounds.
result Enhanced H-consistency bounds under MM noise condition, interpolates between linear and square-root regimes.
Paper tackles hypothesis transfer learning for black-box models.
problem Difficult to build universal machine learning models across different institutions.
method Dynamic Knowledge Distillation (dkdHTL) with instance-wise weighting.
result Empirical results show the effectiveness of dkdHTL.
Study on top-k classification with new loss functions and algorithms.
problem Improving multi-class classification accuracy and cardinality trade-off.
method Introducing cardinality-aware loss functions and deriving their consistency bounds.
result New cardinality-aware algorithms for top-k classification. A framework for ranking with abstention, offering theoretical guarantees and practical effectiveness.
problem Making predictions with limited cost when uncertain.
method Introduces a novel ranking framework with abstention, analyzing theoretical consistency bounds.
result Extensive theoretical analysis including H-consistency bounds for linear and neural network models. Proof of Gaussian ML estimator consistency in linear auto-regressive models.
problem Consistency of Gaussian maximum likelihood estimator in linear auto-regressive models.
method Information-theoretic proof without stability assumptions.
result Nearly optimal non-asymptotic rates for parameter recovery.
CNN-F uses generative feedback to improve neural networks' robustness to perturbations.
problem Neural networks' vulnerability to input perturbations like noise and attacks.
method Enforces self-consistency in neural networks by incorporating generative recurrent feedback.
result CNN-F shows significantly improved adversarial robustness compared to conventional CNNs.
Improved pruning method finds winning neural network subnetworks.
problem Finding a small subnetwork that performs as well as a full neural network.
method Data-dependent pruning criterion using gradient of training loss.
result Data-dependent pruning improves existing pruning algorithms.
Method measures weight similarity in neural networks using normalization and statistical inference.
problem Quantifying weight similarity in non-convex neural networks.
method Chain normalization rule and hypothesis-training-testing statistical inference.
result Weights of identical neural networks converge to similar local solutions.
Natural and social multivariate systems are commonly studied through sets of simultaneous and time-spaced measurements of the observables that drive their dynamics, i.e., through sets of time series. Typically, this is done via hypothesis testing: the statistical properties of the empirical time series are tested again…
We find faster-converging sub-networks that significantly reduce adversarial training time.
problem Finding optimal sub-networks for adversarial training is costly and time-consuming.
method We identify a subset of sub-networks that converge faster during training.
result Sub-networks can reduce adversarial training time by up to 49%.
Study of estimation errors in surrogate loss minimizers, providing stronger guarantees than existing methods.
problem Estimation errors in surrogate loss minimizers for various hypothesis sets.
method Detailed study of H-consistency estimation error bounds, proving general theorems for distribution-dependent and independent settings. result Explicit bounds for zero-one and adversarial losses, showing enhancements under distributional assumptions.
We propose the Insertion-Deletion Transformer, a novel transformer-based neural architecture and training method for sequence generation. The model consists of two phases that are executed iteratively, 1) an insertion phase and 2) a deletion phase. The insertion phase parameterizes a distribution of insertions on the c…
A wild bootstrap method for nonparametric hypothesis tests based on kernel distribution embeddings is proposed. This bootstrap method is used to construct provably consistent tests that apply to random processes, for which the naive permutation-based bootstrap fails. It applies to a large group of kernel tests based on…
Hypothesis testing in singular models is fundamentally about identifiable vs. non-identifiable parameters.
problem Testing in singular models is inherently problematic due to non-identifiability and degeneracy of Fisher information.
method Formalized the overlap obstruction and showed that hypotheses over non-identifiable parameters are untestable, while those over identifiable parameters reduce to classical testing.
result Hypotheses over non-identifiable parameters are untestable, while those over identifiable parameters reduce to classical testing.
Kernel interpolation is inconsistent for norms with smoothness above a constant.
problem Inconsistency of kernel interpolation in reproducing kernel Hilbert spaces.
method Lower bounds for generalization error in Sobolev norms.
result Kernel interpolation is always inconsistent for norms with smoothness above a constant.
Deep learning's anomalous generalization explained by standard frameworks.
problem Anomalous generalization in deep neural networks.
method Intuitive understanding and rigorous characterization using PAC-Bayes and countable hypothesis bounds.
result Deep learning's anomalous generalization can be explained by soft inductive biases.
New method uses LLMs to generate detailed scientific hypotheses.
problem Generating detailed, actionable scientific hypotheses from coarse initial directions.
method Hierarchical search method that incrementally adds details to hypotheses.
result Hierarchical search method consistently outperforms strong baselines on expert-annotated hypotheses.
We propose to investigate test statistics for testing homogeneity in reproducing kernel Hilbert spaces. Asymptotic null distributions under null hypothesis are derived, and consistency against fixed and local alternatives is assessed. Finally, experimental evidence of the performance of the proposed approach on both ar…
A new method detects changes in machine learning models over time.
problem Automatic monitoring of machine learning models trained on evolving data.
method Score-based statistical hypothesis test for change detection.
result The method can detect changes in any number of model components.
Study improves H-consistency bounds for regression analysis.
problem Improving H-consistency bounds for regression analysis. method Generalized theorems and novel H-consistency bounds for various surrogate loss functions. result Derives principled surrogate losses for adversarial regression.
We present a general framework for hypothesis testing on distributions of sets of individual examples. Sets may represent many common data sources such as groups of observations in time series, collections of words in text or a batch of images of a given phenomenon. This observation pattern, however, differs from the c…
Robust tests control type I error under data corruption.
problem Effective hypothesis testing under data corruption.
method General permutation tests using kernel MMD and HSIC metrics.
result Robust tests are minimax optimal and outperform private tests.
This paper introduces new loss functions for balanced multi-class classification.
problem Balancing class imbalance in multi-class classification.
method Introduces two new surrogate loss families: GLA and GCA.
result GCA losses offer stronger theoretical guarantees in imbalanced settings.
Estimates curvature of network manifolds to understand community structure.
problem Understanding the geometry of network models to infer community structure.
method Develops hypothesis tests to determine manifold type, dimension, and curvature from noisy distance matrices.
result Consistently estimates manifold type, dimension, and curvature from Riemannian manifolds of constant curvature.
Sparse Transformers degrade semantic information first, with early layers encoding more.
problem Understanding how sparse Transformers affect learned representations and semantic information.
method Probed Transformers with progressively pruned weights to observe changes in semantic information and model behavior.
result Complex semantic information is first to degrade in sparse Transformers, with early layers encoding more.
New method improves consistency in preference learning for neural networks.
problem Inconsistent surrogate losses in preference learning for neural networks.
method Formulated a margin-shifted ranking framework and introduced Structure-Aware H-consistency. result Proved superior consistency guarantees for capacity-bounded models using heavy-tailed surrogates.
Improved financial market calibration reveals large excess volatility.
problem Large excess volatility in financial markets.
method Extended Chiarella model to handle long-term value drifts, calibrated on multiple asset classes.
result Large excess volatility (factor ≈ 4 for stock indices) and bimodal mispricing distribution.
Efficiently estimates Cox model coefficients without sharing data.
problem Privacy and ownership concerns in multi-center biomedical studies.
method Communication-efficient iterative distributed algorithms for estimation and inference.
result Achieves convergence rate of full-sample estimator with minimal iterations.
Deep neural networks have achieved impressive performance and become the de-facto standard in many tasks. However, troubling phenomena such as adversarial and fooling examples suggest that the generalization they make is flawed. I argue that among the roots of the phenomena are two geometric properties of common deep l…
Study on continuous sequence classification with distribution uncertainty.
problem Classifying continuous sequences with varying distribution uncertainty.
method Proposes distribution-free tests for three test designs: fixed-length, sequential, and two-phase tests.
result Error probabilities decay exponentially fast for all test designs.
Bitcoin reacts negatively to inflation surprises, contrary to belief.
problem Bitcoin's ability to hedge inflation is questioned.
method Examined cryptocurrency responses to macroeconomic news announcements.
result Bitcoin's price decreases by 24 bps in response to inflationary surprises.
We consider a two-sample hypothesis testing problem, where the distributions are defined on the space of undirected graphs, and one has access to only one observation from each model. A motivating example for this problem is comparing the friendship networks on Facebook and LinkedIn. The practical approach to such prob…
Enhanced H-consistency bounds derived under relaxed conditions.
problem Quantifying the relationship between zero-one estimation error and surrogate loss estimation error.
method Relaxing the condition on the surrogate loss conditional regret and presenting a general framework for establishing enhanced H-consistency bounds. result Derivation of more favorable H-consistency bounds in various scenarios. We calculate the realized volatility in the spin model of financial markets and examine the returns standardized by the realized volatility. We find that moments of the standardized returns agree with the theoretical values of standard normal variables. This is the first evidence that the return dynamics of the spin fi…
New method improves feature importance assessment in random forests.
problem Improving feature importance measures for random forests.
method Hypothesis testing via self-normalized feature-residual correlation test (FACT).
result The method provides theoretically justified feature importance tests with controlled type I error and appealing power.
The paper extends graph-based semi-supervised learning to infinite-dimensional Wasserstein space.
problem Graph-based semi-supervised learning in high-dimensional data.
method Laplace Learning in the Wasserstein space, proving variational convergence and characterizing the Laplace-Beltrami operator.
result Consistent classification performance in high-dimensional settings.
Develops a forward variable selection method for interpretable random forest models.
problem Interpreting high-dimensional non-parametric models like random forests.
method Forward variable selection using CRPS as loss function, with hypothesis testing at each step.
result Method selects a smaller set of variables that optimizes predictive performance.
The lottery ticket hypothesis proposes that over-parameterization of deep neural networks (DNNs) aids training by increasing the probability of a "lucky" sub-network initialization being present rather than by helping the optimization process (Frankle & Carbin, 2019). Intriguingly, this phenomenon suggests that initial…
Paper characterizes MDM for consumer choice modeling and prediction.
problem Modeling consumer choice behavior with parsimonious models.
method Establishes necessary and sufficient conditions for MDM consistency.
result Characterization leads to exact set of representable choice probabilities.