Paper studies PSGD for constrained optimization problems and its statistical properties.
problem Online inference for constrained optimization problems.
method Stochastic gradient descent with projection (PSGD) for constrained optimization.
result Limiting distribution of PSGD-based estimates under linear-equality constraints.
Paper studies distributed learning with limited communication bits, achieving optimal error exponents.
problem Distributed hypothesis testing with constant communication bits.
method Geometric approach in distribution spaces, encoding empirical distributions to transmission bits.
result Optimal achievable error exponents and coding schemes for various communication constraints.
Optimal score function estimation via empirical risk minimization
problem Estimating the score function of a probability measure on the flat torus from a sample
method Constraining the hypothesis space to a Sobolev ball
result Minimax estimation rates are achieved
Collider regression improves predictive performance in regression tasks.
problem Discarding prior causal knowledge in regression tasks.
method Collider regression framework incorporating probabilistic causal knowledge from collider structures.
result Proves positive generalization benefit and provides closed-form estimators.
Paper resolves open problems on sample complexity in binary hypothesis testing.
problem Open problems in distributed simple binary hypothesis testing under information constraints.
method One-shot lower bound on Bayes error, streamlined sample complexity formula, reverse data-processing inequality.
result Optimally tight sample complexity bounds for communication-constrained simple binary hypothesis testing.
Novel upper bound for unsupervised domain adaptation considers joint error.
problem Addressing the issue of mixing samples from different classes when matching marginal distributions.
method Proposes a general upper bound that penalizes undesirable joint error, uses constrained hypothesis space, and introduces cross margin discrepancy.
result Our proposal outperforms related approaches in image classification error rates on domain adaptation benchmarks.
Study hypothesis testing under quantized samples with communication constraints, achieving near-optimal sample complexity.
problem Optimizing hypothesis testing with quantized samples and communication constraints.
method Developed a polynomial-time algorithm achieving near-optimal sample complexity under communication constraints.
result Achieved near-optimal sample complexity under communication constraints, with a logarithmic factor increase over unconstrained setting.
We study online active learning for classifying streaming instances within the framework of statistical learning theory. At each time, the learner either queries the label of the current instance or predicts the label based on past seen examples. The objective is to minimize the number of queries while constraining the…
A new method constrains deep networks during fine-tuning to improve generalization.
problem Improving generalization of fine-tuned deep networks.
method A neural network generalisation bound based on distance from initial weights constrains the hypothesis class to a small sphere.
result Empirical evaluation shows superior generalization performance compared to existing methods.
Deep learning models have shown state-of-the-art performance in many inverse reconstruction problems. However, it is not well understood what properties of the latent representation may improve the generalization ability of the network. Furthermore, limited models have been presented for inverse reconstructions over ti…
Formula derived for sample complexity in binary hypothesis testing.
problem Determine the minimum number of samples to distinguish between two distributions.
method Developed a formula for sample complexity in both prior-free and Bayesian settings, using Jensen-Shannon and Hellinger divergences.
result Formula characterizes sample complexity for a wide range of error parameters, up to multiplicative constants.
Paper introduces a new learning framework with U-curve properties for model selection.
problem Learning problems modeled by Hypothesis Space and Learning Space.
method Data-driven general learning algorithm for Model Selection on Learning Space.
result Conditions on Learning Space and loss function lead to U-curve estimated out-of-sample error surfaces.
Study on top-k classification with new loss functions and algorithms.
problem Improving multi-class classification accuracy and cardinality trade-off.
method Introducing cardinality-aware loss functions and deriving their consistency bounds.
result New cardinality-aware algorithms for top-k classification. Robust hypothesis testing designs a test for worst-case distributions using kernel methods.
problem Design a robust test for hypothesis testing under uncertainty sets.
method Data-driven uncertainty sets constructed using kernel mean embeddings and maximum mean discrepancy (MMD). Bayesian and Neyman-Pearson settings investigated.
result Proposed robust kernel tests are exponentially consistent and asymptotically optimal.
Selecting appropriate regularization coefficients is critical to performance with respect to regularized empirical risk minimization problems. Existing theoretical approaches attempt to determine the coefficients in order for regularized empirical objectives to be upper-bounds of true objectives, uniformly over a hypot…
Unified framework for constrained online decision-making.
problem Sequential decisions under stage-wise feasibility constraints.
method Upper counterfactual confidence bounds and generalized eluder dimension.
result Principled foundation for constrained sequential decision-making.
We consider closed immersed hypersurfaces in R3 and R4 evolving by a special class of constrained surface diffusion flows. This class of constrained flows includes the classical surface diffusion flow. In this paper we present a Lifespan Theorem for these flows, which gives a positive lower bound on the time fo…
A universal method for hypothesis tests and confidence sets without regularity conditions.
problem Difficult inference in irregular statistical models.
method Modified likelihood ratio statistic (split LRT).
result Works for any parametric and some nonparametric models.
In this paper, we consider a generic probabilistic discriminative learner from the functional viewpoint and argue that, to make it learn well, it is necessary to constrain its hypothesis space to a set of non-trivial piecewise constant functions. To achieve this goal, we present a scalable unsupervised regularization f…
Study risk bounds for distributed ERM with general loss functions and hypothesis spaces.
problem Limited theoretical analysis for distributed ERM with general loss functions and hypothesis spaces.
method Derive tight risk bounds under assumptions on hypothesis space and loss function.
result Developed more general risk bound for distributed ERM without strong convexity restriction.
This paper develops a new theory for ensemble learning beyond variance reduction.
problem Ensemble learning's effectiveness for stable estimators is not fully explained by variance reduction.
method Develops a general weighting theory for ensemble learning, formalizing ensembles as linear operators and introducing geometric and spectral constraints.
result Structured weights can outperform uniform averaging by reshaping approximation geometry and redistributing spectral complexity.
Finite resources limit false discovery rate control in structured hypothesis spaces.
problem Controlling false discovery rate in hypothesis testing with finite data and structured hypothesis spaces.
method Framework for exact FDR control and adaptive power maximization.
result Exact FDR control and adaptive power maximization.
Paper addresses hypothesis space misspecification in learning from human demonstrations and corrections.
problem Hypothesis space misspecification in learning from human demonstrations and corrections.
method Reason explicitly about how well the robot can explain human inputs given its hypothesis space.
result Demonstrates method on a 7 DOF robot manipulator.
M5-branes' flux quantization linked to non-abelian cohomology.
problem Flux quantization on M5-branes and its implications.
method Analogous to Dirac's charge/flux quantization, constraining M5's flux-quantization law to non-abelian cohomology theory.
result Skyrmion-like and anyonic solitons on M5-branes and open M5-branes.
Time-aware fact-checking improves veracity predictions for time-sensitive claims.
problem Fact-checking decisions should consider temporal information of claims and evidence.
method Investigated four temporal ranking methods to optimize evidence ranking for fact-checking models.
result Time-aware evidence ranking surpasses relevance assumptions and improves veracity predictions for time-sensitive claims.
LLM agents discover cryptocurrency factors under reproducible constraints.
problem Flexibility of LLM agents in empirical discovery leads to uncontrolled search.
method Sequential hypothesis search with fixed data splits and portfolio tests.
result Ridge-combined portfolio achieves 44.55% annualized return in out-of-sample period.
Paper proves unique energy-minimizing curves in constrained spaces.
problem Uniqueness of energy-minimizing curves in constrained spaces.
method Investigated energy-minimizing curves with fixed endpoints in a constrained space.
result Proved that the set of points for which the energy-minimizing curve is not unique has no interior points.
Survey of integrating domain knowledge into DL models.
problem Improving DL model performance with limited data or complex functions.
method Five categories of approaches to inject domain knowledge into DL models.
result Survey identifies five main categories of approaches.
The paper classifies a special family of knots in lens spaces using knot Floer homology.
problem Classifying constrained knots in lens spaces.
method Parameterization by five integers, characterization via spinc structures, and knot Floer homology calculations. result Complete classification of constrained knots based on knot Floer homology.
Energy statistics was proposed by Sz\' ekely in the 80's inspired by Newton's gravitational potential in classical mechanics and it provides a model-free hypothesis test for equality of distributions. In its original form, energy statistics was formulated in Euclidean spaces. More recently, it was generalized to metric…
The paper defines a hypothesis space for deep learning using DNNs.
problem Developing a mathematical framework for deep learning.
method Introducing a Banach space of functions of input variables based on DNNs, proving it's a RKBS, and establishing representer theorems for learning models.
result Solutions to learning problems can be expressed as finite sums of kernel expansions based on training data.
Study risk-constrained Kelly optimization for mutually exclusive outcomes, proving support invariance and developing a structured algorithm.
problem Risk-constrained Kelly optimization for mutually exclusive outcomes with explicit state prices.
method Analyzes the finite mutually exclusive outcome version of risk-constrained Kelly optimization with explicit state prices, proving support invariance and developing a structured algorithm.
result Support is invariant across CRRA parameter and drawdown-surrogate parameter in the overround regime.
In this paper we address the problem of pool based active learning, and provide an algorithm, called UPAL, that works by minimizing the unbiased estimator of the risk of a hypothesis in a given hypothesis space. For the space of linear classifiers and the squared loss we show that UPAL is equivalent to an exponentially…
This paper extends the standard chaining technique to prove excess risk upper bounds for empirical risk minimization with random design settings even if the magnitude of the noise and the estimates is unbounded. The bound applies to many loss functions besides the squared loss, and scales only with the sub-Gaussian or …
Improved financial market calibration reveals large excess volatility.
problem Large excess volatility in financial markets.
method Extended Chiarella model to handle long-term value drifts, calibrated on multiple asset classes.
result Large excess volatility (factor ≈ 4 for stock indices) and bimodal mispricing distribution.
This paper introduces localized discrepancy theories for unsupervised domain adaptation.
problem Improving generalization bounds for unsupervised domain adaptation.
method Localized discrepancies defined on the hypothesis space after localization, leading to smaller and asymmetric values.
result Improved generalization bounds and sample complexity reduction.
We show that the number of unique function mappings in a neural network hypothesis space is inversely proportional to ∏lUl!, where Ul is the number of neurons in the hidden layer l.
The paper refutes the manifold hypothesis for image data and proposes the union of manifolds hypothesis.
problem The manifold hypothesis fails to capture the structure of image data.
method Empirical verification of the union of manifolds hypothesis on image datasets.
result Image data lies on a disconnected set with varying intrinsic dimensions.
Study finds financial constraints explain zero-leverage firms.
problem Why some firms have zero leverage despite various explanations.
method Examined three measures of financial constraints; analyzed firms' behavior before and after levering.
result Firms are financially constrained, not due to managerial entrenchment or market valuation.
Causal modeling has long been an attractive topic for many researchers and in recent decades there has seen a surge in theoretical development and discovery algorithms. Generally discovery algorithms can be divided into two approaches: constraint-based and score-based. The constraint-based approach is able to detect co…
We show that the disagreement coefficient of certain smooth hypothesis classes is O(m), where m is the dimension of the hypothesis space, thereby answering a question posed in \cite{friedman09}.
We show that the homogeneous and the 2-lobe Delaunay tori in the 3-sphere provide the only isothermic constrained Willmore tori in 3-space with Willmore energy below 8π. In particular, every constrained Willmore torus with Willmore energy below 8π and non-rectangular conformal class is non-degenerated.
Improves adversarial robustness by constraining logits with a bounded function.
problem Improving adversarial robustness in deep learning models.
method Addition of a bounded function before softmax to constrain logits.
result Our method improves adversarial robustness without requiring adversarial training.
Develops G-MLKM for better data-target association in constrained spaces.
problem Data-target association problem in constrained spaces with limited sensor information.
method Graph-based multi-layer k-means++ (G-MLKM) method, including MLKM for local space and G-MLKM for general constrained space.
result Improves data-target association accuracy through error correction mechanisms.
Study geodesics in constrained curve spaces, including elastic curves and concentric circles.
problem Geodesics in constrained curve spaces with Sobolev metrics.
method Intrinsic and constructive approaches.
result Construct geodesics in elastic curve and concentric circle spaces.
New algorithms for multitask learning with long-term memory.
problem Learning from tasks partitioned into unknown segments with associated hypotheses.
method Online multitask learning algorithms exploiting segmentation and hypothesis association.
result Regret bounds and efficient algorithms for various hypothesis classes.
Large bundles of myelinated axons, called white matter, anatomically connect disparate brain regions together and compose the structural core of the human connectome. We recently proposed a method of measuring the local integrity along the length of each white matter fascicle, termed the local connectome. If communicat…
Scientific discovery is limited by hypothesis redundancy, and hybrid methods can exploit non-local exploration.
problem Limitation of scientific discovery due to hypothesis redundancy.
method Hybrid discovery systems combining structured local search with LLM-generated non-local proposals.
result Hybrid methods can exploit non-local exploration when three geometric conditions co-occur.