Deep neural networks with ReLU activation achieve optimal nonparametric regression rates.
problem Nonparametric regression with general composition assumptions.
method Sparsely connected deep neural networks with ReLU activation function.
result Achieve minimax rates of convergence under general composition assumption.
The paper proposes a method to efficiently predict using labeled binary trees and analyzes the number of samples needed.
problem Efficiently predicting using compositional nonparametric models.
method A compositional nonparametric method expressed as a labeled binary tree, with a greedy algorithm for regression validation.
result The sufficient number of samples is O(klog(pq)+log(k!)), and the necessary number of samples is Ω(klog(pq)−log(k!)). New method estimates effects of multiple nutrients on blood glucose.
problem Estimating physiological response to multiple nutrient treatments.
method Convolution-based multi-output Gaussian process model.
result Improved prediction accuracy and better interpretation of individual nutrient effects.
Deep-HGP uses Bayesian nonparametric approach for complex data regression.
problem Complex data regression with compositional structures.
method Deep Gaussian processes with a squared-exponential kernel, data-driven lengthscale parameters.
result Posterior distribution optimally recovers unknown true regression curve in terms of quadratic loss.
KernelBiome tackles microbiome research by improving predictive performance and interpretability.
problem Challenges in analyzing high-throughput sequencing data, especially in microbiome research.
method KernelBiome is a kernel-based nonparametric regression and classification framework for compositional data, incorporating prior knowledge and capturing complex signals.
result Improved predictive performance compared to state-of-the-art machine learning methods, with two novel quantities for interpretability.
PEAK tests means of multiple data streams with sequential betting.
problem Testing means of multiple data streams with nonparametric methods.
method Sequential, nonparametric testing using a betting scheme.
result PEAK provides up to 85% reduction in samples for stopping.
Gaussian Processes (GPs) provide a general and analytically tractable way of modeling complex time-varying, nonparametric functions. The Automatic Bayesian Covariance Discovery (ABCD) system constructs natural-language description of time-series data by treating unknown time-series data nonparametrically using GP with …
New test detects differences in heterogeneous datasets.
problem Detecting differences between two samples with unknown heterogeneity.
method Developed a nonparametric testing procedure that handles latent heterogeneity through a composite null.
result The test accurately detects differences in the presence of unknown heterogeneity.
DGPs collapse to near-deterministic transformations, limiting their compositional structure discovery.
problem Limitations of variational inference in DGPs lead to suboptimal posterior approximations.
method Examine alternative variational inference schemes allowing for dependencies across different layers.
result Alternative variational inference schemes can better capture the compositional structure in DGPs.
New geometric approach for analyzing compositional data like gut microbiomes.
problem Analyzing non-negative compositional data with relative values only.
method Reinterpret compositional data as quotient topology of a sphere, using spherical harmonics and reflection group actions.
result Construction of Reproducing Kernel Hilbert Space (RKHS) for compositional data.
The paper tackles deep learning from dependent data, achieving optimal performance.
problem Deep learning from strongly mixing observations, especially with regularization and optimality.
method Sparse-penalized regularization for deep neural networks, oracle inequality for expected excess risk.
result Deep neural network estimator achieves minimax optimal rate for nonparametric autoregression.
Despite its importance, choosing the structural form of the kernel in nonparametric regression remains a black art. We define a space of kernel structures which are built compositionally by adding and multiplying a small number of base kernels. We present a method for searching over this space of structures which mirro…
Develops methods for inference after detecting a change in sequential data.
problem Inference after a detected change in sequential data.
method General framework for constructing confidence sets using only data up to a stopping time.
result First general method for sequential changepoint localization with theoretical guarantees.
This paper presents the beginnings of an automatic statistician, focusing on regression problems. Our system explores an open-ended space of statistical models to discover a good explanation of a data set, and then produces a detailed report with figures and natural-language text. Our approach treats unknown regression…
Paper proposes deep neural networks for nonparametric regression from dependent data.
problem Nonparametric regression from strongly mixing observations.
method Minimum error entropy principle applied to deep neural networks.
result Deep neural networks achieve minimax optimal convergence rates for Gaussian errors.
StackedGP framework integrates and enhances predictions of environmental variables.
problem Improving predictions of environmental variables with quantified uncertainties.
method A network of independently trained Gaussian processes (StackedGP) that integrates different datasets, enhances predictions through intermediate predictions, and propagates uncertainties.
result Validation and application in real data show the effectiveness of the StackedGP model in model composition and cascading predictions.
New model predicts time series quantiles for nonstationary data.
problem Nonparametric probabilistic forecasting of nonstationary univariate time series.
method Composite Quantile Fourier Neural Network (QFNN) for extrapolation-based nonlinear quantile regression.
result Effective in providing high quality and accurate probabilistic predictions.
Develops a nonparametric framework for detecting changes in sequential data.
problem Detecting changes in nonparametrically specified distributions.
method Introduces e-detectors based on e-processes for nonnegative supermartingales.
result Provides bounds on average run length and detection delay.
New method improves MMD estimation without convexity assumptions.
problem Lack of theoretical guarantees for MMD estimation algorithms.
method Preconditioned gradient descent (PGD) scheme for MMD optimization.
result PGD scheme converges globally under specific conditions.
KAPLAN-HR models survival data without manual interactions, outperforming existing methods.
problem Survival analysis challenges with complex covariates and time-varying effects.
method Kolmogorov-Arnold Networks (KAN) for nonparametric hazard estimation.
result KAPLAN-HR matches or exceeds existing methods in clinical survival data.
Improves Gaussian process models for large datasets.
problem Selecting proper covariance functions in Gaussian processes.
method Nonparametric process convolutions and deep GP models.
result Improves performance on benchmarks for GPs, especially for larger datasets.
New adaptive test for NPIV models controls size and has superior power.
problem Testing inequality and equality restrictions in nonparametric IV models.
method Adaptive hypothesis test based on modified leave-one-out sample quadratic distance.
result Adaptive test attains the adaptive minimax rate of testing in L2. A new GP model for non-Gaussian data with explicit inverse warping.
problem Limited expressiveness and computational complexity of Gaussian processes for non-Gaussian data.
method Compositionally-warped Gaussian processes (CWGP) with explicit inverse warping.
result CWGP provides more accurate predictions and shorter computation times than traditional warped GPs.
The study identifies latent concepts from diverse observations without assuming specific models.
problem Lack of general theoretical support for concept learning.
method Develops a nonparametric framework for identifying latent concepts from multiple classes of observations.
result Correctness guarantees for concept identification without parametric assumptions.
The paper develops a neural network method for estimating drift functions of diffusion processes from discrete observations.
problem Nonparametric estimation of drift function for diffusion processes from high-frequency discrete observations.
method Neural network-based estimator for drift function estimation.
result Derives a non-asymptotic convergence rate for the neural network estimator.
Develops a new method for efficient probabilistic inference.
problem Efficient inference for models with dynamic computation graphs.
method Introduces combinator library for Probabilistic Torch framework.
result Models can be trained using stochastic methods that optimize variational or wake-sleep objectives.
Develops a deep learning framework for various data types.
problem Handling nonparametric regression and classification across different data types.
method Introduces a general framework with two estimators: NPDNN and SPDNN, based on data satisfying generalized Bernstein-type inequalities.
result Both NPDNN and SPDNN estimators are minimax optimal in many classical settings.
Controlled interventions provide the most direct source of information for learning causal effects. In particular, a dose-response curve can be learned by varying the treatment level and observing the corresponding outcomes. However, interventions can be expensive and time-consuming. Observational data, where the treat…
Generative Neuro-Symbolic model learns from raw data with rich conceptual representations.
problem Learning rich, general-purpose conceptual representations from raw perceptual inputs.
method Generative Neuro-Symbolic (GNS) model combining symbolic and neural network approaches.
result Model learns from raw data and generalizes to 4 unique tasks.
Researchers develop a new SMC sampler for Wishart processes to improve dynamic covariance inference.
problem Challenging inference of dynamic covariance in various scientific fields.
method Introduce Sequential Monte Carlo (SMC) sampler for the Wishart process.
result SMC sampling provides more robust estimates and out-of-sample predictions of dynamic covariance.
BKP R package models spatially varying binomial probabilities efficiently.
problem Modeling spatially varying binomial probabilities efficiently.
method Beta Kernel Process (BKP) combining localized kernel-weighted likelihoods with conjugate beta priors.
result Closed-form posterior inference without requiring latent variables or intensive MCMC sampling.
We propose a novel class of kernels to alleviate the high computational cost of large-scale nonparametric learning with kernel methods. The proposed kernel is defined based on a hierarchical partitioning of the underlying data domain, where the Nyström method (a globally low-rank approximation) is married with a locall…
Optimizes treatment duration to maximize quality-adjusted lifetime.
problem Balancing risks and benefits in clinical decision making.
method Proposes a weighted estimating equation to adjust for confounding and informative censoring, and a nonparametric estimator for mean counterfactual quality-adjusted lifetime.
result Shows the optimal time for percutaneous endoscopic gastrostomy insertion in ALS patients.
This paper examines how noise affects deep neural networks and improves their performance.
problem The impact of noise on the stability of deep ReLU neural networks for nonparametric regression.
method Investigates the optimal rate of convergence for deep ReLU neural networks under Huber loss, considering the p-th moment of noise and the smoothness of the function.
result The optimal rate of convergence cannot be achieved by ordinary least squares but can be by Huber loss with a properly chosen parameter.
Study analyzes Airbnb lead-time distributions for Nights Booked and Gross Booking Value, finding divergent shapes and tail behavior.
problem Analyzing lead-time distributions for Airbnb demand metrics.
method Compositional analysis of daily lead-time vectors, fitting Gamma, Weibull, and Lognormal distributions, using generalized Pareto for tail inference.
result Lead-time distributions for Nights Booked and Gross Booking Value diverge, with GBV concentrating more in mid-range horizons.
Flexible DNN for survival data, avoiding proportional hazards assumption.
problem Survival analysis with complex interactions and non-proportional hazards.
method Partially linear DNN model with a flexible nonparametric component.
result FLEXI-Haz achieves optimal convergence rates and asymptotic efficiency.
New method uses HDP-HMM and multitaper spectral estimation for automated sleep state classification.
problem Manual sleep scoring is subjective, time-consuming, and doesn't capture neural dynamics.
method Hierarchical Dirichlet Process Hidden Markov Model (HDP-HMM) with multitaper spectral estimation.
result Automated algorithm recovers sleep dynamics and identifies subject-specific microstates.
New method trains DGP models with random feature expansions for scalable inference.
problem Scalability and inference complexity in Deep Gaussian Processes.
method Random feature expansions combined with stochastic variational inference.
result Significantly advanced inference for Deep Gaussian Processes, scalable to large datasets.
Smooth DNNs mitigate the curse of dimensionality in uniform convergence for various regression tasks.
problem The curse of dimensionality in uniform convergence of ReLU networks.
method Analysis of smoothly activated deep neural networks (smooth DNNs), establishing pseudo-dimension bounds and non-asymptotic approximation guarantees.
result Smooth DNNs achieve non-asymptotic uniform convergence rates across multiple statistical contexts, mitigating the curse of dimensionality.
This work is an analytical and numerical study of the composition of several fractals into one and of the relation between the composite dimension and the dimensions of the component fractals. In the case of composition of standard IFS with segments of equal size, the composite dimension can be expressed as a function …
Study on deep neural networks using branching processes and Mehler's formula.
problem Understanding the mathematical role of activation functions in compositional neural networks.
method Connection between compositional kernels and branching processes via Mehler's formula; new random features algorithm.
result Explicit formulas for eigenvalues of compositional kernels quantify complexity.
Uses art composition attributes to guide CycleGAN image translation.
problem Improving image-to-image translation quality.
method Trained ACAN on art composition attributes to influence CycleGAN.
result CycleGAN translations improved with ACAN constraints.
We establish conditions for compositional generalization in machine learning.
problem Achieving compositional generalization in machine learning models.
method We reformulate compositionality as a property of the data-generating process and derive mild conditions on the training distribution and model architecture.
result Our theoretical framework enables compositional generalization under mild conditions.
This paper proves hyperbolicity of virtual knot compositions.
problem Proving hyperbolicity of virtual knot compositions.
method Exploring the composition of hyperbolic virtual knots.
result Strong lower bounds on the volume of compositions.
Develops methods for causal inference in compositional data using instrumental variables.
problem Interpreting summary statistics like diversity indices as causal effects in compositional data.
method Statistical data transformations and regression techniques tailored for compositional data.
result Advantages and limitations of the proposed methods demonstrated on synthetic and real microbiome data.
We study the problem of estimating multiple predictive functions from a dictionary of basis functions in the nonparametric regression setting. Our estimation scheme assumes that each predictive function can be estimated in the form of a linear combination of the basis functions. By assuming that the coefficient matrix …
In classical field theory, the composite fibred manifolds Y -> Z -> X provides the adequate mathematical formulation of gauge models with broken symmetries, e.g., the gauge gravitation theory. This work is devoted to connections on composite fibred manifolds. In particular, we get the horizontal splitting of the vertic…
New algorithm learns optimal actions in complex decision problems.
problem Optimal policy learning in continuous Markov decision problems.
method Nonparametric stochastic compositional gradient descent in RKHS.
result Algorithm converges to optimal policies with low Bellman error.