The paper analyzes a neural network two-sample test using kernel analysis.
problem Determining if two datasets come from the same distribution.
method Time-analysis on a neural tangent kernel (NTK) two-sample test, extending to realistic neural network dynamics.
result Training times needed to detect deviations are well-separated in null and alternative hypothesis scenarios.
Estimates neural representation dimensionality from small sample sizes.
problem Estimating neural representation dimensionality from limited data.
method Proposed a bias-corrected estimator for participation ratio of eigenvalues.
result The estimator is more accurate with finite samples and noise.
New study shows neural networks need many samples for training.
problem How much data is needed to train a ReLU feed-forward neural network?
method Theoretical and empirical analysis of ReLU feed-forward neural networks.
result Generalization error scales at 1 / n 1/\sqrt{n} 1/ n in sample size n n n . Neighborhood sampling affects graph neural network training outcomes.
problem Understanding the impact of neighborhood sampling on graph neural network training.
method Theoretical analysis using neural tangent kernels and Gaussian processes.
result Posterior covariance differs for different neighborhood sampling approaches, indicating no dominant approach.
Residual neural networks don't help overcome sampling complexity issues.
problem Learning invertible residual neural networks from samples is hard due to the curse of dimensionality.
method Investigated invertible residual neural networks and their sampling complexity.
result Invertible residual neural networks still suffer from the curse of dimensionality in sampling complexity.
Paper analyzes sample complexity of polynomial neural networks.
problem Understanding the sample complexity of polynomial neural networks.
method Extends previous literature to polynomial neural networks and analyzes sample complexity.
result Obtains novel results on sample complexity of polynomial neural networks.
Neural Thompson Sampling uses deep neural networks for contextual bandit problems.
problem Solving contextual multi-armed bandit problems.
method Adapts deep neural networks for exploration and exploitation in a novel posterior distribution.
result Guaranteed cumulative regret of O ( T 1 / 2 ) \mathcal{O}(T^{1/2}) O ( T 1/2 ) for bounded reward functions. Generalizes leverage score sampling for neural networks, accelerating kernel methods and deep learning.
problem Accelerating kernel methods and deep learning training.
method Generalizes leverage score sampling to neural networks and proves equivalence to neural tangent kernel ridge regression.
result Equivalence between regularized neural network and neural tangent kernel ridge regression under leverage score sampling initialization.
Neural network accuracy improves with denser training samples.
problem Improving neural network accuracy on unseen test samples.
method Bounding empirical training error smoothed across activation regions and using it to discard high-risk test samples.
result Discarding high-risk test samples based on error bounds improves prediction accuracy by up to 20%.
This paper examines the convergence of adaptive sampling methods for Bayesian neural networks.
problem Uncertainty quantification in deep neural networks, especially for medical applications.
method Locally adaptive and scalable diffusion-based sampling methods.
result These methods can have a substantial bias in the distribution they sample, even in the limit of vanishing step sizes.
This paper introduces a neural sampler for scalable sampling from complex distributions.
problem Efficiently sampling from high-dimensional un-normalized distributions.
method Neural implicit sampler trained with KL and Fisher divergence methods.
result The neural sampler generates large batches of samples with low computational costs.
Improved sampling for Bayesian neural networks reduces vanishing acceptance rates and increases predictive accuracy.
problem Sampling inefficiency in Bayesian neural networks, especially with deep architectures and large datasets.
method Approximate blocked Gibbs sampling to partition and sample subgroups of parameters.
result Increased predictive accuracy and quantification of predictive uncertainty in classification tasks.
The paper explores how splitting data samples influences optimal neural network hyperparameters.
problem Understanding the effectiveness of neural networks and their hyperparameters.
method Investigates the role of sample splitting in neural network hyperparameter selection.
result Optimal hyperparameters derived from sample splitting lead to a neural network model that minimizes prediction risk asymptotically.
This work uses sampling theory to analyze smoothness and error bounds of finite neural networks.
problem Analyzing the function space of finite neural networks and providing error bounds.
method Applying sampling theory to finite neural networks with non-expansive activation functions, considering both deterministic and random sampling.
result Novel error bounds for univariate neural networks under band-limited input assumption, highlighting the advantage of deterministic uniform sampling.
New method uses neural operators for efficient function space optimization.
problem Optimization over function spaces with costly function evaluations.
method Sample-then-optimize approach with neural operator surrogates.
result Better sample efficiency and significant performance gains in experiments.
New sample complexity bounds for linear predictors and neural networks, focusing on initialization.
problem Understanding sample complexity for vector-valued linear predictors and neural networks, especially under initialization-dependent conditions.
method Size-independent bounds on Frobenius norm distance from a fixed reference matrix, applying to vector-valued predictors and neural networks.
result Established new sample complexity bounds for feed-forward neural networks, resolving open questions and introducing a new learnable problem.
New method improves graph neural networks by considering different types of relations in sampling.
problem Current graph neural networks ignore relation types in biomedical graphs, leading to suboptimal performance.
method Proposes relation-dependent sampling for multi-relational graphs to balance relation frequency and importance.
result State-of-the-art graph neural networks achieve better accuracy and efficiency with relation-dependent sampling.
Paper presents a method to summarize HMC samples for neural networks, providing meaningful uncertainty estimates.
problem Lack of interpretable summary statistics for HMC samples in neural networks due to permutation symmetry.
method Introducing a transpositions metric to quantify permutations and using rebasin method to summarize HMC samples.
result Compact representation of HMC samples provides meaningful uncertainty estimates for each weight in a neural network.
New method for Bayesian neural networks reduces inference difficulty.
problem Difficulty in sample-based inference for Bayesian neural networks.
method Embracing mode-connectedness to link overparameterization and sampling difficulty.
result Practical guidelines and deep ensemble approach for effective SBI.
The paper proposes a neural network architecture inspired by Langevin Monte Carlo for sampling from target distributions.
problem Sampling from complex target distributions efficiently.
method A neural network architecture inspired by Langevin Monte Carlo is proposed to map samples from a simple reference distribution to samples from the target.
result The proposed neural network architecture achieves approximation rates in the Wasserstein-2 distance for smooth, log-concave target distributions.
PTSD improves neural samplers by combining diffusion models and PT, enhancing efficiency.
problem Efficiency and correlation issues in neural samplers compared to PT.
method Sequential training of diffusion models across temperatures, combining high-temperature models for approximate lower-temperature samples.
result Significantly improved target evaluation efficiency, outperforming diffusion-based samplers.
Improved TD learning with neural nets reduces sample complexity and overparameterization.
problem Temporal difference learning with neural networks in large state spaces.
method Projection-free and max-norm regularized Neural TD learning, with Lyapunov drift analysis.
result Max-norm regularization significantly improves TD learning's sample complexity and overparameterization.
New algorithms optimize neural networks with ReLU activations using sampling.
problem Optimizing trained neural networks with ReLU activations.
method Iterative algorithm and enhanced algorithm using sampling and neighborhood search.
result The methods reduce the initial MIP problem into smaller LP or MIP problems.
Neural networks benefit from intermediate representations, reducing sample complexity.
problem Understanding how neural networks leverage intermediate representations for hierarchical learning.
method Fixed, randomly initialized neural network as a representation function, compared with raw inputs and other trainable networks.
result Neural representations can achieve improved sample complexities compared to raw inputs, especially for low-rank polynomials.
Wide Bayesian neural networks have a simpler weight posterior, leading to faster MCMC sampling.
problem Sampling from the posterior of wide Bayesian neural networks is challenging.
method Introducing repriorisation, a data-dependent reparameterisation that simplifies the posterior distribution.
result The repriorisation map accelerates MCMC sampling, achieving up to 50x higher effective sample size.
Neural network MCMC sampler maximizes proposal entropy for efficient sampling.
problem Inefficient sampling from complex probability distributions.
method Proposes a neural network MCMC sampler that maximizes proposal entropy.
result Significantly higher efficiency in various sampling tasks.
New method accelerates Parallel Tempering using neural samplers.
problem Challenges in sampling from high-dimensional, multimodal distributions.
method Leverages neural samplers to reduce overlap between distributions.
result Improves sample quality and reduces computational cost.
Proposes a neural network method to combine nonprobability and probability survey samples.
problem Combining nonprobability and probability survey samples for accurate population mean estimation.
method Uses a deep neural network to estimate sampling scores from nonprobability samples and combines them with probability sample information.
result Proposed estimators improve robustness to parametric propensity-score misspecification, especially for nonlinear selection mechanisms.
This work improves sample efficiency in neural function approximation for reinforcement learning.
problem Improving sample efficiency in reinforcement learning with neural function approximation.
method Study of function approximation with two-layer neural networks (ReLU and polynomial activations) under generative and realizability models.
result Significant improvement in sample complexity compared to linear methods.
Deep Neural Networks for image classification have been found to be vulnerable to adversarial samples, which consist of sub-perceptual noise added to a benign image that can easily fool trained neural networks, posing a significant risk to their commercial deployment. In this work, we analyze adversarial samples throug…
New algorithm samples Bayesian neural networks for improved calibration.
problem Improving calibration of Bayesian neural networks.
method Symmetric Minibatch Splitting-UBU (SMS-UBU) algorithm.
result SMS-UBU provides better calibration performance than standard methods.
Paper presents a method to accurately quantify neural network uncertainty without sampling.
problem Uncertainty quantification in neural networks for reliability and robustness.
method Sample-free moment propagation technique for mean vectors and covariance matrices.
result Analytic solution for covariance of nonlinear activation functions.
New study shows exponential sample growth for ReQU neural networks.
problem Computing neural network approximations from samples is challenging.
method Information-based complexity tools.
result Functions can be approximated by ReQU neural networks at arbitrary rates but require exponentially growing samples.
The paper analyzes convergence of neural SDEs as sample size increases.
problem Understanding the limiting behavior of neural SDEs as sample size grows.
method Analyzes Hamilton-Jacobi-Bellman equation and uses stochastic maximum principle.
result Convergence of minima and optimal parameters of neural SDEs as sample size increases.
Imbalanced data classification problem has always been a popular topic in the field of machine learning research. In order to balance the samples between majority and minority class. Oversampling algorithm is used to synthesize new minority class samples, but it could bring in noise. Pointing to the noise problems, thi…
Unified framework for sampling from complex densities using PDEs and neural networks.
problem Sampling from complicated probability densities.
method Dynamical measure transport via PDEs and physics-informed neural networks (PINNs).
result Significantly better mode coverage and high accuracy in sampling.
Training deep neural networks is known to require a large number of training samples. However, in many applications only few training samples are available. In this work, we tackle the issue of training neural networks for classification task when few training samples are available. We attempt to solve this issue by pr…
New method improves sample-efficiency in neural posterior estimation using simulator gradients.
problem High-fidelity posterior estimation with complex physical simulations is time-consuming.
method Neural Posterior Estimation (NPE) with differentiable simulators and gradient information.
result Improves sample-efficiency in posterior density estimation.
A method uses neural networks to approximate sampling distributions of test statistics.
problem Accurate modeling of p-value functions or cdfs for correct confidence set coverage.
method Uses neural networks to model the cdf of test statistics, approximating sampling distributions.
result Neural network approximations of sampling distributions are effective and simple.
LFIS uses a time-dependent velocity field to sample from complex distributions.
problem Sampling from unnormalized density functions.
method LFIS learns a time-dependent velocity field to transport samples from a simple initial distribution to a complex target distribution.
result LFIS achieves state-of-the-art performance on various benchmark problems.
Improved sample efficiency with normalized RBF kernels in neural networks.
problem Learning more with less data in deep learning models.
method Two-phase method to train neural networks with normalized RBF kernels as output layer.
result Normalized RBF kernel networks achieve higher sample efficiency, compactness, and separability.
Study uses neural networks to improve option pricing accuracy.
problem Reducing variance in Monte Carlo estimators for option pricing.
method Characterizes neural networks' universal approximation property and applies it to sampling measures.
result Sampling measures generated by neural networks can approximate optimal measures arbitrarily well.
Paper shows pre-training and transfer learning reduce sample complexity for neural networks.
problem Training high-dimensional supervised learning with limited labeled data.
method Study of single-layer neural networks via online stochastic gradient descent, considering concept shift.
result Pre-training and transfer learning reduce sample complexity by polynomial factors under general assumptions.
New algorithms for neural bandits learn from context and arm features.
problem Learning from context and arm features in combinatorial bandit problems.
method Proposed algorithms CN-UCB and CN-TS using deep neural networks.
result Achieved regret guarantees for the first time in combinatorial neural bandits.
New method uses CNNs to estimate graph means.
problem Estimating the mean of graph-valued data.
method Convolutional Neural Networks (CNNs) for graph morphology learning.
result CNNs reliably recover the sample Frechet mean.
Unified framework for ensemble sampling in nonlinear contextual bandits with provable regret bounds.
problem Efficient exploration in nonlinear contextual bandits with unknown feature dimensions.
method Developed GLM-ES and Neural-ES for generalized linear and neural contextual bandits, respectively, using maximum likelihood estimation on randomly perturbed data.
result Unified high-probability frequentist regret bounds for GLM-ES and Neural-ES, matching state-of-the-art results.
The paper proposes a Gibbs sampler for neural network posterior sampling.
problem Sampling from the posterior of neural networks.
method Adding noise to activations and using a Gibbs sampler.
result The Gibbs sampler achieves similar performance to MCMC methods on real and synthetic data.
Enhances physics-informed neural networks with adaptive sampling and weighting.
problem Challenges in training physics-informed neural networks on complex problems.
method Hybrid adaptive sampling and weighting method.
result Consistently improves prediction accuracy and training efficiency.