Neural programming involves training neural networks to learn programs, mathematics, or logic from data. Previous works have failed to achieve good generalization performance, especially on problems and programs with high complexity or on large domains. This is because they mostly rely either on black-box function eval…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “neural function evaluations”
Deep neural networks can estimate Q-values efficiently on low-dimensional state-action spaces.
problem Estimating the performance of a reinforcement learning policy using data from a different policy.
method Deep fitted Q-evaluation method leveraging manifold structure and convolutional neural networks.
result Sharp error bound for fitted Q-evaluation depends on intrinsic dimension and function space mismatch.
Unified approach optimizes neural network training for various metrics.
problem Training and evaluation of neural network binary classifiers often use different metrics.
method Combines differentiable approximation and probabilistic soft sets.
result Effective in optimizing for metrics like F1-Score across various domains.
FQE with deep neural networks achieves asymptotic normality and finite-sample bounds.
problem Theoretical understanding of FQE with general differentiable function approximators.
method Z-estimation theory applied to FQE with deep neural networks.
result FQE estimation error is asymptotically normal with explicit variance.
New bounds using samplewise evaluated CMI for deep neural networks.
problem Improving generalization bounds for deep neural networks.
method Introduced a new family of information-theoretic generalization bounds using samplewise evaluated conditional mutual information (CMI).
result The new bounds can be tighter than previous ones for deep neural networks.
Distributionally robust risk evaluation with a causality constraint and structural informationq-fin.MF
This work evaluates risks over time using robust measures and neural networks.
problem Distributionally robust risk evaluation over temporal data.
method Characterizes alternative measures using causal optimal transport, approximates test functions by neural networks, and proves sample complexity.
result Framework outperforms classic counterparts in portfolio selection problems.
The performance of deep neural networks crucially depends on good hyperparameter configurations. Bayesian optimization is a powerful framework for optimizing the hyperparameters of DNNs. These methods need sufficient evaluation data to approximate and minimize the validation error function of hyperparameters. However, …
A method for faster neural architecture search using low-fidelity training.
problem Time-consuming evaluations in neural architecture search.
method Bayesian multi-fidelity method with knowledge distillation.
result Training for a few epochs with knowledge distillation leads to better architecture selection.
We propose an efficient transfer Bayesian optimization method, which finds the maximum of an expensive-to-evaluate black-box function by using data on related optimization tasks. Our method uses auxiliary information that represents the task characteristics to effectively transfer knowledge for estimating a distributio…
Novel method for scalable neural network-based blackbox optimization.
problem Scalability challenges in high-dimensional Bayesian Optimization.
method SNBO: Adds new samples using separate criteria for exploration and exploitation, adaptively controlling the sampling region.
result SNBO achieves better function values with 40-60% fewer function evaluations and reduced runtime.
Introduces epistemic deep learning for better uncertainty estimation in neural networks.
problem Uncertainty quantification in deep neural networks.
method Random-set convolutional neural networks with belief function-based loss functions.
result Epistemic approach produces better performance in uncertainty estimation.
New method certifies neural network function space norms from point evaluations.
problem Certifying neural network function space norms from point evaluations alone.
method Combining interval arithmetic enclosures, adaptive marking/refinement, and quadrature-based aggregation.
result Certified computation of , , and norms.
Proposes a meta-learning method for Bayesian optimization with neural network kernels.
problem Improving BO performance on high-dimensional data with complex structure.
method Meta-learning approach that trains a neural network policy to select the next data point to evaluate, using a reinforcement learning framework.
result The proposed method achieves better BO performance than existing methods on text document datasets.
Ensemble techniques are powerful approaches that combine several weak learners to build a stronger one. As a meta learning framework, ensemble techniques can easily be applied to many machine learning techniques. In this paper we propose a neural network extended with an ensemble loss function for text classification. …
Q-NETs use neural networks to estimate integrals of low-dimensional functions efficiently.
problem Estimating integrals of multidimensional functions with costly evaluations.
method Fixed neural networks (Q-NETs) that operate on proxy function parameters to calculate exact integrals over subsets of dimensions.
result Q-NETs can calculate integrals over any subset of dimensions without resampling or retraining the proxy.
Bayesian optimization is an effective methodology for the global optimization of functions with expensive evaluations. It relies on querying a distribution over functions defined by a relatively cheap surrogate model. An accurate model for this distribution over functions is critical to the effectiveness of the approac…
A new framework evaluates HTE estimators using relative error.
problem Lack of robust evaluation methods for HTE estimators.
method Proposes a relative error-based evaluation framework and neural network architecture to estimate nuisance parameters and robustly compare HTE estimators.
result Demonstrates reliable comparisons and improved HTE estimation through the proposed framework and learning algorithm.
Stochastic Bayesian Neural Network improves scalability and performance.
problem Challenges in calculating posterior distribution in Bayesian Neural Networks.
method Maximizes Evidence Lower Bound using Stochastic Evidence Lower Bound objective function.
result Demonstrates improved performance and scalability over previous algorithms.
Automates design of lightweight neural networks for image classification.
problem Designing efficient neural networks for edge devices with limited computational resources.
method Uses the Mesh Adaptive Direct Search (MADS) algorithm to optimize network architecture.
result Achieves comparable performance to standard methods with fewer design trials.
Deep neural networks can interpolate any dataset in the overparametrized regime.
problem Interpolating any dataset with deep neural networks in the overparametrized regime.
method Proving universal approximations and interpolating any dataset with deep neural networks, considering specific conditions on activation functions.
result Interpolation of any dataset is possible in the overparametrized regime with deep neural networks.
Neural networks can approximate functionals on RKHS with error bounds.
problem Approximating functionals on RKHS using neural networks.
method Interpolating orthogonal projections in RKHS using point evaluations.
result Explicit error bounds for various kernels (inverse multiquadric, Gaussian, Sobolev).
New method uses neural operators for efficient function space optimization.
problem Optimization over function spaces with costly function evaluations.
method Sample-then-optimize approach with neural operator surrogates.
result Better sample efficiency and significant performance gains in experiments.
A Benchmark Suite for Evaluating Neural Mutual Information Estimators on Unstructured Datasetsstat.ML
Study introduces a benchmark suite for evaluating neural MI estimators on real-world unstructured datasets.
problem Lack of comprehensive evaluation methods for neural MI estimators on real-world unstructured datasets.
method Developed a benchmark suite using same-class sampling and a binary symmetric channel trick.
result Showed accurate manipulation of true MI values of real-world datasets.
RieszNet and ForestRiesz: Automatic Debiased Machine Learning with Neural Nets and Random Forestscs.LG
Automatic debiasing for causal and policy effects using Neural Nets and Random Forests.
problem Estimating causal and policy effects from high-dimensional or non-parametric regression functions.
method Automatic learning of Riesz representation using Neural Nets and Random Forests.
result Automatic debiasing method performs well compared to state-of-the-art algorithms.
Temporal-difference learning (TD), coupled with neural networks, is among the most fundamental building blocks of deep reinforcement learning. However, due to the nonlinearity in value function approximation, such a coupling leads to nonconvexity and even divergence in optimization. As a result, the global convergence …
We address the problem of learning vector representations for entities and relations in Knowledge Graphs (KGs) for Knowledge Base Completion (KBC). This problem has received significant attention in the past few years and multiple methods have been proposed. Most of the existing methods in the literature use a predefin…
The notion of expense in Bayesian optimisation generally refers to the uniformly expensive cost of function evaluations over the whole search space. However, in some scenarios, the cost of evaluation for black-box objective functions is non-uniform since different inputs from search space may incur different costs for …
Generative adversarial training can be generally understood as minimizing certain moment matching loss defined by a set of discriminator functions, typically neural networks. The discriminator set should be large enough to be able to uniquely identify the true distribution (discriminative), and also be small enough to …
DPM-Solver speeds up DPM sampling to 10-20 function evaluations.
problem Slow sampling from Diffusion Probabilistic Models (DPMs).
method Exact formulation of diffusion ODE solutions, using change-of-variable and exponentially weighted integral.
result Generates high-quality samples in 10-20 function evaluations.
We leverage neural networks as universal approximators of monotonic functions to build a parameterization of conditional cumulative distribution functions (CDFs). By the application of automatic differentiation with respect to response variables and then to parameters of this CDF representation, we are able to build bl…
Sparse perturbations improve convergence in SZO methods for faster training.
problem Dependency of SZO methods on function dimensionality limits their convergence speed.
method Sparse perturbations reduce the effective dimensionality of the optimization problem.
result Sparse SZO optimization leads to faster convergence in training loss and test accuracy.
Stable Neural Flowscs.LG
Stable neural flows ensure robustness and efficiency in deep learning.
problem Ensuring robustness and stability in deep learning models.
method Introducing a stable variant of neural ODEs with a neural network parametrizing an energy functional, solving as an optimal control problem with adjoint sensitivity analysis.
result The proposed model provides robustness against input perturbations and low computational burden.
This paper improves sample efficiency for off-policy evaluation with preference data.
problem Improving sample efficiency for off-policy evaluation with preference data.
method Using a deep neural network to learn the value function and leveraging manifold structure.
result Established a provably efficient guarantee for off-policy evaluation with RLHF.
Improving Infinitely Deep Bayesian Neural Networks with Nesterov's Accelerated Gradient Methodstat.ML
Improved SDE-BNN model reduces NFEs and accelerates convergence.
problem High computational cost and convergence instability in SDE-BNNs.
method Nesterov's Accelerated Gradient (NAG) method integrated into SDE-BNN framework.
result Significantly reduced number of function evaluations (NFEs) and improved predictive accuracy.
Analytic Continuation by Feature Learningcond-mat.str-el
Improved neural network predicts spectral functions more accurately than traditional methods.
problem Reconstructing real-time spectral functions from imaginary-time Green's functions is ill-posed and challenging.
method Feature Learning Network (FL-net) for enhanced prediction accuracy.
result FL-net achieves at least 20% improvement over traditional methods like MEM.
New method encodes function preferences into neural nets for better generalization.
problem Challenges in encoding explicit function preferences in neural network training.
method Function-space empirical Bayes (FSEB) regularization.
result FSEB leads to near-perfect semantic shift detection and improved generalization.
Two new criteria help understand the advantage of deep neural networks.
problem Understanding the advantage of deepening neural networks.
method Proposed two new criteria to evaluate the expressivity of functions computable by deep neural networks.
result Increasing layers is more effective than increasing units in improving the expressivity of deep neural networks.
Cost-aware BO minimizes function evaluations with varying costs.
problem Optimization with varying evaluation costs in expensive functions.
method Cost Apportioned Bayesian Optimization (CArBO) combining initial and cost-cooled phases.
result CArBO finds better hyperparameter configurations with the same cost budget.
New approach to handle ranking function variation in zero-shot NAS.
problem Variation in ranking function outputs due to randomness.
method Viewing ranking function output as a random variable and constructing a stochastic ordering.
result Stochastic ordering boosts performance in neural architecture search.
A new model uses neural networks to efficiently learn multivariate temporal point processes.
problem Efficiently modeling multivariate temporal point processes with low parameter complexity.
method Modeling the cumulative hazard function with neural networks for each variate.
result The proposed model achieves state-of-the-art performance on data fitting and event prediction tasks.
Comparison of non-linear activation functions for deep neural networks on MNIST classification taskcs.LG
Activation functions play a key role in neural networks so it becomes fundamental to understand their advantages and disadvantages in order to achieve better performances. This paper will first introduce common types of non linear activation functions that are alternative to the well known sigmoid function and then eva…
Tensor networks and RNNs are equivalent, improving wave function encoding.
problem Efficiently encoding quantum states in neural networks.
method Generalized RNN architecture for tensor networks, supporting polynomial time wave function evaluation.
result Tensorial RNNs can encode quantum states with lower bond dimensions and higher accuracy.
New method estimates model risk without knowing function class.
problem Evaluating model risk for complex, opaque models.
method Wild refitting with Bregman losses and randomized symmetrization.
result Valid upper bound on excess risk for opaque models.
LALR adapts learning rate for faster convergence in regression and neural nets.
problem Finding optimal learning rates for faster convergence in regression and neural networks.
method Lipschitz continuity theory applied to Mean Absolute Error and Quantile loss functions.
result Adaptive learning rate policy enables up to 20x faster convergence.
Deep neural networks forecast financial return distributions accurately.
problem Forecasting probability distributions of financial returns.
method Used 1D CNN and LSTM architectures with custom loss functions to optimize distribution parameters.
result LSTM with skewed Student's t distribution outperformed classical models in multiple evaluation metrics.
Wide deep neural networks with Gaussian weights approximate Gaussian processes closely.
problem Understanding the approximation of deep neural networks with Gaussian weights to Gaussian processes.
method Established novel rates for the Gaussian approximation of random deep neural networks with Gaussian parameters and Lipschitz activation functions in the wide limit.
result The distance between the network output and the Gaussian approximation scales inversely with the width of the network.
RI-DeepONet learns neural operators from arbitrary sensor data.
problem Discretization of input functions limits practical applications of DeepONet.
method Introduces RI-DeepONet and two dictionary learning algorithms for INRs.
result RINO handles arbitrary sensor data robustly and applies to various problems.
DeepBlip estimates treatment effects over time using neural networks.
problem Estimating treatment effects over time with interpretable blip effects.
method DeepBlip uses a novel double optimization trick to enable simultaneous learning of blip functions with sequential neural networks.
result DeepBlip achieves state-of-the-art performance across various clinical datasets.