Proposes a method to quantify uncertainty in PFNs.
problem Lack of uncertainty quantification in PFNs.
method Martingale posteriors for efficient, tuning-free sampling.
result Proves convergence of proposed sampling procedure.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “Prior-data fitted networks”
PFNs pre-train models on simulated data to predict class probabilities.
problem Training machine learning models on large datasets.
method Pre-train a fixed model on small simulated datasets and use it to infer class probabilities in-context.
result PFNs achieve state-of-the-art performance and improve with larger inference data.
PFN-TS uses Thompson sampling with PFNs to improve contextual bandit performance.
problem Improving contextual bandit performance using Thompson sampling with prior-data fitted networks.
method PFN-TS converts PFN posterior predictives into mean-reward samples using a subsampled predictive central limit theorem.
result PFN-TS achieves the best average rank across nonlinear synthetic and OpenML classification-to-bandit benchmarks.
New method speeds up causal sensitivity analysis.
problem Bounding causal effects in unobserved confounding.
method Amortized approach using prior-data fitted networks.
result Orders of magnitude faster computation.
PFNs4BO uses neural processes for flexible Bayesian Optimization.
problem Efficient surrogate modeling for Bayesian Optimization.
method In-context learning of PFNs to approximate posterior predictive distribution.
result PFNs outperform traditional GP, BNN in BO tasks.
Temporal Causal Prior-Data Fitted Networks (TCPFN) for industrial time series causal discovery
problem Estimating causal effects in industrial time series
method Temporal Causal Prior-Data Fitted Networks
result Zero-shot causal discovery with explicit reliability signals
LaT-PFN model predicts time series with zero-shot capability.
problem Zero-shot time series forecasting.
method In-context latent space learning with JEPA and PFN integration.
result Superior zero-shot predictions compared to baselines.
LC-PFN predicts learning curve performance more accurately and faster than MCMC.
problem Bayesian extrapolation of learning curves is computationally expensive and overly restrictive.
method Prior-Data Fitted Neural Networks (PFNs) for approximate Bayesian inference.
result LC-PFN outperforms MCMC in accuracy and is significantly faster.
TabPFN learns to approximate functions on tabular data.
problem TabPFN tackles function approximation on tabular data.
method Treated as a black-box function approximator generator, observed behavior on varied datasets.
result Observed behavior that is both brilliant and baffling.
New approach uses unlabeled prior data to accelerate exploration in sparse reward tasks.
problem Sparse reward tasks in reinforcement learning.
method Learn reward model from online experience, label prior data, and use concurrently.
result Rapid exploration in challenging sparse-reward domains.
SUPE combines unlabeled data with RL to efficiently explore tasks.
problem Efficient exploration in reinforcement learning with sparse rewards.
method Extract low-level skills using VAE, pseudo-label unlabeled data, and use as off-policy data for online RL.
result SUPE outperforms prior methods across 42 long-horizon tasks.
Transformers can approximate posterior predictive distributions through in-context learning.
problem Bayesian prediction tasks, especially beyond point predictions.
method Gradient descent algorithm targeting posterior predictive mean and variance, followed by nonlinear mappings.
result Transformers can implement algorithms to approximate posterior predictive distributions.
SurvivalPFN simplifies survival analysis through amortized Bayesian inference.
problem Selecting appropriate survival analysis methods requires expertise and can be time-consuming.
method SurvivalPFN uses a prior-data fitted network for in-context Bayesian inference.
result SurvivalPFN achieves strong predictive performance across diverse datasets.
Tabular FMs struggle with reliable uncertainty quantification.
problem Uncertainty quantification in tabular foundation models.
method Compared TabPFN and Gaussian processes (GPs) across various regression tasks.
result GP outperforms TabPFN in data-scarce settings and when kernels are good priors.
AWAC combines offline and online data to accelerate RL learning.
problem Challenges in applying RL to real-world robotic control due to exploration and sample complexity.
method Combines sample-efficient dynamic programming with maximum likelihood policy updates.
result AWAC enables rapid learning of robotic skills with prior data and online experience.
TabPFN models achieve state-of-the-art performance on tabular data tasks.
problem Lack of interpretability in TabPFN models.
method Adaptations of interpretability methods specifically designed for TabPFN, leveraging in-context learning and LOCO.
result Improved interpretability of TabPFN models through efficient computations and scalable data valuation methods.
Transformers can approximate Bayesian inference efficiently.
problem Bayesian methods struggle with deep learning due to prior knowledge and uncertainty capture.
method Prior-Data Fitted Networks (PFNs) approximate posteriors using in-context learning.
result PFNs achieve near-perfect mimicry of Gaussian processes and significant speedups.
TabPFN quickly classifies small tabular data without tuning.
problem Efficiently classifying small tabular datasets.
method Transformer-based model that learns in-context.
result Competitive performance with state-of-the-art methods.
Atomistic structure learningcond-mat.mtrl-sci
One endeavour of modern physical chemistry is to use bottom-up approaches to design materials and drugs with desired properties. Here we introduce an atomistic structure learning algorithm (ASLA) that utilizes a convolutional neural network to build 2D compounds and layered structures atom by atom. The algorithm takes …
Neural networks predictions are unreliable when the input sample is out of the training distribution or corrupted by noise. Being able to detect such failures automatically is fundamental to integrate deep learning algorithms into robotics. Current approaches for uncertainty estimation of neural networks require change…
Drift-Resilient TabPFN learns to adapt to changing data distributions.
problem Real-world data often shifts over time, degrading model performance.
method In-Context Learning with a Prior-Data Fitted Network, using structural causal models.
result Significant performance improvements across various datasets.
TabPFN model shows strong robustness to noisy data.
problem TabPFN tackles robustness to noisy and imperfect tabular data.
method Empirical robustness analysis of TabPFN's attention mechanisms under various perturbations.
result TabPFN maintains high predictive performance and coherent internal behavior under noisy and imperfect data.
Solving power flow (PF) equations is the basis of power flow analysis, which is important in determining the best operation of existing systems, performing security analysis, etc. However, PF equations can be out-of-date or even unavailable due to system dynamics and uncertainties, making traditional numerical approach…
Missing values, irregularly collected samples, and multi-resolution signals commonly occur in multivariate time series data, making predictive tasks difficult. These challenges are especially prevalent in the healthcare domain, where patients' vital signs and electronic records are collected at different frequencies an…
Tabular data analysis shows GBDTs and NNs perform similarly on most datasets.
problem Determining when NNs outperform GBDTs on tabular data.
method Large-scale comparison of 19 algorithms across 176 datasets.
result For many datasets, GBDTs and NNs perform similarly, with hyperparameter tuning often more important than algorithm choice.
Advances few-shot classification by treating it as supervised learning and proposing new training techniques.
problem Formulating the ability of humans to learn from limited data in machine learning.
method Formulated few-shot classification as a supervised learning problem and introduced multi-episode and cross-way training techniques.
result Proposed training strategies accelerate the training process without accuracy loss.
Diverse fault types, fast re-closures, and complicated transient states after a fault event make real-time fault location in power grids challenging. Existing localization techniques in this area rely on simplistic assumptions, such as static loads, or require much higher sampling rates or total measurement availabilit…
New method uses neural networks to estimate parameters without needing detector simulations.
problem Estimating parameters in high-energy physics with detector effects.
method Two-level fitting approach: SRGN (Simulation-level fit based on Reweighting Generator-level events with Neural networks).
result Demonstrated using simulated datasets, SRGN can estimate parameters without detector effects.
Neural networks fit fewer samples than their parameters suggest in practice.
problem Understanding the practical limitations of neural network flexibility.
method Examination of neural network optimization, parameter efficiency, and loss surfaces.
result Neural networks can only fit training sets with significantly fewer samples than their parameters suggest.
New spectral tests assess network model fits efficiently.
problem Determining if network models fit data well and extrapolate.
method Random matrix theory-derived goodness-of-fit tests.
result General approach simplifies parameter selection in network models.
New analysis shows reconstruction attacks are unreliable without prior data knowledge.
problem Privacy and security risks from neural network memorization of training data.
method Complementary analysis of reconstruction methods, proving their unreliability without prior data knowledge.
result Reconstruction attacks are fundamentally unreliable without prior data knowledge, and networks trained more extensively are less susceptible.
Meta-learning for discrete tasks using submodular optimization.
problem Improving performance on new tasks with limited data.
method Discrete submodular optimization for personalized initial solutions.
result Significant reduction in computational cost for new tasks.
New method improves causal structure discovery with Prior-Fitted Networks.
problem Errors in likelihood estimation limit proper causal structure discovery.
method Amortized causal discovery with Prior-Fitted Networks.
result Significant gains in structure recovery compared to baselines.
A new method improves fitting neural data with spiking network models.
problem Fitting spiking network models to neural activity does not produce realistic data.
method Augment log-likelihood with dissimilarity terms measured by summary statistics and optimized via back-propagation.
result The new method generates more realistic neural activity statistics and improves network connectivity inference.
We explore the energy landscape of a simple neural network. In particular, we expand upon previous work demonstrating that the empirical complexity of fitted neural networks is vastly less than a naive parameter count would suggest and that this implicit regularization is actually beneficial for generalization from fit…
A new test assesses how well observed networks fit a specified ERGM model.
problem Testing the goodness of fit for ERGMs with a single network observation.
method Kernel Stein discrepancy combined with a discrete Stein operator for ERGMs, Monte Carlo simulation.
result The test provides theoretical and practical support for assessing ERGM fit.
Exploration of Large Networks with Covariates via Fast and Universal Latent Space Model Fittingstat.ME
Latent space models are effective tools for statistical modeling and exploration of network data. These models can effectively model real world network characteristics such as degree heterogeneity, transitivity, homophily, etc. Due to their close connection to generalized linear models, it is also natural to incorporat…
OccamNet finds interpretable symbolic fits to data efficiently.
problem Complex neural models extrapolate poorly and are hard to interpret.
method Samples functions, biases towards better fits, and uses cross-entropy matching.
result Outperforms state-of-the-art symbolic regression methods on real-world datasets.
neuralGAM package interprets neural networks by fitting them to feature contributions.
problem difficulty understanding neural network decisions
method Generalized Additive Neural Networks (GAM)
result interpretable Deep Learning model with accurate feature contributions
Unified neural network model for astro-particle physics predictions with coverage, systematics, and goodness-of-fit.
problem Lack of statistical uncertainties, coverage, systematic uncertainties, and goodness-of-fit in neural network predictions.
method KL-divergence objective for joint distribution of data and labels, conditional normalizing flows, amortized with neural networks.
result Unified supervised learning and VAEs under stochastic variational inference for event property predictions.
FVI method calculates bicausal OT with neural networks, outperforming other methods.
problem Computing bicausal optimal transport with adapted coupling structures.
method FVI method using multilayer neural networks to approximate value functions.
result FVI method outperforms linear programming and Sinkhorn methods in scalability.
A novel model-selection method for dynamic networks using synthetic data.
problem Classifying and understanding the growth mechanisms of dynamic networks.
method Training a classifier on synthetic network data generated by nine random graph models, using dynamic features that count new links.
result Achieves near-perfect classification of synthetic networks, outperforming state-of-the-art methods.
New law establishes robustness for neural networks with bounded weights.
problem Ensuring robustness of neural networks against adversarial attacks.
method Deriving a lower bound on Lipschitz constant for arbitrary model classes with bounded Rademacher complexity.
result Established a law of robustness for weight-bounded neural networks, requiring log(n) layers for robust fitting.
Neural networks can overfit perfectly to noisy data and then grok near-optimal generalization.
problem Neural networks' ability to overfit perfectly to noisy data and then generalize near-optimally.
method Two-layer ReLU networks trained by gradient descent on XOR cluster data.
result Neural networks can achieve perfect fit to noisy training data and then grok near-optimal generalization.
We propose a novel method that makes use of deep neural networks and gradient decent to perform automated design on complex real world engineering tasks. Our approach works by training a neural network to mimic the fitness function of a design optimization task and then, using the differential nature of the neural netw…
Convolutional Neural Networks (CNNs) have emerged as highly successful tools for image generation, recovery, and restoration. A major contributing factor to this success is that convolutional networks impose strong prior assumptions about natural images. A surprising experiment that highlights this architectural bias t…
New algorithms learn simple staged trees from data, improving model fit.
problem Complex conditional independences in categorical data vectors.
method Structural learning algorithms for simple staged trees, coalescing the underlying tree.
result Data-learned simple staged trees often outperform Bayesian networks in model fit.
New method fits low-rank models for egocentrically sampled networks.
problem Statistical modeling of egocentrically sampled partial networks.
method Graph spectral properties-based approach for low-rank models.
result Consistent recovery of missing subnetworks for sparse networks.