Forward-prediction models enhance physical reasoning, but only for specific tasks.
problem Improving physical reasoning in complex tasks involving many objects.
method Incorporated forward-prediction models into simple physical-reasoning agents and evaluated their performance on the PHYRE benchmark.
result Forward-prediction models improve physical-reasoning performance, especially on complex tasks, but generalization to new task templates is challenging.
Study on predictable forward processes in trading without frequent evaluations.
problem Trading performance evaluation times not matching trading times.
method Solving a linear functional equation to construct predictable forward processes.
result Predictable forward processes are inherently myopic and optimal strategies do not use future information.
KINet learns object interactions without supervision for robotic pushing.
problem Lack of supervised data for object-centric forward prediction.
method End-to-end unsupervised framework using keypoint representation and contrastive estimation.
result Automatically generalizes to unseen scenarios and accurately predicts future states.
PBI inference may not be calibrated if predictive model is inaccurate.
problem Uncertainty quantification in PBI may be unreliable if the predictive model is not accurate.
method Predictive Bayesian inference with a forward predictive model.
result Posterior concentration depends on the predictive model, leading to potential calibration issues.
New method for interpreting non-linear models using forward marginal effects.
problem Interpreting non-linear models' feature effects is challenging.
method Introducing forward marginal effects and partitioning feature space for better interpretation.
result Improved interpretation of non-linear prediction functions.
Paper improves k-NN predictive performance with efficient variable selection.
problem Improving predictive performance of k-NN models. method Efficient forward selection of predictor variables.
result Novel approach approaches outperformance of stepwise selection models.
Study predicts bond yields using machine learning and ultimate forward rates.
problem Forecasting bond yields using ultimate forward rates.
method Applied de Kort-Vellekooptype methodology for UFR estimation, used linear and nonlinear machine learning techniques.
result Nonlinear machine learning models outperform linear models in bond yield forecasting.
A VAE model predicts material properties and microstructures.
problem Building forward and inverse structure-property linkages in materials science.
method Combines VAE with regression, using a two-level prior and multi-modal Gaussian mixture.
result The model achieves accurate forward and inverse predictions of material properties and microstructures.
Forward regression is a statistical model selection and estimation procedure which inductively selects covariates that add predictive power into a working statistical regression model. Once a model is selected, unknown regression parameters are estimated by least squares. This paper analyzes forward regression in high-…
fmeffects package interprets non-linear models in plain language.
problem Interpreting complex non-linear models.
method Forward marginal effects for model-agnostic explanations.
result First software implementation of forward marginal effects.
We introduce a new class of forward performance processes that are endogenous and predictable with regards to an underlying market information set and, furthermore, are updated at discrete times. We analyze in detail a binomial model whose parameters are random and updated dynamically as the market evolves. We show tha…
Audit financial machine learning workflows to detect spurious predictability.
problem Spurious predictability in financial machine learning models.
method Falsification audit testing predictive workflows against synthetic environments.
result Many apparent financial predictions are artifacts, not genuine.
Proposes a new method to learn representations directly optimized for a task.
problem Learning representations optimized for unrelated tasks.
method Jointly learns representation and prediction function for a specific task.
result Learned representations outperform pre-trained ones and are more sample-efficient.
Single-pass method estimates neural network uncertainty.
problem Uncertainty estimation in deep learning requires multiple passes.
method Probabilistic reasoning over neural network depths.
result Single forward pass for uncertainty estimation.
Machine learning speeds up GPR simulations.
problem Computational demands of simulating practical GPR problems.
method Automatic ML-based forward solver framework using gprMax.
result Near-real-time GPR simulations achieved.
Improves predictions by integrating forward-looking views into dynamic factor models.
problem Poor forecasts from historical data when dynamics change.
method Combines historical data with forward-looking views using a dynamic factor model.
result Derives optimal portfolio strategies influenced by both myopic and intertemporal factors.
We propose a novel method to forecast the future from the present using time-reversed data.
problem Forecasting the future from past data, exploiting temporal asymmetry.
method Retrodictive forecasting via inverse MAP optimization over a Conditional Variational Autoencoder (CVAE).
result The method successfully predicts future events in time-reversible and irreversible processes.
Paper introduces efficient uncertainty estimation in LLMs without multiple forward passes.
problem Accurate uncertainty quantification in LLMs remains challenging.
method Evidential Knowledge Distillation to create compact student models.
result Efficient uncertainty estimation achieved with single forward pass.
Develops a multilevel Monte Carlo framework with dropout for efficient uncertainty quantification.
problem Efficiently quantify uncertainty in complex models using dropout.
method Integrates multilevel Monte Carlo with Monte Carlo dropout, creating coupled estimators to reduce variance.
result Demonstrates significant variance reduction and efficiency gains over single-level Monte Carlo dropout.
Paper proposes BMPO to optimize policies using bidirectional models.
problem Model-based reinforcement learning's reliance on forward model accuracy.
method Develops BMPO using both forward and backward models for policy optimization.
result BMPO outperforms state-of-the-art methods in sample efficiency and asymptotic performance.
Reinforcement learning algorithms struggle when the reward signal is very sparse. In these cases, naive random exploration methods essentially rely on a random walk to stumble onto a rewarding state. Recent works utilize intrinsic motivation to guide the exploration via generative models, predictive forward models, or …
A hybrid physics-ML model predicts FO water flux with high accuracy and uncertainty quantification.
problem Challenges in accurately modeling Forward Osmosis water flux due to complex internal mass transfer phenomena.
method Robust Hybrid Physics-ML framework using Gaussian Process Regression (GPR) for uncertainty-aware Jw prediction.
result Achieved a state-of-the-art MAPE of 0.26% and R2 of 0.999 on independent test data.
SGD converges with perturbed forward-backward passes, explained by geometric amplification.
problem Analyzing convergence of SGD with perturbed forward-backward passes in composite optimization.
method Characterized propagation and amplification of perturbations, derived convergence guarantees for non-convex and PL objectives.
result Perturbations cascade through the computational graph, affecting convergence order under specific conditions.
In this paper, we model the impact of oil price volatility on Tehranstock and industry indices in two periods of international sanctions and post-sanction. To analyse the purpose of study, we use Feed-forward neural net-works. The period of study is from 2008 to 2018 that is split in two periods during international en…
Single model predicts multiple subnetworks for robustness.
problem Efficiently ensemble neural networks for robustness and uncertainty.
method Multi-input multi-output (MIMO) configuration to train independent subnetworks.
result Significant improvement in negative log-likelihood, accuracy, and calibration error.
Established PFPPs in complete markets, solving integral equations.
problem Existence of Predictable Forward Performance Processes in complete markets.
method Solving a one-period integral equation using Fourier transform for tempered distributions.
result Closed-form solutions for PFPPs with inverse marginal functions that are completely monotonic.
New bidirectional model predicts magnetohydrodynamics fields and estimates uncertainty.
problem Predicting multiple fields in magnetohydrodynamics with uncertainty.
method Bidirectional autoregressive latent diffusion approach.
result Model can estimate uncertainty without ground truth using self-supervised consistency.
The study compares feed-forward and attention layers in language models.
problem Understanding the role of feed-forward and attention layers in language models.
method Empirical and theoretical analysis in a synthetic setting.
result Feed-forward layers learn simple distributional associations, while attention layers focus on in-context reasoning.
New method estimates data influence efficiently by leveraging test samples.
problem Efficiently estimating influence of training data on model predictions.
method Mirrored Influence Hypothesis, forward pass for test samples.
result Significant improvement in efficiency over existing methods.
PFP-BNNs offer a fast, deterministic approach to Bayesian neural networks.
problem Limited uncertainty handling in traditional neural networks restricts their use in safety-critical settings.
method Probabilistic Forward Pass (PFP) approximates Stochastic Variational Inference (SVI) for efficient BNNs.
result PFP-BNNs achieve up to 4200x speedup over SVI-BNNs while maintaining similar accuracy and uncertainty.
Complex problems may require sophisticated, non-linear learning methods such as kernel machines or deep neural networks to achieve state of the art prediction accuracies. However, high prediction accuracies are not the only objective to consider when solving problems using machine learning. Instead, particular scientif…
This paper introduces forward-looking measures of the network connectedness of fears in the financial system, arising due to the good and bad beliefs of market participants about uncertainty that spreads unequally across a network of banks. We argue that this asymmetric network structure extracted from call and put tra…
Develops a forward variable selection method for interpretable random forest models.
problem Interpreting high-dimensional non-parametric models like random forests.
method Forward variable selection using CRPS as loss function, with hypothesis testing at each step.
result Method selects a smaller set of variables that optimizes predictive performance.
FP uses random projections to train networks without feedback, achieving comparable performance to backpropagation.
problem Training neural networks without feedback from downstream layers.
method Forward Projection (FP) method that uses randomised nonlinear projections and closed-form regression.
result FP achieves comparable generalisation to backpropagation methods with a single forward pass, offering significant speedup.
LSTM and gradient boosting models fail to outperform random chance in predicting MNQ futures.
problem Predicting intraday direction in MNQ futures using LSTM and gradient boosting.
method Comparing LSTM and gradient boosting models on 944 trading days of MNQ futures data.
result No model achieves statistically significant accuracy above random chance.
A new method ranks and selects features without model fitting.
problem Feature importance measures algorithm-specific and need improvement.
method Integrates global sensitivity analysis with forward selection and backward elimination.
result Demonstrates clear advantage over state-of-the-art methods.
Recurrent neural networks like long short-term memory (LSTM) are important architectures for sequential prediction tasks. LSTMs (and RNNs in general) model sequences along the forward time direction. Bidirectional LSTMs (Bi-LSTMs) on the other hand model sequences along both forward and backward directions and are gene…
Bayesian ANN method predicts chaotic systems with uncertainty.
problem Estimating chaotic dynamical systems from noisy data.
method Bayesian Artificial Neural Networks for ODE inverse problems.
result Accurate time predictions and uncertainty bounds.
Simple neural net outperforms complex uncertainty methods.
problem Reliable uncertainty estimation from deterministic models.
method A simple baseline using a single softmax neural net with residual connections and spectral normalization.
result Simple neural net outperforms DUQ and SNGP on uncertainty prediction.
A reliable controller is critical and essential for the execution of safe and smooth maneuvers of an autonomous vehicle.The controller must be robust to external disturbances, such as road surface, weather, and wind conditions, and so on.It also needs to deal with the internal parametric variations of vehicle sub-syste…
ACI identifies cause-effect relationships and causal influence ranges in dynamical systems.
problem Detecting and quantifying causal influence ranges in complex systems.
method Bayesian data assimilation and assimilative causal inference (ACI) to trace causes back from observed effects.
result Mathematically rigorous formulations of forward and backward causal influence ranges (CIRs) for nonlinear dynamical systems.
Study uses AI and ML to predict and optimize corrosion resistance of aluminum alloys.
problem Corrosion resistance of aluminum alloys in marine environments.
method Investigated two ML approaches: direct and inverse, using Random Forest, neural network, and Gaussian Process Regression.
result Gaussian Process Regression with hybrid kernel functions provided superior predictive performance.
Neural encoding and decoding, which aim to characterize the relationship between stimuli and brain activities, have emerged as an important area in cognitive neuroscience. Traditional encoding models, which focus on feature extraction and mapping, consider the brain as an input-output mapper without inner states. In th…
In this paper, we introduce a new form of amortized variational inference by using the forward KL divergence in a joint-contrastive variational loss. The resulting forward amortized variational inference is a likelihood-free method as its gradient can be sampled without bias and without requiring any evaluation of eith…
The paper establishes convergence guarantees for SGMs in 2-Wasserstein distance.
problem Establishing convergence guarantees for SGMs in 2-Wasserstein distance.
method Assuming accurate score estimates and smooth log-concave data distribution, the paper specializes its result to several concrete SGMs with specific forward processes modeled by stochastic differential equations.
result Obtained an upper bound on the iteration complexity for each model and a lower bound for Gaussian data distribution.
Deep learning predicts Bitcoin spot price movements from order books.
problem Predicting cryptocurrency spot price movements from order book data.
method Temporal CNNs trained on 2-second prediction time horizon.
result 71% walk-forward accuracy on coinbase data.
New method reduces diffusion model function evaluations for discrete data.
problem High computational burden in generating samples from masked diffusion models.
method Modified causal attention mask and speculative sampling mechanism for non-factorized predictions.
result Achieved ~2x reduction in required network forward passes.
LayerNorm transformers have dead directions that can be read from their parameters alone.
problem Locating dead directions in LayerNorm transformers
method Using the inverse-scale direction of LayerNorm affine parameters
result Predicted dead direction matches measured bottom singular direction