Dynamic linear models improve travel time prediction for congested freeways.
problem Accurate travel time prediction for congested freeways.
method Dynamic linear models (DLMs) with time-varying parameters.
result Significant improvements in travel time prediction accuracy, especially for short-term predictions.
Locally adaptive interpretable regression improves linear regression's predictability.
problem Linear regression's predictability is limited; it lacks adaptability.
method Locally adaptive interpretable regression (LoAIR) uses neural networks to predict percentile of a Gaussian distribution for regression coefficients.
result LoAIR achieves comparable or better predictive performance than state-of-the-art baselines.
NGSLL combines DNN accuracy with linear model interpretability.
problem Combining high accuracy of DNNs with interpretability of linear models.
method Neural generators of sparse local linear models (NGSLL) using DNNs to approximate non-linear functions.
result Effective in real-world datasets, achieving high predictive performance and interpretability.
Causality-aware methods outperform linear residualization in confounding adjustment for anticausal prediction.
problem Adjusting for confounding in anticausal prediction tasks.
method Causality-aware counterfactual confounding adjustment.
result Causality-aware methods asymptotically outperform linear residualization in predictive performance.
The paper integrates multiple Gaussian process predictions using Monte Carlo sampling.
problem Accurate prediction of variables using multiple models.
method Log-linear pooling of Gaussian process predictions, combined with Monte Carlo sampling.
result The log-linear pooling method improves prediction accuracy compared to linear pooling.
New method for interpreting non-linear models using forward marginal effects.
problem Interpreting non-linear models' feature effects is challenging.
method Introducing forward marginal effects and partitioning feature space for better interpretation.
result Improved interpretation of non-linear prediction functions.
QLA improves Bayesian uncertainty estimation for DNNs without increasing computational cost.
problem Overconfident out-of-distribution predictions from DNNs.
method Proposes Quadratic Laplace Approximation (QLA) to improve Bayesian uncertainty quantification.
result QLA yields modest yet consistent uncertainty estimation improvements over Linearized Laplace Approximation (LLA) on five regression datasets.
This paper improves Bayesian neural nets by using local linearization.
problem Underfitting in Bayesian neural networks.
method Local linearization of Bayesian neural networks to create a generalized linear model (GLM) for predictions.
result The GLM predictive resolves common underfitting problems of the Laplace approximation.
New adaptive models improve prediction accuracy with missing data.
problem Improving prediction accuracy with missing data entries.
method Adaptive optimization approach, learning imputation and regression simultaneously.
result 2-10% improvement in out-of-sample accuracy in strongly non-random missing data settings.
We use supervised learning to identify factors that predict the cross-section of returns and maximum drawdown for stocks in the US equity market. Our data run from January 1970 to December 2019 and our analysis includes ordinary least squares, penalized linear regressions, tree-based models, and neural networks. We fin…
Linear-Core Surrogates combine fast optimization and statistical efficiency in classification and structured prediction.
problem The trade-off between smoothness and margin-based losses in classification and structured prediction.
method Linear-Core (LC) Surrogates, a family of convex loss functions that stitch a linear core to a smooth tail.
result LC Surrogates achieve fast linear consistency rates while maintaining differentiability and strict H-consistency bounds. New methods combine model predictions to avoid linear mixtures' limitations.
problem Combining predictions from different models to avoid linear mixtures' limitations.
method Log-linear pooling (locking) and quantum superposition (quacking) to optimise model weights.
result Demonstrated locking method with illustrative example and practical application.
SELO model predicts link signs better than SDGNN using subgraph encoding and linear optimization.
problem Inferring the sign of links in signed networks with limited sign data.
method Subgraph Encoding via Linear Optimization (SELO) approach to learn edge embeddings.
result SELO model outperforms state-of-the-art methods on multiple real-world signed networks.
Efficiently combines probabilistic predictions using kernel embeddings.
problem Combining multiple probabilistic predictions to improve forecast accuracy.
method Embed predictions into RKHS, optimizes kernel-based scoring rules, finds efficient implementation.
result Generalised linear pool outperforms traditional linear pool in operational wind speed forecasts.
New algorithm accelerates single-pass SGD for generalized linear prediction.
problem Improving single-pass non-quadratic stochastic optimization.
method Data-dependent proximal method incorporating dual-momentum acceleration.
result Momentum acceleration resolves open problem in streaming setting.
Study compares MoE and RNN models for stock price prediction across volatility profiles.
problem Improving stock price prediction accuracy across different volatility levels.
method Dynamic Mixture of Experts model combining RNN and linear models, adjusting weights through a gating network.
result MoE model outperforms individual models in reducing prediction errors.
Unified approach for optimizing predictions in linear programming and inverse problems.
problem Optimizing predictions in linear programming and inverse problems.
method Maximum optimality margin approach.
result Unified approach that balances computational efficiency and theoretical properties.
This paper improves fund net value prediction using ARIMA-LSTM hybrid model.
problem Inaccurate prediction of fund net value due to linear characteristics of models.
method ARIMA-LSTM hybrid model: ARIMA for linear characteristics, LSTM for nonlinear.
result Hybrid model shows higher accuracy in predicting fund net value.
E-LMC improves spatial field prediction accuracy by linearizing complex fields.
problem Predicting complex spatial fields with high accuracy and efficiency.
method Introducing an invertible neural network to linearize nonlinear spatial fields, enabling the use of LMC for nonlinear problems.
result Maximum improvement of about 40% over original LMC, outperforming other models.
Matrix completion aims to predict missing elements in a partially observed data matrix which in typical applications, such as collaborative filtering, is large and extremely sparsely observed. A standard solution is matrix factorization, which predicts unobserved entries as linear combinations of latent variables. We g…
DeepKriging uses DNNs to predict spatial data with improved accuracy and scalability.
problem Predicting spatial processes with non-linear and non-Gaussian data.
method Adds an embedding layer of spatial coordinates with basis functions to DNNs.
result DeepKriging provides non-linear predictions with smaller approximation errors and is scalable for large datasets.
Method predicts future rewards from past actions in a linear Gaussian system.
problem Maximizing cumulative reward in a stochastic multi-armed bandit with linear Gaussian dynamics.
method Proposes a method using a modified Kalman filter to predict future rewards based on past rewards.
result Reward from any action can be used to predict another action's future reward.
Paper explains DRL strategies for portfolio management using linear models.
problem Difficulty in understanding DRL-based trading strategies.
method Empirical approach using linear models and integrated gradients.
result DRL agents show stronger multi-step prediction power than machine learning methods.
Heterosis is the improved or increased function of any biological quality in a hybrid offspring. We have studied yet the largest maize SNP dataset for traits prediction. We develop linear and non-linear models which consider relationships between different hybrids as well as other effect. Specially designed model prove…
Bi-linear feature learning models, like the gated autoencoder, were proposed as a way to model relationships between frames in a video. By minimizing reconstruction error of one frame, given the previous frame, these models learn "mapping units" that encode the transformations inherent in a sequence, and thereby learn …
Enhances linear regression with Kalman filter for loss minimization.
problem Minimizing loss in linear regression models.
method Integrates Kalman filter and SGD for optimal weight updates.
result Develops optimal linear regression equation with minimum area under curve.
The paper investigates non-linear and heavy-tailed predictability in transition-energy financial markets.
problem Incomplete representation of dependence structure in Gaussian-linear forecasting frameworks.
method Develops a hybrid forecasting framework combining Student-t Vector Autoregressions with nonlinear recurrent residual learning architectures.
result The proposed framework consistently improves predictive accuracy relative to conventional models, especially during macro-financial stress.
Proposes a new method for decision-aware learning in optimization.
problem Contextual linear optimization with cost prediction errors.
method Reweighing prediction error by decision regret for decision-aware predictor.
result Improves over predict-then-optimize framework for misspecified models.
New method calibrates probabilistic linear solver for online coverage guarantees.
problem Uncertainty in probabilistic linear solver solutions without coverage guarantees.
method Online conformal prediction-PLS (OCP-PLS) method to calibrate uncertainty thresholds.
result Validates online calibration of uncertainty thresholds via online conformal prediction.
Complex problems may require sophisticated, non-linear learning methods such as kernel machines or deep neural networks to achieve state of the art prediction accuracies. However, high prediction accuracies are not the only objective to consider when solving problems using machine learning. Instead, particular scientif…
This work explains how linear representations in large language models arise from training objectives and gradient descent.
problem Understanding the origins of linear representations in large language models.
method A latent variable model to abstract and formalize concept dynamics, combined with analysis of the softmax cross-entropy objective and gradient descent.
result Linear representations emerge when learning from data matching the latent variable model, and this simple structure suffices to yield linear representations.
While we would like to predict exact values, available incomplete information is rarely sufficient - usually allowing only to predict conditional probability distributions. This article discusses hierarchical correlation reconstruction (HCR) methodology for such prediction on example of usually unavailable bid-ask spre…
New method predicts and optimizes matrix recovery from noisy measurements.
problem Recovering rank-1 matrices from Gaussian measurements with noise.
method Stochastic prox-linear iterative algorithm with trajectory predictions.
result The method converges linearly with accurate predictions of error.
Study the impact of overfitting on linear predictive models' performance.
problem Overfitting reduces the out-of-sample performance of linear predictive trading strategies.
method Computed in- and out-of-sample means and variances of PnLs to derive replication ratios.
result Replication ratio diminishes for complex strategies with many assets.
RestoreAI predicts landmine risk from patterns, improving clearance efficiency.
problem Predicting landmine risk from spatial patterns to enhance clearance efficiency.
method RestoreAI uses landmine patterns for risk prediction, implementing three deminers: linear, curved, and Bayesian.
result RestoreAI significantly boosts clearance efficiency, achieving a 14.37 percentage point increase in cleared landmines per timestep.
Study optimizes prediction error for growing-dimensional PFLM models.
problem Optimizing prediction error for growing-dimensional PFLM models.
method Penalized least-squares approach in RKHS with effective dimension consideration.
result Shows exact upper bound for excess prediction risk in non-asymptotic form.
We analyzed optimism in linear and kernel regression models.
problem Understanding predictive complexity in regression models.
method Derived closed-form asymptotic optimism for linear and kernel regression models.
result Scaled optimism is a useful measure for model complexity.
This paper extends performative prediction to nonlinear cases.
problem Performative prediction's effectiveness is limited by linear assumptions in real-world applications.
method Formulated a maximum margin approach loss function and extended it to nonlinear spaces using kernel methods.
result Derived conditions for performative stability in both linear and nonlinear cases.
Introduces CCR for constructing confidence regions from conformal predictions.
problem Challenges in constructing confidence regions for model parameters.
method Combines conformal prediction intervals for model outputs to establish confidence regions for parameters under minimal assumptions.
result Valid coverage guarantees for finite sample regime, applicable to various model types.
Traditional linear methods for forecasting multivariate time series are not able to satisfactorily model the non-linear dependencies that may exist in non-Gaussian series. We build on the theory of learning vector-valued functions in the reproducing kernel Hilbert space and develop a method for learning prediction func…
Study evaluates machine learning for predicting treatment effects in observational studies.
problem Challenges in measuring treatment effects due to confounding bias in observational studies.
method Simulated two scenarios with and without confounding, using linear and non-linear relationships. Used machine learning models (linear regression, lasso regression, random forest) to predict counterfactuals and treatment effects.
result Machine learning models perform well under linearity but poorly under non-linearity, even in the presence of confounding.
BART and MOTR-BART improve tree-based predictions with local linear models.
problem Non-linearity and high-order interactions in data.
method Bayesian Additive Regression Trees (BART) and Model Trees BART (MOTR-BART) using piecewise linear functions.
result MOTR-BART achieves equal or better performance with fewer trees than BART.
New findings on representation learning beyond linear functions.
problem Achieving diversity in representation learning beyond linear prediction functions.
method Analysis of eluder dimension and empirical risks.
result Diversity holds even with nonlinear prediction functions and multiple layers in neural networks.
A linear multi-factor model is one of the most important tools in equity portfolio management. The linear multi-factor models are widely used because they can be easily interpreted. However, financial markets are not linear and their accuracy is limited. Recently, deep learning methods were proposed to predict stock re…
Improved bipartite link prediction using 2-hop paths.
problem Link prediction in bipartite networks without node attributes.
method Multiply reconstructed adjacency matrix with symmetrically normalized training adjacency matrix to form 2-hop paths.
result 2-hop paths improve link prediction performance.
Work proposes a new framework to improve uncertainty estimation in deep Bayesian models.
problem Traditional training procedures underestimate uncertainty in NLMs, leading to unreliable predictions.
method Introduces a novel training framework that captures useful predictive uncertainties for out-of-distribution inputs.
result Demonstrates that traditional methods for NLMs significantly underestimate uncertainty and propose a new framework to address this issue.
The interpretability of prediction mechanisms with respect to the underlying prediction problem is often unclear. While several studies have focused on developing prediction models with meaningful parameters, the causal relationships between the predictors and the actual prediction have not been considered. Here, we co…
Accurate prediction of user consumption is a key part not only in understanding consumer flexibility and behavior patterns, but in the design of robust and efficient energy saving programs as well. Existing prediction methods usually have high relative errors that can be larger than 30% and have difficulties accounting…