This study compares multivariate vs univariate machine learning for multi-output regression.
problem When to use multivariate ensemble techniques over separate univariate models.
method Comparative analysis of different multivariate approaches for multi-output regression.
result Multivariate ensemble techniques outperform separate univariate models in simulations.
Study assesses drought and late-frost risks in Bavaria using vine copulas.
problem Assessing risks of late-frost and drought in Bavaria due to climate change.
method Used vine copula models for non-Gaussian and asymmetric dependencies, with univariate and bivariate regression analyses.
result Identified 'at-risk' regions for forest adaptation.
New method for multivariate distribution regression using NPT metric.
problem Regression with multivariate distributional responses and Euclidean predictors.
method Fréchet regression with nonparanormal transport (NPT) metric.
result Efficient estimation and granular interpretation of predictor effects.
Kolmogorov-Arnold Networks achieve optimal convergence rates in nonparametric regression.
problem Nonparametric function approximation in multivariate settings.
method Structured additive and multiplicative KANs using B-splines.
result Achieve minimax-optimal convergence rate O(n−2r/(2r+1)) for Sobolev space functions. New model predicts time series quantiles for nonstationary data.
problem Nonparametric probabilistic forecasting of nonstationary univariate time series.
method Composite Quantile Fourier Neural Network (QFNN) for extrapolation-based nonlinear quantile regression.
result Effective in providing high quality and accurate probabilistic predictions.
Shallow neural networks can represent polynomials efficiently.
problem Representing polynomials using shallow neural networks.
method Using shallow neural networks of width 2(R+d)d to represent d-variate polynomials of degree R. result Derives minimax optimal convergence rate for shallow networks to unknown univariate regression functions.
Proposes a model to relate a tensor feature to a univariate outcome using sparse and low-rank components.
problem Relating a univariate outcome to a feature tensor with sparse and low-rank components.
method Divide-and-conquer strategy, stagewise estimation procedure for unit-rank tensor regression.
result The stagewise solution paths converge to those of regularized regression as step size goes to zero.
Gradient optimization improves functional tensor-train regression performance.
problem Low-rank functional regression from scattered data.
method Gradient-based optimization of functional tensor-train parameters.
result Gradient methods outperform standard ALS in low-sample number scenarios.
Mirror flow in shallow neural networks shows similar implicit bias to gradient flow, with key differences in curvature penalties.
problem Analyzing implicit bias in shallow neural networks with mirror flow.
method Characterization through variational problems and scaled potentials.
result Mirror flow with scaled potentials induces a rich class of biases not captured by RKHS norms.
New statistical methods improve explainability of boosting models.
problem Uncertainty quantification for boosting models is computationally intensive and hard to interpret.
method Derive methods for statistical inference using gradient boosting and Boulevard regularization.
result Achieve asymptotically normal predictions with theoretical guarantees and runtime independent of data size.
We introduce a family of adaptive estimators on graphs, based on penalizing the ℓ1 norm of discrete graph differences. This generalizes the idea of trend filtering [Kim et al. (2009), Tibshirani (2014)], used for univariate nonparametric regression, to graphs. Analogous to the univariate case, graph trend filteri…
The paper compares traditional and state-space methods for analyzing foreign exchange risk premia.
problem Analyzing the existence and time-evolving property of foreign exchange risk premia.
method Examines foreign exchange risk premia from simple univariate regressions to state-space methods.
result State-space estimations are more effective in examining the time variability of unobservable risk premia.
The paper develops a new method for estimating non-parametric regression functions with spatio-temporal dependencies.
problem Estimating non-parametric regression functions with spatio-temporal dependencies.
method Locally Adaptive Regression Splines (LARS) with ADMM algorithm.
result The method shows superior performance compared to existing techniques.
Constructs bivariate quantiles using vine copulas for multivariate analysis.
problem Need for research in multivariate quantiles, especially for bivariate responses.
method Constructs bivariate (conditional) quantiles using vine copula based bivariate regression model with a novel tree sequence graph structure.
result Avoids typical shortfalls of regression like transformations, interactions, collinearity, and quantile crossings.
GRM models k-way dependencies in univariate exponential families.
problem Modeling dependencies between variable sets of size k > 2.
method Taking k-th root of sufficient statistics for univariate exponential families.
result GRM models for Poisson and exponential families have no and only slight restrictions on parameters, respectively.
Survey on mean estimation and regression for heavy-tailed data.
problem Estimating mean and regression functions in heavy-tailed distributions.
method Sub-Gaussian mean estimators, median-of-means, trimmed mean, Catoni's estimator.
result Detailed proofs for estimators in heavy-tailed settings.
Unified study of nine multi-output conformal methods with generalized scores.
problem Challenges in extending conformal prediction to multi-output problems.
method Nine conformal methods with generalized multi-output conformity scores.
result Generalized scores ensure asymptotic conditional coverage and exact finite-sample marginal coverage.
Multivariate regression model is a natural generalization of the classical univari- ate regression model for fitting multiple responses. In this paper, we propose a high- dimensional multivariate conditional regression model for constructing sparse estimates of the multivariate regression coefficient matrix that accoun…
Global models outperform univariate benchmarks in complex time series forecasting.
problem Comparing global forecasting models to univariate benchmarks in various challenging scenarios.
method Simulated datasets with controlled characteristics, including homogeneity, complexity, and series lengths. Global forecasting models (RNN, LGBM) compared to univariate techniques.
result Global models like RNN and LGBM are competitive in complex scenarios with short series lengths and heterogeneous data.
New methods for multivariate nonparametric regression reduce dimensionality issues.
problem Nonparametric regression in high dimensions with covariates.
method Introduced entirely monotonic and constrained Hardy-Krause variation LSEs.
result Risk properties and minimax lower bounds for these LSEs.
Multivariate boosted trees improve forecasting and control by capturing correlated predictions.
problem Capturing multivariate target cross-correlations and applying structured penalties to predictions.
method A computationally efficient algorithm for fitting multivariate boosted trees.
result Multivariate trees outperform univariate counterparts in correlated prediction scenarios.
The paper determines the optimal number of machines for parallel computing in kernel ridge regression.
problem How many machines can be used in parallel computing for kernel ridge regression?
method Empirical processes method
result Upper bounds on the number of machines are proven to be un-improvable in two important cases.
MRI image quality affects statistical and predictive analysis of brain morphology.
problem Impact of MRI image quality on statistical and predictive analysis of brain morphology.
method Systematic testing of image quality on univariate statistics and machine learning classification using three large datasets.
result Low-quality MRI data significantly affects detecting significant sex/gender differences in smaller samples, but not in larger ones.
DSPPs improve predictive distributions in scalable regression tasks.
problem Improving predictive distributions in scalable regression tasks.
method Inspired by DGPs, DSPPs use mini-batch training and kernel basis functions for uncertainty control.
result DSPPs provide significantly better calibrated predictive distributions than other methods.
SBAMDT uses adaptive soft splits to model complex decision boundaries.
problem Limited ability of standard decision trees to capture complex decision boundaries.
method Probabilistic additive decision tree model with adaptive soft multivariate splits.
result Demonstrated improved predictive performance on synthetic and real datasets.
Algorithm learns mixtures of linear regressions in subexponential time.
problem Learning mixtures of linear regressions with high accuracy.
method Fourier moment descent method using univariate density estimation and low-degree moments of Fourier transforms.
result First algorithm for learning MLRs in subexponential time.
Brenier isotonic regression extends multi-output isotonic regression using optimal transport.
problem Enforcing cyclic monotonicity in multi-output regression.
method Leverage Kantorovich's optimal transport to find cyclically monotone couplings.
result Brenier isotonic regression outperforms baselines in probability calibration.
Additive isotonic regression attempts to determine the relationship between a multi-dimensional observation variable and a response, under the constraint that the estimate is the additive sum of univariate component effects that are monotonically increasing. In this article, we present a new method for such regression …
Method estimates noise variance in Gaussian process regression.
problem Estimating noise variance in Gaussian process regression models.
method Reduces hyperparameter space, uses marginal likelihood function, derives bounds and asymptotes.
result Computational advantages and robustness compared to traditional methods.
Develops SQR models for multivariate exponential families allowing positive dependencies.
problem Lack of positive dependencies in multivariate graphical models for exponential and Poisson distributions.
method Introduces Square Root Graphical Models (SQR) derived from univariate exponential distributions, with methods for parameter estimation and likelihood approximation.
result Allows for arbitrary positive and negative dependencies in multivariate distributions without constraints on parameter values.
Modeling stock returns and volatility using a bivariate gamma generalized Laplace law.
problem Analyzing stock returns and volatility using a new statistical model.
method Maximum likelihood estimation for a bivariate generalized Laplace distribution, simplifying to linear regression.
result Explicit estimators derived with nonstandard convergence rates for certain parameter configurations.
G-GLN extends GLNs to multiple regression and density modeling.
problem Learning features in deep neural networks.
method G-GLN uses a distributed and local credit assignment mechanism based on optimizing a convex objective.
result G-GLN achieves competitive or state-of-the-art performance on regression benchmarks.
PEER tackles multi-response regression with incomplete outcomes efficiently.
problem Challenges in estimating, predicting, and computing with large-scale multi-response regression and incomplete outcomes.
method PEER converts multi-response regression into parallel univariate-response regressions.
result PEER achieves consistency in estimation, prediction, and variable selection.
Study learns a projection and function in Gaussian models.
problem Learning a one-dimensional projection and a univariate function in high-dimensional Gaussian models.
method Gradient flow dynamics of alternating scheme, RKHS adaptation.
result Gradient flow dynamics converge with rate controlled by Gaussian regularity.
Meta algorithm solves multivariate optimization using univariate optimizers.
problem Multivariate global optimization problems.
method Meta algorithm combining univariate global optimizers.
result Meta algorithm provides robust regret guarantees.
New theory shows large learning rates prevent overfitting in neural networks.
problem Generalization of two-layer ReLU neural networks in noisy regression problems.
method Gradient descent with constant learning rate converges to stable minima.
result Gradient descent with large learning rates finds smooth, sparse fits.
Estimates piecewise polynomials and bounded variation functions using optimal decision trees.
problem Estimating piecewise smooth functions in general dimensions.
method Dyadic CART and Optimal Regression Tree (ORT) estimators for piecewise polynomials and bounded variation functions.
result Oracle inequalities and risk bounds for ORT estimators, demonstrating adaptivity and optimality.
New method extracts aleatoric and epistemic uncertainties from regression-based neural networks.
problem Need for principled uncertainty reasoning in machine learning systems.
method Learning evidential distributions for aleatoric and epistemic uncertainties.
result Allows for the simultaneous extraction of both uncertainties without sampling or out-of-distribution data.
VEST automates feature engineering for time series forecasting.
problem Challenges in time series forecasting with improved performance.
method VEST combines auto-regression with statistical summarization of recent past dynamics.
result VEST significantly improves forecasting performance.
Paper proposes a new GPR-HS framework for accurate VCV estimation in global equity indices.
problem Accurate forecasting of Volatility-Covariance Matrix (VCV) for regulatory processes.
method Hybrid Gaussian Process Regression-Historical Simulation (GPR-HS) framework.
result GPR-HS framework achieves regulatory compliance and outperforms static VaR benchmarks.
Paper proposes a joint quantile regression for VaR and ES forecasting.
problem Forecasting Value at Risk (VaR) and Expected Shortfall (ES) of multiple assets simultaneously.
method Multivariate quantile regression framework with time-varying process for VaR and ES.
result The proposed method outperforms other models in risk measure forecasts.
Infers causal direction from mixed-type multivariate data using information theory.
problem Inferring causal direction from multivariate and mixed-type data.
method Information theoretic approach based on Kolmogorov complexity and Minimum Description Length (MDL) principle.
result Crack algorithm reliably infers causal direction with high accuracy.
The paper uses random matrix theory for multi-task regression, improving time series forecasting.
problem Improving time series forecasting using multi-task regression.
method Applying random matrix theory to multi-task regression problems, deriving closed-form solutions for optimization.
result Provides a robust foundation for hyperparameter optimization in multi-task regression scenarios.
New method uses MDL to infer causal direction between variables.
problem Inferring causal direction from observational data of two variables.
method Information theoretic approach based on Kolmogorov complexity and MDL principle.
result Proposes a compression scheme for encoding local and global functional relations.
New model predicts entire distribution of time series data.
problem Probabilistic forecasting of multivariate time series.
method Deep generative quantile-copula models with latent uniform distribution.
result Single neural network parameterizes joint predictive distribution.
Generative models often fail to preserve joint structure despite matching marginals.
problem Generative models fail to capture complex dependencies beyond univariate marginals.
method Introduced D_Sigma(P,Q) = ||Sigma_P - Sigma_Q||_F to measure covariance-level dependence fidelity.
result Covariance-level divergence can lead to structural instability in downstream inference.
New ensemble methods improve time series forecasting accuracy.
problem Global Forecasting Models (GFM) lack localisation for heterogeneous datasets.
method Ensemble techniques with clustering and varied GFM models.
result Significantly higher accuracy achieved compared to baseline models.
Smart Bayes integrates generative and discriminative features for improved classification.
problem Improving classification performance by combining generative and discriminative modeling.
method Integrates generative likelihood-ratio features into a logistic-regression-style classifier.
result Often outperforms logistic regression and Naive Bayes in simulations and real data.