This study compares multivariate vs univariate machine learning for multi-output regression.
problem When to use multivariate ensemble techniques over separate univariate models.
method Comparative analysis of different multivariate approaches for multi-output regression.
result Multivariate ensemble techniques outperform separate univariate models in simulations.
We propose a family of multivariate Gaussian process models for correlated outputs, based on assuming that the likelihood function takes the generic form of the multivariate exponential family distribution (EFD). We denote this model as a multivariate generalized Gaussian process model, and derive Taylor and Laplace al…
New method improves calibration in multi-output probabilistic models.
problem Challenges in achieving multivariate calibration in multi-output regression.
method General regularization framework to enforce multivariate calibration during training for arbitrary pre-rank functions.
result Significant improvement in calibration across all pre-rank functions without sacrificing predictive accuracy.
Optimal transport improves multivariate prediction uncertainty quantification.
problem Uncertainty quantification in multivariate learning tasks, especially in regression and classification.
method Introducing a novel Conformal Prediction procedure using optimal transport to handle multivariate score functions and construct flexible prediction regions.
result Ensures finite-sample, distribution-free coverage guarantees for multivariate prediction sets.
Improved multivariate conformal prediction by standardizing residuals.
problem Weak conditional coverage in heteroskedastic multivariate settings.
method Natural extension of univariate normalization to multivariate setting, whitening residuals and standardizing local variance.
result Standardized residuals yield asymptotic conditional coverage under certain distributions.
New clustering method uses Wasserstein distance to analyze simulation outputs.
problem Analyzing stochastic simulation outputs to uncover relationships and patterns.
method Agglomerative clustering using regularized Wasserstein distance.
result Identifies staffing plans yielding similar performance outcomes.
Gaussian process model for vector-valued function has been shown to be useful for multi-output prediction. The existing method for this model is to re-formulate the matrix-variate Gaussian distribution as a multivariate normal distribution. Although it is effective in many cases, re-formulation is not always workable a…
We consider the topic of multivariate regression on manifold-valued output, that is, for a multivariate observation, its output response lies on a manifold. Moreover, we propose a new regression model to deal with the presence of grossly corrupted manifold-valued responses, a bottleneck issue commonly encountered in pr…
OTCP extends conformal prediction to multivariate data using optimal transport.
problem Uncertainty quantification in multivariate machine learning models.
method OTCP leverages optimal transport to rank multivariate conformity scores.
result Preserves distribution-free coverage guarantees in multidimensional settings.
A method for constructing tight prediction intervals for multiple numerical outputs.
problem Constructing tight prediction intervals for multiple related numerical outputs.
method A novel coordinate-wise standardization procedure that makes residuals comparable across output dimensions, estimating suitable scaling parameters using calibration data.
result The method produces tighter prediction intervals than existing baselines while maintaining valid simultaneous coverage.
Wide Boosting improves GB's performance on multivariate output tasks.
problem Lack of flexibility in fitting probabilistic multi-dimensional outputs.
method Inserts matrix multiplication between GB output and loss function.
result Wide Boosting outperforms Gradient Boosting on multivariate output tasks.
Aims to optimize complex multivariate systems with constraints.
problem Optimizing force-field systems in physics with large-scale simulations.
method Combines machine learning and experimental design to find feasible input combinations.
result Locates multiple good regions in the input space.
Recently there has been an increasing interest in the multivariate Gaussian process (MGP) which extends the Gaussian process (GP) to deal with multiple outputs. One approach to construct the MGP and account for non-trivial commonalities amongst outputs employs a convolution process (CP). The CP is based on the idea of …
Unified study of nine multi-output conformal methods with generalized scores.
problem Challenges in extending conformal prediction to multi-output problems.
method Nine conformal methods with generalized multi-output conformity scores.
result Generalized scores ensure asymptotic conditional coverage and exact finite-sample marginal coverage.
We introduce a new regression framework, Gaussian process regression networks (GPRN), which combines the structural properties of Bayesian neural networks with the non-parametric flexibility of Gaussian processes. This model accommodates input dependent signal and noise correlations between multiple response variables,…
Despite tremendous progress in outlier detection research in recent years, the majority of existing methods are designed only to detect unconditional outliers that correspond to unusual data patterns expressed in the joint space of all data attributes. Such methods are not applicable when we seek to detect conditional …
Early approaches to multiple-output Gaussian processes (MOGPs) relied on linear combinations of independent, latent, single-output Gaussian processes (GPs). This resulted in cross-covariance functions with limited parametric interpretation, thus conflicting with the ability of single-output GPs to understand lengthscal…
Bayesian method for multivariate autoregressive models with exogenous inputs.
problem Estimating uncertainties in autoregressive models with exogenous inputs.
method Recursive Bayesian estimation via message passing in a factor graph.
result Produces full posterior distributions for autoregressive coefficients and noise precision.
Proposes MVG-CRPS for robust multivariate forecasting.
problem Outliers in multivariate forecasting lead to significant errors.
method Integrates CRPS for MVG distributions, optimizing with MVG-CRPS.
result Improves robustness, accuracy, and uncertainty quantification.
New method extends conformal prediction to multivariate settings using optimal transport.
problem Limited applicability of conformal prediction to multivariate real-valued scores.
method Use optimal transport to define vector-ranks and multivariate quantile regions for finite-sample coverage.
result Constructs the first multivariate conformal predictive distributions with finite-sample calibration.
The paper analyzes the generalizability of linear autoencoders and multivariate linear regression.
problem Limited theoretical understanding of linear autoencoders' performance.
method Proposes a PAC-Bayes bound for multivariate linear regression and shows LAEs as constrained models.
result The proposed PAC-Bayes bound is tight and correlates with practical metrics.
Hidden Markov Models (HMM) have been used for several years in many time series analysis or pattern recognitions tasks. HMM are often trained by means of the Baum-Welch algorithm which can be seen as a special variant of an expectation maximization (EM) algorithm. Second-order training techniques such as Variational Ba…
Develops privacy-preserving multivariate median estimation methods.
problem Lack of rigorous privacy guarantees for robust multivariate location estimation.
method Novel finite-sample performance guarantees for differentially private multivariate depth-based medians.
result Sharp performance guarantees for multivariate depth-based medians under differential privacy.
Proposes a method to generate multivariate prediction intervals for random forests.
problem Uncertainty estimates for iterative design of experiments with multiple correlated model outputs.
method Recalibrated bootstrap method for bagged models.
result Significantly decreases the number of iterations required for satisfactory candidate in sequential learning problems.
Combines MCTM and NF for flexible multivariate density regression with interpretable marginals.
problem Difficult interpretation of flexible NF models and limitations of MCTM in flexibility.
method Hybrid approach combining MCTM for interpretable marginals and NF for complex joint distributions.
result Demonstrates versatility and improved performance compared to MCTM and other NF models.
A neural network with a single hidden layer can't represent certain multivariable functions.
problem Representing certain multivariable functions with a neural network having only one hidden layer.
method Developed a continuum version of a one-hidden-layer neural network with ReLU activation, and proved constraints on its parameters and second derivative.
result Existence of a smooth binary function that cannot be precisely represented by any such neural network.
Multi-output prediction deals with the prediction of several targets of possibly diverse types. One way to address this problem is the so called problem transformation method. This method is often used in multi-label learning, but can also be used for multi-output prediction due to its generality and simplicity. In thi…
Multi-output Gaussian processes (MOGP) are probability distributions over vector-valued functions, and have been previously used for multi-output regression and for multi-class classification. A less explored facet of the multi-output Gaussian process is that it can be used as a generative model for vector-valued rando…
A new variable importance measure for DRFs detects broader impacts on output distributions.
problem Estimating full conditional distributions of multivariate outputs given inputs.
method Based on the drop and relearn principle and MMD distance.
result Consistent and high-performing variable importance measure for DRFs.
Enhances neural forecasting for hierarchically organized time series data.
problem Probabilistic coherent forecasting of time series data across different levels of aggregation.
method Proposes a coherent multivariate mixture output for neural forecasting architectures, optimizing with a composite likelihood objective.
result 13.2% average accuracy improvements on most datasets compared to state-of-the-art baselines.
OFTER predicts multivariate time series online, outperforming baselines.
problem Mid-sized multivariate time series forecasting challenges.
method k-nearest neighbors, Generalized Regression Neural Networks, dimensionality reduction.
result OFTER outperforms state-of-the-art baselines in financial multivariate time series forecasting.
MPVAE learns latent embeddings and label correlations for multi-label classification.
problem Challenging task of predicting multiple targets with label correlations.
method Proposes MPVAE, a novel framework that learns latent embedding spaces and label correlations using a Multivariate Probit model.
result MPVAE outperforms state-of-the-art methods on various application domains and is robust under noisy settings.
CoCAI uses copulas for accurate multivariate time-series forecasting and anomaly detection.
problem Accurate multivariate time-series forecasting and robust anomaly detection.
method Copula-based conformal prediction for multivariate time-series analysis.
result CoCAI provides statistically valid predictive regions and robust anomaly scores.
We propose multivariate nonstationary Gaussian processes for jointly modeling multiple clinical variables, where the key parameters, length-scales, standard deviations and the correlations between the observed output, are all time dependent. We perform posterior inference via Hamiltonian Monte Carlo (HMC). We also prov…
Outlier detection aims to identify unusual data instances that deviate from expected patterns. The outlier detection is particularly challenging when outliers are context dependent and when they are defined by unusual combinations of multiple outcome variable values. In this paper, we develop and study a new conditiona…
We introduce a new category of multivariate conditional generative models and demonstrate its performance and versatility in probabilistic time series forecasting and simulation. Specifically, the output of quantile regression networks is expanded from a set of fixed quantiles to the whole Quantile Function by a univar…
Function trees simplify complex ML models for better understanding.
problem Understanding and interpreting machine learning model predictions.
method Representing a multivariate function as a tree of simpler functions.
result Function trees reveal the global internal structure of functions.
Detecting anomalies in multivariate functional data using Bayesian nonparametric methods.
problem Detecting anomalies in functional data.
method Bayesian nonparametric approach with infinite mixture of multi-output Gaussian processes.
result Anomalous observations assigned to small mixture components.
Data transformation, e.g. feature transformation and selection, is an integral part of any machine learning procedure. In this paper we introduce an information-theoretic model and tools to assess the quality of data transformations in machine learning tasks. In an unsupervised fashion, we analyze the transfer of infor…
Extremes play a special role in Anomaly Detection. Beyond inference and simulation purposes, probabilistic tools borrowed from Extreme Value Theory (EVT), such as the angular measure, can also be used to design novel statistical learning methods for Anomaly Detection/ranking. This paper proposes a new algorithm based o…
The paper develops exact and approximate conformal inference methods for multi-output regression.
problem Uncertainty quantification in multi-output regression predictions.
method Exact derivations and approximations of conformal inference p-values for linear multi-output predictors, and efficient methods for nonlinear predictors.
result Efficient methods for approximating conformal prediction regions for multi-output predictors, both linear and nonlinear.
STAM learns important time steps and variables for multivariate time series prediction.
problem Accurate interpretation of multivariate time series predictions.
method Spatiotemporal attention mechanism (STAM) for multivariate time series modeling.
result STAM maintains state-of-the-art prediction accuracy with improved interpretability.
Proposes a flexible MGP model for dynamic, sparse correlations.
problem Handling dynamic and sparse correlations in multivariate data.
method Non-stationary MGP with dynamic spike-and-slab prior and EM algorithm.
result Captures dynamic and sparse correlations effectively.
The paper monitors artificial neural networks using embeddings and multivariate control charts.
problem Ensuring the stability of artificial neural networks during deployment.
method Monitoring the latent feature representation (embedding) of ANN outputs using multivariate control charts based on data depth and ranks.
result The proposed method effectively detects nonstationary data streams in ANN outputs.
A new model predicts multivariate regression using similarities to data points.
problem Complex, high-dimensional input-output relationships.
method Bayesian mixture-of-experts with conditional Gaussian mixtures and variational Bayes.
result Outperforms competitors in high-dimensional settings.
We propose a probabilistic model for inferring the multivariate function from multiple areal data sets with various granularities. Here, the areal data are observed not at location points but at regions. Existing regression-based models can only utilize the sufficiently fine-grained auxiliary data sets on the same doma…
Deep learning predicts employment changes and industry health.
problem Forecasting short-term employment changes and assessing long-term industry health.
method LSTNet, a multi-scale deep learning model, processes multivariate time series data.
result LSTNet outperforms baseline models in most sectors, especially stable ones.
We propose a general matrix-valued multiple kernel learning framework for high-dimensional nonlinear multivariate regression problems. This framework allows a broad class of mixed norm regularizers, including those that induce sparsity, to be imposed on a dictionary of vector-valued Reproducing Kernel Hilbert Spaces. W…