Bayesian kernel regression improves functional output prediction.
problem Functional output regression in supervised learning.
method Kernel methods, leveraging covariance structure within function values.
result Enhanced prediction accuracy and handling of high-dimensional nonlinearity.
Brenier isotonic regression extends multi-output isotonic regression using optimal transport.
problem Enforcing cyclic monotonicity in multi-output regression.
method Leverage Kantorovich's optimal transport to find cyclically monotone couplings.
result Brenier isotonic regression outperforms baselines in probability calibration.
RVFL NNs perform well without direct links and output bias for regression.
problem Effect of direct links and output bias on RVFL performance.
method Classical and two new methods for generating hidden nodes' parameters tested.
result Direct links and output bias do not significantly improve RVFL accuracy for typical nonlinear regression problems.
New method improves calibration in multi-output probabilistic models.
problem Challenges in achieving multivariate calibration in multi-output regression.
method General regularization framework to enforce multivariate calibration during training for arbitrary pre-rank functions.
result Significant improvement in calibration across all pre-rank functions without sacrificing predictive accuracy.
Study confirms learning rates for vector-valued spectral algorithms, proving consistency.
problem Theoretical confirmation of learning rates for vector-valued spectral algorithms.
method Rigorous analysis of learning rates for various vector-valued spectral algorithms, including kernel ridge regression and gradient descent.
result Upper and lower bounds on learning rates for vector-valued spectral algorithms, proving minimax optimality in various scenarios.
Active learning method reduces labeling cost for regression models with aggregated data.
problem Reducing labeling cost for training regression models with aggregated data.
method Sequentially selects sets to be labeled using mutual information quantifying model parameter uncertainty.
result Achieves better predictive performance with fewer labeled sets.
The paper introduces a new FOR framework using Huber and ε-insensitive losses.
problem Handling outliers and sparsity in functional output regression.
method Proposes a flexible FOR framework with infimal convolution losses and computable algorithms.
result Demonstrates efficiency and effectiveness on synthetic and real-world data.
Solves regression problems with CP by converting to classification.
problem Challenges in CP for heteroscedastic, multimodal, or skewed regression outputs.
method Converts regression to classification, uses CP for classification to obtain CP sets for regression.
result Simple approach yields good results on practical problems.
This paper aims to decrease the time complexity of multi-output relevance vector regression from O(VM^3) to O(V^3+M^3), where V is the number of output dimensions, M is the number of basis functions, and V<M. The experimental results demonstrate that the proposed method is more competitive than the existing method, wit…
New method learns output embeddings for structured prediction.
problem Structured prediction with output embeddings.
method Jointly learns output embedding and regression function.
result Structured predictor is a consistent estimator with smaller complexity.
The paper introduces a method for learning nonparametric Volterra kernels using Gaussian processes.
problem Learning nonparametric nonlinear operators from data.
method NVKM model using Volterra series and Gaussian processes for unobserved and observed input functions.
result The NVKM model can perform both single and multiple output regression and system identification.
We are concerned with obtaining well-calibrated output distributions from regression models. Such distributions allow us to quantify the uncertainty that the model has regarding the predicted target value. We introduce the novel concept of distribution calibration, and demonstrate its advantages over the existing defin…
Neural network predicts functional responses from scalar inputs.
problem Regression of functional responses with large scalar predictors and nonlinear relationships.
method Transform functional response to finite dimensions, design feed-forward neural network, modify output via objective functions, apply roughness penalty.
result Proposed neural network outperforms conventional methods in multiple scenarios.
A scalable method for efficient inference in Gaussian process regression networks.
problem Intractable inference in Gaussian process regression networks (GPRN).
method Tensorization of output space, tensor/matrix-normal variational posteriors, joint optimization, and exploiting Kronecker product structure.
result Captures posterior dependencies and improves inference quality for large number of outputs.
We revisit logistic regression and its nonlinear extensions, including multilayer feedforward neural networks, by showing that these classifiers can be viewed as converting input or higher-level features into Dempster-Shafer mass functions and aggregating them by Dempster's rule of combination. The probabilistic output…
The paper develops exact and approximate conformal inference methods for multi-output regression.
problem Uncertainty quantification in multi-output regression predictions.
method Exact derivations and approximations of conformal inference p-values for linear multi-output predictors, and efficient methods for nonlinear predictors.
result Efficient methods for approximating conformal prediction regions for multi-output predictors, both linear and nonlinear.
This paper introduces hyperspherical prototype networks, which unify classification and regression with prototypes on hyperspherical output spaces. For classification, a common approach is to define prototypes as the mean output vector over training examples per class. Here, we propose to use hyperspheres as output spa…
Industrial process control systems try to keep an output variable within a given tolerance around a target value. PID control systems have been widely used in industry to control input variables in order to reach this goal. However, this kind of Transfer Function based approach cannot be extended to complex processes w…
We study the stability properties of nonlinear multi-task regression in reproducing Hilbert spaces with operator-valued kernels. Such kernels, a.k.a. multi-task kernels, are appropriate for learning prob- lems with nonscalar outputs like multi-task learning and structured out- put prediction. We show that multi-task ke…
Operator-Valued Kernels (OVKs) and associated vector-valued Reproducing Kernel Hilbert Spaces provide an elegant way to extend scalar kernel methods when the output space is a Hilbert space. Although primarily used in finite dimension for problems like multi-task regression, the ability of this framework to deal with i…
We propose a family of multivariate Gaussian process models for correlated outputs, based on assuming that the likelihood function takes the generic form of the multivariate exponential family distribution (EFD). We denote this model as a multivariate generalized Gaussian process model, and derive Taylor and Laplace al…
Proposes GPLFR for predicting high-dimensional outputs with few data.
problem Predicting high-dimensional outputs from limited data.
method GPLFR combines Gaussian process and linear-Gaussian decoding for high-dimensional prediction.
result GPLFR outperforms existing methods in predicting high-dimensional outputs.
For many important problems the quantity of interest is an unknown function of the parameters, which is a random vector with known statistics. Since the dependence of the output on this random vector is unknown, the challenge is to identify its statistics, using the minimum number of function evaluations. This problem …
Splat Regression Models use mixtures of bump functions to approximate complex data.
problem Approximating complex data with high interpretability and accuracy.
method Model outputs are mixtures of heterogeneous and anisotropic bump functions (splats) weighted by output vectors. Fitting splat models reduces to optimization over mixing measures using Wasserstein-Fisher-Rao gradient flows.
result Unified theoretical framework for Gaussian Splatting and flexible approach for diverse problems.
To address functional-output regression, we introduce projection learning (PL), a novel dictionary-based approach that learns to predict a function that is expanded on a dictionary while minimizing an empirical risk based on a functional loss. PL makes it possible to use non orthogonal dictionaries and can then be comb…
Proposes SHORE model for efficient MOR with sparsity and scalability.
problem Challenges of interpretability and scalability in MOR with high-dimensional outputs.
method Incorporates sparsity requirements and a two-stage optimization framework for efficient compression.
result Theoretical and empirical validation of the proposed framework's efficiency and accuracy.
This study compares multivariate vs univariate machine learning for multi-output regression.
problem When to use multivariate ensemble techniques over separate univariate models.
method Comparative analysis of different multivariate approaches for multi-output regression.
result Multivariate ensemble techniques outperform separate univariate models in simulations.
This work simplifies Gaussian process regression for multiple outputs.
problem Exponential computational complexity in Gaussian process regression.
method Approximating the covariance kernel using eigenvalues and functions.
result Significant reduction in training and regression complexity.
The paper presents a novel approach to multi-output regression using probabilistic circuits.
problem Capturing correlations between multiple output dimensions in large-scale regression problems.
method Employing a mixture of single-output Gaussian process experts encoded via a probabilistic circuit.
result The method can capture correlations between output dimensions and often outperforms other approaches.
Reduced-rank method improves least-squares regression under output regularity.
problem Least-squares regression with infinite dimensional outputs.
method Reduced-rank method for solving least-squares problems with output regularity assumptions.
result Learning bounds and improved statistical performance compared to full-rank method.
Neural Processes (NPs) (Garnelo et al 2018a;b) approach regression by learning to map a context set of observed input-output pairs to a distribution over regression functions. Each function models the distribution of the output given an input, conditioned on the context. NPs have the benefit of fitting observed data ef…
Parallel gradient boosting speeds up multi-output regression.
problem Efficiently predicting entire conditional distributions.
method Modification of gradient boosting to use a common descent direction.
result Significantly faster and comparable performance to state-of-the-art boosting libraries.
Proposes a deep ordinal regression framework using optimal transport loss and unimodal output probabilities.
problem Lack of unimodal output probabilities in recent ordinal regression models.
method Introduces a deep learning framework based on optimal transport loss and unimodal output distribution, inspired by the Proportional Odds model.
result Demonstrates improved performance and unimodal output probabilities on real-world datasets compared to existing methods.
A new mutual information lower bound for multimodal regression active learning.
problem Lack of effective acquisition functions for multimodal regression active learning.
method Introduces a Two-Index framework for separating epistemic and aleatoric sources of uncertainty, deriving MI-LB as a closed-form approximation.
result MI-LB consistently outperforms baselines on multimodal regression tasks.
We analyze the problem of regression when both input covariates and output responses are functions from a nonparametric function class. Function to function regression (FFR) covers a large range of interesting applications including time-series prediction problems, and also more general tasks like studying a mapping be…
In presence of sparse noise we propose kernel regression for predicting output vectors which are smooth over a given graph. Sparse noise models the training outputs being corrupted either with missing samples or large perturbations. The presence of sparse noise is handled using appropriate use of ℓ1-norm along-wi…
Multi-output Gaussian processes (MOGP) are probability distributions over vector-valued functions, and have been previously used for multi-output regression and for multi-class classification. A less explored facet of the multi-output Gaussian process is that it can be used as a generative model for vector-valued rando…
This paper introduces a multi-output Gaussian process for censored data.
problem Modeling bias in censored data using correlations between multiple outputs.
method Heteroscedastic multi-output Gaussian process with input-dependent noise and variational inference.
result The model better estimates the true process under complex censoring dynamics.
The paper tackles domain generalization using functional regression.
problem Learning a model that generalizes well across different source distributions.
method Functional regression approach to learn a linear operator between marginal and conditional distributions.
result The proposed algorithm achieves finite sample error bounds for the idealized risk.
A new method quantizes output space for multi-target regression.
problem Predicting multiple continuous targets using shared predictors.
method MRQ method that quantizes output space to model dependencies and scale.
result MRQ achieves high scalability and competitive accuracy.
We present a regression technique for data-driven problems based on polynomial chaos expansion (PCE). PCE is a popular technique in the field of uncertainty quantification (UQ), where it is typically used to replace a runnable but expensive computational model subject to random inputs with an inexpensive-to-evaluate po…
Proposes a method for fair regression using RKHS.
problem Ensuring fairness in regression models with multiple sensitive attributes.
method Uses reproducing kernel Hilbert space (RKHS) to construct a functional space that satisfies MP fairness.
result Derives a closed-form solution for fair regression that is efficient and interpretable.
The recently introduced Tsetlin Machine (TM) has provided competitive pattern classification accuracy in several benchmarks, composing patterns with easy-to-interpret conjunctive clauses in propositional logic. In this paper, we go beyond pattern classification by introducing a new type of TMs, namely, the Regression T…
We consider the problem of learning a structured multi-task regression, where the output consists of multiple responses that are related by a graph and the correlated response variables are dependent on the common inputs in a sparse but synergistic manner. Previous methods such as l1/l2-regularized multi-task regressio…
This paper presents a method for solving the supervised learning problem in which the output is highly nonlinear and discontinuous. It is proposed to solve this problem in three stages: (i) cluster the pairs of input-output data points, resulting in a label for each point; (ii) classify the data, where the correspondin…
A new conformal prediction framework for graph-valued outputs using Z-Gromov-Wasserstein distances.
problem Lack of principled uncertainty quantification for graph-valued supervised prediction.
method Proposes a conformal prediction framework using Z-Gromov-Wasserstein distances for graph-valued outputs.
result Provides distribution-free coverage guarantees for graph-valued outputs.
Optimal transport improves multivariate prediction uncertainty quantification.
problem Uncertainty quantification in multivariate learning tasks, especially in regression and classification.
method Introducing a novel Conformal Prediction procedure using optimal transport to handle multivariate score functions and construct flexible prediction regions.
result Ensures finite-sample, distribution-free coverage guarantees for multivariate prediction sets.
Characterizes learnability of multioutput functions in various settings.
problem Learning multioutput function classes in batch and online settings.
method Characterizes learnability based on single-output restrictions.
result Complete characterization of learnability in multioutput classification and regression.