A scalable algorithm for GP regression selects relevant covariates efficiently.
problem Scalable variable selection in large GP regression models.
method VGPR algorithm using Vecchia approximation for sparse precision matrix, mini-batch subsampling.
result Improved scalability and accuracy in selecting relevant covariates.
Proposes SHORE model for efficient MOR with sparsity and scalability.
problem Challenges of interpretability and scalability in MOR with high-dimensional outputs.
method Incorporates sparsity requirements and a two-stage optimization framework for efficient compression.
result Theoretical and empirical validation of the proposed framework's efficiency and accuracy.
Derives kernel PCA with Nyström method for scalability.
problem Scalability of kernel PCA.
method Nyström method for kernel PCA.
result Provides scalable alternative to full kernel PCA.
Develops a theoretical framework for scalable Gaussian Process regression methods.
problem Limited scalability of Gaussian Process regression for large datasets.
method Introduces and analyzes Nearest Neighbour Gaussian Process (NNGP) and scalable GPnn methods.
result Derives almost sure pointwise limits for predictive criteria and proves risk minimax rates.
We propose a new scalable algorithm for holistic linear regression building on Bertsimas & King (2016). Specifically, we develop new theory to model significance and multicollinearity as lazy constraints rather than checking the conditions iteratively. The resulting algorithm scales with the number of samples n n n in th…
A scalable method for efficient inference in Gaussian process regression networks.
problem Intractable inference in Gaussian process regression networks (GPRN).
method Tensorization of output space, tensor/matrix-normal variational posteriors, joint optimization, and exploiting Kronecker product structure.
result Captures posterior dependencies and improves inference quality for large number of outputs.
This paper shows using classification instead of regression improves deep RL scalability.
problem Challenges in training value functions for large networks in deep RL.
method Used categorical cross-entropy loss instead of mean squared error regression.
result Significant improvements in performance and scalability across various domains.
A scalable Bayesian linear regression framework for spatial data.
problem Scalable methodologies for analyzing large spatial datasets.
method Conjugate Bayesian linear regression framework.
result Exact sampling from joint posterior distribution without iterative algorithms.
PAN+SR tackles scalable symbolic regression for large p p p datasets.
problem Symbolic regression struggles with large number of input variables and measurement error.
method Combines ab initio nonparametric variable selection with SR to pre-screen and reduce search complexity.
result PAN+SR consistently enhances 19 SR methods' performance on challenging datasets.
Exact and scalable algorithm for Gaussian process regression with Matérn correlations.
problem Efficient Gaussian process regression with Matérn correlations.
method Novel kernel packet theory and sparse representation of covariance matrix.
result Significantly superior to existing alternatives in computational time and predictive accuracy.
BKTR models spatiotemporal data with scalable tensor regression.
problem High computational cost in applying STVC to large-scale spatiotemporal data.
method Summarize STVC coefficients in a tensor, reformulate as low-rank tensor regression, incorporate GP priors for local dependencies.
result BKTR efficiently models large spatiotemporal datasets with reduced parameters and local dependencies.
Scalable kernel methods for large datasets using Fourier representations and NUFFT.
problem Cubic complexity in kernel methods limits their use on large-scale datasets.
method Fourier representation of kernels combined with NUFFT for O(n log n) complexity.
result Achieves minimax convergence rates and processes up to tens of billions of samples.
FaStR improves scalability for time-aware RS with varying coefficients.
problem Limited applicability of structured regression models to large-scale data with categorical effects and many interactions.
method Combines structured additive regression and factorization approaches in a neural network-based model implementation.
result FaStR scales better and performs competitively with other time-aware RS in prediction performance.
A new method quantizes output space for multi-target regression.
problem Predicting multiple continuous targets using shared predictors.
method MRQ method that quantizes output space to model dependencies and scale.
result MRQ achieves high scalability and competitive accuracy.
Securely trains regression models with secret sharing for data collaboration.
problem Balancing data collaboration for technological improvements with security concerns.
method Secret sharing scheme for scalable and efficient secure multiparty training.
result Scalable and efficient protocols for training linear and logistic regression models.
A scalable PyTorch framework for non-crossing quantile regression.
problem Non-crossing quantile regression to avoid impossible negative probability densities.
method CJQR-ALM combining Augmented Lagrangian Method, differentiable pinball loss, and L-BFGS optimization.
result Achieves near-zero crossing rates on large datasets within minutes.
Gaussian processes (GP) are Bayesian non-parametric models that are widely used for probabilistic regression. Unfortunately, it cannot scale well with large data nor perform real-time predictions due to its cubic time cost in the data size. This paper presents two parallel GP regression methods that exploit low-rank co…
Gaussian processes (GP) are Bayesian non-parametric models that are widely used for probabilistic regression. Unfortunately, it cannot scale well with large data nor perform real-time predictions due to its cubic time cost in the data size. This paper presents two parallel GP regression methods that exploit low-rank co…
As a non-parametric Bayesian model which produces informative predictive distribution, Gaussian process (GP) has been widely used in various fields, like regression, classification and optimization. The cubic complexity of standard GP however leads to poor scalability, which poses challenges in the era of big data. Hen…
SoftKI combines SKI and variational methods for scalable GP regression.
problem Scalable Gaussian Process regression on high-dimensional datasets.
method SoftKI approximates kernel via softmax interpolation from a smaller number of learned points.
result SoftKI is competitive with other approximated GP methods for modest data dimensions.
Bayesian optimization (BO) is a model-based approach for gradient-free black-box function optimization. Typically, BO is powered by a Gaussian process (GP), whose algorithmic complexity is cubic in the number of evaluations. Hence, GP-based BO cannot leverage large amounts of past or related function evaluations, for e…
TNP-KR improves scalability of NPs with Transformer blocks and attention mechanisms.
problem Scalability bottleneck in NPs and GPs, especially for large datasets.
method Introduces TNP-KR with KRBlock, kernel-based attention, and two attention mechanisms.
result TNP-KR with DKA outperforms Performer and achieves state-of-the-art results.
The main contribution of the paper is a new approach to subspace clustering that is significantly more computationally efficient and scalable than existing state-of-the-art methods. The central idea is to modify the regression technique in sparse subspace clustering (SSC) by replacing the ℓ 1 \ell_1 ℓ 1 minimization with a g…
Bayesian tensor train kernel machine uses Laplace approximation for scalable GP regression.
problem Scalability limitations of Gaussian process regression.
method Bayesian tensor train kernel machine with Laplace approximation and variational inference.
result VI replaces cross-validation and offers up to 65x faster training.
DSoftKI scales GP regression with full derivative observations.
problem Efficiently fitting and predicting full derivative observations in Gaussian Processes.
method Extends SoftKI by using local temperature vectors for interpolation, enabling encoding of local directional sensitivity.
result DSoftKI achieves accurate predictions and scales to larger datasets with full derivative observations.
We detail distributed algorithms for scalable, secure multiparty linear regression and feature selection at essentially the same speed as plaintext regression. While the core geometric ideas are simple, the recognition of their broad utility when combined is novel. Our scheme opens the door to efficient and secure geno…
FIRAL is a scalable active learning algorithm for multiclass classification.
problem Scalability issues with FIRAL in large datasets.
method Proposed an approximate algorithm with reduced storage and computational complexity.
result Demonstrated strong scalability and accuracy on large datasets.
A new method for efficient Gaussian process regression reduces complexity and improves scalability.
problem Efficient Gaussian process regression for large datasets.
method Learnable coreset-based variational inference for Gaussian processes.
result CVGP reduces the dimensionality of the variational parameter search space to linear complexity.
Heteroscedastic regression considering the varying noises among observations has many applications in the fields like machine learning and statistics. Here we focus on the heteroscedastic Gaussian process (HGP) regression which integrates the latent function and the noise function together in a unified non-parametric B…
A new algorithm speeds up sparse regression for discovering equations from data.
problem Learning governing equations from vast data with unsatisfying descriptions.
method SPRINT: a fast algorithm using bisection and analytic bounds to identify optimal rank-1 modifications.
result A calculation that would take millions of years can be done in a day.
TSRGA scales multivariate linear regression for feature-distributed data.
problem Multivariate linear regression for feature-distributed data with high dimensions and many computing nodes.
method Two-stage relaxed greedy algorithm (TSRGA) for multivariate linear regression.
result TSRGA is highly scalable and can yield low-rank coefficient estimates.
BSA-TNP improves NP scalability and accuracy for spatiotemporal data.
problem Scalability and accuracy trade-off in Neural Processes.
method Introduces KRBlocks, group-invariant attention biases, and BSA for scalable spatiotemporal inference.
result BSA-TNP matches or exceeds accuracy of best models while training faster.
New GPnn method achieves scalable regression with low computational cost.
problem Inefficient Gaussian Process (GP) regression for large datasets.
method GP nearest-neighbour (GPnn) prediction with robustness and limiting behaviour exploration.
result GPnn achieves high MSE accuracy with minimal parameter estimation effort, even in gross misspecification.
A scalable method for accurate inference of low-dimensional parameters in high-dimensional linear regression.
problem Statistical inference for low-dimensional parameters in high-dimensional linear regression models.
method Mean-field variational Bayes approach, focusing on nuisance parameters and conditional distributions.
result Competitive numerical performance and theoretical guarantees for estimation and uncertainty quantification.
Improved Gaussian Process regression using TQFF over RFF and Gaussian QFF.
problem Limited performance of Quadrature Fourier Features (QFF) in approximating highly oscillatory functions.
method Developed Trigonometric Quadrature Fourier Features (TQFF) using a novel non-Gaussian quadrature rule.
result TQFF provides better approximation accuracy and fewer features compared to RFF and Gaussian QFF.
New method scales Gaussian processes with derivatives using variational inference.
problem Scaling Gaussian processes with derivative information for high-dimensional problems.
method Introducing inducing directional derivatives to sparsify derivative information using variational inference.
result Achieves fully scalable Gaussian process regression with derivatives.
New scalable GP approximation using Fourier series decomposition.
problem Scalability and accuracy in Gaussian process approximations.
method Harmonic kernel decomposition (HKD) to decompose kernels orthogonally.
result Significantly outperforms standard variational methods in scalability and accuracy.
Paper tackles DR problem with scalable signature-based approach.
problem Memory and computation cost issues in DR solutions.
method Signature-based features, novel distance approximator.
result Proposes scalable DR solution with reduced estimation uncertainty.
New method for scalable learning of IRT models from large datasets.
problem Efficiently learning latent variables in IRT models from large numbers of examinees and items.
method Leveraging logistic regression and coresets for scalable IRT training.
result Scalable learning of IRT models from large data is achieved.
Spike-and-slab priors are improved for high-dimensional Bayesian regression.
problem Prohibitive computational costs for existing samplers in high-dimensional settings.
method Proposes Scalable Spike-and-Slab ( S 3 S^3 S 3 ) for high-dimensional Bayesian regression. result Improves computational cost to max { n 2 p t , n p } \max\{ n^2 p_t, np \} max { n 2 p t , n p } per iteration, demonstrating significant speed-ups and quality gains. New scalable algorithm for non-negative linear regression with entropy-regularized OT loss.
problem Generalizing task-specific linear models to broader applications.
method Sinkhorn-like scaling iterations for convex penalty and datafit terms.
result Simple multiplicative updates for various penalty and datafit terms.
The paper analyzes convergence rates of Gaussian process approximations for scalable regression.
problem Characterizing convergence rates of Gaussian process approximations for scalable regression.
method Analysis of kernel functions and dataset-size n n n for isotropic kernels like Matérn and squared-exponential. result Upper and lower bounds on predictive MSE and calibration metric convergence rates are derived.
Derives a Matern Gaussian process on hypergraphs for regression and embedding.
problem Regression and embedding of vertices in hypergraphs with uncertainty.
method Derives a Matern Gaussian process on hypergraphs, embeds vertices into latent space, identifies inducing vertices for scalable inference.
result Enables estimation of regression models with hypergraph structure informed correlation and uncertainty.
Improves Gaussian process regression without bias.
problem Bias in Gaussian process regression estimates.
method Adaptive computation selection to minimize bias.
result Guaranteed small bias in log marginal likelihood estimates.
This paper tackles efficient and scalable estimation of a complex model involving stochastic linear combinations of non-linear regressions.
problem Estimating a model involving stochastic linear combinations of non-linear regressions efficiently and scalably.
method The paper provides algorithms for estimating the model under specific assumptions about the variate vector and sample size, using techniques like zero-bias transformation and sub-sampling.
result The paper provides theoretical guarantees for the estimation of the model, showing that the estimation errors are of the order O ( p n ) O(\sqrt{\frac{p}{n}}) O ( n p ) and O ( 1 p + p n ) O(\frac{1}{\sqrt{p}}+\sqrt{\frac{p}{n}}) O ( p 1 + n p ) with high probability. The combination of inducing point methods with stochastic variational inference has enabled approximate Gaussian Process (GP) inference on large datasets. Unfortunately, the resulting predictive distributions often exhibit substantially underestimated uncertainties. Notably, in the regression case the predictive varian…
Improved GP models for fast training and good performance.
problem Training scalable Gaussian process models efficiently.
method Cross-validation and nearest neighbor truncation for scalable GP training.
result Our method offers fast training and excellent predictive performance.
A new method reparameterizes ridge regression for faster, more interpretable results.
problem Challenges in selecting hyperparameter α for ridge regression.
method Fractional Ridge Regression (FRR) reparameterizes RR in terms of the ratio γ.
result FRR solutions vary with different γ, avoiding wasted calculations and manual exploration.