Many real-world regression problems demand a measure of the uncertainty associated with each prediction. Standard decision forests deliver efficient state-of-the-art predictive performance, but high-quality uncertainty estimates are lacking. Gaussian processes (GPs) deliver uncertainty estimates, but scaling GPs to lar…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “large-scale regression”
Proposes GBBHE for efficient large-scale regression.
problem Large-scale regression problems.
method Gradient Boosting with binary histogram partition and ensemble learning.
result Improves computational efficiency and performance on large datasets.
We propose a new two stage algorithm LING for large scale regression problems. LING has the same risk as the well known Ridge Regression under the fixed design setting and can be computed much faster. Our experiments have shown that LING performs well in terms of both prediction accuracy and computational efficiency co…
FaStR improves scalability for time-aware RS with varying coefficients.
problem Limited applicability of structured regression models to large-scale data with categorical effects and many interactions.
method Combines structured additive regression and factorization approaches in a neural network-based model implementation.
result FaStR scales better and performs competitively with other time-aware RS in prediction performance.
ParK efficiently solves kernel ridge regression for large datasets.
problem Large-scale kernel ridge regression efficiency and accuracy.
method Partitioning feature space with random projections and iterative optimization.
result Provably maintains statistical accuracy with reduced space and time complexity.
Improved ridge regression with Frequent Directions for large-scale tasks.
problem Improving performance of ridge regression for large-scale data.
method Combines Frequent Directions with iterative optimization schemes.
result Achieves high accuracy in estimating bias and variance for sketched ridge regression.
This paper tackles hyperparameter tuning for large-scale kernel ridge regression.
problem Hyperparameter tuning is crucial but often left to users, hindering efficiency and usability.
method Proposes a complexity regularization criterion based on a data-dependent penalty for efficient optimization.
result Demonstrates the benefit of the proposed approach through extensive empirical evaluation.
We propose LOCO, an algorithm for large-scale ridge regression which distributes the features across workers on a cluster. Important dependencies between variables are preserved using structured random projections which are cheap to compute and must only be communicated once. We show that LOCO obtains a solution which …
Scalable kernel methods for large datasets using Fourier representations and NUFFT.
problem Cubic complexity in kernel methods limits their use on large-scale datasets.
method Fourier representation of kernels combined with NUFFT for O(n log n) complexity.
result Achieves minimax convergence rates and processes up to tens of billions of samples.
New algorithms solve large-scale convex regression problems.
problem Large-scale convex regression with subgradient regularization.
method Active set type algorithm on dual QP, approximate optimization, randomized augmentation.
result Solves problems with n=10^5 and d=10 in minutes.
Efficiently maps indoor magnetic fields with SKI and D-SKI.
problem Computing large-scale magnetic field maps in indoor environments.
method Structured kernel interpolation (SKI) with derivatives (D-SKI) for Gaussian process regression.
result Achieves better accuracy and faster computation than state-of-the-art methods.
We propose a practical and scalable Gaussian process model for large-scale nonlinear probabilistic regression. Our mixture-of-experts model is conceptually simple and hierarchically recombines computations for an overall approximation of a full Gaussian process. Closed-form and distributed computations allow for effici…
Data-dependent hashing has recently attracted attention due to being able to support efficient retrieval and storage of high-dimensional data such as documents, images, and videos. In this paper, we propose a novel learning-based hashing method called "Supervised Discrete Hashing with Relaxation" (SDHR) based on "Super…
ASkotch solves large-scale KRR faster and better than existing methods.
problem Challenges in scaling full Kernel Ridge Regression (KRR) to large datasets.
method ASkotch: A scalable, accelerated, iterative method for full KRR.
result ASkotch provides better solutions faster than state-of-the-art solvers for full and inducing points KRR.
Parallel integrative learning for large-scale multi-response regression with incomplete outcomesstat.ME
PEER tackles multi-response regression with incomplete outcomes efficiently.
problem Challenges in estimating, predicting, and computing with large-scale multi-response regression and incomplete outcomes.
method PEER converts multi-response regression into parallel univariate-response regressions.
result PEER achieves consistency in estimation, prediction, and variable selection.
In order to scale standard Gaussian process (GP) regression to large-scale datasets, aggregation models employ factorized training process and then combine predictions from distributed experts. The state-of-the-art aggregation models, however, either provide inconsistent predictions or require time-consuming aggregatio…
We propose a new method for input variable selection in nonlinear regression. The method is embedded into a kernel regression machine that can model general nonlinear functions, not being a priori limited to additive models. This is the first kernel-based variable selection method applicable to large datasets. It sides…
Efficiently trains deep Gaussian processes on large datasets.
problem Large-scale data and multi-scale features in function approximation.
method Combines variational learning with MCMC for efficient and accurate training.
result Highly efficient and accurate deep GP training on large-scale data.
Graph neural networks improve network localization accuracy and efficiency.
problem Network localization in large-scale networks.
method Adopted graph neural networks for nonlinear regression.
result GNN outperforms state-of-the-art benchmarks in network localization.
A new method resolves permutation issues in shuffled linear regression for large-scale applications.
problem Estimating latent features through linear transformation with unknown permutations.
method Spectral matching method to align spectral components of measurement and feature covariances.
result Achieves accurate estimates in shuffled LS and LASSO settings with sufficient samples.
We propose a novel method designed for large-scale regression problems, namely the two-stage best-scored random forest (TBRF). "Best-scored" means to select one regression tree with the best empirical performance out of a certain number of purely random regression tree candidates, and "two-stage" means to divide the or…
Ordinal regression (OR) is a special multiclass classification problem where an order relation exists among the labels. Recent years, people share their opinions and sentimental judgments conveniently with social networks and E-Commerce so that plentiful large-scale OR problems arise. However, few studies have focused …
Efficient algorithms speed up adversarial training for linear models.
problem Adversarial training for linear models is computationally expensive.
method Tailored optimization algorithms for regression and classification.
result Significantly faster convergence rates for large-scale problems.
Online BSP-Forest improves space partitioning for large-scale classification and regression.
problem Efficient space partitioning for large-scale classification and regression problems.
method Developed an online BSP-Forest framework that expands space coverage and refines partition structure in real-time.
result Guaranteed universal consistency for both classification and regression problems.
Flexible empirical Bayes for large-scale multiple linear regression.
problem Large-scale multiple linear regression with flexible priors and efficient computation.
method Adaptive shrinkage priors combined with variational approximations for hyperparameter estimation.
result The posterior mean from the empirical Bayes method solves a penalized regression problem.
A new algorithm approximates logistic regression probabilities efficiently.
problem Efficiently approximating probabilities in logistic regression for large datasets.
method Randomized sampling-based algorithm with leverage scores.
result Accurate approximations to estimated probabilities with smaller sample sizes.
New method for scalable inference in large-scale regression models with complex error structures.
problem Challenges in statistical inference for large-scale regression models with dependent errors.
method Generalized Method of Wavelet Moments with Exogenous variables (GMWMX).
result Statistical validity and scalability of GMWMX for linear models with complex error structures.
SRF improves kernel approximation and GP regression performance.
problem Efficient kernel approximation and Bayesian kernel learning in large-scale regression problems.
method Stein variational gradient descent to generate high-quality random features and approximate spectral measure posteriors.
result SRF outperforms traditional approaches in kernel approximation and GP regression.
The paper tackles high-dimensional mixed linear regression with unknown parameters and proposes methods for estimation, confidence intervals, and hypothesis testing.
problem High-dimensional mixed linear regression with unknown parameters and covariance structure.
method Iterative high-dimensional EM algorithm for estimating regression vectors, debiased estimators for individual coordinates, and large-scale multiple testing procedure.
result Asymptotic normality of debiased estimators and FDR control for hypothesis testing.
In this paper, we propose hybrid building/floor classification and floor-level two-dimensional location coordinates regression using a single-input and multi-output (SIMO) deep neural network (DNN) for large-scale indoor localization based on Wi-Fi fingerprinting. The proposed scheme exploits the different nature of th…
We propose a variable decomposition algorithm -greedy block coordinate descent (GBCD)- in order to make dense Gaussian process regression practical for large scale problems. GBCD breaks a large scale optimization into a series of small sub-problems. The challenge in variable decomposition algorithms is the identificati…
In this paper, we revisit the large-scale constrained linear regression problem and propose faster methods based on some recent developments in sketching and optimization. Our algorithms combine (accelerated) mini-batch SGD with a new method called two-step preconditioning to achieve an approximate solution with a time…
New method speeds up learning of complex dynamical systems.
problem Efficiently learning large-scale dynamical systems from finite data.
method Random projections (sketching) to boost kernel-based Koopman operator estimators.
result The proposed estimators maintain accuracy while significantly reducing computation time.
Researchers parallelize neural kernels for large-scale data, achieving state-of-the-art accuracy.
problem Limited scalability of neural kernels on large datasets.
method Massively parallel computation across many GPUs, combined with a distributed, preconditioned conjugate gradients algorithm.
result Achieved state-of-the-art accuracy of 91.2% on CIFAR-5m dataset using neural kernels.
We propose a novel algorithm for large-scale regression problems named histogram transform ensembles (HTE), composed of random rotations, stretchings, and translations. First of all, we investigate the theoretical properties of HTE when the regression function lies in the Hölder space , , $…
This work tackles regression on non-Euclidean spaces, specifically positive-definite matrices with the Bures-Wasserstein metric.
problem Regression on non-Euclidean spaces, specifically positive-definite matrices with the Bures-Wasserstein metric.
method Developed a sufficient condition for the existence of a minimizer of the conditional barycenter problem, characterized the optimization landscape, and developed a projection-free algorithm for approximate computation of first-order stationary points.
result The objective is free of local maxima under the sufficient condition, and the algorithm enables the use of stochastic Riemannian optimization methods for large-scale setups.
Semi-supervised ordinal regression (SOR) problems are ubiquitous in real-world applications, where only a few ordered instances are labeled and massive instances remain unlabeled. Recent researches have shown that directly optimizing concordance index or AUC can impose a better ranking on the data than optimizing t…
A major challenge for building statistical models in the big data era is that the available data volume far exceeds the computational capability. A common approach for solving this problem is to employ a subsampled dataset that can be handled by available computational resources. In this paper, we propose a general sub…
Paper shows noisy labels can improve PLR variable selection.
problem Variable selection in PLR is challenging due to noisy labels.
method Proposes a novel ADMM-based algorithm to fuse noisy labels.
result Fused noisy labels improve PLR performance in estimation and classification.
Subsampling methods have been recently proposed to speed up least squares estimation in large scale settings. However, these algorithms are typically not robust to outliers or corruptions in the observed covariates. The concept of influence that was developed for regression diagnostics can be used to detect such corrup…
This paper reviews recent advances in Gaussian process regression methods.
problem Handling uncertainties and scalability in large-scale systems with sparse data.
method Factorised Gaussian process methods, including hierarchical off-diagonal low-rank approximation and GP with Kronecker structures.
result These methods provide scalable solutions with inherent uncertainty assessment.
Simple methods improve regression transferability estimation.
problem Estimating how well regression models transfer between tasks.
method Two simple, computationally efficient approaches based on negative regularized mean squared error.
result Significantly outperform existing methods in accuracy and efficiency.
Paper proposes PPMM for fast estimation of large-scale OTM.
problem Estimation of large-scale optimal transport maps (OTM) is challenging due to the curse of dimensionality.
method Combines projection pursuit regression and sufficient dimension reduction to adaptively select projection directions.
result PPMM consistently estimates the most informative projection direction and weakly converges to the target OTM.
We outline an inherent weakness of tensor factorization models when latent factors are expressed as a function of side information and propose a novel method to mitigate this weakness. We coin our method \textit{Kernel Fried Tensor}(KFT) and present it as a large scale forecasting tool for high dimensional data. Our re…
In this paper, we study large-scale convex optimization algorithms based on the Newton method applied to regularized generalized self-concordant losses, which include logistic regression and softmax regression. We first prove that our new simple scheme based on a sequence of problems with decreasing regularization para…
Heteroscedastic regression considering the varying noises among observations has many applications in the fields like machine learning and statistics. Here we focus on the heteroscedastic Gaussian process (HGP) regression which integrates the latent function and the noise function together in a unified non-parametric B…
Daily streamflow forecasting through data-driven approaches is traditionally performed using a single machine learning algorithm. Existing applications are mostly restricted to examination of few case studies, not allowing accurate assessment of the predictive performance of the algorithms involved. Here we propose sup…
Local polynomial regression (Fan and Gijbels 1996) is an important class of methods for nonparametric density estimation and regression problems. However, straightforward implementation of local polynomial regression has quadratic time complexity which hinders its applicability in large-scale data analysis. In this pap…