Off-the-shelf Gaussian Process (GP) covariance functions encode smoothness assumptions on the structure of the function to be modeled. To model complex and non-differentiable functions, these smoothness assumptions are often too restrictive. One way to alleviate this limitation is to find a different representation of …
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “Feature-space regression”
ParK efficiently solves kernel ridge regression for large datasets.
problem Large-scale kernel ridge regression efficiency and accuracy.
method Partitioning feature space with random projections and iterative optimization.
result Provably maintains statistical accuracy with reduced space and time complexity.
Re-calibrating Photometric Redshift Probability Distributions Using Feature-space Regressionastro-ph.IM
New method calibrates photometric redshift PDFs more accurately.
problem Inaccurate photometric redshift uncertainties lead to systematic errors.
method Local re-calibration using feature-space regression of Probability Integral Transform (PIT) distributions.
result Calibrated PDFs are more accurate at all locations in feature space.
One-class classification (OCC) deals with the classification problem in which the training data has data points belonging only to target class. In this paper, we study a one-class classification algorithm, One-Class Classification by Ensembles of Regression models (OCCER), that uses regression methods to address OCC pr…
Neural networks perform differently when regression is treated as classification.
problem Understanding why neural networks perform better when regression is treated as classification.
method Analyzing two-layer ReLU networks and their feature spaces, focusing on the cross entropy loss vs. square loss.
result The support of the measure induced by the square loss differs from that of the cross entropy loss, indicating optimization difficulties.
Sliced Inverse Regression reduces parameter space for estimating complex financial models.
problem High-dimensional parameter space in stochastic differential equations.
method Sliced Inverse Regression for dimension reduction.
result Reduced computational costs in estimating parameters.
Oblique BART improves tree-based predictions.
problem Axis-aligned decision rules in BART can be suboptimal.
method Developed an oblique version of BART using data-adaptive hyperplane partitions.
result Oblique BART outperformed axis-aligned BART and other tree methods on benchmarks.
Novel bounds for logistic regression coreset construction and feature selection.
problem Efficiently summarize and reduce logistic regression inputs.
method Feature space sketching for logistic regression.
result Tight bounds for coreset construction and feature selection.
Proposes a new method for multi-class classification with well-calibrated predictions.
problem Improving the accuracy and reliability of multi-class classification models.
method Trains data in a latent space induced by an -dimensional simplex, then extends and fits a regression model.
result Demonstrates a well-calibrated classifier with improved prediction and calibration properties.
Although operator-valued kernels have recently received increasing interest in various machine learning and functional data analysis problems such as multi-task learning or functional regression, little attention has been paid to the understanding of their associated feature spaces. In this paper, we explore the potent…
Interpretable Kernelsstat.ML
This paper makes kernels interpretable for wide feature matrices.
problem Loss of interpretability in kernel methods for wide feature matrices.
method Re-express kernel solution as a linear combination of original features with a special metric.
result Interpretable kernel solutions for wide feature matrices are possible.
New theory explains when pre-trained models can improve downstream tasks.
problem Lack of theoretical understanding of task similarity for transfer learning.
method Feature-centric viewpoint, theoretical results on transferability phase diagram.
result Transfer learning outperforms training from scratch when target task is well represented in feature space.
UACQR improves CQR by separating aleatoric and epistemic uncertainties.
problem Ineffective CQR for problems with varying quantile regressor performance.
method Integrates aleatoric and epistemic uncertainties in CQR.
result UACQR provides stronger conditional coverage in simulated and real-world data.
New method learns output embeddings for structured prediction.
problem Structured prediction with output embeddings.
method Jointly learns output embedding and regression function.
result Structured predictor is a consistent estimator with smaller complexity.
Optimal subgroup selectionmath.ST
Optimizes subgroup selection in clinical trials.
problem Identifying regions in feature space where a regression function exceeds a threshold.
method Formulates subgroup selection as constrained optimisation, determining minimax optimal rate for regret.
result Determines the minimax optimal rate for regret in sample size and Type I error probability.
Standard Gaussian Process (GP) regression, a powerful machine learning tool, is computationally expensive when it is applied to large datasets, and potentially inaccurate when data points are sparsely distributed in a high-dimensional feature space. To address these challenges, a new multiscale, sparsified GP algorithm…
Paper develops methods for semi-supervised Fréchet regression.
problem High costs of obtaining non-Euclidean labels.
method Proposes semi-supervised NW Fréchet regression and semi-supervised kNN Fréchet regression.
result Demonstrates superior performance over supervised methods.
Efficient and accurate low-rank approximations of multiple data sources are essential in the era of big data. The scaling of kernel-based learning algorithms to large datasets is limited by the O(n^2) computation and storage complexity of the full kernel matrix, which is required by most of the recent kernel learning a…
We propose a novel adversarial training method in feature space that improves model robustness and computational efficiency.
problem Improving model robustness against adversarial input perturbations with computational efficiency.
method Shift from input to feature-space perturbations, reformulating the adversarial training problem in reproducing kernel Hilbert spaces, enabling exact solution of inner maximization and efficient optimization.
result The feature-perturbed formulation is a relaxation of the original problem and provides a regularized estimator that adapts to noise and function smoothness.
WiGS improves active learning for regression by dynamically selecting informative samples.
problem Reducing labeling costs in regression tasks.
method Formulated as a reinforcement learning problem, WiGS adapts the exploration-investigation balance.
result WiGS outperforms static methods in accuracy and labeling efficiency, especially in irregular data density.
PARC uses piecewise linear predictors for regression and classification.
problem Multivariate regression and classification problems.
method Alternates between ridge and softmax regression, and cluster assignment based on accuracy and separability.
result Converges to a local minimum in a finite number of steps.
Inference in popular nonparametric Bayesian models typically relies on sampling or other approximations. This paper presents a general methodology for constructing novel tractable nonparametric Bayesian methods by applying the kernel trick to inference in a parametric Bayesian model. For example, Gaussian process regre…
Dynamic regressor selection (DRS) systems work by selecting the most competent regressors from an ensemble to estimate the target value of a given test pattern. This competence is usually quantified using the performance of the regressors in local regions of the feature space around the test pattern. However, choosing …
We introduce single-set spectral sparsification as a deterministic sampling based feature selection technique for regularized least squares classification, which is the classification analogue to ridge regression. The method is unsupervised and gives worst-case guarantees of the generalization power of the classificati…
New model selects uncorrelated and discriminative features for unsupervised feature selection.
problem Selecting uncorrelated and discriminative features in high-dimensional data.
method Adaptive graph-based generalized regression model with uncorrelated constraint and -norm regularization.
result The model effectively selects uncorrelated and discriminative features, improving clustering performance.
Fitting statistical models is computationally challenging when the sample size or the dimension of the dataset is huge. An attractive approach for down-scaling the problem size is to first partition the dataset into subsets and then fit using distributed algorithms. The dataset can be partitioned either horizontally (i…
Density-Regression improves deep uncertainty estimation with faster inference.
problem Efficient uncertainty estimation under distribution shifts with modern deep models.
method Leverages density function for fast inference and distance-aware feature space.
result Density-Regression achieves competitive uncertainty estimation performance.
Kernel methods are an extremely popular set of techniques used for many important machine learning and data analysis applications. In addition to having good practical performances, these methods are supported by a well-developed theory. Kernel methods use an implicit mapping of the input data into a high dimensional f…
CPCR mitigates bias in PCR for overparameterized models.
problem Bias in Principal Component Regression (PCR) for overparameterized models.
method Calibrated Principal Component Regression (CPCR) learns a low-variance prior in the PC subspace and calibrates the model in the original feature space.
result CPCR outperforms standard PCR in overparameterized settings, improving prediction across multiple problems.
We propose to solve a label ranking problem as a structured output regression task. We adopt a least square surrogate loss approach that solves a supervised learning problem in two steps: the regression step in a well-chosen feature space and the pre-image step. We use specific feature maps/embeddings for ranking data,…
We consider the problem of estimating a regression function in the common situation where the number of features is small, where interpretability of the model is a high priority, and where simple linear or additive models fail to provide adequate performance. To address this problem, we present GapTV, an approach that …
Study compares non-parametric models for predicting medical insurance reimbursement delays.
problem Estimating the time-lapse between medical insurance reimbursement.
method Comparative study of four non-parametric regression models (KNNs, SVMs, Decision Trees, Random Forests) using R-squared metric.
result Each model's performance varies with training data size, feature space, and hyperparameters.
Correcting for Selection Bias and Missing Response in Regression using Privileged Informationmath.ST
Proposes methods to correct bias and missing data in regression models.
problem Nonignorable selection bias and missing response in regression models.
method Imputation-based and importance weighted regression methods, including repeated regression and doubly robust combination.
result Repeated regression can effectively correct bias and outperforms weighted regression in extrapolation.
Deep Embedding Kernelstat.ML
In this paper, we propose a novel supervised learning method that is called Deep Embedding Kernel (DEK). DEK combines the advantages of deep learning and kernel methods in a unified framework. More specifically, DEK is a learnable kernel represented by a newly designed deep architecture. Compared with pre-defined kerne…
Slow feature analysis (SFA) is an unsupervised-learning algorithm that extracts slowly varying features from a multi-dimensional time series. A supervised extension to SFA for classification and regression is graph-based SFA (GSFA). GSFA is based on the preservation of similarities, which are specified by a graph struc…
The paper presents a new framework for complex Support Vector Regression as well as Support Vector Machines for quaternary classification. The method exploits the notion of widely linear estimation to model the input-out relation for complex-valued data and considers two cases: a) the complex data are split into their …
We study the convergence of the predictive surface of regression trees and forests. To support our analysis we introduce a notion of adaptive concentration for regression trees. This approach breaks tree training into a model selection phase in which we pick the tree splits, followed by a model fitting phase where we f…
In this paper, a multi-layer architecture (in a hierarchical fashion) by stacking various Kernel Ridge Regression (KRR) based Auto-Encoder for one-class classification is proposed and is referred as MKOC. MKOC has many layers of Auto-Encoders to project the input features into new feature space and the last layer was r…
New method creates adaptive prediction intervals for regression models.
problem Need to quantify uncertainty in regression model predictions.
method Regression trees and Random Forests trained on conformity scores.
result Superior scalability and performance compared to baselines.
We address the problem of merging graph and feature-space information while learning a metric from structured data. Existing algorithms tackle the problem in an asymmetric way, by either extracting vectorized summaries of the graph structure or adding hard constraints to feature-space algorithms. Following a different …
We consider the problem of estimating a regression function in the common situation where the number of features is small, where interpretability of the model is a high priority, and where simple linear or additive models fail to provide adequate performance. To address this problem, we present Maximum Variance Total V…
Study characterizes cryospheric spectral feature space using joint PC+t-SNE approach.
problem Characterize cryospheric spectral feature space for remote sensing applications.
method Compare and contrast two approaches for identifying feature space basis vectors via dimensionality reduction (PCA and t-SNE).
result Joint characterization reveals distinct continua and clusters of ice reflectance properties.
Skew-adaptive method improves prediction intervals for regression.
problem Improving prediction intervals for regression models, especially in cases of skewness and varying scales.
method Develops a skew-adaptive extension of split conformal prediction using an asymmetric interval family and gauge approach.
result Preserves marginal validity and adapts to local scale and skewness, with efficiency gains over existing methods.
A Generalized Meta-loss function for regression and classification using privileged informationcs.LG
Learning using privileged information (LUPI) is a powerful heterogenous feature space machine learning framework that allows a machine learning model to learn from highly informative or privileged features which are available during training only to generate test predictions using input space features which are availab…
Gaussian process regression loses locality in high dimensions, affecting molecular energy surface fitting.
problem Loss of locality in high-dimensional Gaussian process regression.
method Analysis of Matern family kernels and multi-zeta basis functions.
result The property of locality disappears in high dimensions, impacting regression quality.
Stacking methods improve the prediction performance of regression models. A simple way to stack base regressions estimators is by combining them linearly, as done by \citet{breiman1996stacked}. Even though this approach is useful from an interpretative perspective, it often does not lead to high predictive power. We pr…
Compressed Gaussian Processstat.ML
Nonparametric regression for massive numbers of samples (n) and features (p) is an increasingly important problem. In big n settings, a common strategy is to partition the feature space, and then separately apply simple models to each partition set. We propose an alternative approach, which avoids such partitioning and…
A nonparametric approach for policy learning for POMDPs is proposed. The approach represents distributions over the states, observations, and actions as embeddings in feature spaces, which are reproducing kernel Hilbert spaces. Distributions over states given the observations are obtained by applying the kernel Bayes' …