We present generalization bounds for the TS-MKL framework for two stage multiple kernel learning. We also present bounds for sparse kernel learning formulations within the TS-MKL framework.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “two-stage kernels”
Improved learning theory for kernel distribution regression with two-stage sampling.
problem Distribution regression problem and two-stage sampling setting.
method Kernel methods, near-unbiased condition, new error bounds, convergence rates.
result Strictly improved convergence rates for three important classes of kernels.
Localized Kernel Projection Outlyingness: A Two-Stage Approach for Multi-Modal Outlier Detectioncs.LG
Two-stage framework detects multi-modal outliers.
problem Detecting outliers in multi-modal data structures.
method Combines global kernel PCA and local clustering stages.
result Significantly outperforms existing methods on challenging datasets.
CobBO optimizes expensive functions in high dimensions by using a two-stage kernel approach.
problem Bayesian optimization struggles in high dimensions due to computational inefficiency.
method Coordinate backoff Bayesian Optimization with two-stage kernels.
result CobBO finds solutions comparable to or better than other methods in high dimensions.
A two-stage GPR framework with automatic kernel search and subsampling improves prediction accuracy and efficiency.
problem Inaccurate predictions due to misspecified mean and kernel functions in Gaussian Process Regression.
method Two-stage GPR, automatic kernel search, subsampling for hyperparameter initialization.
result Competitive or better performance compared to full dataset training, robust on real-world datasets.
Paper develops a new method for distribution regression with indefinite kernels.
problem Distribution regression with indefinite kernels.
method Coefficient-based regularized distribution regression with two-stage sampling.
result Optimal learning rates derived for the algorithm under mild conditions.
We focus on the distribution regression problem: regressing to a real-valued response from a probability distribution. Although there exist a large number of similarity measures between distributions, very little is known about their generalization performance in specific learning tasks. Learning problems formulated on…
Kernel methods estimate causal effects with a single proxy for deterministic confounders.
problem Estimating causal effects with a single proxy for an unobserved confounder.
method Two kernel-based methods: two-stage regression and maximum moment restriction.
result Both kernel methods can consistently estimate the causal effect.
SGD improves DR by solving two-stage sampling problems.
problem Improving the learning properties of SGD for distribution regression.
method Applying SGD to two-stage sampling problems in distribution regression.
result Theoretical guarantees for SGD's performance in DR, with optimal bounds.
End-to-end kernel learning using generative RFFs for improved performance.
problem Improving kernel learning performance and generalization.
method Develops a generative network via RFFs to implicitly learn the kernel, followed by a linear classifier, jointly trained by ERM.
result Shows superior generalization performance over classical methods in real-world tasks.
Paper introduces robust distribution regression using kernel methods.
problem Distribution regression from probability measures to real-valued responses.
method Introduces a robust loss function and a windowing function for two-stage sampling problems.
result Shows improved learning rates and robustness with the robust distribution regression (RDR) scheme.
Attention-only transformers learn from context via two stages of inference.
problem Learning from corrupted token sequences in minimal transformers.
method Two-stage empirical Bayes interpretation: kernel-weighted posterior mean and particle dynamics.
result Effective denoising without explicit noise schedules, showing posterior-mean recovery under asymptotic conditions.
We present a novel algorithm for non-linear instrumental variable (IV) regression, DualIV, which simplifies traditional two-stage methods via a dual formulation. Inspired by problems in stochastic programming, we show that two-stage procedures for non-linear IV regression can be reformulated as a convex-concave saddle-…
In this paper, we propose a variable selection method for general nonparametric kernel-based estimation. The proposed method consists of two-stage estimation: (1) construct a consistent estimator of the target function, (2) approximate the estimator using a few variables by l1-type penalized estimation. We see that the…
Method estimates treatment effects with continuous values, correcting for confounding.
problem Estimating treatment effects with continuous values, dealing with confounding.
method Two-stage kernel ridge regression: first stage learns response, second stage corrects for distribution shift.
result Optimal learning bounds achieved without estimating treatment density, adapts to unknown overlap and kernel spectral decay.
With the advent of kernel methods, automating the task of specifying a suitable kernel has become increasingly important. In this context, the Multiple Kernel Learning (MKL) problem of finding a combination of pre-specified base kernels that is suitable for the task at hand has received significant attention from resea…
In this paper, we present a novel two-stage metric learning algorithm. We first map each learning instance to a probability distribution by computing its similarities to a set of fixed anchor points. Then, we define the distance in the input data space as the Fisher information distance on the associated statistical ma…
We focus on the distribution regression problem: regressing to vector-valued outputs from probability measures. Many important machine learning and statistical tasks fit into this framework, including multi-instance learning and point estimation problems without analytical solution (such as hyperparameter or entropy es…
Proposes KAR for nonlinear causal discovery using kernel methods.
problem Learning causal relationships in nonlinear settings.
method Kernel anchor regression (KAR) with improved three-stage nonparametric regression.
result KAR outperforms existing methods in nonlinear causal discovery.
Due to the growing ubiquity of unlabeled data, learning with unlabeled data is attracting increasing attention in machine learning. In this paper, we propose a novel semi-supervised kernel learning method which can seamlessly combine manifold structure of unlabeled data and Regularized Least-Squares (RLS) to learn a ne…
DFIV uses deep features for IV regression, achieving optimal rates.
problem Optimal IV regression with deep features for complex target functions.
method Two-stage approach: deep feature learning followed by IV regression.
result DFIV achieves minimax optimal learning rate under certain conditions.
This paper presents a novel multitask multiple kernel learning framework that efficiently learns the kernel weights leveraging the relationship across multiple tasks. The idea is to automatically infer this task relationship in the \textit{RKHS} space corresponding to the given base kernels. The problem is formulated a…
Optimal CATE estimation with structured contrast functions using KRR.
problem Estimating CATEs with complex response functions in RKHS.
method Unified two-stage kernel ridge regression method for structured contrast functions.
result Minimax rates governed by contrast function complexity, enabling adaptation.
Kernel method improves instrumental variable regression rates.
problem Nonparametric instrumental variable regression with weak instruments.
method Kernel-based two-stage least-squares method, strong convergence analysis.
result Minimax optimal rates for instrumental regression under standard assumptions.
DKGM improves image quality by debiasing kernel-based models.
problem Improving image quality in generative models without losing detail.
method Two-stage framework: KDE for density estimation, debiasing algorithm inspired by stochastic approximation.
result DKGM achieves comparable image quality to state-of-the-art models on CIFAR10.
Fast nonparametric conditional independence testing via two-stage regression
problem Fast nonparametric conditional independence testing
method BLITZ (Broad-to-Local Independence Testing via residualiZation)
result Better null calibration than fast kernel, random-feature, and regression-based competitors
Instrumental variable (IV) regression is a strategy for learning causal relationships in observational data. If measurements of input X and output Y are confounded, the causal relationship can nonetheless be identified if an instrumental variable Z is available that influences X directly, but is conditionally independe…
A new method for distribution regression using sliced Wasserstein distance.
problem Learning functions over spaces of probabilities.
method Proposes an OT-based estimator using the Sliced Wasserstein distance.
result Proves universal consistency and excess risk bounds for the proposed estimator.
New method preserves privacy while improving machine learning accuracy.
problem Privacy-preserving machine learning for daily data.
method Compressive Privacy and multi-kernel method.
result Improved utility classification accuracy with privacy preservation.
A new deep neural network tackles nonlinear functional regression with improved dimensionality reduction.
problem Nonlinear functional regression in infinite-dimensional functional data analysis.
method Functional deep neural network with adaptive kernel embedding and projection steps.
result Explicit rates of approximating nonlinear smooth functionals are derived, and the network is shown to be effective in both simulated and real datasets.
Proposes CCME framework for estimating heterogeneous treatment effects.
problem Estimating heterogeneous treatment effects in complex distributions.
method Embeds conditional distributions into RKHS, develops meta-estimators for CCME.
result Establishes finite-sample convergence rates and double robustness for CCME estimators.
New algorithm tackles high-dimensional simulation optimization, converging efficiently.
problem High-dimensional simulation optimization challenges.
method Sparse grid experimental design combined with kernel ridge regression using Brownian field kernel, followed by expected improvement strategy.
result Established upper bounds on convergence rate, demonstrating superior performance in practice.
Enhanced Hopfield model boosts memory retrieval capacity.
problem Memory retrieval in modern Hopfield models with limited capacity.
method Introduces a learnable feature map transforming energy function into kernel space, minimizing separation loss for uniform memory distribution.
result Significant reduction in metastable states, enhancing memory capacity and retrieval accuracy.
New method shows supervised learning can mimic unsupervised learning effectively.
problem The fundamental difference between supervised and unsupervised learning.
method A two-stage procedure where unsupervised model selection is followed by adding outputs without changing parameters.
result Asymptotic out-of-sample risk bounds for various models trained without access to labels.
The paper analyzes how generated data improves adversarial training in high-dimensional regression.
problem Improving adversarial training in high-dimensional regression.
method Theoretical analysis of a two-stage training approach with generated data and pseudo-labels.
result Two-stage adversarial training achieves better performance than ridgeless training in high-dimensional linear regression.
We present a convex approach to probabilistic segmentation and modeling of time series data. Our approach builds upon recent advances in multivariate total variation regularization, and seeks to learn a separate set of parameters for the distribution over the observations at each time point, but with an additional pena…
SARD improves adversarial robustness in two-stage L2D systems.
problem Adversarial attacks can manipulate query allocation in two-stage L2D systems.
method Introduces SARD, a convex learning algorithm with provable guarantees.
result SARD significantly improves robustness under adversarial attacks while maintaining strong clean performance.
Two-stage recommender systems show better performance when components interact rather than operate independently.
problem Two-stage recommender systems are often treated as sums of their parts, ignoring interactions between components.
method Used synthetic and real-world data to demonstrate interactions between ranker and nominators. Derived a generalization lower bound and proposed a Mixture-of-Experts approach to learn optimal item pools.
result Independent nominator training can lead to performance on par with random recommendations, highlighting the importance of interactions.
A new framework integrates classification and regression tasks in multi-task learning.
problem Jointly solving classification and regression tasks in multi-task scenarios.
method Two-Stage Learning-to-Defer (L2D) framework with a unified deferral mechanism.
result Unified deferral mechanism ensures convergence to the Bayes-optimal rejector.
Distributional Random Forests for Complex Survey Designs on Reproducing Kernel Hilbert Spacesstat.ME
SDRF estimates complex survey designs for conditional distributions.
problem Estimating conditional distributions under complex survey designs.
method Survey-calibrated distributional random forest (SDRF) with pseudo-population bootstrap and MMD split criterion.
result Established design consistency and model consistency for survey designs.
CASP selects reliable policies for two-stage recommender systems by considering both value and support.
problem The selection of a generator in two-stage recommender systems affects both the policy value and the data support used to estimate it.
method CASP combines doubly robust value estimation with a support-burden penalty.
result CASP selects lower-burden policies when estimated value and support credibility are in tension.
Bayesian deconditioning improves downscaling of spatial fields.
problem Challenges in refining low-resolution spatial fields with high-resolution information.
method Proposes a Bayesian formulation of deconditioning to solve the inverse problem of conditional expectation.
result Shows substantial improvements in atmospheric field downscaling over existing methods.
Proposes a two-stage method for testing variable interactions with FDR control.
problem Testing pairwise interactions in high-dimensional data with dependence.
method Two-stage testing procedure with FDR control using Cramér type moderate deviation technique.
result The proposed method controls FDR and has comparable or improved statistical power.
To integrate strategic, tactical and operational decisions, the two-stage optimization has been widely used to guide dynamic decision making. In this paper, we study the two-stage stochastic programming for complex systems with unknown response estimated by simulation. We introduce the global-local metamodel assisted t…
Bayesian optimization tackles non-convex, two-stage stochastic problems efficiently.
problem Solving non-convex, two-stage stochastic optimization problems with expensive, black-box evaluations.
method Knowledge-gradient-based acquisition function for joint optimization of first- and second-stage variables.
result Comparable and superior empirical results compared to alternatives.
Two-stage recommender systems struggle with exploration, leading to linear regret.
problem Linear regret in two-stage recommender systems due to exploration issues.
method Proposed a method to synchronize exploration strategies between the ranker and nominators using LinUCB.
result Demonstrated the effectiveness of the proposed algorithm experimentally.
Paper develops a new algorithm for distribution regression with optimal learning rates.
problem Distribution regression with limited second-stage samples.
method Multi-penalty regularization in a reproducing kernel Hilbert space.
result Derives optimal learning rates for distribution regression.
Proxy methods adapt to distribution shifts without explicitly modeling latent confounders.
problem Adapting to distribution shifts under latent variable confounding.
method Proximal causal learning, two-stage kernel estimation.
result Proxy methods outperform other methods in adapting to complex distribution shifts.