Counterexamples show HSIC feature selection misses critical features.
problem Feature selection using HSIC misses important features.
method Feature selection via HSIC maximization.
result HSIC feature selection can miss critical features.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “HSIC”
New research optimizes HSIC estimation rate for translation-invariant kernels.
problem Optimizing the rate of HSIC estimation for translation-invariant kernels.
method Proved minimax optimal rate of for HSIC estimation.
result Optimality of various HSIC estimators proven.
Efficiently explains model outputs using HSIC, a dependence measure.
problem Efficiently explain model outputs for various architectures.
method HSIC, RKHS, Reproducing Kernel Hilbert Spaces, black-box attribution.
result Up to 8 times faster than previous methods while maintaining fidelity.
New method tests causal association using noise contrastive backdoor adjustment.
problem Testing causal association in complex settings with many confounders.
method Backdoor-HSIC (bd-HSIC) using HSIC for independence testing.
result Calibrated and powerful for binary and continuous treatments with many confounders.
Maximizes image representation dependence for self-supervised learning.
problem Learning meaningful image representations from unlabeled data.
method Maximizes Hilbert-Schmidt Independence Criterion (HSIC) between image transformations and identity.
result Matches state-of-the-art performance on ImageNet and other vision tasks.
We introduce the HSIC (Hilbert-Schmidt independence criterion) bottleneck for training deep neural networks. The HSIC bottleneck is an alternative to the conventional cross-entropy loss and backpropagation that has a number of distinct advantages. It mitigates exploding and vanishing gradients, resulting in the ability…
New method speeds up HSIC for multiple variables.
problem Quadratic computational complexity of HSIC for multiple variables.
method Nyström approximation to HSIC for .
result Consistent Nyström HSIC estimator for .
Proposes HSIC-Lasso for selective inference in non-linear data.
problem Detecting influential features in non-linear and high-dimensional data.
method Model-free HSIC-Lasso based on truncated Gaussians and polyhedral lemma.
result Tight control of type-I error even for small sample sizes.
A statistical test of independence may be constructed using the Hilbert-Schmidt Independence Criterion (HSIC) as a test statistic. The HSIC is defined as the distance between the embedding of the joint distribution, and the embedding of the product of the marginals, in a Reproducing Kernel Hilbert Space (RKHS). It has …
Maximum mean discrepancy (MMD), also called energy distance or N-distance in statistics and Hilbert-Schmidt independence criterion (HSIC), specifically distance covariance in statistics, are among the most popular and successful approaches to quantify the difference and independence of random variables, respectively. T…
Unified framework for optimal kernel tests across MMD, HSIC, and KSD.
problem Optimal testing in kernel-based hypothesis testing frameworks.
method Unified derivation of minimax rates, adaptive kernel selection methods.
result Unified power results across MMD, HSIC, and KSD.
The article introduces practical estimators for kernel discrepancies.
problem Estimating kernel discrepancies accurately and efficiently.
method Presented various estimators for MMD, HSIC, and KSD, including V-statistics, U-statistics, and incomplete U-statistics. Stressed the importance of kernel bandwidth and introduced adaptive estimators.
result Adaptive estimators combining multiple estimators with various kernels address the problem of kernel selection.
Nonlinear Sparse Generalized Canonical Correlation Analysis for Multi-view High-dimensional Datastat.ML
New methods integrate nonlinear, sparse, and multi-view aspects for high-dimensional data analysis.
problem Integrating nonlinear dependence, sparsity, and multi-view data in high-dimensional datasets.
method Proposes HSIC-SGCCA, SA-KGCCA, and TS-KGCCA methods for multi-view high-dimensional data analysis.
result HSIC-SGCCA outperforms competing methods in multi-view variable selection.
Recent works investigated the generalization properties in deep neural networks (DNNs) by studying the Information Bottleneck in DNNs. However, the mea- surement of the mutual information (MI) is often inaccurate due to the density estimation. To address this issue, we propose to measure the dependency instead of MI be…
We describe a novel non-parametric statistical hypothesis test of relative dependence between a source variable and two candidate target variables. Such a test enables us to determine whether one source variable is significantly more dependent on a first target variable or a second. Dependence is measured via the Hilbe…
This work improves independence tests for high-dimensional data.
problem Detecting subtle dependencies between high-dimensional random variables with complex distributions.
method Develops two approaches to learn powerful independence tests using variational mutual information and HSIC.
result Optimized HSIC tests generally outperform other approaches on detecting structured dependence.
A new non parametric approach to the problem of testing the independence of two random process is developed. The test statistic is the Hilbert Schmidt Independence Criterion (HSIC), which was used previously in testing independence for i.i.d pairs of variables. The asymptotic behaviour of HSIC is established when compu…
Method analyzes hyperparameters using HSIC for better neural network performance.
problem Complex hyperparameter spaces in deep learning.
method Goal-oriented sensitivity analysis using HSIC.
result Robust analysis index quantifies hyperparameters' impact.
HSIC-based method explains GNN structures.
problem Interpreting GNNs' complex dynamics.
method Flexible model-agnostic explanation using HSIC.
result Identifies crucial structures in GNNs.
New statistics improve kernel independence testing efficiency.
problem Improving efficiency in kernel independence testing.
method Adapting martingale MMD construction to joint independence problem.
result Two new statistics achieve finite-sample consistency with linear per-test cost.
Kernel dependence measures yield accurate estimates of nonlinear relations between random variables, and they are also endorsed with solid theoretical properties and convergence rates. Besides, the empirical estimates are easy to compute in closed form just involving linear algebra operations. However, they are hampere…
Paper proposes an algorithm to learn DAGs with indirect dependencies.
problem Learning DAGs misses indirect dependencies in local variables.
method Two-phase algorithm using high-order HSIC for local optimization.
result OT algorithm outperforms existing methods in structure estimation.
New method disentangles hidden data structures using HSIC and supervision.
problem Tackles the challenge of interpreting high-dimensional data.
method Supervised Independent Subspace Principal Component Analysis (sisPCA) using HSIC.
result Identifies and separates hidden data structures effectively.
Spectral Non-Convex Optimization for Dimension Reduction with Hilbert-Schmidt Independence Criterionstat.ML
The Hilbert Schmidt Independence Criterion (HSIC) is a kernel dependence measure that has applications in various aspects of machine learning. Conveniently, the objectives of different dimensionality reduction applications using HSIC often reduce to the same optimization problem. However, the nonconvexity of the object…
In this paper, we propose a new kernel-based co-occurrence measure that can be applied to sparse linguistic expressions (e.g., sentences) with a very short learning time, as an alternative to pointwise mutual information (PMI). As well as deriving PMI from mutual information, we derive this new measure from the Hilbert…
A new kernel test avoids permutations for independence testing.
problem Intractable null distributions of kernel statistics.
method Developed xHSIC and xdCov, avoiding permutations.
result New tests have limiting Gaussian distributions under null.
Discusses MultiFIT for multivariate dependence, comparing it to HSIC tests.
problem Comparing Multiscale Fisher's Independence Test (MultiFIT) to HSIC tests for multivariate dependence.
method Compares MultiFIT to HSIC tests, highlighting exact level control and performance limitations.
result Observes performance limitations of MultiFIT in terms of test power.
A crucial challenge in image-based modeling of biomedical data is to identify trends and features that separate normality and pathology. In many cases, the morphology of the imaged object exhibits continuous change as it deviates from normality, and thus a generative model can be trained to model this morphological con…
Proposes a deep network for multi-class classification using spectral training and Gaussian kernel.
problem Multi-class classification with deep networks.
method Spectral training with linear weights and Gaussian kernel activation, constrained on Stiefel Manifold.
result Theoretical guarantee of global optimum and insight into network generalization.
CDSSL improves representation quality by integrating linear and nonlinear dependencies.
problem Scarcity of labeled data and neglect of nonlinear dependencies in SSL.
method CDSSL combines linear correlations and nonlinear dependencies using HSIC in RKHS.
result CDSSL enhances representation quality on diverse benchmarks.
Paper proposes a hybrid loss function for graph self-supervised learning.
problem Improving self-supervised representation learning for graphs.
method Hybrid loss function combining VICReg and HSIC.
result Hybrid loss function outperformed other methods in 4 out of 7 cases.
The paper uses a graph autoencoder to learn unbiased plant-pollinator interaction embeddings.
problem Sampling bias in citizen science data affects ecological network analysis.
method Bipartite graph variational autoencoder with HSIC for fairness.
result The method mitigates sampling bias and provides unbiased embeddings.
Brain imaging data are important in brain sciences yet expensive to obtain, with big volume (i.e., large p) but small sample size (i.e., small n). To tackle this problem, transfer learning is a promising direction that leverages source data to improve performance on related, target data. Most transfer learning methods …
Framework for generating multiple clusterings from multi-view data.
problem Challenges in finding optimal clustering criteria and handling incomplete multi-view data.
method DiMVMC framework that optimizes multiple decoder deep networks to complete data views and generate shared representations.
result DiMVMC outperforms state-of-the-art competitors in generating multiple clusterings with high diversity and quality.
Cheap Permutation Testingmath.ST
Cheap permutation tests speed up distribution testing without sacrificing accuracy.
problem Efficiently testing distribution differences and independence.
method Group datapoints into bins and permute only these bins, using stored sufficient statistics.
result Cheap permutation tests maintain the accuracy and optimality of standard tests but are significantly faster.
Graph structured data has wide applicability in various domains such as physics, chemistry, biology, computer vision, and social networks, to name a few. Recently, graph neural networks (GNN) were shown to be successful in effectively representing graph structured data because of their good performance and generalizati…
A novel double-space tensor-product RKHS framework for hybrid uncertainty sensitivity analysis.
problem Quantifying the influence of hybrid aleatory and epistemic uncertainties on high-dimensional system responses.
method A novel double-space tensor-product RKHS framework for sensitivity analysis under hybrid uncertainty.
result Concurrent double Möbius inversion orthogonally decomposes global dependence measure into pure aleatory effects, pure epistemic effects, and their interaction contributions.
We introduce Information Condensing Active Learning (ICAL), a batch mode model agnostic Active Learning (AL) method targeted at Deep Bayesian Active Learning that focuses on acquiring labels for points which have as much information as possible about the still unacquired points. ICAL uses the Hilbert Schmidt Independen…
The ability of a human being to extrapolate previously gained knowledge to other domains inspired a new family of methods in machine learning called transfer learning. Transfer learning is often based on the assumption that objects in both target and source domains share some common feature and/or data space. In this p…
We propose a novel kernel based post selection inference (PSI) algorithm, which can not only handle non-linearity in data but also structured output such as multi-dimensional and multi-label outputs. Specifically, we develop a PSI algorithm for independence measures, and propose the Hilbert-Schmidt Independence Criteri…
Learning the kernel functions used in kernel methods has been a vastly explored area in machine learning. It is now widely accepted that to obtain 'good' performance, learning a kernel function is the key challenge. In this work we focus on learning kernel representations for structured regression. We propose use of po…
We introduce a general non-parametric independence test between right-censored survival times and covariates, which may be multivariate. Our test statistic has a dual interpretation, first in terms of the supremum of a potentially infinite collection of weight-indexed log-rank tests, with weight functions belonging to …
A new computationally efficient dependence measure, and an adaptive statistical test of independence, are proposed. The dependence measure is the difference between analytic embeddings of the joint distribution and the product of the marginals, evaluated at a finite set of locations (features). These features are chose…
NLARS: Minimum Redundancy Maximum Relevance Feature Selection for Large and High-dimensional Datastat.ML
We propose a feature selection method that finds non-redundant features from a large and high-dimensional data in nonlinear way. Specifically, we propose a nonlinear extension of the non-negative least-angle regression (LARS) called NLARS, where the similarity between input and output is measured through the norm…
Robust tests control type I error under data corruption.
problem Effective hypothesis testing under data corruption.
method General permutation tests using kernel MMD and HSIC metrics.
result Robust tests are minimax optimal and outperform private tests.
Machine learning methods are used to discover complex nonlinear relationships in biological and medical data. However, sophisticated learning models are computationally unfeasible for data with millions of features. Here we introduce the first feature selection method for nonlinear learning problems that can scale up t…
New method optimizes fairness in predictive models for continuous sensitive attributes.
problem Enforcing full statistical independence on continuous sensitive attributes is too restrictive.
method Functional bilevel optimization (FBO) and ITD algorithms.
result Achieves lowest or near-lowest fairness-accuracy regret on synthetic and real datasets.
Proposes a new feature selection method integrating feature relationships.
problem Feature selection in machine learning models.
method Integrates feature-feature and feature-target relationships via penalized mRMR.
result Correctly identifies inactive features, reducing false discoveries.