CKA with Gaussian RBF kernels converges linearly as bandwidth increases.
problem Understanding the behavior of CKA with large bandwidth Gaussian kernels.
method Analyzing the convergence of CKA based on Gaussian RBF kernels in the large-bandwidth limit.
result CKA based on Gaussian RBF kernels converges linearly as bandwidth increases.
Study neural communication systems with bandwidth-limited channels.
problem Reliable message transmission despite noisy channels.
method Jointly model compression and error correction with neural networks; introduce prior for missing information; use auxiliary latent variables.
result Joint neural communication systems outperform separate models under expected information loss.
We provide a way to infer about existence of topological circularity in high-dimensional data sets in Rd from its projection in R2 obtained through a fast manifold learning map as a function of the high-dimensional dataset X and a particular choice of a positive real σ known as band…
Study bandwidth-limited training and inference of language models.
problem Training and inference of language models on scattered data with limited bandwidth.
method Analyzed two protocols: Federated Probe-Logit Distillation (FPLD) for training and Federated Conformal RAG (FC-RAG) for inference.
result Explicit high-probability KL-consistency rate and distribution-free marginal-coverage bound for Federated Conformal RAG.
Proposes GRAB-MDM for robust multiview data fusion.
problem Limited theoretical guarantees for multiview fusion methods in noisy high-dimensional data.
method Generalized Robust Adaptive-Bandwidth Multiview Diffusion Maps (GRAB-MDM) with adaptive bandwidth selection.
result Adaptive bandwidths lead to robust recovery of shared intrinsic structure in noisy multiview data.
Reduces data transfer for neural network inference on limited bandwidth.
problem Limited bandwidth during neural network inference.
method Automatic selection of relevant input data parts.
result Significant reduction in data transfer without compromising model quality.
A streaming algorithm estimates quadratic covariation from financial data efficiently.
problem Estimating quadratic covariation from ultra-high-frequency financial data with limited memory.
method Formulated multi-scale, realized kernel, pre-averaging, and modulated realized covariance estimators with fixed bandwidth.
result Fixed bandwidth estimators require higher bandwidth for positive semidefiniteness.
Paper proves MS convergence for radially symmetric kernels with large bandwidths.
problem Proving convergence of mean shift algorithm with radially symmetric kernels.
method Analyzes convergence of mean shift algorithm with radially symmetric, positive definite kernels.
result Guaranteed convergence for sufficiently large bandwidth in any dimension.
The paper analyzes Kernel Density Estimation in high dimensions with varying data and dimensionality.
problem High-dimensional Kernel Density Estimation with growing data and dimensionality.
method Examines the behavior of Kernel Density Estimators in the regime where both data points and dimensionality grow with a fixed ratio.
result Three distinct statistical regimes are identified for Kernel-based density estimates, each with different statistical properties.
Local Gaussian correlation struggles in tails but a new method improves it.
problem Local Gaussian correlation's limitations in tail dependence.
method A new adaptive bandwidth method for LGC, optimizing for local effective sample size.
result Adaptive bandwidths outperform global ones in moderate dependence, but not in strong or weak dependence.
Extends FC-RAG to anytime-valid sequential coverage for language model swarms.
problem Maintain distribution-free coverage for a swarm of weak language models over time.
method Introduces Anytime-FC-RAG, a sequential extension with a summable calibration-deviation budget.
result Achieves time-uniform alarm validity and safety under predictable adaptive control.
A-FADMM improves FL scalability and privacy via wireless channel perturbations and interference.
problem Challenges in model training due to wireless channel randomness and interference.
method Formulated a novel constrained optimization problem and proposed A-FADMM framework.
result Proves convergence and privacy guarantees for A-FADMM under time-varying channels.
Large-scale distributed training requires significant communication bandwidth for gradient exchange that limits the scalability of multi-node training, and requires expensive high-bandwidth network infrastructure. The situation gets even worse with distributed training on mobile devices (federated learning), which suff…
New method for spot volatility estimation with reduced microstructure noise.
problem Estimating spot volatility from noisy high-frequency data.
method Pre-averaging/kernel estimator to handle microstructure noise.
result Optimal bandwidth selection and kernel functions for minimal variance.
The paper proves convergence of graph Laplacian with kNN self-tuned kernels.
problem Theoretical and practical challenges in choosing kernel bandwidth for graph-based analysis.
method Develops and analyzes a new family of kNN self-tuned kernels for graph Laplacian convergence.
result Proves convergence of graph Laplacian to manifold Laplacian for new kNN self-tuned kernels.
Changing kernel bandwidth during training improves kernel regression performance.
problem Improving kernel regression performance with varying model complexity.
method Investigated changing the bandwidth of a translational-invariant kernel during training for kernel regression using gradient descent.
result Kernel regression exhibits double descent behavior with decreasing model complexity (bandwidth).
A new method for faster bandwidth selection in Gaussian kernel ridge regression.
problem Efficiently selecting the bandwidth in Gaussian kernel ridge regression.
method Formulated an approximate Jacobian expression for bandwidth selection, proposing a closed-form heuristic.
result Our method is as accurate as cross-validation and marginal likelihood maximization but up to six orders of magnitude faster.
Typical large-scale recommender systems use deep learning models that are stored on a large amount of DRAM. These models often rely on embeddings, which consume most of the required memory. We present Bandana, a storage system that reduces the DRAM footprint of embeddings, by using Non-volatile Memory (NVM) as the prim…
New method uses reinforcement learning to accurately estimate available network bandwidth.
problem Accurate and fast estimation of available bandwidth in networks with varying cross-traffic.
method Employed reinforcement learning, specifically the ε-greedy algorithm in a multi-armed bandit approach. result Proposed method identifies available bandwidth with high precision and converges under various challenging conditions.
The paper bounds bandwidth and focal radius for manifolds with positive isotropic curvature.
problem Bounding bandwidth and focal radius for manifolds with positive isotropic curvature.
method Using spectral properties of a twisted de Rham-Hodge operator.
result Upper bounds on bandwidth and focal radius are derived for hypersurfaces in PIC manifolds.
Support vector data description (SVDD) is a popular technique for detecting anomalies. The SVDD classifier partitions the whole space into an inlier region, which consists of the region near the training data, and an outlier region, which consists of points away from the training data. The computation of the SVDD class…
Consistency of the kernel density estimator requires that the kernel bandwidth tends to zero as the sample size grows. In this paper we investigate the question of whether consistency is possible when the bandwidth is fixed, if we consider a more general class of weighted KDEs. To answer this question in the affirmativ…
Algorithm selects variables and bandwidths for geographically weighted regression.
problem Estimating variable subsets and bandwidths for geographically weighted regression.
method Mathematical programming-based approach integrating variable selection and bandwidth estimation.
result Proposed algorithm provides stable spatially varying patterns with competitive explanatory power.
Kernel Estimation is one of the most widely used estimation methods in non-parametric Statistics, having a wide-range of applications, including spot volatility estimation of stochastic processes. The selection of bandwidth and kernel function is of great importance, especially for the finite sample settings commonly e…
New method selects optimal bandwidth for price return density estimation, impacting efficient market hypothesis evaluation.
problem Estimating the complexity of price return distributions using kernel density estimation.
method Proposes a new complexity measure to select optimal bandwidth, avoiding overfitting and underfitting.
result Optimal bandwidth selection leads to clearer evaluation of the efficient market hypothesis.
Support vector data description (SVDD) is a popular anomaly detection technique. The SVDD classifier partitions the whole data space into an inlier region, which consists of the region near the training data, and an outlier region, which consists of points away from the training data. The computation of the SVDD classi…
This paper improves bandwidth selectors for SPBNs to enhance their performance.
problem Suboptimal density estimation and reduced predictive performance in SPBNs due to normal rule bandwidth selection.
method Theoretical framework for state-of-the-art bandwidth selectors (cross-validation and plug-in methods) are established and evaluated.
result Cross-validation selectors outperform the normal rule, especially in high sample size scenarios.
We consider recommendation systems that need to operate under wireless bandwidth constraints, measured as number of broadcast transmissions, and demonstrate a (tight for some instances) tradeoff between regret and bandwidth for two scenarios: the case of multi-armed bandit with context, and the case where there is a la…
Deep Gaussian processes can have non-degenerate and non-Gaussian limits.
problem Understanding the behavior of deep Gaussian processes as depth grows.
method Studying the limit of compositional Gaussian processes where each layer is a Gaussian process.
result Identified a sharp bandwidth threshold above which the limit is degenerate, and proved that for bandwidths below this threshold, the limit is a non-degenerate and non-Gaussian distribution.
Study on sensor fusion algorithms under high dimensional noise.
problem Behavior of sensor fusion algorithms under high dimensional noise.
method Analysis of NCCA and AD algorithms using Gaussian kernel.
result Robustness of NCCA and AD to high dimensional noise depends on SNR and bandwidth selection.
This paper proposes a new method for automatically selecting the optimal kernel bandwidth in density estimation.
problem The challenge of selecting the optimal kernel bandwidth in unsupervised density estimation.
method The approach uses a topology-based loss function for automated bandwidth selection.
result Demonstrates the potential of the topology-based approach across different dimensions.
Study shows that ridgeless Gaussian kernel regression overfits even with varying bandwidth or dimensionality.
problem Analyzing overfitting in Gaussian kernel ridgeless regression with varying bandwidth or dimensionality.
method Examined the behavior of minimum norm interpolating solutions for fixed and increasing dimensions under varying bandwidth and sample size.
result Ridgeless solutions are never consistent and can be worse than null predictor with large enough noise, even with varying bandwidth or dimensionality.
Shared workspace improves neural module coordination in deep learning.
problem Pairwise interactions in deep learning models lack global coordination.
method Introduce a shared global workspace with bandwidth limitations among neural modules.
result Capacity limitations encourage specialization and synchronization.
New approach to adaptively select bandwidths in nonparametric regression.
problem Adaptive bandwidth selection in nonparametric regression.
method Inspired by ℓ2-norms of interval projections, introduces a new bandwidth selection procedure. result Obtains non-asymptotic risk bounds for local polynomial regression methods that adapt to local Hölder exponent.
New method tightens federated probe-logit distillation rates under varying bandwidths.
problem Estimating conditional distributions in federated learning with heterogeneous bandwidth constraints.
method Developed a new federated probe-logit distillation (FPLD) method with optimal allocation for varying bandwidths.
result Achieved matching lower and upper bounds for the minimax rate under heterogeneous bandwidths.
Study provides bounds for estimating intrinsic dimension using Gaussian kernels.
problem Estimating intrinsic dimension from data.
method Finite-sample concentration and anti-concentration bounds for Gaussian kernel sums.
result Explicit dependence on sample size, bandwidth, and geometric parameters.
We propose a flexible nonparametric regression method for ultrahigh-dimensional data. As a first step, we propose a fast screening method based on the favored smoothing bandwidth of the marginal local constant regression. Then, an iterative procedure is developed to recover both the important covariates and the regress…
Memory bandwidth bottleneck is a major challenges in processing machine learning (ML) algorithms. In-memory acceleration has potential to address this problem; however, it needs to address two challenges. First, in-memory accelerator should be general enough to support a large set of different ML algorithms. Second, it…
Support Vector Data Description (SVDD) provides a useful approach to construct a description of multivariate data for single-class classification and outlier detection with various practical applications. Gaussian kernel used in SVDD formulation allows flexible data description defined by observations designated as sup…
We explore the performance of several automatic bandwidth selectors, originally designed for density gradient estimation, as data-based procedures for nonparametric, modal clustering. The key tool to obtain a clustering from density gradient estimators is the mean shift algorithm, which allows to obtain a partition not…
Edge filters reduce video data transmission to datacenters.
problem Strain on wide area network infrastructure due to video camera deployments.
method FilterForward system with lightweight edge filters and microclassifiers.
result Reduces bandwidth use by an order of magnitude.
Proposes a new method for clustering compressed data.
problem Limited communication bandwidth and low-power consumption.
method Joint Variational Autoencoders with Bernoulli mixture models (VAB).
result The model can perform clustering in the compressed data domain.
Scalable web crawling using noisy change-indicating signals.
problem Optimizing web page freshness with limited bandwidth and noisy side information.
method Proposes a scalable crawling algorithm that uses noisy side information optimally.
result Achieves constant total rate of crawling without spikes in bandwidth usage.
PCA-Triage optimizes sensor data sampling for IoT networks.
problem Excessive sensor data in IoT networks exceeds available bandwidth.
method PCA-Triage uses streaming incremental PCA loadings to adaptively triage sensor data.
result PCA-Triage achieves high inference performance with minimal bandwidth usage.
The study optimizes Gaussian process approximations for finite-rank models.
problem Posterior behavior of finite-rank approximations differs from parent GP priors.
method Locally supported basis expansions with dependent Gaussian coefficients.
result Finite-rank expansions inherit the same posterior contraction rate as parent GP priors.
Paper uses deep learning to optimize resource allocation in ultra-reliable communications.
problem Optimizing resource allocation for ultra-reliable and low-latency communications.
method Unsupervised deep learning for joint power and bandwidth allocation.
result Deep learning finds an approximated optimal solution with QoS constraints.
Estimates bandwidth for CMC initial data sets.
problem Estimating bandwidth for constant mean curvature (CMC) initial data sets.
method Three independent proofs: stability of null expansion, spacetime harmonic function perturbation, Dirac operator.
result Generalized Gromov's band width estimate to CMC initial data sets.
Estimators of information theoretic measures such as entropy and mutual information are a basic workhorse for many downstream applications in modern data science. State of the art approaches have been either geometric (nearest neighbor (NN) based) or kernel based (with a globally chosen bandwidth). In this paper, we co…