In this paper, we consider parameter recovery for non-overlapping convolutional neural networks (CNNs) with multiple kernels. We show that when the inputs follow Gaussian distribution and the sample size is sufficiently large, the squared loss of such CNNs is in a basin of attraction…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,042 papers · 148 categories
Trend · papers per month
48 results for “multiplicative polynomial kernel”
A new kernel improves Volterra series model selection and prediction.
problem Hard identification of Volterra series from limited data.
method Proposes a novel regularization network using a multiplicative polynomial kernel.
result Better selection of influential monomials improves model prediction.
We analyze kernel matrices in polynomial high-dimensional settings and explain double descent in KRR.
problem Understanding the spectrum of kernel matrices in polynomial high-dimensional settings and its implications for KRR risk.
method Generalized decomposition of kernel matrices into low-rank spike matrix, identity, and Gegenbauer matrix.
result The test error in KRR can exhibit double descent behavior, depending on effective regularization and signal-to-noise ratio.
Compact Random Feature Mapsstat.ML
Kernel approximation using randomized feature maps has recently gained a lot of interest. In this work, we identify that previous approaches for polynomial kernel approximation create maps that are rank deficient, and therefore do not utilize the capacity of the projected feature space effectively. To address this chal…
We study the expressive power of kernel methods and the algorithmic feasibility of multiple kernel learning for a special rich class of kernels. Specifically, we define \emph{Euclidean kernels}, a diverse class that includes most, if not all, families of kernels studied in literature such as polynomial kernels and radi…
New algorithm speeds up polynomial kernel approximations.
problem Efficiently approximating polynomial kernels of high degree.
method Oblivious sketching combined with novel sampling.
result Polynomial factor slowdown removed in running time.
Quillen proved that, if a Hermitian bihomogeneous polynomial is strictly positive on the unit sphere, then repeated multiplication of the standard sesquilinear form to this polynomial eventually results in a sum of Hermitian squares. Catlin-D'Angelo and Varolin deduced this positivstellensatz of Quillen from the eventu…
For supervised and unsupervised learning, positive definite kernels allow to use large and potentially infinite dimensional feature spaces with a computational cost that only depends on the number of observations. This is usually done through the penalization of predictor functions by Euclidean or Hilbertian norms. In …
The nonzero level sets in -dimensional flat affine space of a translationally homogeneous function are improper affine spheres if and only if the Hessian determinant of the function is equal to a nonzero constant multiple of the th power of the function. The exponentials of the characteristic polynomials of certa…
MixCIT tests conditional independence for mixed data types efficiently and reliably.
problem Testing conditional independence for mixed data types, especially when at least one is continuous.
method Graph-based test statistic comparing kernel similarities, debiased local-polynomial approach for continuous variables.
result Unified, efficient, and statistically guaranteed solution across heterogeneous data types.
Unified analysis of kernel-based and locally adaptive bandit optimization methods.
problem Performance of bandit optimization algorithms in RKHS functions.
method Investigates the relationship between kernel regularity and algorithmic performance, characterizing spectral properties of various kernels.
result Unified framework for analyzing kernel-based and locally adaptive bandit algorithms, deriving explicit regret bounds.
The paper recovers missing data entries of high-rank matrices using polynomial polynomials.
problem Recovering missing entries of high-rank matrices with low intrinsic dimension.
method Developed a new polynomial matrix completion method using the kernel trick and relaxation of rank objective.
result Identified complete matrix of minimum intrinsic dimension by minimizing rank in high-dimensional feature space.
Study finds polynomial convergence rate for Farey sequences linked to Riemann hypothesis.
problem Understanding convergence rates of maximum mean discrepancies for Farey sequences.
method Identifying positive-semidefinite kernels and their polynomial convergence rates.
result Polynomial convergence rate of maximum mean discrepancies of Farey sequences is equivalent to the Riemann hypothesis.
A method for interpreting SVMs using polynomial kernels, revealing model complexity.
problem Interpreting SVMs built with truncated orthogonal polynomial kernels.
method Orthogonal Representation Contribution Analysis (ORCA) with normalized Orthogonal Kernel Contribution (OKC) indices.
result The method reveals structural aspects of model complexity not captured by predictive accuracy.
New algorithms solve linear algebra problems in sublinear time.
problem Numerical linear algebra problems, especially with structured matrices.
method Sublinear time algorithms using matrix-vector multiplications.
result Solve problems like least squares regression and low rank approximation in sublinear time.
We consider the problem of simultaneously learning to linearly combine a very large number of kernels and learn a good predictor based on the learnt kernel. When the number of kernels to be combined is very large, multiple kernel learning methods whose computational cost scales linearly in are intractable. We p…
Three-layer networks learn complex hierarchical polynomials of multiple nonlinear features.
problem Understanding how neural networks learn hierarchical features of multiple nonlinear inputs.
method Examine a broad class of functions using three-layer neural networks, showing complete recovery and efficient learning.
result Three-layer neural networks trained via gradient descent can learn hierarchical polynomials of multiple nonlinear features efficiently.
Researchers create integral representations for two-layer ReLU networks with quantitative bounds.
problem Approximating functions with two-layer ReLU networks using explicit integral representations.
method Developed integral representations involving harmonic extension and projection, providing bounds.
result Functions can be approximated with errors independent of dimension or degree, depending on coefficients and distribution.
Learning the kernel functions used in kernel methods has been a vastly explored area in machine learning. It is now widely accepted that to obtain 'good' performance, learning a kernel function is the key challenge. In this work we focus on learning kernel representations for structured regression. We propose use of po…
Kernel discriminant analysis uses nonlinear embeddings to improve classification.
problem Limited effectiveness of linear discriminant analysis in capturing nonlinear features.
method Study of nonlinear embeddings in kernel discriminant analysis using polynomial and Gaussian kernels, solving generalized eigenvalue problems.
result Polynomial and Gaussian discriminants capture class differences through population moments and randomized projections.
Adapts manifold structure for better clustering performance.
problem Lack of consideration for local manifold structure in existing multiple kernel k-means methods.
method Adopts manifold adaptive kernel to integrate local manifold structure of kernels.
result Proposed method outperforms state-of-the-art methods.
Kernel methods can learn hierarchical polynomials efficiently.
problem Learning hierarchical structure from data.
method Iteratively reweighting kernel machines using derivatives.
result Efficient learning of hierarchical polynomials.
MKLpy simplifies Multiple Kernel Learning in Python.
problem Learning optimal kernel functions from data.
method Python-based framework for Multiple Kernel Learning algorithms.
result Maximizes usability and simplifies development of novel solutions.
Paper proves convergence rates for Gaussian kernel ridge regression.
problem Understanding convergence rates for Gaussian kernel ridge regression.
method Establishes polynomial convergence rates for KRR with fixed hyperparameters.
result First polynomial convergence rates for Gaussian kernel ridge regression.
FPC tackles big data challenges with polynomial kernels and ADMM.
problem Efficiently classify massive data with scalability and storage challenges.
method Polynomial feature mapping and ADMM for non-smooth convex optimization.
result FPC significantly reduces computational burden and storage memory without sacrificing generalization ability.
Kernel DRO uses RKHS to optimize under distributional uncertainty.
problem Optimizing under distributional uncertainty with limited knowledge.
method Kernel DRO using RKHS ambiguity sets and duality theory.
result Unified approach to robust and stochastic optimization.
New method uses multiple kernels to improve SVGD performance.
problem Sub-optimal performance of single kernel in SVGD.
method Combines multiple kernels to approximate optimal kernel, using Kernelized Stein Discrepancy (KSD) and constructing Multiple Kernel SVGD (MK-SVGD).
result Consistently matches or outperforms competing methods in experiments.
Deep Gaussian Processes with polynomial kernels can collapse rapidly without proper hyperparameter tuning.
problem The collapse of Deep Gaussian Processes with polynomial kernels without careful hyperparameter tuning.
method Analysis using the Berry-Esseen Theorem and observation of prior behavior.
result The prior of a Deep Gaussian Process collapses rapidly towards zero or places negligible mass on low norm functions without proper hyperparameter tuning.
New method proves Jones Polynomial's connect sum property.
problem Jones Polynomial's behavior under connect sums.
method Trip matrix method for calculating Jones Polynomial.
result Jones Polynomial is multiplicative under connect sums.
Hermite polynomials improve private data generation by reducing feature count.
problem Infinite-dimensional features in kernel mean embedding are impractical for private data generation.
method Replace random features with Hermite polynomial features, leveraging their ordered nature.
result Hermite polynomial features yield a more accurate approximation of kernel mean embedding with fewer features.
RPN unifies various models with a reconciled polynomial network.
problem Unifying diverse models for deep function learning.
method RPN disentangles functions into inner products of expansion and reconciliation functions.
result RPN accurately approximates underlying functions for data distributions.
Graph potentials link to topological QFTs, with computational methods.
problem Defining a topological quantum field theory using graph potentials.
method Using colored trivalent graphs and birational type to define a topological QFT.
result Graph potentials' birational type depends on the graph's homotopy type.
Kernel methods have been widely applied to machine learning and other questions of approximating an unknown function from its finite sample data. To ensure arbitrary accuracy of such approximation, various denseness conditions are imposed on the selected kernel. This note contributes to the study of universal, characte…
Learning Schizophrenia Imaging Genetics Data Via Multiple Kernel Canonical Correlation Analysisq-bio.QM
Kernel and Multiple Kernel Canonical Correlation Analysis (CCA) are employed to classify schizophrenic and healthy patients based on their SNPs, DNA Methylation and fMRI data. Kernel and Multiple Kernel CCA are popular methods for finding nonlinear correlations between high-dimensional datasets. Data was gathered from …
This work introduces a polynomial kernel method for inferring ODE models.
problem Estimating future behavior of dynamical systems from observations.
method Parametric polynomial kernel regression using Backpropagation and Stochastic Gradient Descent.
result Successfully tracks future behavior of chaotic dynamical systems over long time periods.
Study analyzes learnability of RKHS under L∞ norm for kernel methods.
problem Understand performance of kernel methods and random feature models.
method Relate L∞ learnability to kernel spectrum decay and establish sample complexity bounds.
result Conditions for efficient L∞ learning of RKHS identified.
The paper examines the optimality of kernel methods in high-dimensional clustering.
problem Understanding the optimality of kernel methods in high-dimensional data clustering.
method High-dimensional Gaussian clustering, exponential kernel function, kernel k-means, semi-definite relaxation.
result The exponential kernel function optimally recovers clusters in high-dimensional data, matching information-theoretic limits up to a factor of √2.
Polynomial algorithm for multiplication on one-hole torus skein algebra.
problem Complexity of multiplicative structure in skein algebra.
method Provided a polynomial algorithm for one-hole torus.
result Closed form formulas for multiplication of curves with low crossing number.
Study reveals an equivalence principle for the spectrum of random inner-product kernel matrices in polynomial scaling.
problem Understanding the spectrum of random kernel matrices in polynomial scaling regimes.
method Investigates random matrices with nonlinear kernel functions applied to inner products of uniformly distributed vectors.
result The spectrum of the random kernel matrix is asymptotically equivalent to a simpler matrix model through free additive convolution.
SimpleMKKM improves multi-kernel clustering efficiency.
problem Efficient multi-kernel clustering.
method Re-formulated minimization-maximization problem into a smooth minimization, solved with gradient descent.
result Outperforms state-of-the-art multi-kernel clustering alternatives.
Invariant kernels reduce rank and improve generalization across dimensions.
problem Symmetry in high-dimensional data impacts kernel matrix rank and learning algorithms.
method Compute invariant polynomial kernel ranks under various groups acting on data.
result Symmetry decreases kernel rank, making it independent of data dimension.
TKRR improves KRR performance by aligning target functions with kernels.
problem Improving kernel ridge regression performance through target alignment.
method Focuses on truncated kernel ridge regression (TKRR) with an additional spectral truncation parameter.
result TKRR can achieve faster rates than full KRR, reaching parametric rates.
Paper proposes a robust LPR method using similarity kernels.
problem Outliers and high-leverage points affect traditional LPR's accuracy.
method Integrates predictor and response variables in weighting mechanism using a conditional density kernel.
result Lower empirical bias compared to iterative robust LOWESS.
Deep networks can perfectly classify two low-dimensional manifolds on a sphere with large depth and width.
problem Binary classification of two low-dimensional submanifolds on a sphere.
method Analysis of a deep fully-connected neural network trained to separate two submanifolds of the unit sphere.
result Randomly-initialized gradient descent can perfectly classify the two manifolds with high probability when the network depth is large relative to certain geometric and statistical properties of the data.
Extended symmetric union with multiple tangle regions and Alexander polynomial properties.
problem Characterizing knots with multiple tangle regions.
method Generalizing the symmetric union construction to include multiple tangle regions and analyzing the Alexander polynomial.
result The Alexander polynomial of the constructed knot is the product of the Alexander polynomials of the tangles and the square of the partial knot's Alexander polynomial.
New framework establishes positivity of DNTK for PINNs.
problem Establishing positivity of NTK for PINNs with multiple differential operators.
method Proposed Differential Neural Tangent Kernel (DNTK) for PINNs.
result Positivity of infinite width DNTK for various activation functions and differential operators.
Study examines boundedness of oscillating singular integrals on specific Lie groups.
problem Investigating boundedness of oscillating singular integrals on Lie groups of polynomial growth.
method Presented kernel criteria in terms of sub-Riemannian structure and Fourier analysis.
result Extended classical oscillating conditions for boundedness of oscillating convolution operators.
Identifying Outliers using Influence Function of Multiple Kernel Canonical Correlation Analysisstat.ML
Imaging genetic research has essentially focused on discovering unique and co-association effects, but typically ignoring to identify outliers or atypical objects in genetic as well as non-genetics variables. Identifying significant outliers is an essential and challenging issue for imaging genetics and multiple source…