Perceptrons are neuronal devices capable of fully discriminating linearly separable classes. Although straightforward to implement and train, their applicability is usually hindered by non-trivial requirements imposed by real-world classification problems. Therefore, several approaches, such as kernel perceptrons, have…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,051 papers · 148 categories
Trend · papers per month
48 results for “kernel Perceptron”
This study approximates neural network features for modeling relations and attention mechanisms.
problem Approximating neural network features for modeling relations and attention mechanisms.
method Analyzes inner products of multi-layer perceptrons for universal approximation of symmetric and asymmetric relation functions.
result Universal approximation of relation functions and attention mechanisms using inner products of neural networks.
Developed AI models for multi-gas detection in near IR spectrums.
problem Detecting multiple gases in near IR spectrums.
method Used Monte Carlo KNN and multi-resolution CNN, synthesized near IR spectrums, optimized kernel sizes and channels.
result Multi-resolution CNN outperforms other models.
Efficient algorithms for online multiclass linear classification with bandit feedback under linear separability conditions.
problem Efficient online multiclass linear classification with bandit feedback for separable data.
method Design of efficient algorithms based on kernel Perceptron for strong and weak linear separability conditions.
result Near-optimal mistake bounds of $O\left( K/γ^2
ight)$ for strong separability and for weak separability.
Ahpatron improves online kernel learning with tighter mistake bounds.
problem Improving mistake bounds in online kernel learning with budget constraints.
method Introducing Ahpatron, a new model that uses an aggressive updating rule and a budget maintenance mechanism to approximate AVP.
result Ahpatron achieves tighter mistake bounds compared to previous models.
In this paper we propose solving localized multiple kernel learning (LMKL) using LMKL-Net, a feedforward deep neural network. In contrast to previous works, as a learning principle we propose {\em parameterizing} both the gating function for learning kernel combination weights and the multiclass classifier in LMKL usin…
Improved perceptron design mitigates vanishing gradient problem.
problem Vanishing gradient problem in deep multilayer perceptrons.
method Auto-rotating perceptron (ARP) design with dynamic activation region.
result Neural networks with ARP units achieve better learning performance.
Kernel methods and MLPs perform similarly to linear models in high dimensions.
problem Understanding the performance of kernel methods and MLPs in high-dimensional settings.
method Analysis of kernel methods and MLPs in a high-dimensional regime with proportional asymptotics.
result Linear models are optimal in high-dimensional settings when data is generated by kernel models with nonlinear relationships.
Discrete perceptronsmath.PR
Perceptrons have been known for a long time as a promising tool within the neural networks theory. The analytical treatment for a special class of perceptrons started in seminal work of Gardner \cite{Gar88}. Techniques initially employed to characterize perceptrons relied on a statistical mechanics approach. Many of su…
A new kernel simplifies deep learning code and boosts performance.
problem Complex and hard-to-maintain DL codebases.
method Introducing a batch-reduce GEMM kernel for deep learning.
result Deep learning algorithms can be implemented with just 3K lines of code.
Study of infinitely deep but narrow neural networks using NTK theory.
problem Analyzing the role of depth in deep learning with overparameterized networks.
method Infinite-depth limit analysis of MLP and CNN using Neural Tangent Kernel (NTK) theory.
result Established trainability guarantee for infinitely deep but narrow neural networks.
We analyze training dynamics in Gaussian mixture models using a comparison theorem.
problem Analyzing training algorithms with Gaussian mixture data.
method Applying a Gaussian comparison theorem to a specific family of training algorithms.
result Validated dynamic mean-field expressions and provided iterative refinement schemes.
New kernels from ELU and GELU networks reveal non-trivial fixed points.
problem Understanding fixed-point dynamics in deep neural networks with ELU and GELU activations.
method Deriving covariance functions and analyzing fixed-point dynamics of ELU and GELU networks.
result ELU and GELU networks exhibit non-trivial fixed-point dynamics, explaining implicit regularization in overparameterized models.
Paper proposes online algorithms for multiclass classification with partial labels.
problem Classifying data with partial labels.
method Avg Perceptron, Max Perceptron, Avg Pegasos, Max Pegasos algorithms.
result Mistake bounds for Avg Perceptron and regret bound for Avg Pegasos.
ResNets promote smoother interpolations than MLPs, enhancing generalization.
problem Understanding the difference in smoothness between ResNets and MLPs.
method Neural Tangent Kernel (NTK) analysis during gradient descent training.
result ResNet's NTK results in smoother interpolations than MLPs.
We comment on the fact that gradient ascent for logistic regression has a connection with the perceptron learning algorithm. Logistic learning is the "soft" variant of perceptron learning.
m-arcsinh improves SVM and MLP reliability and speed in scikit-learn.
problem Improving SVM and MLP reliability and speed in scikit-learn.
method Modified arcsinh function for kernel and activation in SVM and MLP.
result Competitive classification performance and reliability of SVM and MLP with m-arcsinh.
We study and provide exposition to several phenomena that are related to the perceptron's compression. One theme concerns modifications of the perceptron algorithm that yield better guarantees on the margin of the hyperplane it outputs. These modifications can be useful in training neural networks as well, and we demon…
Kernel-based online learning has often shown state-of-the-art performance for many online learning tasks. It, however, suffers from a major shortcoming, that is, the unbounded number of support vectors, making it non-scalable and unsuitable for applications with large-scale datasets. In this work, we study the problem …
A note proves the binary perceptron's capacity is less than 0.847.
problem Determining the capacity of the binary perceptron.
method Conditional first moment method combined with known results on the spherical perceptron.
result Proves the binary perceptron's capacity is less than 0.847.
New priors for deep neural networks converge to Gaussian processes.
problem Improving the performance and stability of deep neural networks.
method Extending prior distributions to include non-zero means and partially exchangeable priors, leading to a new Gaussian process model.
result The new Gaussian process model avoids pathologies and improves performance on regression problems.
FFN addresses spectral bias in neural value approximation, improving reinforcement learning performance.
problem Spectral bias in neural value approximation, leading to slow convergence and poor performance.
method Proposes Fourier feature networks (FFN) to overcome spectral bias by using a composite neural tangent kernel.
result FFN achieves state-of-the-art performance on challenging continuous control domains with faster convergence and better stability.
Paper analyzes mistake and generalization of MNIC classifiers.
problem Understanding the performance of interpolating classifiers.
method Elementary analyses of MNIC's regret and generalization.
result MNIC generalizes with a rate proportional to the norm of the interpolating solution and inversely proportional to the number of data points.
Adding node feature kernels improves GCN robustness to graph perturbations.
problem GCNs' robustness to graph perturbations is a concern.
method Introduced random GCN and added node feature kernels to message passing.
result Perturbations of the graph structure can significantly degrade GCN performance.
Quantum algorithms improve perceptron learning efficiency.
problem Improving quantum algorithms for perceptron learning.
method Revisiting and correcting a flawed quantum version space perceptron algorithm, proposing quantum-enhanced cutting-plane algorithms.
result Improved complexity bounds for quantum perceptron learning.
Efficient algorithms find solutions in a rare well-connected cluster at low constraint densities.
problem Finding solutions in the symmetric binary perceptron at low density.
method Formal proof of existence of a subdominant connected cluster and application of an efficient multiscale majority algorithm.
result An efficient algorithm can find solutions in a subdominant connected cluster with high probability.
Bayesian Perceptron offers fully Bayesian neural networks without complex computations.
problem Lack of uncertainty quantification in neural networks.
method Bayesian inference framework for perceptron training and predictions in closed-form.
result Analytical expressions for perceptron's output and weight learning provided.
A new algorithm finds a separating hyperplane with fewer updates.
problem Finding a separating hyperplane with minimal updates.
method Optimistic Perceptron algorithm.
result The Optimistic Perceptron finds a separating hyperplane with no more than $ rac{1}{γ}$ updates.
Perceptron is a classic online algorithm for learning a classification function. In this paper, we provide a novel extension of the perceptron algorithm to the learning to rank problem in information retrieval. We consider popular listwise performance measures such as Normalized Discounted Cumulative Gain (NDCG) and Av…
VQC-MLPNet combines quantum and classical elements for scalable quantum machine learning.
problem Challenges in expressivity, trainability, and noise resilience of VQCs.
method Hybrid architecture with a VQC generating weights for a classical MLP during training.
result Improved expressivity, trainability, and robustness compared to standalone quantum or hybrid approaches.
Quantum Perceptron Modelsquant-ph
We demonstrate how quantum computation can provide non-trivial improvements in the computational and statistical complexity of the perceptron model. We develop two quantum algorithms for perceptron learning. The first algorithm exploits quantum information processing to determine a separating hyperplane using a number …
A universal collection of 4 invariants improves neural network accuracy for molecular dynamics.
problem Improving accuracy of neural networks in molecular dynamics.
method Developed a universal collection of 4 smooth scalar invariants on M(3) x M(3) and evaluated their effectiveness in a PONITA neural network architecture.
result Using a universal collection of invariants significantly improves neural network accuracy.
The paper introduces a new method to improve model generalization by routing model copies through permutations.
problem Improving model generalization in machine learning.
method The method replicates a model \(M\) times and rewire the contexts in which local learning messages are computed using permutations.
result The method improves generalization by structured message sharing rather than coupling parameters.
Develops a new theory for neural systems stability and width effects.
problem Stability and finite-width effects in deep neural systems.
method Gauge-covariant stochastic effective field theory using classical commuting fields.
result Predicts the edge of chaos and low-frequency spectral deformation.
Equivalence between algorithmic instability and transition to replica symmetry breaking in perceptron learning systemscond-mat.dis-nn
Binary perceptron's instability linked to replica symmetry breaking.
problem Understanding the relationship between algorithmic instability and replica symmetry breaking in binary perceptron learning.
method Established the connection between algorithmic instability and replica symmetry breaking by comparing the instability condition around the fixed point to the instability for breaking the replica symmetric solution of the free energy function.
result The instability condition around the algorithmic fixed point is identical to the instability for breaking the replica symmetric saddle point solution of the free energy function.
We propose a new type of hidden layer for a multilayer perceptron, and demonstrate that it obtains the best reported performance for an MLP on the MNIST dataset.
DKL-KAN combines deep learning and kernel methods for scalable, expressive models.
problem Combining deep learning's depth with kernel methods' flexibility for scalable models.
method DKL-KAN uses Kolmogorov-Arnold Networks (KAN) to optimize kernel attributes within a Gaussian process framework.
result DKL-KAN outperforms DKL-MLP on datasets with a low number of observations and DKL-MLP on large datasets.
Unified framework for accelerated Perceptron and related problems.
problem Finding optimal linear threshold functions for classification.
method Modern acceleration techniques, specifically optimistic online learning.
result Improved convergence rates for various Perceptron-related problems.
SIGTRON improves classification accuracy for imbalanced datasets.
problem Improving classification accuracy for imbalanced datasets.
method SIGTRON is a new sigmoid function with a convex loss function for imbalanced classification.
result SIGTRON models outperform existing methods in balanced and imbalanced datasets.
Study binary perceptrons' capacity using random duality theory.
problem Characterize the capacity of binary perceptrons with general thresholds.
method Utilized fully lifted random duality theory (fl RDT) to characterize the capacity.
result Characterizations match replica symmetry breaking predictions and uncover the capacity for zero-threshold scenario.
The study improves the perceptron's storage capacity by optimizing variable selection.
problem Distinguishing genuine structure from random correlations in high-dimensional data.
method Replica method from statistical mechanics for optimal variable selection.
result Optimal variable selection can surpass the Cover--Gardner bound for pattern classification.
In this paper, we aim at introducing a new machine learning model, namely reconciled polynomial machine, which can provide a unified representation of existing shallow and deep machine learning models. Reconciled polynomial machine predicts the output by computing the inner product of the feature kernel function and va…
Modified Perceptron handles strategic agents with limited position changes.
problem Learning linear classifiers in the presence of strategic agents that can manipulate their positions.
method Developed a modified Perceptron algorithm with bounded mistakes under various manipulation costs.
result The modified Perceptron achieves bounded mistakes even when manipulation costs are unknown.
Study analyzes learning dynamics in nonlinear perceptrons using stochastic-process approach.
problem Understanding the roles of nonlinearity and input-data distribution in neural network learning.
method Stochastic-process approach to derive flow equations for learning dynamics in nonlinear perceptrons.
result Input-data noise affects learning speed differently under supervised and reinforcement learning.
A new geometric perceptron model improves 3D shape classification.
problem Challenges in geometric tasks involving point clouds using machine learning.
method Introduces multilayer geometric perceptron (MLGP) with geometric neurons.
result MLGP outperforms vanilla MLP in 3D shape classification and noise resistance.
The study analyzes multi-class teacher-student perceptron performance and generalization errors.
problem Analyzing multi-class classification with the teacher-student perceptron.
method Deriving asymptotic expressions for Bayes-optimal and empirical risk minimization (ERM) generalization errors.
result Regularised cross-entropy minimization yields close-to-optimal accuracy for multi-class classification.
Improved mistake bound for group linear separable cases in online multiclass linear classification.
problem Improving mistake bounds for online multiclass linear classification under group linear separable conditions.
method Refined group weak linear separability condition and rational kernel approach.
result Achieved a mistake bound of under group weak linear separable condition.
The paper analyzes phase transitions in transfer learning for perceptrons.
problem Understanding when transfer learning from a source task to a target task is beneficial.
method Theoretical analysis of a pair of related perceptron learning tasks.
result Reveals a phase transition from negative to positive transfer as task similarity changes.