A note proves the binary perceptron's capacity is less than 0.847.
problem Determining the capacity of the binary perceptron.
method Conditional first moment method combined with known results on the spherical perceptron.
result Proves the binary perceptron's capacity is less than 0.847.
It has been known for a long time that the classical spherical perceptrons can be used as storage memories. Seminal work of Gardner, \cite{Gar88}, started an analytical study of perceptrons storage abilities. Many of the Gardner's predictions obtained through statistical mechanics tools have been rigorously justified. …
Perceptrons have been known for a long time as a promising tool within the neural networks theory. The analytical treatment for a special class of perceptrons started in seminal work of Gardner \cite{Gar88}. Techniques initially employed to characterize perceptrons relied on a statistical mechanics approach. Many of su…
New method lowers spherical perceptron capacity using fully lifted random duality theory.
problem Tackles the negative spherical perceptron capacity, a long-standing open problem.
method Develops fully lifted random duality theory (fl RDT) to characterize capacity.
result Shows remarkable closed-form analytical relations for practical capacity values.
Deep ReLU networks show that 4 layers suffice for unique input recovery.
problem Injectivity capacity of deep ReLU networks.
method Developed a program connecting deep ReLU injectivity to an l-extension of the ℓ0 spherical perceptrons, using random duality theory. result Only 4 layers are needed for unique input recovery, showing expansion saturation effect.
The study calculates the injectivity capacity of ReLU networks using a novel mathematical approach.
problem Determining the injectivity capacity of ReLU networks layers.
method Employing fully lifted random duality theory (fl RDT) to handle the ℓ0 spherical perceptron and implicitly the ReLU layers injectivity. result The lifting mechanism converges remarkably fast with relative corrections not exceeding 0.1%.
One way to recognise an object is to study how the echo has been shaped during the interaction with the target. Wideband sonar allows the study of the energy distribution for a large range of frequencies. The frequency distribution contains information about an object, including its inner structure. This information is…
This paper proposes an improved design of the perceptron unit to mitigate the vanishing gradient problem. This nuisance appears when training deep multilayer perceptron networks with bounded activation functions. The new neuron design, named auto-rotating perceptron (ARP), has a mechanism to ensure that the node always…
In this paper, we propose online algorithms for multiclass classification using partial labels. We propose two variants of Perceptron called Avg Perceptron and Max Perceptron to deal with the partial labeled data. We also propose Avg Pegasos and Max Pegasos, which are extensions of Pegasos algorithm. We also provide mi…
We comment on the fact that gradient ascent for logistic regression has a connection with the perceptron learning algorithm. Logistic learning is the "soft" variant of perceptron learning.
Linear classifiers in product space forms improve scRNA-seq data classification.
problem Linear classification in products of Euclidean, spherical, and hyperbolic spaces.
method Novel formulations of linear classifiers on Riemannian manifolds, proving expressive power, and formalizing perceptron and SVM classifiers.
result Linear classifiers in product space forms have the same expressive power as in Euclidean space of the same dimension.
We study and provide exposition to several phenomena that are related to the perceptron's compression. One theme concerns modifications of the perceptron algorithm that yield better guarantees on the margin of the hyperplane it outputs. These modifications can be useful in training neural networks as well, and we demon…
Quantum algorithms improve perceptron learning efficiency.
problem Improving quantum algorithms for perceptron learning.
method Revisiting and correcting a flawed quantum version space perceptron algorithm, proposing quantum-enhanced cutting-plane algorithms.
result Improved complexity bounds for quantum perceptron learning.
Efficient algorithms find solutions in a rare well-connected cluster at low constraint densities.
problem Finding solutions in the symmetric binary perceptron at low density.
method Formal proof of existence of a subdominant connected cluster and application of an efficient multiscale majority algorithm.
result An efficient algorithm can find solutions in a subdominant connected cluster with high probability.
Injectivity of ReLU networks studied using statistical physics.
problem When can the input of a ReLU neural network be inferred from its output?
method Connection to spherical integral geometry and statistical physics.
result Replica symmetry-breaking theory and Gordon's min--max theorem provide insights into the injectivity threshold.
Bayesian Perceptron offers fully Bayesian neural networks without complex computations.
problem Lack of uncertainty quantification in neural networks.
method Bayesian inference framework for perceptron training and predictions in closed-form.
result Analytical expressions for perceptron's output and weight learning provided.
A new algorithm finds a separating hyperplane with fewer updates.
problem Finding a separating hyperplane with minimal updates.
method Optimistic Perceptron algorithm.
result The Optimistic Perceptron finds a separating hyperplane with no more than $ rac{1}{γ}$ updates.
Perceptron is a classic online algorithm for learning a classification function. In this paper, we provide a novel extension of the perceptron algorithm to the learning to rank problem in information retrieval. We consider popular listwise performance measures such as Normalized Discounted Cumulative Gain (NDCG) and Av…
Perceptrons are neuronal devices capable of fully discriminating linearly separable classes. Although straightforward to implement and train, their applicability is usually hindered by non-trivial requirements imposed by real-world classification problems. Therefore, several approaches, such as kernel perceptrons, have…
We demonstrate how quantum computation can provide non-trivial improvements in the computational and statistical complexity of the perceptron model. We develop two quantum algorithms for perceptron learning. The first algorithm exploits quantum information processing to determine a separating hyperplane using a number …
Binary perceptron's instability linked to replica symmetry breaking.
problem Understanding the relationship between algorithmic instability and replica symmetry breaking in binary perceptron learning.
method Established the connection between algorithmic instability and replica symmetry breaking by comparing the instability condition around the fixed point to the instability for breaking the replica symmetric solution of the free energy function.
result The instability condition around the algorithmic fixed point is identical to the instability for breaking the replica symmetric saddle point solution of the free energy function.
We propose a new type of hidden layer for a multilayer perceptron, and demonstrate that it obtains the best reported performance for an MLP on the MNIST dataset.
Unified framework for accelerated Perceptron and related problems.
problem Finding optimal linear threshold functions for classification.
method Modern acceleration techniques, specifically optimistic online learning.
result Improved convergence rates for various Perceptron-related problems.
Machine learning models solve inverse eigenvalue problems for symmetric potentials and refractive indices.
problem Solving inverse eigenvalue problems for symmetric potentials and refractive indices.
method Supervised regression models (k-Nearest Neighbours, Random Forests, Multi-Layer Perceptron) trained on eigenvalue datasets.
result Machine learning methods can numerically solve inverse eigenvalue problems under appropriate parameter tuning.
Study binary perceptrons' capacity using random duality theory.
problem Characterize the capacity of binary perceptrons with general thresholds.
method Utilized fully lifted random duality theory (fl RDT) to characterize the capacity.
result Characterizations match replica symmetry breaking predictions and uncover the capacity for zero-threshold scenario.
The study improves the perceptron's storage capacity by optimizing variable selection.
problem Distinguishing genuine structure from random correlations in high-dimensional data.
method Replica method from statistical mechanics for optimal variable selection.
result Optimal variable selection can surpass the Cover--Gardner bound for pattern classification.
Modified Perceptron handles strategic agents with limited position changes.
problem Learning linear classifiers in the presence of strategic agents that can manipulate their positions.
method Developed a modified Perceptron algorithm with bounded mistakes under various manipulation costs.
result The modified Perceptron achieves bounded mistakes even when manipulation costs are unknown.
Study analyzes learning dynamics in nonlinear perceptrons using stochastic-process approach.
problem Understanding the roles of nonlinearity and input-data distribution in neural network learning.
method Stochastic-process approach to derive flow equations for learning dynamics in nonlinear perceptrons.
result Input-data noise affects learning speed differently under supervised and reinforcement learning.
A new geometric perceptron model improves 3D shape classification.
problem Challenges in geometric tasks involving point clouds using machine learning.
method Introduces multilayer geometric perceptron (MLGP) with geometric neurons.
result MLGP outperforms vanilla MLP in 3D shape classification and noise resistance.
The study analyzes multi-class teacher-student perceptron performance and generalization errors.
problem Analyzing multi-class classification with the teacher-student perceptron.
method Deriving asymptotic expressions for Bayes-optimal and empirical risk minimization (ERM) generalization errors.
result Regularised cross-entropy minimization yields close-to-optimal accuracy for multi-class classification.
The paper analyzes phase transitions in transfer learning for perceptrons.
problem Understanding when transfer learning from a source task to a target task is beneficial.
method Theoretical analysis of a pair of related perceptron learning tasks.
result Reveals a phase transition from negative to positive transfer as task similarity changes.
Geometric vector perceptrons improve protein structure learning.
problem Learning from protein structure with efficient and natural representations.
method Introducing geometric vector perceptrons to extend dense layers for Euclidean vectors, integrating geometric and relational reasoning.
result Improves model quality assessment and computational protein design over existing methods.
The paper analyzes a simple neural network model with algebraic methods.
problem Finding minima of a ridge-regularized mean squared error for ReLU perceptrons.
method Developed a Divide-Enumerate-Merge strategy using computational algebra.
result Identifies both isolated and connected minima of the RR-MSE.
New bounds on neural network capacity for treelike sign perceptrons using RDT.
problem Determining the capacity of treelike sign perceptrons neural networks.
method Random Duality Theory (RDT) to establish upper bounds.
result Mathematically rigorous bounds on network capacity for any number of neurons.
Study shows perceptrons with random labels perform similarly to Gaussian data.
problem The assumption of Gaussian input data is often seen as a limitation in machine learning.
method Analyzed generalized linear classification (perceptron model) with random labels.
result Minimum training loss is independent of data covariance for high-dimensional input data.
This work concerns estimation of multidimensional nonlinear regression models using multilayer perceptron (MLP). The main problem with such model is that we have to know the covariance matrix of the noise to get optimal estimator. however we show that, if we choose as cost function the logarithm of the determinant of t…
Paper introduces spherical knot mosaics for knot and link invariants.
problem Representing knots on a sphere with tiles.
method Tiling a 2-sphere with 11 knot mosaic tiles to define new invariants.
result New knot invariants derived from spherical mosaic tiling.
PAC-Bayesian bounds for MLPs with cross entropy loss validated.
problem Generalization bounds for MLPs with cross entropy loss.
method Introduced probabilistic explanations and proved PAC-Bayesian bounds using ELBO.
result MLPs with cross entropy loss inherently guarantee PAC-Bayesian generalization bounds.
Study geodesics on spherical polyhedra, estimating their number.
problem Counting simple closed geodesics on spherical polyhedra.
method Examined regular spherical octahedra, cubes, and tetrahedra.
result Estimated the number of simple closed geodesics on spherical polyhedra.
Artificial Neural Networks(ANN) has been phenomenally successful on various pattern recognition tasks. However, the design of neural networks rely heavily on the experience and intuitions of individual developers. In this article, the author introduces a mathematical structure called MLP algebra on the set of all Multi…
The abstract proves spherical surface decompositions with conical singularities.
problem Decomposing surfaces with spherical metrics and conical singularities.
method Geometric triangulations and irreducible components of standard shapes.
result Spherical polygons, including half-spherical concave polygons, can be arbitrarily complicated.
For the n-dimensional spherical pedal curve pedγ,P with respect to an n-dimensional spherical unit speed curve γ and a given point P∈Sn, we define the spherical orthotomic curve of γ relative to the point P, and classify singularities of spherical orthotomic curves.
Breast cancer is the most frequently reported cancer type among the women around the globe and beyond that it has the second highest female fatality rate among all cancer types. Despite all the progresses made in prevention and early intervention, early prognosis and survival prediction rates are still unsatisfactory. …
Study spherical curves with curvature dependent on distance to a great circle.
problem Understanding spherical curves with curvature dependent on distance to a great circle.
method Introducing spherical angular momentum, characterizing known curves, finding new families, and obtaining arc length parametrizations.
result New families of spherical curves with intrinsic equations in elementary or Jacobi elliptic functions.
Learning to rank is a supervised learning problem where the output space is the space of rankings but the supervision space is the space of relevance scores. We make theoretical contributions to the learning to rank problem both in the online and batch settings. First, we propose a perceptron-like algorithm for learnin…
Study calculates eigenvalues and eigenfunctions for spherical triangles and finds fundamental gap behavior.
problem Understanding eigenvalues and gaps in spherical triangles.
method Explicit computation of Dirichlet eigenvalues and eigenfunctions for spherical lunes and triangles.
result Fundamental gap of spherical triangles increases as the angle of the lune decreases.
We utilize Wi-Fi communications from smartphones to predict their mobility mode, i.e. walking, biking and driving. Wi-Fi sensors were deployed at four strategic locations in a closed loop on streets in downtown Toronto. Deep neural network (Multilayer Perceptron) along with three decision tree based classifiers (Decisi…
New findings show fundamental group is not audible in spherical space forms.
problem Isospectral spherical space forms with non-cyclic fundamental groups.
method Revisited and found new examples of spherical space forms.
result Fundamental group is not audible among spherical space forms.