A note proves the binary perceptron's capacity is less than 0.847.
problem Determining the capacity of the binary perceptron.
method Conditional first moment method combined with known results on the spherical perceptron.
result Proves the binary perceptron's capacity is less than 0.847.
Binary perceptron's instability linked to replica symmetry breaking.
problem Understanding the relationship between algorithmic instability and replica symmetry breaking in binary perceptron learning.
method Established the connection between algorithmic instability and replica symmetry breaking by comparing the instability condition around the fixed point to the instability for breaking the replica symmetric solution of the free energy function.
result The instability condition around the algorithmic fixed point is identical to the instability for breaking the replica symmetric saddle point solution of the free energy function.
Efficient algorithms find solutions in a rare well-connected cluster at low constraint densities.
problem Finding solutions in the symmetric binary perceptron at low density.
method Formal proof of existence of a subdominant connected cluster and application of an efficient multiscale majority algorithm.
result An efficient algorithm can find solutions in a subdominant connected cluster with high probability.
Study binary perceptrons' capacity using random duality theory.
problem Characterize the capacity of binary perceptrons with general thresholds.
method Utilized fully lifted random duality theory (fl RDT) to characterize the capacity.
result Characterizations match replica symmetry breaking predictions and uncover the capacity for zero-threshold scenario.
A new algorithm finds a separating hyperplane with fewer updates.
problem Finding a separating hyperplane with minimal updates.
method Optimistic Perceptron algorithm.
result The Optimistic Perceptron finds a separating hyperplane with no more than $ rac{1}{γ}$ updates.
Study potential computational gaps in symmetric binary perceptrons using fl-RDT.
problem Potential statistical-computational gaps in symmetric binary perceptrons.
method Parametric utilization of fully lifted random duality theory (fl-RDT).
result Observation of a computational gap SCG=αc−αa in SBP. The study analyzes multi-class teacher-student perceptron performance and generalization errors.
problem Analyzing multi-class classification with the teacher-student perceptron.
method Deriving asymptotic expressions for Bayes-optimal and empirical risk minimization (ERM) generalization errors.
result Regularised cross-entropy minimization yields close-to-optimal accuracy for multi-class classification.
The goal of Ordinal Regression is to find a rule that ranks items from a given set. Several learning algorithms to solve this prediction problem build an ensemble of binary classifiers. Ranking by Projecting uses interdependent binary perceptrons. These perceptrons share the same direction vector, but use different bia…
EP algorithm for efficient feature selection in binary classification.
problem Sparse feature selection in binary classification.
method Statistical mechanics inspired expectation propagation (EP) on a diluted Bayesian classifier.
result EP is a robust and competitive algorithm in terms of variable selection, estimation accuracy, and computational complexity.
New method finds rare dense clusters in asymmetric binary perceptrons, resolving algorithmic hardness.
problem Resolving algorithmic hardness in asymmetric binary perceptrons.
method Fully lifted random duality theory (fl RDT) and large deviation upgrade (sfl LD RDT).
result Local entropy breaks down for constraint densities in (0.77, 0.78) interval, matching current solver limits.
New insights into binary perceptron reveal phase transitions and algorithmic thresholds.
problem Understanding the statistical-computational gap in binary perceptron models.
method Application of fully lifted random duality theory (fl RDT) to uncover structural changes.
result Numerical estimates of constraint density thresholds align with theoretical predictions.
This paper connects ultrametric overlap gap properties to parametric RDT for symmetric binary perceptrons.
problem Characterizing statistical computational gaps in symmetric binary perceptrons.
method Developed an analytical union-bounding program to rigorously upper-bound constraint densities of ultrametric overlap gap properties.
result Obtained tightest bounds at the first two levels of ultrametric overlap gap properties, closely approaching parametric RDT estimates.
Binary encoding enables neural networks to extrapolate periodic functions.
problem Extrapolating periodic functions without prior knowledge of their form.
method Normalized Base-2 Encoding (NB2E) for continuous numerical values.
result MLPs using NB2E can successfully extrapolate diverse periodic signals.
Paper introduces r-DEP classifier for binary classification tasks.
problem No natural ordering for feature patterns in practical situations.
method Introduces reduced dilation-erosion (r-DEP) classifier using multi-valued mathematical morphology.
result r-DEP classifiers outperform traditional SVCs in balanced accuracy.
The geometrical features of the (non-convex) loss landscape of neural network models are crucial in ensuring successful optimization and, most importantly, the capability to generalize well. While minimizers' flatness consistently correlates with good generalization, there has been little rigorous work in exploring the…
Study analyzes learning dynamics in nonlinear perceptrons using stochastic-process approach.
problem Understanding the roles of nonlinearity and input-data distribution in neural network learning.
method Stochastic-process approach to derive flow equations for learning dynamics in nonlinear perceptrons.
result Input-data noise affects learning speed differently under supervised and reinforcement learning.
Machine learning struggles to predict binary options movements due to randomness.
problem Predicting binary options movements using machine learning.
method Tested multiple machine learning models (RF, LR, GB, kNN) and neural networks (MLP, LSTM) on EUR/USD currency pairs.
result None of the models surpassed the ZeroR baseline accuracy, indicating randomness in binary options.
We propose Very Simple Classifier (VSC) a novel method designed to incorporate the concepts of subsampling and locality in the definition of features to be used as the input of a perceptron. The rationale is that locality theoretically guarantees a bound on the generalization error. Each feature in VSC is a max-margin …
Stochasticity and limited precision of synaptic weights in neural network models are key aspects of both biological and hardware modeling of learning processes. Here we show that a neural network model with stochastic binary weights naturally gives prominence to exponentially rare dense regions of solutions with a numb…
New method initializes sigmoidal MLPs for interpretable shapes.
problem Creating interpretable decision boundaries in neural networks.
method Introducing a geometry-aware initialization for sigmoidal multi-layer perceptrons (MLPs) using tropical geometry.
result Sigmoidal MLPs can have decision boundaries aligned with prescribed shapes at initialization.
We prove lognormal distribution for symmetric perceptron model, solving key conjectures.
problem Understanding the performance of learning algorithms in neural networks.
method Lognormal distribution characterization and small graph conditioning method.
result Established lognormal distribution and several conjectures for the symmetric perceptron model.
Binary and multiclass epilepsy detection methods using EEG features.
problem Epilepsy diagnosis from EEG data.
method Feature extraction from power spectrum, spectrogram, and bispectrogram; eight machine learning algorithms used.
result Random forest and backpropagation algorithms achieved highest accuracy for binary and multiclass classification.
We have applied a little-known data transformation to subsets of the Surveillance, Epidemiology, and End Results (SEER) publically available data of the National Cancer Institute (NCI) to make it suitable input to standard machine learning classifiers. This transformation properly treats the right-censored data in the …
Stochastic binary hidden units in a multi-layer perceptron (MLP) network give at least three potential benefits when compared to deterministic MLP networks. (1) They allow to learn one-to-many type of mappings. (2) They can be used in structured prediction problems, where modeling the internal structure of the output i…
Perceptrons have been known for a long time as a promising tool within the neural networks theory. The analytical treatment for a special class of perceptrons started in seminal work of Gardner \cite{Gar88}. Techniques initially employed to characterize perceptrons relied on a statistical mechanics approach. Many of su…
We introduce a novel Entropy-driven Monte Carlo (EdMC) strategy to efficiently sample solutions of random Constraint Satisfaction Problems (CSPs). First, we extend a recent result that, using a large-deviation analysis, shows that the geometry of the space of solutions of the Binary Perceptron Learning Problem (a proto…
pystacked combines machine learning models for improved predictions.
problem Improving machine learning model performance through stacking.
method Stacked generalization using Python's scikit-learn with various base learners.
result Enhanced predictive models through combining multiple machine learning algorithms.
Paper proposes online algorithms for multiclass classification with partial labels.
problem Classifying data with partial labels.
method Avg Perceptron, Max Perceptron, Avg Pegasos, Max Pegasos algorithms.
result Mistake bounds for Avg Perceptron and regret bound for Avg Pegasos.
SIGTRON improves classification accuracy for imbalanced datasets.
problem Improving classification accuracy for imbalanced datasets.
method SIGTRON is a new sigmoid function with a convex loss function for imbalanced classification.
result SIGTRON models outperform existing methods in balanced and imbalanced datasets.
This paper proposes an improved design of the perceptron unit to mitigate the vanishing gradient problem. This nuisance appears when training deep multilayer perceptron networks with bounded activation functions. The new neuron design, named auto-rotating perceptron (ARP), has a mechanism to ensure that the node always…
We comment on the fact that gradient ascent for logistic regression has a connection with the perceptron learning algorithm. Logistic learning is the "soft" variant of perceptron learning.
We study and provide exposition to several phenomena that are related to the perceptron's compression. One theme concerns modifications of the perceptron algorithm that yield better guarantees on the margin of the hyperplane it outputs. These modifications can be useful in training neural networks as well, and we demon…
Quantum algorithms improve perceptron learning efficiency.
problem Improving quantum algorithms for perceptron learning.
method Revisiting and correcting a flawed quantum version space perceptron algorithm, proposing quantum-enhanced cutting-plane algorithms.
result Improved complexity bounds for quantum perceptron learning.
Bayesian Perceptron offers fully Bayesian neural networks without complex computations.
problem Lack of uncertainty quantification in neural networks.
method Bayesian inference framework for perceptron training and predictions in closed-form.
result Analytical expressions for perceptron's output and weight learning provided.
Perceptron is a classic online algorithm for learning a classification function. In this paper, we provide a novel extension of the perceptron algorithm to the learning to rank problem in information retrieval. We consider popular listwise performance measures such as Normalized Discounted Cumulative Gain (NDCG) and Av…
GMLP learns feature groups for tabular data without known structure.
problem Deep learning for tabular data with unknown feature interactions.
method Group-wise operations and sparse feature grouping matrix learned through temperature annealing softmax.
result GMLP achieves state-of-the-art classification performance on various datasets.
Perceptrons are neuronal devices capable of fully discriminating linearly separable classes. Although straightforward to implement and train, their applicability is usually hindered by non-trivial requirements imposed by real-world classification problems. Therefore, several approaches, such as kernel perceptrons, have…
We demonstrate how quantum computation can provide non-trivial improvements in the computational and statistical complexity of the perceptron model. We develop two quantum algorithms for perceptron learning. The first algorithm exploits quantum information processing to determine a separating hyperplane using a number …
We propose a new type of hidden layer for a multilayer perceptron, and demonstrate that it obtains the best reported performance for an MLP on the MNIST dataset.
Unified framework for accelerated Perceptron and related problems.
problem Finding optimal linear threshold functions for classification.
method Modern acceleration techniques, specifically optimistic online learning.
result Improved convergence rates for various Perceptron-related problems.
We investigate the problem of sequentially predicting the binary labels on the nodes of an arbitrary weighted graph. We show that, under a suitable parametrization of the problem, the optimal number of prediction mistakes can be characterized (up to logarithmic factors) by the cutsize of a random spanning tree of the g…
The study improves the perceptron's storage capacity by optimizing variable selection.
problem Distinguishing genuine structure from random correlations in high-dimensional data.
method Replica method from statistical mechanics for optimal variable selection.
result Optimal variable selection can surpass the Cover--Gardner bound for pattern classification.
Modified Perceptron handles strategic agents with limited position changes.
problem Learning linear classifiers in the presence of strategic agents that can manipulate their positions.
method Developed a modified Perceptron algorithm with bounded mistakes under various manipulation costs.
result The modified Perceptron achieves bounded mistakes even when manipulation costs are unknown.
A new geometric perceptron model improves 3D shape classification.
problem Challenges in geometric tasks involving point clouds using machine learning.
method Introduces multilayer geometric perceptron (MLGP) with geometric neurons.
result MLGP outperforms vanilla MLP in 3D shape classification and noise resistance.
The paper analyzes phase transitions in transfer learning for perceptrons.
problem Understanding when transfer learning from a source task to a target task is beneficial.
method Theoretical analysis of a pair of related perceptron learning tasks.
result Reveals a phase transition from negative to positive transfer as task similarity changes.
Geometric vector perceptrons improve protein structure learning.
problem Learning from protein structure with efficient and natural representations.
method Introducing geometric vector perceptrons to extend dense layers for Euclidean vectors, integrating geometric and relational reasoning.
result Improves model quality assessment and computational protein design over existing methods.
It has been known for a long time that the classical spherical perceptrons can be used as storage memories. Seminal work of Gardner, \cite{Gar88}, started an analytical study of perceptrons storage abilities. Many of the Gardner's predictions obtained through statistical mechanics tools have been rigorously justified. …
The paper analyzes a simple neural network model with algebraic methods.
problem Finding minima of a ridge-regularized mean squared error for ReLU perceptrons.
method Developed a Divide-Enumerate-Merge strategy using computational algebra.
result Identifies both isolated and connected minima of the RR-MSE.