Logistic regression connects to perceptron learning via gradient ascent.
problem No specific problem stated; focuses on connection between algorithms.
method Gradient ascent for logistic regression compared to perceptron learning.
result Gradient ascent for logistic regression is a soft variant of perceptron learning.
Improved perceptron design mitigates vanishing gradient problem.
problem Vanishing gradient problem in deep multilayer perceptrons.
method Auto-rotating perceptron (ARP) design with dynamic activation region.
result Neural networks with ARP units achieve better learning performance.
Quantum algorithms improve perceptron learning efficiency.
problem Improving quantum algorithms for perceptron learning.
method Revisiting and correcting a flawed quantum version space perceptron algorithm, proposing quantum-enhanced cutting-plane algorithms.
result Improved complexity bounds for quantum perceptron learning.
Bayesian Perceptron offers fully Bayesian neural networks without complex computations.
problem Lack of uncertainty quantification in neural networks.
method Bayesian inference framework for perceptron training and predictions in closed-form.
result Analytical expressions for perceptron's output and weight learning provided.
Binary perceptron's instability linked to replica symmetry breaking.
problem Understanding the relationship between algorithmic instability and replica symmetry breaking in binary perceptron learning.
method Established the connection between algorithmic instability and replica symmetry breaking by comparing the instability condition around the fixed point to the instability for breaking the replica symmetric solution of the free energy function.
result The instability condition around the algorithmic fixed point is identical to the instability for breaking the replica symmetric saddle point solution of the free energy function.
Study analyzes learning dynamics in nonlinear perceptrons using stochastic-process approach.
problem Understanding the roles of nonlinearity and input-data distribution in neural network learning.
method Stochastic-process approach to derive flow equations for learning dynamics in nonlinear perceptrons.
result Input-data noise affects learning speed differently under supervised and reinforcement learning.
Perceptron is a classic online algorithm for learning a classification function. In this paper, we provide a novel extension of the perceptron algorithm to the learning to rank problem in information retrieval. We consider popular listwise performance measures such as Normalized Discounted Cumulative Gain (NDCG) and Av…
A new algorithm finds a separating hyperplane with fewer updates.
problem Finding a separating hyperplane with minimal updates.
method Optimistic Perceptron algorithm.
result The Optimistic Perceptron finds a separating hyperplane with no more than $ rac{1}{γ}$ updates.
The paper analyzes phase transitions in transfer learning for perceptrons.
problem Understanding when transfer learning from a source task to a target task is beneficial.
method Theoretical analysis of a pair of related perceptron learning tasks.
result Reveals a phase transition from negative to positive transfer as task similarity changes.
L-Perceptron improves breast cancer diagnosis and survival prediction.
problem Improving early prognosis and survival prediction rates for breast cancer.
method Proposes a novel type of perceptron (L-Perceptron) for better accuracy and sensitivity.
result Achieves 97.42% and 98.73% accuracy and sensitivity in Wisconsin Breast Cancer dataset.
We demonstrate how quantum computation can provide non-trivial improvements in the computational and statistical complexity of the perceptron model. We develop two quantum algorithms for perceptron learning. The first algorithm exploits quantum information processing to determine a separating hyperplane using a number …
Geometric vector perceptrons improve protein structure learning.
problem Learning from protein structure with efficient and natural representations.
method Introducing geometric vector perceptrons to extend dense layers for Euclidean vectors, integrating geometric and relational reasoning.
result Improves model quality assessment and computational protein design over existing methods.
Efficient algorithms find solutions in a rare well-connected cluster at low constraint densities.
problem Finding solutions in the symmetric binary perceptron at low density.
method Formal proof of existence of a subdominant connected cluster and application of an efficient multiscale majority algorithm.
result An efficient algorithm can find solutions in a subdominant connected cluster with high probability.
Unified framework for accelerated Perceptron and related problems.
problem Finding optimal linear threshold functions for classification.
method Modern acceleration techniques, specifically optimistic online learning.
result Improved convergence rates for various Perceptron-related problems.
The perceptron's compression is explored and applied to neural networks.
problem Understanding and improving the perceptron's margin and compression.
method Modifications to the perceptron algorithm and deductions from compression.
result Better guarantees on the margin of the perceptron's hyperplane.
Perceptrons have been known for a long time as a promising tool within the neural networks theory. The analytical treatment for a special class of perceptrons started in seminal work of Gardner \cite{Gar88}. Techniques initially employed to characterize perceptrons relied on a statistical mechanics approach. Many of su…
The study analyzes multi-class teacher-student perceptron performance and generalization errors.
problem Analyzing multi-class classification with the teacher-student perceptron.
method Deriving asymptotic expressions for Bayes-optimal and empirical risk minimization (ERM) generalization errors.
result Regularised cross-entropy minimization yields close-to-optimal accuracy for multi-class classification.
The study improves the perceptron's storage capacity by optimizing variable selection.
problem Distinguishing genuine structure from random correlations in high-dimensional data.
method Replica method from statistical mechanics for optimal variable selection.
result Optimal variable selection can surpass the Cover--Gardner bound for pattern classification.
Modified Perceptron handles strategic agents with limited position changes.
problem Learning linear classifiers in the presence of strategic agents that can manipulate their positions.
method Developed a modified Perceptron algorithm with bounded mistakes under various manipulation costs.
result The modified Perceptron achieves bounded mistakes even when manipulation costs are unknown.
Paper proposes online algorithms for multiclass classification with partial labels.
problem Classifying data with partial labels.
method Avg Perceptron, Max Perceptron, Avg Pegasos, Max Pegasos algorithms.
result Mistake bounds for Avg Perceptron and regret bound for Avg Pegasos.
A new geometric perceptron model improves 3D shape classification.
problem Challenges in geometric tasks involving point clouds using machine learning.
method Introduces multilayer geometric perceptron (MLGP) with geometric neurons.
result MLGP outperforms vanilla MLP in 3D shape classification and noise resistance.
Study shows perceptrons with random labels perform similarly to Gaussian data.
problem The assumption of Gaussian input data is often seen as a limitation in machine learning.
method Analyzed generalized linear classification (perceptron model) with random labels.
result Minimum training loss is independent of data covariance for high-dimensional input data.
Learning to rank is a supervised learning problem where the output space is the space of rankings but the supervision space is the space of relevance scores. We make theoretical contributions to the learning to rank problem both in the online and batch settings. First, we propose a perceptron-like algorithm for learnin…
A new model of learning corrects for chance to improve learning outcomes.
problem The importance of chance-corrected measures in learning.
method Developed two models: Informatron and AdaBook, based on empirical psychological results.
result Chance correction facilitates learning, as shown by computational results.
A note proves the binary perceptron's capacity is less than 0.847.
problem Determining the capacity of the binary perceptron.
method Conditional first moment method combined with known results on the spherical perceptron.
result Proves the binary perceptron's capacity is less than 0.847.
Introduces matrix MLP for learning symmetric positive definite matrices.
problem Learning structured parameters like symmetric positive definite matrices.
method Develops matrix multilayer perceptron (matrix MLP) for structured parameter learning.
result Extends variational autoencoder (VAE) for dense covariance matrices.
We present a neural-network valuation of financial derivatives in the case of fat-tailed underlying asset returns. A two-layer perceptron is trained on simulated prices taking into account the well-known effect of volatility smile. The prices of the underlier are generated using fractional calculus algorithms, and opti…
Perceptrons are neuronal devices capable of fully discriminating linearly separable classes. Although straightforward to implement and train, their applicability is usually hindered by non-trivial requirements imposed by real-world classification problems. Therefore, several approaches, such as kernel perceptrons, have…
PCGs encompass a broader range of neural networks.
problem Understanding the broader scope of neural network models.
method Proving PCGs as a superset of feedforward neural networks.
result PCGs represent a wider class of neural network models.
Paper explores offline and online image recognition using neural networks.
problem Challenges in evolving image recognition through different settings.
method Used Convolutional Neural Networks and Multi-layer Perceptrons.
result Encouraging preliminary results in offline and online image classification.
We propose a new type of hidden layer for a multilayer perceptron, and demonstrate that it obtains the best reported performance for an MLP on the MNIST dataset.
EP algorithm for efficient feature selection in binary classification.
problem Sparse feature selection in binary classification.
method Statistical mechanics inspired expectation propagation (EP) on a diluted Bayesian classifier.
result EP is a robust and competitive algorithm in terms of variable selection, estimation accuracy, and computational complexity.
Proposes using MLP for predicting optimal penalty in changepoint detection.
problem Predicting optimal penalty for changepoints in sequences.
method Uses a multilayer perceptron (MLP) with ReLU activation function to predict penalty.
result Improves accuracy and F1 score compared to existing models.
Study binary perceptrons' capacity using random duality theory.
problem Characterize the capacity of binary perceptrons with general thresholds.
method Utilized fully lifted random duality theory (fl RDT) to characterize the capacity.
result Characterizations match replica symmetry breaking predictions and uncover the capacity for zero-threshold scenario.
A new scoring function improves ordinal regression by combining binary predictions.
problem Improving ordinal regression prediction accuracy.
method Introduces a cumulative sum scoring function to combine binary predictions.
result Simplified formulation and two online learning algorithms converge under rank separability condition.
We prove lognormal distribution for symmetric perceptron model, solving key conjectures.
problem Understanding the performance of learning algorithms in neural networks.
method Lognormal distribution characterization and small graph conditioning method.
result Established lognormal distribution and several conjectures for the symmetric perceptron model.
New scaling laws explain deep learning performance growth.
problem Understanding neural network performance growth.
method Analyzed entire training dynamics of various architectures.
result Identified two dynamical scaling laws.
Improved stability in DQN using neural networks with modifications.
problem Stability issues in DQN with neural networks.
method Simple modifications to neural network structure.
result Improved stability in DQN learning.
Article proposes a profitable intraday trading strategy for Chinese stocks.
problem Intraday trading opportunities in Chinese stock market.
method Markowitz optimization and Multilayer Perceptron (MLP) for stock price prediction.
result Validation of Markowitz portfolio optimization and MLP for intraday stock price prediction.
Noise causes learning plateaus in neural networks.
problem Plateau phenomena in online learning due to vanishing gradients.
method Analysis of stochastic gradient descent in multi-layer perceptrons.
result Noise induces synchronisation leading to strong plateaus.
Deep neural networks predict walking, biking, and driving from Wi-Fi signals.
problem Predicting human mobility modes using Wi-Fi signals.
method Deployed Wi-Fi sensors at four locations, developed and tested multiple classifiers (MLP, Decision Tree, Bagged Decision Tree, Random Forest).
result Multilayer Perceptron achieved 86.52% correct predictions of mobility modes.
The paper addresses the mismatch between theoretical machine learning and practical quantization.
problem The mismatch between theoretical machine learning and practical quantization.
method Proposes a framework for reasoning about learning under arbitrary quantizations and proves the convergence of quantization-aware algorithms.
result Proves the convergence of quantization-aware versions of the Perceptron and Frank-Wolfe algorithms.
It has been known for a long time that the classical spherical perceptrons can be used as storage memories. Seminal work of Gardner, \cite{Gar88}, started an analytical study of perceptrons storage abilities. Many of the Gardner's predictions obtained through statistical mechanics tools have been rigorously justified. …
Deep learning outperforms traditional methods in estimating OU process parameters.
problem Parameter estimation of the Ornstein-Uhlenbeck process is challenging.
method Used a multi-layer perceptron to estimate OU process parameters compared to traditional methods like Kalman filter and maximum likelihood estimation.
result Deep learning method outperforms traditional methods in parameter estimation of the OU process.
New view: Deep GCNs learn to anti-oversmooth during training.
problem Performance drop in deep GCNs due to oversmoothing.
method Interpreted GCN as MLP + graph regularization, analyzed training process.
result Deep GCNs learn to anti-oversmooth during training, not over-smooth.
New bounds on neural network capacity for treelike sign perceptrons using RDT.
problem Determining the capacity of treelike sign perceptrons neural networks.
method Random Duality Theory (RDT) to establish upper bounds.
result Mathematically rigorous bounds on network capacity for any number of neurons.
The paper analyzes a simple neural network model with algebraic methods.
problem Finding minima of a ridge-regularized mean squared error for ReLU perceptrons.
method Developed a Divide-Enumerate-Merge strategy using computational algebra.
result Identifies both isolated and connected minima of the RR-MSE.
LMKL-Net uses deep neural networks to solve localized multiple kernel learning faster and more efficiently.
problem Localized multiple kernel learning (LMKL) optimization problem.
method LMKL-Net employs a feedforward deep neural network with attentional networks and multilayer perceptrons to learn kernel combination weights and multiclass classifiers.
result LMKL-Net outperforms state-of-the-art MKL solvers in accuracy and is trained much faster and with less memory.