We propose a new type of hidden layer for a multilayer perceptron, and demonstrate that it obtains the best reported performance for an MLP on the MNIST dataset.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “MultiLayer Perceptrons”
This work concerns estimation of multidimensional nonlinear regression models using multilayer perceptron (MLP). The main problem with such model is that we have to know the covariance matrix of the noise to get optimal estimator. however we show that, if we choose as cost function the logarithm of the determinant of t…
PAC-Bayesian bounds for MLPs with cross entropy loss validated.
problem Generalization bounds for MLPs with cross entropy loss.
method Introduced probabilistic explanations and proved PAC-Bayesian bounds using ELBO.
result MLPs with cross entropy loss inherently guarantee PAC-Bayesian generalization bounds.
Artificial Neural Networks(ANN) has been phenomenally successful on various pattern recognition tasks. However, the design of neural networks rely heavily on the experience and intuitions of individual developers. In this article, the author introduces a mathematical structure called MLP algebra on the set of all Multi…
We utilize Wi-Fi communications from smartphones to predict their mobility mode, i.e. walking, biking and driving. Wi-Fi sensors were deployed at four strategic locations in a closed loop on streets in downtown Toronto. Deep neural network (Multilayer Perceptron) along with three decision tree based classifiers (Decisi…
A Practical Method for Constructing Equivariant Multilayer Perceptrons for Arbitrary Matrix Groupscs.LG
New method constructs equivariant neural networks for arbitrary matrix groups.
problem Challenges in constructing equivariant neural networks for complex groups.
method Completely general algorithm for solving equivariant layers of matrix groups.
result Constructs multilayer perceptrons equivariant to multiple groups including O(1,3), O(5), Sp(n), and Rubik's cube group.
A new geometric perceptron model improves 3D shape classification.
problem Challenges in geometric tasks involving point clouds using machine learning.
method Introduces multilayer geometric perceptron (MLGP) with geometric neurons.
result MLGP outperforms vanilla MLP in 3D shape classification and noise resistance.
Proposes using MLP for predicting optimal penalty in changepoint detection.
problem Predicting optimal penalty for changepoints in sequences.
method Uses a multilayer perceptron (MLP) with ReLU activation function to predict penalty.
result Improves accuracy and F1 score compared to existing models.
PCGs encompass a broader range of neural networks.
problem Understanding the broader scope of neural network models.
method Proving PCGs as a superset of feedforward neural networks.
result PCGs represent a wider class of neural network models.
Intraday trading strategy based on time series and machine learning for Chinese stock marketq-fin.TR
Article proposes a profitable intraday trading strategy for Chinese stocks.
problem Intraday trading opportunities in Chinese stock market.
method Markowitz optimization and Multilayer Perceptron (MLP) for stock price prediction.
result Validation of Markowitz portfolio optimization and MLP for intraday stock price prediction.
Global demographic and economic changes have a critical impact on the total energy consumption, which is why demographic and economic parameters have to be taken into account when making predictions about the energy consumption. This research is based on the application of a multiple linear regression model and a neura…
This paper proposes an improved design of the perceptron unit to mitigate the vanishing gradient problem. This nuisance appears when training deep multilayer perceptron networks with bounded activation functions. The new neuron design, named auto-rotating perceptron (ARP), has a mechanism to ensure that the node always…
A new KAN variant uses sinusoidal activations to approximate functions.
problem Approximating multivariable functions using neural networks.
method Replacing inner and outer functions in Kolmogorov-Arnold representation with weighted sinusoidal functions.
result The new KAN variant outperforms fixed-frequency Fourier transform and achieves comparable performance to MLPs.
Long-term lane change prediction model predicts maneuvers with 75% accuracy.
problem Predicting long-term lane changes for safer autonomous driving.
method Introduced three models: logistic regression, MLP, and RNN. Used NGSIM dataset with new labeling scheme.
result Developed model predicts 75% of lane changes with an average advanced time of 8.05 seconds.
Group invariant and equivariant Multilayer Perceptrons (MLP), also known as Equivariant Networks, have achieved remarkable success in learning on a variety of data structures, such as sequences, images, sets, and graphs. Using tools from group theory, this paper proves the universality of a broad class of equivariant M…
This work proves the asymptotic freeness of layerwise Jacobians in MLPs with Haar orthogonal matrices.
problem Proving the asymptotic freeness of layerwise Jacobians in multilayer perceptrons (MLPs).
method Replacing each layer's parameter matrix with itself multiplied by a Haar orthogonal matrix, and using the invariance of the MLP.
result Proves the asymptotic freeness of layerwise Jacobians in MLPs with Haar orthogonal matrices.
New method breaks symmetry in neural networks, improving sample efficiency.
problem Symmetry in neural networks limits their ability to learn unique features.
method Introduces 'relaxed equivariance' to overcome symmetry limitations.
result Equivariant multilayer perceptrons (E-MLPs) can now break symmetry at the sample level.
Like most learning algorithms, the multilayer perceptrons (MLP) is designed to learn a vector of parameters from data. However, in certain scenarios we are interested in learning structured parameters (predictions) in the form of symmetric positive definite matrices. Here, we introduce a variant of the MLP, referred to…
Designs an MLP from LDA for multi-Gaussian class classification.
problem Classifying inputs with multiple Gaussian distributions.
method Interprets MLP as generalized LDA, using LDAs for half-space partitioning, neurons for subspace isolation, and merging for class-wise representation.
result Automatic feedforward design for MLP architecture and weights.
Deep learning models compare performance on Limit Order Book tasks.
problem Comparing Deep Learning models for High Frequency Trading.
method Reviewed and compared state-of-the-art models on the same dataset.
result Multilayer Perceptrons perform comparably to CNN-LSTM architectures.
Kolmogorov-Arnold Networks in Low-Data Regimes: A Comparative Study with Multilayer Perceptronscs.LG
This study compares MLPs and KANs in low-data regimes, finding MLPs with personalized activation functions outperform KANs.
problem Comparing MLPs and KANs in low-data regimes.
method Introduced an effective technique for designing MLPs with unique, parameterized activation functions for each neuron.
result MLPs with personalized activation functions achieve significantly higher predictive accuracy with only a modest increase in parameters, especially in low-data regimes.
In this work we use Recurrent Neural Networks and Multilayer Perceptrons to predict NYSE, NASDAQ and AMEX stock prices from historical data. We experiment with different architectures and compare data normalization techniques. Then, we leverage those findings to question the efficient-market hypothesis through a formal…
Emerging Statistical Machine Learning Techniques for Extreme Temperature Forecasting in U.S. Citiesstat.AP
The paper analyzes extreme temperature forecasting using machine learning models.
problem Forecasting extreme temperatures in U.S. cities.
method Auto-Regressive Integrated Moving Average, Exponential Smoothing, Multilayer Perceptrons, Gaussian Processes.
result Multilayer Perceptrons were found to be the most effective approach for forecasting extreme temperatures.
These days human beings are facing many environmental challenges due to frequently occurring drought hazards. It may have an effect on the countrys environment, the community, and industries. Several adverse impacts of drought hazard are continued in Pakistan, including other hazards. However, early measurement and det…
Proposes graph neural network layers for manifold-valued graphs.
problem Graphs with features in a Riemannian manifold.
method Diffusion layer and tangent multilayer perceptron.
result Outperforms state-of-the-art networks on Alzheimer's classification.
Models with transparent inner structure and high classification performance are required to reduce potential risk and provide trust for users in domains like health care, finance, security, etc. However, existing models are hard to simultaneously satisfy the above two properties. In this paper, we propose a new hierarc…
VQC-MLPNet combines quantum and classical elements for scalable quantum machine learning.
problem Challenges in expressivity, trainability, and noise resilience of VQCs.
method Hybrid architecture with a VQC generating weights for a classical MLP during training.
result Improved expressivity, trainability, and robustness compared to standalone quantum or hybrid approaches.
New sampling method estimates Shapley values more accurately.
problem Exponential time complexity of computing Shapley values.
method Multilinear sampling algorithm based on game theory.
result Our method reduces variance and provides more accurate Shapley value estimations.
In-Context Learning as Nonparametric Conditional Probability Estimation: Risk Bounds and Optimalitystat.ML
Paper analyzes risk bounds for in-context learning in multiclass classification.
problem Risk bounds for in-context learning in multiclass classification.
method Formalizes tasks as sequences of labeled examples and queries, estimates conditional class probabilities, establishes oracle inequality for KL divergence.
result ICL achieves minimax optimal rate for conditional probability estimation.
WavPool improves deep neural networks with wavelet-based pooling.
problem Improving efficiency and performance of deep neural networks.
method Introducing WavPool, a wavelet-transform-based pooling layer.
result WavPool outperforms existing network architectures by 10% on CIFAR-10.
Despite the success of deep learning in domains such as image, voice, and graphs, there has been little progress in deep representation learning for domains without a known structure between features. For instance, a tabular dataset of different demographic and clinical factors where the feature interactions are not gi…
Effective theory for Transformer initialization improves model performance.
problem Improving performance of Transformers at initialization.
method Effective-theory analysis of signal propagation in wide and deep Transformers.
result Particular width scalings of initialization and training hyperparameters.
This work concerns testing the number of parameters in one hidden layer multilayer perceptron (MLP). For this purpose we assume that we have identifiable models, up to a finite group of transformations on the weights, this is for example the case when the number of hidden units is know. In this framework, we show that …
Neural Decision Treesstat.ML
In this paper we propose a synergistic melting of neural networks and decision trees (DT) we call neural decision trees (NDT). NDT is an architecture a la decision tree where each splitting node is an independent multilayer perceptron allowing oblique decision functions or arbritrary nonlinear decision function if more…
Deep learning models price options using volatility surfaces.
problem Pricing exotic options with high accuracy and efficiency.
method Variational autoencoder for volatility surface compression, multilayer perceptron for option pricing.
result Trained model achieves high accuracy across American and Asian options.
This paper explains double descent using VC theory.
problem Understanding the generalization of overfitting neural networks.
method VC-theoretical analysis of double descent.
result Double descent can be explained by classical VC-generalization bounds.
An appropriate choice of the activation function (like ReLU, sigmoid or swish) plays an important role in the performance of (deep) multilayer perceptrons (MLP) for classification and regression learning. Prototype-based classification learning methods like (generalized) learning vector quantization (GLVQ) are powerful…
Capacity of the treelike sign perceptrons neural networks with one hidden layer -- RDT based upper boundscond-mat.dis-nn
New bounds on neural network capacity for treelike sign perceptrons using RDT.
problem Determining the capacity of treelike sign perceptrons neural networks.
method Random Duality Theory (RDT) to establish upper bounds.
result Mathematically rigorous bounds on network capacity for any number of neurons.
Machine learning classifies braids and discovers new invariants.
problem Classifying and discovering invariants of braids and flat braids.
method Supervised learning with neural networks to classify braids as trivial or non-trivial.
result Found new convenient invariants of braids, including a complete invariant of flat braids.
Overwhelming theoretical and empirical evidence shows that mildly overparametrized neural networks -- those with more connections than the size of the training data -- are often able to memorize the training data with accuracy. This was rigorously proved for networks with sigmoid activation functions and, very …
A Monte Carlo k-nearest neighbours (KNN) and a multi-resolution convolutional neural network (CNN) were developed to detect the presences of multiple gasses in near infrared (IR) spectrums. High Resolution Transmission database was used to synthesize the near IR spectrums. Monte Carlo KNN determined the optimal kernel …
On Expected Accuracycs.LG
We empirically investigate the (negative) expected accuracy as an alternative loss function to cross entropy (negative log likelihood) for classification tasks. Coupled with softmax activation, it has small derivatives over most of its domain, and is therefore hard to optimize. A modified, leaky version is evaluated on…
We present here a new model and algorithm which performs an efficient Natural gradient descent for Multilayer Perceptrons. Natural gradient descent was originally proposed from a point of view of information geometry, and it performs the steepest descent updates on manifolds in a Riemannian space. In particular, we ext…
This work studies the entity-wise topical behavior from massive network logs. Both the temporal and the spatial relationships of the behavior are explored with the learning architectures combing the recurrent neural network (RNN) and the convolutional neural network (CNN). To make the behavioral data appropriate for th…
Interest rates mappingq-fin.ST
The present study deals with the analysis and mapping of Swiss franc interest rates. Interest rates depend on time and maturity, defining term structure of the interest rate curves (IRC). In the present study IRC are considered in a two-dimensional feature space - time and maturity. Geostatistical models and machine le…
Neural networks learn to mimic brain neurons with two-input activation functions, improving performance and robustness.
problem Training neural networks to mimic the complex interactions of brain neurons.
method Developed a network-in-network architecture with two-input activation functions, optimized hyperparameters, and compared to conventional ReLU networks.
result Two-input activation functions can learn soft XOR functions, improving network performance and robustness.
The ability to witness non-local correlations lies at the core of foundational aspects of quantum mechanics and its application in the processing of information. Commonly, this is achieved via the violation of Bell inequalities. Unfortunately, however, their systematic derivation quickly becomes unfeasible as the scena…
Study of infinitely deep but narrow neural networks using NTK theory.
problem Analyzing the role of depth in deep learning with overparameterized networks.
method Infinite-depth limit analysis of MLP and CNN using Neural Tangent Kernel (NTK) theory.
result Established trainability guarantee for infinitely deep but narrow neural networks.