The exclusive or (xor) function is one of the simplest examples that illustrate why nonlinear feedforward networks are superior to linear regression for machine learning applications. We review the xor representation and approximation problems and discuss their solutions in terms of probabilistic logic and associative …
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “XOR Function”
SGD efficiently learns the XOR function with near-optimal sample complexity.
problem Learning the XOR function with a 2-layer neural network.
method Minibatch SGD on a 2-layer neural network with ReLU activations, focusing on signal-finding and signal-heavy phases.
result Achieves population error with samples.
Neural networks learn to mimic brain neurons with two-input activation functions, improving performance and robustness.
problem Training neural networks to mimic the complex interactions of brain neurons.
method Developed a network-in-network architecture with two-input activation functions, optimized hyperparameters, and compared to conventional ReLU networks.
result Two-input activation functions can learn soft XOR functions, improving network performance and robustness.
SGD quickly learns a spurious XOR feature before the signal feature, revealing learning dynamics.
problem Over-reliance on spurious correlations in neural networks trained by SGD.
method Theoretical analysis of SGD on two-layer ReLU networks trained on XOR data.
result SGD learns the spurious feature first and exponentially fast, dominating the signal feature.
Single neuron with ADA learns XOR and outperforms other functions.
problem Classifying linearly non-separable data.
method Proposed a new artificial neuron with apical dendrite activation.
result ADA function achieves 100% accuracy on XOR and superior performance on benchmark datasets.
When applying Machine Learning techniques to problems, one must select model parameters to ensure that the system converges but also does not become stuck at the objective function's local minimum. Tuning these parameters becomes a non-trivial task for large models and it is not always apparent if the user has found th…
Study on neuron dynamics for XOR classification with zero-margin.
problem Understanding neural network training dynamics in zero-margin classification problems.
method Analysis of Gaussian XOR problem, focusing on neuron block dynamics and generalization without margin assumptions.
result Neurons cluster into four directions and block-level signals evolve coherently, essential for reliable prediction in the Gaussian setting.
We show linear XOR classification is possible and propose equality separation for anomaly detection.
problem Linearly separating XOR data.
method Equality separation, adapting SVM objective for data within/outside margin.
result Equality separation can detect both seen and unseen anomalies.
FleXOR trains fractional quantization for neural networks, improving accuracy and size.
problem Quantization limits to integer bits restricts compression and accuracy.
method Encryption algorithm with XOR gates for fractional bits during inference.
result FleXOR achieves high accuracy with fractional sub-1-bit weights.
Training an artificial neural network involves an optimization process over the landscape defined by the cost (loss) as a function of the network parameters. We explore these landscapes using optimisation tools developed for potential energy landscapes in molecular science. The number of local minima and transition sta…
Efficiently classifies binary labels with XOR queries, even under noisy conditions.
problem Binary classification with unknown labels using XOR queries.
method Effective query type and an efficient inference algorithm for noisy conditions.
result Achieves information-theoretic limit on optimal number of queries.
New method disentangles feature importance scores in machine learning.
problem Misinterpretation of feature importance scores due to interactions and dependencies.
method Derive DIP (Disentangled Importance) decomposition of feature importance scores.
result DIP decomposition uniquely separates standalone contributions from interactions and dependencies.
Study of local optima in neural networks for feature interactions.
problem NNs struggle with local optima in feature interactions for small datasets.
method Proposed a node pruning and feature selection algorithm to improve NN performance.
result NNs have many non-equivalent local optima in XOR-like data with irrelevant variables.
Empirical evidence suggests that neural networks with ReLU activations generalize better with over-parameterization. However, there is currently no theoretical analysis that explains this observation. In this work, we provide theoretical and empirical evidence that, in certain cases, overparameterized convolutional net…
Study on neural networks forgetting in continual learning.
problem Forgetting in neural networks during continual learning.
method Gradient descent analysis on XOR-cluster datasets.
result Explicit bounds on forgetting rate and generalization gap.
A natural p-classes generalization of the eXclusive OR problem, the subtraction modulo p, where p is prime, is presented and solved using a single fully connected hidden layer with p-neurons. Although the problem is very simple, the landscape is intricate and challenging and represents an interesting benchmark for grad…
Neural networks can overfit perfectly to noisy data and then grok near-optimal generalization.
problem Neural networks' ability to overfit perfectly to noisy data and then generalize near-optimally.
method Two-layer ReLU networks trained by gradient descent on XOR cluster data.
result Neural networks can achieve perfect fit to noisy training data and then grok near-optimal generalization.
This paper models the crowdsourced labeling/classification problem as a sparsely encoded source coding problem, where each query answer, regarded as a code bit, is the XOR of a small number of labels, as source information bits. In this paper we leverage the connections between this problem and well-studied codes with …
Eigen-Spike Emergence and Quadratic Equivalents for Conjugate Kernels on Nonlinearly Separable Datastat.ML
Study of eigenvalues in nonlinear kernels for classification of separable data.
problem Understanding the applicability of linear equivalents in nonlinearly separable data classification.
method Analysis of conjugate kernels and their quadratic equivalents for a canonical nonlinearly separable dataset (XOR problem).
result Identification of regimes where nonlinear kernels deviate from linear equivalents, leading to label-aligned eigenspaces.
New method trains Boltzmann machines without supervision.
problem Training unsupervised learning models.
method Mixed binary quadratic feasibility problem formulation.
result Theory validated on XOR patterns.
Model compression techniques, such as pruning and quantization, are becoming increasingly important to reduce the memory footprints and the amount of computations. Despite model size reduction, achieving performance enhancement on devices is, however, still challenging mainly due to the irregular representations of spa…
New measure LMN explains neural network grokking.
problem Delayed generalization after memorization in neural networks.
method Defined LMN to measure network complexity, showing LMN correlates with test losses linearly.
result LMN reveals intriguing XOR network behavior and is a promising complexity measure.
Overparameterisation can limit the benefits of curriculum learning in neural networks.
problem The ineffectiveness of curriculum learning in deep learning applications.
method Analytical study connecting curriculum learning and overparameterisation in an online learning setting for a 2-layer network in the XOR-like Gaussian Mixture problem.
result High degree of overparameterisation can limit the benefit from curricula.
Study neural networks by mapping correlations, revealing essential statistics.
problem Understanding information processing in trained neural networks.
method Characterize neural network as distribution transformations, focusing on correlation functions.
result Higher-order correlations are crucial for internal layers, while input layer captures more.
Paper proposes GrokTransfer to eliminate delayed generalization in neural networks.
problem Delayed generalization in neural networks, compromising predictability and efficiency.
method Trains a smaller, weaker model to reach a nontrivial test performance, then uses its learned input embedding to initialize the stronger model.
result GrokTransfer enables the target model to generalize directly without delay, across various tasks.
Reformulates binary classification on manifolds using Yang-Mills-Higgs theory.
problem Binary classification on non-contractible spaces.
method Formulates binary classification as a Yang-Mills-Higgs variational problem, encoding data as a functor.
result Reveals a geometric interpretation of binary classification and solves XOR on the torus.
Explaining neural network computation in terms of probabilistic/fuzzy logical operations has attracted much attention due to its simplicity and high interpretability. Different choices of logical operators such as AND, OR and XOR give rise to another dimension for network optimization, and in this paper, we study the o…
Gradient descent amplifies random features in neural networks to useful ones.
problem Generalization in neural networks trained on corrupted data.
method Characterization of feature-learning process in two-layer ReLU networks trained by gradient descent.
result Gradient descent amplifies random features to useful ones, achieving near optimal generalization error.
For the tensor PCA (principal component analysis) problem, we propose a new hierarchy of increasingly powerful algorithms with increasing runtime. Our hierarchy is analogous to the sum-of-squares (SOS) hierarchy but is instead inspired by statistical physics and related algorithms such as belief propagation and AMP (ap…
A model of associative memory is studied, which stores and reliably retrieves many more patterns than the number of neurons in the network. We propose a simple duality between this dense associative memory and neural networks commonly used in deep learning. On the associative memory side of this duality, a family of mo…
o1Neuro neural network approximates complex functions and converges quickly.
problem Approximating complex functions and ensuring convergence in neural networks.
method Sparse indicator activation neurons, population and sample level convergence properties.
result o1Neuro achieves optimal model approximation and convergence with high probability.
This paper optimizes high-dimensional oblique splits for decision trees, enhancing performance and computational efficiency.
problem Enhancing decision tree performance and computational efficiency in high-dimensional data.
method Established Sufficient Impurity Decrease (SID) convergence for -sparse oblique splits, proposing progressive trees for iterative refinement.
result Demonstrated that SID function class expands with -sparsity, enabling capture of complex data-generating processes.
A new random forest algorithm uncovers feature interdependencies better than traditional methods.
problem Tackles the sub-optimality of greedy decision tree implementations in random forests.
method Presented a 'stepwise lookahead' variation of random forests that considers multiple split nodes simultaneously.
result Significantly outperforms greedy random forests in uncovering feature interdependencies, especially in high-noise environments.
Proposes a weaker faithfulness assumption for causal discovery.
problem Violation of the faithfulness assumption in causal discovery.
method Proposes a new assumption called 2-adjacency faithfulness and a modified Grow and Shrink algorithm.
result Proves the correctness of the modified algorithm under weaker assumptions.
SafeML: Safety Monitoring of Machine Learning Classifiers through Statistical Difference Measurecs.LG
SafeML monitors ML systems for safety and security risks.
problem Ensuring safety and explainability of ML systems in safety-critical domains.
method Statistical difference measures of ECDF to detect distributional shifts.
result Approach can detect invalid application contexts of ML components.
A current research trend in neurocomputing involves the design of novel artificial neural networks incorporating the concept of time into their operating model. In this paper, a novel architecture that employs stigmergy is proposed. Computational stigmergy is used to dynamically increase (or decrease) the strength of a…
Spiking neural networks (SNNs) have garnered a great amount of interest for supervised and unsupervised learning applications. This paper deals with the problem of training multi-layer feedforward SNNs. The non-linear integrate-and-fire dynamics employed by spiking neurons make it difficult to train SNNs to generate de…
Boosting improves accuracy with fewer calls to weak learners for certain concept classes.
problem Improving accuracy of learning algorithms with limited weak learner calls.
method Combines boosting and list-decodable codes to achieve better performance for specific concept classes.
result A new boosting algorithm that achieves strong learning with fewer calls to weak learners and additional samples.
In this paper, we explore degrees of freedom in deep sigmoidal neural networks. We show that the degrees of freedom in these models is related to the expected optimism, which is the expected difference between test error and training error. We provide an efficient Monte-Carlo method to estimate the degrees of freedom f…
It is well-known that the robustness of artificial neural networks (ANNs) is important for their wide ranges of applications. In this paper, we focus on the robustness of the classification ability of a spiking neural network which receives perturbed inputs. Actually, the perturbation is allowed to be arbitrary styles.…
Much of studies on neural computation are based on network models of static neurons that produce analog output, despite the fact that information processing in the brain is predominantly carried out by dynamic neurons that produce discrete pulses called spikes. Research in spike-based computation has been impeded by th…
High-dimensional SGD limits show surprising dynamics and phase transitions.
problem Understanding SGD in high dimensions and its scaling limits.
method Proving limit theorems for SGD trajectories in high dimensions, choosing summary statistics, initialization, and step-size.
result Critical scaling regime for step-size, new correction term, and complex diffusive limits.
Designing effective and efficient classifier for pattern analysis is a key problem in machine learning and computer vision. Many the solutions to the problem require to perform logic operations such as `and', `or', and `not'. Classification and regression tree (CART) include these operations explicitly. Other methods s…
Signed pairwise interactions conflate uniqueness, redundancy, and synergy
problem Signed pairwise interactions conflate uniqueness, redundancy, and synergy
method Stochastic Hi-Fi
result Stochastic Hi-Fi recovers structure missed by scalar baselines
Classification and regression in which the inputs are graphs of arbitrary size and shape have been paid attention in various fields such as computational chemistry and bioinformatics. Subgraph indicators are often used as the most fundamental features, but the number of possible subgraph patterns are intractably large …
Two-layer networks favor simple features, especially in complex datasets.
problem Simplicity bias in neural networks over-reliing on simple features.
method Characterization of two-layer neural networks with small weights and gradient flow.
result Features learned in middle training stages are more useful for out-of-distribution transfer.
DD algorithm tracks test error from train error without validation data.
problem Systematic generalization gap between train and test errors in modern model training.
method Decoupled descent (DD) algorithm that cancels data reuse biases via approximate message passing.
result DD algorithm rigorously demonstrates zero-cost validation and 100% data utilization.
ChaosNet uses chaotic neural networks for classification with high accuracy on limited data.
problem Classification with limited training data.
method ChaosNet architecture with GLS neurons and a novel learning algorithm exploiting topological transitivity.
result ChaosNet achieves high accuracy (73.89% - 98.33%) with as few as 7 training samples per class.