Neural networks learn modular arithmetic but not all, extending known solutions to generalize.
problem Neural networks struggle with modular arithmetic, especially for polynomials.
method Developed analytical solutions for MLP networks to learn modular addition and multiplication, then combined these solutions to generalize on arbitrary modular polynomials.
result Neural networks can learn and generalize solutions to modular polynomials, supporting the hypothesis that some polynomials are learnable.
Modular deep learning framework using pairwise labels without backpropagation.
problem Efficiently training deep neural networks with limited supervision.
method Stacked linear models in feature spaces, provably optimal modular learning framework.
result High accuracy (94.88%) achieved with minimal labeled examples (1 per class).
Artificial neural networks (ANNs) have achieved significant success in tackling classical and modern machine learning problems. As learning problems grow in scale and complexity, and expand into multi-disciplinary territory, a more modular approach for scaling ANNs will be needed. Modular neural networks (MNNs) are neu…
Modular neural networks generalize better with less data.
problem Theoretical and practical understanding of how modularity improves neural network generalization.
method Theoretical analysis of sample complexity, development of a novel learning rule.
result Modular networks require fewer samples to generalize compared to nonmodular networks, especially in high-dimensional tasks.
Soft modularization improves sample efficiency and performance in reinforcement learning.
problem Challenges in training multiple tasks jointly in reinforcement learning.
method Explicit modularization technique on policy representation, soft modularization method.
result Improves sample efficiency and performance over strong baselines in robotics manipulation tasks.
PICLE uses probabilistic models to efficiently evaluate and compose modules for continual learning.
problem Challenging search space of module compositions in continual learning.
method Probabilistic framework to cheaply compute module compositions' fitness.
result First modular CL algorithm to achieve perceptual, few-shot, and latent transfer.
RNNs solve modular addition tasks using low rank and sparse Fourier structures.
problem Solving modular addition tasks with recurrent neural networks.
method Identified low rank structures and sparse Fourier representations in RNN weights.
result RNNs robust to removing individual frequencies but degrade with more ablation.
Survey of methods to train deep architectures without E2EBP.
problem Training deep architectures without end-to-end backpropagation.
method Modular training, weakly modular hybrids.
result Provably optimal alternatives to E2EBP can match or surpass E2EBP performance.
New methods detect modular structure in neural networks, revealing surprising effects of dropout.
problem Detecting functional modules in neural networks for learning, compositionality, and generalization.
method Two families of methods: upstream and downstream, to define similarity between units.
result Dropout dramatically increased modularity, and there's little agreement between upstream and downstream methods.
Training a Neural Network (NN) with lots of parameters or intricate architectures creates undesired phenomena that complicate the optimization process. To address this issue we propose a first modular approach to NN design, wherein the NN is decomposed into a control module and several functional modules, implementing …
Unified policy controls diverse agents through modular neural networks.
problem Learning control policies for various agent morphologies.
method Shared Modular Policies (SMP) with decentralized control and message passing.
result A single modular policy controls multiple agent morphologies.
Enhances multi-modular models by directing information flow between components.
problem Improving predictive performance in multi-modular models with misspecification.
method Introduces Semi-Modular Inference (SMI) with an influence parameter to control information flow between modules.
result SMI allows for tunable and directed information flow, improving prediction in some settings.
New framework for modular reinforcement learning reduces sample complexity.
problem Achieving independent credit assignment in reinforcement learning.
method Defining modular credit assignment as minimizing algorithmic mutual information, introducing modularity criterion for causal analysis.
result Single-step temporal difference action-value methods meet the modularity criterion, improving sample efficiency.
The study shows how modular learning can adapt to new tasks.
problem Adapting to new tasks in an ever-changing environment.
method Task segmentation, modular learning, memory-based ensembling.
result The system demonstrates robustness to catastrophic forgetting and increasing positive transfer.
Modular RL modules solve complex 3D Sokoban tasks.
problem Solving complex, integrated tasks combining visual, physical, and abstract reasoning.
method Compose RL modules in a sense-plan-act hierarchy, using only model-free methods.
result Modular RL outperforms state-of-the-art monolithic RL on Mujoban.
PyTorch Frame simplifies multi-modal tabular learning with modular data and model handling.
problem Handling complex multi-modal tabular data in deep learning.
method A PyTorch-based framework that provides a data structure, model abstraction, and integration with external models.
result Demonstrated the effectiveness of PyTorch Frame in implementing and applying diverse tabular models to complex multi-modal tabular data.
New methods for scalable inference in modular models with misspecified sub-models.
problem Model misspecification in multi-modular models complicates evidence combination.
method Variational methods for approximating Cut and SMI posteriors, and Variational Meta-Posterior.
result Feasibility of analysis with multiple cuts using a single set of variational parameters.
Study on AI-driven modeling for high burnup accident-tolerant fuels in SMRs.
problem Design and optimization of high burnup accident-tolerant fuels for SMRs.
method Artificial intelligence and multi-scale modeling (neutronics, thermal hydraulics, fuel performance).
result Demonstrated the effectiveness of AI in modeling and optimizing SMR fuels.
Recursive Feature Machines show grokking in modular arithmetic without neural networks.
problem Grokking in modular arithmetic tasks.
method Recursive Feature Machines (RFM) with Average Gradient Outer Product (AGOP).
result RFM and neural networks learn block-circulant features to solve modular arithmetic.
Modular meta-learning is a new framework that generalizes to unseen datasets by combining a small set of neural modules in different ways. In this work we propose abstract graph networks: using graphs as abstractions of a system's subparts without a fixed assignment of nodes to system subparts, for which we would need …
A modular GP framework for efficient transfer learning.
problem Efficiently transfer knowledge across different tasks or datasets.
method Modular variational Gaussian processes (GPs) with a dictionary of well-fitted GPs.
result Reduces computational costs and allows the transfer of uncertainty metrics.
Proposes a CL technique to improve accuracy and reduce forgetting.
problem Sequential task learners struggle with forgetting information from previous tasks.
method Extracts modular parts of neural networks and estimates task relatedness.
result Remarkable performance gain in robustness to forgetting for EWC and GEM methods.
We find and propose an explanation for a large variety of modularity-related symmetries in problems of 3-manifold topology and physics of 3d N=2 theories where such structures a priori are not manifest. These modular structures include: mock modular forms, SL(2,Z) Weil representations, quantum mo…
Torch-Points3D simplifies 3D deep learning research and reproducibility.
problem Lack of transparency and reproducibility in 3D deep learning research.
method Modular framework with quality-of-life features, standardized protocols, and open-source implementation.
result Facilitates fair and rigorous evaluation of 3D deep learning methods.
Transformers learn to solve modular arithmetic tasks by in-context learning and skill composition.
problem Understanding how large language models generalize to unseen tasks in modular arithmetic.
method Pre-training on a set of modular arithmetic tasks and evaluating out-of-distribution performance.
result Transformers require two transformer blocks for out-of-distribution generalization, and deeper models exhibit transient out-of-distribution performance.
modAL is a modular active learning framework for Python, aimed to make active learning research and practice simpler. Its distinguishing features are (i) clear and modular object oriented design (ii) full compatibility with scikit-learn models and workflows. These features make fast prototyping and easy extensibility p…
Researchers found the global topology of the Eisenstein-Picard modular surface.
problem Understanding the global topology of the Eisenstein-Picard modular surface.
method Quotient space of the complex hyperbolic plane by the modular group.
result Determined the global topology of the Eisenstein-Picard modular surface as a 4-orbifold.
Improved graph clustering with modularity and coarsening for attributes and communities.
problem Inaccurate community detection and computational inefficiency in graph clustering.
method Integrates coarsening and modularity maximization, using a loss function with log-determinant, smoothness, and modularity components.
result Superior clustering outcomes, proven consistent under DC-SBM, and efficient algorithm integration with GNNs and VGAEs.
Study modular surfaces in Lorentz-Minkowski 3-space, classifying and analyzing their curvature and applications.
problem Understanding the curvature properties of modular surfaces in Lorentz-Minkowski space.
method Analyzing the sign of Gaussian and mean curvature, classifying surfaces, and applying to conformal field theories.
result Complete classification of zero Gaussian curvature modular surfaces and non-existence of non-planar maximal modular surfaces.
Study modular forms over Γ^0(2) and anomaly cancellation formulas.
problem Anomaly cancellation formulas for modular forms over Γ^0(2).
method Study and analysis of modular forms over Γ^0(2).
result Anomaly cancellation formulas derived for modular forms over Γ^0(2).
Our aim is to introduce and advocate non-Σ (non-symmetric) modular operads. While ordinary modular operads were inspired by the structure of the moduli space of stable complex curves, non-Σ modular operads model surfaces with open strings outputs. An immediate application of our theory is a short proof that the mod…
We focus on two supervised visual reasoning tasks whose labels encode a semantic relational rule between two or more objects in an image: the MNIST Parity task and the colorized Pentomino task. The objects in the images undergo random translation, scaling, rotation and coloring transformations. Thus these tasks involve…
BOML unifies meta-learning methods into a common bilevel optimization framework.
problem Meta-learning methods with diverse modeling aspects.
method Modularized bilevel optimization library in Python.
result Unified solution for various meta-learning formulations.
Fuchsian groups with a modular embedding have the richest arithmetic properties among non-arithmetic Fuchsian groups. But they are very rare, all known examples being related either to triangle groups or to Teichmueller curves. In Part I of this paper we study the arithmetic properties of the modular embedding and deve…
Study explores how neural networks and Transformers learn modular arithmetic with multiple inputs.
problem Understanding how neural networks and Transformers learn modular arithmetic with multiple inputs.
method Analytical characterization of features learned by neural networks and Transformers, focusing on margin maximization and Fourier spectra.
result Neural networks and Transformers require a minimum neuron count of \( m \geq 2^{2k-2} \cdot (p-1) \) to solve modular addition problems with \( k \) inputs and modulus \( p \).
A framework for modular training of robust generative models.
problem Training large generative models is resource-intensive and requires heuristic tuning.
method Modular training using a gating mechanism and a minimax game to find a robust gate.
result The modular approach can theoretically outperform monolithic baselines and is scalable.
Improved algorithm for modular links provides upper volume bounds.
problem Understanding the geometry of modular links and Lorenz links.
method Bunch algorithm to study modular links and provide upper volume bounds.
result First upper volume bound independent of word exponents and quadratic in braid index.
Geodesics on modular surface yield arithmetic 3-manifolds.
problem Understanding arithmetic properties of modular surfaces.
method Constructing geodesics and analyzing their lifts.
result Complements of canonical lifts are arithmetic 3-manifolds.
We introduce the notion of the modular class of a Lie algebroid equipped with a Nambu structure. In particular, we recover the modular class of a Nambu-Poisson manifold M with its Nambu tensor Λ as the modular class of the tangent Lie algebroid TM with Nambu structure Λ. We show that many known properties of th…
Contemporary sensorimotor learning approaches typically start with an existing complex agent (e.g., a robotic arm), which they learn to control. In contrast, this paper investigates a modular co-evolution strategy: a collection of primitive agents learns to dynamically self-assemble into composite bodies while also lea…
New modular forms for anomaly cancellation formulas on any dimensional manifolds.
problem Constructing new modular forms for anomaly cancellation formulas.
method Using E8 bundles, constructing modular forms on any dimensional manifolds. result Derived new anomaly cancellation formulas and applications.
MOCA uses modular attention to estimate causal effects from complex data.
problem Estimating causal effects from observational data with complex, non-linear, and high-dimensional treatment and outcome mechanisms.
method MOCA is a transformer-based framework that separates treatment and outcome modeling through modular design and one-way attention mechanism, with cutting-feedback to prevent outcome influence on treatment representations.
result MOCA outperforms classical estimators and machine learning approaches across various simulated and real-world scenarios.
Quantum modularity proved for SU(2) TQFT signature on genus 2 surfaces.
problem Proving quantum modularity of SU(2) TQFT signature for genus 2 surfaces.
method Using quantum modularity of generalized Dedekind sums associated with modular forms and trigonometric sum expressions.
result Quantum modularity of SU(2) TQFT signature on genus 2 surfaces proved.
New modularity function improves clustering of spatially embedded networks.
problem Improving clustering in spatially embedded networks for unsupervised learning.
method Developed a new modularity function and compared its performance with existing methods.
result Our modularity function outperforms existing methods in partitioning 2D and 3D granular assemblies.
Motivated by a question of Hirzebruch on the possible topological types of cusp cross-sections of Hilbert modular varieties, we give a necessary and sufficient condition for a manifold M to be diffeomorphic to a cusp cross-section of a Hilbert modular variety. Specialized to Hilbert modular surfaces, this proves that e…
Countable modular groups found on surfaces with infinite type.
problem Finding modular groups of infinite type surfaces.
method Proving countable modular groups for orientable infinite type surfaces.
result Every orientable infinite type surface has a countable modular group.
Sine activation functions enable two-layer neural networks to learn modular addition more efficiently.
problem Learning modular addition with two-layer neural networks.
method Introduced and analyzed sine activation functions, providing theoretical and empirical evidence.
result Sine activation functions allow for constant-width network realizations of modular addition, whereas ReLU networks require linear width scaling.
Quantum modularity proven for specific theta series.
problem Proving quantum modularity for partial theta series with periodic coefficients.
method Explicit proof using Kontsevich-Zagier series and colored Jones polynomials.
result Kontsevich-Zagier series is a weight 3/2 quantum modular form.