A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Study reduces NAS search cost by generating multiple complex architectures in one shot.
Study examines how network architecture handles increasing data complexity.
Gradient-based method prunes large models to create transferable architectures.
Simpler neural architecture search method using random architectures and regression.
New findings show modern neural networks have finite sample complexity in o-minimal structures.
In neural architecture search (NAS), the space of neural network architectures is automatically explored to maximize predictive accuracy for a given task. Despite the success of recent approaches, most existing methods cannot be directly applied to large scale problems because of their prohibitive computational complex…
DNArch learns CNN architectures by backpropagation.
The learnability of different neural architectures can be characterized directly by computable measures of data complexity. In this paper, we reframe the problem of architecture selection as understanding how data determines the most expressive and generalizable architectures suited to that data, beyond inductive bias.…
A graph VAE framework optimizes neural architectures in a continuous space.
Deep neural networks are over-parameterized, which implies that the number of parameters are much larger than the number of samples used to train the network. Even in such a regime deep architectures do not overfit. This phenomenon is an active area of research and many theories have been proposed trying to understand …
Residual neural networks don't help overcome sampling complexity issues.
Develops a complexity measure for neural networks based on quantum statistical mechanics.
Neural Architecture Search has shown potential to automate the design of neural networks. Deep Reinforcement Learning based agents can learn complex architectural patterns, as well as explore a vast and compositional search space. On the other hand, evolutionary algorithms offer higher sample efficiency, which is criti…
Transformer architecture struggles with complex tasks due to limitations in function composition.
Introduces a framework using information theory for understanding machine learning.
This work explores how neural architecture search can improve adversarial robustness without adversarial training.
Deep feedforward and recurrent networks have achieved impressive results in many perception and language processing applications. This success is partially attributed to architectural innovations such as convolutional and long short-term memory networks. The main motivation for these architectural innovations is that t…
Optimal transport kernels improve neural architecture search efficiency.
New distance metric for neural architecture search reduces search space complexity.
Different neural network (NN) architectures have different advantages. Convolutional neural networks (CNNs) achieved enormous success in computer vision, while recurrent neural networks (RNNs) gained popularity in speech recognition. It is not known which type of NN architecture is the best fit for classification of co…
To solve the big topic modeling problem, we need to reduce both time and space complexities of batch latent Dirichlet allocation (LDA) algorithms. Although parallel LDA algorithms on the multi-processor architecture have low time and space complexities, their communication costs among processors often scale linearly wi…
Recent studies on neural architecture search have shown that automatically designed neural networks perform as good as expert-crafted architectures. While most existing works aim at finding architectures that optimize the prediction accuracy, these architectures may have complexity and is therefore not suitable being d…
New hierarchical search algorithm improves neural architecture design across different operator sets.
With a view to bridging the gap between deep learning and symbolic AI, we present a novel end-to-end neural network architecture that learns to form propositional representations with an explicitly relational structure from raw pixel data. In order to evaluate and analyse the architecture, we introduce a family of simp…
Regularization of Deep Neural Networks (DNNs) for the sake of improving their generalization capability is important and challenging. The development in this line benefits theoretical foundation of DNNs and promotes their usability in different areas of artificial intelligence. In this paper, we investigate the role of…
Neural network architectures found by sophistic search algorithms achieve strikingly good test performance, surpassing most human-crafted network models by significant margins. Although computationally efficient, their design is often very complex, impairing execution speed. Additionally, finding models outside of the …
New framework combines simple machines into complex ones for better neural network performance.
Tree-based LSTM improves sequential regression with missing data.
Recent advances in Neural Architecture Search (NAS) have produced state-of-the-art architectures on several tasks. NAS shifts the efforts of human experts from developing novel architectures directly to designing architecture search spaces and methods to explore them efficiently. The search space definition captures pr…
Superior performance and ease of implementation have fostered the adoption of Convolutional Neural Networks (CNNs) for a wide array of inference and reconstruction tasks. CNNs implement three basic blocks: convolution, pooling and pointwise nonlinearity. Since the two first operations are well-defined only on regular-s…
When using reinforcement learning (RL) algorithms to evaluate a policy it is common, given a large state space, to introduce some form of approximation architecture for the value function (VF). The exact form of this architecture can have a significant effect on the accuracy of the VF estimate, however, and determining…
Paper introduces DNTs to clone black-box models efficiently.
Paper tackles end-to-end training of complex neural networks using DIP method.
The paper extends the functional geometry of the visual cortex to more complex architectures using contactization and symplectization.
New architecture improves decision-making in dense traffic.
Novel neural computer learns algorithmic solutions for symbolic tasks.
TabNAS improves neural architecture search for tabular datasets by rejecting suboptimal architectures.
We explore efficient neural architecture search methods and show that a simple yet powerful evolutionary algorithm can discover new architectures with excellent performance. Our approach combines a novel hierarchical genetic representation scheme that imitates the modularized design pattern commonly adopted by human ex…
Improves stability in hyperbolic neural networks for complex data generation.
MixerFlow combines MLP-Mixer with normalizing flows for efficient image modeling.
There is a general trend towards solving problems suited to deep learning with more complex deep learning architectures trained on larger training sets. This requires longer compute times and greater data parallelization or model parallelization. Both data and model parallelism have been historically faster in paramete…
When using reinforcement learning (RL) algorithms it is common, given a large state space, to introduce some form of approximation architecture for the value function (VF). The exact form of this architecture can have a significant effect on an agent's performance, however, and determining a suitable approximation arch…
Unified transformer-based LT-TTD improves ranking efficiency and quality.
New findings on how convolutional architectures approximate time series data.
Extends neural net safety guarantees by proving structural properties.
End-to-end learning refers to training a possibly complex learning system by applying gradient-based learning to the system as a whole. End-to-end learning system is specifically designed so that all modules are differentiable. In effect, not only a central learning machine, but also all "peripheral" modules like repre…
Cell-based NAS search spaces are redundant and lack novelty.
The angular power spectrum characterizes neural network complexity.