Differentiable Algorithm Networks (DAN) enable composable robot learning.
problem Training robots to learn from limited data and imperfect models.
method Composable architecture of neural network modules, each encoding a differentiable robot algorithm and model, trained end-to-end from data.
result DAN modules adapt to one another and compensate for imperfect models and algorithms, achieving best overall system performance.
End-to-end models classify composers from musical scores.
problem Classifying composers from musical scores using machine learning.
method Pooled and convolutional architectures for feature extraction.
result Models achieve high accuracy on a large corpus of scores.
A new language for neural architecture search decouples search spaces and algorithms.
problem Current neural architecture search methods are limited to specific use-cases and lack general-purpose constructs.
method Proposes a formal language for encoding search spaces over general computational graphs, allowing modular, composable, and reusable encodings.
result The language enables easy experimentation with different search spaces and algorithms without reinventing the wheel.
Research has shown that widely used deep neural networks are vulnerable to carefully crafted adversarial perturbations. Moreover, these adversarial perturbations often transfer across models. We hypothesize that adversarial weakness is composed of three sources of bias: architecture, dataset, and random initialization.…
A framework for designing and evaluating new GCN variants.
problem Designing and evaluating new graph convolutional network (GCN) variants.
method Propose a framework to compose networks using building blocks of GCN.
result Several newly composed variants are useful alternatives and competitive with original GCNs.
Randomly initialized sentence encoders perform well on tasks, suggesting learning is key.
problem The role of sentence encoder architectures in language tasks.
method Random initialization and fixed architecture approach to evaluate sentence encoders.
result Priors do not leverage additional information, learning is necessary.
Choosing appropriate architectures and regularization strategies for deep networks is crucial to good predictive performance. To shed light on this problem, we analyze the analogous problem of constructing useful priors on compositions of functions. Specifically, we study the deep Gaussian process, a type of infinitely…
Deep learning (DL) advances state-of-the-art reinforcement learning (RL), by incorporating deep neural networks in learning representations from the input to RL. However, the conventional deep neural network architecture is limited in learning representations for multi-task RL (MT-RL), as multiple tasks can refer to di…
Efficient neural networks compute various differential operators cheaply.
problem Efficient computation of higher time complexity differential operators.
method Restricted neural network architectures with diagonal and hollow Jacobian matrices, allowing efficient extraction of dimension-wise derivatives.
result Demonstrated efficient computation of differential operators for various applications.
Transformer architecture struggles with complex tasks due to limitations in function composition.
problem Transformer architecture's limitations in handling complex tasks.
method Used Communication Complexity to prove limitations in composing functions.
result Transformer layer is incapable of handling large domain functions, even when domains are small.
Quantum datasets improve QML performance.
problem Benchmarking QML on classical datasets is uncertain.
method Introduced NTangled dataset of quantum states with varying entanglement.
result QML models trained on NTangled dataset outperform classical models.
Interstellar searches for recurrent architecture to enhance KG embedding.
problem Learning long-term information in KGs.
method Recurrent neural architecture search for relational paths.
result Effectiveness and efficiency of searched models.
HMCNAS generates competitive neural architectures without human-defined parameters.
problem Lack of human-defined parameters in Neural Architecture Search.
method Combines Hidden Markov Chains and Bayesian Optimization for autonomous search space generation and competitive model generation.
result HMCNAS generates competitive models in a short time without human-defined parameters.
The ability to transfer in reinforcement learning is key towards building an agent of general artificial intelligence. In this paper, we consider the problem of learning to simultaneously transfer across both environments (ENV) and tasks (TASK), probably more importantly, by learning from only sparse (ENV, TASK) pairs …
Enhanced neural model detects misinformation by considering author context.
problem Plateaued accuracy in traditional rumor classifiers.
method Ensemble neural architecture with word-level attention and author context.
result Promising results on Twitter dataset, detecting rumors more accurately.
Artificial neural networks typically have a fixed, non-linear activation function at each neuron. We have designed a novel form of piecewise linear activation function that is learned independently for each neuron using gradient descent. With this adaptive activation function, we are able to improve upon deep neural ne…
Study reduces NAS search cost by generating multiple complex architectures in one shot.
problem Finding multiple neural architectures with varying complexities efficiently.
method Uses importance sampling to generate and update multiple distributions of architectures.
result Reduces search cost by finding multiple architectures with different complexities in a single search.
Paper optimizes trading profits by predicting price direction using ensemble models.
problem Predicting short-term price movements in financial markets.
method Developed a two-layer ensemble architecture optimized through grid search.
result Improvement of 20% over benchmark strategies.
LP algorithm optimizes neural networks with architectural constraints.
problem Training neural networks with specific architectural constraints.
method Lagrangian optimization and saddle point search in adjoint space.
result LP algorithm is fully parallelizable and feasible for training deep networks.
LumièreNet creates lecture videos from audio narration.
problem Creating high-quality lecture videos from audio narration.
method Modular deep-learning architecture that learns from audio to video.
result Synthesizes high-quality lecture videos from any length of audio.
GNNs generalize CNNs for graph data, showing equivariance and stability.
problem Processing signals on graphs.
method Graph convolutional filters, nonlinearities, stacked layers.
result GNNs converge to graphon neural networks under graph convergence.
New findings show disentangled latent representations are not enough for robust compositional generalization.
problem Deep learning models struggle with compositional generalization, especially in out-of-distribution samples.
method Investigated a 2D Gaussian generation task with fully disentangled inputs, then forced disentangled latent representations into full-dimensional output space.
result Forcing disentangled latent representations into full-dimensional output space enables robust compositional generalization.
Deep learning detects cyber-attacks in smart grid systems.
problem Cyber-attacks on smart grid systems.
method Deep learning-based intrusion detection system trained on industrial dataset.
result Proposed system outperforms Naive Bayes, SVM, and Random Forest.
Forecasting based on financial time-series is a challenging task since most real-world data exhibits nonstationary property and nonlinear dependencies. In addition, different data modalities often embed different nonlinear relationships which are difficult to capture by human-designed models. To tackle the supervised l…
Finding the best neural network architecture requires significant time, resources, and human expertise. These challenges are partially addressed by neural architecture search (NAS) which is able to find the best convolutional layer or cell that is then used as a building block for the network. However, once a good buil…
Generating music has a few notable differences from generating images and videos. First, music is an art of time, necessitating a temporal model. Second, music is usually composed of multiple instruments/tracks with their own temporal dynamics, but collectively they unfold over time interdependently. Lastly, musical no…
A model of music needs to have the ability to recall past details and have a clear, coherent understanding of musical structure. Detailed in the paper is a deep reinforcement learning architecture that predicts and generates polyphonic music aligned with musical rules. The probabilistic model presented is a Bi-axial LS…
A new complexity measure for neural networks improves upon classical methods.
problem Lack of a refined complexity measure for comparing different neural network architectures, especially permutation-invariant ones.
method Introduced an equivalence relation among linear functions and counted them relative to this relation.
result The new complexity measure clearly distinguishes between different models and increases exponentially with depth.
Recurrent neural networks (RNNs) are powerful architectures to model sequential data, due to their capability to learn short and long-term dependencies between the basic elements of a sequence. Nonetheless, popular tasks such as speech or images recognition, involve multi-dimensional input features that are characteriz…
The backpropagation of error algorithm (BP) is impossible to implement in a real brain. The recent success of deep networks in machine learning and AI, however, has inspired proposals for understanding how the brain might learn across multiple layers, and hence how it might approximate BP. As of yet, none of these prop…
Mathematical reasoning---a core ability within human intelligence---presents some unique challenges as a domain: we do not come to understand and solve mathematical problems primarily on the back of experience and evidence, but on the basis of inferring, learning, and exploiting laws, axioms, and symbol manipulation ru…
Recurrent neural networks (RNNs) are notoriously difficult to train. When the eigenvalues of the hidden to hidden weight matrix deviate from absolute value 1, optimization becomes difficult due to the well studied issue of vanishing and exploding gradients, especially when trying to learn long-term dependencies. To cir…
New approach learns causally disentangled latent structures in generative models.
problem Fundamental tension between expressivity and structure in latent structure learning.
method Added a context module to an arbitrarily complex model to learn causally disentangled concepts.
result Causally disentangled representations can be composed for out-of-distribution generation.
Sentinel improves time series forecasting by modeling both temporal and channel dependencies.
problem Limited effectiveness of existing transformer-based architectures in multivariate time-series forecasting.
method Proposes Sentinel, a full transformer-based architecture with multi-patch attention mechanism.
result Sentinel achieves better or comparable performance compared to state-of-the-art approaches.
Improved code translation by preserving structure with composed fine-tuning.
problem Improving code translation accuracy with unlabeled code outputs.
method Pre-trained denoiser to capture output structure, composed fine-tuning to fine-tune predictor.
result Composed fine-tuning significantly improves generalization over standard fine-tuning.
FinRL-X unifies trading components for AI and rule-based strategies.
problem Inconsistent between research and live deployment in trading platforms.
method Modular architecture integrating data processing, strategy construction, backtesting, and execution.
result Unified protocol supports AI and rule-based trading components without altering execution.
Method transfers knowledge without label overlap, source data, or target architecture consistency.
problem Difficulties in transfer learning due to label mismatch, restricted source data, and specialized target architectures.
method Uses deep generative models in two stages: pseudo pre-training and pseudo semi-supervised learning.
result Outperforms scratch training and knowledge distillation methods.
In this paper, we explicitly construct the Calabi composition of multiple affine hyperspheres possibly including some points viewing as 0-dimensional hypersheres. Then we compute all the basic affine invariants of the composed affine hyperspheres, proving that the composed affine hypersphere is symmetric one if and onl…
NAS improves gliomas segmentation on MRI scans.
problem Designing optimal deep learning architectures for brain tumor segmentation.
method Neural architecture search with probabilistic parameter learning for MRI brain tumor segmentation.
result Two optimal neural architectures for brain tumor segmentation were discovered.
We present a new model, Predictive State Recurrent Neural Networks (PSRNNs), for filtering and prediction in dynamical systems. PSRNNs draw on insights from both Recurrent Neural Networks (RNNs) and Predictive State Representations (PSRs), and inherit advantages from both types of models. Like many successful RNN archi…
Causal deep learning tackles causal inference using tensor factor analysis.
problem Addressing causal questions in data using neural networks.
method Tensor factor analysis and neural network architectures (causal capsules, tensor transformer, multilinear projection algorithm).
result Derives deep neural networks for causal inference with tensor factor analysis.
The use of distributions and high-level features from deep architecture has become commonplace in modern computer vision. Both of these methodologies have separately achieved a great deal of success in many computer vision tasks. However, there has been little work attempting to leverage the power of these to methodolo…
This work proposes a method to compose visual relations more faithfully.
problem Composing relations between objects in images is challenging due to their entanglement.
method Represent each relation as an unnormalized density (energy-based model) to compose relations factorizedly.
result The proposed method generates and edits scenes with multiple sets of relations more faithfully.
NAS-Navigator automates neural network architecture search with visual steering.
problem Difficulty in configuring large neural networks efficiently.
method Formulates architecture optimization as graph space exploration, trains all candidate architectures in one-shot.
result Allows analysts to effectively select and guide the search for optimal neural network architectures.
A simple CNN architecture outperforms complex ones in P300 detection.
problem Efficiently detecting P300 component from EEG signals.
method Used a simple CNN architecture with a single depthwise separable 1D convolutional layer followed by a fully connected Sigmoid neuron.
result A single depthwise separable 1D convolutional layer with four filters achieved competitive performance.
Financial markets modeled like brain networks using dMNC.
problem Understanding latent dynamics in financial markets.
method Biologically inspired framework using dMNC.
result Structural persistence, regime shifts, and early warning signals identified.
New CNN architecture improves pediatric image segmentation by homogenizing pose and size.
problem Challenges in segmenting pediatric images due to pose and size heterogeneity.
method Spatial Transformer Network (STN) for pose and scale invariance, combined with UNet for segmentation.
result Improved pediatric segmentation, especially renal tumor delineation, with accelerated processing.
While convolutional neural networks (CNNs) have recently made great strides in supervised classification of data structured on a grid (e.g. images composed of pixel grids), in several interesting datasets, the relations between features can be better represented as a general graph instead of a regular grid. Although re…