Paper explores physics-informed deep learning for system reliability assessment.
problem Limited study on deep learning for system reliability assessment.
method Physics-informed deep learning approach for system reliability assessment.
result Physics-informed deep learning can alleviate computational challenges and combine measurement data and mathematical models.
In this paper, a new deep reinforcement learning based augmented general sequence tagging system is proposed. The new system contains two parts: a deep neural network (DNN) based sequence tagging model and a deep reinforcement learning (DRL) based augmented tagger. The augmented tagger helps improve system performance …
Deep learning networks are approximated using dynamical systems theory.
problem Understanding the approximation capabilities of deep learning networks.
method Modeling deep residual networks as continuous-time dynamical systems and using approximation theories in Lp. result Established general sufficient conditions for universal approximation of deep residual networks.
Among the machine learning applications to business, recommender systems would take one of the top places when it comes to success and adoption. They help the user in accelerating the process of search while helping businesses maximize sales. Post phenomenal success in computer vision and speech recognition, deep learn…
Deep learning system generates new Chinese fonts via style variables.
problem Efficiently design new Chinese fonts.
method End-to-end deep learning system generating new style fonts via interpolation of latent style-related embedding variables.
result Smooth transition between different font styles achieved.
StarNet trains deep models without gradients using linear equations.
problem Training deep generative models with gradients.
method Solving determined systems of linear equations.
result Least-square bounds for latent codes and model parameters.
Stable deep models learn dynamical systems with formal stability guarantees.
problem Difficulties in making formal claims about stability of deep network dynamics models.
method Jointly learning a dynamics model and Lyapunov function to ensure non-expansiveness.
result Proposes an approach for stable deep learning of dynamical systems.
Physics-guided model improves deep learning for nonlinear systems.
problem Intractable inference of nonlinear dynamical systems from data.
method Physics-guided Deep Markov Model (PgDMM) using neural networks.
result Improved performance on nonlinear systems with structured latent space.
Research proposes a test case generation system for deep learning models using dataset properties.
problem Automated generation of extensive test cases for deep learning models is challenging.
method Measures dataset quality and proposes a test case generation system guided by dataset properties.
result Systematic test case generation for deep learning models is effective.
Automated testing improves deep learning model accuracy by 259.2%.
problem Ensuring robustness of deep learning models through automated testing.
method Jointly optimizing differential behaviors and neuron coverage; generating corner-cases; applying transformations and GANs.
result Deep learning model accuracy increased by 259.2% using automated generated corner cases.
Generalized linear models with nonlinear feature transformations are widely used for large-scale regression and classification problems with sparse inputs. Memorization of feature interactions through a wide set of cross-product feature transformations are effective and interpretable, while generalization requires more…
DeepRSCN models nonlinear systems using stochastic configurations.
problem Modeling nonlinear dynamic systems efficiently.
method Incrementally constructed deep reservoir computing framework with random parameters and online weight updates.
result DeepRSCN outperforms single-layer networks in efficiency, learning, and generalization.
A novel deep probabilistic model for dynamic systems forecasting.
problem Probabilistic forecasting in dynamic systems.
method Combining deep generative models and state space models with recurrent neural networks and variational sequence models.
result Outperforms existing models in system identification benchmarks and real-world centrifugal compressor forecasting.
Improves deep learning robustness by considering task and model.
problem Adversarial attacks on deep learning systems.
method Binary and interval label encoding strategy to redefine classification tasks and design corresponding loss functions.
result Our method enhances robustness without sacrificing accuracy.
We consider the application of deep generative models in propagating uncertainty through complex physical systems. Specifically, we put forth an implicit variational inference formulation that constrains the generative model output to satisfy given physical laws expressed by partial differential equations. Such physics…
Deep SSMs use neural networks to identify complex systems.
problem Identifying nonlinear systems with high uncertainty.
method Deep state space models with neural networks.
result Deep SSMs outperform traditional methods on benchmarks.
Paper tackles interpretability issues in deep learning models.
problem Lack of understanding of deep learning models' decision-making processes.
method Integrates concepts from machine learning, quantum computation, and quantum field theory.
result Demonstrates a many valued quantum logic system in Convolutional Deep Belief Networks.
Paper uses deep learning for systemic risk measures.
problem Computing optimal capital allocations for systemic risk.
method Deep learning algorithms to solve primal and dual problems.
result Deep learning provides fair risk allocations.
This survey presents a review of state-of-the-art deep neural network architectures, algorithms, and systems in vision and speech applications. Recent advances in deep artificial neural network algorithms and architectures have spurred rapid innovation and development of intelligent vision and speech systems. With avai…
How to understand deep learning systems remains an open problem. In this paper we propose that the answer may lie in the geometrization of deep networks. Geometrization is a bridge to connect physics, geometry, deep network and quantum computation and this may result in a new scheme to reveal the rule of the physical w…
Deep learning models generate languages that lack abstract reasoning.
problem Lack of abstract reasoning in deep learning-generated languages.
method Analyzed emergent language from two multi-agent games with compositional measures.
result Deep learning solutions often fail to generalize to out-of-training examples.
Autonomous AI systems will be entering human society in the near future to provide services and work alongside humans. For those systems to be accepted and trusted, the users should be able to understand the reasoning process of the system, i.e. the system should be transparent. System transparency enables humans to fo…
Complex high dimensional stochastic dynamic systems arise in many applications in the natural sciences and especially biology. However, while these systems are difficult to describe analytically, "snapshot" measurements that sample the output of the system are often available. In order to model the dynamics of such sys…
Deep neural networks with memory learn reduced equations from partial data.
problem Constructing governing equations for unknown dynamical systems from limited data.
method Formulate a discrete approximation of memory integrals, use deep neural networks to incorporate history terms.
result Deep neural networks can learn reduced equations with memory from partial data.
Deep networks analyzed using IFS theory for stability and generalization.
problem Stability and generalization of deep neural networks.
method Viewing deep architectures as place-dependent IFS and applying results from random dynamical systems.
result Derivation of a Wasserstein generalization bound and a new training objective.
It is well established that neural networks with deep architectures perform better than shallow networks for many tasks in machine learning. In statistical physics, while there has been recent interest in representing physical data with generative modelling, the focus has been on shallow neural networks. A natural ques…
Generative model generates images with multiple object classes.
problem Generating images with multiple object classes.
method Conditional Deep Convolutional GAN architecture, stabilized against collapse.
result System generates diverse samples with inter-object relationships.
dynoGP uses deep Gaussian processes for dynamic system identification.
problem System identification for complex dynamical systems.
method Interconnecting linear dynamic GPs and static GPs to model dynamic and static nonlinearities.
result Demonstrates effectiveness of the approach using both simulated and real-world data.
Fault detection problem for closed loop uncertain dynamical systems, is investigated in this paper, using different deep learning based methods. Traditional classifier based method does not perform well, because of the inherent difficulty of detecting system level faults for closed loop dynamical system. Specifically, …
This work extracts stochastic dynamical systems with α-stable Lévy noise.
problem Extracting data-driven governing laws of dynamical systems with non-Gaussian noise.
method End-to-end deep learning approach for learning drift and diffusion coefficients for α-stable Lévy noise. result Effectiveness of the method confirmed by numerical experiments.
Deep learning improves conversational recommender systems.
problem Creating effective goal-oriented dialogue systems through natural language interactions.
method Apply deep learning techniques to conversational recommender systems.
result Deep learning models enhance user preference learning in CRS.
Deep neural network generates symbolic equations from data.
problem Lack of insight into underlying mappings from traditional deep learning.
method Combines deep learning flexibility with symbolic solutions.
result Accurately generates governing equations for dynamical systems.
Traditional vision-based hand gesture recognition systems is limited under dark circumstances. In this paper, we build a hand gesture recognition system based on microwave transceiver and deep learning algorithm. A Doppler radar sensor with dual receiving channels at 5.8GHz is used to acquire a big database of hand ges…
Generalized ResNet learns unknown dynamical systems using neural networks.
problem Learning unknown dynamical systems with deep neural networks.
method A generalized ResNet framework using discrepancy as model correction.
result Generalized ResNet produces more accurate predictions than standard ResNet.
Deep QMC method accurately computes electronic excited states.
problem Accurate calculation of electronic excited states in large systems.
method Extends variational QMC with deep neural networks for excited states.
result Consistently achieves high accuracy for low-lying excited states.
We present MILABOT: a deep reinforcement learning chatbot developed by the Montreal Institute for Learning Algorithms (MILA) for the Amazon Alexa Prize competition. MILABOT is capable of conversing with humans on popular small talk topics through both speech and text. The system consists of an ensemble of natural langu…
Adaptive neural network approximates stochastic system densities.
problem Approximating high-dimensional stochastic dynamical systems.
method Temporal KRnet (tKRnet) trained with adaptive collocation points and temporal decomposition.
result Improves density approximation for stochastic systems without curse of dimensionality.
To assure cyber security of an enterprise, typically SIEM (Security Information and Event Management) system is in place to normalize security event from different preventive technologies and flag alerts. Analysts in the security operation center (SOC) investigate the alerts to decide if it is truly malicious or not. H…
Study on deep learning IDS resistance against adversarial attacks.
problem Vulnerabilities in deep learning-based IDS against adversarial attacks.
method Apply min-max optimization to train IDS against adversarial samples.
result Adversarial attack methods can be used in continuous domains and boost IDS robustness.
Generative adversarial network for probabilistic forecasting of random systems.
problem Forecasting random dynamical systems without distributional assumptions.
method Recurrent neural network and generative adversarial network (GAN) with regularization based on maximum mean discrepancy (MMD).
result The proposed model successfully forecasts complex stochastic processes with multiple-step predictions.
Aug-Gen improves music generation by augmenting training data with model-produced examples.
problem Lack of high-quality training data for specific musical styles.
method Dataset augmentation using generated examples during training.
result Aug-Gen improves both training duration and quality of generated music.
Symmetry in inverse problems leads to multiple solutions, but breaking symmetry helps deep learning.
problem Symmetry in physical systems causes multiple solutions in inverse problems, hindering deep learning.
method Careful symmetry breaking on training data helps solve inverse problems and improve deep learning performance.
result Symmetry breaking on training data significantly improves deep learning performance in inverse problems.
TensorWatch enables real-time interactive analysis of deep learning training.
problem Challenges in diagnosing and exploring deep learning models during training.
method Modeling inspection and diagnostic tasks as streams using a map-reduce paradigm.
result Real-time interactive queries on live deep learning training processes.
This work proposes a mathematical framework for loss landscapes and optimization in deep neural networks.
problem The effectiveness of gradient-based optimization in over-parameterized neural networks.
method A modern view and mathematical framework of loss landscapes and efficient optimization in over-parameterized machine learning models.
result Wide neural networks satisfy the PL∗ condition, explaining (S)GD convergence to a global minimum. Proposes a deep hybrid model for better recommendation systems.
problem Limited studies on hybrid recommender systems and the need for more advanced approaches.
method Integrates deep learning with ID embeddings and auxiliary features for improved recommendation.
result Improves recommendation results over deep learning models using ID embeddings.
Deep RL improves power control and scheduling for wireless multicast systems.
problem Scalable power control and scheduling for wireless multicast networks.
method Deep reinforcement learning with function approximation using a deep neural network.
result Deep RL can learn optimal power control policies for large systems.
A framework uses deep generative modeling to design metamaterials efficiently.
problem Designing metamaterials with complex properties is challenging due to high-dimensional design space and high computational cost.
method A variational autoencoder (VAE) and a regressor are trained on a large database to map microstructures to a latent space, enabling interpolation and manipulation of microstructures.
result The latent space provides a distance metric for shape similarity and encoding meaningful patterns of variation, enabling efficient design of microstructures and multiscale systems.
Computing equilibrium states in condensed-matter many-body systems, such as solvated proteins, is a long-standing challenge. Lacking methods for generating statistically independent equilibrium samples in "one shot", vast computational effort is invested for simulating these system in small steps, e.g., using Molecular…