GAN Lab helps non-experts learn GANs through interactive visualization.
problem Teaching complex deep learning models like GANs to non-experts.
method Interactive visualization tool integrating model structure and training dynamics.
result Users can interactively train and visualize GANs, understanding training dynamics.
Users create melodies with AI, harmonized in the style of Bach.
problem Making music composition accessible to non-experts.
method Simplified interface, machine learning model Coconet, optimized for web.
result Users spent 350 years worth of time playing with the Bach Doodle.
In this paper, we present a new wrapper feature selection approach based on Jensen-Shannon (JS) divergence, termed feature selection with maximum JS-divergence (FSMJ), for text categorization. Unlike most existing feature selection approaches, the proposed FSMJ approach is based on real-valued features which provide mo…
The paper introduces a new divergence measure for variational autoencoders to improve reconstruction and generation.
problem Balancing reconstruction and generalizability in latent space of variational autoencoders.
method Presented a regularisation mechanism based on skew-geometric Jensen-Shannon divergence.
result The skew-geometric Jensen-Shannon divergence leads to better reconstruction and generation in variational autoencoders.
Improved GANs estimate convergence rate for density estimation.
problem Improving the accuracy of density estimation with GANs.
method Proved an oracle inequality for JS divergence between GAN estimate and true density.
result JS-divergence rate of convergence is (logn/n)2β/(2β+d). The t-distributed Stochastic Neighbor Embedding (tSNE) algorithm has become in recent years one of the most used and insightful techniques for the exploratory data analysis of high-dimensional data. tSNE reveals clusters of high-dimensional data points at different scales while it requires only minimal tuning of its pa…
TensorFlow Probability introduces JointDistributions for probabilistic programming.
problem Specifying models in probabilistic programming languages.
method Declarative representations of directed graphical models.
result JointDistributions for TensorFlow Probability.
Google's Machine Learning framework TensorFlow was open-sourced in November 2015 [1] and has since built a growing community around it. TensorFlow is supposed to be flexible for research purposes while also allowing its models to be deployed productively. This work is aimed towards people with experience in Machine Lea…
Mesh-TensorFlow enables efficient deep learning on large clusters.
problem Memory constraints and inefficiency in batch-splitting for large models.
method Introduces Mesh-TensorFlow for specifying general tensor computations across a multi-dimensional mesh of processors.
result Trains Transformer models with up to 5 billion parameters on TPU meshes of up to 512 cores.
AdaNet automates ensemble learning with TensorFlow.
problem Automatic ensemble learning for high-quality models.
method Inspired by AdaNet algorithm, integrates TensorFlow ecosystem, offers flexible API.
result Efficiently accelerates training on various hardware.
This paper improves financial simulations using Tensor Processing Units and Tensorflow.
problem Estimating sensitivities in financial models efficiently.
method Utilizing Tensor Processing Units and Tensorflow for fast and automated differentiation.
result Single line of code for estimating sensitivities in financial models.
TF-GNN simplifies graph neural networks in TensorFlow.
problem Handling rich heterogeneous graph data in machine learning.
method A scalable library with a Keras message passing API.
result Enables low-code solutions for broader developers.
Implicit generative models are difficult to train as no explicit density functions are defined. Generative adversarial nets (GANs) present a minimax framework to train such models, which however can suffer from mode collapse due to the nature of the JS-divergence. This paper presents a learning by teaching (LBT) approa…
This paper considers the problem of estimating a high-dimensional vector of parameters θ∈Rn from a noisy observation. The noise vector is i.i.d. Gaussian with known variance. For a squared-error loss function, the James-Stein (JS) estimator is known to dominate the simple maximum-likelihood (…
TensorFI injects faults in TensorFlow programs to assess their reliability.
problem Ensuring reliability of machine learning systems in safety-critical domains.
method TensorFI is a flexible fault injection framework for TensorFlow applications.
result TensorFI evaluates the resilience of 12 ML programs, including autonomous vehicle DNNs.
TF-Coder simplifies tensor manipulation programming in TensorFlow.
problem Difficulty in programming with TensorFlow due to steep learning curve.
method Bottom-up weighted enumerative search with value-based pruning and type/value filtering.
result Solves 63 out of 70 real-world tasks within 5 minutes.
Trieste optimizes black-box functions using TensorFlow for efficient decision-making.
problem Optimizing complex, black-box functions efficiently.
method Bayesian optimization and active learning with TensorFlow integration.
result Trieste enables efficient exploration of black-box functions.
A probabilistic framework optimizes quantum clustering parameters.
problem Optimizing length parameters for quantum clustering sensitivity.
method Bayesian optimization of control parameters within a probabilistic framework.
result Optimized clustering yields better concordance with known data structure.
The TensorFlow Distributions library implements a vision of probability theory adapted to the modern deep-learning paradigm of end-to-end differentiable computation. Building on two basic abstractions, it offers flexible building blocks for probabilistic computation. Distributions provide fast, numerically stable metho…
TensorFlow Probability MCMC toolkit improves MCMC efficiency for modern hardware.
problem Inefficient MCMC algorithms on modern hardware.
method Design and implementation of a new MCMC toolkit for TensorFlow.
result Improved MCMC efficiency on modern hardware.
This paper improves NAQ method for faster convergence on Tensorflow.
problem Non-convex optimization problems.
method Modified Nesterov's Accelerated Quasi-Newton (NAQ) method on Tensorflow.
result mNAQ converges better and faster than first and second order optimizers.
AutoGraph improves Python coding for machine learning, combining ease and performance.
problem The trade-off between ease of writing and scalability in machine learning code.
method Source code transformation using staged programming in Python, delaying type-dependent decisions until runtime.
result Usability improvements with no performance loss compared to native TensorFlow graphs.
Spektral simplifies graph neural networks with TensorFlow and Keras.
problem Building graph neural networks efficiently.
method Open-source library implementing various graph neural network methods.
result Performance of implemented methods in node, graph, and graph regression tasks.
TF-Replicator simplifies distributed ML for researchers.
problem Complex distributed ML code for researchers.
method Abstraction over TensorFlow, data-parallel and model-parallel.
result Strong scalability without distributed systems expertise.
C-SURE improves complex-valued deep learning models by shrinking estimates, outperforming MLE and SurReal.
problem Improving accuracy and robustness of complex-valued deep learning models.
method Proposes a Stein's unbiased risk estimate (SURE) for complex-valued data and integrates it into a prototype CNN classifier.
result C-SURE outperforms SurReal and MLE in accuracy and robustness on complex-valued datasets.
Software tool enhances sampling efficiency in neural networks.
problem Efficiently sampling from posterior distributions in machine learning models.
method Ensemble quasi-Newton preconditioning method for Langevin dynamics.
result Significant improvement in sampling efficiency demonstrated.
Deep learning improves sensor performance optimization.
problem Optimizing sensor performance based on key metrics.
method Re-approach non-linear regression using deep learning with Keras and Tensorflow.
result Deep learning models improve sensor performance optimization.
Recent work has shown that Field-Programmable Gate Arrays (FPGAs) play an important role in the acceleration of Machine Learning applications. Initial specification of machine learning applications are often done using a high-level Python-oriented framework such as Tensorflow, followed by a manual translation to either…
GPflow is a Gaussian process library that uses TensorFlow for its core computations and Python for its front end. The distinguishing features of GPflow are that it uses variational inference as the primary approximation method, provides concise code through the use of automatic differentiation, has been engineered with…
TFCheck detects training issues in ML programs using TensorFlow.
problem Training programs often contain inconsistencies and bugs.
method Developed a TensorFlow library (TFCheck) with verification routines.
result TFCheck successfully detects training issues in ML code.
WDAIL uses Wasserstein distance for more effective reward shaping in IL.
problem Fixed reward functions in GAIL limit performance on complex tasks.
method Introduces Wasserstein distance and PPO for improved reward shaping and stability.
result Significant performance improvement in complex MuJoCo tasks.
TF Boosted Trees (TFBT) is a new open-sourced frame-work for the distributed training of gradient boosted trees. It is based on TensorFlow, and its distinguishing features include a novel architecture, automatic loss differentiation, layer-by-layer boosting that results in smaller ensembles and faster prediction, princ…
While accelerators such as GPUs have limited memory, deep neural networks are becoming larger and will not fit with the memory limitation of accelerators for training. We propose an approach to tackle this problem by rewriting the computational graph of a neural network, in which swap-out and swap-in operations are ins…
EagerPy simplifies writing code for multiple deep learning frameworks.
problem Code duplication and framework lock-in when using different deep learning libraries.
method Integrates multiple frameworks into a single Python framework.
result Automatic compatibility across PyTorch, TensorFlow, JAX, and NumPy.
Non-determinism from GPUs dominates ResNet training accuracy variability.
problem Variability in ResNet training accuracy due to GPU non-determinism.
method Analysis of TensorFlow ResNet training on GPUs, comparing fixed seeds vs. different seeds.
result 74% of ResNet model variability is due to GPU non-determinism.
Deep learning models can take weeks to train on a single GPU-equipped machine, necessitating scaling out DL training to a GPU-cluster. However, current distributed DL implementations can scale poorly due to substantial parameter synchronization over the network, because the high throughput of GPUs allows more data batc…
This paper compares machine learning models for pricing European options.
problem Pricing European options using traditional methods like Black Scholes Model.
method Google AutoML Regressor, TensorFlow Neural Networks, and XGBoost Gradient Boosting Decision Trees.
result All models outperformed the Black Scholes Model in terms of mean absolute error.
Federated Learning system for mobile devices.
problem Training models on decentralized data.
method High-level design of a scalable Federated Learning system.
result Solutions to challenges in federated learning.
Develops a convex duality framework for analyzing GANs.
problem Analyzing how GANs behave under different discriminator constraints.
method Introduces a convex duality framework to interpret GANs under constrained discriminators.
result Shows that the GAN formulation can be interpreted as minimizing a divergence to penalized moments of the data distribution.
tf_geometric simplifies graph deep learning in TensorFlow.
problem Efficient graph deep learning in TensorFlow.
method Kernel libraries and infrastructures for GNNs.
result tf_geometric supports various graph tasks and provides efficient GNN models.
The paper optimizes neural network inference on mobile GPUs.
problem Limited computing power and thermal constraints on mobile CPUs.
method Leverage mobile GPUs for neural network inference.
result Real-time inference of deep neural networks on Android and iOS devices.
Adds recursion to deep learning frameworks for better handling of recursive data structures.
problem Lack of support for recursion in existing deep learning frameworks.
method Complements existing frameworks with recursive execution of dataflow graphs and APIs for recursive definitions.
result Recursive implementation reduces training and inference time by more effectively using resources.
InferPy simplifies probabilistic modeling with deep neural networks in Python.
problem Complex probabilistic models with deep neural networks.
method User-friendly API for defining, learning, and evaluating models.
result Compact and simple way to define general hierarchical probabilistic models.
This paper improves neural network training performance by optimizing concurrency and operation scheduling.
problem Managing and scheduling fine-grained operations in neural network training for high performance.
method Extending TensorFlow runtime to enable automatic concurrency control and scheduling, using performance modeling.
result Achieved 33% average performance improvement on neural network models, up to 49%.
We developed a faster method for calculating neuron importance in neural networks.
problem Assigning importance to individual neurons in deep learning models.
method We developed Neuron Integrated Gradients, a scalable implementation of Total Conductance.
result Neuron Integrated Gradients is faster and empirically stronger than DeepLIFT.
Training modern deep learning models requires large amounts of computation, often provided by GPUs. Scaling computation from one GPU to many can enable much faster training and research progress but entails two complications. First, the training library must support inter-GPU communication. Depending on the particular …
A novel Python framework for Bayesian optimization known as GPflowOpt is introduced. The package is based on the popular GPflow library for Gaussian processes, leveraging the benefits of TensorFlow including automatic differentiation, parallelization and GPU computations for Bayesian optimization. Design goals focus on…
This paper studies backdoor attacks in federated learning and proposes defenses.
problem Detecting and defending against backdoor attacks in federated learning.
method Study of backdoor attacks and defenses on the EMNIST dataset, focusing on non-malicious clients.
result Norm clipping and weak differential privacy can mitigate backdoor attacks without significantly affecting overall performance.