Stan models are compiled to generative languages and extended with new features.
problem Lack of direct support for variational inference and deep models in Stan.
method Comprehensive compilation scheme to convert Stan models to generative languages, and extension of Stan with new features.
result NumPyro backend yields a 2.3x speedup compared to Stan in geometric mean over 26 benchmarks.
Tracr compiles programs into transformer models for interpretability.
problem Uncertainty in understanding transformer model outputs due to unknown learned programs.
method Tracr compiles human-readable programs into known structure transformer models.
result Known structure of Tracr-compiled models serves as ground-truth for interpretability.
CompILE learns reusable segments from demonstrations for hierarchical task execution.
problem Learning reusable, variable-length segments of hierarchical behavior from demonstrations.
method Unsupervised, fully-differentiable sequence segmentation module for latent encoding and re-composition.
result Model generalizes to longer sequences and unseen environments, learns task boundaries and event encodings.
Relay simplifies deep learning compilation across diverse hardware.
problem Challenges in extending DL frameworks to diverse models and hardware.
method Relay introduces a unified, expressive intermediate representation and extension mechanisms for hardware portability.
result Relay achieves competitive performance across various hardware platforms.
LIC compiles probabilistic models to generate efficient MCMC proposals.
problem Creating accurate Metropolis-Hastings proposals for Bayesian inference.
method Integrates probabilistic graphical models and neural networks in an open-source framework to optimize proposal distributions.
result LIC produces more efficient and robust MCMC proposals compared to existing methods.
Forward inference techniques such as sequential Monte Carlo and particle Markov chain Monte Carlo for probabilistic programming can be implemented in any programming language by creative use of standardized operating system functionality including processes, forking, mutexes, and shared memory. Exploiting this we have …
Julia programs and ML models are compiled and offloaded to Google Cloud TPUs.
problem Efficiently compiling and executing machine learning models on Google Cloud TPUs.
method Developed a method to compile and offload Julia programs and ML models to Google Cloud TPUs using the Google XLA compiler.
result Successfully fused a VGG19 model into a single TPU executable, achieving 0.23s evaluation time on a batch of 100 images.
Chameleon optimizes neural network compilation for faster execution and shorter time.
problem Faster execution and shorter compilation time for neural networks.
method Adaptive code optimization using reinforcement learning and adaptive sampling.
result 4.45x speed up in optimization time over AutoTVM, 5.6% improvement in inference time.
ReLeASE uses reinforcement learning and adaptive sampling to optimize neural network compilation.
problem Efficiently optimizing neural network compilation with shorter compilation time.
method Formulated as a reinforcement learning problem, with adaptive sampling to focus on representative points.
result 4.45x speed up in optimization time over AutoTVM, 5.6% improvement in inference time.
Deep RL optimizes compiler passes for better performance.
problem Designing optimal compiler optimization sequences is hard and often suboptimal.
method Employed deep reinforcement learning to learn optimal pass orderings.
result Achieved up to 1.32x speedup on previously unseen programs.
swTVM optimizes deep learning code for Sunway processors.
problem Efficiently compiling deep learning models for Sunway processors.
method Extends TVM to support Sunway, leveraging architecture features.
result 1.79x improvement on average across benchmarks.
XLA compiler extension improves memory efficiency for machine learning.
problem Memory constraints limit the scalability of memory-intensive machine learning algorithms.
method Developed an XLA compiler extension that adjusts algorithm data-flow representation to fit memory limits.
result k-nearest neighbour and sparse Gaussian process regression can be run at larger scales.
CINIC-10 combines CIFAR-10 and ImageNet images.
problem Creating a dataset similar to ImageNet but with fewer classes.
method Combining CIFAR-10 and downsampled ImageNet images.
result CINIC-10 provides a balanced dataset for model training.
Paper introduces HISA for efficient FHE computations.
problem Efficiently evaluating encrypted neural networks.
method Developed HISA for FHE applications, including compiler and runtime.
result Generated code is faster than hand-optimized implementations.
Myia compiler optimizes ML models with efficient AD for array programming.
problem Efficient automatic differentiation for array programming in ML.
method Introduces a new graph-based IR that supports function calls, higher-order functions, and recursion.
result Myia compiler enables efficient AD using source transformation without a tape, supporting higher-order derivatives.
Conference compiles problems on foliations and diffeomorphisms.
problem Challenges in foliations and diffeomorphism groups.
method Compilation of problems from conference participants.
result Compilation of 20+ problems on foliations and diffeomorphisms.
We introduce Dimple, a fully open-source API for probabilistic modeling. Dimple allows the user to specify probabilistic models in the form of graphical models, Bayesian networks, or factor graphs, and performs inference (by automatically deriving an inference engine from a variety of algorithms) on the model. Dimple a…
Improved inference in probabilistic programs using attention mechanisms.
problem Inference failure in existing IC network architectures due to long-range dependency issues.
method Inference compilation with attention mechanism to model latent variables.
result Attention mechanism enhances proposal distributions to better match true posterior.
Efficient FPGA virtualization for deep learning reduces user isolation and overhead.
problem Poor isolation and heavy re-compilation overhead in FPGA-based DNN accelerators.
method Two-level instruction dispatch module, multi-core hardware resources pool, tiling-based instruction frame package, two-stage static-dynamic compilation.
result 1.07-1.69x and 1.88-3.12x throughput improvement over previous designs.
It is time-consuming and error-prone to implement inference procedures for each new probabilistic model. Probabilistic programming addresses this problem by allowing a user to specify the model and having a compiler automatically generate an inference procedure for it. For this approach to be practical, it is important…
SPoC uses search to translate pseudocode into correct programs with error localization.
problem Mapping pseudocode to functionally correct long programs.
method Search-based approach guided by compilation errors for credit assignment.
result Search improves synthesis success rate from 25.6% to 44.7%.
MACER accelerates error repair by modularly identifying and applying fixes.
problem Automated compilation error repair for novice programmers.
method Modular segregation of repair process into identification and application, using discriminative learning techniques.
result MACER outperforms existing methods by 20% on popular errors and is competitive on all error types.
Paper optimizes GEMM for deep learning models, improving performance.
problem Limited GEMM optimization in deep learning frameworks restricts performance on different hardware.
method Proposes two novel algorithms: Greedy Best First Search (G-BFS) and Neighborhood Actor Advantage Critic (N-A2C) based on TVM framework.
result Significant performance improvements in GEMM computation time, achieving up to 40% savings.
We develop a technique for generalising from data in which models are samplers represented as program text. We establish encouraging empirical results that suggest that Markov chain Monte Carlo probabilistic programming inference techniques coupled with higher-order probabilistic programming languages are now sufficien…
The following is a compilation of some techniques in Alexandrov's geometry which are directly connected to convexity.
This survey was compiled from lectures and problem sessions at the International Conference on Geometric Topology at the Mathematical Research and Conference Center in Bedlewo, Poland in July 2005.
Specialized Deep Learning (DL) acceleration stacks, designed for a specific set of frameworks, model architectures, operators, and data types, offer the allure of high performance while sacrificing flexibility. Changes in algorithms, models, operators, or numerical systems threaten the viability of specialized hardware…
We study the problem of building generative models of natural source code (NSC); that is, source code written and understood by humans. Our primary contribution is to describe a family of generative models for NSC that have three key properties: First, they incorporate both sequential and hierarchical structure. Second…
Reinforcement learning optimizes neural network execution costs.
problem Minimizing execution cost of neural network computation graphs.
method Offline training of a reinforcement learning optimizer for generalization to unseen graphs.
result Significant improvements in execution time and memory usage compared to baselines.
This paper compiles and calculates triple point numbers for surface-links in Yoshikawa's table.
problem Determining the triple point number of surface-links in Yoshikawa's table.
method Using broken sheet diagrams, the paper compiles known triple point numbers and calculates or bounds the remaining ones.
result Compilation and calculation of triple point numbers for surface-links in Yoshikawa's table.
TVO tightens variational inference bounds for deep models.
problem Improving variational inference bounds for deep models.
method Introduces thermodynamic variational objective (TVO) connecting variational inference and thermodynamic integration.
result TVO provides tighter lower bound to log marginal likelihood than ELBO.
PFP-BNNs offer a fast, deterministic approach to Bayesian neural networks.
problem Limited uncertainty handling in traditional neural networks restricts their use in safety-critical settings.
method Probabilistic Forward Pass (PFP) approximates Stochastic Variational Inference (SVI) for efficient BNNs.
result PFP-BNNs achieve up to 4200x speedup over SVI-BNNs while maintaining similar accuracy and uncertainty.
We introduce a method for using deep neural networks to amortize the cost of inference in models from the family induced by universal probabilistic programming languages, establishing a framework that combines the strengths of probabilistic programming and deep learning methods. We call what we do "compilation of infer…
Abstract collects open problems in billiards and symplectic geometry.
problem Open problems in billiards and symplectic geometry.
method Compilation of open problems from discussions.
result Compilation of open problems.
ProGraML uses graph-based machine learning to improve program optimization and analysis.
problem Improving program optimization and analysis with machine learning.
method Low-level, language agnostic graph representation and message passing neural networks.
result ProGraML achieves an average 94.0 F1 score on a benchmark dataset, significantly outperforming state-of-the-art approaches.
Hybrid deep learning algorithm optimizes register allocation for compiler.
problem Efficiently coloring interference graphs for register allocation.
method Deep learning network trained on random graphs, augmented with a color correction phase.
result Hybrid algorithm performs well compared to optimal and greedy register allocators.
Paper proposes a faster method for evaluating DNN hardware and software designs.
problem Reducing time for evaluating different DNN hardware and software designs.
method Using virtual hardware models to estimate DNN performance at the concept phase.
result Up to 92% accuracy in predicting DNN inference processing time.
Predicting the number of clock cycles a processor takes to execute a block of assembly instructions in steady state (the throughput) is important for both compiler designers and performance engineers. Building an analytical model to do so is especially complicated in modern x86-64 Complex Instruction Set Computer (CISC…
A new kernel simplifies deep learning code and boosts performance.
problem Complex and hard-to-maintain DL codebases.
method Introducing a batch-reduce GEMM kernel for deep learning.
result Deep learning algorithms can be implemented with just 3K lines of code.
CoCoPIE shows AI can run on regular devices without special hardware.
problem Real-time AI on mobile devices without special hardware.
method Compression-compiler co-design using CoCoPIE framework.
result Real-time AI applications on off-the-shelf mobile devices.
Kjolstad et. al. proposed a tensor algebra compiler. It takes expressions that define a tensor element-wise, such as fij(a,b,c,d)=exp[−∑k=04((aik+bjk)2cii+di+k3)], and generates the corresponding compute kernel code. For machine learning, especially deep learni…
This paper compiles formulas involving differential operators and interior products.
problem Scattered identities in differential geometry involving various operators.
method Compilation and extension of formulas using the Schouten-Nijenhuis bracket and interior product.
result New formulas involving the de Rham codifferential and interior product.
Graph-based approach repairs programs from diagnostic feedback.
problem Learning to repair programs from limited labeled data and compiler error messages.
method Introduces program-feedback graph and graph neural network for reasoning, and self-supervised learning with unlabeled programs.
result DrRepair significantly outperforms prior work, achieving high repair rates.
A framework for privacy-preserving DNN pruning and acceleration.
problem Privacy concerns in DNN weight pruning for mobile devices.
method ADMM-based iterative pruning with synthetic data, compiler optimizations.
result 4.2X, 2.5X, and 2.0X speedup with almost no accuracy loss.
CodeX improves DNN acceleration on FPGAs by encoding and customizing bitwidth.
problem Efficiently accelerating deep neural networks on FPGAs with limited memory.
method Nonlinear encoding, automated bitwidth customization, FPGA streaming buffers.
result Average 4.65x throughput improvement on MNIST, SVHN, CIFAR-10.
Instead of requiring a domain expert to specify the probabilistic dependencies of the data, in this work we present an approach that uses the relational DB schema to automatically construct a Bayesian graphical model for a database. This resulting model contains customized distributions for columns, latent variables th…
Improved genetic programming by optimizing mutation operators for continuous program search.
problem Small syntactic mutations in genetic programming can lead to unpredictable behavioral shifts.
method Learned a compact trading-strategy DSL, created a block-factorized embedding, and designed geometry-compiled mutation operators.
result Geometry-compiled mutation operators discover strong strategies using fewer evaluations and achieve higher Sharpe ratios.
OpenEDS dataset captures eye images for VR applications.
problem Lack of high-quality eye image datasets for VR applications.
method Captured eye images using VR head-mounted display with synchronized cameras.
result Baseline experiment achieved 98.3% mIoU for pupil, iris, sclera segmentation.