Transformer model improves source code summarization.
problem Generating readable summaries of source code.
method Transformer model with self-attention mechanism for code representation.
result Transformer model outperforms state-of-the-art techniques.
Paper presents a code authorship attribution attack using adversarial learning.
problem Misleading attribution of source code using machine learning methods.
method Exploits adversarial examples and semantics-preserving code transformations guided by Monte-Carlo tree search.
result Demonstrates substantial effect on attribution methods, reducing accuracy from over 88% to 1%.
Tangent automates derivatives in Python, improving expressiveness and performance.
problem Efficiently calculating derivatives for complex models in Python.
method Source-code transformation for dynamically typed array programming.
result Demonstrates improved expressiveness and performance in automatic differentiation.
Tree-Transformer improves grammar correction in code and natural language.
problem Grammar correction in tree-structured data.
method Tree-Transformer neural network architecture for tree-structured data.
result Significant improvement in grammar correction accuracy.
Study makes code models robust to small changes that keep functionality.
problem Vulnerability of deep neural networks to adversarial examples in source code.
method Defined a powerful adversary and adversarial training to learn robust models.
result Significant gains in robustness demonstrated across different languages and architectures.
Automatic differentiation (AD) is an essential primitive for machine learning programming systems. Tangent is a new library that performs AD using source code transformation (SCT) in Python. It takes numeric functions written in a syntactic subset of Python and NumPy as input, and generates new Python functions which c…
COSET benchmarks neural program embeddings using diverse source-code datasets.
problem Evaluating neural program embeddings is challenging due to lack of straightforward metrics.
method COSET framework with labeled programs, transformations, and a pilot study.
result COSET identifies strengths and weaknesses of neural models and program characteristics.
Kymatio simplifies scattering transforms for Python.
problem Signal processing and machine learning applications.
method Wavelet scattering transform implemented in Python.
result Efficient, GPU-accelerated implementation.
DiffTaichi enables fast, differentiable physical simulations with shorter code.
problem Building efficient differentiable physical simulators.
method Differentiable programming language (DiffTaichi) that generates gradients using source code transformations and a light-weight tape.
result Differentiable physical simulators written in DiffTaichi are faster and more concise than existing methods.
Model learns code representations from comments for data analysis tasks.
problem Lack of descriptive labels for analyzing large code corpora.
method Weakly supervised transformer architecture for joint code and comment representation.
result Model achieves 38% accuracy increase over expert-supplied heuristics.
Improved source code summarization using extended Tree-LSTM.
problem Challenges in applying LSTM to structured source code.
method Extended Tree-LSTM for abstract syntax trees (ASTs).
result Multi-way Tree-LSTM achieves better results than state-of-the-art techniques.
AutoGraph improves Python coding for machine learning, combining ease and performance.
problem The trade-off between ease of writing and scalability in machine learning code.
method Source code transformation using staged programming in Python, delaying type-dependent decisions until runtime.
result Usability improvements with no performance loss compared to native TensorFlow graphs.
Signatory calculates signature and logsignature transforms efficiently on CPU and GPU.
problem Efficient computation of signature and logsignature transforms for machine learning.
method CPU and GPU parallelism, backpropagation, efficient precomputation strategies, algorithmic improvements.
result Substantial speedups on CPU and GPU, including real-world applications.
Improved hybrid acoustic model using interleaved self-attention and convolution.
problem Limited application of transformer in hybrid acoustic models.
method Proposed a model structure with interleaved self-attention and 1D convolution.
result Competitive recognition results on Librispeech dataset.
Deep neural network predicts semantic labels for source code.
problem Difficulty in labeling and understanding new programming languages and functionalities.
method Language-agnostic deep convolutional neural network trained on Stack Overflow code snippets.
result Mean area under ROC of 0.957 and top-1 accuracy of 86.6% on Github code documents.
New algorithm for uncertain time series classification.
problem Uncertainty in time series data.
method Uncertain dissimilarity measure based on Euclidean distance and uncertain shapelet transform.
result Effectiveness of the uncertain shapelet transform algorithm on state-of-the-art datasets.
Wavelets help compress neural networks efficiently.
problem Efficiently compressing linear layers in neural networks.
method Learnable wavelet transforms to compress RNNs.
result Wavelet compressed RNNs have fewer parameters and perform competitively.
Survey on embedding techniques in source code.
problem Applying word embedding techniques to source code.
method Collection and categorization of articles from related work and scholarly searches.
result Word embedding has been successfully applied to various granularities of source code.
ConfFlow uses transformer networks to generate molecular conformations efficiently.
problem Efficient generation of valid conformations for large molecules.
method Flow-based model using transformer networks that directly samples in coordinate space.
result ConfFlow improves accuracy by up to 40% for large molecule conformations.
Paper explores understanding of neural source code embeddings.
problem Lack of understanding of contents and characteristics of code2vec embeddings.
method Small case study using code2vec embeddings to create binary SVM classifiers and compare performance with handcrafted features.
result Code2vec embeddings perform similarly to handcrafted features and have more evenly distributed information gains.
New model creates code semantics vectors for better understanding.
problem Improving code understanding and embedding quality.
method Siamese recurrent neural network on Python source code.
result Model significantly outperforms bag-of-tokens embeddings.
Backdoors can be implanted in neural models of source code, and we detect and remove them.
problem Vulnerability of neural models to backdoors in source code.
method Defined and implemented various backdoor classes, adapted robust statistics algorithms, and detected poisoned data through spectral signatures.
result Demonstrated the ease of injecting and removing backdoors in neural models of source code.
Neural networks model code edits from source code changes.
problem Modeling dynamic changes in source code.
method Developed neural networks to learn and predict code edits.
result Attentional and pointer network components provide best performance.
Paper shows semi-supervised clustering is equivalent to a type of source coding.
problem Multiclass labeling of unlabeled elements with noisy binary queries.
method Locally encodable source coding approach, using pairwise queries.
result Lower bounds on the number of queries required for correct labeling.
Transfer learning boosts deep learning in source code modeling.
problem Deep learning models are problem-specific and data-hungry.
method Transfer learning to improve performance of deep learning models.
result The proposed approach outperforms state-of-the-art models in source code suggestion task.
Equity-Transformer solves NP-hard min-max routing problems efficiently.
problem Min-max routing problems with multiple agents and large-scale applications.
method Sequential planning approach with Transformer and equitable workload distribution inductive biases.
result Significant runtime and cost reductions in min-max mTSP and min-max mPDP tasks.
Algorithm improves query recommendations with immediate user feedback.
problem Lack of adaptability to immediate user feedback in query recommendation algorithms.
method Augmented transformer-based causal language models with multi-armed bandit framework.
result Substantial improvement in per-round regret compared to state-of-the-art models.
Proposes a tabular transformer model to maintain feature effect intelligibility.
problem Losing marginal feature effects in deep tabular transformer networks.
method Adapts tabular transformer networks to identify marginal feature effects.
result The model accurately identifies marginal feature effects, matching black-box performance while maintaining intelligibility.
We study the problem of building generative models of natural source code (NSC); that is, source code written and understood by humans. Our primary contribution is to describe a family of generative models for NSC that have three key properties: First, they incorporate both sequential and hierarchical structure. Second…
Convolutional neural network summarizes code comments across multiple languages.
problem Insufficient or missing comments in source code.
method Language-agnostic encoder-decoder model with open vocabulary.
result Comparable results to state-of-the-art on single-language data; first results on multi-language data.
We present the first general purpose framework for marginal maximum a posteriori estimation of probabilistic program variables. By using a series of code transformations, the evidence of any probabilistic program, and therefore of any graphical model, can be optimized with respect to an arbitrary subset of its sampled …
Develops deep learning for logical code segmentation.
problem Lack of logically segmented source code.
method Novel deep learning approach to generate logical code segments.
result Improves software analysis tasks like commenting, bug detection, and code synthesis.
Improves code2vec for Java classes by obfuscating variable names.
problem Code2vec's reliance on variable names makes it vulnerable to typos and attacks.
method Obfuscate variable names during code2vec training and aggregate method embeddings for class-level predictions.
result Obfuscated variable names improve model's robustness and accuracy.
Increasing numbers of software vulnerabilities are discovered every year whether they are reported publicly or discovered internally in proprietary code. These vulnerabilities can pose serious risk of exploit and result in system compromise, information leaks, or denial of service. We leveraged the wealth of C and C++ …
CRCCA framework improves non-linear CCA with compressed representations.
problem Non-linear CCA for multi-view data with limited samples.
method Information-theoretic compressed representation framework (CRCCA) based on lattice quantization.
result The CRCCA framework provides theoretical bounds and optimality conditions, offering a flexible and computationally efficient solution.
This work proposes a meta-learning approach for better adaptation of source code models.
problem Adapting source code models to unseen local contexts.
method Formulated as a meta-learning problem, selecting targeted information for adaptation.
result Improved performance in code auto-completion tasks, especially for identifiers and literals.
Unsupervised method constructs knowledge graph from text and code.
problem Lack of structured knowledge in scientific literature and code.
method Word embedding, clustering, and dimensionality reduction techniques.
result Enhanced understanding of scientific literature and code.
Imputation-free method learns tabular data with missing values using transformer.
problem Machine learning on tabular data with missing values often leads to unreliable outcomes due to synthetic imputation.
method Incremental attention learning (IFIAL) using transformer with attention masks.
result IFIAL outperforms state-of-the-art methods in 17 diverse tabular data sets.
This paper studies communication efficiency in federated learning by optimizing the sum-rate-distortion function for indirect multiterminal source coding.
problem Indirect multiterminal source coding in federated learning where edge devices send noisy gradients to the server.
method Analyzes the rate region for the quadratic vector Gaussian CEO problem under unbiased estimator and derives an explicit formula for the sum-rate-distortion function.
result Derives an explicit formula for the sum-rate-distortion function in the special case of identical gradients over edge devices.
We provide complete source code for building a fundamental industry classification based on publically available and freely downloadable data. We compare various fundamental industry classifications by running a horserace of short-horizon trading signals (alphas) utilizing open source heterotic risk models (https://ssr…
This study evaluates feature scaling across 14 datasets and 12 techniques in ML.
problem Impact of feature scaling on machine learning performance and computational costs.
method Systematic evaluation of 12 scaling techniques across 14 datasets and 16 ML algorithms.
result Wide variation in model performance due to feature scaling, especially for non-ensemble models.
MethaneMapper detects methane emissions with high accuracy and reduced model size.
problem Challenges in detecting and quantifying methane emissions from AVIRIS-NG data.
method Spectral absorption wavelength aware transformer network, introducing two novel modules.
result Achieves 0.63 mAP in detection and reduces model size by 5x.
DeepRC uses Hopfield networks and attention to classify immune repertoires.
problem Classifying the vast number of immunosequences of an individual.
method Integrates transformer-like attention into deep learning architectures.
result DeepRC outperforms other methods in predictive performance.
SCC identifies 21 programming languages with 75% accuracy.
problem Classifying code snippets among 21 programming languages.
method Multinomial Naive Bayes classifier trained on Stack Overflow posts.
result SCC achieves 75% accuracy, significantly higher than PLI.
New model learns execution of code using GNNs.
problem Stagnation of computer system performance due to Moore's Law.
method Multi-task GNN over low-level code and program state.
result Improved performance on dynamic tasks (26% and 45% over state-of-the-art).
A framework and benchmark for deep batch active learning in neural networks.
problem Efficiently acquiring labels for neural network regression.
method Framework of base kernels, transformations, and selection methods; use of sketched finite-width neural tangent kernels and clustering.
result Proposed method outperforms state-of-the-art on benchmark, scales to large data sets.
We give complete algorithms and source code for constructing statistical risk models, including methods for fixing the number of risk factors. One such method is based on eRank (effective rank) and yields results similar to (and further validates) the method set forth in an earlier paper by one of us. We also give a co…
New seq2seq model can copy entire spans, outperforming simpler models in editing tasks.
problem Editing documents or source code using seq2seq models with explicit token copying.
method Extended seq2seq model capable of copying entire input spans to output in one step, new training and inference methods.
result New model consistently outperforms simpler baselines in editing tasks of natural language and source code.