Predicting the runtime complexity of a programming code is an arduous task. In fact, even for humans, it requires a subtle analysis and comprehensive knowledge of algorithms to predict time complexity with high fidelity, given any code. As per Turing's Halting problem proof, estimating code complexity is mathematically…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,694 papers · 148 categories
Trend · papers per month
48 results for “task codes”
Contrastive Code Representation Learning improves code summarization and type inference.
problem Code representations are sensitive to edits, hindering downstream semantic understanding tasks.
method ContraCode: a contrastive pre-training task that learns code functionality.
result Contrastive pre-training improves code summarization and type inference accuracy.
BERT-XML automates ICD coding from EHR notes using BERT pretraining.
problem Manual ICD coding is time-consuming and expensive in hospitals.
method Trained a BERT model on EHR notes, adapting it for multi-label attention.
result First large-scale ICD-10 classification model using millions of EHR notes.
Neural networks learn task-specific features, influenced by nonlinearity.
problem Understanding the nature of task-dependent feature learning in neural networks.
method Investigation of fully-connected, wide neural networks using Bayesian framework.
result The nature of internal representations depends on neuronal nonlinearity, leading to analog, redundant, or sparse coding schemes.
As the performance of computer systems stagnates due to the end of Moore's Law, there is a need for new models that can understand and optimize the execution of general purpose code. While there is a growing body of work on using Graph Neural Networks (GNNs) to learn representations of source code, these representation…
Sparse codes improve optimal control tasks with correlated inputs.
problem Optimal control tasks with correlated feature inputs.
method Used a sparse code to represent natural images in an optimal control task solved with neuro-dynamic programming.
result An over-complete sparse code increases memory capacity and learning speed beyond a complete code.
Single neural network learns multiple tasks from combined data.
problem Can a single neural network learn multiple unrelated tasks?
method Investigates how task representations affect joint learning; uses various task encoding methods.
result Single neural network can learn multiple tasks from combined data, even when tasks are unrelated and different.
Study investigates predictive coding models for phonemic learning.
problem Understanding how predictive coding models generalize to different languages and dataset sizes.
method Investigated Autoregressive Predictive Coding and Contrastive Predictive Coding models in phoneme discrimination tasks for two languages with varying dataset sizes.
result Contrastive Predictive Coding model converges rapidly and outperforms Autoregressive Predictive Coding on both languages.
Semantic code search is the task of retrieving relevant code given a natural language query. While related to other information retrieval tasks, it requires bridging the gap between the language used in code (often abbreviated and highly technical) and natural language more suitable to describe vague concepts and ideas…
Many software analysis methods have come to rely on machine learning approaches. Code segmentation - the process of decomposing source code into meaningful blocks - can augment these methods by featurizing code, reducing noise, and limiting the problem space. Traditionally, code segmentation has been done using syntact…
Machine learning models that take computer program source code as input typically use Natural Language Processing (NLP) techniques. However, a major challenge is that code is written using an open, rapidly changing vocabulary due to, e.g., the coinage of new variable and method names. Reasoning over such a vocabulary i…
The ability to generate natural language sequences from source code snippets has a variety of applications such as code summarization, documentation, and retrieval. Sequence-to-sequence (seq2seq) models, adopted from neural machine translation (NMT), have achieved state-of-the-art performance on these tasks by treating…
The abundance of open-source code, coupled with the success of recent advances in deep learning for natural language processing, has given rise to a promising new application of machine learning to source code. In this work, we explore the use of a Siamese recurrent neural network model on Python source code to create …
Machine learning and deep learning in particular has been recently used to successfully address many tasks in the domain of code such as finding and fixing bugs, code completion, decompilation, type inference and many others. However, the issue of adversarial robustness of models for code has gone largely unnoticed. In…
While sparse coding-based clustering methods have shown to be successful, their bottlenecks in both efficiency and scalability limit the practical usage. In recent years, deep learning has been proved to be a highly effective, efficient and scalable feature learning tool. In this paper, we propose to emulate the sparse…
For the task of generating complex outputs such as source code, editing existing outputs can be easier than generating complex outputs from scratch. With this motivation, we propose an approach that first retrieves a training example based on the input (e.g., natural language description) and then edits it to the desir…
Unlike traditional programs (such as operating systems or word processors) which have large amounts of code, machine learning tasks use programs with relatively small amounts of code (written in machine learning libraries), but voluminous amounts of data. Just like developers of traditional programs debug errors in the…
CORAL: COde RepresentAtion Learning with Weakly-Supervised Transformers for Analyzing Data Analysiscs.LG
Model learns code representations from comments for data analysis tasks.
problem Lack of descriptive labels for analyzing large code corpora.
method Weakly supervised transformer architecture for joint code and comment representation.
result Model achieves 38% accuracy increase over expert-supplied heuristics.
CodNN uses error-correcting codes to make neural networks more resilient to noise.
problem Neural networks are sensitive to noise, especially in critical applications.
method Construct robust neural networks by coding data or internal layers with error-correcting codes.
result Parity codes can guarantee robustness for a wide range of neural networks, including binarized networks.
New algorithm for active learning from feedback coding.
problem Efficiently selecting examples for labeling in active machine learning.
method Formalized structural similarities between active learning and feedback channel coding, developed an optimal transport-based algorithm called Approximate Posterior Matching (APM).
result Learning performance comparable to existing methods at reduced computational cost.
Coded Federated Learning speeds up training in edge computing networks.
problem Slow convergence in Federated Learning due to heterogeneity and stochastic fluctuations.
method Exploiting statistical properties of compute and communication delays, distributed kernel embedding, and random Fourier features.
result Significant performance gains for CodedFedL in distributed non-linear regression and classification problems.
Transformer models waste resources on long-context tasks.
problem Redundant attention computations in Transformer models for long-context tasks.
method Reformulate sequence modeling as supervised learning, analyze attention sparsity, formulate attention optimization as linear coding problem, propose Dynamic Group Attention.
result DGA reduces computational costs while maintaining performance.
Convex Codingcs.LG
Inspired by recent work on convex formulations of clustering (Lashkari & Golland, 2008; Nowozin & Bakir, 2008) we investigate a new formulation of the Sparse Coding Problem (Olshausen & Field, 1997). In sparse coding we attempt to simultaneously represent a sequence of data-vectors sparsely (i.e. sparse approximation (…
Transformer model improves source code summarization.
problem Generating readable summaries of source code.
method Transformer model with self-attention mechanism for code representation.
result Transformer model outperforms state-of-the-art techniques.
Code-switching, the alternation of languages within a conversation or utterance, is a common communicative phenomenon that occurs in multilingual communities across the world. This survey reviews computational approaches for code-switched Speech and Natural Language Processing. We motivate why processing code-switched …
Study re-evaluates MIMIC-III codes, finding many are under-coded.
problem Validity of MIMIC-III clinical codes is questionable.
method Open-source, reproducible methodology for assessing codes.
result Most frequently assigned codes are under-coded up to 35%
In recent years, deep learning models have shown great potential in source code modeling and analysis. Generally, deep learning-based approaches are problem-specific and data-hungry. A challenging issue of these approaches is that they require training from starch for a different related problem. In this work, we propo…
With the recent success of embeddings in natural language processing, research has been conducted into applying similar methods to code analysis. Most works attempt to process the code directly or use a syntactic tree representation, treating it like sentences written in a natural language. However, none of the existin…
Tasks like code generation and semantic parsing require mapping unstructured (or partially structured) inputs to well-formed, executable outputs. We introduce abstract syntax networks, a modeling framework for these problems. The outputs are represented as abstract syntax trees (ASTs) and constructed by a decoder with …
Sparse coding has shown its power as an effective data representation method. However, up to now, all the sparse coding approaches are limited within the single domain learning problem. In this paper, we extend the sparse coding to cross domain learning problem, which tries to learn from a source domain to a target dom…
MISIM improves code similarity systems with neural learning.
problem Improving accuracy of code semantics similarity systems.
method Context-aware semantics structure and neural code similarity scoring algorithm.
result MISIM achieves 8.08% better accuracy than state-of-the-art systems.
PCGs encompass a broader range of neural networks.
problem Understanding the broader scope of neural network models.
method Proving PCGs as a superset of feedforward neural networks.
result PCGs represent a wider class of neural network models.
In order to submit a claim to insurance companies, a doctor needs to code a patient encounter with both the diagnosis (ICDs) and procedures performed (CPTs) in an Electronic Health Record (EHR). Identifying and applying relevant procedures code is a cumbersome and time-consuming task as a doctor has to choose from arou…
New seq2seq model can copy entire spans, outperforming simpler models in editing tasks.
problem Editing documents or source code using seq2seq models with explicit token copying.
method Extended seq2seq model capable of copying entire input spans to output in one step, new training and inference methods.
result New model consistently outperforms simpler baselines in editing tasks of natural language and source code.
Effective representation learning of electronic health records is a challenging task and is becoming more important as the availability of such data is becoming pervasive. The data contained in these records are irregular and contain multiple modalities such as notes, and medical codes. They are preempted by medical co…
A new model for sequential memory using temporal predictive coding.
problem Forming accurate memory of sequential stimuli in the brain.
method Proposes a novel PC-based model called temporal predictive coding (tPC).
result Shows that tPC models can accurately memorize and retrieve sequential inputs.
We address the problem of any-code completion - generating a missing piece of source code in a given program without any restriction on the vocabulary or structure. We introduce a new approach to any-code completion that leverages the strict syntax of programming languages to model a code snippet as a tree - structural…
Sparse coding has been popularly used as an effective data representation method in various applications, such as computer vision, medical imaging and bioinformatics, etc. However, the conventional sparse coding algorithms and its manifold regularized variants (graph sparse coding and Laplacian sparse coding), learn th…
Program comprehension is a fundamental task in software development and maintenance processes. Software developers often need to understand a large amount of existing code before they can develop new features or fix bugs in existing programs. Being able to process programming language code automatically and provide sum…
We investigate the use of sparse coding and dictionary learning in the context of multitask and transfer learning. The central assumption of our learning method is that the tasks parameters are well approximated by sparse linear combinations of the atoms of a dictionary on a high or infinite dimensional space. This ass…
A new approach predicts next observations without explicit decoding for better control.
problem High-dimensional observations and unknown dynamics in real-world control tasks.
method Proposes a novel information-theoretic LCE approach using predictive coding to develop a decoder-free model.
result The model reliably learns a controllable latent space leading to superior performance.
Distributed gradient descent (DGD) is an efficient way of implementing gradient descent (GD), especially for large data sets, by dividing the computation tasks into smaller subtasks and assigning to different computing servers (CSs) to be executed in parallel. In standard parallel execution, per-iteration waiting time …
The goal of predictive sparse coding is to learn a representation of examples as sparse linear combinations of elements from a dictionary, such that a learned hypothesis linear in the new representation performs well on a predictive task. Predictive sparse coding algorithms recently have demonstrated impressive perform…
This paper develops coding techniques to reduce the running time of distributed learning tasks. It characterizes the fundamental tradeoff to compute gradients (and more generally vector summations) in terms of three parameters: computation load, straggler tolerance and communication cost. It further gives an explicit c…
In several natural language tasks, labeled sequences are available in separate domains (say, languages), but the goal is to label sequences with mixed domain (such as code-switched text). Or, we may have available models for labeling whole passages (say, with sentiments), which we would like to exploit toward better po…
Finding optimal correction of errors in generic stabilizer codes is a computationally hard problem, even for simple noise models. While this task can be simplified for codes with some structure, such as topological stabilizer codes, developing good and efficient decoders still remains a challenge. In our work, we syste…
Dynamical system models (including RNNs) often lack the ability to adapt the sequence generation or prediction to a given context, limiting their real-world application. In this paper we show that hierarchical multi-task dynamical systems (MTDSs) provide direct user control over sequence generation, via use of a latent…
A new predictive coding algorithm improves machine learning performance.
problem Improving performance of variational inference algorithms in high-dimensional, structured problems.
method Divide-and-Conquer Predictive Coding (DCPC) that respects generative model correlation structure and performs maximum-likelihood updates.
result DCPC achieves better numerical performance and accurate inference in structured generative models.