Deep learning improves decoding of constrained sequence codes, reducing errors and increasing throughput.
problem Errors during transmission of constrained sequence codes.
method Deep learning, specifically MLP and CNN networks.
result Achieved low bit error rates close to MAP decoding and improved system throughput.
Paper studies distributed learning with limited communication bits, achieving optimal error exponents.
problem Distributed hypothesis testing with constant communication bits.
method Geometric approach in distribution spaces, encoding empirical distributions to transmission bits.
result Optimal achievable error exponents and coding schemes for various communication constraints.
Deep generative models have been successfully used to learn representations for high-dimensional discrete spaces by representing discrete objects as sequences and employing powerful sequence-based deep models. Unfortunately, these sequence-based models often produce invalid sequences: sequences which do not represent a…
A new method optimizes slicing directions for SW distances to improve high-dimensional probability measure comparison.
problem Challenging identification of informative slicing directions for SW distances.
method Constrained learning approach to optimize slicing directions, using continuous relaxations and gradient-based primal-dual approach.
result Demonstrated efficacy in learning more informative slicing directions on various high-dimensional data.
MTDS improves sequence generation adaptability via latent code control.
problem Lack of adaptability in sequence generation models like RNNs.
method Hierarchical multi-task dynamical systems (MTDS) with latent code control.
result MTDS enables style transfer, interpolation, and morphing in generated sequences.
CODE2SEQ generates natural language sequences from code snippets.
problem Generating natural language descriptions from code.
method CODE2SEQ represents code as AST paths and uses attention to select relevant paths.
result CODE2SEQ outperforms previous models for code-to-text tasks.
SequenceR uses seq-to-seq learning to fix bugs in code.
problem Fixing bugs in code using machine learning.
method Sequence-to-sequence learning with copy mechanism, trained on curated code samples.
result SequenceR can perfectly predict and find correct patches for bugs.
Generative model learns compact codes for video recovery.
problem Efficiently represent and reconstruct videos from missing data.
method Generative network trained to map compact latent codes to images, with low-rank and similarity constraints.
result Can recover true video sequences even if not in pretrained network's range.
We introduce new definitions of universal and superuniversal computable codes, which are based on a code's ability to approximate Kolmogorov complexity within the prescribed margin for all individual sequences from a given set. Such sets of sequences may be singled out almost surely with respect to certain probability …
We propose using canonical correlation analysis (CCA) to generate features from sequences of medical billing codes. Applying this novel use of CCA to a database of medical billing codes for patients with diverticulitis, we first demonstrate that the CCA embeddings capture meaningful relationships among the codes. We th…
A new neural sequence prediction method using error-correcting codes improves accuracy and convergence.
problem Improving neural sequence prediction accuracy and speed.
method Error-correcting output codes, separable code maximization, latent variable mixture sampling.
result Consistent improvements on language modeling datasets and text generation tasks.
AI generates a sequence of death causes from hospital records.
problem Accurate death reporting for vital statistics and policy formulation.
method Neural machine translation models to generate causal chains, incorporating medical domain knowledge.
result Achieved 16.04 BLEU score for generating accurate causal chains.
MedGraph learns patient visit embeddings from EMRs, capturing both attributes and temporal sequences.
problem Limited EMR embedding methods fail to capture patient demographics, utilisation, and code descriptions.
method MedGraph constructs an attributed bipartite graph and uses a point process to model temporal sequences.
result MedGraph outperforms state-of-the-art methods in medical risk prediction tasks.
A framework that edits existing examples to generate structured outputs.
problem Generating complex outputs like code is hard.
method Retrieve a training example, edit it, and use a computationally efficient retrieval model.
result Retrieve-and-edit framework significantly boosts performance on code and card tasks.
skscope simplifies sparsity-constrained optimization in Python.
problem Tedious mathematical deduction and programming for sparsity-constrained optimization.
method Introduces skscope, a Python library that allows users to solve sparsity-constrained optimization problems by just programming the objective function.
result skscope enables state-of-the-art solvers to quickly attain sparse solutions in high-dimensional spaces, achieving up to 80x speedup.
SWIFT improves time series forecasting on edge devices with wavelet decomposition.
problem Efficient time series forecasting on resource-constrained devices.
method Wavelet decomposition, cross-band fusion, and shared linear mapping.
result SWIFT achieves state-of-the-art performance on multiple datasets.
Develops a method to optimize tax codes with practical constraints.
problem Translating optimal taxation theory into practical tax codes.
method Constrained optimization framework for piecewise linear tax functions.
result Generates reforms that meet theoretical and practical constraints.
New method learns flows between multiple distributions efficiently.
problem Learning dynamic transport maps between multiple empirical distributions.
method Combining flow matching and dynamic optimal transport with potential terms.
result OTP-FM achieves state-of-the-art performance on various datasets.
Paper uses RL to solve constrained combinatorial optimization problems.
problem Constrained combinatorial optimization problems.
method Extending NCO theory to CMDPs, using RL with penalty signals.
result Proposes a superior method for solving constrained problems.
Three physics-constrained regression exercises for image velocimetry and turbulence modeling.
problem Image velocimetry and turbulence modeling challenges.
method Physics-constrained regression exercises implemented as toy problems.
result Python codes provided for all exercises.
The paper tackles gradual domain adaptation with manifold-constrained DRO, showing error bounds across distributions.
problem Gradual domain adaptation challenge with manifold-constrained data distributions.
method Distributionally Robust Optimization (DRO) with an adaptive Wasserstein radius.
result Theoretical bounds on classification error across distributions, demonstrating error propagation dynamics.
GIRNet tackles multi-tasking with mixed domain sequences, improving sentiment and tagging tasks.
problem Labeling sequences with mixed domain data and position-specific inference.
method Unified position-sensitive multi-task RNN architecture with gated state sequences from auxiliary data.
result GIRNet achieves new state-of-the-art performance in sentiment classification, POS tagging, and target position-sensitive annotation.
This work tackles sparse coding in DLRA for interpretable multiway data.
problem Sparse coding in DLRA for interpretable multiway data.
method Proposes a new sparse-coding subproblem (MSC) and several algorithms to solve it.
result DLRA extends low-rank approximations, reducing variance and enhancing interpretability.
Paper improves MIRACLE for faster, more robust neural network compression.
problem Efficiently compressing neural networks while maintaining performance.
method Introduces Mean-KL parameterization to constrain compression cost.
result Mean-KL parameterization leads to twice as fast convergence and more robust compression.
Deep RL optimizes compiler passes for better performance.
problem Designing optimal compiler optimization sequences is hard and often suboptimal.
method Employed deep reinforcement learning to learn optimal pass orderings.
result Achieved up to 1.32x speedup on previously unseen programs.
Study on predicting sequences with Gaussian constraints, linking to intrinsic volumes and metric complexity.
problem Predicting sequences almost as well as the best Gaussian distribution with mean in a given subset.
method Expressed minimax regret in terms of intrinsic volumes, established comparison inequality for Wills functional, characterized global covering numbers and local Gaussian widths.
result Sharp estimates on the log-Laplace transform of intrinsic volume sequence for a general nonconvex set.
PLIT identifies plant lncRNAs from RNA-seq data with high accuracy.
problem Inaccurate identification of lncRNAs in plant transcriptomic datasets.
method PLIT uses L1 regularization and iRF classification to select optimal features from sequence and codon-bias data.
result PLIT outperforms existing CPC tools in identifying lncRNAs in plant RNA-seq datasets.
Study shows limits of volume-constrained sets are finite unions of Wulff shapes.
problem Analyzing the behavior of sets with degenerating ellipticity.
method Proving rigidity of L1-accumulation points of volume-constrained almost-critical sets. result Limits of volume-constrained sets are finite unions of φ-Wulff shapes. Study makes code models robust to small changes that keep functionality.
problem Vulnerability of deep neural networks to adversarial examples in source code.
method Defined a powerful adversary and adversarial training to learn robust models.
result Significant gains in robustness demonstrated across different languages and architectures.
A new method for optimizing non-decomposable metrics with constraints.
problem Optimizing complex machine learning objectives with thresholded constraints.
method Formulate rate-constrained optimization using the Implicit Function theorem and solve with gradient-based methods.
result Demonstrated effectiveness over existing methods on benchmark datasets.
New seq2seq model can copy entire spans, outperforming simpler models in editing tasks.
problem Editing documents or source code using seq2seq models with explicit token copying.
method Extended seq2seq model capable of copying entire input spans to output in one step, new training and inference methods.
result New model consistently outperforms simpler baselines in editing tasks of natural language and source code.
Optimal control problems on Riemannian manifolds are solved by penalizing constraint violations.
problem Optimal control problems with velocity constraints on Riemannian manifolds.
method Penalizing constraint violations and showing convergence to hard-constrained solutions.
result Solutions to soft-constrained problems converge to solutions of hard-constrained problems as penalty parameter increases.
A statistical physics model for the time evolutions of stock portfolios is proposed. In this model the time series of price changes are coded into the sequences of up and down spins. The Hamiltonian of the system is introduced and is expressed by spin-spin interactions as in spin glass models of disordered magnetic sys…
Inspired by recent work on convex formulations of clustering (Lashkari & Golland, 2008; Nowozin & Bakir, 2008) we investigate a new formulation of the Sparse Coding Problem (Olshausen & Field, 1997). In sparse coding we attempt to simultaneously represent a sequence of data-vectors sparsely (i.e. sparse approximation (…
We compare alternative computing strategies for solving the constrained lasso problem. As its name suggests, the constrained lasso extends the widely-used lasso to handle linear constraints, which allow the user to incorporate prior information into the model. In addition to quadratic programming, we employ the alterna…
We introduce the problem of reconstructing a sequence of multidimensional real vectors where some of the data are missing. This problem contains regression and mapping inversion as particular cases where the pattern of missing data is independent of the sequence index. The problem is hard because it involves possibly m…
Graph ConvNet improves ncRNA classification accuracy.
problem Classifying non-coding RNA sequences into families.
method Graph Convolutional Network model trained on raw RNA graphs.
result 85.73% accuracy and 85.61% F1-score over 13 classes.
Tree-Transformer improves grammar correction in code and natural language.
problem Grammar correction in tree-structured data.
method Tree-Transformer neural network architecture for tree-structured data.
result Significant improvement in grammar correction accuracy.
End-to-end learning of codes for secure BPSK communication in Gaussian wiretap channel.
problem Secure communication in the presence of an adversary for BPSK modulation.
method Learning finite-length codes using deep neural networks (DNNs) with mutual information estimation.
result Learned codes achieve near-optimal security performance, as demonstrated by numerical results.
DQA efficiently quantizes deep neural network activations for resource-constrained devices.
problem Efficiently quantizing deep neural network activations for resource-constrained devices.
method DQA uses simple shifting-based operations and Huffman coding for sub-6-bit quantization.
result DQA achieves significantly better accuracy than direct quantization and state-of-the-art methods.
Paper improves deep learning models for cardiac potential reconstruction.
problem Improving generalization of sequence models for cardiac potential reconstruction.
method Constrained stochasticity and global aggregation of temporal information in latent space.
result Improved generalization of inverse reconstruction networks.
We discuss algorithms for estimating the Shannon entropy h of finite symbol sequences with long range correlations. In particular, we consider algorithms which estimate h from the code lengths produced by some compression algorithm. Our interest is in describing their convergence with sequence length, assuming no limit…
CPS solves inverse problems using forward passes and constrained particle seeking.
problem Solving inverse problems with limited forward observation information.
method Gradient-free approach that reformulates inverse problem as constrained optimization.
result CPS achieves results comparable to gradient-based methods while outperforming alternatives.
Bayesian method detects Markov order in network paths more reliably.
problem Detecting Markov order in constrained categorical sequences.
method Multi-order Bayesian modelling framework.
result Bayesian method detects correct Markov order more reliably than competing methods.
Smoothed fitness landscape improves protein optimization.
problem Infeasibility of combinatorially large protein sequence space.
method Formulate protein fitness as a graph signal, smooth using Tikunov regularization, and optimize with Gibbs sampling.
result 2.5 fold fitness improvement over training set.
It is often hypothesized that a crucial role for recurrent connections in the brain is to constrain the set of possible response patterns, thereby shaping the neural code. This implies the existence of neural codes that cannot arise solely from feedforward processing. We set out to find such codes in the context of one…
Scout-Nd optimizes parameters of stochastic simulators efficiently.
problem Optimizing parameters of stochastic, computationally expensive simulators.
method Scout-Nd algorithm, reducing gradient noise, multi-fidelity schemes.
result Demonstrates better performance compared to existing methods.
Seq2seq models predict complex multi-physics systems' time evolution.
problem Predicting the time-evolution of complex multi-physics systems.
method Sequence-to-sequence models applied to multi-physics simulations.
result Seq2seq models accurately emulate complex systems and predict their evolution.