2-bit coding scheme improves near neighbor search efficiency.
problem Efficiently estimating data similarities in near neighbor search.
method Developed 2-bit quantization and nonlinear estimators.
result 2-bit strategy significantly reduces computational complexity.
2-bit quantization improves RNN performance on resource-limited devices.
problem Large models and high latency on resource-constrained devices.
method Quantize weights and activations into multiple binary codes using alternating minimization.
result 2-bit quantization achieves significant memory saving and inference acceleration.
ActNN reduces neural network training memory by 2-bit quantization.
problem Limited memory in training neural networks.
method Randomly quantizes activations to 2 bits, proving convergence and proposing mixed-precision strategies.
result Reduces activation memory footprint by 12x with negligible accuracy loss.
Quantum algorithm improves sparse vector recovery from noisy measurements.
problem Accurately recover sparse vectors from noisy linear measurements.
method Formulated as a QUBO task, solved using quantum technology.
result Quantum approach outperforms classical methods in sparse coding.
DFTerNet improves human activity recognition on portable devices with 2-bit quantization and dynamic fusion.
problem Resource constraints and unequal fusion strategies limit practical human activity recognition on portable devices.
method 2-bit Convolutional Neural Networks with dynamic fusion strategies for different activity types.
result Exceeds baseline model performance by up to ~5% on OPPORTUNITY and PAMAP2 datasets.
Improved 2-bit covariance estimator with reduced operator norm error and no tuning needed.
problem Improving 2-bit covariance estimation with reduced operator norm error and no tuning needed.
method Proposed a new 2-bit covariance matrix estimator using triangular dithering scales.
result Improved operator norm error rate that depends on effective rank of covariance matrix, closing theoretical gap.
Quantized-TinyLLaVA reduces communication costs in split learning for multimodal models.
problem High communication costs in split learning for multimodal models.
method Integrates a compression module that quantizes intermediate features into discrete representations before transmission.
result Achieves an approximate 87.5% reduction in communication overhead with 2-bit quantization.
Maximizing trading volume in online learning framework between traders.
problem Maximizing the total number of trades between traders with unknown valuations.
method Developed algorithms for brokers to maximize trading volume under different feedback scenarios.
result Achieved logarithmic and poly-logarithmic regret rates for different feedback models.
Discretizing input space improves DLN robustness against adversarial attacks.
problem Improving machine learning models' resistance to adversarial attacks.
method Input discretization and Binary Neural Networks (BNNs).
result 2-bit input discretization significantly enhances adversarial robustness with minimal accuracy loss.
Autoencoder shapes QAM symbols for improved GMI performance.
problem Improving Geometric Mean Information (GMI) in QAM constellation shapes.
method End-to-end learning of bit mappings using autoencoders.
result Achieved up to 0.2 bits/QAM symbol gain in GMI.
vqSGD reduces communication in distributed optimization with convergence guarantees.
problem Reduction of communication cost in distributed optimization.
method Vector quantization schemes based on convex hull of a point set.
result Asymptotic reduction in communication cost with convergence guarantees.
Our work specifies the fundamental cost of using secure aggregation in federated learning.
problem Training a distributed model with differential privacy constraints.
method Characterized the communication cost required for optimal accuracy under differential privacy, achieved by a linear scheme.
result The fundamental communication cost is $ ilde{O}\left( \min(n^2\varepsilon^2, d)
ight)$ bits per client, both sufficient and necessary.
Low-bit training framework reduces energy consumption in CNNs.
problem Reducing energy consumption in convolutional neural networks.
method Low-bit training framework using MLS tensor format with dynamic quantization.
result Achieves superior trade-off between accuracy and bit-width.
New method reduces deep learning complexity on IoT devices.
problem High computational complexity limits deep learning on IoT devices.
method Local quantization region for low-bit data representation.
result Models retain accuracy with reduced computational complexity.
Paper proposes a training framework for deploying DNNs on analog NVM crossbars, achieving significant efficiency gains.
problem Challenges in deploying deep learning models on microcontrollers due to limited compute and memory resources.
method Developed a training algorithm to eliminate tuning and propose unipolar-weighted matrices to reduce crossbar area and simplify hardware.
result Achieved up to 92.91% accuracy with 2-bit crossbars and up to 45% energy reduction.
We study the following generalized matrix rank estimation problem: given an n×n matrix and a constant c≥0, estimate the number of eigenvalues that are greater than c. In the distributed setting, the matrix of interest is the sum of m matrices held by separate machines. We show that any deterministic…
New memory-query tradeoffs for convex optimization algorithms.
problem Optimizing memory usage for convex optimization algorithms.
method Analyzing randomized first-order algorithms for minimizing convex functions.
result Cutting plane methods are optimal in terms of memory and query complexity.
KD technique improves QDNN performance with reduced hyper-parameters.
problem Restoring performance loss in QDNNs due to quantization.
method Applied KD with reduced hyper-parameters, including a new coefficient reduction technique.
result Achieved 92.7% test accuracy on CIFAR-10 and 67.0% on CIFAR-100 with 2-bit weights.
This paper enables deep network inference on microcontrollers with improved accuracy and reduced memory usage.
problem Deploying deep networks on resource-constrained edge-devices with low memory and computational constraints.
method Mixed low-bitwidth compression, rule-based iterative procedure for bit precision determination, quantization-aware retraining, and integer-only model conversion.
result Improved Top1 accuracy of 68% on a 2MB FLASH memory STM32H7 microcontroller, 8% higher than 8-bit implementations.
Develops deep learning for logical code segmentation.
problem Lack of logically segmented source code.
method Novel deep learning approach to generate logical code segments.
result Improves software analysis tasks like commenting, bug detection, and code synthesis.
A new statistical concept, lepto-variance, is defined for stock returns using Regression Trees.
problem Understanding the underlying structure of stock returns using statistical methods.
method Defining lepto-variance as the variance that cannot be removed by any regression tree of a specific depth and analyzing stock returns with 1- and 2-bit Regression Trees.
result Lepto-variance quantifies the resolving power of Regression Trees for stock returns, decomposing total variance into lepto-variance and macro-variance.
Gauss codes help uniquely identify virtual doodles.
problem Identifying virtual doodles uniquely.
method Introduced left canonical Gauss codes.
result Oriented virtual doodles uniquely presented by left canonical Gauss codes.
SQWA improves low-precision DNNs with model averaging and quantization.
problem Designing good generalization DNNs with quantized weights.
method Floating-point model training, direct quantization, multiple low-precision models, weight averaging, re-quantization, fine-tuning, loss visualization.
result State-of-the-art results for 2-bit QDNNs on CIFAR-100 and ImageNet datasets.
TreeCaps improves code comprehension for software developers.
problem Processing code efficiently for software developers.
method Tree-based capsule networks for capturing code syntactical structures and dependencies.
result TreeCaps outperforms other approaches in classifying program functionalities.
Study re-evaluates MIMIC-III codes, finding many are under-coded.
problem Validity of MIMIC-III clinical codes is questionable.
method Open-source, reproducible methodology for assessing codes.
result Most frequently assigned codes are under-coded up to 35%
Paper proposes new gradient codes for robust distributed machine learning.
problem Robustness against slow machines (stragglers) in distributed machine learning.
method Constructs gradient codes based on probabilistic and Kronecker product methods.
result Gradient codes exist for a wide range of system parameters with comparable error performance.
Gradient coding improves distributed learning efficiency.
problem Mitigating straggler issues in distributed learning.
method Gradient coding using cyclic MDS codes and expander graphs.
result Approximate gradient codes enable faster convergence and less computation.
New model creates code semantics vectors for better understanding.
problem Improving code understanding and embedding quality.
method Siamese recurrent neural network on Python source code.
result Model significantly outperforms bag-of-tokens embeddings.
New quantum codes improve error correction with local tests.
problem Improving quantum error correction efficiency.
method Introducing hemicubic codes and exploiting their local testability.
result Quantum codes with improved local testability and error correction.
Deepcode codes outperform traditional ones by 3 orders of magnitude.
problem Designing reliable codes for feedback channels with Gaussian noise.
method Integrating deep learning with information theory for novel codes.
result Deepcode codes outperform known codes by 3 orders of magnitude in reliability.
CoNCRA uses CNN to find code snippets matching developer intent.
problem Finding relevant code snippets using general search engines.
method Convolutional Neural Network approach to code retrieval.
result Improved code retrieval by 5% on average, top 3 positions 80% of the time.
Paper proposes graph-based separable transforms for video coding.
problem Improving video coding efficiency by better capturing residual block statistics.
method Derives graph-based separable transforms (GBSTs) from line graphs with weights determined by parameters.
result GBSTs achieve about 0.4% average coding gain over existing transforms in VVC.
Contrastive Code Representation Learning improves code summarization and type inference.
problem Code representations are sensitive to edits, hindering downstream semantic understanding tasks.
method ContraCode: a contrastive pre-training task that learns code functionality.
result Contrastive pre-training improves code summarization and type inference accuracy.
Adversarial attacks found to be effective on code models.
problem Adversarial robustness of code models.
method Instantiated adversarial attacks for code, demonstrated vulnerability, and improved robustness.
result Neural models for code are vulnerable to adversarial attacks.
Deep learning predicts ICD codes with high accuracy for patient phenotyping.
problem Variability in ICD code assignment by coders.
method Deep learning model trained on demographics, lab results, and medications.
result Model predictions outperform coder assigned ICD codes in accuracy.
Deep neural network predicts semantic labels for source code.
problem Difficulty in labeling and understanding new programming languages and functionalities.
method Language-agnostic deep convolutional neural network trained on Stack Overflow code snippets.
result Mean area under ROC of 0.957 and top-1 accuracy of 86.6% on Github code documents.
This work tackles runtime complexity prediction for code, using machine learning and a new dataset.
problem Predicting runtime complexity of code is hard and mathematically impossible.
method Modelled as a machine learning task, using feature engineering and code embeddings, with a new dataset.
result Achieved state-of-the-art results in runtime complexity prediction.
Transformer model improves source code summarization.
problem Generating readable summaries of source code.
method Transformer model with self-attention mechanism for code representation.
result Transformer model outperforms state-of-the-art techniques.
SLM models code syntax as trees to generate any programming language code.
problem Generating any piece of code in a given language without restrictions.
method Structural language modeling (SLM) decomposes code into ASTs and estimates probabilities over nodes.
result SLM model generates arbitrary code in any language, outperforming previous methods.
Investigates neural codes and their embeddings, proving conjectures and introducing new code types.
problem Analyzing neural codes and their embedding dimensions.
method Combinatorial, topological, and algebraic analysis; proving conjectures; introducing new neural code types.
result Proves conjectures about neural codes and their embeddings, introduces new code types.
Sparse coding approximates the data sample as a sparse linear combination of some basic codewords and uses the sparse codes as new presentations. In this paper, we investigate learning discriminative sparse codes by sparse coding in a semi-supervised manner, where only a few training samples are labeled. By using the m…
This paper automates the creation of low precision deep learning operators for mobile devices.
problem Deploying large deep learning models on low power devices is challenging due to limited compute capabilities and energy budgets.
method Introduces a workflow to generate high-performance low precision deep learning operators for multiple CPU architectures, including optimizations like memory tiling and vectorization.
result 1-bit and 2-bit convolutions achieve up to 16x and 2.3x speedups over 16-bit integer baselines on ARM Cortex-A53 CPU.
Gradient codes use block designs to resist adversarial stragglers in distributed computing.
problem Mitigating slow machines (stragglers) in distributed gradient-based methods.
method Gradient coding based on balanced incomplete block designs (BIBDs) to resist adversarial selection of stragglers.
result Adversarial stragglers have no advantage over random selection, and codes based on symmetric BIBDs maximize the adversarial threshold.
Inspired by recent work on convex formulations of clustering (Lashkari & Golland, 2008; Nowozin & Bakir, 2008) we investigate a new formulation of the Sparse Coding Problem (Olshausen & Field, 1997). In sparse coding we attempt to simultaneously represent a sequence of data-vectors sparsely (i.e. sparse approximation (…
Survey reviews code-switched speech and language processing.
problem Processing code-switched text and speech for multilingual communities.
method Reviews computational approaches and lists available resources.
result Essential for building intelligent agents that interact in multilingual settings.
Topological theory for qLDPC codes enables non-Clifford gates and magic state injection.
problem Fault-tolerant quantum computation in qLDPC codes with non-Clifford gates and magic state resources.
method Developed a topological theory using simplicial or CW complex structures and deformation retraction.
result Achieved non-Clifford gates and magic state injection in qLDPC codes with constant rate and polynomial distance.
We introduce new definitions of universal and superuniversal computable codes, which are based on a code's ability to approximate Kolmogorov complexity within the prescribed margin for all individual sequences from a given set. Such sets of sequences may be singled out almost surely with respect to certain probability …
Machine learning predicts accurate cloning of printed graphical codes.
problem Clonability of anti-counterfeiting printable graphical codes.
method Simple system composed of fully connected neural network layers.
result Accurate estimation of digital codes from printed counterparts in certain cases.