This paper explores how random sampling and coding can speed up approximate matrix multiplication.
problem Efficiently computing large-scale matrix multiplications in distributed systems.
method Proposes two schemes: coding for recovery and random sampling for approximation.
result Investigates tradeoffs between recovery threshold and approximation error.
New gradient coding schemes reduce decoding error in both random and adversarial straggler settings.
problem Creating efficient approximate gradient coding schemes for distributed optimization.
method Introduced novel approximate gradient codes based on expander graphs, achieving optimal decoding coefficients.
result Achieved nearly optimal error in random setting and nearly half the error in adversarial setting compared to existing codes.
We use matricial free energy to regularize autoencoders, producing Gaussian-like codes.
problem Generating Gaussian-like codes for autoencoders.
method Define a differentiable loss function based on singular values of the code matrix, minimizing matricial free energy.
result Minimizing matricial free energy results in Gaussian-like codes that generalize.
Paper reinterprets majorizing measure theorem in terms of coding theory.
problem Understanding boundedness of random processes.
method Information-theoretic perspective using variable-length codes.
result Boundedness of random processes linked to efficient coding.
ForestHash combines random forests and CNNs for efficient data hashing.
problem Efficiently hashing large datasets while preserving similarity.
method Random forests with light-weight CNNs, grouping classes, and information-theoretic aggregation.
result Significantly outperforms state-of-the-art hashing methods for image retrieval.
Hash codes are a very efficient data representation needed to be able to cope with the ever growing amounts of data. We introduce a random forest semantic hashing scheme with information-theoretic code aggregation, showing for the first time how random forest, a technique that together with deep learning have shown spe…
This note improves code quality for equientropic channels by maximizing marginal entropy.
problem Improving code quality for equientropic channels with finite bit usage.
method Characterizes code quality by an upper bound on expected minimum error probability for equientropic channels, showing random coding maximizes marginal entropy.
result For equientropic channels, random coding maximizes marginal entropy and achieves minimal upper bound on expected minimum error probability.
Constructs manifolds from quantum codes with novel geometric properties.
problem Creating manifolds with specific geometric constraints.
method Reverse engineering manifolds from quantum code chain complexes.
result First examples of power law Z2 systolic freedom. 2-bit coding scheme improves near neighbor search efficiency.
problem Efficiently estimating data similarities in near neighbor search.
method Developed 2-bit quantization and nonlinear estimators.
result 2-bit strategy significantly reduces computational complexity.
This paper establishes information-theoretic limits in estimating a finite field low-rank matrix given random linear measurements of it. These linear measurements are obtained by taking inner products of the low-rank matrix with random sensing matrices. Necessary and sufficient conditions on the number of measurements …
Two new coding schemes improve the efficient communication of noisy data.
problem Efficient communication of noisy data in machine learning.
method Ordered Random Coding (ORC) and Hybrid Coding Scheme.
result Improved coding schemes over existing approaches.
Hyperplane hashing aims at rapidly searching nearest points to a hyperplane, and has shown practical impact in scaling up active learning with SVMs. Unfortunately, the existing randomized methods need long hash codes to achieve reasonable search accuracy and thus suffer from reduced search speed and large memory overhe…
In (\cite{zhang2014nonlinear,zhang2014nonlinear2}), we have viewed machine learning as a coding and dimensionality reduction problem, and further proposed a simple unsupervised dimensionality reduction method, entitled deep distributed random samplings (DDRS). In this paper, we further extend it to supervised learning …
New gradient codes use sparse graphs to speed up distributed computation with slight inaccuracy.
problem Straggler effect in distributed algorithms causing slowest nodes to dictate overall time.
method Sparse random graphs for gradient coding to achieve approximate accuracy.
result Approximate gradient codes can significantly increase robustness to stragglers.
A new stochastic solver improves Convolutional Sparse Coding efficiency.
problem Efficiency and sparsity in Convolutional Sparse Coding.
method Randomized subsampling strategy in spatial domain for online learning.
result Improved execution time with no loss in learning quality.
Coded Federated Learning speeds up training in edge computing networks.
problem Slow convergence in Federated Learning due to heterogeneity and stochastic fluctuations.
method Exploiting statistical properties of compute and communication delays, distributed kernel embedding, and random Fourier features.
result Significant performance gains for CodedFedL in distributed non-linear regression and classification problems.
We introduce the Randomized Dependence Coefficient (RDC), a measure of non-linear dependence between random variables of arbitrary dimension based on the Hirschfeld-Gebelein-Rényi Maximum Correlation Coefficient. RDC is defined in terms of correlation of random non-linear copula projections; it is invariant with respec…
New algorithm for multi-player bandits with collision-dependent rewards.
problem Stochastic multi-player multi-armed bandits with collision-dependent reward distributions.
method Error-Correction Collision Communication (EC3) algorithm.
result EC3 algorithm achieves optimal regret approaching centralized MP-MAB regret.
New insights into image compression trade-offs with private randomness.
problem Trade-off between compression rate and perceptual quality in image compression.
method Characterization of rate-distortion trade-off with private randomness under different realism constraints.
result Encoder private randomness is not useful if compression rate is below source entropy, even with limited common and decoder private randomness.
Paper explores how neural codes can be derived from spike timing patterns.
problem Lack of computational models for spike timing information.
method Minimalistic abstraction of recurrent connections using information-theoretic techniques.
result Neural codes derived from polychronous groups meet benchmarks for linear classification and capacity.
Paper improves MIRACLE for faster, more robust neural network compression.
problem Efficiently compressing neural networks while maintaining performance.
method Introduces Mean-KL parameterization to constrain compression cost.
result Mean-KL parameterization leads to twice as fast convergence and more robust compression.
Gradient codes adapt to varying straggler counts in distributed learning.
problem Mitigating slow worker nodes (stragglers) in distributed machine learning.
method Proposes a flexible gradient coding scheme that concatenates codes for different straggler tolerances, adapting to actual straggler counts.
result Significantly lower latency compared to fixed-tolerance gradient codes.
Word embeddings reveal multiple senses, which can be recovered using sparse coding.
problem Understanding word senses in polysemous words.
method Sparse coding of word embeddings to recover multiple senses.
result Sparse coding can approximate multiple word senses, with each sense associated with a discourse atom.
Gradient codes use block designs to resist adversarial stragglers in distributed computing.
problem Mitigating slow machines (stragglers) in distributed gradient-based methods.
method Gradient coding based on balanced incomplete block designs (BIBDs) to resist adversarial selection of stragglers.
result Adversarial stragglers have no advantage over random selection, and codes based on symmetric BIBDs maximize the adversarial threshold.
The paper extends lossy coding to nonlinear latent representations.
problem Learning finite-dimensional coding schemes with nonlinear reconstruction maps.
method Generalizes Maurer--Pontil framework to nonlinear maps, connects to generative modeling, and provides generalization bounds.
result Established a connection to approximate generative modeling and presented generalization bounds.
New method compresses neural networks using random code, improving efficiency.
problem Large memory footprint of deep neural networks.
method Training a variational distribution over weights, encoding using Kullback-Leibler divergence.
result Achieves state-of-the-art compression rates and test performance.
CodNN uses error-correcting codes to make neural networks more resilient to noise.
problem Neural networks are sensitive to noise, especially in critical applications.
method Construct robust neural networks by coding data or internal layers with error-correcting codes.
result Parity codes can guarantee robustness for a wide range of neural networks, including binarized networks.
Paper uses RL to optimize bit-flipping decoding for binary codes.
problem Improving bit-flipping decoding for binary linear codes.
method Mapped iterative decoding algorithms to MDPs for reinforcement learning.
result Learned BF decoders offer performance-complexity trade-offs and near-optimal performance.
Paper develops a decoder for sparse codes without encoder matrix, achieving optimal recovery.
problem Designing a decoder for sparse codes from linear measurements alone.
method Matrix factorization to recover encoder and sparse coding matrices from measurements.
result Decoder-Expander Based Factorisation recovers encoder and sparse coding matrix at optimal measurement rate with high probability.
In compressed sensing, we wish to reconstruct a sparse signal x from observed data y. In sparse coding, on the other hand, we wish to find a representation of an observed signal y as a sparse linear combination, with coefficients x, of elements from an overcomplete dictionary. While many algorithms are competit…
The paper predicts run times for Gaussian chemistry code.
problem Accurate run time prediction for complex scientific codes.
method Characterized data set, explored regression methods.
result Promising future directions for run time prediction.
Paper uses AI to predict medications from medical codes, improving accuracy in healthcare.
problem Predicting medications from incomplete or incorrect medical codes is challenging.
method Robust Recurrent Neural Networks (RNNs) with decay mechanism and noise injection.
result The method accurately predicts medication orders from contaminated medical codes.
A new method uses LDPC codes to improve gradient descent in distributed computing.
problem Mitigating the effect of straggling processors in distributed computing.
method Encoding the second-moment of data with LDPC codes and iterative decoding.
result The method outperforms existing schemes in real distributed computing setups.
Categorical Co-Frequency Analysis clusters diagnoses to predict hospital readmissions.
problem Predicting patients' risk of 30-day hospital readmission.
method Categorical Co-Frequency Analysis (CoFA) measures diagnosis similarity using random forests.
result Identified three groups of diagnoses with varying readmission risk.
Study finds color-coded DMSs improve driver behavior.
problem Improving driver behavior through dynamic message signs.
method Random forest algorithm for route diversion, choice, and compliance analyses.
result Color-coded DMSs are more effective than alphanumeric ones.
Continuous time random walks impose a random waiting time before each particle jump. Scaling limits of heavy tailed continuous time random walks are governed by fractional evolution equations. Space-fractional derivatives describe heavy tailed jumps, and the time-fractional version codes heavy tailed waiting times. Thi…
Framework translates images between domains without supervision.
problem Challenges in unsupervised image-to-image translation, especially handling multimodality.
method Proposes a Multimodal Unsupervised Image-to-Image Translation (MUNIT) framework, decomposing images into content and style codes.
result Demonstrates improved generation of diverse outputs from a single source image.
Method learns topological states from randomized measurements.
problem Detecting topologically ordered two-dimensional states on quantum processors.
method Variational tensor network tomography with randomized measurements.
result Demonstrated ability to learn ground states of surface code and quantum spin liquid states.
Orthogonal coding matrices improve multi-class classification accuracy across various datasets.
problem Improving multi-class classification accuracy using orthogonal coding matrices.
method Optimized orthogonal coding matrices for multi-class classification, compared with other methods.
result Orthogonal coding matrices generally outperform random ECCs and are faster than 1 vs. 1.
Paper addresses data reconstruction from privacy-protected templates using STCA.
problem Reconstructing privacy-sensitive data from protected templates.
method Sparse ternary coding with ambiguization (STCA) for privacy preservation.
result STCA maintains theoretical performance against deep reconstruction attacks for synthetic data but requires special measures for real images.
Paper introduces EM-KSH and EM-SPLH methods for supervised hashing.
problem Efficiently optimize retrieval speed and storage cost while preserving semantic information.
method Convert supervised hashing formulations to CRF, solve consistency equations using linear approximation of sigmoid function.
result Experimental results show superior performance of EM-KSH and EM-SPLH.
AL methods show inconsistent performance gains over random sampling, highlighting variability in neural network-based approaches.
problem Inconsistent performance of AL methods using neural networks.
method Different types of AL algorithms (uncertainty based, diversity based, and committee based) were tested under identical experimental settings.
result AL methods do not consistently outperform random sampling, revealing variability in performance metrics.
Machine learning model predicts DFT total energy to complete basis set limit.
problem Finding a model to extrapolate DFT calculations to complete basis set limit.
method Quantile-random-forest model trained on binary solids data.
result Random-forest model achieves <25% symmetric MAPE for both DFT codes.
Transformer models waste resources on long-context tasks.
problem Redundant attention computations in Transformer models for long-context tasks.
method Reformulate sequence modeling as supervised learning, analyze attention sparsity, formulate attention optimization as linear coding problem, propose Dynamic Group Attention.
result DGA reduces computational costs while maintaining performance.
PV-RNN model uses predictive coding to predict and recognize data sequences.
problem Addressing variational RNNs' challenges in learning meaningful latent representations and transferring future observations.
method Inspired by predictive coding, PV-RNN dynamically changes latent stochasticity and uses prediction errors for information transfer.
result PV-RNN outperforms standard variational Bayes models in predicting unseen sequences and generalizes well.
Neurogenesis-inspired online learning adapts model architecture in changing environments.
problem Continuous adaptation of model architecture in non-stationary environments.
method Online dictionary-learning framework with adaptive addition and deletion of units, inspired by neurogenesis.
result Significant improvement in performance on nonstationary data compared to fixed-size online sparse coding.
Two machine learning models detect anomalies in ER claims, saving up to 40% in improper payments.
problem Improper health insurance payments from fraud and upcoding.
method Two machine learning models: an upcoding model based on severity code distributions and a random forest model for claim sorting.
result Random forest model saved 12% to 40% in improper payments compared to a baseline approach.
PLIT identifies plant lncRNAs from RNA-seq data with high accuracy.
problem Inaccurate identification of lncRNAs in plant transcriptomic datasets.
method PLIT uses L1 regularization and iRF classification to select optimal features from sequence and codon-bias data.
result PLIT outperforms existing CPC tools in identifying lncRNAs in plant RNA-seq datasets.