Paper presents CeNN quantization for efficient CPS applications.
problem Efficient processing for CPS applications, especially in telemedicine and ADAS.
method Incremental quantization of CeNNs with various strategies.
result Achieved up to 7.8x speedup with no performance loss.
Efficiently processes dynamic inputs in AI writing assistants with incremental computation.
problem Efficiently updating AI models in real-time with dynamic inputs.
method Incremental computing using vector quantization to filter and reuse intermediate values in neural networks.
result Comparable accuracy with 12.1X fewer operations for processing dynamic inputs.
Network quantization is an effective solution to compress deep neural networks for practical usage. Existing network quantization methods cannot sufficiently exploit the depth information to generate low-bit compressed network. In this paper, we propose two novel network quantization approaches, single-level network qu…
The thesis explores how to integrate machine learning with hardware constraints.
problem Designing efficient neural networks for real-time processing with hardware limitations.
method Developed a library for training and converting sparse quantized neural networks to hardware.
result Demonstrated how to design and optimize neural networks for FPGA-based hardware.
We introduce incremental variational inference and apply it to latent Dirichlet allocation (LDA). Incremental variational inference is inspired by incremental EM and provides an alternative to stochastic variational inference. Incremental LDA can process massive document collections, does not require to set a learning …
The paper extends cluster validity indices for incremental analysis.
problem Providing incremental alternatives for cluster validation.
method Extending iCVI family to include 6 incremental indices and examining their behavior under under- and over-partitioning.
result Over-partitioning is more challenging to detect than under-partitioning.
EILearn learns incrementally using previous classifier knowledge.
problem Incremental learning with previous data.
method Retains and uses previous classifier knowledge, monitors performance, eliminates poorly performing classifiers.
result Outperforms existing incremental learning approaches.
Paper proposes faster incremental subclass discriminant analysis.
problem Efficiently classify subclasses in incremental data.
method Exact and approximate linear and kernelized solutions.
result Superior training time and accuracy compared to existing methods.
Time and Sales of corn futures traded electronically on the CME Group Globex are studied. Theories of continuous prices turn upside down reality of intra-day trading. Prices and their increments are discrete and obey lattice probability distributions. A function for systematic evolution of futures trading volume is pro…
Depth helps neural networks learn simpler solutions incrementally.
problem Understanding why deep neural networks generalize well despite complex architectures.
method Formal definition of incremental learning dynamics, theoretical analysis of depth and initialization effects, experiments with various models.
result Incremental learning dynamics can arise in deeper models, but not in shallow ones, under specific conditions.
Survey on quantization methods on Kähler manifolds.
problem None explicitly stated; focuses on methods.
method Deformation quantization, geometric quantization, Berezin-Toeplitz quantization, BV quantization.
result New relationships among quantization methods on Kähler manifolds.
Paper tackles few-shot class-incremental learning with a neural gas network.
problem Incrementally learn new classes from very few labelled samples without forgetting old classes.
method Proposes TOPIC framework using a neural gas network to preserve class topology and adapt to new samples.
result Significantly outperforms other methods on CIFAR100, miniImageNet, and CUB200 datasets.
A new model learns preferences incrementally without personal data.
problem Incremental session-based recommendation without personal data.
method Memory Augmented Neural model (MAN) that combines a neural recommender with a nonparametric memory.
result MAN consistently outperforms existing methods in incremental session-based recommendation.
The paper classifies quantizable functions and explores symmetry in quantization methods.
problem Classifying quantizable functions and understanding symmetry in quantization methods.
method Deformation quantization and geometric quantization methods are compared and classified.
result Formal quantizable functions are of a specific form and relate to Hamiltonian Killing vector fields.
Incremental versions of batch algorithms are often desired, for increased time efficiency in the streaming data setting, or increased memory efficiency in general. In this paper we present a novel algorithm for incremental kernel PCA, based on rank one updates to the eigendecomposition of the kernel matrix, which is mo…
This paper introduces a differentiable, scalable quantization method for neural networks.
problem Previous quantization methods lacked differentiability and scalability.
method The approach is differentiable and scalable, using bit-shifting and logarithmic quantization.
result The method achieves comparable accuracy to state-of-the-art approaches with less training time and lower inference cost.
Paper proposes CNE-net to tackle incremental learning in (T)ACSA tasks.
problem Catastrophic forgetting in multi-task incremental learning for (T)ACSA.
method Category Name Embedding network (CNE-net) with shared encoder and decoder.
result State-of-the-art performance on (T)ACSA benchmark datasets.
This paper introduces Deep Incremental Boosting, a new technique derived from AdaBoost, specifically adapted to work with Deep Learning methods, that reduces the required training time and improves generalisation. We draw inspiration from Transfer of Learning approaches to reduce the start-up time to training each incr…
GraphSAIL updates GNN-based recommender models incrementally to reduce computation time and improve frequent updates.
problem Incremental updates in GNN-based recommender systems are computationally expensive and prone to forgetting.
method GraphSAIL uses a graph structure preservation strategy to update GNN models incrementally, preserving long-term preferences and properties.
result GraphSAIL reduces computation time and improves frequent updates compared to other incremental learning techniques.
This paper presents a novel approach for incremental semiparametric inverse dynamics learning. In particular, we consider the mixture of two approaches: Parametric modeling based on rigid body dynamics equations and nonparametric modeling based on incremental kernel methods, with no prior information on the mechanical …
While the harmonic function solution performs well in many semi-supervised learning (SSL) tasks, it is known to scale poorly with the number of samples. Recent successful and scalable methods, such as the eigenfunction method focus on efficiently approximating the whole spectrum of the graph Laplacian constructed from …
StatQAT optimizes quantization for deep networks, reducing computational cost and memory usage.
problem Optimal quantization parameters selection for deep neural networks with diverse data distributions.
method Statistical error analysis framework for uniform and floating-point quantization, iterative and analytic quantizers designed for arbitrary and Gaussian-like distributions.
result Improved accuracy and stability in training low-precision neural networks.
Smart Quantization adapts binary and ternary quantization for neural networks.
problem Resource constraints in deploying neural networks on devices with limited resources.
method Adaptive combination of binary and ternary quantization with a regularization function.
result Adapts quantization depth during training to maintain high model accuracy.
Data is continuously generated by modern data sources, and a recent challenge in machine learning has been to develop techniques that perform well in an incremental (streaming) setting. In this paper, we investigate the problem of private machine learning, where as common in practice, the data is not given at once, but…
This study optimizes quantized neural networks by considering model architecture and quantization types.
problem Optimizing quantized neural networks for low-power, high-throughput applications.
method Holistic approach including training methods and quantization-friendly architecture design.
result Deeper models are more sensitive to activation quantization, while wider models improve resilience to both weight and activation quantization.
We analyze the question whether sliding window time averages applied to stationary increment processes converge to a limit in probability. The question centers on averages, correlations, and densities constructed via time averages of the increment x(t,T)=x(t+T)-x(t)and the assumption is that the increment is distribute…
RATQ is a new quantizer for optimizing noisy gradients in machine learning.
problem Optimizing noisy gradients in stochastic optimization.
method RATQ uses Hadamard transform and adaptive uniform quantization, and achieves near-optimal performance.
result RATQ nearly achieves information theoretic lower bounds for optimization accuracy.
New algorithm improves on EM for streaming data, outperforming existing methods.
problem Processing high-volume, streaming data efficiently.
method Incremental stochastic Majorization-Minimization (MM) algorithm.
result The algorithm converges to a stationary point with vanishing gradient.
This paper presents a novel adaptive resonance theory (ART)-based modular architecture for unsupervised learning, namely the distributed dual vigilance fuzzy ART (DDVFA). DDVFA consists of a global ART system whose nodes are local fuzzy ART modules. It is equipped with the distinctive features of distributed higher-ord…
Incremental clustering approaches have been proposed for handling large data when given data set is too large to be stored. The key idea of these approaches is to find representatives to represent each cluster in each data chunk and final data analysis is carried out based on those identified representatives from all t…
Extends ONNX for quantized neural networks with new formats and operators.
problem Handling arbitrary-precision quantization in neural networks.
method Introduces new formats and operators in ONNX to represent quantized neural networks.
result Enabled representation of uniform quantization in neural networks.
New method for quantizing symplectic manifolds with Lagrangian bundles.
problem Quantization of symplectic manifolds with Lagrangian bundles.
method A new construction of strict deformation quantization.
result Established a correspondence between differential operators and principal symbols.
HMQ improves quantization for edge devices with mixed precision.
problem Efficient quantization for edge devices with uniform, power-of-two thresholds.
method Introduces HMQ, a mixed precision quantization block that repurposes Gumbel-Softmax for searching over quantization schemes.
result Achieves competitive and state-of-the-art results on ImageNet despite restrictions.
Introduces sheaf quantization, a topological approach to geometric quantization.
problem Topological realization of WKB-states in geometric quantization.
method Enhancement of constructible sheaves, Betti counterpart of Fukaya--Floer theory.
result Introduction to sheaf quantization as a topological realization of WKB-states.
The article defines and compares two types of quantizations on compact manifolds.
problem Quantization on arbitrary compact smooth manifolds.
method Embedding into CP^n and inducing quantizations from there.
result Generalizations of earlier quantization methods.
Proposes MEDIC to improve incremental learning by preventing forgetting and intransigence.
problem Challenges of forgetting old knowledge and intransigence on new knowledge in incremental learning.
method Maximum Entropy Regularizer (MER) and DropOut Sampling (DOS) to penalize uncertain knowledge and reduce class imbalance.
result Proposed method 'MEDIC' outperforms state-of-the-art algorithms in accuracy, forgetting, and intransigence.
Stochastic Gradient Trees learn decision trees incrementally.
problem Learning decision trees using stochastic gradient information.
method Incremental learning setting, soft splits not used, new tree not constructed per update.
result Performs similarly to standard incremental classification trees, outperforms state of the art incremental regression trees, comparable to batch multi-instance learning methods.
Quantized Adam reduces communication cost in deep learning training.
problem Reducing communication cost in distributed deep learning training.
method Gradient and weight quantization with error feedback in Adam.
result Proposed methods converge to first-order stationary points.
Proposes a robust neural network quantization method.
problem Training model's dependency on specific quantization methods.
method Intrinsic robustness to various quantization processes.
result Single model capable of operating at various bit-widths and policies.
The paper studies quantization on symplectic manifolds with real polarizations, comparing different quantization methods.
problem Quantization on compact symplectic manifolds with real polarizations.
method Geometric quantization, Toeplitz operators, Fourier transforms, asymptotic expansion of traces.
result Deformation quantization is realized through asymptotic traces of Toeplitz operators.
Quantizes neural networks using frame theory for improved accuracy.
problem Improving neural network efficiency and accuracy through quantization.
method Sigma-Delta (ΣΔ) quantization with finite unit-norm tight frames. result Error bound between original and quantized neural networks derived.
Unified finetuning of all quantization degrees of freedom achieves state-of-the-art 4-bit quantization.
problem Achieving high accuracy in quantized neural networks while maintaining speed and resource constraints.
method Quantization-aware finetuning (QFT) that jointly optimizes all quantization degrees of freedom.
result 4-bit weight quantization results on-par with state-of-the-art (SoTA) within PTQ constraints.
It was recently shown that architectural, regularization and rehearsal strategies can be used to train deep models sequentially on a number of disjoint tasks without forgetting previously acquired knowledge. However, these strategies are still unsatisfactory if the tasks are not disjoint but constitute a single increme…
Abstract proposes a new categorical approach to quantization of Poisson algebras.
problem Quantization of Poisson algebras.
method Defining quantization categories as subcategories of R-module categories with classical limits.
result Categories of strict deformation quantization, prequantization, and matrix regularization are equivalent, while Poisson enveloping algebra is not.
Tensor decompositions are invaluable tools in analyzing multimodal datasets. In many real-world scenarios, such datasets are far from being static, to the contrary they tend to grow over time. For instance, in an online social network setting, as we observe new interactions over time, our dataset gets updated in its "t…
A new generative classification strategy outperforms existing methods in class-incremental learning.
problem Incrementally training deep neural networks to recognize new classes is challenging.
method Proposes learning the joint distribution p(x,y) and performing classification using Bayes' rule, implemented with variational autoencoders and importance sampling.
result Performs very well on continual learning benchmarks, outperforming existing baselines.
The statistical properties of the increments x(t+T) - x(t) of a financial time series depend on the time resolution T on which the increments are considered. A non-parametric approach is used to study the scale dependence of the empirical distribution of the price increments x(t+T) - x(t) of S&P Index futures, for time…
For classification of the high frequency trading quantities, waiting times, price increments within and between sessions are referred to as the a-, b-, and c-increments. Statistics of the a-b-c-increments are computed for the Time & Sales records posted by the Chicago Mercantile Exchange Group for the futures traded on…