Model compression has been widely adopted to obtain light-weighted deep neural networks. Most prevalent methods, however, require fine-tuning with sufficient training data to ensure accuracy, which could be challenged by privacy and security issues. As a compromise between privacy and performance, in this paper we inve…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “few-shot compression”
New algorithm PRACTISE accelerates networks with tiny sets, reducing latency by 22%.
problem Accelerating networks with tiny training sets to improve performance and privacy.
method Proposes a new algorithm PRACTISE based on finetune convexity hypothesis and feature mimicking.
result PRACTISE outperforms previous methods by reducing latency by 22% on ImageNet-1k.
The ability to generalize quickly from few observations is crucial for intelligent systems. In this paper we introduce APL, an algorithm that approximates probability distributions by remembering the most surprising observations it has encountered. These past observations are recalled from an external memory module and…
There has been a recent trend in training neural networks to replace data structures that have been crafted by hand, with an aim for faster execution, better accuracy, or greater compression. In this setting, a neural data structure is instantiated by training a network over many epochs of its inputs until convergence.…
Meta-learning improves few-shot acoustic event detection.
problem Detecting new audio events with limited labeled data.
method Formulated few-shot AED problem; explored supervised and meta-learning approaches.
result Meta-learning achieves superior performance in few-shot AED.
Fine-tuning a deep network trained with the standard cross-entropy loss is a strong baseline for few-shot learning. When fine-tuned transductively, this outperforms the current state-of-the-art on standard datasets such as Mini-ImageNet, Tiered-ImageNet, CIFAR-FS and FC-100 with the same hyper-parameters. The simplicit…
A concise review of recent few-shot meta-learning methods.
problem Mimicking human fast adaptation to new concepts based on prior knowledge.
method Categorized into four branches based on technical characteristics.
result Current challenges and future prospects identified.
FROB model improves robustness and reliable confidence for few-shot OoD detection.
problem Challenges in few-shot classification and OoD detection due to limited samples and adversarial attacks.
method FROB model combines support boundary generation and few-shot Outlier Exposure (OE) for improved robustness and reliable confidence.
result FROB achieves generalization to unseen OoD and maintains robustness independent of few-shot number.
Bayesian method improves few-shot classification accuracy.
problem Few-shot classification with small labeled datasets.
method Gaussian process classifier with Pólya-Gamma augmentation and one-vs-each softmax.
result Improved accuracy and uncertainty quantification.
In the few-shot scenario, a learner must effectively generalize to unseen classes given a small support set of labeled examples. While a relatively large amount of research has gone into few-shot learning for image classification, little work has been done on few-shot video classification. In this work, we address the …
Accurate image classification given small amounts of labelled data (few-shot classification) remains an open problem in computer vision. In this work we examine how the known texture bias of Convolutional Neural Networks (CNNs) affects few-shot classification performance. Although texture bias can help in standard imag…
New online few-shot learning model for context-aware recognition.
problem Few-shot learning in online, continuous settings with spatiotemporal context.
method Proposed new dataset and online versions of existing few-shot learning approaches.
result Contextual prototypical memory model improves performance.
Advances few-shot classification by treating it as supervised learning and proposing new training techniques.
problem Formulating the ability of humans to learn from limited data in machine learning.
method Formulated few-shot classification as a supervised learning problem and introduced multi-episode and cross-way training techniques.
result Proposed training strategies accelerate the training process without accuracy loss.
Paper accelerates Bayesian few-shot classification using mirror descent.
problem Non-conjugate inference in Bayesian few-shot classification.
method Integrates mirror descent-based variational inference into Gaussian process-based few-shot classification.
result Accelerated convergence and improved uncertainty quantification.
LMs perform poorly in true few-shot learning without held-out examples.
problem Evaluating few-shot performance of language models without access to held-out examples.
method Evaluated two model selection criteria (cross-validation and minimum description length) for choosing LM prompts and hyperparameters in true few-shot learning.
result Selection criteria often prefer models that perform worse than random selection, suggesting overestimation of few-shot ability.
Paper introduces negative margin loss for better few-shot classification accuracy.
problem Improving few-shot classification accuracy with metric learning.
method Introduces negative margin loss and analyzes its impact on feature discriminability.
result Negative margin loss outperforms regular softmax loss on few-shot classification benchmarks.
Paper introduces privacy-preserving few-shot learning for images.
problem Privacy risk in few-shot learning systems.
method Discrete embedding vectors and one-way hash functions.
result Achieves computational pan privacy without storing embeddings.
CosML combines domain-specific meta-learners for cross-domain few-shot classification.
problem Generalizing to unseen domains while meta-learning on multiple seen domains.
method CosML trains domain-specific meta-learners and combines their meta-parameters in the parameter space.
result CosML outperforms state-of-the-art methods and achieves strong cross-domain generalization.
Paper improves few-shot classification accuracy using feature distribution preprocessing.
problem Challenges of few-shot classification due to limited labelled samples.
method Proposes a novel transfer-based method that preprocesses feature vectors to Gaussian-like distributions and uses optimal-transport inspired algorithms.
result Achieves state-of-the-art accuracy on standardized vision benchmarks.
ProtoTransfer learns from unlabeled data to classify unseen tasks.
problem Few-shot classification with limited labeled data.
method Self-supervised prototypical transfer learning.
result ProtoTransfer outperforms unsupervised meta-learning methods.
Novel GNN model tackles few-shot learning with improved performance.
problem Few-shot learning with GNN suffers from over-fitting and over-smoothing.
method Proposes Attentive GNN with triple-attention mechanism.
result Improves GNN performance for few-shot learning tasks.
Method improves few-shot one-class classification.
problem Learning binary classifier with data from only one class.
method Modified MAML algorithm to learn initialization for few-shot OCC.
result Method leads to better results than classical approaches.
Few-Shot Diffusion Models generate new samples from small image sets.
problem Generating new samples from a few images.
method Conditional Denoising Diffusion Probabilistic Models (DDPM) with patch-based input set information.
result FSDM can generate samples from previously unseen classes conditioned on as few as 5 samples.
BayPrAnoMeta tackles few-shot industrial image anomaly detection with Bayesian methods.
problem Challenges in industrial image anomaly detection, especially class imbalance and scarcity of labeled samples.
method Bayesian Proto-MAML approach with probabilistic normality models and Bayesian posterior predictive likelihood.
result Consistent and significant AUROC improvements over existing methods in few-shot anomaly detection.
Paper defines benchmarks for learning new tasks sequentially.
problem Efficient evaluation of continual few-shot learning.
method Theoretical framework and flexible benchmarks.
result Introduction of SlimageNet64 for efficient evaluation.
Interpretable neural model for few-shot time-series classification.
problem Few-shot time-series classification challenges.
method Dual Prototypical Shapelet Networks (DPSN) framework.
result DPSN framework outperforms state-of-the-art methods, especially with limited data.
Few-shot learning algorithms aim to learn model parameters capable of adapting to unseen classes with the help of only a few labeled examples. A recent regularization technique - Manifold Mixup focuses on learning a general-purpose representation, robust to small changes in the data distribution. Since the goal of few-…
Paper proposes LMM-PQS for cross-domain few-shot learning.
problem Cross-domain few-shot learning problem.
method Generates pseudo query images and fine-tunes feature extraction modules with a large margin mechanism.
result LMM-PQS outperforms baseline models in cross-domain few-shot learning.
Improved few-shot learning with lower-level neural network embeddings.
problem Limited data scenarios in few-shot learning.
method Graph-based meta-learning framework using hidden layer feature embeddings.
result Utilization of lower-level neural network embeddings improves classifier accuracy.
TIM maximizes mutual information for few-shot learning, outperforming state-of-the-art methods.
problem Few-shot learning with limited labeled data.
method Transductive Information Maximization (TIM) with alternating-direction solver.
result Significant improvement in accuracy across various datasets and networks.
Metric-based few-shot learning methods try to overcome the difficulty due to the lack of training examples by learning embedding to make comparison easy. We propose a novel algorithm to generate class representatives for few-shot classification tasks. As a probabilistic model for learned features of inputs, we consider…
This paper tackles few-shot classification by improving GAN-based data augmentation.
problem Improving few-shot classification performance using GANs with limited data.
method Fine-tuning GANs for few-shot classification, addressing training and evaluation challenges.
result Semi-supervised fine-tuning is a more effective approach for few-shot classification with limited data.
Graph Prototypical Networks improve few-shot node classification on attributed networks.
problem Few-shot node classification in attributed networks with limited labeled instances.
method Graph Prototypical Networks (GPN) using meta-learning to extract meta-knowledge and identify informative labeled instances.
result GPN achieves superior performance in few-shot node classification.
Bayesian online meta-learning framework tackles catastrophic forgetting in few-shot classification.
problem Catastrophic forgetting in few-shot classification problems.
method Bayesian online learning, meta-learning, Laplace approximation, variational inference.
result Framework effectively achieves goal of overcoming catastrophic forgetting in few-shot classification.
The field of few-shot learning has been laboriously explored in the supervised setting, where per-class labels are available. On the other hand, the unsupervised few-shot learning setting, where no labels of any kind are required, has seen little investigation. We propose a method, named Assume, Augment and Learn or AA…
Unified RL meta-learning framework for few-shot optimization.
problem Few-shot learning optimization problems.
method Generic RL meta-learning framework that learns optimal optimization algorithms.
result Significantly improved performance on few-shot tasks.
Task-adaptive clustering improves few-shot learning with unlabeled data.
problem Handling unseen tasks with limited labeled data.
method Task-conditioned clustering in a new projection space.
result State-of-the-art semi-supervised few-shot classification performance.
Meta-learning improves feature extraction for few-shot tasks.
problem Understanding why meta-learning models perform better on few-shot classification.
method Developed hypotheses and a regularizer to improve standard training routines.
result Meta-learned models outperform classical training routines in few-shot classification.
Proposes a method for weakly-supervised object localization to improve few-shot learning.
problem Challenges of few-shot learning, especially with fine-grained categories.
method Introduces a Self-Attention Based Complementary Module (SAC Module) for weakly-supervised object localization.
result Significantly outperforms state-of-the-art methods on benchmark datasets, especially for fine-grained few-shot tasks.
Enhances few-shot image classification using unlabelled examples.
problem Few-shot image classification with limited labeled data.
method Transductive meta-learning combining soft k-means clustering and neural feature extractor.
result State-of-the-art performance on Meta-Dataset, mini-ImageNet, and tiered-ImageNet benchmarks.
Novel loss function improves few-shot classification performance.
problem Few-shot classification struggles with embedding space for new classes.
method Proposes self-compacting softmax loss (SSL) to prevent embedding space fullness.
result SSL leads to significant improvement in state-of-the-art performance.
Few-shot learning has become essential for producing models that generalize from few examples. In this work, we identify that metric scaling and metric task conditioning are important to improve the performance of few-shot algorithms. Our analysis reveals that simple metric scaling completely changes the nature of few-…
Improved few-shot learning with LSSVM and transductive modules.
problem Few-shot learning with limited data and samples.
method Introducing LSSVM as a base learner and transductive modules to enhance classification accuracy.
result FSLSTM achieves state-of-the-art performance on miniImageNet and CIFAR-FS benchmarks.
We propose to study the problem of few-shot learning with the prism of inference on a partially observed graphical model, constructed from a collection of input images whose label can be either observed or not. By assimilating generic message-passing inference algorithms with their neural-network counterparts, we defin…
The successful application of deep learning to many visual recognition tasks relies heavily on the availability of a large amount of labeled data which is usually expensive to obtain. The few-shot learning problem has attracted increasing attention from researchers for building a robust model upon only a few labeled sa…
New PAC-Bayesian bounds explain few-shot learning performance gaps.
problem Gap between PAC-Bayesian theory and practice in few-shot learning.
method Developed new PAC-Bayesian bounds for few-shot learning, derived MAML and Reptile from these bounds, and introduced a new PACMAML algorithm.
result PAC-Bayesian bounds explain performance of MAML and Reptile, and outperform existing algorithms.
Self-augmentation improves deep networks for few-shot learning with minimal training data.
problem Improving deep networks' generalization to unseen classes with limited training examples.
method Self-augmentation using self-mix and self-distillation techniques, combined with regional dropout and local representation learning.
result The method outperforms state-of-the-art few-shot learning methods on prevalent benchmarks.
LSTMs show surprising few-shot learning ability, improving on MAML.
problem Few-shot learning with limited data.
method Revisited LSTM approach with Outer Product LSTM (OP-LSTM).
result OP-LSTM outperforms MAML on simple few-shot tasks, but not complex ones.