BTZSC benchmarks zero-shot text classification across diverse models.
problem Systematically comparing zero-shot text classification across various models.
method Comprehensive benchmark of 22 datasets, comparing NLI cross-encoders, embedding models, rerankers, and instruction-tuned LLMs.
result Rerankers and instruction-tuned LLMs outperform NLI cross-encoders, with rerankers setting a new state-of-the-art.
ZegOT uses optimal transport to zero-shot segment images with text prompts.
problem Zero-shot semantic segmentation with limited image-text alignment knowledge.
method ZegOT uses optimal transport to match multiple text prompts with frozen image embeddings.
result ZegOT achieves state-of-the-art performance in zero-shot semantic segmentation.
Automates zero-shot classification by scoring and weighting prompts.
problem Improving zero-shot accuracy through prompt ensembling.
method Automatic prompt scoring and weighting method.
result Method outperforms existing techniques on various benchmarks.
Out-of-domain (OOD) detection for low-resource text classification is a realistic but understudied task. The goal is to detect the OOD cases with limited in-domain (ID) training data, since we observe that training data is often insufficient in machine learning applications. In this work, we propose an OOD-resistant Pr…
OTSeg uses multi-prompt Sinkhorn attention to improve zero-shot semantic segmentation.
problem Leveraging pre-trained CLIP knowledge to align text embeddings with pixel embeddings.
method OTSeg employs Multi-Prompts Sinkhorn (MPS) and Multi-Prompts Sinkhorn Attention (MPSA) to enhance semantic feature matching.
result OTSeg achieves state-of-the-art performance in zero-shot semantic segmentation tasks.
Expands weak supervision by allowing partial labels from multiple noisy sources.
problem Creating models without labeled data using heuristic labelers.
method Probabilistic generative model estimating partial label accuracies.
result Improved model accuracy on various tasks (8.6% on text, comparable to zero-shot methods on images).
CLIP learns joint image-text representations for zero-shot learning.
problem Understanding and improving zero-shot transfer performance in CLIP.
method Formal study of transferrable representation learning and analysis of zero-shot transfer performance.
result Proposes a new CLIP-type approach that outperforms existing methods.
ZeroSCROLLS benchmarks zero-shot natural language understanding over long texts.
problem Evaluate natural language understanding models over long texts without training data.
method Adapt six tasks from SCROLLS benchmark and add four new datasets, including novel aggregation tasks.
result Claude outperforms ChatGPT, and GPT-4 achieves highest average score.
Zero-shot contrastive loss improves text-guided image style transfer without extra training.
problem Stochastic nature of diffusion models leads to trade-offs between style transformation and content preservation.
method Proposes a zero-shot contrastive loss for diffusion models that doesn't require additional fine-tuning or auxiliary networks.
result Method outperforms existing methods while preserving content and requiring no additional training.
MuLan links music audio to natural language tags.
problem Traditional music tagging systems use rigid attributes; MuLan aims to link audio directly to natural language.
method Joint audio-text embedding model trained on 44 million music recordings and text annotations.
result MuLan's embeddings enable zero-shot functionalities and transfer learning.
Generates coherent storybooks from plain text using diffusion models.
problem Ensuring coherency in a sequence of images for storytelling applications.
method Combines pre-trained LLM and text-guided Latent Diffusion Model for zero-shot generation.
result Outperforms state-of-the-art image editing baselines in generating coherent storybooks.
Language models help text classification tasks by predicting next words.
problem Lack of theoretical understanding of why language models perform well on downstream tasks.
method Mathematical study of the connection between next word prediction and text classification, formalizing it and quantifying the benefit.
result Language models that are ε-optimal in cross-entropy learn features that can solve classification tasks with linear approximation.
We empirically characterize the performance of discriminative and generative LSTM models for text classification. We find that although RNN-based generative models are more powerful than their bag-of-words ancestors (e.g., they account for conditional dependencies across words in a document), they have higher asymptoti…
Dissertation tackles zero-shot anomaly detection, focusing on consistent anomalies and proposing CoDeGraph framework.
problem Consistent anomalies bias distance-based zero-shot anomaly detection methods.
method Formalized consistent anomalies, identified similarity scaling and neighbor-burnout phenomena, introduced CoDeGraph framework.
result CoDeGraph effectively suppresses consistent anomalies in zero-shot anomaly detection.
Telescope detects LLM generated text by measuring token repetition probability.
problem Distinguishing LLM generated text from human writing.
method Telescope Perplexity, evaluating token repetition probability.
result Telescope Perplexity enables effective zero-shot LLM detection.
Ground-A-Video edits videos without training, preserving intended changes.
problem Complex multi-attribute video editing with omitted or wrong changes.
method Grounding-guided video-to-video translation with Cross-Frame Gated Attention.
result Zero-shot multi-attribute video editing with improved accuracy and frame consistency.
Aggregates diverse zero-shot LLM outputs for better corporate disclosure classification.
problem Combining varied zero-shot LLM predictions for improved stock return prediction.
method Multi-prompt framework with three fixed zero-shot LLM classifiers, logistic meta-classifier aggregation.
result Aggregated model outperforms single classifiers and baseline models, increasing balanced accuracy from 0.566 to 0.606.
Generative classifiers show surprising human-like performance.
problem Comparing generative and discriminative models for object recognition.
method Built on recent advances in generative modeling to create classifiers and compared them to discriminative models.
result Generative classifiers outperform discriminative models in several key areas, including shape bias and out-of-distribution accuracy.
We address the problem of learning fine-grained cross-modal representations. We propose an instance-based deep metric learning approach in joint visual and textual space. The key novelty of this paper is that it shows that using per-image semantic supervision leads to substantial improvement in zero-shot performance ov…
New study finds best language model architecture and pretraining objective for zero-shot tasks.
problem Evaluating which language model architectures and pretraining objectives best enable zero-shot generalization.
method Compared three model architectures and two pretraining objectives across 170 billion tokens, with and without finetuning.
result Causal decoder-only models trained on autoregressive language modeling exhibit strongest zero-shot generalization.
NAMEx merges experts using Nash bargaining for improved performance.
problem Sparse Mixture of Experts merging strategies lack a principled weighting mechanism.
method Reinterpreting expert merging through game theory, introducing Nash Merging and complex momentum.
result NAMEx consistently outperforms competing methods across various tasks and system sizes.
We propose a comprehensive end-to-end pipeline for Twitter hashtags recommendation system including data collection, supervised training setting and zero shot training setting. In the supervised training setting, we have proposed and compared the performance of various deep learning architectures, namely Convolutional …
Zero-shot learning transfers knowledge from seen classes to novel unseen classes to reduce human labor of labelling data for building new classifiers. Much effort on zero-shot learning however has focused on the standard multi-class setting, the more challenging multi-label zero-shot problem has received limited attent…
This paper proposes a zero-shot learning approach for audio classification based on the textual information about class labels without any audio samples from target classes. We propose an audio classification system built on the bilinear model, which takes audio feature embeddings and semantic class label embeddings as…
Paper explores using LLMs for zero-shot reinforcement learning in continuous spaces.
problem Leveraging LLMs for continuous state spaces in reinforcement learning.
method Disentangled In-Context Learning (DICL) to handle multivariate data and control signal.
result DICL produces well-calibrated uncertainty estimates in reinforcement learning settings.
Zero-Shot learning has been shown to be an efficient strategy for domain adaptation. In this context, this paper builds on the recent work of Bucher et al. [1], which proposed an approach to solve Zero-Shot classification problems (ZSC) by introducing a novel metric learning based objective function. This objective fun…
The paper develops a method to identify LLM-generated text without training.
problem Identifying LLM-generated text among human-generated content.
method Modeling LLM text as a sequential process and designing zero-shot statistical tests.
result The method can distinguish between text from known LLMs and non-sanctioned models with low false positive rates.
Object recognition systems usually require fully complete manually labeled training data to train the classifier. In this paper, we study the problem of object recognition where the training samples are missing during the classifier learning stage, a task also known as zero-shot learning. We propose a novel zero-shot l…
We present a generative framework for generalized zero-shot learning where the training and test classes are not necessarily disjoint. Built upon a variational autoencoder based architecture, consisting of a probabilistic encoder and a probabilistic conditional decoder, our model can generate novel exemplars from seen/…
A new framework uses text descriptions to improve protein design.
problem Lack of effective methods to incorporate textual descriptions in protein design.
method ProteinDT framework that combines text and protein structural information.
result ProteinDT significantly improves protein design accuracy and performance.
MoleculeSTM learns from molecule structures and texts for better drug design.
problem Lack of integration between chemical structures and textual knowledge in AI drug discovery.
method Jointly learns chemical structures and texts via contrastive learning, using a large dataset.
result MoleculeSTM achieves state-of-the-art performance in zero-shot tasks like structure-text retrieval and molecule editing.
Deep learning has improved performance on many natural language processing (NLP) tasks individually. However, general NLP models cannot emerge within a paradigm that focuses on the particularities of a single metric, dataset, and task. We introduce the Natural Language Decathlon (decaNLP), a challenge that spans ten ta…
Paper introduces Latent-CLIP for efficient text-image comparison in latent space.
problem Efficiently compare text and images in latent space without costly decoding.
method Trains CLIP model in latent space, uses Latent-CLIP rewards for noise optimization, and guides generation away from harmful content.
result Latent-CLIP matches CLIP performance on text-image classification and harmful content detection.
LLMs struggle with zero-shot annotation tasks due to model-internalized priors.
problem Impact of model-internalized priors on LLM performance in zero-shot annotation tasks.
method Investigated three dimensions: familiarity, decision stickiness, and susceptibility to misaligned task definitions.
result Nearly two-thirds of zero-shot errors are resistant to correction, with a rescue rate of 34.8%. Definition-Specific Familiarity (DSF) shows a positive association with model performance.
Paper proposes CCVAE for generalized zero-shot domain adaptation.
problem Adapting to unseen classes in target domain with limited labeled data.
method Coupled Conditional Variational Autoencoder (CCVAE).
result CCVAE generates synthetic target domain features for unseen classes.
Zero-shot understanding of accidents from surveillance videos using vision-language models
problem Accident understanding from surveillance videos
method Three-stage pipeline with vision-language similarity, metadata-driven multi-prompt reasoning, and entropy-gated pairwise adjudicator
result Substantial improvement in harmonic-mean score over baseline
Generative autoencoders offer a promising approach for controllable text generation by leveraging their latent sentence representations. However, current models struggle to maintain coherent latent spaces required to perform meaningful text manipulations via latent vector operations. Specifically, we demonstrate by exa…
This paper analyzes financial sentiment using LLMs and FinBERT, improving accuracy with few-shot examples.
problem Financial sentiment analysis for market evaluation.
method Application of large language models and FinBERT, with focus on prompt engineering and few-shot learning.
result GPT-4o achieves similar sentiment classification accuracy to FinBERT with fewer examples.
Despite the advancement of supervised image recognition algorithms, their dependence on the availability of labeled data and the rapid expansion of image categories raise the significant challenge of zero-shot learning. Zero-shot learning (ZSL) aims to transfer knowledge from labeled classes into unlabeled classes to r…
Study evaluates ensemble methods for zero-shot uncertainty quantification with diffusion models.
problem Quantifying uncertainty in zero-shot regression problems using diffusion models.
method Used diffusion probabilistic models for ensemble prediction and evaluated their effectiveness on various regression tasks.
result Ensemble methods consistently improve model prediction accuracy across different regression tasks.
A new topology design improves zero-shot classification performance in contrastive learning.
problem Improving zero-shot classification performance in contrastive visual-textual alignment.
method Proposed an alternative topology design using multiple class tokens and an oblique manifold with negative inner product.
result Improves zero-shot classification performance by an average of 6.1%.
AudioPaLM combines text and speech models to improve speech processing and translation.
problem Improving speech understanding and generation models.
method Unified multimodal architecture combining text and speech models.
result AudioPaLM significantly outperforms existing speech translation systems.
Obtaining common representations from different modalities is important in that they are interchangeable with each other in a classification problem. For example, we can train a classifier on image features in the common representations and apply it to the testing of the text features in the representations. Existing m…
Open-FinLLMs tackle financial tasks with multimodal capabilities.
problem Financial LLMs lack multimodal capabilities and real-world applicability.
method Developed Open-FinLLMs, an open-source multimodal financial LLM suite.
result Open-FinLLMs outperform advanced financial and general LLMs in diverse tasks.
New method improves RL/IL agents' adaptability to unseen environments.
problem Current RL/IL techniques struggle with generalizing to unseen environments.
method Zero-shot compositional policy learning with multi-modal fusion and attention mechanism.
result Language grounding enhances generalization across varied environments.
pRSL combines probabilistic rules to improve multi-label classification.
problem Modeling the structure between multi-label classes for better performance.
method Uses probabilistic propositional logic rules and belief propagation to combine predictions from multiple classifiers.
result pRSL achieves state-of-the-art performance on various benchmark datasets.
Unified framework for generating meteorological time series from text.
problem Lack of large-scale, physically grounded multimodal datasets and architectures ignoring spectral-temporal structure.
method Introduce MeteoCap-3B dataset and MTransformer model.
result State-of-the-art generation quality, accurate cross-modal alignment, strong semantic controllability.
Guided Flows enhance sample quality in conditional image generation and text-to-speech.
problem Improving sample quality in conditional generative models.
method Integrating classifier-free guidance into Flow Matching (FM) models for Continuous Normalizing Flows (CNFs).
result Guided Flows significantly improve sample quality in conditional image generation and text-to-speech synthesis.