TL-ANDI distills context from source data to improve transfer learning for TFMs.
problem Limited transfer learning due to context-size constraints and distribution shifts.
method TL-ANDI uses posterior-aware distillation to construct a compact source context and locally distills labels.
result Improves transfer performance by addressing context-size and distribution shifts.
LLMs compress financial texts, but distort decision-making.
problem LLMs compress financial texts, altering decision-making.
method Analyzed two diagnostic patterns: decontextualization and model dependency. Proposed Agentic Context Compression.
result LLM-compressed financial texts alter decision-making.
Models for audio source separation usually operate on the magnitude spectrum, which ignores phase information and makes separation performance dependant on hyper-parameters for the spectral front-end. Therefore, we investigate end-to-end source separation in the time-domain, which allows modelling phase information and…
Transformers can learn Markov processes with constant depth, surprising results.
problem Understanding how transformers learn context in Markov processes.
method Empirical study and theoretical analysis of attention-based transformers on Markov data.
result Transformers with constant depth can achieve low test loss on Markov sequences, matching empirical and theoretical findings.
This paper examines risks and uncertainties of changing data sources in machine learning for official statistics.
problem Risks and uncertainties associated with changing data sources in machine learning for official statistics.
method An overview of risks, causes, and repercussions of changing data sources, with a checklist of measures.
result Maintaining integrity, reliability, consistency, and relevance in official statistics.
Much information available on the web is copied, reused or rephrased. The phenomenon that multiple web sources pick up certain information is often called trend. A central problem in the context of web data mining is to detect those web sources that are first to publish information which will give rise to a trend. We p…
Improves medical note processing by training model on related concepts and global context.
problem Scarce and imbalanced labeled training data limits generalizability of automated abbreviation disambiguation models.
method Data augmentation using related medical concepts and global context information within medical notes.
result Model accuracy improved by almost 14% on CASI dataset and 4% on i2b2 dataset.
CAMul forecasts with calibrated and accurate multi-view time-series data.
problem Combining diverse data sources for reliable time-series forecasting.
method CAMul integrates multi-modal data views dynamically, assigning importance based on context.
result CAMul outperforms state-of-the-art models by 25% in accuracy and calibration.
Transformers with MLP heads outperform linear baselines in in-context learning.
problem Understanding in-context learning in Transformers with nonlinear MLP heads.
method Analyzing a model with two-layer MLPs trained via gradient steps and fully optimized, under high-dimensional asymptotics.
result Nonlinear MLPs enhance ICL performance, particularly on nonlinear tasks.
Developing a visual platform for faster astronomical source cataloging.
problem Speeding up cataloging of large area surveys in radio astronomy.
method Integration of advanced source finding and classification tools into a visual analytic platform.
result Improvement and acceleration of cataloging process in astronomical surveys.
Magnetoencephalography (MEG) and Electroencephalography (EEG) source estimates have thus far mostly been derived sample by sample, i.e., independent of each other in time. However, neuronal assemblies are heavily interconnected, constraining the temporal evolution of neural activity in space as detected by MEG and EEG.…
This article addresses the modeling of reverberant recording environments in the context of under-determined convolutive blind source separation. We model the contribution of each source to all mixture channels in the time-frequency domain as a zero-mean Gaussian random variable whose covariance encodes the spatial cha…
Contextualized ML learns context-dependent effects using deep learning.
problem Learning heterogeneous and context-dependent effects in data.
method Applying deep learning to the meta-relationship between contextual information and context-specific parametric models.
result Unified framework for cluster analysis and cohort modeling.
New algorithm improves source separation with multi-trial supervision.
problem Non-convex optimization and interpretability of independent components.
method Proximal gradient-type algorithm in invertible matrices with backpropagation for joint learning.
result Increased success rate of non-convex optimization and improved interpretability.
New theory allows ICA without assuming non-Gaussian sources.
problem Traditional ICA struggles with Gaussian sources.
method Developed identifiability theory based on second-order statistics and sparsity.
result Identifiability theory and estimation methods validated experimentally.
LangDA improves domain adaptation for semantic segmentation by learning context-aware scene descriptions.
problem Improving domain adaptation for semantic segmentation with dense prediction tasks.
method LangDA learns contextual relationships between objects via VLM-generated scene descriptions and aligns image features with text representation.
result LangDA sets new state-of-the-art across three DASS benchmarks, outperforming existing methods.
This work proposes a meta-learning approach for better adaptation of source code models.
problem Adapting source code models to unseen local contexts.
method Formulated as a meta-learning problem, selecting targeted information for adaptation.
result Improved performance in code auto-completion tasks, especially for identifiers and literals.
Backdoors can be implanted in neural models of source code, and we detect and remove them.
problem Vulnerability of neural models to backdoors in source code.
method Defined and implemented various backdoor classes, adapted robust statistics algorithms, and detected poisoned data through spectral signatures.
result Demonstrated the ease of injecting and removing backdoors in neural models of source code.
Two types of nonidentifiability in latent position graphs identified and characterized.
problem Identifying and characterizing nonidentifiability in latent position random graph models.
method Defined and examined subspace nonidentifiability and model-based nonidentifiability, providing examples and characterizing limits.
result Characterized the limits of model-based nonidentifiability and obtained additional limiting results for specific graph models.
This paper tackles rare word problem in low-resource language pairs using NMT.
problem Rare word problem in neural machine translation, especially for low-resource languages.
method Three solutions: enhanced source context, morphology learning, and wordnet synonyms.
result Significant improvements in BLEU scores (+1.0 points) on English-Vietnamese and Japanese-Vietnamese.
Non-negative blind source separation (non-negative BSS), which is also referred to as non-negative matrix factorization (NMF), is a very active field in domains as different as astrophysics, audio processing or biomedical signal processing. In this context, the efficient retrieval of the sources requires the use of sig…
A number of modern learning tasks involve estimation from heterogeneous information sources. This includes classification with labeled and unlabeled data as well as other problems with analogous structure such as competitive (game theoretic) problems. The associated estimation problems can be typically reduced to solvi…
Out-of-vocabulary word translation is a major problem for the translation of low-resource languages that suffer from a lack of parallel training data. This paper evaluates the contributions of target-language context models towards the translation of OOV words, specifically in those cases where OOV translations are der…
Modern audio source separation techniques rely on optimizing sequence model architectures such as, 1D-CNNs, on mixture recordings to generalize well to unseen mixtures. Specifically, recent focus is on time-domain based architectures such as Wave-U-Net which exploit temporal context by extracting multi-scale features. …
Blind source separation (BSS) is a very popular technique to analyze multichannel data. In this context, the data are modeled as the linear combination of sources to be retrieved. For that purpose, standard BSS methods all rely on some discrimination principle, whether it is statistical independence or morphological di…
This work introduces a method to decompose uncertainty in in-context learning for large language models.
problem Understanding the sources of uncertainty in in-context learning for large language models.
method Variational uncertainty decomposition framework without sampling from latent parameter posterior.
result Quantitative and qualitative validation of decomposed epistemic and aleatoric uncertainties.
Distributed, online data mining systems have emerged as a result of applications requiring analysis of large amounts of correlated and high-dimensional data produced by multiple distributed data sources. We propose a distributed online data classification framework where data is gathered by distributed data sources and…
FibQuant improves KV-cache compression for long-context inference.
problem Memory traffic bottleneck in long-context inference due to KV cache growth.
method Introduces FibQuant, a universal vector quantizer that combines Beta-quantile radii, Fibonacci/Roberts-Kronecker directions, and Lloyd-Max refinement.
result FibQuant achieves high compression rates with minimal loss in attention cosine similarity.
EmTract extracts emotions from financial social media text.
problem Understanding investor emotions in financial markets.
method Annotated data, DistilBERT model, embedding space augmentation.
result EmTract outperforms existing emotion classifiers.
Sparse Blind Source Separation (sparse BSS) is a key method to analyze multichannel data in fields ranging from medical imaging to astrophysics. However, since it relies on seeking the solution of a non-convex penalized matrix factorization problem, its performances largely depend on the optimization strategy. In this …
ASKAP observes a region of the Galactic plane, identifying 3963 radio sources.
problem Characterizing radio sources in the Galactic plane using ASKAP observations.
method 912 MHz observations with 15 ASKAP antennas, source characterization using CAESAR.
result Differential source counts in agreement with previous surveys, spectral index estimation for sources.
New method improves source separation using adversarial NMF.
problem Improving source separation in single channel signals.
method Adversarial training of non-negative matrix factorization (NMF).
result Adversarial NMF leads to better signal reconstruction.
This paper is concerned with structured machine learning, in a supervised machine learning context. It discusses how to make joint structured learning on interdependent objects of different nature, as well as how to enforce logical con-straints when predicting labels. We explain how this need arose in a Document Unders…
New insights into how depth and width affect in-context learning in deep models.
problem Understanding how various resources impact in-context learning in deep models.
method Analyzed linear regression in a deep linear self-attention model, varying resources like depth, width, context length, and training steps.
result Increasing depth improves in-context learning even at infinite context length, contrary to previous findings.
Contrastive Code Representation Learning improves code summarization and type inference.
problem Code representations are sensitive to edits, hindering downstream semantic understanding tasks.
method ContraCode: a contrastive pre-training task that learns code functionality.
result Contrastive pre-training improves code summarization and type inference accuracy.
Optimal domain adaptation model using Fisher's Linear Discriminant.
problem Improving classification accuracy across different domains.
method Convex combination of source and target hypotheses, derived under 0-1 loss.
result Effective classifier can be computed without direct source task information.
Purpose: In the present work we describe the correction of diffusion-weighted MRI for site and scanner biases using a novel method based on invariant representation. Theory and Methods: Pooled imaging data from multiple sources are subject to variation between the sources. Correcting for these biases has become very im…
LLMs can be tricked into recalling facts based on context clues.
problem Manipulation of LLMs' factual recall through context changes.
method Mathematical exploration of transformers' associative memory properties.
result Transformers use self-attention and value matrix for associative memory.
Digital currencies exhibit multifractality due to heavy-tailed returns and temporal correlations.
problem Understanding market inefficiencies and predicting volatility in digital currencies.
method Multifractal cross-correlation analysis (MFCCA) and multifractal detrended fluctuation analysis (MFDFA).
result Temporal correlations are the primary source of multifractality in digital currency markets.
BeST metric selects best source models for transfer learning.
problem Efficiently selecting top source models for a new target task.
method Quantization-level optimization procedure for similarity measure.
result BeST consistently identifies best source models for target tasks.
Research in deep reinforcement learning (RL) has coalesced around improving performance on benchmarks like the Arcade Learning Environment. However, these benchmarks conspicuously miss important characteristics like abrupt context-dependent shifts in strategy and temporal sensitivity that are often present in real-worl…
Improves text classification on new domains using distance-based measures and dynamic domain selection.
problem Improving text classification performance on new domains with limited labeled data.
method Develops DistanceNet and DistanceNet-Bandit models using distance measures to adapt to new domains.
result DistanceNet and DistanceNet-Bandit models outperform baseline methods in unsupervised domain adaptation.
Unsupervised learning can leverage large-scale data sources without the need for annotations. In this context, deep learning-based auto encoders have shown great potential in detecting anomalies in medical images. However, state-of-the-art anomaly scores are still based on the reconstruction error, which lacks in two e…
We introduce a new representation learning approach for domain adaptation, in which data at training and test time come from similar but different distributions. Our approach is directly inspired by the theory on domain adaptation suggesting that, for effective domain transfer to be achieved, predictions must be made b…
New method improves source separation using NMF and adversarial learning.
problem Source separation in single channel data.
method Maximum Discrepancy Generative Regularization applied to NMF.
result Improvement in reconstructed signals, especially in weak supervision scenarios.
This paper benchmarks FinGPT for financial datasets using open-source large language models.
problem Challenges in integrating GPT-based models with financial datasets.
method Instruction Tuning paradigm for open-source large language models adapted for financial contexts.
result Demonstrates the effectiveness and adaptability of FinGPT in financial tasks.
In the following paper we investigate the question: when is a transitive topological groupoid continuously isomorphic to a Lie groupoid? We present many results on the matter which may be considered generalizations of the Hilbert's fifth problem to this context. Most notably we present a "solution" to the problem for p…
Human irrationality can improve AI design, study shows.
problem Improving AI by learning from human decision-making biases.
method Developed a novel POMDP model to simulate human decision-making in contextual choice tasks.
result Reinforcement learners can exploit human irrationalities to make better decisions.