We introduce a new measure to evaluate the transferability of representations learned by classifiers. Our measure, the Log Expected Empirical Prediction (LEEP), is simple and easy to compute: when given a classifier trained on a source data set, it only requires running the target data set through this classifier once.…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “Transfer Measure”
Study measures impact of data and neural net similarity on transferability in restaurant sales data.
problem Identify indicators for successful transferability of neural nets across different data sets.
method Empirical study on sales data from six restaurants, calculating indicators based on data and neural net similarities.
result Negative correlations between transferability and indicators, allowing better model performance and fewer transfers.
Transfer learning aims at building robust prediction models by transferring knowledge gained from one problem to another. In the semantic Web, learning tasks are enhanced with semantic representations. We exploit their semantics to augment transfer learning by dealing with when to transfer with semantic measurements an…
Given a set of heterogeneous source datasets with their classifiers, how can we quickly find the most useful source dataset for a specific target task? We address the problem of measuring transferability between source and target datasets, where the source and the target have different feature spaces and distributions.…
CLS measures dataset similarity through decision rule performance.
problem Measuring dataset similarity in machine learning, especially for transfer learning and domain adaptation.
method Cross-Learning Score (CLS) measures similarity through bidirectional generalization performance of decision rules, linking to cosine similarity under canonical linear models.
result CLS effectively measures dataset similarity and transferability, validated on synthetic and real-world datasets.
Measures neural network information transfer for generalization.
problem Estimating the generalizable information in neural networks.
method Proposes Information Transfer () based on prequential coding.
result Consistently correlates with generalizable information in neural networks.
Paper proves convergence of measure transfer schemes using slicing and matching.
problem Iterative schemes for measure transfer and approximation problems.
method Slicing-and-matching procedure, stochastic gradient descent on Wasserstein space.
result Almost sure convergence proof for stochastic slicing-and-matching schemes.
Information-theoretic measures for non-linear causality detection: application to social media sentiment and cryptocurrency pricesphysics.data-an
Information transfer between time series is calculated by using the asymmetric information-theoretic measure known as transfer entropy. Geweke's autoregressive formulation of Granger causality is used to find linear transfer entropy, and Schreiber's general, non-parametric, information-theoretic formulation is used to …
Study shows benefits of transfer learning with neural networks.
problem Understanding generalization errors in transfer learning.
method Mean-field analysis applied to -ERM and fine-tuning.
result Established conditions for generalization error and convergence rates.
Paper explores properties of slice-matching operators for measure transfer.
problem Efficiently transferring measures in high dimensions.
method Examines an associated slice-matching operator with source, target measures and slicing directions.
result Establishes invariance, equivariance, Lipschitz continuity, and error bounds.
The paper analyzes and minimizes transferability of adversarial attacks between models in an ensemble.
problem Adversarial attacks can transfer between models, posing security risks.
method Introduces a gradient-based measure to assess and reduce transferability, and uses it during training to increase robustness.
result Demonstrates that the gradient-based measure can be used to increase an ensemble's robustness to adversarial attacks.
Unified approach for sample aggregation in transfer learning across various divergence measures.
problem Optimizing sample aggregation from source to target distributions for improved target performance.
method Unified algorithmic approach that adapts to multiple divergence measures via a weak modulus of transfer.
result Unified approach achieves near optimal rates in terms of the unknown strong modulus, applicable in more general settings.
New findings show transfer learning is possible even when density ratios are unbounded.
problem Transfer learning under unbounded density ratios.
method Low-degree polynomial estimators, general transfer inequality over .
result Non-trivial transfer learning possible under mild assumptions, including log-concave measures.
New method speeds up lead-lag detection between asynchronous time series.
problem Slow inference of lead-lag networks between long time series.
method Derive asymptotic distribution of Transfer Entropy and introduce time-shifted time series.
result Statistically validated lead-lag networks between time series.
Paper introduces a novel measure to analyze excess error in classification under covariate shift.
problem Analyzing excess error in classification under covariate shift.
method Utilizes vicinity information to characterize excess error.
result Faster or competitive convergence rates compared to previous techniques.
Task-specific scores are often used to optimize for and evaluate the performance of conditional text generation systems. However, such scores are non-differentiable and cannot be used in the standard supervised learning paradigm. Hence, policy gradient methods are used since the gradient can be computed without requiri…
Time Series Classification (TSC) has been an important and challenging task in data mining, especially on multivariate time series and multi-view time series data sets. Meanwhile, transfer learning has been widely applied in computer vision and natural language processing applications to improve deep neural network's g…
This work transfers causal knowledge between tasks for Individual Treatment Effect estimation.
problem Estimating Individual Treatment Effects (ITE) requires a large amount of data, making it challenging.
method The authors introduce a practical framework for efficient transfer of causal knowledge between tasks, using a Causal Inference Task Affinity (CITA) measure.
result ITE knowledge transfer can significantly reduce the amount of data needed for ITE estimation.
Continuity of earthquake flow map transfers Teichmüller dynamics results.
problem Transfer results from Teichmüller dynamics to earthquake flow.
method Analyze continuity of earthquake flow map and its inverse.
result Transfer results from Teichmüller dynamics to earthquake flow.
Local Granger Causalityq-bio.QM
Calculates local Granger causality for Gaussian and nonlinear systems.
problem Understanding causal influence in complex systems.
method Vector autoregression and information-theoretic approach.
result Local Granger causality offers a robust and fast method for time-directed information transfer.
We prove exponential decay of correlations for Hölder continuous observables with respect to any Gibbs measure for contact Anosov flows admitting Pesin sets with exponentially small tails. This is achieved by establishing strong spectral estimates for certain Ruelle transfer operators for such flows.
Training a source model optimally for its own task is suboptimal for downstream transfer.
problem The optimality of a source model for its own task hinders downstream transfer performance.
method Analyzes L2-SP ridge regression, characterizes transfer-optimal source penalty, and identifies alignment-dependent effects.
result Transfer benefits from stronger source regularization when aligned imperfectly, and from weaker regularization when aligned perfectly.
Paper tackles robust transfer learning with unreliable source data.
problem Challenges in robust transfer learning stemming from ambiguity in Bayes classifiers and weak transferable signals.
method Introduces ambiguity level, proposes Transfer Around Boundary (TAB) model, establishes general theorem.
result Demonstrates efficiency and robustness of TAB model improving classification while avoiding negative transfer.
When related learning tasks are naturally arranged in a hierarchy, an appealing approach for coping with scarcity of instances is that of transfer learning using a hierarchical Bayes framework. As fully Bayesian computations can be difficult and computationally demanding, it is often desirable to use posterior point es…
New technique explains convergence in ML models with data modifications.
problem Understanding convergence of ML models under data changes.
method Analogue of Fatou's lemma and gamma-convergence.
result Relevance and applications in general ML tasks and domain adaptation.
Paper tackles continuous transfer learning with evolving target domains.
problem Challenges of negative transfer in evolving target domains.
method Proposes label-informed C-divergence for measuring distribution shift and negative transfer.
result Demonstrates effectiveness of TransLATE framework in minimizing classification error and C-divergence.
We analyze a simple asset transfer model in which the transfer amount is a fixed fraction of the giver's wealth. The model is analyzed in a new way by Laplace transforming the master equation, solving it analytically and numerically for the steady-state distribution, and exploring the solutions for various values o…
Transfer learning improves model robustness against adversarial attacks.
problem Understanding how transfer learning affects model robustness against adversarial attacks.
method Extensive empirical evaluations of white-box and black-box attacks on fine-tuned transfer learning models.
result Adversarial examples are more transferable when fine-tuning is used than when networks are trained independently.
Study improves model fit by transferring info from related datasets.
problem Improving model fit on target data using source data.
method Proposes a transfer learning algorithm for GLMs, derives error bounds, and introduces detection of informative sources.
result Theoretical and practical improvements over classical methods in high-dimensional GLM settings.
This paper proposes a new method to improve domain adaptation by distinguishing between marginal and dependence structure differences.
problem Existing domain adaptation methods fail to differentiate between marginal and dependence structure differences, leading to suboptimal transferability.
method The paper introduces a new approach that measures and optimizes the differences in internal dependence structure separately from marginals.
result The new method significantly improves transferability and robustness compared to existing benchmarks on real-world datasets.
TL-ANDI distills context from source data to improve transfer learning for TFMs.
problem Limited transfer learning due to context-size constraints and distribution shifts.
method TL-ANDI uses posterior-aware distillation to construct a compact source context and locally distills labels.
result Improves transfer performance by addressing context-size and distribution shifts.
Telecommunication (Telco) outdoor position recovery aims to localize outdoor mobile devices by leveraging measurement report (MR) data. Unfortunately, Telco position recovery requires sufficient amount of MR samples across different areas and suffers from high data collection cost. For an area with scarce MR samples, i…
This paper defines and quantifies transferability in domain generalization.
problem Understanding and quantifying transferability between domains.
method Formal definition and estimation of transferability, upper bound for target error.
result Many algorithms do not learn transferable features, proposing a new algorithm.
CONCERT improves transfer learning by borrowing partial information from auxiliary datasets.
problem Inefficiency of global similarity measures in transfer learning for high-dimensional data.
method Conditional spike-and-slab prior with covariate-specific priors for robust partial information transfer.
result CONCERT achieves variable selection and information transfer simultaneously, improving performance on the target.
Study improves hypothesis transfer learning for functional linear models.
problem Incompatible TL techniques for high-dimensional FLR methods due to infinite-dimensional nature of functional data.
method Proposes two algorithms for hypothesis transfer learning in RKHS framework, leveraging RKHS distance and aggregation techniques.
result Establishes asymptotic lower bounds and matching upper bounds for the proposed algorithms, demonstrating their effectiveness.
We propose Gaussian optimal transport for Image style transfer in an Encoder/Decoder framework. Optimal transport for Gaussian measures has closed forms Monge mappings from source to target distributions. Moreover interpolates between a content and a style image can be seen as geodesics in the Wasserstein Geometry. Usi…
URL benchmark evaluates uncertainty quantification in pretrained models.
problem Need for reliable uncertainty estimates in transferable pretrained models.
method Proposes URL benchmark to measure transferability of representations and uncertainty estimates.
result Transferable uncertainty quantification remains challenging but not contradictory to traditional goals.
Quantifying Causal Coupling Strength: A Lag-specific Measure For Multivariate Time Series Related To Transfer Entropyphysics.data-an
While it is an important problem to identify the existence of causal associations between two components of a multivariate time series, a topic addressed in Runge et al. (2012), it is even more important to assess the strength of their association in a meaningful way. In the present article we focus on the problem of d…
Despite increasing focus on data publication and discovery in materials science and related fields, the global view of materials data is highly sparse. This sparsity encourages training models on the union of multiple datasets, but simple unions can prove problematic as (ostensibly) equivalent properties may be measure…
Transfer Learning (TL) has shown great potential to accelerate Reinforcement Learning (RL) by leveraging prior knowledge from past learned policies of relevant tasks. Existing transfer approaches either explicitly computes the similarity between tasks or select appropriate source policies to provide guided explorations…
AutoTransfer improves subject transfer learning for biosignal datasets.
problem Subject transfer learning for challenging biosignal datasets.
method Regularization framework with mutual information or divergence penalties.
result Improves subject transfer learning performance on EEG, EMG, and ECoG datasets.
Adversarial examples are of wide concern due to their impact on the reliability of contemporary machine learning systems. Effective adversarial examples are mostly found via white-box attacks. However, in some cases they can be transferred across models, thus enabling them to attack black-box models. In this work we ev…
Causal discovery is a fundamental problem in statistics and has wide applications in different fields. Transfer Entropy (TE) is a important notion defined for measuring causality, which is essentially conditional Mutual Information (MI). Copula Entropy (CE) is a theory on measurement of statistical independence and is …
Novel model improves clinical risk prediction by transferring knowledge between tasks over time.
problem Negative transfer in multi-task learning for clinical risk prediction.
method Temporal Probabilistic Asymmetric Multi-Task Learning (TPAMTL).
result Significantly outperforms various deep learning models for time-series prediction.
Many machine intelligence techniques are developed in E-commerce and one of the most essential components is the representation of IDs, including user ID, item ID, product ID, store ID, brand ID, category ID etc. The classical encoding based methods (like one-hot encoding) are inefficient in that it suffers sparsity pr…
Develops methods to measure and set function-space learning rates in neural networks.
problem Measuring and optimizing changes in neural network output functions.
method Efficient methods to measure and set function-space learning rates, requiring minimal computational overhead.
result Demonstrates FLeRM (Function-space Learning Rate Matching) for hyperparameter transfer across model scales.
Novel method uses information theory to measure causal influences during transient neural events.
problem Characterizing network interactions during transient neural events.
method Structural Causal Models, Information Theory, Transfer Entropy, Dynamic Causal Strength, Relative Dynamic Causal Strength.
result Introduced a novel measure, relative Dynamic Causal Strength, with theoretical and empirical support.
Unified approach to domain generalization by aligning gradients and Hessians.
problem Developing models that generalize well across unseen domains.
method Moment Alignment, extending transfer measure to DG, aligning derivatives across domains.
result Moment Alignment unifies gradient and Hessian matching approaches, improving generalizability.