On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,742 papers · 148 categories

Trend · papers per month

count % of corpus

11 results for “VLMs”

VLM Judges Can Rank but Cannot Score: Task-Dependent Uncertainty in Multimodal Evaluationcs.LG

VLM judges rank well but score poorly; task difficulty and annotation quality affect interval width.

problem VLMs as judges lack reliability indicators in multimodal evaluations.

method Conformal prediction using score-token log-probabilities.

result Evaluation uncertainty is task-dependent, affecting interval width and reliability.

VLM judges multimodal evaluation conformal prediction uncertainty quantification task difficulty

2026-04-28abs ↗pdf ↗

Martingale Doppelgänger-Eval: An Identification Framework for Auditing Candlestick Understanding in Vision-Language Modelsq-fin.CP

Martingale Doppelgänger-Eval benchmarks VLMs on candlestick evidence vs. trend extrapolation

problem Auditing whether VLMs use chart evidence or trend extrapolation

method Proving formal limitations and designing controlled mechanisms

result Identifying regression coefficients for evidence vs. trend

Martingale Doppelgänger-Eval VLMs candlestick trend extrapolation

2026-06-16abs ↗pdf ↗

Breach in the Shield: Unveiling the Vulnerabilities of Large Language Modelscs.LG

Study identifies key parameters and input dimensions making LLMs and VLMs brittle.

problem Vulnerability of large language and vision-language models to perturbations.

method Proposed FI measure based on information geometry to quantify sensitivity.

result Small subset of high FI parameters significantly contribute to brittleness.

Large Language Models Vision-Language Models Vulnerability Stability Measure Information Geometry

2025-03-28abs ↗pdf ↗

PyFi: Toward Pyramid-like Financial Image Understanding for VLMs via Adversarial Agentsq-fin.CP

PyFi uses adversarial agents to train VLMs on financial image understanding.

problem Training VLMs to understand complex financial questions.

method PyFi-600K dataset and adversarial MCTS mechanism.

result Fine-tuned VLMs improve by 19.52% and 8.06% on financial question accuracy.

financial image understanding vision language models adversarial training Monte Carlo Tree Search question chains

2025-12-11abs ↗pdf ↗

VERA-V: Variational Inference Framework for Jailbreaking Vision-Language Modelscs.CR

VERA-V uses variational inference to discover vulnerabilities in multimodal vision-language models.

problem Existing methods for jailbreaking vision-language models are brittle, limited, and focus on single attacks.

method VERA-V recasts jailbreak discovery as learning a joint posterior distribution over text-image prompts, using variational inference and three complementary strategies.

result VERA-V consistently outperforms state-of-the-art baselines, achieving up to 53.75% higher attack success rate.

vision-language models variational inference jailbreaking multimodal attacks language models

2025-10-20abs ↗pdf ↗

How Much Data Is Enough? Uniform Convergence Bounds for Generative & Vision-Language Models under Low-Dimensional Structurecs.LG

The study examines how much data is needed for generative and vision-language models to make reliable predictions.

problem Ensuring reliable predictions with low data for models used in medical decision support.

method Analyzes uniform convergence bounds for VLM-induced classifiers under low-dimensional semantic representations.

result Finite-sample uniform convergence bounds for accuracy and calibration functionals of VLM-induced classifiers.

generative models vision-language models uniform convergence calibration semantic embeddings

2025-12-28abs ↗pdf ↗

Training-free Diffusion Model Alignment with Sampling Demonscs.CV

Demon aligns diffusion models without retraining or backpropagation.

problem Aligning diffusion models with user preferences.

method Stochastic optimization to control noise distribution.

result Significantly improves aesthetics scores for text-to-image generation.

diffusion models alignment stochastic optimization sampling demons user preferences

2024-10-08abs ↗pdf ↗

LangDA: Building Context-Awareness via Language for Domain Adaptive Semantic Segmentationcs.CV

LangDA improves domain adaptation for semantic segmentation by learning context-aware scene descriptions.

problem Improving domain adaptation for semantic segmentation with dense prediction tasks.

method LangDA learns contextual relationships between objects via VLM-generated scene descriptions and aligns image features with text representation.

result LangDA sets new state-of-the-art across three DASS benchmarks, outperforming existing methods.

domain adaptation semantic segmentation context-awareness language models dense prediction

2025-03-17abs ↗pdf ↗

HyperINF: Unleashing the HyperPower of the Schulz's Method for Data Influence Estimationcs.LG

HyperINF improves influence function estimation for large models with better accuracy and efficiency.

problem Inaccurate and computationally expensive influence function estimation for large-scale models.

method HyperINF leverages Schulz's iterative algorithm and GFIM for low-rank approximation of Hessian matrix.

result HyperINF achieves superior accuracy and performance compared to existing methods on LoRA-tuned models.

influence function large-scale models Schulz's method GFIM matrix approximation

2024-10-07abs ↗pdf ↗

MM-DREX: Multimodal-Driven Dynamic Routing of LLM Experts for Financial Tradingq-fin.TR

MM-DREX adapts LLM experts for financial trading via dynamic routing.

problem Challenges of non-stationary financial markets and static expert designs.

method MM-DREX uses a VLM-powered dynamic router to allocate expert weights and designs heterogeneous trading experts.

result Significantly outperforms 15 baselines across key metrics.

financial trading large language models dynamic routing non-stationary markets vision-language models

2025-09-05abs ↗pdf ↗

Scaling DoRA: High-Rank Adaptation via Factored Norms and Fused Kernelscs.LG

DoRA improves adaptation efficiency for large models by factoring norms and fusing kernels.

problem High-rank DoRA is computationally expensive and infeasible on common GPUs.

method Factored norms and fused Triton kernels to reduce memory and speed up computation.

result Fused implementation is up to 2.0x faster for inference and 1.9x faster for gradient computation.

DoRA weight-decomposition factored norms fused kernels vision-language models

2026-03-23abs ↗pdf ↗