Language model benchmarks often misrepresent true understanding, revealing vulnerabilities in evaluation methods.
problem Language model benchmarks fail to accurately reflect true language understanding and adaptability.
method Systematic analysis of NLP evaluation frameworks, identifying vulnerabilities in static benchmarks, human evaluation protocols, and LLM-as-judge frameworks.
result Current evaluation methods are unreliable and need improvement to accurately assess LLM performance.
Despite the remarkable performance of deep neural networks on various computer vision tasks, they are known to be susceptible to adversarial perturbations, which makes it challenging to deploy them in real-world safety-critical applications. In this paper, we conjecture that the leading cause of adversarial vulnerabili…
Increasing numbers of software vulnerabilities are discovered every year whether they are reported publicly or discovered internally in proprietary code. These vulnerabilities can pose serious risk of exploit and result in system compromise, information leaks, or denial of service. We leveraged the wealth of C and C++ …
Improved estimation of concentration using half-spaces for adversarial vulnerability.
problem Understanding the concentration of measure phenomenon and its impact on adversarial vulnerability.
method Extending Gaussian Isoperimetric Inequality to non-spherical Gaussian measures and arbitrary ℓ_p-norms, using half-spaces to estimate concentration.
result Proposed method finds tighter intrinsic robustness bounds, providing evidence against concentration as a cause of adversarial vulnerability.
New benchmarks focus on LLM risk in finance, not just accuracy.
problem Standard benchmarks ignore LLM financial risks, leading to unsafe deployment.
method Three-level agenda: model, workflow, and system stress-testing.
result Hidden weaknesses in LLMs are missed by traditional benchmarks.
Generative text classifiers are most vulnerable to membership inference attacks.
problem Privacy threat from Membership Inference Attacks on generative text classifiers.
method Comprehensive empirical evaluation of generative, discriminative, and pseudo-generative classifiers across various datasets.
result Generative classifiers explicitly modeling P(X,Y) are most vulnerable to membership leakage. VERA-V uses variational inference to discover vulnerabilities in multimodal vision-language models.
problem Existing methods for jailbreaking vision-language models are brittle, limited, and focus on single attacks.
method VERA-V recasts jailbreak discovery as learning a joint posterior distribution over text-image prompts, using variational inference and three complementary strategies.
result VERA-V consistently outperforms state-of-the-art baselines, achieving up to 53.75% higher attack success rate.
GTA is the first backdoor attack on GNNs, demonstrating vulnerabilities in graph-oriented security models.
problem Vulnerability of graph neural networks to backdoor attacks.
method Graph-oriented triggers, dynamic adaptation, model-agnostic, attack-extensible.
result Demonstrates severe threats to graph classification and node classification tasks.
Benchmark evaluates AI-generated financial QA hallucinations, highlighting system vulnerabilities.
problem Ensuring factual accuracy of AI-generated financial QA outputs.
method Developed a benchmark dataset and evaluated six detection methods under clean and noisy conditions.
result LLM-based judges and embedding methods perform best, but degrade under noisy conditions.
This paper investigates the resilience and robustness of Deep Reinforcement Learning (DRL) policies to adversarial perturbations in the state space. We first present an approach for the disentanglement of vulnerabilities caused by representation learning of DRL agents from those that stem from the sensitivity of the DR…
Model for optimal cybersecurity investment considering clustered cyberattacks.
problem Optimal investment in cybersecurity to reduce system vulnerability under clustered cyberattacks.
method Developed a continuous-time stochastic model using a Hawkes process, extended Gordon-Loeb model, solved as a Markovian stochastic optimal control problem.
result Investment policies that account for attack clustering lead to more effective and responsive strategies, improving upon static and Poisson-based approaches.
Deep neural networks are vulnerable to adversarial examples, which becomes one of the most important research problems in the development of deep learning. While a lot of efforts have been made in recent years, it is of great significance to perform correct and complete evaluations of the adversarial attack and defense…
The detection of software vulnerabilities (or vulnerabilities for short) is an important problem that has yet to be tackled, as manifested by the many vulnerabilities reported on a daily basis. This calls for machine learning methods for vulnerability detection. Deep learning is attractive for this purpose because it a…
We take the holistic approach of computing an OTC claim value that incorporates credit and funding liquidity risks and their interplays, instead of forcing individual price adjustments: CVA, DVA, FVA, KVA. The resulting nonlinear mathematical problem features semilinear PDEs and FBSDEs. We show that for the benchmark v…
Each year, thousands of software vulnerabilities are discovered and reported to the public. Unpatched known vulnerabilities are a significant security risk. It is imperative that software vendors quickly provide patches once vulnerabilities are known and users quickly install those patches as soon as they are available…
A membership inference attack (MIA) against a machine-learning model enables an attacker to determine whether a given data record was part of the model's training data or not. In this paper, we provide an in-depth study of the phenomenon of disparate vulnerability against MIAs: unequal success rate of MIAs against diff…
This paper benchmarks time-series adversarial defenses and attacks.
problem Adversarial attacks on time-series data.
method Detailed benchmarking of adversarial defense methods in the L∞ threat model. result Adversarial defenses offer robustness against both strong white-box and black-box attacks.
Over the past few years, neural networks were proven vulnerable to adversarial images: targeted but imperceptible image perturbations lead to drastically different predictions. We show that adversarial vulnerability increases with the gradients of the training objective when viewed as a function of the inputs. Surprisi…
AFD learns features resistant to adversarial attacks.
problem Vulnerability of neural networks to adversarial attacks.
method AFD learns features invariant to adversarial perturbations through a game of predictive and robust features.
result AFD effectively resists a wide range of attack types and strengths.
Deep neural networks are vulnerable to adversarial examples, which poses security concerns on these algorithms due to the potentially severe consequences. Adversarial attacks serve as an important surrogate to evaluate the robustness of deep learning models before they are deployed. However, most of existing adversaria…
New framework improves adversarial robustness in one-stage L2D.
problem Adversarial robustness in one-stage Learning-to-Defer (L2D).
method Formalizes attacks, proposes cost-sensitive adversarial surrogate losses, establishes theoretical guarantees.
result Improves robustness against untargeted and targeted attacks while preserving clean performance.
New risk measures assess cryptocurrency market vulnerabilities during financial distress.
problem Capturing systemic risk in cryptocurrency markets during financial distress.
method Introducing Vulnerability Conditional Risk Measures (VCoES) and related measures.
result Validated theoretical insights and demonstrated practical relevance in cryptocurrency market.
Enhances GNN robustness during inference using Conditional Random Fields.
problem Vulnerability of GNNs to adversarial attacks.
method Post-hoc approach using Conditional Random Fields (CRF).
result Improves robustness of GNNs across various models.
Preschool attendance correlates with lower developmental vulnerabilities in Queensland, Australia.
problem Understanding the relationship between preschool attendance and developmental vulnerabilities in different regions.
method Data Analysis and Machine Learning to identify clusters of socio-demographic variables.
result Identified three clusters with varying socio-demographic variables affecting the relationship between preschool attendance and developmental vulnerabilities.
The exponential increase in dependencies between the cyber and physical world leads to an enormous amount of data which must be efficiently processed and stored. Therefore, computing paradigms are evolving towards machine learning (ML)-based systems because of their ability to efficiently and accurately process the eno…
Study on measuring vulnerability of neural network parameters via corruption.
problem Understanding the robustness and generalization of deep neural networks.
method Proposes an indicator to measure parameter robustness via parameter corruption and provides a gradient-based estimation.
result Demonstrates the effectiveness of the proposed indicator and training method in improving parameter robustness and accuracy.
PriceSeer benchmarks LLMs in real-time stock prediction.
problem Evaluating LLMs' stock prediction accuracy and robustness.
method Real-time benchmark with 110 U.S. stocks, internal and external information expansion.
result LLMs perform suboptimally in long-term predictions due to fake news and specific industries.
Study reveals how high-dimensional models are vulnerable to consistent adversarial attacks.
problem Understanding the vulnerability of high-dimensional linear classifiers to adversarial attacks.
method Introducing a new error metric to quantify model vulnerability, and rigorously characterizing these metrics in asymptotic settings.
result As models become more overparameterized, their vulnerability to label-preserving perturbations increases.
Thousands of security vulnerabilities are discovered in production software each year, either reported publicly to the Common Vulnerabilities and Exposures database or discovered internally in proprietary code. Vulnerabilities often manifest themselves in subtle ways that are not obvious to code reviewers or the develo…
We are concerned with the vulnerability of computer vision models to distributional shifts. We formulate a combinatorial optimization problem that allows evaluating the regions in the image space where a given model is more vulnerable, in terms of image transformations applied to the input, and face it with standard se…
The paper analyzes how much data points can be altered to change their rank in nearest neighbor searches.
problem Vulnerability of nearest neighbor search in high-dimensional data.
method Statistical analysis of perturbation needed to change neighbor rank.
result Derived statistical distribution of perturbation needed to modify neighbor rank.
Machine-learning models have been recently used for detecting malicious Android applications, reporting impressive performances on benchmark datasets, even when trained only on features statically extracted from the application, such as system calls and permissions. However, recent findings have highlighted the fragili…
Using a suitable change of probability measure, we obtain a novel Poisson series representation for the arbitrage- free price process of vulnerable contingent claims in a regime-switching market driven by an underlying continuous- time Markov process. As a result of this representation, along with a short-time asymptot…
New method certifies joint adversarial robustness of model ensembles.
problem Ensuring robustness of model ensembles against adversarial attacks.
method Proposes a novel technique to certify joint robustness, building on prior work on single-model robustness certification.
result Demonstrates the effectiveness of certifying joint robustness of ensembles, improving understanding of ensemble defenses.
DVERGE diversifies adversarial vulnerabilities to enhance robust ensemble models.
problem Diverse adversarial vulnerabilities for robust ensemble models.
method Isolates and diversifies adversarial vulnerabilities through distillation and training.
result Achieves higher robustness against transfer attacks compared to previous methods.
Foresight Arena benchmarks AI forecasting on real-world markets, isolating predictive edge.
problem Evaluating AI forecasting ability in real-world markets is challenging due to overfitting, centralized trust, and conflated metrics.
method Permissionless, on-chain benchmark using probabilistic forecasts, commit-reveal protocol, and smart contracts.
result Demonstrates the need for 350 predictions to reliably distinguish agents of different skill levels.
New method defends RL agents from poisoning attacks without MDP knowledge.
problem Poisoning attacks on RL systems can cause learning failures.
method Generic poisoning framework for online RL, Vulnerability-Aware Adversarial Critic Poison (VA2C-P).
result Successfully prevents RL agents from learning good policies or converging to target policies.
Motivated by the problem of automated repair of software vulnerabilities, we propose an adversarial learning approach that maps from one discrete source domain to another target domain without requiring paired labeled examples or source and target domains to be bijections. We demonstrate that the proposed adversarial l…
Benchmark evaluates LLM trading agents by masking identifiers to prevent memory leaks.
problem Evaluate LLM trading agents without relying on market memory or noise.
method Data-side masking protocol, Barra-style performance attribution framework.
result LLM agents' returns are largely explained by market and style exposure, not stock selection.
Improved security of smart contracts by classifying them into four categories.
problem Detecting and classifying vulnerabilities in smart contracts efficiently.
method Used AWD-LSTM for multi-class classification, addressing class imbalance.
result Achieved a weighted average Fbeta score of 90.0%.
An intriguing property of deep neural networks is their inherent vulnerability to adversarial inputs, which significantly hinders their application in security-critical domains. Most existing detection methods attempt to use carefully engineered patterns to distinguish adversarial inputs from their genuine counterparts…
This paper extensively evaluates the vulnerability of capsule networks to different adversarial attacks. Recent work suggests that these architectures are more robust towards adversarial attacks than other neural networks. However, our experiments show that capsule networks can be fooled as easily as convolutional neur…
Designing models that are robust to small adversarial perturbations of their inputs has proven remarkably difficult. In this work we show that the reverse problem---making models more vulnerable---is surprisingly easy. After presenting some proofs of concept on MNIST, we introduce a generic tilting attack that injects …
Graph neural network (GNN), as a powerful representation learning model on graph data, attracts much attention across various disciplines. However, recent studies show that GNN is vulnerable to adversarial attacks. How to make GNN more robust? What are the key vulnerabilities in GNN? How to address the vulnerabilities …
Study reveals how neural network smoothness affects their vulnerability to adversarial attacks.
problem Understanding adversarial vulnerability in deep learning networks.
method Analysis of manifold smoothness and generalization capability of deep neural networks trained with local errors.
result High generalization accuracy requires a fast power-law decay of eigen-spectrum of hidden representations.
The success of modern Artificial Intelligence (AI) technologies depends critically on the ability to learn non-linear functional dependencies from large, high dimensional data sets. Despite recent high-profile successes, empirical evidence indicates that the high predictive performance is often paired with low robustne…
Boosting algorithms predict financial vulnerability of farmers in Chile and Tunisia.
problem Predict financial vulnerability of farmers in Chile and Tunisia using environmental data.
method Interpretable boosting algorithms based on ridge-regularized generalized linear models.
result Interaction effects improve predictive power only when included in two-step boosting.
Machine learning models that can exploit the inherent structure in data have gained prominence. In particular, there is a surge in deep learning solutions for graph-structured data, due to its wide-spread applicability in several fields. Graph attention networks (GAT), a recent addition to the broad class of feature le…