arXiv research

Proposes a method to evaluate generalizability in causal inference models.

Sampling a fraction of pairs can match full evaluation in machine learning losses.

A new cost-frugal HPO method controls training cost during optimization.

Frengression models causal data flexibly and faithfully.

New model considers varying costs in learning, outperforming existing methods.

Frugal method predicts multiple local electricity loads efficiently.

A new optimizer saves significant shots in quantum machine learning.

Frugal Flows learn complex data and infer marginal causal effects.

Spectral regularization improves learning over combinatorial spaces with limited data.

In many real-world scenarios where data is high dimensional, test time acquisition of features is a non-trivial task due to costs associated with feature acquisition and evaluating feature value. The need for highly confident models with an extremely frugal acquisition of features can be addressed by allowing a feature…

New approach selects sparse features without validation.

Model learns brevity by exposing to easy problems, improving efficiency without explicit length penalties.

We present an LDA approach to entity disambiguation. Each topic is associated with a Wikipedia article and topics generate either content words or entity mentions. Training such models is challenging because of the topic and vocabulary size, both in the millions. We tackle these problems using a novel distributed infer…

Unimodal approach for group-level emotion recognition without individual features.

The family of temporal difference (TD) methods span a spectrum from computationally frugal linear methods like TD(λ) to data efficient least squares methods. Least square methods make the best use of available data directly computing the TD solution and thus do not require tuning a typically highly sensitive learning r…

We provide a mathematical definition of fragility and antifragility as negative or positive sensitivity to a semi-measure of dispersion and volatility (a variant of negative or positive "vega") and examine the link to nonlinear effects. We integrate model error (and biases) into the fragile or antifragile context. Unli…

Paper introduces SPADE method to protect classifiers from OOD and adversarial samples.

Algorithms typically come with tunable parameters that have a considerable impact on the computational resources they consume. Too often, practitioners must hand-tune the parameters, a tedious and error-prone task. A recent line of research provides algorithms that return nearly-optimal parameters from within a finite …

ASPIRE improves amortized posterior inference for Bayesian inverse problems.

A new method finds DAG models without ground truth.

Optimal transport aggregation combines distributed MoE models efficiently.

When building large-scale machine learning (ML) programs, such as big topic models or deep neural nets, one usually assumes such tasks can only be attempted with industrial-sized clusters with thousands of nodes, which are out of reach for most practitioners or academic researchers. We consider this challenge in the co…

A computational theory reduces agent evaluation errors and speeds up processes.

Develops a method to evaluate OPE robustness to hyperparameters and policies.

This paper evaluates and validates cluster results using external and internal evaluation methods.

Optimizes crowdsourced preference-based subjective evaluation with online learning.

We conduct a large-scale, systematic study to evaluate the existing evaluation methods for natural language generation in the context of generating online product reviews. We compare human-based evaluators with a variety of automated evaluation procedures, including discriminative evaluators that measure how well machi…

Our research extends the Bilingual Evaluation Understudy (BLEU) evaluation technique for statistical machine translation to make it more adjustable and robust. We intend to adapt it to resemble human evaluation more. We perform experiments to evaluate the performance of our technique against the primary existing evalua…

Study finds AUC is most consistent across different prevalence in binary classification.

Cramming method evaluates learned policies from contextual bandits efficiently.

Cer-Eval saves LLM evaluation costs while maintaining accuracy.

The study addresses biases in evaluating molecular optimization methods and proposes methods to reduce these biases.

Paper addresses off-policy evaluation and learning with covariate shift.

AXE evaluates explanations to avoid misleading Rashomon set model selection.

We consider evaluation methods for payoffs with an inherent financial risk as encountered for instance for portfolios held by pension funds and insurance companies. Pricing such payoffs in a way consistent to market prices typically involves combining actuarial techniques with methods from mathematical finance. We prop…

New method for evaluating LLMs reduces bias in open-ended evaluations.

Progress in machine learning is measured by careful evaluation on problems of outstanding common interest. However, the proliferation of benchmark suites and environments, adversarial attacks, and other complications has diluted the basic evaluation model by overwhelming researchers with choices. Deliberate or accident…

New method improves consistency of reinforcement learning performance evaluations.

This paper considers the challenge of evaluating a set of classifiers, as done in shared task evaluations like the KDD Cup or NIST TREC, without expert labels. While expert labels provide the traditional cornerstone for evaluating statistical learners, limited or expensive access to experts represents a practical bottl…

Bayesian approach quantifies uncertainty in LLM evaluations.

Combines curvature descriptors with TDA for graph model evaluation.

We present and prove properties of a new offline policy evaluator for an exploration learning setting which is superior to previous evaluators. In particular, it simultaneously and correctly incorporates techniques from importance weighting, doubly robust evaluation, and nonstationary policy evaluation approaches. In a…

Paper tackles efficient evaluation of natural stochastic policies in offline RL.

MLModelScope streamlines ML/DL model evaluation and benchmarking.

Study evaluates saliency maps on artificial data with different backgrounds.

Tiny benchmarks reduce LLM evaluation costs by using fewer examples.

A new method for faster multi-objective optimization by evaluating objectives separately.

Language model benchmarks often misrepresent true understanding, revealing vulnerabilities in evaluation methods.