Paper explores methods to improve web search ranking.
problem Improving web search ranking accuracy with varied test data.
method Model Interpolation and Boosting Algorithm.
result Model Interpolation achieves best results on open test sets.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,932 papers · 148 categories
Trend · papers per month
48 results for “open test sets”
Research improves open-set learning by leveraging unlabelled data.
problem Learning between observed and unobserved novel categories.
method Unified policy of positive and unlabelled learning, semi-supervised learning, and open-set recognition.
result Achieves state-of-the-art results in open-set learning.
Often, when dealing with real-world recognition problems, we do not need, and often cannot have, knowledge of the entire set of possible classes that might appear during operational testing. In such cases, we need to think of robust classification methods able to deal with the "unknown" and properly reject samples belo…
This work bridges continual learning, active learning, and open set recognition in deep neural networks.
problem Protecting previously acquired representations from catastrophic forgetting in deep neural networks.
method Surveying the literature and proposing a consolidated view to integrate open set recognition and active learning principles.
result Joint improvement in alleviating catastrophic forgetting, querying data, selecting task orders, and robust open world application.
Paper develops a deep neural network for open set incremental learning of new authors.
problem Classifying unseen examples from previously unseen classes.
method Deep neural network clustering and retraining for new classes.
result Incremental learning model that continuously learns new classes.
The Sample Complexity of Distributed Simple Binary Hypothesis Testing under Information Constraintscs.IT
Paper resolves open problems on sample complexity in binary hypothesis testing.
problem Open problems in distributed simple binary hypothesis testing under information constraints.
method One-shot lower bound on Bayes error, streamlined sample complexity formula, reverse data-processing inequality.
result Optimally tight sample complexity bounds for communication-constrained simple binary hypothesis testing.
This paper presents a study on discriminative artificial neural network classifiers in the context of open-set speaker identification. Both 2-class and multi-class architectures are tested against the conventional Gaussian mixture model based classifier on enrolled speaker sets of different sizes. The performance evalu…
Reduces test set maintenance effort by 80-100%.
problem Lack of proper and up-to-date test sets in real-world scenarios.
method Simple technique to reduce labeling effort.
result Significant reduction in test set maintenance effort (80-100%).
Open category detection is the problem of detecting "alien" test instances that belong to categories or classes that were not present in the training data. In many applications, reliably detecting such aliens is central to ensuring the safety and accuracy of test set predictions. Unfortunately, there are no algorithms …
Camera model identification refers to the problem of linking a picture to the camera model used to shoot it. As this might be an enabling factor in different forensic applications to single out possible suspects (e.g., detecting the author of child abuse or terrorist propaganda material), many accurate camera model att…
Random forest can be adapted for open-set recognition with improved performance.
problem Handling unknown classes in real-world classification tasks.
method Incorporating distance metric learning and distance-based open-set recognition into random forest.
result The proposed method outperforms state-of-the-art open-set recognition methods.
Paper develops a new method for open-set and imbalanced classification with valid prediction sets.
problem Tackles open-set and imbalanced classification with new prediction methods.
method Develops a new family of conformal p-values and a selective sample splitting algorithm.
result Valid prediction sets with valid coverage in open-set scenarios and informative predictions under extreme class imbalance.
This paper evaluates knowledge graph completion models under the open-world assumption, revealing unexpected behavior of metrics.
problem Evaluation of knowledge graph completion models often assumes a closed-world assumption, which can lead to misleading results.
method The paper studies KGC evaluation under the open-world assumption, analyzing the behavior of metrics like MRR and Hits@K.
result Metrics like MRR and Hits@K can show significant degradation under the open-world assumption, leading to incorrect model comparisons.
New method for estimating class proportions in open-set label shift data.
problem Estimating class proportions and distributions when test data includes novel classes.
method Semiparametric density ratio model framework with maximum empirical likelihood estimators and confidence intervals.
result Improved estimation accuracy and classification performance compared to existing methods.
S2OSC improves OSC by filtering and re-training models with out-of-class instances.
problem Difficulty in classifying out-of-class instances during inference.
method S2OSC combines semi-supervised learning, out-of-class instance filtering, and model re-training.
result S2OSC achieves state-of-the-art performance in various OSC tasks.
CGDL improves open set recognition by learning conditional Gaussian distributions.
problem Handling unknown samples in real-world recognition tasks.
method Conditional Gaussian Distribution Learning (CGDL) with probabilistic ladder architecture.
result CGDL significantly outperforms baseline methods on standard image datasets.
Using sequence to sequence algorithms for query expansion has not been explored yet in Information Retrieval literature nor in Question-Answering's. We tried to fill this gap in the literature with a custom Query Expansion engine trained and tested on open datasets. Starting from open datasets, we built a Query Expansi…
Viral Load Inference in Non-Adaptive Pooled Testingcond-mat.stat-mech
New method infers viral load from pooled tests.
problem Inefficient viral load inference in pooled testing.
method Message passing algorithm with PCR noise function.
result Accurate viral load inference possible.
Proposes a new framework for open set recognition using conditional probabilistic generative models.
problem Unknown samples can mislead traditional deep neural networks during testing.
method Conditional Probabilistic Generative Models (CPGM) that combine generative models with discriminative information.
result Significantly outperforms baselines on multiple benchmark datasets.
Proves uniform K-stability is open in Kähler cone.
problem Stability of Kähler metrics in complex geometry.
method Introduced new norm on test configurations and estimates for non-archimedean energy functionals.
result Uniform K-stability is an open condition in the Kähler cone.
In real-world recognition/classification tasks, limited by various objective factors, it is usually difficult to collect training samples to exhaust all classes when training a recognizer or classifier. A more realistic scenario is open set recognition (OSR), where incomplete knowledge of the world exists at training t…
The paper proves the openness of K-semistability for Fano varieties.
problem Stability of K-semistability in families of log Fano pairs.
method By showing the stability threshold is a constructible function and proving special test configurations arise from log canonical places.
result The stability threshold is a constructible function on fibers, and any minimizer of the stability threshold exists.
Throughout the last decade, random forests have established themselves as among the most accurate and popular supervised learning methods. While their black-box nature has made their mathematical analysis difficult, recent work has established important statistical properties like consistency and asymptotic normality b…
COME replaces entropy minimization to prevent model collapse.
problem Overconfidence in entropy minimization leads to model collapse.
method COME explicitly models uncertainty with a Dirichlet prior distribution.
result COME achieves state-of-the-art performance on various TTA settings.
End-to-end speech recognition system trained on GPUs and CPUs.
problem Building state-of-the-art speech recognition systems.
method Utilizes CPUs and GPUs for training, data augmentation, and neural network updates. Uses vocal tract length perturbation and acoustic simulator for data augmentation. Employed Horovod allreduce for training.
result Achieved 7.92% WER on proprietary English Bixby open domain test set using a Bidirectional Full Attention (BFA) model.
GRETEL unifies GCE evaluation across various settings.
problem Lack of standardized evaluation for Graph Counterfactual Explanations.
method Unified framework for testing GCE methods in diverse settings.
result GRETEL promotes reproducible evaluations of GCE techniques.
AutoML serves as the bridge between varying levels of expertise when designing machine learning systems and expedites the data science process. A wide range of techniques is taken to address this, however there does not exist an objective comparison of these techniques. We present a benchmark of current open source Aut…
The Machine Recognition of Crystallization Outcomes (MARCO) initiative has assembled roughly half a million annotated images of macromolecular crystallization experiments from various sources and setups. Here, state-of-the-art machine learning algorithms are trained and tested on different parts of this data set. We fi…
This article contains a detailed study, in the toric case, of the test configuration geodesic rays defined by Phong-Sturm. We show that the `Bergman approximations' of Phong-Sturm converge in C^1 to the geodesic ray and that the geodesic ray itself is C^{1,1} and no better. The \kahler metrics associated to the geodesi…
Develops a statistical test for IV, improving feature selection reliability.
problem Lack of statistical justification in conventional IV-based feature selection.
method Establishes connection with Jeffreys divergence and proposes a nonparametric hypothesis test.
result The J-Divergence test provides rigorous guarantees and is more reliable than traditional IV thresholds.
In open set recognition (OSR), almost all existing methods are designed specially for recognizing individual instances, even these instances are collectively coming in batch. Recognizers in decision either reject or categorize them to some known class using empirically-set threshold. Thus the decision threshold plays a…
FinanceBench benchmarks LLMs on financial QA, revealing limitations.
problem Evaluating LLMs' performance on financial question answering.
method Developed a comprehensive test suite (FinanceBench) with 10,231 questions, tested 16 models, and manually reviewed answers.
result Existing LLMs have significant limitations for financial QA, especially GPT-4-Turbo.
Develops a data-driven fault diagnosis framework for time-series data.
problem Fault diagnosis of dynamic systems using imbalanced and unknown fault classes.
method Kullback-Leibler divergence, data-driven fault classification, open-set classification.
result Framework handles imbalanced datasets, class overlapping, and unknown faults.
Quick and accurate medical diagnosis is crucial for the successful treatment of a disease. Using machine learning algorithms, we have built two models to predict a hematologic disease, based on laboratory blood test results. In one predictive model, we used all available blood test parameters and in the other a reduced…
New method TLC improves transductive learning bounds.
problem Sharp generalization bounds for transductive learning.
method Transductive Local Complexity (TLC) framework.
result Nearly sharp bounds consistent with inductive results.
Bayesian networks are typically faithful, with implications for causal inference.
problem Determining the typicality of faithfulness in Bayesian networks.
method Analysis of Bayesian networks over a given DAG, parametrized by conditional exponential families, and nonparametric conditional densities.
result The faithful Bayesian networks are dense and open with respect to the total variation metric, extending existing results for specific classes of Bayesian networks.
Over-parameterized CNNs show U-shaped test risk with depth increase.
problem Understanding the impact of depth on test risk in over-parameterized CNNs.
method Empirical image classification experiments and linear regression framework.
result Test risk is U-shaped with increasing depth in over-parameterized CNNs.
For precision medicine and personalized treatment, we need to identify predictive markers of disease. We focus on Alzheimer's disease (AD), where magnetic resonance imaging scans provide information about the disease status. By combining imaging with genome sequencing, we aim at identifying rare genetic markers associa…
New robustness test for kernel goodness-of-fit tests.
problem Lack of robustness in existing kernel goodness-of-fit tests.
method Proposes a new robust kernel goodness-of-fit test using kernel Stein discrepancy (KSD) balls.
result First robust kernel goodness-of-fit test addressing both qualitative and quantitative robustness.
Unified framework for optimal kernel tests across MMD, HSIC, and KSD.
problem Optimal testing in kernel-based hypothesis testing frameworks.
method Unified derivation of minimax rates, adaptive kernel selection methods.
result Unified power results across MMD, HSIC, and KSD.
This paper improves confidence measurement in deep metric learning models.
problem Measuring confidence in deep metric learning models is challenging.
method Approximates class distributions using Gaussian kernel smoothing and calibrates the confidence metric.
result Improves generalization and robustness of deep metric learning models.
The package High-dimensional Metrics (\Rpackage{hdm}) is an evolving collection of statistical methods for estimation and quantification of uncertainty in high-dimensional approximately sparse models. It focuses on providing confidence intervals and significance testing for (possibly many) low-dimensional subcomponents…
Hypothesis testing for graphs has been an important tool in applied research fields for more than two decades, and still remains a challenging problem as one often needs to draw inference from few replicates of large graphs. Recent studies in statistics and learning theory have provided some theoretical insights about …
Study shows different trajectory prediction models generalize better under OoD conditions.
problem Comparing trajectory prediction models' robustness across different datasets.
method Training models on Argoverse 2 and testing on Waymo Open Motion, and vice versa, with various augmentation strategies.
result Smallest model with highest inductive bias performs best in OoD generalization.
New handwritten digits dataset for Kannada script.
problem Lack of datasets for Kannada numeral digits.
method Developed Kannada-MNIST and Dig-MNIST datasets.
result Initial CNN accuracy is lower than MNIST, indicating a challenge in generalization.
Open-world evasion attacks using OOD adversarial examples outperform in-distribution attacks.
problem Evasion attacks on deep learning systems in open-world environments.
method Using 11 neural network models trained on 3 datasets, the study demonstrates the effectiveness of OOD adversarial examples against state-of-the-art detectors and defenses.
result OOD adversarial examples can achieve up to 4x higher success rates than in-distribution adversarial examples against known defenses.
Study improves statistical power for detecting algorithmic bias in educational data.
problem Challenges in measuring algorithmic bias using ABROCA due to skewed distribution.
method Investigates ABROCA's distributional properties and proposes nonparametric randomization tests.
result ABROCA-based bias assessments are underpowered in typical EDM sample sizes.
When Alpha Disappears: A One-Switch Benchmark for Decision-Time Leakage in Financial Backtestsq-fin.RM
Benchmark detects decision-time leakage in financial backtests.
problem Detecting decision-time leakage in financial machine-learning backtests.
method Toggles one evaluation convention at a time around a clean -open reference, holding other factors fixed.
result Inflation is highly selective, affecting specific features and execution methods.