Research
On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

169,341 papers · 148 categories

Trend · papers per month

2805598391,118 · Jun 202019922001200920182026
48 results for absent data

Random forests handle categorical predictors natively but overlook 'absent levels' can bias models.

problem Bias in decision tree models due to 'absent levels' problem.
method Examined with Leo Breiman and Adele Cutler's random forests FORTRAN code and the randomForest R package.
result Simple heuristics can help mitigate the effects of the absent levels problem.

SBIC learns patterns in imbalanced datasets using empirical similarity and synthetic data.

problem Classification failure in imbalanced datasets.
method SBIC uses an empirical similarity function and absent data to optimize weights and find minority class data points.
result SBIC outperforms other classification techniques for imbalanced datasets.

Researchers show how to secretly train models with hidden data, detect usage with high confidence.

problem Protecting training data from traceability in large language models.
method Gradient-based optimization to learn secret sequences absent from training data.
result Secret sequences can be learned by models without performance degradation, detectable with high confidence.

High-dimensional spectroscopy data makes ML models achieve near-perfect accuracy, even when chemical distinctions are absent.

problem Why machine learning models achieve near-perfect accuracy in spectroscopic classification tasks without chemically meaningful features.
method Theoretical analysis grounded in the Feldman-Hajek theorem and concentration of measure, combined with specific experiments on synthetic and real fluorescence spectra.
result Infinitesimal distributional differences in high-dimensional spaces can lead to perfect separability, making models achieve near-perfect accuracy in spectroscopy.

In arXiv:0905.2622v1 and arXiv:0910.4785v1, Bray and Khuri outlined an approach to prove the Penrose inequality for general initial data sets of the Einstein equations. In this paper we extend this approach so that it may be applied to a charged version of the Penrose inequality. Moreover, assuming that the initial dat…

2012-07-23abs ↗pdf ↗

Study bounds noise level in linear regression with dependent data.

problem Analyzing noise level in linear regression with dependent data.
method Derive upper bounds for random design linear regression with ββ-mixing data, without realizability assumptions.
result Correctly recovers the noise level of the problem, exhibiting graceful degradation with misspecification.

We generalize the Arbitrage Pricing Theory (APT) to include the contribution of virtual arbitrage opportunities. We model the arbitrage return by a stochastic process. The latter is incorporated in the APT framework to calculate the correction to the APT due to the virtual arbitrage opportunities. The resulting relatio…

1999-02-03abs ↗pdf ↗

Detect hidden confounding in observational data using multiple environments.

problem Detect hidden confounding in observational data.
method Theoretical framework and simulation studies to test for hidden confounding.
result The proposed procedure correctly predicts hidden confounding, especially when bias is large.

Paper improves deep learning models using contrastive predictive coding for semi-supervised learning.

problem Limited labeled data in semi-supervised learning.
method Contrastive predictive coding technique to improve deep learning models with unlabeled data.
result Proposed cpc-SSL and ccpc-SSL models effectively use unlabeled data, scaling well to large datasets.

We give geometric explanations and proofs of various mirror symmetry conjectures for TnT^{n}-invariant Calabi-Yau manifolds when instanton corrections are absent. This uses fiberwise Fourier transformation together with base Legendre transformation. We discuss mirror transformations of (i) moduli spaces of complex stru…

2000-09-27abs ↗pdf ↗

New method for estimating class proportions in open-set label shift data.

problem Estimating class proportions and distributions when test data includes novel classes.
method Semiparametric density ratio model framework with maximum empirical likelihood estimators and confidence intervals.
result Improved estimation accuracy and classification performance compared to existing methods.

Developed a diffusion model on spherical data, addressing geometric and stochastic challenges.

problem Diffusion models on spherical data face unique geometric and stochastic issues.
method Extended spectral diffusion to spherical harmonics, introducing modified stochastic differential equations.
result Introduced a geometry-dependent inductive bias in spectral diffusion models.

A new model family of zero-inflated Gaussian processes improves prediction and interpretability of rare event data.

problem Poor performance of conventional machine learning on zero-inflated datasets.
method Sparse kernels and latent probit Gaussian processes to zero out kernel rows and columns.
result Improves prediction of zero-inflated data and interpretability of latent mixing models.

BCF models estimate causal effects on multiple outcomes in TIMSS data.

problem Estimating causal effects on multiple outcomes in educational data.
method Bayesian Additive Regression Trees (BART) for multivariate causal inference.
result Positive and negative effects of home study conditions and school absence on student achievement.

Study shows sample complexity for logistic regression with normal covariates.

problem Estimating parameters of logistic regression with normal design.
method Analyzes sample complexity in terms of dimension and inverse temperature.
result Shows two change-points in sample complexity curve based on inverse temperature.

FedForest adapts RF for federated learning, improving performance and efficiency.

problem Adapting RF for federated learning with heterogeneous data.
method FedForest uses a novel splitting procedure to aggregate client statistics, allowing non-parametric personalization.
result FedForest's federated RF achieves performance close to centralized models while being communication-efficient.

New approach improves domain adaptation by relaxing distribution alignment constraints.

problem Improving domain adaptation when target distribution differs from source distribution.
method Asymmetrically-relaxed distribution alignment to minimize target error under varying conditions.
result Empirical and theoretical benefits demonstrated on synthetic and real datasets.

Pairwise discriminators stabilize adversarial training by ensuring the generator's alignment is preserved.

problem Stability issues in adversarial training when using sub-optimal discriminators.
method Introducing a family of objectives using pairwise discriminators, ensuring the generator's alignment is preserved.
result Only the generator needs to converge, and the alignment is preserved with any discriminator.

Model learns code representations from comments for data analysis tasks.

problem Lack of descriptive labels for analyzing large code corpora.
method Weakly supervised transformer architecture for joint code and comment representation.
result Model achieves 38% accuracy increase over expert-supplied heuristics.

Improved neural keyphrase generation by beam search with reward functions.

problem Sequence length bias and beam diversity issues in neural keyphrase generation.
method Beam search decoding strategy with word-level and ngram-level reward functions.
result Significant improvement in generating diverse and accurate keyphrases.

SynC generates synthetic population from aggregated data using Gaussian copula.

problem Generating individual-level data from aggregated datasets is challenging.
method SynC removes outliers, fits with Gaussian copula, merges datasets, and scales them.
result SynC efficiently combines multiple datasets into synthetic individual-level data.

Paper discovers sub-interval relationships in time series data.

problem Finding complex patterns of relationships between time series data.
method Proposes a novel approach to find most interesting sub-interval relationships (SIR) in a pair of time series.
result Discovered statistically significant sub-interval relationships with physical interpretation.

Combining self-training and contrastive learning improves performance under distribution shift.

problem Improving performance under distribution shift using unlabeled data.
method Combining self-training and contrastive learning techniques.
result Combined method achieves 3-8% higher accuracy than either approach independently.

New method improves neural network robustness without sacrificing generalization.

problem Robustness and generalization are often at odds in neural networks.
method Distributionally robust loss function bridging robustness and generalization.
result Certified robustness against data evasion and poisoning attacks with guaranteed generalization.

We discuss the local and global problems for the equivalence of geometric structures of an arbitrary order and, in later sections, attention is given to what really matters, namely the equivalence with respect to transformations belonging to a given pseudo-group of transformations. We first give attention to general pr…

2014-12-29abs ↗pdf ↗

Analysis of ridge regression under concept shift reveals nontrivial effects on generalization performance.

problem Understanding and mitigating the impact of distribution shift in machine learning models.
method Derivation of exact prediction risk expression in the thermodynamic limit for ridge regression under concept shift.
result Reveals a phase transition and nonmonotonic data dependence of test performance under concept shift.

Retail investors set interest rates for P2P loans based on borrower characteristics.

problem Understanding how individual investors price credit risk in online consumer loan auctions.
method Reverse auction framework, analyzing interest rate variance and borrower characteristics.
result Retail investors exhibit strong predictability in pricing, with gender and marital status influencing interest rates.

Global well-posedness and asymptotic convergence for vacuum Einstein's equations proved.

problem Proving global well-posedness and asymptotic convergence for vacuum Einstein's equations.
method Integrable damping mechanism induced by cosmological constant.
result Future-global solutions converge smoothly to a limiting metric of constant negative scalar curvature.

Methodology to measure lag relevance in time series models.

problem Measuring lag relevance in machine learning models for univariate time series.
method Ghost variables, Shapley values, additive importance measures, auto-relevance and partial auto-relevance functions, one-step forecast.
result Calculated relevance measures successfully demonstrate expected lag structure in almost all cases.

Many applications require the ability to judge uncertainty of time-series forecasts. Uncertainty is often specified as point-wise error bars around a mean or median forecast. Due to temporal dependencies, such a method obscures some information. We would ideally have a way to query the posterior probability of the enti…

2012-11-13abs ↗pdf ↗

Link prediction is one of the fundamental problems in network analysis. In many applications, notably in genetics, a partially observed network may not contain any negative examples of absent edges, which creates a difficulty for many existing supervised learning approaches. We develop a new method which treats the obs…

2013-01-29abs ↗pdf ↗

Addressing the ongoing examination of high-frequency trading practices in financial markets, we report the results of an extensive empirical study estimating the maximum possible profitability of the most aggressive such practices, and arrive at figures that are surprisingly modest. By "aggressive" we mean any trading …

2010-07-15abs ↗pdf ↗