Research
On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,742 papers · 148 categories

Trend · papers per month

12.1%24.1%36.2%48.2% · Jun 202019922001200920172026
48 results for Data Practices

Classifies privacy policy segments for better user understanding.

problem Difficulty in understanding privacy policies due to legal jargon.
method Uses machine learning and deep learning techniques to classify privacy policy segments.
result Identifies data practices in privacy policies for better user comprehension.

Study highlights how lazy data practices in fair ML research can unfairly impact minority groups.

problem Lazy data practices in fair ML research can unfairly impact minority groups.
method Systematic study of 280 experiments across 142 publications on 142 datasets.
result Unreflective data practices lead to biased findings and unfair treatment of minorities.

Semi-supervised wrapper methods are concerned with building effective supervised classifiers from partially labeled data. Though previous works have succeeded in some fields, it is still difficult to apply semi-supervised wrapper methods to practice because the assumptions those methods rely on tend to be unrealistic i…

2016-11-12abs ↗pdf ↗

Aggregation defenses improve deep learning models' robustness against data poisoning attacks.

problem Data poisoning attacks manipulate deep learning models with malicious training samples.
method Deep Partition Aggregation, efficiency improvements, data-to-complexity ratio, poisoning overfitting phenomenon.
result Aggregation defenses boost poisoning robustness through the poisoning overfitting phenomenon.

Addresses theoretical and practical aspects of Gaussian differential privacy.

problem Theoretical and practical challenges in privacy-preserving data analysis.
method Discussion of f-differential privacy and Gaussian differential privacy.
result Gaussian differential privacy can enhance privacy in various applications.

Machine learning suffers from poor design, data, and evaluation practices.

problem Ad hoc design, poor data hygiene, and lack of statistical rigor in model evaluation.
method Examines the entire machine learning process from design to evaluation, highlighting common pitfalls and providing recommendations.
result Common pitfalls in machine learning research and development are identified and actionable recommendations are provided.

Differentially private learning on real-world data poses challenges for standard machine learning practice: privacy guarantees are difficult to interpret, hyperparameter tuning on private data reduces the privacy budget, and ad-hoc privacy attacks are often required to test model privacy. We introduce three tools to ma…

2018-12-07abs ↗pdf ↗

New algorithm ensures global convergence in deep neural networks beyond NTK regime.

problem Existing global convergence guarantees do not apply to practical deep networks.
method Proposes an algorithm with global convergence guarantees under the expressivity condition.
result Algorithm ensures global convergence in practical settings beyond NTK regime.

The study compares machine learning models for depression detection and highlights the importance of feature selection.

problem The choice of features in machine learning models for depression detection is crucial.
method Comparison of seven machine learning models on depression detection tasks.
result Optimal feature selection is essential for accurate and clinically acceptable classification solutions.

Improves PU learning for imbalanced data with practical AUL estimation and new training method.

problem Training binary classifiers on datasets with only positive and unlabeled samples.
method Asymptotic unbiased AUL estimation and ProbTagging for imbalanced data.
result ProbTagging increases AUC by up to 10% on industrial and artificial data sets.

This paper explores good practices for AI explainability in finance.

problem Complex financial models lack transparency and interpretability.
method Exploring good practices for deploying explainability in AI-based financial systems.
result Developing effective XAI tools for the financial industry.

This work analyzes Fréchet regression using comparison geometry, providing theoretical and practical insights.

problem Analyzing data on complex structures like manifolds and graphs.
method Theoretical analysis through comparison geometry, focusing on existence, uniqueness, and stability of the Fréchet mean.
result Key results on the existence, uniqueness, and stability of the Fréchet mean, along with statistical guarantees for nonparametric regression.

This chapter focuses on developing datasets for machine learning, addressing data preparation challenges.

problem Challenges in collecting and transforming data for practical machine learning applications.
method Detailed process of dataset development including data collection, transformation, quality evaluation, and distribution.
result Provides insights and practical tools for effective dataset development and management.

Neural networks fit fewer samples than their parameters suggest in practice.

problem Understanding the practical limitations of neural network flexibility.
method Examination of neural network optimization, parameter efficiency, and loss surfaces.
result Neural networks can only fit training sets with significantly fewer samples than their parameters suggest.

Bayesian methods promise to fix many shortcomings of deep learning, but they are impractical and rarely match the performance of standard methods, let alone improve them. In this paper, we demonstrate practical training of deep networks with natural-gradient variational inference. By applying techniques such as batch n…

2019-06-06abs ↗pdf ↗

This guide clarifies techniques for assessing and comparing model calibration and performance.

problem Assessing and comparing the calibration and performance of predictive models in insurance and actuarial practice.
method Clarifies statistical techniques for assessing model calibration and comparing models, emphasizing the importance of specifying the prediction target functional and choosing the appropriate scoring function.
result Provides guidance for the practical choice of scoring functions and illustrates results with real data case studies.

New framework provides privacy guarantees for practical federated learning.

problem Inadequate privacy guarantees for federated learning due to restrictive assumptions.
method Fed-α\alpha-NormEC, integrating multiple local updates, partial client participation, and standard assumptions.
result Provably convergent and differentially private federated learning framework.

Causal discovery algorithms infer causal relations from data based on several assumptions, including notably the absence of measurement error. However, this assumption is most likely violated in practical applications, which may result in erroneous, irreproducible results. In this work we show how to obtain an upper bo…

2018-10-18abs ↗pdf ↗

Short-and-sparse deconvolution (SaSD) is the problem of extracting localized, recurring motifs in signals with spatial or temporal structure. Variants of this problem arise in applications such as image deblurring, microscopy, neural spike sorting, and more. The problem is challenging in both theory and practice, as na…

2019-08-28abs ↗pdf ↗

Paper shows local SGD outperforms mini-batch SGD under certain conditions.

problem Proving local SGD's superiority in distributed learning with heterogeneous data.
method New lower and upper bounds for local SGD under first-order heterogeneity assumptions.
result Local SGD is min-max optimal under certain conditions, resolving understanding of distributed optimization.

This paper reviews PU learning evaluation methods and provides practical recommendations.

problem Evaluating PU learning methods when only positive and unlabelled data are available.
method Critical review of 51 articles proposing PU classifiers and alternative predictive accuracy measures.
result Practical recommendations for improving PU learning evaluation.

A practical one-shot federated learning algorithm for cross-silo setting.

problem Limited applicability of existing one-shot federated learning algorithms due to specific model support and lack of privacy guarantees.
method FedKT, a one-shot federated learning algorithm that supports any classification models and provides differential privacy guarantees.
result FedKT significantly outperforms other state-of-the-art federated learning algorithms with a single communication round.

Histogram binning method proven with guarantees without splitting data.

problem Proving theoretical guarantees for histogram binning without sample splitting.
method Using Markov property of order statistics to prove calibration guarantees for original method.
result Proves histogram binning has strong calibration guarantees without sample splitting.

SAGDA generates synthetic agricultural datasets to improve ML in African farming.

problem Data scarcity in African agriculture limits machine learning innovations.
method SAGDA is an open-source Python toolkit that generates, augments, and validates synthetic agricultural datasets.
result SAGDA enhances ML applications in agriculture, such as yield prediction and fertilizer recommendation.

Confounding biases recommender systems, even when data seems fully observed.

problem Unmeasured features influencing both treatment and outcome in recommender systems.
method Illustrations and simulation studies showing how common practices introduce confounding.
result Standard recommender system practices can introduce confounding, reducing performance.

Topic models provide a useful method for dimensionality reduction and exploratory data analysis in large text corpora. Most approaches to topic model inference have been based on a maximum likelihood objective. Efficient algorithms exist that approximate this objective, but they have no provable guarantees. Recently, a…

2012-12-19abs ↗pdf ↗

Faced with massive data, is it possible to trade off (statistical) risk, and (computational) space and time? This challenge lies at the heart of large-scale machine learning. Using k-means clustering as a prototypical unsupervised learning problem, we show how we can strategically summarize the data (control space) in …

2016-05-02abs ↗pdf ↗

Domain adaptation provides a powerful set of model training techniques given domain-specific training data and supplemental data with unknown relevance. The techniques are useful when users need to develop models with data from varying sources, of varying quality, or from different time ranges. We build CrossTrainer, a…

2019-05-07abs ↗pdf ↗

Develops methods to improve demand counterfactuals from imperfect proxies.

problem Imperfect proxies in demand models lead to biased counterfactuals and invalid inference.
method Practical toolkit for market-level and individual data, requiring minimal computation.
result Improves substitution prediction and counterfactual performance.

This paper introduces TDA and TSI for better business analytics.

problem Nonlinear, multi-scale business datasets under-represented by traditional tools.
method Topological Data Analysis (TDA) and Topological Stability Index (TSI).
result TSI reveals structural variability in business data.

Gradient Boosting Decision Trees (GBDTs) have become very successful in recent years, with many awards in machine learning and data mining competitions. There have been several recent studies on how to train GBDTs in the federated learning setting. In this paper, we focus on horizontal federated learning, where data sa…

2019-11-11abs ↗pdf ↗

Tree-based variational inference improves PLN model for hierarchical count data.

problem Limited applicability of PLN model in ecosystems due to lack of hierarchical tree structures.
method Introduced PLN-Tree model integrating structured variational inference techniques.
result Enhanced generative improvements and practical interpretability in microbiome modeling.

This paper provides a practical method to extract caplet volatilities from quoted data.

problem Extracting caplet volatilities from quoted data is complex and not straightforward.
method The paper presents a constructive algorithm based on criteria and robust outlier detection. It includes direct interpolation, bootstrap methods, and global search methods.
result The paper introduces methods to extract caplet volatilities that are arbitrage-free and consistent with quoted data.

Study shows data attribution methods are sensitive to hyperparameters, making tuning costly.

problem Hyperparameter sensitivity in data attribution methods makes tuning impractical.
method Theoretical analysis and lightweight procedure for selecting regularization value without retraining.
result Proposes a lightweight procedure for selecting regularization value without model retraining.