Survival analysis models research reproducibility, offering new insights.
problem Reproducibility crisis in machine learning research.
method Survival analysis to model reproducibility as a continuous process.
result Survival analysis provides deeper insights into research reproducibility.
Proposes a method to improve neural architectures reproducibly.
problem Lack of reproducibility in Neural Architecture Transformer (NAT).
method Differentiable Neural Architecture Transformation (DNAT).
result DNAT outperforms NAT and is applicable to various models and datasets.
The study uses reproducing kernels to model bond discount curves.
problem Estimating bond discount curves under no-arbitrage conditions.
method Introduced reproducing kernels as a regression basis for estimating bond discount curves.
result Reproducing kernels provide a tractable solution for calibrating models to market data.
Study on reproducibility in optimization with bounds on limits.
problem Limits of reproducibility in noisy or error-prone optimization procedures.
method Defined a quantitative measure of reproducibility and analyzed convex optimization settings.
result Revealed a fundamental trade-off between computation and reproducibility.
Reproducibility of modeling is a problem that exists for any machine learning practitioner, whether in industry or academia. The consequences of an irreproducible model can include significant financial costs, lost time, and even loss of personal reputation (if results prove unable to be replicated). This paper will fi…
Clarifies the scope of 'reproducibility' in AI and ML.
problem Unclear meaning of 'reproducibility' in AI and ML.
method Analyzes the community's works on reproducibility.
result Identifies eight general topic areas of reproducibility.
LLM-based trading systems vary in execution realism and reproducibility.
problem LLM-based trading systems vary in execution realism and reproducibility.
method Reproducibility audit of 30 trade-relevant primary studies.
result LLM-based trading systems vary in execution realism and reproducibility.
This chapter introduces reproducibility in machine learning for medical imaging.
problem Lack of reproducibility in machine learning for medical imaging.
method Distinguishes and defines types of reproducibility, outlines requirements, and discusses utility.
result Discussion on benefits and a plea for a non-dogmatic approach to reproducibility.
New algorithms ensure reproducibility and optimal convergence in convex optimization.
problem Trade-off between reproducibility and convergence rate in convex optimization.
method Regularization-based algorithms for smooth convex minimization and minimax optimization.
result Achieves optimal reproducibility and near-optimal gradient complexity for various oracle settings.
Evaluating the computational reproducibility of data analysis pipelines has become a critical issue. It is, however, a cumbersome process for analyses that involve data from large populations of subjects, due to their computational and storage requirements. We present a method to predict the computational reproducibili…
CK simplifies ML model deployment and reproducibility with open APIs and DevOps.
problem Making ML models reproducible and deployable across different environments.
method Decompose complex systems into reusable sub-components with unified APIs and DevOps principles.
result Automatically co-design and optimize ML models for speed, accuracy, energy, and size.
BayesBag improves reproducibility of Bayesian inference under model misspecification.
problem Bayesian posteriors can be unreliable and inconsistent under model misspecification.
method Apply bagging to the Bayesian posterior to improve reproducibility.
result Bagged posteriors typically satisfy reproducibility criteria under misspecification.
The paper aims to improve reproducibility of machine learning experiments.
problem Lack of reproducibility in machine learning experiments.
method Investigates factors affecting reproducibility and proposes FAIR data practices.
result Preliminary results show the role of ProvBook in capturing and comparing provenance.
The paper advocates for interpretable, accountable, reproducible machine learning in medicine.
problem Black box models in medicine lack transparency and regulatory approval.
method Intrinsically interpretable modeling approaches and collaborative learning paradigms.
result Interpretable machine learning models can support clinical decisions and gain regulatory approval.
In this paper we solve support vector machines in reproducing kernel Banach spaces with reproducing kernels defined on nonsymmetric domains instead of the traditional methods in reproducing kernel Hilbert spaces. Using the orthogonality of semi-inner-products, we can obtain the explicit representations of the dual (nor…
NeurIPS 2019 program improves reproducibility in machine learning.
problem Ensuring machine learning research results are reproducible and reliable.
method Code submission policy, reproducibility challenge, and checklist integration.
result Improved reproducibility standards across the machine learning community.
Study assesses consistency and reproducibility of LLMs in finance and accounting tasks.
problem Consistency and reproducibility of LLM outputs in finance and accounting research.
method Extensive experimentation with 50 independent runs across 5 tasks using 3 OpenAI models.
result Task-specific patterns of consistency and reproducibility, with binary classification and sentiment analysis achieving near-perfect reproducibility.
Challenge encourages reproducible deep learning methods.
problem Inconsistent training behavior in deep learning competitions.
method Evaluate methods based on their training procedures, retrain in controlled settings.
result Guaranteed reproducibility and generalization of methods.
Earlier we proposed the stochastic point process model, which reproduces a variety of self-affine time series exhibiting power spectral density S(f) scaling as power of the frequency f and derived a stochastic differential equation with the same long range memory properties. Here we present a stochastic differential eq…
In this paper, we discuss the approaches we took and trade-offs involved in making a paper on a conceptual topic in pattern recognition research fully reproducible. We discuss our definition of reproducibility, the tools used, how the analysis was set up, show some examples of alternative analyses the code enables and …
Paper explores RKHS properties for derivative and integral operators.
problem Establishing sufficient conditions for reproducing property in RKHS.
method Establishing reproducing property for combinations of composition operators.
result Provides framework for regularized learning algorithms involving function values, gradients, or operators.
Study evaluates 21 KG embedding models, highlighting model architecture, training approach, and loss function importance.
problem Difficulty in fair and thorough comparisons of knowledge graph embedding models due to heterogeneity in implementations, training, and evaluation.
method Reimplemented and evaluated 21 models in PyKEEN, providing insights into reproducibility and best practices.
result Best practices and configurations for each model were identified, showing the importance of model architecture, training approach, and loss function.
A modular framework for knowledge distillation simplifies experiments and reproducibility.
problem Difficulty in reproducing knowledge distillation studies due to lack of generalized frameworks.
method A configuration-driven PyTorch framework for knowledge distillation studies.
result Demonstrates efficient training strategies and various knowledge distillation methods.
Paper characterizes embeddability of function spaces into Lp-type RKBS via metric entropy.
problem Characterizing embeddability of function spaces into Lp-type RKBS. method Establishes a connection between metric entropy growth and embeddability.
result A bound on metric entropy growth allows embedding into Lp-type RKBS. This study assesses the reproducibility of 1H-MRS scans across different vendors and sessions.
problem Lack of harmonization in magnetic resonance spectroscopy protocols among vendors.
method Analysis of CV and ICC for within- and between-sessions, and correlation coefficients for across machines.
result Metabolite concentrations are highly reproducible across different vendors and sessions.
Study compares 29 emulators across 60 test functions and 40 datasets.
problem Comparing the strengths and weaknesses of different emulators.
method Large-scale, fully reproducible comparison using R package duqling.
result Detailed empirical insights into emulator strengths and weaknesses.
New activations improve deep network reproducibility without sacrificing accuracy.
problem Deep networks' reproducibility issues, especially on distributed systems.
method Developed SmeLU activations, smoother than ReLU, to enhance reproducibility.
result SmeLU activations provide better accuracy-reproducibility tradeoffs.
Survey on reproducibility and distortion issues in text clustering and topic modeling.
problem Reproducibility and misleading cluster geometry in unsupervised learning for text categorization.
method Systematic literature review of text clustering and topic modeling from 2011-2022.
result Outliers and initialization issues are significant factors in text clustering and topic modeling.
In this paper, we reproduce the experiments of Artetxe et al. (2018b) regarding the robust self-learning method for fully unsupervised cross-lingual mappings of word embeddings. We show that the reproduction of their method is indeed feasible with some minor assumptions. We further investigate the robustness of their m…
FairGround offers a diverse dataset corpus for fair ML research.
problem Lack of diverse, well-annotated datasets in fair ML research.
method Unified framework and Python package for reproducible fair ML research.
result Advances reproducibility and generalizability of fair ML research.
The reproducibility of scientific research has become a point of critical concern. We argue that openness and transparency are critical for reproducibility, and we outline an ecosystem for open and transparent science that has emerged within the human neuroimaging community. We discuss the range of open data sharing re…
Deep neural networks define suitable reproducing kernel Banach spaces.
problem Characterizing the function spaces of deep neural networks.
method Reproducing kernel Banach spaces and variational results.
result Deep neural networks define suitable reproducing kernel Banach spaces.
Simultaneous reproduction of all financial stylized facts is so difficult that most existing stochastic process-based and agent-based models are unable to achieve the goal. In this study, by extending the decision-making structure of Minority Game, we propose a novel agent-based model called "Speculation Game," for a b…
This is a report for reproducibility challenge of NeurlIPS 2019 on the paper Competitive Gradient Descent (Schafer et al., 2019). The paper introduces a novel algorithm for the numerical computation of Nash equilibria of competitive two-player games. It avoids oscillatory and divergent behaviours seen in alternating gr…
What makes a paper independently reproducible? Debates on reproducibility center around intuition or assumptions but lack empirical results. Our field focuses on releasing code, which is important, but is not sufficient for determining reproducibility. We take the first step toward a quantifiable answer by manually att…
We study reproducing kernel Hilbert spaces (RKHS) on a Riemannian manifold. In particular, we discuss under which condition Sobolev spaces are RKHS and characterize their reproducing kernels. Further, we introduce and discuss a class of smoother RKHS that we call diffusion spaces. We illustrate the general results with…
Fine-tuned open-source LLMs match or exceed closed-source models in social science research.
problem Limited scalability and high costs of large LLMs in social science research.
method Fine-tuning open-source models for specific tasks, exploring training set size effects, proposing hybrid workflow.
result Small, fine-tuned open-source LLMs achieve equal or superior performance to commercial alternatives.
Develops vector-valued RKBS for neural networks and operators.
problem Understanding function spaces of Rd-valued neural networks and neural operators. method Defines and constructs vector-valued RKBS (vv-RKBS) without restrictive assumptions.
result Establishes Representer Theorem for neural architectures.
We propose a model of fractal point process driven by the nonlinear stochastic differential equation. The model is adjusted to the empirical data of trading activity in financial markets. This reproduces the probability distribution function and power spectral density of trading activity observed in the stock markets. …
ERICA assesses reproducibility in cluster analysis.
problem Lack of a unified framework for evaluating cluster analysis replicability.
method ERICA (iterative clustering assignments) method to quantify replicability.
result Demonstrates ERICA's ability to identify reproducible cluster structure.
This paper extends mirror descent to Banach spaces with reproducing kernels.
problem Optimizing in Banach spaces with reproducing kernels.
method Mirror descent algorithm adapted for Banach spaces with reproducing kernels.
result Mirror descent achieves linear convergence in certain conditions and standard convergence in a constrained setting.
Personal income distributions in Japan are analyzed empirically and a simple stochastic model of the income process is proposed. Based on empirical facts, we propose a minimal two-factor model. Our model of personal income consists of an asset accumulation process and a wage process. We show that these simple processes…
The paper uses Banach spaces to analyze neural networks.
problem Understanding the function spaces of neural networks.
method Theory of reproducing kernel Banach spaces.
result Representer theorem for wide class of Banach spaces.
Anti-Distillation improves reproducibility of deep networks by making ensemble predictions more diverse.
problem Deep networks are prone to high prediction differences, making them less reproducible.
method Anti-Distillation uses ensembles to force predictions to be more different and diverse.
result Anti-Distillation reduces prediction differences by making ensemble predictions more diverse.
Study improves LLMs for PPI analysis by addressing uncertainty.
problem Uncertainty in LLM predictions for PPIs.
method Fine-tuned LLaMA-3 and BioMedGPT models, LoRA ensembles, Bayesian LoRA for UQ.
result Competitive PPI identification performance across diverse disease contexts.
Machine learning algorithms designed to characterize, monitor, and intervene on human health (ML4H) are expected to perform safely and reliably when operating at scale, potentially outside strict human supervision. This requirement warrants a stricter attention to issues of reproducibility than other fields of machine …
Novel RKHS approach solves complex financial model equations.
problem Calibrating singular local stochastic volatility models.
method Reproducing Kernel Hilbert Space (RKHS) regularization.
result Regularized model is well-posed and replicates option prices.
Derives properties of heat kernel for Rumin complex on Heisenberg groups.
problem Analyzing heat kernel properties for Rumin complex.
method Derives properties of heat equation with Hodge operator on Heisenberg groups.
result Constructs Calderón reproducing formula using heat kernel for Rumin forms.