Researchers share pitfalls and improvements in implementing Hinton's capsule network.
problem Implementation pitfalls in Hinton's capsule network hindered progress in the field.
method Identifying and addressing common mistakes in capsule network implementations.
result Improved implementation of Hinton's capsule network outperforms existing open-source implementations.
Model-agnostic interpretation methods can mislead if not used carefully.
problem Misinterpretation of machine learning models due to improper use of techniques.
method General pitfalls of model-agnostic interpretation methods.
result Many pitfalls exist when using global interpretation techniques for machine learning models.
Study pitfalls of deep learning ensembles in uncertainty estimation.
problem Pitfalls in in-domain uncertainty estimation and ensembling in deep learning.
method Exploration of standards for uncertainty quantification and broad study of ensembling techniques.
result Many sophisticated ensembling techniques are equivalent to a simple ensemble of few networks.
Cross-validation pitfalls in change-point regression are addressed with new approaches.
problem Cross-validation's prediction error-based criterion may lead to under- or over-estimation of change-points.
method Proposes two approaches: absolute error loss and modified holdout sets.
result Consistent estimation of the number of change-points under certain conditions.
Machine learning suffers from poor design, data, and evaluation practices.
problem Ad hoc design, poor data hygiene, and lack of statistical rigor in model evaluation.
method Examines the entire machine learning process from design to evaluation, highlighting common pitfalls and providing recommendations.
result Common pitfalls in machine learning research and development are identified and actionable recommendations are provided.
This paper evaluates metrics for graph generative models, addressing common pitfalls.
problem Evaluating and comparing graph generative models effectively.
method Systematic evaluation of MMD, analysis of synthetic and real graphs, practical recommendations.
result MMD can be problematic; practical solutions are provided.
When faced with high frequency streams of data, clustering raises theoretical and algorithmic pitfalls. We introduce a new and adaptive online clustering algorithm relying on a quasi-Bayesian approach, with a dynamic (i.e., time-dependent) estimation of the (unknown and changing) number of clusters. We prove that our a…
When confronted with massive data streams, summarizing data with dimension reduction methods such as PCA raises theoretical and algorithmic pitfalls. Principal curves act as a nonlinear generalization of PCA and the present paper proposes a novel algorithm to automatically and sequentially learn principal curves from d…
Machine Learning (ML) and Deep Learning (DL) innovations are being introduced at such a rapid pace that model owners and evaluators are hard-pressed analyzing and studying them. This is exacerbated by the complicated procedures for evaluation. The lack of standard systems and efficient techniques for specifying and pro…
Survey of topological 4-manifold theory, highlighting foundational theorems and pitfalls.
problem Understanding the topological properties of 4-dimensional manifolds.
method Compilation and explanation of foundational theorems, with cautionary notes.
result Many intuitive results in differential topology are not true in the topological category.
In this paper, we briefly review the basic scheme of the pseudoinverse learning (PIL) algorithm and present some discussions on the PIL, as well as its variants. The PIL algorithm, first presented in 1995, is a non-gradient descent and non-iterative learning algorithm for multi-layer neural networks and has several adv…
Bayesian optimization has emerged as a strong candidate tool for global optimization of functions with expensive evaluation costs. However, due to the dynamic nature of research in Bayesian approaches, and the evolution of computing technology, using Bayesian optimization in a parallel computing environment remains a c…
The paper addresses pitfalls in calibrating option pricing models using deep learning.
problem Calibrating option pricing models to market data using deep learning.
method Identifies and resolves issues in existing approaches, improving model performance and accuracy.
result Proposes solutions that enhance the accuracy and performance of deep learning models in option pricing.
Study identifies pitfalls in assessing hierarchies for multi-class classification.
problem Lack of understanding in selecting hierarchies for multi-class classification.
method Analyzed and compared popular approaches to extracting hierarchies.
result Hierarchy quality becomes irrelevant when using powerful classifiers.
Benchmark assesses LLMs' causal inference skills, revealing significant limitations.
problem Lack of rigorous evaluation of LLMs' causal inference capabilities.
method CausalPitfalls benchmark with structured challenges and grading rubrics.
result Significant limitations in current LLMs' statistical causal inference.
Paper discusses challenges in deploying ML models for structural engineering.
problem Challenges in deploying machine learning models for structural engineering applications.
method Illustrates challenges through two examples, focusing on model overfitting, underspecification, training data representativeness, variable omission bias, and cross-validation.
result Highlights the importance of rigorous model validation techniques.
New categorization of community detection methods to avoid pitfalls.
problem Misuse of community detection methods in practice.
method Descriptive vs. inferential approaches.
result Inferential methods provide clearer insights into network formation.
Bayesian optimization improves molecule design by addressing three pitfalls.
problem Bayesian optimization pitfalls cause poor performance in molecule design.
method Identified and addressed three pitfalls: incorrect prior width, over-smoothing, and inadequate acquisition function maximization.
result Basic BO setup achieves highest performance on PMO benchmark.
Unified framework for ranking-and-selection with multiple correct answers and non-answerable estimates
problem Fixed-precision ranking-and-selection in structured settings with non-unique answers and non-answerable estimates
method Unified framework based on answer-wise acceptance sets, restricted generalized likelihood ratio stopping, and answer-pitfall decomposition
result Unified recipe performs well across a broad range of pure-exploration problems
Kernel quadrature improves CRPS estimation for probabilistic time-series forecasting.
problem Intractable integrations in CRPS evaluation metrics lead to improper rankings of forecasting models.
method Introduced kernel quadrature approach for unbiased CRPS estimation and scalable computation.
result Our approach consistently outperforms existing CRPS estimators.
This paper improves conditional sampling for VAEs by overcoming structural issues.
problem Computational intractability of conditional sampling in VAEs.
method Proposes two methods to address pitfalls in Metropolis-within-Gibbs (MWG) for VAEs.
result Improved performance on sampling tasks.
Optimizes pure exploration in linear bandits with a new algorithm.
problem Best-arm identification in linear stochastic bandits.
method Developed the first asymptotically optimal algorithm for fixed-confidence pure exploration in linear bandits.
result Avoids the pitfall of a simple but difficult instance and bypasses the need to solve an optimal design problem.
This paper evaluates test selection methods for deep neural networks, revealing their limitations.
problem Challenges in testing deep learning systems due to high labeling costs.
method Analysis and empirical testing of 11 test selection methods on five datasets.
result Test selection methods can fail under certain conditions, leading to significant drops in test relative coverage.
Deep learning applied to SAR data is explored in this paper.
problem Limited use of deep learning in SAR data processing.
method Introduction to relevant deep learning models, analysis of SAR data characteristics, review of state-of-the-art applications, and future research directions.
result Unlocking the potential of deep learning in SAR data processing.
This paper highlights the overlooked role of preprocessing hyperparameters in machine learning model performance.
problem The neglect of preprocessing hyperparameters in machine learning model evaluation.
method Empirical review and illustration of different procedures for generating and evaluating prediction models.
result Users may fail to report or account for preprocessing hyperparameter optimization, leading to exaggerated performance claims.
Deep RL fails on deceptive games, revealing algorithm weaknesses.
problem Characterizing and understanding failures of deep reinforcement learning.
method Testing A2C on four deceptive games using a game framework.
result Deep RL fails in specific ways that differ from planning-based agents.
Piecewise-linear regression trees improve tree-based regression with theoretical and practical benefits.
problem Improving tree-based regression models with theoretical guarantees and practical tractability.
method Regularized piecewise-linear node-splitting criterion, LASSO-type and ℓ2 regularization, variable selection procedure. result New high-probability generalization error bounds for piecewise-linear regression trees.
New findings show a balance between data fit and complexity in kernel hyperparameters.
problem Overcorrelation due to reparametrization of kernel hyperparameters.
method Reparametrization of kernel hyperparameters and analysis of marginal likelihood.
result Data fit term influences all other kernel hyperparameters, not just the complexity penalty.
LLA shows strong performance in Bayesian optimization but has unbounded search space issues.
problem Applying LLA in unbounded search spaces for Bayesian optimization.
method Linearized-Laplace approximation applied to Bayesian optimization problems.
result LLA demonstrates strong performance but also presents unbounded search space challenges.
Success in the quest for artificial intelligence has the potential to bring unprecedented benefits to humanity, and it is therefore worthwhile to investigate how to maximize these benefits while avoiding potential pitfalls. This article gives numerous examples (which should by no means be construed as an exhaustive lis…
New datasets reveal neural networks can rely on simple features, leading to poor generalization.
problem Neural networks' reliance on simple features can lead to poor generalization and robustness.
method Designing datasets with varying levels of simplicity and incorporating non-robustness.
result Neural networks can exclusively rely on the simplest feature, leading to poor performance on complex data.
Machine learning methods have gained a great deal of popularity in recent years among public administration scholars and practitioners. These techniques open the door to the analysis of text, image and other types of data that allow us to test foundational theories of public administration and to develop new theories. …
Proposes a new undersampling method for imbalanced data classification.
problem Challenges of oversampling and undersampling in imbalanced data.
method Bilevel optimization framework for identifying optimal subset of majority training data.
result Improves F1 scores by up to 10% compared to state-of-the-art methods.
Deep learning method for fair derivative pricing.
problem Fair pricing of financial derivatives with hedging.
method Deep reinforcement learning, modified equal risk pricing framework.
result Derivative prices are arbitrage-free and more tractable.
Statistical analysis reveals pitfalls in climate network construction.
problem Uncertainty in correlation estimates affects network characteristics.
method Simulated isotropic random fields and resampling procedures.
result Spurious link bundles and high-degree clusters are common.
TBAL reduces manual annotation but requires validated data.
problem Creating large, high-quality labeled datasets.
method Threshold-based auto-labeling using human validation data.
result Sample complexity bounds on validation data needed.
This document provides a tutorial description of the use of the MDL principle in complex graph analysis. We give a brief summary of the preliminary subjects, and describe the basic principle, using the example of analysing the size of the largest clique in a graph. We also provide a discussion of how to interpret the r…
Statistical tests for fairness in admissions data reveal hidden patterns.
problem Simpson's paradox in admissions data hides true gender bias.
method Introduces a new statistical test based on Pearl's instrumental-variable inequalities.
result Statistical tests for fairness coincide with causal notions for the Berkeley admissions case.
Paper proposes an ensemble of attacks to evaluate adversarial robustness more reliably.
problem Insufficient evaluation of adversarial defenses leads to incorrect robustness assessments.
method Developed two extensions of PGD-attack and combined them with two complementary attacks.
result Identified several broken defenses with lower robust test accuracy than reported.
This document is an invited chapter covering the specificities of ABC model choice, intended for the incoming Handbook of ABC by Sisson, Fan, and Beaumont (2017). Beyond exposing the potential pitfalls of ABC based posterior probabilities, the review emphasizes mostly the solution proposed by Pudlo et al. (2016) on the…
Framework adds human knowledge to AI decisions to improve outcomes.
problem Conflict between AI recommendations and human insights.
method Develops a framework for integrating human knowledge as a guardrail for AI decisions.
result Human knowledge can improve AI decisions, especially in specific pitfalls.
This letter uses the Block Maxima Extreme Value approach to quantify catastrophic risk in international equity markets. Risk measures are generated from a set threshold of the distribution of returns that avoids the pitfall of using absolute returns for markets exhibiting diverging levels of risk. From an application t…
We construct Zero-Coupon Bond markets driven by a cylindrical Brownian motion in which the notion of generalized portfolio has important flaws: There exist bounded smooth random variables with generalized hedging portfolios for which the price of their risky part is +∞ at each time. For these generalized portfol…
New method AdaMod stabilizes deep neural network training by limiting adaptive learning rates.
problem Adaptive learning rates can produce extremely large values at the start of training, hindering learning.
method AdaMod uses adaptive and momental upper bounds to restrict learning rates dynamically.
result AdaMod eliminates large learning rates and improves training on complex networks.
Abstract MDPs enable strategic exploration and fast reward transfer in complex environments.
problem Challenging to learn accurate MDPs for high-dimensional states.
method Learn an abstract MDP over low-dimensional coarse states, using an abstraction function.
result Achieves superhuman performance on Pitfall! and higher reward with fewer samples.
C-index varies among software, complicating model comparison.
problem Variation in C-index calculations across software.
method Comparison of C-index estimators in R and Python.
result Different implementations yield varying results.
Generative Adversarial Networks simulate elevator group control without extensive data.
problem Lack of historical real-world data for system testing.
method Used GANs to generate simulation data for a multi-car elevator system.
result GANs can be used as substitutes for expensive simulation runs, but fine-tuning is needed.
In this paper we sketch some reflections on the pitfalls and inconsistencies of the research program - currently dominant among the profession - aimed at providing microfoundations to macroeconomics along a Walrasian perspective. We argue that such a methodological approach constitutes an unsatisfactory answer to a wel…