Researchers review challenges in interpreting additive models, especially neural additive models.
problem Challenges in interpreting additive models, particularly neural additive models.
method Review of generalized additive models and discussion of nonidentifiability.
result Challenges in claiming interpretability or suitability for safety-critical applications of additive models.
Monotonic neural additive models simplify machine learning for credit scoring.
problem Complex machine learning methods make models less transparent and fair.
method Introducing monotonic neural additive models that simplify neural networks while maintaining regulatory compliance.
result Monotonic neural additive models achieve similar accuracy to complex neural networks but are more transparent and fair.
Bayesian principles improve neural additive models for better feature selection and uncertainty.
problem Lack of calibrated uncertainties and feature selection in neural additive models.
method Augmenting NAMs with Bayesian principles to provide credible intervals, feature selection, and interaction ranking.
result Improved performance on tabular datasets and real-world medical tasks.
Neural model improves option pricing by calibrating additive process term structure.
problem Calibrating additive process models for option pricing with time-dependent parameters.
method Proposes neural term structure model using feedforward neural networks to represent term structure.
result Improves option pricing accuracy with neural term structure model.
NAMLSS models provide interpretable neural regression for location, scale, and shape.
problem Lack of interpretability in deep learning models for complex data distributions.
method Combines classical statistical methods with DNNs for distributional regression.
result Achieves visual interpretability and predictive power of deep learning models.
New models improve machine learning accuracy and transparency in finance.
problem Black-box machine learning models lack interpretability in regulated industries.
method Introducing generalized groves of neural additive models with clear feature categories and interactions.
result Generalized groves of neural additive models achieve high accuracy with predominantly linear and sparse nonlinear components.
Neural Additive Models combine neural nets with interpretable models for high stakes tasks.
problem Inability to understand how neural networks make decisions.
method Combines neural nets with generalized additive models to create Neural Additive Models (NAMs).
result NAMs are more accurate than intelligible models and as accurate as state-of-the-art generalized additive models.
neuralGAM package interprets neural networks by fitting them to feature contributions.
problem difficulty understanding neural network decisions
method Generalized Additive Neural Networks (GAM)
result interpretable Deep Learning model with accurate feature contributions
ProtoNAM models tabular data with neural networks, making predictions transparent.
problem Tabular data analysis using neural networks lacks transparency and accuracy compared to tree-based methods.
method ProtoNAM introduces prototypes into neural networks to model tabular data while maintaining explainability.
result ProtoNAM outperforms existing NN-based GAMs and provides insights into learned feature patterns.
New image classifier uses hierarchical max-pooling with local pooling.
problem Improving image classification accuracy with variable spatial relationships.
method Introduces a hierarchical max-pooling model with additional local pooling for convolutional neural networks.
result Demonstrates improved performance in estimating image features.
Improved meta-learning for dynamics using additional structured knowledge.
problem Meta-learning for dynamics with limited raw observations.
method Extended Neural ODE Process model to use privileged information.
result Improved accuracy and calibration on simulated dynamics tasks.
Combining additive models and neural networks allows to broaden the scope of statistical regression and extend deep learning-based approaches by interpretable structured additive predictors at the same time. Existing attempts uniting the two modeling approaches are, however, limited to very specific combinations and, m…
dnamite simplifies NAMs for feature selection and survival analysis.
problem Handling complex machine learning tasks on large-scale data.
method Python package implementing Neural Additive Models (NAMs) for feature selection and survival analysis.
result dnamite provides a scikit-learn style interface for training NAMs.
SIAN bridges simple models to neural networks by identifying necessary feature combinations.
problem The gap between simple models and powerful neural networks in performance.
method Feature interaction detection and sparse selection algorithm.
result Competitive performance across multiple tabular datasets with optimal tradeoff.
RNNs solve modular addition tasks using low rank and sparse Fourier structures.
problem Solving modular addition tasks with recurrent neural networks.
method Identified low rank structures and sparse Fourier representations in RNN weights.
result RNNs robust to removing individual frequencies but degrade with more ablation.
New neural networks combine additive regression with traditional architectures.
problem Performance limitations and high parameter requirements of traditional neural networks.
method Introduce hybrid deep additive neural networks with simpler activation and basis functions.
result Hybrid neural networks achieve better performance with fewer parameters.
Proposes DAK model for improved GP computations.
problem Challenges in high-dimensional GP layers in DKL.
method Additive structure and induced prior approximation for GP units.
result Outperforms state-of-the-art DKL methods in regression and classification.
The paper addresses monotonicity in machine learning models for fairness and accountability.
problem Ensuring fairness and accountability in transparent machine learning models.
method Study of three types of monotonicity (individual, weak pairwise, strong pairwise) and propose monotonic groves of neural additive models.
result Monotonic groves of neural additive models maintain transparency, accountability, and fairness.
GAMI-Net improves neural network interpretability while maintaining accuracy.
problem Lack of interpretability in neural network models.
method GAMI-Net is a disentangled feedforward network with multiple additive subnetworks designed for capturing main effects and pairwise interactions, considering sparsity, heredity, and marginal clarity.
result GAMI-Net achieves superior interpretability and competitive prediction accuracy compared to explainable boosting machine and other models.
Paper introduces GAMs for interpretable learning-to-rank models.
problem Need for transparent ranking models in legal or policy scenarios.
method Developed generalized additive models (GAMs) for ranking tasks using neural networks.
result Neural ranking GAMs achieve better performance than traditional GAMs while maintaining interpretability.
This review compares GAMs and neural networks on real-world tabular data.
problem Comparing the performance and characteristics of GAMs and neural networks in tabular data applications.
method Systematic review following PRISMA guidelines, extracting and analysing key attributes from 143 papers and 430 datasets.
result No consistent evidence of superiority for either GAMs or neural networks, with performance trade-offs depending on dataset characteristics.
Machine Learning algorithms are increasingly being used in recent years due to their flexibility in model fitting and increased predictive performance. However, the complexity of the models makes them hard for the data analyst to interpret the results and explain them without additional tools. This has led to much rese…
AxNN improves model interpretability without sacrificing predictive power.
problem Black-box nature of machine learning models hinders interpretation and explanation.
method AxNN combines ensembles of generalized additive model networks and additive index models.
result AxNN achieves both good predictive performance and model interpretability.
Theoretical analysis of deep neural networks for time series data.
problem Theoretical development for deep neural networks on temporally dependent observations is lacking.
method Established non-asymptotic bounds for prediction error of deep neural networks under mixing-type assumptions.
result Deep neural networks can model non-linear time series data with additional logarithmic factors due to dependence.
This paper proposes a novel approach to train deep neural networks by unlocking the layer-wise dependency of backpropagation training. The approach employs additional modules called local critic networks besides the main network model to be trained, which are used to obtain error gradients without complete feedforward …
SNAM improves NAM's accuracy and feature selection via group sparsity.
problem Improving interpretability and accuracy in deep learning models.
method Employing group sparsity regularization in neural additive models (SNAM).
result SNAM provably converges to zero training loss and achieves exact support recovery.
We show that Neural Ordinary Differential Equations (ODEs) learn representations that preserve the topology of the input space and prove that this implies the existence of functions Neural ODEs cannot represent. To address these limitations, we introduce Augmented Neural ODEs which, in addition to being more expressive…
This paper proposes an embedding-based neural network for more accurate investment return prediction.
problem Accurately predicting investment returns requires understanding industry knowledge and news, as well as leveraging relevant theories.
method The approach uses embedding to encode investment IDs into low-dimensional vectors, leveraging dual branches to separate different information, and employs the swish activation function.
result The proposed embedding-based dual branch model outperforms traditional machine learning models like Xgboost, Lightgbm, and Catboost on the Ubiquant Market Prediction dataset.
CoxSE combines deep learning with self-explaining neural networks for survival analysis.
problem Improving predictive power of Cox Proportional Hazards model while maintaining explainability.
method Proposes CoxSE, a locally explainable Cox proportional hazards model using SENN, and CoxSENAM, a hybrid model with NAM.
result CoxSE provides more stable and consistent explanations while maintaining predictive power.
Improved Bayesian optimization for conditional parameter spaces.
problem Efficient global optimization of expensive-to-evaluate functions in conditional parameter spaces.
method Additive tree-structured covariance function for conditional parameter optimization.
result Significantly improved sample-efficiency and wider applicability compared to existing methods.
Bayesian Additive Regression Networks use neural networks for regression tasks.
problem Regression tasks with small neural networks and ensemble learning.
method Bayesian Additive Regression Tree principles applied to small neural networks, Gibbs sampling for ensemble learning.
result BARN provides more consistent and often more accurate results than shallow neural networks, BART, and ordinary least squares.
A method interprets black-box models using an ensemble of gradient boosting machines.
problem Local and global interpretation of black-box models.
method An ensemble of gradient boosting machines (GBMs) to form a generalized additive model.
result Efficiency and properties demonstrated on synthetic and real datasets.
WrapNet optimizes inference for low-resolution neural networks by using 8-bit additions.
problem Reducing multiplication complexity in low-resolution neural networks.
method Adapting neural networks to use low-resolution (8-bit) additions in accumulators, with a cyclic activation layer and overflow penalty regularizer.
result Achieves comparable classification accuracy to 32-bit counterparts using low-resolution additions.
Neural networks play an increasingly important role in the field of machine learning and are included in many applications in society. Unfortunately, neural networks suffer from adversarial samples generated to attack them. However, most of the generation approaches either assume that the attacker has full knowledge of…
This paper characterizes how randomized neural networks generalize well in multi-dimensional tasks.
problem Understanding the generalization of randomized neural networks in multi-dimensional tasks.
method Characterizes RSNs as an IGAM formalized by an optimization problem with a regularization functional and loss.
result RSNs generalize well in multi-dimensional tasks, akin to spline regression under certain conditions.
BayesFlow learns complex models using neural networks.
problem Estimating parameters in complex, non-likelihood models.
method Invertible neural networks for global Bayesian inference.
result Global probabilistic mapping from data to parameters.
DREAM model improves computational efficiency for non-linear effects in relational event models.
problem Efficiently modeling non-linear effects in dynamic relational networks.
method Introduces Deep Relational Event Additive Model (DREAM) using Neural Additive Models.
result Demonstrates superior computational efficiency compared to traditional REM approaches.
Cyber-physical systems often consist of entities that interact with each other over time. Meanwhile, as part of the continued digitization of industrial processes, various sensor technologies are deployed that enable us to record time-varying attributes (a.k.a., time series) of such entities, thus producing correlated …
Training a neural network model can be a lifelong learning process and is a computationally intensive one. A severe adverse effect that may occur in deep neural network models is that they can suffer from catastrophic forgetting during retraining on new data. To avoid such disruptions in the continuous learning, one ap…
New model learns graph neural networks equivariant to various transformations.
problem Learning equivariant graph neural networks for complex transformations.
method E(n)-Equivariant Graph Neural Networks (EGNNs) that are computationally efficient and scalable.
result Achieves competitive or better performance without higher-order representations.
We propose a neural machine translation architecture that models the surrounding text in addition to the source sentence. These models lead to better performance, both in terms of general translation quality and pronoun prediction, when trained on small corpora, although this improvement largely disappears when trained…
We present a new method for uncertainty estimation and out-of-distribution detection in neural networks with softmax output. We extend softmax layer with an additional constant input. The corresponding additional output is able to represent the uncertainty of the network. The proposed method requires neither additional…
A scalable Bayesian additive model for stellar flare detection using Gaussian process inference and hidden Markov models.
problem Bayesian time-series modeling for astronomical datasets
method Generative surrogate framework with Variational Autoencoder and neural network forward pass
result Significant reduction in computational time for stellar flare detection
Deep neural networks tackle high-dimensional nonparametric interaction models.
problem Estimating structured regression functions in high-dimensional data.
method Analyze kth order nonparametric interaction models in growing and high dimensions, introducing debiasing techniques. result Debiased deep neural networks achieve optimal rates of convergence in both growing and high dimensions.
Existing approaches to combine both additive and multiplicative neural units either use a fixed assignment of operations or require discrete optimization to determine what function a neuron should perform. This leads either to an inefficient distribution of computational resources or an extensive increase in the comput…
FNNs detect EEG signals without position dependence.
problem Detecting EEG signals without position dependence.
method Shift invariant functional neural networks (FNNs) using FDA methods.
result FNNs outperform FDA benchmarks in EEG classification.
Improved noise estimation in latent neural SDEs enhances model accuracy.
problem Latent neural SDEs underestimate noise, limiting their stochastic dynamics modeling.
method Explicit additional noise regularization in the loss function.
result Model accurately captures diffusion component of stochastic time series data.
The objective of this work is to take advantage of deep neural networks in order to make next day crime count predictions in a fine-grain city partition. We make predictions using Chicago and Portland crime data, which is augmented with additional datasets covering weather, census data, and public transportation. The c…