Soft-AE interprets autoencoders with adaptable wavelet units.
problem Lack of interpretability in autoencoders.
method Proposes Soft-AE with adaptable wavelet units and GenLU.
result Soft-AE offers interpretability and competitive performance.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,051 papers · 148 categories
Trend · papers per month
48 results for “Interpretable Units”
Pruning neural networks reduces parameters without sacrificing interpretability.
problem Reducing unnecessary structure in neural networks to improve efficiency.
method Examined the effect of pruning on the number of hidden units learning disentangled representations.
result Pruning does not harm interpretability until a significant portion of parameters are removed.
Analyzes GAN units for better understanding and improvement.
problem Lack of understanding GAN internal representations and artifacts.
method Interpretable units identification, causal effect quantification, contextual relationship examination.
result Visualizes and understands GANs at various levels, enabling new insights and improvements.
CRITS improves time series classification with interpretable local explanations.
problem Lack of detailed explanations in time series classification models.
method CRITS uses convolutional kernels, max-pooling, and rectified linear units to extract feature weights.
result CRITS provides intrinsically interpretable local explanations without requiring gradients or random perturbations.
Combining interpretability and stability methods improves DNN robustness.
problem Improving interpretability and robustness of deep neural networks.
method Combining interpretability (conductance) and stability (binary classifier) methods to detect and discard wrong predictions.
result Combining interpretability and stability methods increases model robustness.
In this paper we propose and investigate a novel nonlinear unit, called unit, for deep neural networks. The proposed unit receives signals from several projections of a subset of units in the layer below and computes a normalized norm. We notice two interesting interpretations of the unit. First…
The approach we present is a modification of the Morse theory for unital C*-algebras. We provide tools for the geometric interpretation of noncommutative CW complexes. These objects were introduced and studied in [2],[7] and [14]. Some examples to illustrate these geometric information in practice are given. A classifi…
Deep networks respond to specific linguistic units, not arbitrary patterns.
problem Understanding how deep convolutional networks interpret natural language.
method Concept alignment method based on unit responsiveness to replicated text.
result Deep networks selectively respond to morphemes, words, and phrases, not arbitrary patterns.
Bayesian units improve speech recognition with minimal parameters.
problem Improving speech recognition models with fewer parameters.
method Derived Bayesian recurrent units integrated into deep learning frameworks.
result Adding Bayesian units improves speech recognition performance.
Most of the parameters in large vocabulary models are used in embedding layer to map categorical features to vectors and in softmax layer for classification weights. This is a bottle-neck in memory constraint on-device training applications like federated learning and on-device inference applications like automatic spe…
Paper interprets deep learning using decision trees and Haar wavelets.
problem Understanding the function approximation capabilities of ReLU deep learning.
method Constructing a deep learning structure equivalent to a forest and approximating Haar wavelet functions with ReLU deep learning.
result ReLU deep learning can be considered as decision trees and approximates Haar wavelet functions with arbitrary precision.
Dirichlet pruning compresses neural networks by removing unimportant units.
problem Compressing large neural network models without sacrificing performance.
method Assigns Dirichlet distribution over network layers' units and uses variational inference to estimate parameters.
result Achieves state-of-the-art compression performance on larger architectures like VGG and ResNet.
CoDA Nets improve interpretability in neural networks.
problem Improving interpretability in neural networks.
method Dynamic Alignment Units (DAUs) for input-dependent linear transformations.
result CoDA Nets achieve on par results with ResNet and VGG models on complex datasets.
For every finite collection of curves on a surface, we define an associated (semi-)norm on the first homology group of the surface. The unit ball of the dual norm is the convex hull of its integer points. We give an interpretation of these points in terms of certain coorientations of the original collection of curves. …
Foreign exchange markets show that currency units (= accounting or nominal price units) are variables. Technical and economic progress evidence that the consumer baskets (= purchasing power units or real price units) are also variables. In contrast, all physical measurement units are constants and either defined in the…
We aim to create the highest possible quality of treatment-control matches for categorical data in the potential outcomes framework. Matching methods are heavily used in the social sciences due to their interpretability, but most matching methods do not pass basic sanity checks: they fail when irrelevant variables are …
A remarkable characteristic of overparameterized deep neural networks (DNNs) is that their accuracy does not degrade when the network's width is increased. Recent evidence suggests that developing compressible representations is key for adjusting the complexity of large networks to the learning task at hand. However, t…
Interpreting the prediction mechanism of complex models is currently one of the most important tasks in the machine learning field, especially with layered neural networks, which have achieved high predictive performance with various practical data sets. To reveal the global structure of a trained neural network in an …
Set-Sequence model learns cross-sectional dynamics directly from time series data.
problem Predicting large cross-sections of time series data with latent cross-sectional dynamics.
method A model that learns cross-sectional structure directly, enhancing expressivity and eliminating manual feature engineering.
result Significantly outperforms strong baselines in equity portfolio optimization and loan risk prediction.
Dual neural network architecture improves accuracy and interpretability.
problem Improving neural network interpretability and accuracy.
method Stacked recurrent and feedforward layers, binary activation function.
result Binary activation leads to simpler, more interpretable models with higher accuracy.
Project uses GANs to recognize facial expressions and emotions from-the-wild with dual model approach.
problem Facial expression and emotion recognition in real-world scenarios.
method Created a dual GAN model architecture for Action Units and Valence Arousal annotations.
result Dual GAN model achieved better results than single model for emotion recognition.
Let M be a G2-manifold. We consider an almost CR-structure on the sphere bundle of unit tangent vectors on M, called the CR twistor space. This CR-structure is integrable if and only if M is a holonomy G2 manifold. We interpret G2-instanton bundles as CR-holomorphic bundles on its twistor space.
MEMGAN uses memory to improve anomaly detection by isolating abnormal data.
problem Weak guarantees for detecting anomalous data in classical algorithms.
method Memory-augmented Generative Adversarial Networks (MEMGAN) with a memory module.
result MEMGAN provides strong guarantees for anomaly detection with improved reconstruction.
Determining whether hypotensive patients in intensive care units (ICUs) should receive fluid bolus therapy (FBT) has been an extremely challenging task for intensive care physicians as the corresponding increase in blood pressure has been hard to predict. Our study utilized regression models and attention-based recurre…
Automates feature selection and weighting in molecular systems.
problem Optimal feature selection and alignment in molecular systems.
method Differentiable Information Imbalance (DII) method for automated feature ranking and scaling.
result Automated feature selection and scaling that preserves information content and interpretability.
This paper uses a geometric approach to understand how normalization layers affect neural network optimization.
problem Understanding the effect of normalization layers on optimization in neural networks.
method Introduces a spherical framework to study optimization dynamics of neural networks with normalization layers from a geometric perspective.
result Derives the first effective learning rate expression of Adam and shows that SGD with NLs is equivalent to a constrained variant of Adam.
ParaRNN: An Interpretable and Parallelizable Recurrent Neural Network for Time-Dependent Datastat.ML
ParaRNN improves RNN interpretability and parallelizability for time-dependent data.
problem Limited interpretability and slow training of RNNs.
method Parallelized RNN with additive representation and recurrence features.
result ParaRNN achieves comparable performance to vanilla RNNs but with improved interpretability and efficiency.
Equivariant neural networks use symmetry to interpret complex data.
problem Interpreting and understanding the behavior of equivariant neural networks.
method Decompose layers into simple representations and analyze nonlinear activation functions.
result Equivariant neural networks can be interpreted using a filtration generalizing Fourier series.
The study identifies patient subgroups with enhanced or diminished opioid treatment effects.
problem Lack of prescribing guidelines for opioids leading to adverse outcomes.
method Generative model using mixture distribution and sparsity to discover subgroups with treatment effects.
result Human-interpretable insights on subgroups with enhanced or diminished treatment effects.
We propose a sparse and low-rank tensor regression model to relate a univariate outcome to a feature tensor, in which each unit-rank tensor from the CP decomposition of the coefficient tensor is assumed to be sparse. This structure is both parsimonious and highly interpretable, as it implies that the outcome is related…
Bilinear MLPs offer a new way to interpret deep learning models without complex nonlinearities.
problem Lack of mechanistic understanding in how MLPs compute.
method Introduced bilinear MLPs without element-wise nonlinearities, analyzed their weights using tensor and eigendecomposition.
result Bilinear MLPs provide interpretable weight structures and enable adversarial attacks and overfitting analysis.
A novel approach combines interpretability and performance in machine learning models.
problem Lack of transparency in black box machine learning models.
method Semiparametric approach using ideas from sufficient dimension reduction and influence function based estimators.
result Optimized model combining interpretability and performance, demonstrated through simulations and a real-world ICU patient data application.
In this paper, we introduce an alternative approach, namely GEN (Genetic Evolution Network) Model, to the deep learning models. Instead of building one single deep model, GEN adopts a genetic-evolutionary learning strategy to build a group of unit models generations by generations. Significantly different from the well…
Researchers transform equations and define integral operators on a ball.
problem Transforming equations from half space to ball.
method Identify Poisson kernel, define extension operator, prove inequalities.
result Uniqueness of extremal functions in limit case.
In this work, we propose an infinite restricted Boltzmann machine~(RBM), whose maximum likelihood estimation~(MLE) corresponds to a constrained convex optimization. We consider the Frank-Wolfe algorithm to solve the program, which provides a sparse solution that can be interpreted as inserting a hidden unit at each ite…
Deep neural networks for ordinal outcomes combining image and tabular data.
problem Lack of interpretable models for ordinal outcomes in mixed data types.
method Ordinal Neural Network Transformation Models (ONTRAMs) integrating DL and classical ordinal regression.
result ONTRAMs achieve performance equivalent to standard multi-class DL models but are faster and more interpretable.
Model predicts COVID-19 progression with interpretability.
problem Accurate and credible forecasting of COVID-19 progression.
method Integrates machine learning into disease modeling, uses interpretable encoders.
result More accurate forecasts than state-of-the-art alternatives.
Deep learning predicts ICU mortality with enhanced interpretability.
problem Improving mortality prediction accuracy and clinician trust in AI.
method Trained a deep learning model on MIMIC-III to interpret nursing notes.
result Model reaches ROC of 0.8629, outperforming SAPS-II.
The paper introduces a method to explain redundancy in deep CNNs using unit impulse response.
problem Redundancy in deep CNNs leads to unnecessary computations and increased cost.
method Empirical demonstration and unit impulse response analysis to identify and quantify redundancy across layers and depth.
result Identifies and quantifies redundancy in deep CNNs, providing better insights into their internal dynamics.
Paper introduces models to discover complex structures in large hypergraphs.
problem Understanding dependency structures in complex systems represented as hypergraphs.
method Probabilistic models treating classes of similar units as nodes in a latent hypergraph, using low-rank representations.
result Improves link prediction and discovers interpretable structures in diverse real-world systems.
Partial soft-matching distance improves neural representation comparison by allowing some neurons to remain unmatched.
problem Neural representations are noisy and contain outliers, making traditional matching methods unreliable.
method Extends soft-matching distance to a partial optimal transport setting, allowing some neurons to remain unmatched.
result Partial soft-matching provides robust correspondences that are more reliable under noise and outliers.
Closed-form polynomial approximations replace MLPs in transformers, enabling new interpretability methods.
problem Replacing MLPs with polynomial approximations for transformer models.
method Theoretical derivation of closed-form least-squares approximations of MLPs and GLUs using polynomial functions.
result Polynomial approximations explain over 95% of MLP and GLU outputs' variance, enabling interpretability.
Generative Kernel PCA explores latent spaces for data interpretation and novelty detection.
problem Exploring latent spaces of datasets for better data interpretation.
method Generative Kernel PCA using hidden and visible units similar to Restricted Boltzmann Machines.
result Gradually moving in the latent space allows for interpretation of components and detection of novel patterns.
Method estimates heterogeneous causal effects on networks using orthogonal learning.
problem Challenges in estimating causal effects on networks due to treatment effects on both treated and neighbors, and network homophily.
method Two-stage orthogonal learning framework: first stage uses graph neural networks for nuisance components, second stage residualizes and interpretable attention-based model for causal effects.
result Improves heterogeneous effect estimation and supports interpretable analyses.
Extends deep learning with interpretable additive models.
problem Identifiability issues between neural networks and additive models.
method Orthogonalization cell to separate deep neural network and structured model parts.
result Stable estimation and interpretability of structured model parts.
Novel method combines neural network features with survival models for ICU infections.
problem Improving predictive models of ICU infections while maintaining interpretability.
method Semi-parametric approach combining low-resolution and high-resolution data.
result Improved predictive power with interpretability maintained.
High-dimensional models can outperform simpler ones in causal inference.
problem Estimating average treatment effects with many covariates.
method High-dimensional linear regression and synthetic control with many control units.
result Adding more control units can improve imputation performance even when pre-treatment fit is perfect.
Disease progression models are instrumental in predicting individual-level health trajectories and understanding disease dynamics. Existing models are capable of providing either accurate predictions of patients prognoses or clinically interpretable representations of disease pathophysiology, but not both. In this pape…