This work prunes CNN filters based on their functionality, not just size.
problem Redundant filters in CNNs waste computation resources.
method Functionality-oriented filter pruning method.
result Pruning based on functionality optimizes computation and interprets filter importance.
Proposes interpretable filters in CNNs for object classification.
problem Learning interpretable filters in deep CNNs without additional annotations.
method Assigns each filter in a CNN to an object part during training.
result Interpretable filters are more semantically meaningful than traditional filters.
Convolutional neural networks (CNNs) achieve state-of-the-art performance in a wide variety of tasks in computer vision. However, interpreting CNNs still remains a challenge. This is mainly due to the large number of parameters in these networks. Here, we investigate the role of compression and particularly pruning fil…
ISVAE enhances interpretability in time series clustering using a novel filter bank.
problem Improving interpretability in time series clustering models.
method Integrates a Filter Bank (FB) into a Variational Autoencoder (VAE) to enhance interpretability and clusterability.
result ISVAE produces a more interpretable and separable encoding with enhanced clusterability.
A guide to using low-pass graph filters for network data.
problem Understanding and processing graph data with low-pass filters.
method Definition and application of low-pass graph filters to graph data.
result Low-pass filters effectively retain lower frequency graph data contents.
DualSMC combines filtering and planning for continuous POMDPs.
problem Handling multi-modal state distributions and uncertainty in continuous POMDPs.
method DualSMC network that combines SMC for filtering and planning, with adversarial particle filter and uncertainty-dependent policy.
result DualSMC effectively handles complex observations and remains interpretable.
SC-Net learns interpretable filters for inverse problems, achieving optimal convergence and super-resolution.
problem Solving ill-posed inverse problems with effective regularization and interpretability.
method SC-Net operates in the spectral domain, learning a pointwise adaptive filter function based on signal-to-noise ratio.
result SC-Net achieves optimal convergence rate and zero-shot super-resolution, matching theoretical bounds.
Improved Kalman filter for non-linear, non-Gaussian data.
problem Estimating hidden variables with non-linear, non-Gaussian observations.
method Reproduces and extends Burkhart et al.'s discriminative Kalman filter.
result Enhanced filter performance for complex observation models.
This text investigates relations between two well-known family of algorithms, matrix factorisations and recursive linear filters, by describing a probabilistic model in which approximate inference corresponds to a matrix factorisation algorithm. Using the probabilistic model, we derive a matrix factorisation algorithm …
FrequentNet uses frequency domain basis vectors for image classification, making models more interpretable and efficient.
problem Image classification models are often complex and hard to interpret.
method FrequentNet selects filter vectors from frequency domain basis vectors instead of training them with back propagation.
result The method improves interpretability and efficiency of image classification models.
New algorithms for interpreting complex multivariate functions.
problem Hard interpretation of multivariate functions due to many parameters.
method Filtered tensor decompositions of derivative information.
result Nonparametric estimates of smooth decoupled functions.
Improved volatility estimation using SV-PF-RNN.
problem Estimating true volatility in the presence of market noise.
method SV-PF-RNN: hybrid neural network and particle filter architecture.
result SV-PF-RNN outperforms basic particle filter.
New particle filter estimates model evidence without bias.
problem Unbiased estimation of marginal likelihood for model comparison.
method Particle filter with rejection control.
result Unbiased estimation of marginal likelihood.
We cast Amari's natural gradient in statistical learning as a specific case of Kalman filtering. Namely, applying an extended Kalman filter to estimate a fixed unknown parameter of a probabilistic model from a series of observations, is rigorously equivalent to estimating this parameter via an online stochastic natural…
In an effort to understand the meaning of the intermediate representations captured by deep networks, recent papers have tried to associate specific semantic concepts to individual neural network filter responses, where interesting correlations are often found, largely by focusing on extremal filter responses. In this …
Adaptive spectral RL method enhances RL performance and interpretability.
problem Balancing interpretability and performance in reinforcement learning.
method Spectral based linear RL approach with adaptive regularization.
result Near-optimal bounds for parameter estimation and generalization error.
The paper audits trading filters, finding a high save-to-miss ratio.
problem Improving the efficiency and accuracy of trading filters in decentralized exchanges.
method A precision audit of filter rules against real trading data, classifying rejection events.
result Conservative save-to-miss ratio of 3.7 : 1, with wider interpretation of 14.8 : 1.
Proposes a probabilistic optimization method for large-scale problems.
problem Large-scale regularized optimization problems.
method Develops a probabilistic interpretation of the incremental proximal gradient algorithm and uses Bayesian filtering.
result Makes it possible to solve large-scale problems using well-known Bayesian filters.
Many businesses are using recommender systems for marketing outreach. Recommendation algorithms can be either based on content or driven by collaborative filtering. We study different ways to incorporate content information directly into the matrix factorization approach of collaborative filtering. These content-booste…
Three situations in which filtering theory is used in mathematical finance are illustrated at different levels of detail. The three problems originate from the following different works: 1) On estimating the stochastic volatility model from observed bilateral exchange rate news, by R. Mahieu, and P. Schotman; 2) A stat…
Unsupervised learning filters tweets for emergency services during crises.
problem Challenges in filtering relevant information from social web data during disasters.
method Multi-task domain adversarial attention network for unsupervised domain adaptation.
result The multi-task model outperforms single task models in filtering relevant tweets.
Proposes a Gaussian process for graph signals using adaptive spectral kernels.
problem Predicting signals on graph nodes with various structures.
method Spectral kernel learning approach that incorporates a polynomial function in the graph spectral domain.
result The model accurately recovers ground truth spectral filters and outperforms in real-world graph data.
The kernel least mean squares (KLMS) algorithm is a computationally efficient nonlinear adaptive filtering method that "kernelizes" the celebrated (linear) least mean squares algorithm. We demonstrate that the least mean squares algorithm is closely related to the Kalman filtering, and thus, the KLMS can be interpreted…
Improved model-based estimation through tempered Bayes filter.
problem Improving predictive accuracy in partially-observable stochastic systems.
method Developed tempered Bayes filter combining likelihood and full posterior tempering.
result Tempered Bayes filter achieves improved predictive performance over the Bayes filter baseline.
Ridge regression linked to Poisson resetting in statistical physics.
problem Understanding and extending ridge regularization in machine learning.
method Connecting stochastic resetting from statistical physics with ridge regularization in machine learning, using renewal processes.
result Exact filter identities for ridge regularization in various reset laws, including exponential and non-exponential.
We present the collaborative Kalman filter (CKF), a dynamic model for collaborative filtering and related factorization models. Using the matrix factorization approach to collaborative filtering, the CKF accounts for time evolution by modeling each low-dimensional latent embedding as a multidimensional Brownian motion.…
Neuron Shapley identifies key neurons in deep networks, improving model accuracy and fairness.
problem Identifying responsible neurons in deep networks for better model performance and fairness.
method Neuron Shapley framework quantifies neuron contributions, accounting for interactions.
result Removing just 30 critical filters can destroy model accuracy, revealing network function.
Develops an online learning framework for Bayesian joint filtering.
problem Streaming inference of nonlinear state-space models.
method Variational inference and sequential Monte Carlo.
result Efficient approximation of filtering posterior for a wide class of models.
Bayesian technique compresses RNNs by 100x without tuning.
problem Large parameter size in RNNs, especially embeddings.
method Bayesian sparsification for RNNs and vocabulary filtering.
result RNNs can be compressed dozens or hundreds of times.
Introduces Gaussian Processes and Relevance Vector Machines, connecting them to Kalman filtering.
problem Regression, smoothing, interpolation, and filtering problems.
method Bayesian kernel-based methods, Gaussian Processes, Relevance Vector Machines, connections to Kalman filtering.
result Developed a common framework for understanding these methods.
The paper stabilizes PD term structures under forecast uncertainty using a Kalman filter with an anchored observation model.
problem Stable estimation of lifetime PDs under forecast uncertainty.
method Reformulated in state-space framework, introduced an anchored observation model.
result Asymptotic stochastic stability of error dynamics, leading to smoother projections.
KalmanNet uses neural networks to improve state estimation in systems with unknown dynamics.
problem State estimation of systems with non-linear dynamics and partial information.
method KalmanNet integrates a recurrent neural network with the Kalman filter to handle non-linearities and model mismatches.
result KalmanNet outperforms classic filtering methods in systems with both mismatched and accurate domain knowledge.
MFCNs use sparse graphs to approximate manifold convergence.
problem Understanding manifold neural networks (MNNs).
method Sparse graph approximation for manifold convergence.
result Method converges to continuum limit as data points increase.
In this paper we study the estimation of changing trends in time-series using ℓ1 trend filtering. This method generalizes 1D Total Variation (TV) denoising for detection of step changes in means to detecting changes in trends, and it relies on a convex optimization problem for which there are very efficient numer…
The bane of one-class collaborative filtering is interpreting and modelling the latent signal from the missing class. In this paper we present a novel Bayesian generative model for implicit collaborative filtering. It forms a core component of the Xbox Live architecture, and unlike previous approaches, delineates the o…
Explicit encoding of group actions in deep features makes it possible for convolutional neural networks (CNNs) to handle global deformations of images, which is critical to success in many vision tasks. This paper proposes to decompose the convolutional filters over joint steerable bases across the space and the group …
This work ranks CNN filters based on their importance.
problem Unclear role of CNN neurons in producing output.
method Two methods: Shapley value game theory and Importance switch variational inference.
result Filters with higher importance are more crucial for output.
Neurons predict future scalar inputs by learning top modes of lag vectors.
problem Predicting future scalar inputs with physiological delays.
method Normal Mode Decomposition to extract independently evolving modes.
result Temporal filters of neurons correspond to left eigenvectors of a generalized eigenvalue problem.
Extends V-IP framework to use LLMs for generating task-relevant concepts, improving interpretability and performance.
problem Limited applicability of V-IP to small-scale tasks due to manual data annotation.
method Integrates Foundational Models with Large Language and Multimodal Models to generate and annotate concepts.
result FM+V-IP achieves better test performance with fewer concepts/queries compared to other frameworks.
Proposes M-CHMM for robust modeling of multivariate healthcare time series.
problem Challenges in analyzing multivariate healthcare time series data.
method Mixture of coupled hidden Markov models (M-CHMM) with two sampling algorithms.
result Improves data fit, handles missing and noisy measurements, and enhances prediction accuracy.
New research connects evolutionary dynamics to Bayesian learning.
problem Connecting evolutionary biology and Bayesian learning.
method Rigorous mathematical proof using Kushner-Stratonovich equation and gradient flows.
result Discrete time filtering equations converge to Stratonovich interpretation of Kushner-Stratonovich equation.
Introduces MFCNs for better understanding manifold neural networks.
problem Understanding manifold neural networks (MNNs).
method Filter-combine framework on high-dimensional point clouds, approximating manifold by sparse graph.
result Method converges to continuum limit as data points increase.
GAMI-Tree uses model-based trees to fit low-order fANOVA models.
problem Fitting interpretable fANOVA models with low-order interactions.
method GAMI-Tree uses model-based trees and a new interaction filtering method.
result GAMI-Tree outperforms EBM and GAMI-Net in predictive performance and interpretability.
Kalman Filters are one of the most influential models of time-varying phenomena. They admit an intuitive probabilistic interpretation, have a simple functional form, and enjoy widespread adoption in a variety of disciplines. Motivated by recent variational methods for learning deep generative models, we introduce a uni…
A new method learns state and proposal dynamics in state-space models using neural networks.
problem Inference in non-linear state-space models.
method StateMixNN method using neural networks for proposal and transition distributions.
result Significantly improved recovery of hidden state, especially in highly non-linear scenarios.
Triangulation filters spurious circuits in multilingual models.
problem Unreliable explanations of multilingual models across languages.
method Formalizes reference families and introduces triangulation as a causal acceptance rule.
result Triangulation provides a falsifiable standard for mechanistic claims.
ScDCFNet improves multiscale image classification with reduced model size.
problem Improving performance in multiscale image classification.
method Decomposed convolutional filters for ST-equivariant CNNs.
result ScDCFNet achieves significantly improved performance in multiscale image classification.
The paper addresses instability in CNNs' first layer by proving max pooling's shift invariance.
problem Instability in CNNs' first layer, leading to sensitivity to small input shifts.
method Establishing conditions for max pooling's shift invariance and deriving a measure of stability.
result Max pooling approximates a nearly shift-invariant complex modulus under certain conditions.