Convolutional neural networks (CNNs) achieve state-of-the-art performance in a wide variety of tasks in computer vision. However, interpreting CNNs still remains a challenge. This is mainly due to the large number of parameters in these networks. Here, we investigate the role of compression and particularly pruning fil…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “filter importance”
Prunes CNNs by removing redundant filters with provable guarantees.
problem Redundant filters in over-parameterized neural networks.
method Sampling-based approach using saliency scores and importance sampling.
result Consistently generates sparser and more efficient models.
Pruning CNNs by removing less important filters based on empirical loss changes.
problem Reducing memory and computation requirements for CNNs on resource-limited devices.
method Developed a novel filter importance norm based on empirical loss changes, and used sampling and ranking to prune filters.
result Reduced 60% of parameters and 64% of FLOPs with less than 0.6% accuracy drop.
The ability to track a moving vehicle is of crucial importance in numerous applications. The task has often been approached by the importance sampling technique of particle filters due to its ability to model non-linear and non-Gaussian dynamics, of which a vehicle travelling on a road network is a good example. Partic…
The study reveals flaws in pruning criteria and proposes a new assumption for better filter selection.
problem Flaws in existing pruning criteria for CNNs.
method Empirical experiments and Convolutional Weight Distribution Assumption.
result The Convolutional Weight Distribution Assumption improves filter selection in pruning.
An Automatic Relevance Determination Prior Bayesian Neural Network for Controlled Variable Selectionstat.ML
Bayesian neural network improves feature selection and prediction.
problem Improving feature selection and prediction accuracy in neural networks.
method BNN-ARD with l2-norm feature importance measure.
result Improves variable selection and predictive performance on real-world data.
Convolutional neural networks (CNNs) in recent years have made a dramatic impact in science, technology and industry, yet the theoretical mechanism of CNN architecture design remains surprisingly vague. The CNN neurons, including its distinctive element, convolutional filters, are known to be learnable features, yet th…
It is not easy to design and run Convolutional Neural Networks (CNNs) due to: 1) finding the optimal number of filters (i.e., the width) at each layer is tricky, given an architecture; and 2) the computational intensity of CNNs impedes the deployment on computationally limited devices. Oracle Pruning is designed to rem…
Sequential Monte Carlo techniques are useful for state estimation in non-linear, non-Gaussian dynamic models. These methods allow us to approximate the joint posterior distribution using sequential importance sampling. In this framework, the dimension of the target distribution grows with each time step, thus it is nec…
Advances deep network embedding through multi-filtering GCN.
problem Existing attribute embedding methods fail to capture different aspects of node features.
method Multi-filtering Graph Convolution Neural Network (GCN) framework.
result Significant improvement in link prediction and node classification tasks with limited training data.
A new method estimates time-varying parameters in earth system models using offline and online data assimilation.
problem Estimating time-varying parameters in complex earth system models.
method Hybrid Offline Online Parameter Estimation with Particle Filtering (HOOPE-PF)
result HOOPE-PF outperforms existing methods, especially with small ensemble sizes.
Proposes IFCDA framework to improve cross-domain adaptation.
problem Negative transfer and difficulty in handling category-irrelevant losses in DA.
method Importance filtered mechanism to generate filtered soft labels, combined with graph-based label propagation.
result Significantly improves performance in both Closed-Set and Open-Set DA scenarios.
This paper focuses on spectral filters on graphs, namely filters defined as elementwise multiplication in the frequency domain of a graph. In many graph signal processing settings, it is important to transfer a filter from one graph to another. One example is in graph convolutional neural networks (ConvNets), where the…
New method filters large networks from financial data to reveal key subnetworks.
problem Filtering large dimensional networks to isolate key constituents.
method Exploits spectral properties of high-dimensional data networks, tuning for sparsity and consistency.
result Shows method can interpolate between zero and maximal filtering, preserving spectral properties.
Shrinkage and spectral filtering of correlation matrices: a comparison via the Kullback-Leibler distancephysics.data-an
The problem of filtering information from large correlation matrices is of great importance in many applications. We have recently proposed the use of the Kullback-Leibler distance to measure the performance of filtering algorithms in recovering the underlying correlation matrix when the variables are described by a mu…
New MCFOs improve learning generative models and time series inference.
problem Challenges in learning generative models and inferring latent trajectories for time series.
method Proposed Monte Carlo filtering objectives (MCFOs) for joint learning and adaptive proposals.
result MCFOs lead to efficient and stable model learning and explain data well.
Improved stock volume prediction using Kalman Filters with various hidden states.
problem Improving accuracy of intraday trading volume prediction.
method Extended Kalman Filter with various hidden states for different stocks, using cross-validation to determine optimal state number.
result Demonstrated improved accuracy through comparison experiments and numerical analysis.
A new spectrum attention mechanism improves time series classification.
problem Improving robustness and classification accuracy in time series classification.
method Proposes a spectrum attention mechanism (SAM) to filter and highlight important frequency components, using L1 regularization and a tumbling window for segmentation.
result Experimental results show that the proposed SSAM method produces better feature representations and improves classification accuracy.
This paper presents a fast Bayesian filtering technique for state estimation.
problem Bottleneck in Bayesian inference for state estimation from noisy sensor data.
method Processor-native uncertainty tracking for uncertainty propagation and inference.
result Deterministic approximate filtering with up to 805x speedup and competitive accuracy.
We consider filtering in high-dimensional non-Gaussian state-space models with intractable transition kernels, nonlinear and possibly chaotic dynamics, and sparse observations in space and time. We propose a novel filtering methodology that harnesses transportation of measures, convex optimization, and ideas from proba…
A Bloom filter approach combined with Transformer models improves accuracy for machine learning tasks on opaque IDs.
problem Improving accuracy for machine learning tasks on opaque IDs with large vocabulary sizes.
method Applying hash functions to map opaque IDs to multiple hash tokens, similar to a Bloom filter, and using a multi-layer Transformer to process these digests.
result Models outperform those without hashing and sampled softmax, achieving high accuracy with a smaller computational budget.
BeMF improves recommendation reliability in recommender systems.
problem Improving reliability in recommender systems beyond accuracy.
method Bernoulli Matrix Factorization (BeMF) for model-based collaborative filtering.
result BeMF selects more reliable predictions, improving recommendation quality.
The accuracy of least squares calibration using option premiums and particle filtering of price data to find model parameters is determined. Derivative models using exponential Lévy processes are calibrated using regularized weighted least squares with respect to the minimal entropy martingale measure. Sequential impor…
Develops NFCF to reduce gender bias in social media recommendation systems.
problem Reduces gender bias in collaborative filtering systems on social media data.
method Pre-training and fine-tuning neural collaborative filtering with bias correction techniques.
result Achieves better performance and fairness in gender de-biased recommendations.
Neuron Shapley identifies key neurons in deep networks, improving model accuracy and fairness.
problem Identifying responsible neurons in deep networks for better model performance and fairness.
method Neuron Shapley framework quantifies neuron contributions, accounting for interactions.
result Removing just 30 critical filters can destroy model accuracy, revealing network function.
On Data-Selective Learningeess.SP
Adaptive filters are applied in several electronic and communication devices like smartphones, advanced headphones, DSP chips, smart antenna, and teleconference systems. Also, they have application in many areas such as system identification, channel equalization, noise reduction, echo cancellation, interference cancel…
On-line estimation plays an important role in process control and monitoring. Obtaining a theoretical solution to the simultaneous state-parameter estimation problem for non-linear stochastic systems involves solving complex multi-dimensional integrals that are not amenable to analytical solution. While basic sequentia…
ATPF combines PF and EnKF for better inference in complex systems.
problem Weight degeneracy in PF and approximation errors in EnKF.
method Adversarial learning to improve posterior matching and incorporate kernel methods for optimization.
result ATPF provides theoretical guarantees and practical advantages over PF and EnKF.
This paper introduces the kernel mixture network, a new method for nonparametric estimation of conditional probability densities using neural networks. We model arbitrarily complex conditional densities as linear combinations of a family of kernel functions centered at a subset of training points. The weights are deter…
A new filter design improves system identification accuracy.
problem Improving system identification accuracy for various system types.
method Generalized proportionate-type normalized subband adaptive filter (GPtNSAF) using least squares on subband errors with a sparsity penalty.
result GPtNSAF benefits from increasing subbands more than sparsity for quasi-sparse or dispersive systems, and both aspects are complementary for sparse systems.
We propose a principled algorithm for robust Bayesian filtering and smoothing in nonlinear stochastic dynamic systems when both the transition function and the measurement function are described by non-parametric Gaussian process (GP) models. GPs are gaining increasing importance in signal processing, machine learning,…
A major issue in harmonic analysis is to capture the phase dependence of frequency representations, which carries important signal properties. It seems that convolutional neural networks have found a way. Over time-series and images, convolutional networks often learn a first layer of filters which are well localized i…
In today's day and age when almost every industry has an online presence with users interacting in online marketplaces, personalized recommendations have become quite important. Traditionally, the problem of collaborative filtering has been tackled using Matrix Factorization which is linear in nature. We extend the wor…
Improved product recommendations using deep learning.
problem Sparse customer purchasing data for personalized recommendations.
method Deep Collaborative Filtering (NCF) with latent variables and Bayesian Optimization.
result NCF achieved highest NDCG performance on proprietary dataset.
A new Fourier model improves ODE prediction.
problem Improving the accuracy of ODE solutions, especially for periodic functions.
method Constructing a Fourier state space model and a hybrid model combining Taylor and Fourier methods.
result The hybrid model can predict ODE solutions more accurately, especially for periodic functions.
Advances in collaborative filtering and ranking methods.
problem Improving recommendation systems efficiency and accuracy.
method Graph information encoding, pairwise and listwise approaches, regularization techniques, personalization.
result New methods significantly improve recommendation system performance.
A method for optimal Bayesian filtering using progressive particle flow and optimal transport maps.
problem Optimizing Bayesian filtering with deterministic particles to avoid degeneration.
method Progressive flow of particles through a sequence of sub-steps, each using an optimal transport map to replace non-equally weighted particles with equally weighted ones.
result The method avoids particle degeneration and simplifies the filtering process by not requiring inversions or monotonicity constraints.
New training algorithm enhances SNNs for temporal signal processing.
problem Lack of robust training algorithms for large-scale SNNs.
method Formulated SNN as IIR filters, proposed training algorithm for optimal synapse filter kernels and weights.
result Model and training algorithm outperform state-of-the-art approaches in accuracy.
New research connects evolutionary dynamics to Bayesian learning.
problem Connecting evolutionary biology and Bayesian learning.
method Rigorous mathematical proof using Kushner-Stratonovich equation and gradient flows.
result Discrete time filtering equations converge to Stratonovich interpretation of Kushner-Stratonovich equation.
Feature selection is frequently used as a pre-processing step to machine learning. It is a process of choosing a subset of original features so that the feature space is optimally reduced according to a certain evaluation criterion. The central objective of this paper is to reduce the dimension of the data by finding a…
Controlled interacting particle systems such as the ensemble Kalman filter (EnKF) and the feedback particle filter (FPF) are numerical algorithms to approximate the solution of the nonlinear filtering problem in continuous time. The distinguishing feature of these algorithms is that the Bayesian update step is implemen…
State of the art music recommender systems mainly rely on either matrix factorization-based collaborative filtering approaches or deep learning architectures. Deep learning models usually use metadata for content-based filtering or predict the next user interaction by learning from temporal sequences of user actions. D…
The filtering-clustering models, including trend filtering and convex clustering, have become an important source of ideas and modeling tools in machine learning and related fields. The statistical guarantee of optimal solutions in these models has been extensively studied yet the investigations on the computational as…
Efficient algorithm predicts unknown linear systems with long-term memory.
problem Predicting unknown and partially observed linear dynamical systems with long-term memory.
method Bounding the generalized Kolmogorov width of the Kalman filter model using spectral methods and conducting tight convex relaxation.
result Competes with Kalman filter in hindsight with only logarithmic regret.
We characterize and study variable importance (VIMP) and pairwise variable associations in binary regression trees. A key component involves the node mean squared error for a quantity we refer to as a maximal subtree. The theory naturally extends from single trees to ensembles of trees and applies to methods like rando…
Despite the importance of sparsity signal models and the increasing prevalence of high-dimensional streaming data, there are relatively few algorithms for dynamic filtering of time-varying sparse signals. Of the existing algorithms, fewer still provide strong performance guarantees. This paper examines two algorithms f…
This paper addresses the problem of filtering with a state-space model. Standard approaches for filtering assume that a probabilistic model for observations (i.e. the observation model) is given explicitly or at least parametrically. We consider a setting where this assumption is not satisfied; we assume that the knowl…
This paper focuses on spectral graph convolutional neural networks (ConvNets), where filters are defined as elementwise multiplication in the frequency domain of a graph. In machine learning settings where the dataset consists of signals defined on many different graphs, the trained ConvNet should generalize to signals…