Develops a tensor network framework to reduce RNN complexity for high-dimensional sequence modeling.
problem Exponential parameter growth in RNNs for large multidimensional data.
method Embeds a multi-linear graph filter in a tensor network architecture to approximate RNN hidden states.
result Demonstrates superior performance and reduced complexity compared to traditional RNNs.
Algorithm estimates tensors from sparse observations with robust error bounds.
problem Estimating tensors from sparse noisy observations.
method Similarity-based collaborative filtering algorithm for tensor estimation.
result Achieves sample complexity nearly matching conjectured lower bound.
This work speeds up fHMM analysis by tensor algebra.
problem Scalability issues in analyzing factorial hidden Markov models.
method Tensorized algorithms and scalable filtering methods.
result Significant improvement in computational performance.
New model improves recommendation systems by analyzing user-item interactions.
problem Improving recommendation systems for better user-item interactions.
method Sliced Anti-symmetric Decomposition (SAD) model using tensor decomposition.
result SAD produces the most consistent personalized preferences compared to SOTA models.
SG-NTF completes HDI tensors with spectral mapping and spatio-temporal gating.
problem High-dimensional and incomplete tensor completion.
method Spectra-Guided Neural Tucker Factorization (SG-NTF) with Spatio-Temporal Co-Gating (STCG).
result Maintains competitive completion accuracy with parameter efficiency.
New algorithms for interpreting complex multivariate functions.
problem Hard interpretation of multivariate functions due to many parameters.
method Filtered tensor decompositions of derivative information.
result Nonparametric estimates of smooth decoupled functions.
We prove the correspondence between the information geometry of a signal filter and a Kähler manifold. The information geometry of a minimum-phase linear system with a finite complex cepstrum norm is a Kähler manifold. The square of the complex cepstrum norm of the signal filter corresponds to the Kähler potential. The…
Recently, many researchers have been focusing on the definition of neural networks for graphs. The basic component for many of these approaches remains the graph convolution idea proposed almost a decade ago. In this paper, we extend this basic component, following an intuition derived from the well-known convolutional…
Functional tensors unify probabilistic programming with automatic differentiation.
problem Designing probabilistic programming systems that can handle diverse inference strategies.
method Introducing functional tensors that capture benefits of tensors and continuous probability distributions.
result Functional tensors enable parallel exact inference for various modeling motifs.
Given a finitely generated and projective Lie-Rinehart algebra, we show that there is a continuous homomorphism of complete commutative Hopf algebroids between the completion of the finite dual of its universal enveloping Hopf algebroid and the associated convolution algebra. The topological Hopf algebroid structure of…
New method estimates and completes tensors from ordinal data, improving accuracy and efficiency.
problem Estimating and completing tensors from incomplete, ordinal observations.
method Multi-linear cumulative link model with rank-constrained M-estimator.
result The proposed estimator achieves faster convergence and is minimax optimal.
Efficiently factorize tensors in streaming data with coreset selection.
problem Efficiently factorize tensors in streaming data.
method Online filtering and kernelization techniques to select a coreset of vectors.
result CP decomposition of coreset approximates full data tensor decomposition.
In this paper, we investigate the statistical convergence rate of a Bayesian low-rank tensor estimator. Our problem setting is the regression problem where a tensor structure underlying the data is estimated. This problem setting occurs in many practical applications, such as collaborative filtering, multi-task learnin…
A substantial progress in development of new and efficient tensor factorization techniques has led to an extensive research of their applicability in recommender systems field. Tensor-based recommender models push the boundaries of traditional collaborative filtering techniques by taking into account a multifaceted nat…
Motivated by the needs of online large-scale recommender systems, we specialize the decoupled extended Kalman filter (DEKF) to factorization models, including factorization machines, matrix and tensor factorization, and illustrate the effectiveness of the approach through numerical experiments on synthetic and on real-…
Kronecker trend filtering improves lattice data smoothing.
problem Estimating smooth functions on lattice data.
method Penalized least squares with Kronecker products of univariate trend filtering penalties.
result Kronecker trend filtering outperforms linear smoothers in high dimensions.
SPIDER uses deep neural networks for streaming tensor factorization.
problem Lack of effective approach for deep tensor factorization of streaming data.
method Bayesian neural networks with spike-and-slab prior, Taylor expansions, moment matching, and EPI framework.
result Effective incremental updates for latent factors and NN weights.
Tensor methods have emerged as a powerful paradigm for consistent learning of many latent variable models such as topic models, independent component analysis and dictionary learning. Model parameters are estimated via CP decomposition of the observed higher order input moments. However, in many domains, additional inv…
Tensors or {\em multi-way arrays} are functions of three or more indices (i,j,k,⋯) -- similar to matrices (two-way arrays), which are functions of two indices (r,c) for (row,column). Tensors have a rich history, stretching over almost a century, and touching upon numerous disciplines; but they have only recent…
Kähler information manifolds for signal filters in weighted Hardy spaces are explored.
problem Developing a geometric framework for signal processing filters in weighted Hardy spaces.
method Introducing weighted Hardy spaces and smooth transformations of transfer functions, demonstrating the Kähler manifold structure.
result The Riemannian geometry of weighted Hardy norms for transfer functions forms a Kähler manifold.
Holistic Filter Pruning reduces DNN complexity efficiently.
problem Redundant parameters in deep neural networks.
method Holistic Filter Pruning (HFP) for efficient DNN training.
result Achieves state-of-the-art performance with 60% reduction in multiplications on ImageNet.
TF-Coder simplifies tensor manipulation programming in TensorFlow.
problem Difficulty in programming with TensorFlow due to steep learning curve.
method Bottom-up weighted enumerative search with value-based pruning and type/value filtering.
result Solves 63 out of 70 real-world tasks within 5 minutes.
Paper describes a state sum formula for a graph coloring polynomial.
problem Counting n-face colorings of ribbon graphs for various n. method Combines topological quantum field theory and diagrammatic tensors.
result Describes a state sum formula for the total face color polynomial.
It is well known that multiplication operations in convolutional layers of common CNNs consume a lot of time during inference stage. In this article we present a flexible method to decrease both computational complexity of convolutional layers in inference as well as amount of space to store them. The method is based o…
New method constructs multilayer networks from financial data, capturing dependencies across different risk factors.
problem Difficult construction of multilayer networks, neglecting time delays and interdependencies.
method Tucker tensor autoregression for direct multilayer network construction.
result Captures within and between connections, identifies strong interconnections between volumes and prices layers.
Bayesian model for cancer drug studies maps dose-response curves.
problem Mapping dose-response curves in cancer drug studies.
method Bayesian Tensor Filtering (BTF) with low-dimensional embeddings and structured shrinkage priors.
result BTF outperforms state-of-the-art methods in cancer drug studies.
We present a new model, Predictive State Recurrent Neural Networks (PSRNNs), for filtering and prediction in dynamical systems. PSRNNs draw on insights from both Recurrent Neural Networks (RNNs) and Predictive State Representations (PSRs), and inherit advantages from both types of models. Like many successful RNN archi…
Compact neural networks improve reinforcement learning performance with less data.
problem Sample inefficiency in deep reinforcement learning.
method Tensor factorization and wavelet scattering for compact representation.
result Achieved 2-10 times fewer total coefficients with comparable performance.
Motivated by large-scale Collaborative-Filtering applications, we present a Non-Commuting Latent Factor (NCLF) tensor-completion approach for modeling three-way arrays, which is diagonal like the standard PARAFAC, but wherein different terms distinguish different kinds of three-way relations of co-clusters, as determin…
This paper focuses on spectral filters on graphs, namely filters defined as elementwise multiplication in the frequency domain of a graph. In many graph signal processing settings, it is important to transfer a filter from one graph to another. One example is in graph convolutional neural networks (ConvNets), where the…
Automatically designs normalization and activation layers together.
problem Designing normalization and activation layers separately.
method Unified tensor-to-tensor computation graph, low-level mathematical functions, rejection protocols, multi-objective evolution.
result Discovery of EvoNorms with novel structures.
A new SOHP filter improves trend estimation in economic time series.
problem Improving trend estimation in nonlinear economic time series.
method Recursive application of one-sided HP filter on updated cyclical components, combined with an incremental HP filtering algorithm.
result Better performance of SOHP filter compared to other HP-type filters on real economic data.
Deep density methods improve filtering in high-dimensional systems.
problem Nonlinear filtering in high-dimensional systems.
method Two deep density methods based on Feynman-Kac formulas and neural networks.
result Logarithmic deep backward stochastic differential equation filter outperforms classical methods in high dimensions.
The sophisticated structure of Convolutional Neural Network (CNN) allows for outstanding performance, but at the cost of intensive computation. As significant redundancies inevitably present in such a structure, many works have been proposed to prune the convolutional filters for computation cost reduction. Although ex…
Gradient filters track moving parameters under noisy data and misspecification.
problem Tracking multidimensional time-varying parameters under noisy observations and model misspecification.
method Gradient-based filters update parameters using the gradient of a postulated objective function, evaluated at either the predicted or updated parameters.
result Novel sufficient conditions for exponential stability of the filtered parameter path, and finite-sample and asymptotic mean squared error bounds.
We simplify Bayesian filtering by framing it as optimization, making it practical for high-dimensional systems.
problem Bayesian filtering struggles in high-dimensional state spaces like neural networks.
method We frame Bayesian filtering as optimization, using gradient descent for nonlinear cases.
result Our method results in effective, robust, and scalable filters for high-dimensional systems.
Efficiently computes sparse signature coefficients using kernels.
problem Lack of efficient methods for sparse signature coefficients.
method Signature kernels and PDE-based methods.
result Sparse groups of signature coefficients can be isolated effectively.
The paper proves curvature identities for symplectic connections.
problem Curvature tensor identities on symplectic connections.
method Invariant theory of the symplectic group, analogous to Riemannian or Kahler geometry.
result Describes the first space of p-covariant curvature identities.
Develops an inverse particle filter for cognitive systems.
problem Tracking cognitive adversaries in counter-adversarial applications.
method Global filtering approach using Monte Carlo methods and differentiable I-PF.
result Demonstrates convergence to optimal inverse filter and improved estimation performance.
Convolutional neural networks (CNNs) achieve state-of-the-art performance in a wide variety of tasks in computer vision. However, interpreting CNNs still remains a challenge. This is mainly due to the large number of parameters in these networks. Here, we investigate the role of compression and particularly pruning fil…
A novel method reduces dimensionality for filtering SRNs with observed variables.
problem Challenges in estimating hidden state variables in SRNs with limited observations.
method Filtered Markovian Projection (Filtered MP) for dimensionality reduction in filtering.
result Filtered MP guarantees consistency and superior computational efficiency in high dimensions.
Kernel learning FBSDE filter improves nonlinear filtering efficiency.
problem Nonlinear filtering problem in high-dimensional systems.
method Iterative and adaptive meshfree approach using forward backward SDE and KDE.
result Rigorous convergence analysis provided, supporting empirical results.
New method filters large networks from financial data to reveal key subnetworks.
problem Filtering large dimensional networks to isolate key constituents.
method Exploits spectral properties of high-dimensional data networks, tuning for sparsity and consistency.
result Shows method can interpolate between zero and maximal filtering, preserving spectral properties.
Many nonlinear extensions of the Kalman filter, e.g., the extended and the unscented Kalman filter, reduce the state densities to Gaussian densities. This approximation gives sufficient results in many cases. However, this filters only estimate states that are correlated with the observation. Therefore, sequential esti…
DCIts interprets complex time series data with interpretable coefficients.
problem Interpreting nonlinear multivariate time series data.
method Deep convolutional architecture with a Focuser and Modeler components.
result DCIts provides interpretable coefficients and interaction patterns.
Paper proves convergence of Kalman filter on Stiefel manifolds with measurement errors.
problem Filtering constant particle with measurement errors on Stiefel manifolds.
method Extended Kalman filter applied to Stiefel manifold-valued observations.
result Convergence of the extended Kalman filter proved for constant system process.
Improved Kalman filter for non-linear, non-Gaussian data.
problem Estimating hidden variables with non-linear, non-Gaussian observations.
method Reproduces and extends Burkhart et al.'s discriminative Kalman filter.
result Enhanced filter performance for complex observation models.
This work analyzes the stability of graph filters under large perturbations.
problem Stability of graph filters under large edge rewires.
method Proves a bound on stability using frequency response and community structure.
result Graph filter stability depends on perturbation to community structure.