Stock networks, constructed from stock price time series, are a well-established tool for the characterization of complex behavior in stock markets. Following Mantegna's seminal paper, the linear Pearson's correlation coefficient between pairs of stocks has been the usual way to determine network edges. Recently, possi…
We present a probabilistic framework for nonlinearities, based on doubly truncated Gaussian distributions. By setting the truncation points appropriately, we are able to generate various types of nonlinearities within a unified framework, including sigmoid, tanh and ReLU, the most commonly used nonlinearities in neural…
This paper addresses the following question of neural network identifiability: Does the input-output map realized by a feed-forward neural network with respect to a given nonlinearity uniquely specify the network architecture, weights, and biases? Existing literature on the subject Sussman 1992, Albertini, Sontag et al…
Study on identifying and inferring nonlinear dynamics on unknown networks.
problem Identifying network structure in nonlinear dynamic systems with unknown interactions.
method Showed network structure is not generically identified, requiring sufficient spectral heterogeneity. Developed necessary and sufficient conditions for identification and proposed a semiparametric estimator.
result Necessary and sufficient conditions for identification of network structure in nonlinear dynamic systems.
Nonlinear MCMC improves Bayesian machine learning sampling.
problem Sampling problems in Bayesian machine learning.
method Nonlinear MCMC technique with convergence guarantees.
result Improves sampling in Bayesian neural networks.
Study adapts β-TCVAE for fMRI to recover nonlinear brain components.
problem Capturing nonlinear brain dynamics in fMRI data.
method Adapted β-TCVAE framework for fMRI data. result Recovery of meaningful nonlinear spatial components in fMRI data.
New findings on neural network identifiability using affine symmetries.
problem Identifying all neural networks that produce a given function.
method Examined affine symmetries of nonlinearities and their impact on neural network identifiability.
result Symmetries can be used to find a rich set of networks giving rise to the same function, except for a special case.
Generalizes memory and forecasting capacities for nonlinear recurrent networks with dependent inputs.
problem Understanding memory and forecasting capabilities in networks with dependent inputs.
method Formulated bounds for memory and forecasting capacities in terms of network size and input properties.
result Proved that memory capacity for linear recurrent networks with independent inputs is given by the rank of the controllability matrix.
Shallow nonlinear networks can separate classes linearly with polynomially scaling width.
problem Understanding the linear separability of deep networks' features.
method Modeling inputs as a union of low-dimensional subspaces and using random weights and quadratic activations.
result Shallow nonlinear networks can achieve linear separation with polynomially scaling width.
We introduce nonlinear higher-order label spreading for semi-supervised learning.
problem Efficient semi-supervised learning on graphs with complex label spreading.
method We add nonlinearity to label spreading through higher-order graph structures, proving convergence and demonstrating efficiency on various datasets.
result Our nonlinear higher-order label spreading algorithm converges to the global solution and performs favorably compared to classical methods.
Finslerian graph neural networks recover nonlinear diffusion geometry
problem Graph neural networks on point clouds
method Estimates of the Finsler Laplacian
result Recovery of Finsler geometry
Neural networks learn task-specific features, influenced by nonlinearity.
problem Understanding the nature of task-dependent feature learning in neural networks.
method Investigation of fully-connected, wide neural networks using Bayesian framework.
result The nature of internal representations depends on neuronal nonlinearity, leading to analog, redundant, or sparse coding schemes.
NNNH uses neural networks to model complex event patterns.
problem Analyzing multi-dimensional nonlinear Hawkes processes with mutual excitation and inhibition.
method NNNH employs feedforward neural networks to model individual kernels and base intensity, optimizing parameters via Stochastic Gradient Descent.
result NNNH accurately captures complexities of nonlinear Hawkes processes, as demonstrated by numerical experiments.
Over the last decade, both the neural network and kernel adaptive filter have successfully been used for nonlinear signal processing. However, they suffer from high computational cost caused by their complex/growing network structures. In this paper, we propose two random Euler filters for complex-valued nonlinear filt…
Enhances RSCNs with hybrid regularization for nonlinear dynamics.
problem Modeling nonlinear dynamic systems with uncertainties.
method Recurrent stochastic configuration networks with hybrid regularization.
result The method outperforms other models in nonlinear system identification and industrial tasks.
TRNN combines tensor geometry with neural network nonlinearity for HD data.
problem Modeling high-dimensional data with preserved tensor geometry and nonlinear interactions.
method Introduces TRNN that integrates tensor geometry and neural network nonlinearity.
result TRNN preserves tensor geometry while offering nonlinearity.
The study sets limits on how well nonlinear models can generalize from training data.
problem Understanding the limits of generalization for nonlinear learning models.
method Deriving explicit generalization lower bounds for multi-layer neural networks and linear regression.
result Explicit bounds for general biased estimators in nonlinear networks, showing unacceptable performance for unbiased estimators.
Theory explains deep nonlinear networks' plateaus and transitions.
problem Understanding long plateaus and feature acquisition transitions in deep nonlinear networks.
method Derived an exact identity for Frobenius norms, classified activation functions, and reduced matrix flow to a scalar ODE.
result Escape time law τ⋆=Θ(ε−(r−2)) for deep nonlinear networks, where r is the number of bottleneck layers. Three-layer networks learn complex hierarchical polynomials of multiple nonlinear features.
problem Understanding how neural networks learn hierarchical features of multiple nonlinear inputs.
method Examine a broad class of functions using three-layer neural networks, showing complete recovery and efficient learning.
result Three-layer neural networks trained via gradient descent can learn hierarchical polynomials of multiple nonlinear features efficiently.
Nonlinear RNNs' memory capacity varies widely, making it impractical.
problem The usefulness of memory capacity as a metric for linear RNNs is questioned.
method Analysis of random nonlinear RNNs with varying input scales.
result Memory capacity of nonlinear RNNs is arbitrary and impractical.
Study approximates nonlinear functionals using deep ReLU networks.
problem Approximating nonlinear continuous functionals with neural networks.
method Constructs continuous piecewise linear interpolation under simple triangulation, analyzes rates of approximation.
result Established rates of approximation for functional deep ReLU networks.
Neural networks simplify uncertainty quantification of locally nonlinear systems.
problem Estimating statistics of responses in large-scale locally nonlinear dynamical systems.
method Decomposes response into nominal linear system and a neural network-estimated pseudoforce.
result Neural networks can efficiently estimate pseudoforce containing nonlinear and uncertain information.
Deep network improves electrical tomography across multiple frequencies.
problem Nonlinear multi-frequency electrical impedance tomography (mfEIT) for tissue conductivity estimation.
method Integrates graph neural networks (GNNs) into the iterative Proximal Regularized Gauss Newton (PRGN) framework to reconstruct tissue concentrations accurately.
result Accurate reconstruction of overlapping tissue fraction concentrations across multiple frequencies.
Proposes neural dynamic mode decomposition for end-to-end modeling of nonlinear dynamics.
problem Understanding and modeling nonlinear dynamical systems.
method Trains neural networks to minimize forecast error based on spectral decomposition in the lifted space.
result Demonstrates effectiveness in eigenvalue estimation and forecast performance.
Unified neural network for linear and nonlinear dimension reduction.
problem Efficiently perform linear and nonlinear sufficient dimension reduction.
method Belted and Ensembled Neural Network (BENN) framework.
result Unified framework for both linear and nonlinear dimension reduction.
Despite the widespread practical success of deep learning methods, our theoretical understanding of the dynamics of learning in deep neural networks remains quite sparse. We attempt to bridge the gap between the theory and practice of deep learning by systematically analyzing learning dynamics for the restricted case o…
A new deep neural network tackles nonlinear functional regression with improved dimensionality reduction.
problem Nonlinear functional regression in infinite-dimensional functional data analysis.
method Functional deep neural network with adaptive kernel embedding and projection steps.
result Explicit rates of approximating nonlinear smooth functionals are derived, and the network is shown to be effective in both simulated and real datasets.
Conventional research attributes the improvements of generalization ability of deep neural networks either to powerful optimizers or the new network design. Different from them, in this paper, we aim to link the generalization ability of a deep network to optimizing a new objective function. To this end, we propose a \…
ResGCN detects anomalies in attributed networks by capturing sparsity and nonlinearity.
problem Detecting anomalous nodes in attributed networks.
method Attention-based deep residual modeling using Graph Convolutional Networks.
result ResGCN effectively detects anomalies in attributed networks.
This note introduces a regression technique for finding a class of nonlinear integro-differential operators from data. The method parametrizes the spatial operator with neural networks and Fourier transforms such that it can fit a class of nonlinear operators without needing a library of a priori selected operators. We…
Framework for designing nonlinearities in neural networks with slope constraints.
problem Designing nonlinearities with specific properties for signal processing.
method Variational framework with regularization for slope constraints and optimization of adaptive splines.
result Adaptive nonuniform linear splines achieve global optimum in constrained optimization.
New nonlinear saliency maps improve deep neural network interpretability.
problem Lack of understanding why and how deep neural networks make decisions.
method Developed novel nonlinear saliency maps to better interpret deep neural networks.
result Nonlinear saliency maps provide more specific drivers of classification on complex examples.
DiffSlack learns neural networks with nonlinear constraints via learnable slack variables.
problem Enforcing nonlinear inequality constraints in neural networks.
method DiffSlack reformulates inequalities as equalities with learnable slack variables, predicting them as part of the network output.
result DiffSlack achieves higher planning success rates and stronger geometric constraint satisfaction compared to existing methods.
The paper explores how the depth of neural networks affects their ability to represent data accurately.
problem Understanding the implicit bias and rank of neural networks with large depth.
method Analyzing the convergence of representation cost to a notion of rank as network depth increases, and investigating conditions for recovering the true rank of data.
result There is a range of network depths where the true rank of data is recovered, and this affects the topology of class boundaries.
Our objective is to estimate the unknown compositional input from its output response through an unknown system after estimating the inverse of the original system with a training set. The proposed methods using artificial neural networks (ANNs) can compete with the optimal bounds for linear systems, where convex optim…
Unified framework identifies nonlinear systems using characteristic curves and neural networks.
problem Balancing interpretability and flexibility in nonlinear system identification.
method Combines differential equation structure with neural networks, using characteristic curves as modular components.
result NN-CC approach outperforms other methods in complex nonlinear systems.
DeepBayes uses neural networks to efficiently estimate parameters in complex dynamical models.
problem Estimating parameters in stochastic, nonlinear dynamical models is challenging.
method DeepBayes leverages deep recurrent neural networks to learn an estimator that minimizes mean-squared error.
result DeepBayes achieves asymptotically equivalent performance to Bayesian estimation methods.
Kernel methods are powerful tools to capture nonlinear patterns behind data. They implicitly learn high (even infinite) dimensional nonlinear features in the Reproducing Kernel Hilbert Space (RKHS) while making the computation tractable by leveraging the kernel trick. Classic kernel methods learn a single layer of nonl…
Paper develops a method to learn causal networks with non-invertible functions.
problem Identifying causal relationships from observational data with non-invertible functional relationships.
method Proposes a test for non-invertible bivariate causal models and develops a method to incorporate this test in structure learning of DAGs.
result Our algorithms outperform existing DAG learning methods in identifying causal graphical structures.
Improved stability analysis of neural network systems using Zames-Falb multipliers.
problem Analyzing stability of linear systems with neural network nonlinearities.
method Using integral quadratic constraints, sector-bounded and slope-restricted structure, and acausal Zames-Falb multipliers.
result Flexible and versatile framework for stability analysis with improved computational efficiency.
DeepRSCN models nonlinear systems using stochastic configurations.
problem Modeling nonlinear dynamic systems efficiently.
method Incrementally constructed deep reservoir computing framework with random parameters and online weight updates.
result DeepRSCN outperforms single-layer networks in efficiency, learning, and generalization.
New algorithm learns nonlinear phenomena from noisy local measurements without data exchange.
problem Learning nonlinear phenomena from noisy local measurements in a decentralized network.
method Non-parametric learning algorithm that spreads information only between neighboring nodes.
result Non-asymptotic estimation error bounds for the proposed method.
LSTM neural networks improve fiber nonlinearities in coherent systems.
problem Compensating fiber nonlinearities in digital coherent systems.
method Utilization of Long short-term memory (LSTM) neural networks.
result LSTM neural networks provide superior performance compared to digital back propagation, especially in multi-channel scenarios.
New neural architectures with multivariate nonlinearities are optimal in function space.
problem Optimality of neural architectures with multivariate nonlinearities.
method Construction of Banach spaces via k-plane transform and sparsity-promoting norm, proving representer theorem. result Neural architectures with multivariate nonlinearities are optimal in function space.
Multiview analysis aims at extracting shared latent components from data samples that are acquired in different domains, e.g., image, text, and audio. Classic multiview analysis, e.g., canonical correlation analysis (CCA), tackles this problem via matching the linearly transformed views in a certain latent domain. More…
Bayesian neural networks improve macroeconomic forecasting and model nonlinearities.
problem Handling small T, big K macroeconomic datasets with temporal dependence.
method Developed Bayesian neural networks with mixture activation functions, shrinkage priors, and stochastic volatility.
result BNNs produce precise density forecasts, often better than other methods.
Deep SSMs use neural networks to identify complex systems.
problem Identifying nonlinear systems with high uncertainty.
method Deep state space models with neural networks.
result Deep SSMs outperform traditional methods on benchmarks.
A neural network and evolutionary algorithm framework designs nonlinear optical molecules.
problem Designing efficient nonlinear optical materials.
method Multi-stage Bayesian neural network (msBNN) and corrected Lewis-mode group contribution method (cLGC) combined with evolutionary algorithm (EA).
result Accurately and efficiently designs molecules with different optical properties using a small data set.