Paper transforms deep rectifier networks into shallow ones for analysis.
problem Understanding the complexity of deep neural networks.
method Transformation of deep rectifier networks into shallow ones.
result Shallow networks can represent deep networks with fewer functions.
RDL-Net improves speech enhancement with fewer parameters and better performance.
problem Improving speech enhancement with fewer parameters and better performance.
method Proposes RDL-Net, a CNN combining residual and dense aggregations without over-allocating parameters.
result RDL-Net achieves higher speech enhancement performance with fewer parameters and lower computational requirements.
A new method learns brain dynamics from limited rs-fMRI data.
problem Limited training data for brain connectivity dynamics.
method Residual D-net with recurrent multi-scale representation.
result Significantly higher classification accuracy for MCI vs NC.
Study on preventing early training failure modes in deep ReLU nets.
problem Early training failure modes in deep ReLU nets.
method Proved and avoided two failure modes: exploding/vanishing mean activation length and exponentially large variance of activation length.
result Correct initialization and architecture can prevent early training failure modes in deep ReLU nets.
A major contributing factor to the recent advances in deep neural networks is structural units that let sensory information and gradients to propagate easily. Gating is one such structure that acts as a flow control. Gates are employed in many recent state-of-the-art recurrent models such as LSTM and GRU, and feedforwa…
Deep neural net solves multi-agent optimal trading problem.
problem Optimal trade execution for multiple agents and assets.
method Residual U-net with self-attention for viscosity solution approximation.
result Neural network approach outperforms finite difference methods.
Res-SE-Net boosts Resnet performance by enhancing bridge-connections.
problem Reduced accuracy in Resnet due to lack of feature map contribution from bridge-connections.
method Proposed Res-SE-Net architecture with Squeeze-and-Excitation (SE) block to quantify and weight feature map importance.
result Res-SE-Net generalizes better than Resnet and SE-Resnet on CIFAR-10 and CIFAR-100 datasets.
PSiLON Net uses L1 weight normalization and 1-path-norm regularization for efficient learning and sparsity.
problem Efficient learning and sparsity in neural networks with limited data.
method PSiLON Net employs L1 weight normalization and 1-path-norm regularization to simplify the 1-path-norm and achieve efficient learning and near-sparse parameters. result PSiLON Net achieves reliable optimization and strong performance in the small data regime.
Residual networks improve noise stability, leading to better performance.
problem Degradation problem in training Residual Neural Networks.
method Introduced simplified versions of ResNets and PlnNets, showing equivalence in expressive power.
result Residual networks have better noise stability, leading to improved generalization.
A two-stage CNN approach for brain tumor segmentation.
problem Brain tumor segmentation with class imbalance.
method Cascade V-Net architecture with ROI masks.
result Improved brain tumor segmentation accuracy.
Visualizes classification accuracy and label bias in neural nets and trees.
problem Identifying mislabeled cases in neural net and tree-based classifications.
method Silhouette plots and quasi residual plots of PAC (probability of alternative class).
result Comparison of different classifications using silhouette width and PAC plots.
TradeMech nets trades without changing counterparty relationships.
problem Netting trades without altering counterparty exposure in complex financial networks.
method Transforms contracts into chains and cycles, nets designated object multilaterally, and replaces contracts with new multiparty agreements.
result Maximal multilateral netting of a designated object while preserving each agent's profit and counterparty risk.
Deep learning improves MRI image quality from down-sampled data.
problem Improving MRI image quality from accelerated, down-sampled k-space data.
method Deep Residual Dense U-Net architecture with Residual Dense Block and new loss function.
result The proposed method achieves better performance in reconstructing high-quality images from down-sampled k-space data.
Lat-Net compresses Lattice Boltzmann simulations using neural networks.
problem High computational and memory demands in fluid simulations.
method Convolutional autoencoders and residual connections in a fully differentiable scheme.
result Computational and memory efficient neural network that generalizes to large grid sizes and complex geometries.
Bayesian Neural Nets improve model stability and fit.
problem Improving model stability and fit in time series prediction.
method Assign Bayesian Neural Nets to drift and diffusion terms of SDE, infer posterior using SGLD.
result Significantly improved stability and better model fit on benchmarks.
This paper improves fund net value prediction using ARIMA-LSTM hybrid model.
problem Inaccurate prediction of fund net value due to linear characteristics of models.
method ARIMA-LSTM hybrid model: ARIMA for linear characteristics, LSTM for nonlinear.
result Hybrid model shows higher accuracy in predicting fund net value.
Proposes Π-Nets, polynomial neural networks, for improved representation power.
problem Improving representation power in deep learning models.
method Introduces Π-Nets, a new class of deep polynomial neural networks. result Demonstrates Π-Nets outperform standard DCNNs and achieve state-of-the-art results. Deep learning speeds up whole heart MRI to 30 seconds.
problem Long acquisition times in whole heart MRI.
method Deep learning, specifically a 3D residual U-Net, to reconstruct high-resolution images from low-resolution data.
result Super-resolution images show better edge sharpness and fewer artefacts than low-resolution images.
BiGraphNet generalizes graph neural networks for more efficient operations.
problem Fragmented graph neural network architectures hinder optimization.
method Explicitly separates input and output nodes, enabling new efficient operations.
result BiGraphNet accelerates and scales computations in hierarchical networks.
Simple neural net outperforms complex uncertainty methods.
problem Reliable uncertainty estimation from deterministic models.
method A simple baseline using a single softmax neural net with residual connections and spectral normalization.
result Simple neural net outperforms DUQ and SNGP on uncertainty prediction.
Paper presents a new method for efficient deep learning with over-complete dictionaries.
problem Learning an over-complete basis for optimal reconstruction without optimization.
method Multiscale Residual Mixture of PCA with a hierarchical approach.
result Exponential decrease of error with depth in a recursive deep approach.
CounteRGAN generates realistic, actionable counterfactuals for machine learning models.
problem Creating realistic and actionable counterfactuals for machine learning models.
method Applying Residual GANs to improve counterfactual realism and actionability.
result CounteRGAN produces counterfactuals with improved realism and actionability, achieving real-time applicability.
PDE-Net learns PDEs from data using neural networks.
problem Learning PDEs from complex system dynamics.
method Proposes PDE-Net, a feed-forward deep network to learn differential operators and nonlinear responses.
result PDE-Net can accurately predict dynamics and uncover hidden PDE models.
Proposes polynomial neural networks for improved function approximation in various tasks.
problem Improving function approximation in various tasks like image generation, face verification, and 3D mesh representation learning.
method Introduces polynomial neural networks (Π-Nets) and three tensor decompositions to reduce parameter count and enhance expressiveness. result Demonstrates that Π-Nets can produce state-of-the-art results in challenging tasks without non-linear activation functions. GAMI-Net improves neural network interpretability while maintaining accuracy.
problem Lack of interpretability in neural network models.
method GAMI-Net is a disentangled feedforward network with multiple additive subnetworks designed for capturing main effects and pairwise interactions, considering sparsity, heredity, and marginal clarity.
result GAMI-Net achieves superior interpretability and competitive prediction accuracy compared to explainable boosting machine and other models.
Gradient descent converges to global minima for ResNets with linearly scaled width.
problem Understanding the convergence of deep residual networks with varying network width and dataset size.
method Analyzing the Jacobian of ResNets and applying gradient descent for quadratic loss.
result Gradient descent converges to global minima for ResNets with linearly scaled width and independent of depth.
Proposes using MR images to create synthetic CT images for prostate segmentation.
problem Creating high-quality annotations for prostate segmentation in CT scans.
method CycleGAN algorithm to create synthetic CT images from MR images, using a 2.5D Residual U-Net for segmentation.
result Automatic delineation of prostate from real CT scans achieved with comparable results to radiologist annotations.
Wide hidden layer TCM nets capacity analyzed using RDT and fl RDT.
problem Capacity analysis of wide hidden layer TCM nets.
method Employed Fully Lifted Random Duality Theory (fl RDT) for capacity characterization.
result Explicit, closed form capacity characterizations for a generic class of hidden layer activations.
New framework analyzes deep neural networks using feature probabilities.
problem Degenerate situation in over-parameterized DNNs.
method Mean-field framework representing DNNs by feature probabilities and functions.
result Global convergence proof for over-parameterized Res-Net training.
Deep neural nets classify sleep stages from raw PSG data with high accuracy.
problem Automatically classifying sleep stages from raw polysomnogram signals.
method Deep residual neural networks trained on 50 convolutional layers.
result Best model achieved 84.1% accuracy and 0.746 Cohen's kappa.
Neural surrogate predicts SPN rates from token trajectories.
problem Challenging parameter estimation in SPNs with covariates.
method 1D Convolutional Residual Network trained on Gillespie-simulated SPN realizations.
result Surrogate predicts rate-function coefficients with RMSE = 0.043.
Paper studies M-estimators with derivatives and residual distribution for robust adaptive tuning.
problem Tackles robustness and adaptive tuning of M-estimators with heavy-tailed noise.
method Provides formulae for derivatives, characterizes residual distribution, proposes adaptive criterion.
result Characterizes distribution of residuals and proposes adaptive criterion as out-of-sample error proxy.
ResNets can be as good as linear predictors under certain conditions.
problem Prove that ResNets can achieve the same performance as linear predictors.
method Rigorous mathematical proof of ResNets' optimization landscape properties.
result ResNets can achieve the same performance as linear predictors without local minima issues.
Machine learning helps estimate risk premiums of stocks without knowing their factors.
problem Estimate risk premiums of stocks without knowing their underlying factors.
method Used elastic-net machine learning to project stock returns onto peers and construct replicate portfolios.
result Unique stocks have higher SARP and excess returns than ubiquitous stocks.
The purpose of this paper is introducing rigorous methods and formulas for bilateral counterparty risk credit valuation adjustments (CVA's) on interest-rate portfolios. In doing so, we summarize the general arbitrage-free valuation framework for counterparty risk adjustments in presence of bilateral default risk, as de…
Complexity measures for neural nets with general activations using path-based norms.
problem Control complexity of neural networks with arbitrary activation functions.
method Approximate general activations with ReLU networks and derive path-based norms for complexity control.
result Preliminary analyses of function spaces and regularized estimators.
Deep neural net improves sleep stage classification from raw EEG.
problem Improving accuracy of sleep stage classification from raw EEG data.
method 34-layer deep residual ConvNet architecture trained on raw single channel EEG data.
result Proposed network outperforms state-of-the-art methods in sleep staging accuracy.
Deep learning model predicts subsurface flow dynamics.
problem Predicting dynamic subsurface flow in channelized geological systems.
method Residual U-Net and Convolutional LSTM networks trained on pressure and saturation maps.
result Surrogate model accurately predicts pressure, saturation, and well rates for new realizations.
Transform classical network structures to graph CNN for better graph recognition.
problem Transforming classical network structures to graph CNN for better graph recognition.
method Review and introduce ResNet, Inception, and DenseNet into graph CNN, constructing G_ResNet, G_Inception, G_DenseNet.
result Demonstrated how different network structures work on graph CNN in the graph recognition task.
New robust estimator improves variable selection and coefficient estimation in linear regression with heavy-tailed errors and outliers.
problem Heavy-tailed errors and anomalous predictors in high-dimensional regression.
method Adaptive PENSE estimator for robust variable selection and estimation.
result Adaptive PENSE estimator provides reliable results even under very heavy-tailed errors and aberrant predictors.
ACNN improves medical image segmentation with full resolution and higher IoUs.
problem Reduced spatial resolution in current DCNNs hinders medical image segmentation.
method Proposes ACNN using atrous convolution, cascaded atrous II-blocks, residual learning, and Instance Normalization.
result ACNN achieves higher IoUs than U-Net and Deeplabv3+ with fewer parameters.
A novel approach for 3D lung nodule segmentation using adaptive ROI and multi-view residual learning.
problem Inaccurate nodule segmentation due to fixed ROI and redundant structures.
method Two-stage approach: 2D ROI patch-wise investigation with adaptive ROI strategy, followed by 2D and 3D VOI investigation with deep residual U-Net.
result Significantly robust and accurate nodule segmentation compared to previous methods.
Adaptive probabilistic load forecasting improves performance in power systems.
problem Complexity of electricity load forecasting due to changing drivers and local generation.
method Adaptive probabilistic approach using Kalman filter and online gradient descent.
result Adaptive probabilistic forecasts improve performance in both point and probabilistic forecasting.
Improved sparse-view CT images with deep learning sinogram interpolation.
problem Sparse-view CT images quality improvement with limited projection data.
method Combination of U-Net and residual learning for sinogram interpolation.
result Significantly improved CT image quality (RMSE and SSIM metrics) over standard methods.
Characterizes discrete nets with characteristic properties on larger parameter rectangles.
problem Classify discrete and smooth surfaces with specific properties.
method Imposes characteristic properties on larger parameter rectangles for nets.
result Characterizes discrete multi-nets leading to classical smooth surfaces.
The paper explores discrete isothermic nets using checkerboard patterns in quadrilateral nets.
problem Defining and understanding discrete isothermic nets in quadrilateral nets.
method Using checkerboard patterns and discrete differential geometry to define and analyze isothermic nets.
result The class of isothermic nets is invariant under dualization and Moebius transformations.
The recent financial crisis has stressed the need to understand financial systems as networks of interdependent countries, where cross-border financial linkages play the fundamental role. It has also been emphasized that the relevance of these networks relies on the representation of changes follow-on the occurrence of…
Deep nets outperform shallow nets in complex feature realization.
problem Realizing complex data features with deep nets.
method Refined covering number estimates and analysis of approximation rates.
result Deep nets can improve performance without additional capacity costs for complex features.