DCN-V2 improves deep & cross network for web-scale learning to rank systems.
problem Efficiently learning feature interactions in large-scale recommender systems.
method Proposes DCN-V2, an improved framework for deep & cross network learning.
result DCN-V2 outperforms state-of-the-art algorithms on benchmark datasets.
Many studies have been undertaken by using machine learning techniques, including neural networks, to predict stock returns. Recently, a method known as deep learning, which achieves high performance mainly in image recognition and speech recognition, has attracted attention in the machine learning field. This paper im…
Proposes a method to solve deep neural networks' local minimum problem.
problem Local minimum problem in deep neural networks training.
method Transforms cross-entropy loss into risk-averse error criterion, adjusts RSI, and uses convexity region.
result Trained deep learning machine is expected to be inside a global minimum's attraction basin.
Introduces NQ network for non-crossing quantile learning.
problem Quantile crossing issue in distributional learning.
method Non-negative activation functions ensure monotonic distributions.
result Effective for distributional reinforcement learning and causal effect estimation.
As a highlighting research topic in the multimedia area, cross-media retrieval aims to capture the complex correlations among multiple media types. Learning better shared representation and distance metric for multimedia data is important to boost the cross-media retrieval. Motivated by the strong ability of deep neura…
Deep neural networks enforce non-crossing quantile regression curves.
problem Estimating quantile regression curves without crossing.
method Penalized deep ReQU neural networks with a non-crossing penalty.
result Established non-asymptotic risk and error bounds for the estimated QRP.
In deep neural network, the cross-entropy loss function is commonly used for classification. Minimizing cross-entropy is equivalent to maximizing likelihood under assumptions of uniform feature and class distributions. It belongs to generative training criteria which does not directly discriminate correct class from co…
Proposes a non-crossing deep neural network quantile regression method.
problem Quantile crossing in nonparametric quantile regression.
method Non-crossing constraints via rectified linear unit penalty function.
result Established non-asymptotic upper bounds for excess risk.
We identify a class of over-parameterized deep neural networks with standard activation functions and cross-entropy loss which provably have no bad local valley, in the sense that from any point in parameter space there exists a continuous path on which the cross-entropy loss is non-increasing and gets arbitrarily clos…
Hopfield networks outperform deep-learning methods in portfolio optimization.
problem Optimizing portfolios and managing asset allocation efficiently.
method Application of Hopfield networks to portfolio optimization, using combinatorial purged cross-validation.
result Modern Hopfield Networks perform on par or better than deep-learning methods, with faster training times and better stability.
Feature engineering has been the key to the success of many prediction models. However, the process is non-trivial and often requires manual feature engineering or exhaustive searching. DNNs are able to automatically learn feature interactions; however, they generate all the interactions implicitly, and are not necessa…
This paper shows using classification instead of regression improves deep RL scalability.
problem Challenges in training value functions for large networks in deep RL.
method Used categorical cross-entropy loss instead of mean squared error regression.
result Significant improvements in performance and scalability across various domains.
Learning social media data embedding by deep models has attracted extensive research interest as well as boomed a lot of applications, such as link prediction, classification, and cross-modal search. However, for social images which contain both link information and multimodal contents (e.g., text description, and visu…
Cross-Domain Collaborative Filtering (CDCF) provides a way to alleviate data sparsity and cold-start problems present in recommendation systems by exploiting the knowledge from related domains. Existing CDCF models are either based on matrix factorization or deep neural networks. Either of the techniques in isolation m…
Geometry-aware models improve cross-subject EEG decoding accuracy.
problem Strong inter-subject variability in motor imagery decoding.
method Discriminative Congruence Transform (DCT), Deep Linear DCT (DLDCT), Deep DCT-UNet (DDCT-UNet).
result Improves transductive cross-subject accuracy by 2-3%.
Proposes a deep learning model for timely and accurate recommendations.
problem Inability to provide timely recommendations and ranking issues with implicit feedback.
method Unified cross-network solution using listwise ranking for implicit data.
result Superior performance in accuracy, novelty, and diversity compared to baselines.
We introduce deep scale-spaces (DSS), a generalization of convolutional neural networks, exploiting the scale symmetry structure of conventional image recognition tasks. Put plainly, the class of an image is invariant to the scale at which it is viewed. We construct scale equivariant cross-correlations based on a princ…
We propose a max-pooling based loss function for training Long Short-Term Memory (LSTM) networks for small-footprint keyword spotting (KWS), with low CPU, memory, and latency requirements. The max-pooling loss training can be further guided by initializing with a cross-entropy loss trained network. A posterior smoothin…
As the success of deep learning reaches more grounds, one would like to also envision the potential limits of deep learning. This paper gives a first set of results proving that certain deep learning algorithms fail at learning certain efficiently learnable functions. The results put forward a notion of cross-predictab…
The cross-domain recommendation technique is an effective way of alleviating the data sparse issue in recommender systems by leveraging the knowledge from relevant domains. Transfer learning is a class of algorithms underlying these techniques. In this paper, we propose a novel transfer learning approach for cross-doma…
We introduce the HSIC (Hilbert-Schmidt independence criterion) bottleneck for training deep neural networks. The HSIC bottleneck is an alternative to the conventional cross-entropy loss and backpropagation that has a number of distinct advantages. It mitigates exploding and vanishing gradients, resulting in the ability…
Adam achieves optimal convergence in deep ReLU networks via novel Kakeya bounds.
problem Training deep ReLU networks using Adam in non-smooth settings.
method Stratified Morse theory and Kakeya bounds to analyze region crossings and convergence.
result First global-optimal convergence for Adam in non-smooth, non-convex ReLU landscapes.
Predict stock movement by considering cross effects among stocks.
problem Challenges in predicting stock price movement due to cross effects among stocks.
method Multi-GCGRU framework combining GCN and GRU, encoding cross effects from financial domain knowledge and data-driven relationships.
result Our model outperforms other baselines in predicting stock movement.
This paper investigates the impact of normalization on deep neural networks for click-through rate prediction.
problem The effect of normalization on deep neural network models for CTR estimation.
method Systematic study of various normalization approaches applied to feature embedding and MLP part of DNN models.
result Correct normalization significantly enhances model performance, as demonstrated by extensive experiments on real-world datasets.
Few-shot network compression improves accuracy with minimal data.
problem High estimation errors from original network during inference.
method Cross distillation, layer-wise knowledge distillation approach.
result Cross distillation significantly improves student network's accuracy with few training instances.
This paper improves image super-resolution by integrating cross-scale non-local attention.
problem Improving image super-resolution by leveraging long-range and cross-scale feature correlations.
method Proposes a Cross-Scale Non-Local (CS-NL) attention module integrated into a recurrent neural network.
result Significantly improved performance on SISR benchmarks.
Pipeline integrates cross-sectional and longitudinal multi-omics data for IBD research.
problem Integrating diverse data types from the same individuals for disease understanding.
method Statistical and deep learning methods for variable selection, feature extraction, and joint integration.
result Identified microbial pathways, metabolites, and genes discriminating IBD status.
This paper concerns automated vehicles negotiating with other vehicles, typically human driven, in crossings with the goal to find a decision algorithm by learning typical behaviors of other vehicles. The vehicle observes distance and speed of vehicles on the intersecting road and use a policy that adapts its speed alo…
Meta learning enables cross-domain Hamiltonian dynamics.
problem Adapting to new physical systems with different laws.
method Graph Neural Network (GNN) with meta learning.
result Unified Hamiltonian representation across multiple system domains.
Researchers propose better probabilistic models for deep learning.
problem Using cross-entropy loss for non-categorical data.
method Introducing continuous-categorical distribution and proposing probabilistic alternatives.
result Potential for outperformance in deep learning models with proper probabilistic treatment.
Given two networks with the same training loss on a dataset, when would they have drastically different test losses and errors? Better understanding of this question of generalization may improve practical applications of deep networks. In this paper we show that with cross-entropy loss it is surprisingly simple to ind…
Researchers find a class/cross-class structure in deep learning spectra.
problem Understanding the structure of deep learning classifiers.
method Empirical spectral analysis of deepnet classifiers.
result The cross-class structure explains various features in deepnet spectra.
XMixup improves transfer learning accuracy by 1.9% with less training time.
problem Efficiently transfer knowledge from large source datasets to target tasks with small samples.
method Cross-domain Mixup technique that selects auxiliary samples from source datasets and augments training samples via mixup strategy.
result Improves accuracy by 1.9% on average over six real-world transfer learning datasets.
DMT enhances deep neural networks to better preserve data structures.
problem Preserving geometric, topological, and distributional structures of data in NLDR.
method Deep manifold transformation (DMT) using cross-layer LGP constraints.
result DMT networks outperform existing NLDR methods in preserving data structures.
In recent years, there have been numerous developments towards solving multimodal tasks, aiming to learn a stronger representation than through a single modality. Certain aspects of the data can be particularly useful in this case - for example, correlations in the space or time domain across modalities - but should be…
Deep CNNs diagnose chest X-rays for COVID-19 and other pneumonia.
problem Diagnosing pneumonia from chest X-rays.
method Training and testing VGG16, VGG19, InceptionResNetV2, InceptionV3, and Xception networks on chest X-rays.
result VGG16 with dropout achieves high accuracy and sensitivity.
DHEN improves CVR prediction for ads with multitask learning and auxiliary loss.
problem Predicting conversion rates in ad-recommendation systems.
method DHEN integrates multiple feature-crossing modules and uses a multitask learning framework, ablation studies, and self-supervised auxiliary loss.
result DHEN achieves state-of-the-art performance in CVR prediction.
In this paper, we propose a novel structure for a cross-modal data association, which is inspired by the recent research on the associative learning structure of the brain. We formulate the cross-modal association in Bayesian inference framework realized by a deep neural network with multiple variational auto-encoders …
Survey of multimodal deep generative models for diverse data types.
problem Inference of shared representations and cross-modal generation from heterogeneous multimodal data.
method Variational autoencoders and other deep generative models.
result A comprehensive survey of multimodal deep generative models.
FuncNN package enables deep learning with functional covariates.
problem Lack of software for deep learning with functional covariates.
method Developed an R package using keras architecture, introducing functions for model building, predictions, and cross-validation.
result First package for deep learning with functional covariates.
Deep learning detects APT attacks with high accuracy and low false positives.
problem Detecting Advanced Persistent Threat (APT) attacks in network traffic.
method Used C5.0 decision tree, Bayesian network, and 6-layer deep neural network models on NSL-KDD dataset.
result Deep learning model achieved 98.85% accuracy and 1.13 false positive rate.
This paper uses deep learning to detect money laundering in cross-border transactions.
problem Detecting money laundering in cross-border transactions is challenging due to complexity and scale.
method Unsupervised learning models, including CNNs and hybrid CNNGRU architectures, were tested for anomaly detection.
result Hybrid Convolutional-Recurrent Neural Integration Model (CRNIM) showed superior performance.
Optimizes tensor rank selection for neural network compression.
problem Finding optimal tensor rank for regression models.
method Analyzes population expressions for training-testing discrepancy under Gaussian design.
result Optimal rank minimizes prediction error and aligns with cross-validation.
Deep learning predicts drug prescriptions across global health records.
problem Predicting drug prescriptions in chronic disease patients.
method Adaptive cross-global attention graph kernel network with support vector machine.
result Model outperforms current methods in accuracy and interpretability.
MetFA aligns source and target domains for cross-device image classification.
problem Learning discriminative class boundaries across different domains.
method Distance metric guided feature alignment (MetFA) for domain-invariant and discriminative feature extraction.
result MetFA outperforms state-of-the-art methods in cross-device image classification.
EIDGM model estimates DE parameters from RCS data.
problem Estimating DE parameters from RCS data with heterogeneities.
method Physics-informed neural network emulator + Wasserstein GAN parameter generator.
result EIDGM accurately captures diverse parameter distributions.
CAggNet improves medical image segmentation by fusing coarse and fine features.
problem Medical image segmentation accuracy and efficiency.
method Crossing Aggregation Network with nested skip connections and weighted aggregation.
result CAggNet achieves more accurate and efficient segmentation compared to existing methods.
DNN2LR bridges DNN power and LR interpretability.
problem Combining DNN power and LR interpretability for real-world tabular data.
method Automatic feature crossing method based on DNN interpretation inconsistencies.
result DNN2LR outperforms complex DNN models and feature crossing methods.