Memory-efficient learning for large-scale imaging systems.
problem Memory limitations in GPUs for real-world large-scale inverse problems.
method Exploits reversibility of network layers to enable data-driven design.
result Demonstrated on small-scale and large-scale real-world systems.
DistPre predicts traffic speeds efficiently for large networks.
problem Fine-grained, accurate speed prediction for large-scale transportation networks.
method Customizes LSTM models on a cluster, sharing trained models between detectors.
result Efficient and accurate fine-grained traffic-speed prediction.
Method analyzes large-scale network data to detect communication pattern shifts.
problem Analyzing large-scale time-series network data is challenging.
method Temporal encoder embedding method using ground-truth or estimated vertex labels.
result Detects communication pattern shifts across all levels of network structure.
Randomized spectral co-clustering speeds up large-scale directed networks.
problem Co-clustering directed networks efficiently for large-scale data.
method Randomized spectral co-clustering algorithms using random-projection and random-sampling techniques.
result Theoretical and numerical validation of approximation and misclustering error rates.
PALMS reconstructs large-scale networks efficiently with parallel computing.
problem Reconstructing large-scale latent networks from observed dynamics is computationally challenging.
method PALMS (Parallel Adaptive Lasso with Multi-directional Signals) framework for distributed network reconstruction.
result PALMS substantially reduces computational complexity and storage requirements.
The motivation for this paper is to apply Bayesian structure learning using Model Averaging in large-scale networks. Currently, Bayesian model averaging algorithm is applicable to networks with only tens of variables, restrained by its super-exponential complexity. We present a novel framework, called LSBN(Large-Scale …
New method infers causal factors from large-scale data without full graph reconstruction.
problem Inferring causal variables from large-scale systems without full causal graph reconstruction.
method Supervised learning on simulated data using a neural network and subsampled-ensemble inference.
result Efficiently identifies causal relationships in large-scale gene regulatory networks.
New neural network method simplifies high-dimensional data.
problem Scalability issues in nonlinear sufficient dimension reduction.
method Stochastic neural network with adaptive gradient algorithm.
result Proposed method outperforms existing methods on large-scale data.
BayesDLL offers a PyTorch library for Bayesian deep learning with large models.
problem Bayesian inference for large-scale deep networks.
method Variational inference, MC-dropout, stochastic-gradient MCMC, Laplace approximation.
result BayesDLL can handle Vision Transformers and pre-trained model weights as priors.
Graph neural networks improve network localization accuracy and efficiency.
problem Network localization in large-scale networks.
method Adopted graph neural networks for nonlinear regression.
result GNN outperforms state-of-the-art benchmarks in network localization.
Gradient-free deep learning for large datasets.
problem Training deep neural networks on large-scale datasets is resource-intensive and requires specialized techniques.
method Recursive Local Representation Alignment (RLRA) for gradient-free training.
result RLRA achieves comparable performance to backprop while converging faster and being parallelizable.
Study evaluates machine learning methods for large-scale network reliability, revealing ANN's and PR's performance.
problem Tackles the NP-hard problem of approximating binary-state network reliability for large-scale systems.
method Compares 20 machine learning methods across three reliability regimes and evaluates their performance on large-scale networks.
result Large-scale networks with arc reliability ≥ 0.9 exhibit near-unity system reliability, enabling computational simplifications.
Hybrid approach for large-scale network synchronization using KF and PTP.
problem Synchronization of large-scale networks in 5G.
method Combines Kalman Filtering and PTP for pairwise synchronization, and Factor Graphs and Belief Propagation for end-to-end synchronization.
result Error in offset estimation remains below 5 ns in simulations.
Deep neural networks provide meaningful uncertainty estimates for large-scale simulations.
problem Uncertainty estimates for deep neural network predictions from large-scale simulations.
method General variational inference approach to calibrate Bayesian uncertainties.
result Calibrated Bayesian uncertainties preserved physics-correlations in predicted quantities.
DALC customizes LSTM models for detectors in large-scale traffic networks.
problem Fine-grained traffic prediction for large-scale transportation networks.
method Formulated as a finite Markov decision process, introduced ALC algorithm for automatic customization, and developed DALC for distributed customization.
result DALC provides higher prediction accuracy than Apache Spark MLlib approaches.
Efficiently applies NTK to large-scale datasets using random features.
problem Computational limitations of kernel methods for large-scale datasets.
method Proposes a sketching-based algorithm combining random features of arc-cosine kernels to construct an efficient feature map of the NTK.
result Achieves comparable error bounds to exact kernel methods but with significantly reduced feature dimensionality.
This paper proposes a convolutional neural network (CNN)-based method that learns traffic as images and predicts large-scale, network-wide traffic speed with a high accuracy. Spatiotemporal traffic dynamics are converted to images describing the time and space relations of traffic flow via a two-dimensional time-space …
We study the problem of large-scale network embedding, which aims to learn latent representations for network mining applications. Previous research shows that 1) popular network embedding benchmarks, such as DeepWalk, are in essence implicitly factorizing a matrix with a closed form, and 2)the explicit factorization o…
Paper studies randomized spectral clustering for large-scale networks.
problem Computational challenges in large-scale network community detection.
method Randomized sketching algorithms for spectral clustering.
result Theoretical bounds for approximation, misclassification, and link probability estimation.
Graph neural networks improve El Niño forecasts.
problem Improving seasonal forecasting accuracy for El Niño-Southern Oscillation.
method Designing a novel graph connectivity learning module to model large-scale spatial interactions with ENSO forecasting.
result Our model \graphino outperforms state-of-the-art models for forecasts up to six months ahead.
ConvNets improve nonstationary covariance estimation for large-scale spatial data.
problem Estimating nonstationary spatial covariance functions on large scales.
method Convolutional Neural Networks (ConvNets) for subregion identification and selection.
result Enhanced accuracy in parameter estimation using ConvNet-based partitioning.
There are many surprising and perhaps counter-intuitive properties of optimization of deep neural networks. We propose and experimentally verify a unified phenomenological model of the loss landscape that incorporates many of them. High dimensionality plays a key role in our model. Our core idea is to model the loss la…
Proposes DC-S3GD for efficient large-scale decentralized neural network training.
problem Training large-scale decentralized neural networks efficiently.
method Decentralized stale-synchronous version of DC-ASGD with gradient correction.
result Achieves state-of-the-art results in training Convolutional Neural Networks.
Augments graph node features to improve GNN performance.
problem Improving graph neural networks' performance on large-scale datasets.
method Iteratively augments node features with gradient-based adversarial perturbations.
result Boosts model performance in node classification, link prediction, and graph classification tasks.
Real time large scale streaming data pose major challenges to forecasting, in particular defying the presence of human experts to perform the corresponding analysis. We present here a class of models and methods used to develop an automated, scalable and versatile system for large scale forecasting oriented towards saf…
LargeMvC-Net improves scalability of multi-view clustering.
problem Scalability issues in multi-view clustering.
method Deep unfolding of multi-view clustering into a network architecture with three modules.
result LargeMvC-Net consistently outperforms state-of-the-art methods in scalability and effectiveness.
Deep GNNs and self-supervision boost graph learning at scale.
problem Efficiently deploying GNNs at large scale remains challenging.
method Two large-scale GNNs: a deep transductive node classifier and a very deep inductive graph regressor.
result Award-level performance on MAG240M and PCQM4M benchmarks.
Post-processes deep networks with StoNet to quantify uncertainty.
problem Uncertainty quantification in predictions from large-scale deep neural networks.
method Feeds DNN output into StoNet, trains StoNet with sparse penalty, constructs prediction intervals.
result Proposed approach constructs honest confidence intervals with shorter lengths and better calibration.
Annotating the right data for training deep neural networks is an important challenge. Active learning using uncertainty estimates from Bayesian Neural Networks (BNNs) could provide an effective solution to this. Despite being theoretically principled, BNNs require approximations to be applied to large-scale problems, …
High-performance neural network training on a cluster of Pentium III processors.
problem Extremely computationally intensive neural network training.
method Distributed training on a Linux-based cluster of Pentium III processors.
result 1.73 million parameter neural network trained at 163.3 GFLOPS/s.
The study introduces measures of collective mobility from aggregated OD data.
problem Understanding large-scale mobility patterns from aggregated data.
method Developed a framework using synthetic and real data to interpret network-level mobility.
result Aggregated mobility measures reveal network structure and flow constraints.
Many modern big data applications feature large scale in both numbers of responses and predictors. Better statistical efficiency and scientific insights can be enabled by understanding the large-scale response-predictor association network structures via layers of sparse latent factors ranked by importance. Yet sparsit…
Click through rate (CTR) prediction is very important for Native advertisement but also hard as there is no direct query intent. In this paper we propose a large-scale event embedding scheme to encode the each user browsing event by training a Siamese network with weak supervision on the users' consecutive events. The …
We study the problem of large scale, multi-label visual recognition with a large number of possible classes. We propose a method for augmenting a trained neural network classifier with auxiliary capacity in a manner designed to significantly improve upon an already well-performing model, while minimally impacting its c…
This paper studies large-scale dynamical networks where the current state of the system is a linear transformation of the previous state, contaminated by a multivariate Gaussian noise. Examples include stock markets, human brains and gene regulatory networks. We introduce a transition matrix to describe the evolution, …
We propose a nonparametric approach to link prediction in large-scale dynamic networks. Our model uses graph-based features of pairs of nodes as well as those of their local neighborhoods to predict whether those nodes will be linked at each time step. The model allows for different types of evolution in different part…
Water pollution is a major global environmental problem, and it poses a great environmental risk to public health and biological diversity. This work is motivated by assessing the potential environmental threat of coal mining through increased sulfate concentrations in river networks, which do not belong to any simple …
Communication networks have evolved from specialized, research and tactical transmission systems to large-scale and highly complex interconnections of intelligent devices, increasingly becoming more commercial, consumer-oriented, and heterogeneous. Propelled by emergent social networking services and high-definition st…
Efficiently attacks large-scale graphs without using the whole graph.
problem Vulnerability of graph neural networks to adversarial attacks.
method Simplified Gradient-based Attack (SGA) method for large-scale graphs.
result SGA achieves significant time and memory efficiency improvements.
A new model for sparse networks improves consistency of score estimators.
problem Statistical inference in large sparse networks.
method General pairwise comparison model with flexible parametrization.
result Maximum likelihood estimator is uniformly consistent under sparse conditions.
Study of 40+ complexity measures in 10,000 deep networks.
problem Generalization of deep networks in various settings.
method Systematic study of 10,000 convolutional networks with varied hyperparameters.
result Surprising failures and promising measures for further research.
Protein interactions constitute the fundamental building block of almost every life activity. Identifying protein communities from Protein-Protein Interaction (PPI) networks is essential to understand the principles of cellular organization and explore the causes of various diseases. It is critical to integrate multipl…
This paper analyzes convergence of large-scale Transformers with weight decay.
problem Understanding optimization guarantees in large-scale Transformer training.
method Construct mean-field limit, show gradient flow convergence to PDE, demonstrate global minimum consistency.
result Gradient flow reaches global minimum in large-scale Transformers with small weight decay.
Paper introduces models to discover complex structures in large hypergraphs.
problem Understanding dependency structures in complex systems represented as hypergraphs.
method Probabilistic models treating classes of similar units as nodes in a latent hypergraph, using low-rank representations.
result Improves link prediction and discovers interpretable structures in diverse real-world systems.
Generative models for graphs have been typically committed to strong prior assumptions concerning the form of the modeled distributions. Moreover, the vast majority of currently available models are either only suitable for characterizing some particular network properties (such as degree distribution or clustering coe…
We present Distributed Equivalent Substitution (DES) training, a novel distributed training framework for large-scale recommender systems with dynamic sparse features. DES introduces fully synchronous training to large-scale recommendation system for the first time by reducing communication, thus making the training of…
New method improves adversarial learning with unlabeled data.
problem Poor quality of pseudo labels on unlabeled data.
method Robust Co-training (RCT) using deep co-training.
result RCT significantly outperforms baselines in adversarial robustness.
FaStR improves scalability for time-aware RS with varying coefficients.
problem Limited applicability of structured regression models to large-scale data with categorical effects and many interactions.
method Combines structured additive regression and factorization approaches in a neural network-based model implementation.
result FaStR scales better and performs competitively with other time-aware RS in prediction performance.