NetDP predicts loan defaults using network data, addressing cold-start issues.
problem Cold-start problem in default prediction for new users.
method Combines unsupervised and supervised network representations, using parameter-server for scalability.
result Effectiveness in cold-start problem, especially for new users.
Etsy uses novel embeddings to improve user recommendations based on item interactions.
problem Improving personalized recommendations for users based on diverse item interactions.
method Learning interaction-based item embeddings to encode co-occurrence patterns of item and interaction types.
result Taking interaction type into account improves user shopping behavior modeling accuracy.
Federated learning platform for drug discovery without sharing data.
problem Lack of secure collaboration in drug discovery.
method Industry-scale federated learning platform using cryptographic aggregation.
result Generated new scientific discoveries in drug discovery.
Industrial-scale podcast recommender system optimizes long-term listening journeys.
problem Optimizing long-term listening experiences in podcast recommendation systems.
method Reinforcement learning approach to optimize user listening journeys over months.
result Significantly improved long-term performance in A/B tests compared to short-term metrics.
Compression of Neural Networks (NN) has become a highly studied topic in recent years. The main reason for this is the demand for industrial scale usage of NNs such as deploying them on mobile devices, storing them efficiently, transmitting them via band-limited channels and most importantly doing inference at scale. I…
Paper proposes scalable privacy-preserving DNN for industrial applications.
problem Data isolation and scalability issues in deep neural networks.
method Split computation graph into private and neutral server parts; use cryptographic techniques for private data.
result Demonstrates practicality of the proposed scalable privacy-preserving DNN.
Temporal Causal Prior-Data Fitted Networks (TCPFN) for industrial time series causal discovery
problem Estimating causal effects in industrial time series
method Temporal Causal Prior-Data Fitted Networks
result Zero-shot causal discovery with explicit reliability signals
CorePPR combines PageRank and CoreRank for scalable GNNs.
problem Expensive message passing in GNNs for large graphs.
method Learnable convex combination of approximate PPR and CoreRank, dynamic node selection.
result CorePPR outperforms PPRGo on large graphs.
Similar models predict similarly, reducing overfitting risk.
problem Excessive reuse of test data in machine learning.
method Proved model similarity mitigates overfitting and provided a generalization bound.
result Model similarity reduces the risk of overfitting, even when accuracy levels suggest otherwise.
Real-time advertising allows advertisers to bid for each impression for a visiting user. To optimize specific goals such as maximizing revenue and return on investment (ROI) led by ad placements, advertisers not only need to estimate the relevance between the ads and user's interests, but most importantly require a str…
While deep learning models have achieved state-of-the-art accuracies for many prediction tasks, understanding these models remains a challenge. Despite the recent interest in developing visual tools to help users interpret deep learning models, the complexity and wide variety of models deployed in industry, and the lar…
Snorkel DryBell uses weak supervision to speed up machine learning model development.
problem Costly label data in machine learning applications.
method Flexible ingestion of organizational knowledge, cross-feature production serving, scalable execution.
result Comparable quality to hand-labeled models, 52% performance improvement on average.
As the Industrial Internet of Things (IIoT) grows, systems are increasingly being monitored by arrays of sensors returning time-series data at ever-increasing 'volume, velocity and variety' (i.e. Industrial Big Data). An obvious use for these data is real-time systems condition monitoring and prognostic time to failure…
LineFlow is a framework for training RL agents to control production lines.
problem Designing control systems for production lines is challenging.
method Introduces LineFlow, an extensible Python framework for simulating and training RL agents.
result RL agents approach optimal performance in well-understood scenarios but face challenges in complex industrial lines.
What is a systematic way to efficiently apply a wide spectrum of advanced ML programs to industrial scale problems, using Big Models (up to 100s of billions of parameters) on Big Data (up to terabytes or petabytes)? Modern parallelization strategies employ fine-grained operations and scheduling beyond the classic bulk-…
Expands small recommendation datasets to industrial scale.
problem Disconnection between academic and industrial data scales.
method Randomized fractal expansions using Kronecker Graph Theory.
result Generated synthetic data sets with 1.2B ratings, 2.2M users, and 855K items.
SNGP improves DNNs' uncertainty estimation with minimal changes.
problem Uncertainty estimation in deep learning models for real-time applications.
method Formalizing uncertainty as a minimax problem, SNGP adds weight normalization and replaces the output layer with a Gaussian process.
result SNGP outperforms other single-model approaches in uncertainty estimation across vision and language tasks.
In E-commerce advertising, where product recommendations and product ads are presented to users simultaneously, the traditional setting is to display ads at fixed positions. However, under such a setting, the advertising system loses the flexibility to control the number and positions of ads, resulting in sub-optimal p…
Model shows wealth taxes can cause sudden emigration waves, impacting GDP.
problem Estimating the economic impact of wealth taxes on emigration.
method Developed a social contagion model with tipping-point dynamics, embedded in Fokker-Planck framework.
result Micro-to-macro extrapolation requires five conditions to hold, violating each.
Reincarnating RL reuses prior work to accelerate RL progress.
problem Efficiency and accessibility in reinforcement learning for large-scale applications.
method Transfer of learned policies between RL agents or design iterations, focusing on value-based RL.
result Demonstrated gains in performance over tabula rasa RL on various tasks.
The rise of Big Data has led to new demands for Machine Learning (ML) systems to learn complex models with millions to billions of parameters, that promise adequate capacity to digest massive datasets and offer powerful predictive analytics thereupon. In order to run ML algorithms at such scales, on a distributed clust…
Extends Shifts dataset for MS lesion segmentation and marine vessel power estimation.
problem Distributional shift in training and deployment data for ML models.
method Develops new datasets for high-risk industrial applications.
result Demonstrates robustness and uncertainty estimation in new industrial tasks.
This study reveals the critical role of scale vectors in large language models, improving optimization and expressivity.
problem Understanding and optimizing the scale vectors in large language models.
method Systematic study of scale vectors from expressivity, optimization, and architectural perspectives; theoretical and empirical analysis of weight decay; proposing and evaluating improvements.
result Scale vectors improve optimization through a self-amplifying preconditioning effect and are beneficial for expressivity in certain architectures.
SNGP improves single-model deep uncertainty by enhancing distance-awareness.
problem Improving uncertainty estimation in deep learning models, especially for real-time applications.
method SNGP improves distance-awareness of DNNs through spectral normalization and Gaussian process layers.
result SNGP outperforms other single-model approaches in prediction, calibration, and out-of-domain detection.