Introduces resemblance structure for large scale geometry.
problem Defining similarity in large scale geometry.
method Axiomatizing the concept of resemblance for subsets of a set.
result Large scale resemblance structures can induce nearness and generalize large scale properties.
We study the concept of coarse disjointness and large scale n-to-1 functions. As a byproduct, we obtain an Ostrand-type characterization of asymptotic dimension for coarse structures. It is shown that properties like finite asymptotic dimension, coarse finitism, large scale weak paracompactness, ect. are all invari…
Memory-efficient learning for large-scale imaging systems.
problem Memory limitations in GPUs for real-world large-scale inverse problems.
method Exploits reversibility of network layers to enable data-driven design.
result Demonstrated on small-scale and large-scale real-world systems.
Introduces bounded scale measure and generalizes property A.
problem Defining property A for large scale spaces with bounded geometry.
method Introduces bounded scale measure, shows its coarse invariance, and generalizes property A.
result Definition of property A for large scale spaces with bounded scale measure is a coarse invariant.
Efficiently solves large-scale robust portfolio optimization problems.
problem High computational demands in large-scale robust portfolio optimization.
method Extended supporting hyperplane approximation for distributionally robust portfolio problems.
result Significantly reduces computational time from several thousand seconds to just a few.
New principles needed for scaling large language models, challenging traditional regularization methods.
problem The shift from generalization to scaling in machine learning requires new guiding principles.
method Examining the effectiveness of traditional regularization methods in the scaling-centric era.
result Traditional principles of regularization may not generalize to larger scales, highlighting new phenomena like scaling law crossover.
Paper presents an efficient algorithm for learning minimax risk classifiers with large-scale data.
problem Efficient learning of minimax risk classifiers for large-scale data with multiple classes.
method Combination of constraint and column generation for efficient learning.
result 10x speedup for general large-scale data and 100x speedup with many classes.
We present an idea of unifying small scale (topology, proximity spaces, uniform spaces) and large scale (coarse spaces, large scale spaces). It relies on an analog of multilinear forms from Linear Algebra. Each form has a large scale compactification and those include all well-known compactifications: Higson corona, Gr…
BanditLP optimizes personalized recommendations for large-scale systems.
problem Optimizing personalized recommendations for large-scale systems with constraints.
method Unified neural Thompson Sampling for learning and large-scale linear programming for action selection.
result Consistent gains over strong baselines in experiments and business win in LinkedIn's email marketing system.
Let M be a compact manifold. We show the identity component Homeo0(M) of the group of self-homeomorphisms of M has a well-defined quasi-isometry type, and study its large scale geometry. Through examples, we relate this large scale geometry to both the topology of M and the dynamics of group actions on M. T…
Many real-world regression problems demand a measure of the uncertainty associated with each prediction. Standard decision forests deliver efficient state-of-the-art predictive performance, but high-quality uncertainty estimates are lacking. Gaussian processes (GPs) deliver uncertainty estimates, but scaling GPs to lar…
Efficient knockoffs for large-scale feature selection.
problem Large-scale feature selection problems.
method Gaussian model-X knockoffs with efficient methods for solving semidefinite programs.
result Efficient knockoffs can be generated with linear complexity in the dimension.
Scale of data and scale of computation infrastructures together enable the current deep learning renaissance. However, training large-scale deep architectures demands both algorithmic improvement and careful system configuration. In this paper, we focus on employing the system approach to speed up large-scale training.…
SHAKE-GNN scales GNNs for large graphs with multi-scale representations.
problem Scaling Graph Neural Networks (GNNs) to large graphs.
method SHAKE-GNN uses a hierarchy of Kirchhoff Forests for stochastic multi-resolution graph decompositions.
result SHAKE-GNN achieves competitive performance on large-scale graph classification benchmarks.
Paper introduces scalable clustering for large datasets with outliers.
problem Lack of scalable algorithms for large datasets with outliers.
method Provable robust clustering algorithm based on loss minimization for Gaussian mixture models.
result Algorithm provides high accuracy with theoretical guarantees and outperforms existing methods.
Study how large-scale flows align small-scale vortices in 3D Euler equations.
problem Understanding how large-scale flows align small-scale vortices in 3D Euler equations.
method Constructing a Lagrangian coordinate to identify when the Lie bracket is zero and investigating the locality of the pressure term.
result Clarified conditions under which small-scale vortices are aligned by large-scale flows.
Efficiently trains deep Gaussian processes on large datasets.
problem Large-scale data and multi-scale features in function approximation.
method Combines variational learning with MCMC for efficient and accurate training.
result Highly efficient and accurate deep GP training on large-scale data.
Large-batch training approaches have enabled researchers to utilize large-scale distributed processing and greatly accelerate deep-neural net (DNN) training. For example, by scaling the batch size from 256 to 32K, researchers have been able to reduce the training time of ResNet50 on ImageNet from 29 hours to 2.2 minute…
A topology on a set X is the same as a projection (i.e. an idempotent linear operator) cl:2X→2X satisfying A⊂cl(A) for all A⊂X. That's a good way to summarize Kuratowski's closure operator. Basic geometry on a set X is a dot product ⋅:2X×2X→2Y. Its equivalent form is an or…
The paper shows how to efficiently generate large Gaussian process samples with reliability guarantees.
problem Generating large-scale Gaussian process samples efficiently and with reliability.
method Demonstrates scaling data generation to large \(n\) while providing high probability guarantees.
result Efficiently generates large Gaussian process samples with reliability guarantees.
New kernel model scales to large datasets.
problem Challenges in scaling kernel machines to large datasets.
method Introduced EigenPro 3.0, an algorithm based on projected dual preconditioned SGD.
result Ability to scale model and data sizes independently.
Method analyzes large-scale network data to detect communication pattern shifts.
problem Analyzing large-scale time-series network data is challenging.
method Temporal encoder embedding method using ground-truth or estimated vertex labels.
result Detects communication pattern shifts across all levels of network structure.
A new algorithm speeds up CP decomposition for large tensors.
problem Efficiently processing large-scale tensors in real-time.
method Randomized online CP decomposition (ROCP) algorithm.
result ROCP reduces computing time and memory usage significantly.
Paper proposes PPMM for fast estimation of large-scale OTM.
problem Estimation of large-scale optimal transport maps (OTM) is challenging due to the curse of dimensionality.
method Combines projection pursuit regression and sufficient dimension reduction to adaptively select projection directions.
result PPMM consistently estimates the most informative projection direction and weakly converges to the target OTM.
Paper proposes data quality measures for large-scale high-dimensional data.
problem Lack of practical data quality measures for large-scale high-dimensional data.
method Proposes two data quality measures: class separability and in-class variability. Efficient algorithms based on random projections and bootstrapping are provided.
result Efficient algorithms for computing data quality measures on large-scale high-dimensional data.
Theory explains neural network scaling with dataset and model size.
problem Neural network scaling laws with dataset and model size.
method Identified variance-limited and resolution-limited scaling behaviors.
result Four scaling regimes explained: infinite data, infinite width, resolution-limited, and large width.
New method infers causal factors from large-scale data without full graph reconstruction.
problem Inferring causal variables from large-scale systems without full causal graph reconstruction.
method Supervised learning on simulated data using a neural network and subsampled-ensemble inference.
result Efficiently identifies causal relationships in large-scale gene regulatory networks.
Large models follow power laws in performance with dataset size or parameters.
problem Understanding neural scaling laws in large language models.
method Joint generative data model and random feature model.
result Modeling and solving the dual limit reveals insights into scaling laws.
We study in this paper a variant of Wasserstein barycenter problem, which we refer to as tree-Wasserstein barycenter, by leveraging a specific class of ground metrics, namely tree metrics, for Wasserstein distance. Drawing on the tree structure, we propose an efficient algorithmic approach to solve the tree-Wasserstein…
Natural image statistics exhibit hierarchical dependencies across multiple scales. Representing such prior knowledge in non-factorial latent tree models can boost performance of image denoising, inpainting, deconvolution or reconstruction substantially, beyond standard factorial "sparse" methodology. We derive a large …
DistPre predicts traffic speeds efficiently for large networks.
problem Fine-grained, accurate speed prediction for large-scale transportation networks.
method Customizes LSTM models on a cluster, sharing trained models between detectors.
result Efficient and accurate fine-grained traffic-speed prediction.
This work aims to create a large-scale model for critical care time series data.
problem Lack of large-scale datasets and distribution shifts in critical care time series data.
method Harmonized dataset creation and transfer learning research.
result Established a foundation for large-scale multi-variate time series models in critical care.
We propose a practical and scalable Gaussian process model for large-scale nonlinear probabilistic regression. Our mixture-of-experts model is conceptually simple and hierarchically recombines computations for an overall approximation of a full Gaussian process. Closed-form and distributed computations allow for effici…
Deep GNNs and self-supervision boost graph learning at scale.
problem Efficiently deploying GNNs at large scale remains challenging.
method Two large-scale GNNs: a deep transductive node classifier and a very deep inductive graph regressor.
result Award-level performance on MAG240M and PCQM4M benchmarks.
Proposes GBBHE for efficient large-scale regression.
problem Large-scale regression problems.
method Gradient Boosting with binary histogram partition and ensemble learning.
result Improves computational efficiency and performance on large datasets.
New algorithms estimate Jacobian matrices for large-scale machine learning.
problem Efficiently computing search directions for large nonlinear least squares.
method Exploit low-rank structure in Hessian to estimate Jacobian matrices.
result Two algorithms perform well compared to state-of-the-art methods.
We introduce the notion of large scale inductive dimension for asymptotic resemblance spaces. We prove that the large scale inductive dimension and the asymptotic dimensiongrad are equal in the class of r-convex metric spaces. This class contains the class of all geodesic metric spaces and all finitely generated groups…
We define a complete Riemannian manifold X to be large-scale conformally rigid if all groups that are quasi-isometric to some complete Riemannian manifold of bounded geometry conformal to X are quasi-isometric to X. We prove that many 3-manifolds, including Euclidean 3-space, hyperbolic 3-space and the product of the h…
We introduce a few variants on Frank-Wolfe style algorithms suitable for large scale optimization. We show how to modify the standard Frank-Wolfe algorithm using stochastic gradients, approximate subproblem solutions, and sketched decision variables in order to scale to enormous problems while preserving (up to constan…
Uniformity and proximity are two different ways for defining small scale structures on a set. Coarse structures are large scale counterparts of uniform structures. In this paper, motivated by the definition of proximity, we develop the concept of asymptotic resemblance as a relation between subsets of a set to define a…
Large scale online kernel learning aims to build an efficient and scalable kernel-based predictive model incrementally from a sequence of potentially infinite data points. A current key approach focuses on ways to produce an approximate finite-dimensional feature map, assuming that the kernel used has a feature map wit…
This paper analyzes convergence of large-scale Transformers with weight decay.
problem Understanding optimization guarantees in large-scale Transformer training.
method Construct mean-field limit, show gradient flow convergence to PDE, demonstrate global minimum consistency.
result Gradient flow reaches global minimum in large-scale Transformers with small weight decay.
New method estimates bidirectional causal effects in large-scale systems.
problem Estimating bidirectional causal effects in systems with mutual dependence and heteroskedasticity.
method Heteroskedasticity-based identification with online kernel learning and random Fourier features.
result Superior accuracy and stability compared to single equation and polynomial approximations.
Root cause analysis in a large-scale production environment is challenging due to the complexity of services running across global data centers. Due to the distributed nature of a large-scale system, the various hardware, software, and tooling logs are often maintained separately, making it difficult to review the logs…
Analyzed US firm data 1970-2019, identifying scale effects and distributional forms.
problem Understanding differences between small and large firms over time.
method Examined all public US firms, used stylized facts and DLN distribution analysis.
result Small firms are systematically different from large firms, with scale-dependent heteroskedasticity.
This paper tackles hyperparameter tuning for large-scale kernel ridge regression.
problem Hyperparameter tuning is crucial but often left to users, hindering efficiency and usability.
method Proposes a complexity regularization criterion based on a data-dependent penalty for efficient optimization.
result Demonstrates the benefit of the proposed approach through extensive empirical evaluation.
Chemical space is so large that brute force searches for new interesting molecules are infeasible. High-throughput virtual screening via computer cluster simulations can speed up the discovery process by collecting very large amounts of data in parallel, e.g., up to hundreds or thousands of parallel measurements. Bayes…
A new, fast kernel test for large data.
problem Efficient kernel two-sample tests for high-dimensional, large-scale data.
method A new kernel-based test that is computationally efficient and robust to high dimensions.
result The new test performs well across various alternatives and dimensions.