Novel neural network solves PDEs with multi-scale resolution.
problem Solving time-dependent PDEs with varying spatial and temporal scales.
method Multi-scale message passing neural network with temporal and spatial gating modules.
result Outperforms baselines on PDEs with diverse scales.
Improved algorithm for multidimensional scaling reduces stress.
problem Stress in multidimensional scaling.
method Proposed modifications of the smacof algorithm.
result Convergent majorization algorithm for Kruskal's stress formula two.
We introduce a few variants on Frank-Wolfe style algorithms suitable for large scale optimization. We show how to modify the standard Frank-Wolfe algorithm using stochastic gradients, approximate subproblem solutions, and sketched decision variables in order to scale to enormous problems while preserving (up to constan…
Nearest Neighbors Algorithm is a Lazy Learning Algorithm, in which the algorithm tries to approximate the predictions with the help of similar existing vectors in the training dataset. The predictions made by the K-Nearest Neighbors algorithm is based on averaging the target values of the spatial neighbors. The selecti…
Paper introduces scalable clustering for large datasets with outliers.
problem Lack of scalable algorithms for large datasets with outliers.
method Provable robust clustering algorithm based on loss minimization for Gaussian mixture models.
result Algorithm provides high accuracy with theoretical guarantees and outperforms existing methods.
New analysis of stochastic approximation with non-expansive mappings.
problem Finite-time analysis of two-time-scale stochastic approximation with non-expansive mappings.
method Studied two-time-scale stochastic approximation algorithms with non-expansive mappings and projection steps.
result Last-iterate mean square residual error decays at a rate O(1/k1/4−ε). A new algorithm speeds up CP decomposition for large tensors.
problem Efficiently processing large-scale tensors in real-time.
method Randomized online CP decomposition (ROCP) algorithm.
result ROCP reduces computing time and memory usage significantly.
Algorithm identifies fractal system's scaling exponents in high dimensions.
problem Statistical identification of Hurst distribution in high-dimensional fractal systems.
method Wavelet random matrices, modified spectral clustering, model selection.
result Algorithm consistently estimates Hurst distribution in moderately high dimensions.
We introduce online learning algorithms which are independent of feature scales, proving regret bounds dependent on the ratio of scales existent in the data rather than the absolute scale. This has several useful effects: there is no need to pre-normalize data, the test-time and test-space complexity are reduced, and t…
We introduce online learning algorithms which are independent of feature scales, proving regret bounds dependent on the ratio of scales existent in the data rather than the absolute scale. This has several useful effects: there is no need to pre-normalize data, the test-time and test-space complexity are reduced, and t…
Improved loss scaling for stochastic momentum algorithms in high dimensions.
problem Improving loss scaling for stochastic momentum algorithms in high dimensions.
method Dimension-adapted Nesterov acceleration (DANA) scales momentum hyperparameters based on model size and data complexity.
result DANA improves loss scaling exponents across various data and target complexities.
Proposes a method to predict time series data using scale information.
problem Sequential data exhibits different behavior at different scales.
method Scale-based inference using k-nearest neighbors and sequential prediction. result Asymptotically, the algorithm produces the best prediction error.
A new method for automatically learning metric scaling in metric-based meta-learning.
problem Lack of principled method for learning metric scaling parameter.
method Developed a variational metric scaling framework for automatic metric scaling parameter learning.
result Consistently improves the performance of existing metric-based meta-algorithms.
New algorithm reduces regret bounds for Bayesian optimization with unknown hyperparameters.
problem Optimizing black-box functions with unknown hyperparameters, especially length scale.
method Length Scale Balancing (LB) - aggregating multiple surrogate models with varying length scales.
result LB achieves a regret bound only logaritically away from the oracle algorithm.
Paper presents an efficient algorithm for learning minimax risk classifiers with large-scale data.
problem Efficient learning of minimax risk classifiers for large-scale data with multiple classes.
method Combination of constraint and column generation for efficient learning.
result 10x speedup for general large-scale data and 100x speedup with many classes.
New algorithm reduces robust optimization scale for better constraint satisfaction.
problem Finding robust solutions to optimization problems with unknown constraints.
method Empirical domain reduction to determine robustness scale.
result Our algorithm's scale is less affected by parameter dimensionality.
UCB algorithm adapted for large-scale, non-sub-Gaussian problems.
problem Selecting the best alternative from a large set of options with non-sub-Gaussian performance distributions.
method Adapted UCB algorithm for non-sub-Gaussian settings, focusing on sample size and meta-UCB selection.
result UCB algorithms can achieve sample optimality in large-scale, non-sub-Gaussian problems.
Generalized algorithm for translation and scale-invariant prediction.
problem Sequential prediction with expert advice, focusing on translation and scale invariance.
method Designing a generalized online algorithm using the universal prediction perspective to compete against a generic class of expert selection strategies.
result No preliminary knowledge of loss sequences is required; performance bounds are stable under arbitrary scalings and translations.
Efficient algorithms for large Maxent models improve wildfire probability predictions.
problem Training large-scale, non-smooth Maxent models efficiently for big data.
method First-order optimization algorithms using Kullback-Leibler divergence.
result Our algorithms outperform state-of-the-art methods by one order of magnitude.
New algorithms optimize risk for large datasets, improving efficiency.
problem Optimizing risk for large datasets with robust methods.
method Proposed algorithms for distributionally robust optimization with CVaR and χ² divergence uncertainty sets.
result Algorithms require independent gradient evaluations of training set size and parameters, suitable for large-scale applications.
Paper solves robust multi-dimensional scaling with accelerated projections.
problem Localize point locations from noisy pairwise distances.
method Alternating projections with tangent space acceleration.
result Linear convergence of reconstructed points to original points.
FibeRed reduces complex data dimensions while preserving topology.
problem Hard embedding of topologically complex datasets in low-dimensional Euclidean space.
method Modeling datasets with vector bundles, reducing fibers while preserving topology.
result FibeRed learns topologically faithful embeddings in lower dimensions than existing methods.
Deep RL strategies outperform classical models in trading.
problem Designing profitable trading strategies for futures contracts.
method Deep Reinforcement Learning with volatility scaling.
result Deep RL algorithms outperformed classical models with positive profits.
Scale of data and scale of computation infrastructures together enable the current deep learning renaissance. However, training large-scale deep architectures demands both algorithmic improvement and careful system configuration. In this paper, we focus on employing the system approach to speed up large-scale training.…
This thesis tackles large-scale learning with kernel methods and proposes scalable algorithms for lifelong robot learning.
problem The challenge of scaling kernel methods to large datasets.
method We analyze and develop approximate learning algorithms, including Nyström and random features, to improve scalability.
result Our methods enable robots to learn continuously and adapt to changing environments efficiently.
This paper tackles ranking-based performance normalization for optimization algorithms.
problem Ranking optimization algorithms across diverse numerical scales disrupts performance comparisons.
method Introduces absolute ranking and a sampling-based computational method to address numerical scale variation.
result Provides a more robust framework for assessing performance across multiple algorithms and problems.
Recently, deep neural networks (DNNs) have shown advantages in accelerating optimization algorithms. One approach is to unfold finite number of iterations of conventional optimization algorithms and to learn parameters in the algorithms. However, these are forward methods and are indeed neither iterative nor convergent…
We consider a variant of online convex optimization in which both the instances (input vectors) and the comparator (weight vector) are unconstrained. We exploit a natural scale invariance symmetry in our unconstrained setting: the predictions of the optimal comparator are invariant under any linear transformation of th…
Paper analyzes convergence rates of two time-scale AC and NAC algorithms.
problem Finite-sample convergence rate analysis of two time-scale AC and NAC algorithms.
method Developed novel techniques for bias error and convergence rate analysis.
result Established non-asymptotic convergence rates for two time-scale AC and NAC.
We propose HAMSI (Hessian Approximated Multiple Subsets Iteration), which is a provably convergent, second order incremental algorithm for solving large-scale partially separable optimization problems. The algorithm is based on a local quadratic approximation, and hence, allows incorporating curvature information to sp…
Proposes MamBO for efficient high-dimensional large-scale optimization.
problem High-dimensional and large-scale optimization problems in machine learning and simulation.
method Combines subsampling and subspace embeddings with model aggregation to address uncertainty in surrogate models.
result Improves robustness of Bayesian optimization algorithm and achieves superior performance.
The present contribution suggests the use of a multidimensional scaling (MDS) algorithm as a visualization tool for manifold-valued elements. A visualization tool of this kind is useful in signal processing and machine learning whenever learning/adaptation algorithms insist on high-dimensional parameter manifolds.
Spectral clustering algorithms typically require a priori selection of input parameters such as the number of clusters, a scaling parameter for the affinity measure, or ranges of these values for parameter tuning. Despite efforts for automating the process of spectral clustering, the task of grouping data in multi-scal…
New algorithm handles bandit problems under translations and scales.
problem Adversarial multi-armed bandit problems with arbitrary translations and scales.
method Innovative online algorithm invariant to translations and scales, using universal prediction.
result Second-order regret bounds, unaffected by affine transformations of losses.
A new algorithm DC2 for large-scale kernel learning and clustering.
problem Efficiently handle large-scale kernel learning and clustering problems.
method Divide-and-conquer approach using recursive random projections for data partition and compression.
result Achieves clustering accuracy comparable to fast approximate spectral clustering algorithms with lower running time.
Study on continual learning with Twitter data, developing ConGraD algorithm.
problem Personalized online language learning on a massive scale.
method Developed POLL problem setting, collected Firehose datasets, and introduced ConGraD algorithm.
result ConGraD algorithm outperforms prior continual learning methods on Firehose datasets.
Improves RLHF sample efficiency by scaling reward complexity polynomially.
problem Exponential sample complexity in RLHF algorithms for skewed preferences.
method SE-POPO, an online RLHF algorithm that achieves polynomial sample complexity.
result SE-POPO outperforms existing algorithms in sample efficiency.
Deep reinforcement learning (deep RL) has been successful in learning sophisticated behaviors automatically; however, the learning process requires a huge number of trials. In contrast, animals can learn new tasks in just a few trials, benefiting from their prior knowledge about the world. This paper seeks to bridge th…
The Perona-Malik model has been very successful at restoring images from noisy input. In this paper, we reinterpret the Perona-Malik model in the language of Gaussian scale mixtures and derive some extensions of the model. Specifically, we show that the expectation-maximization (EM) algorithm applied to Gaussian scale …
Randomized spectral co-clustering speeds up large-scale directed networks.
problem Co-clustering directed networks efficiently for large-scale data.
method Randomized spectral co-clustering algorithms using random-projection and random-sampling techniques.
result Theoretical and numerical validation of approximation and misclustering error rates.
New algorithm clusters hyperspectral images at multiple scales.
problem Clustering hyperspectral images at various scales.
method M-SRDL algorithm using spectral-spatial diffusion distances.
result More accurate clustering labels achieved with spatial regularization.
Adaptive algorithms improve performance in non-convex optimization across various scenarios.
problem Improper handling of noise scales, gradient magnitudes, and smoothness in non-convex optimization.
method Design and analysis of noise-adaptive, scale-free, and generalized algorithms.
result Adaptive algorithms achieve optimal rates and performance in diverse optimization settings.
Proposes GBBHE for efficient large-scale regression.
problem Large-scale regression problems.
method Gradient Boosting with binary histogram partition and ensemble learning.
result Improves computational efficiency and performance on large datasets.
Bregman divergences play a central role in the design and analysis of a range of machine learning algorithms. This paper explores the use of Bregman divergences to establish reductions between such algorithms and their analyses. We present a new scaled isodistortion theorem involving Bregman divergences (scaled Bregman…
Paper studies randomized spectral clustering for large-scale networks.
problem Computational challenges in large-scale network community detection.
method Randomized sketching algorithms for spectral clustering.
result Theoretical bounds for approximation, misclassification, and link probability estimation.
New method provides formal uncertainty guarantees for image classifiers.
problem Uncertainty quantification for image classifiers without formal guarantees.
method Adapts conformal prediction to give stable, formal coverage guarantees.
result Method outperforms existing approaches in coverage and set size.
We consider online learning with linear models, where the algorithm predicts on sequentially revealed instances (feature vectors), and is compared against the best linear function (comparator) in hindsight. Popular algorithms in this framework, such as Online Gradient Descent (OGD), have parameters (learning rates), wh…
Natural image statistics exhibit hierarchical dependencies across multiple scales. Representing such prior knowledge in non-factorial latent tree models can boost performance of image denoising, inpainting, deconvolution or reconstruction substantially, beyond standard factorial "sparse" methodology. We derive a large …