A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
SOBER framework optimizes Bayesian optimization tasks efficiently.
Develops a flexible batched experimentation framework for limited adaptivity.
Adaptive batch sizes improve active learning efficiency and flexibility.
Dynamic neural network toolkits such as PyTorch, DyNet, and Chainer offer more flexibility for implementing models that cope with data of varying dimensions and structure, relative to toolkits that operate on statically declared computations (e.g., TensorFlow, CNTK, and Theano). However, existing toolkits - both static…
FlexServe simplifies deployment of PyTorch models as REST endpoints.
In this work we develop a new algorithm for regularized empirical risk minimization. Our method extends recent techniques of Shalev-Shwartz [02/2015], which enable a dual-free analysis of SDCA, to arbitrary mini-batching schemes. Moreover, our method is able to better utilize the information in the data defining the ER…
New algorithm reduces overfitting in neural networks.
QC-ST and CoCo methods correct batch effects in metabolomics data.
Takagi-Sugeno-Kang (TSK) fuzzy systems are flexible and interpretable machine learning models; however, they may not be easily optimized when the data size is large, and/or the data dimensionality is high. This paper proposes a mini-batch gradient descent (MBGD) based algorithm to efficiently and effectively train TSK …
Proposes a new semi-parametric framework for batched bandits with covariates.
A significant advance in accelerating neural network training has been the development of normalization methods, permitting the training of deep models both faster and with better accuracy. These advances come with practical challenges: for instance, batch normalization ties the prediction of individual examples with o…
mSAM improves generalization by making models flatter.
Adaptive gradient-based optimizers such as Adagrad and Adam are crucial for achieving state-of-the-art performance in machine translation and language modeling. However, these methods maintain second-order statistics for each parameter, thus introducing significant memory overheads that restrict the size of the model b…
BMBO-DARN optimizes expensive functions with varying fidelities.
When learning policies for real-world domains, two important questions arise: (i) how to efficiently use pre-collected off-policy, non-optimal behavior data; and (ii) how to mediate among different competing objectives and constraints. We thus study the problem of batch policy learning under multiple constraints, and o…
VOPy optimizes multiple objectives with flexible cone-based ordering.
Utilizing recently introduced concepts from statistics and quantitative risk management, we present a general variant of Batch Normalization (BN) that offers accelerated convergence of Neural Network training compared to conventional BN. In general, we show that mean and standard deviation are not always the most appro…
New convergence results for NGVI with various step sizes and sample sizes.
The AMIDST Toolbox is a software for scalable probabilistic machine learning with a spe- cial focus on (massive) streaming data. The toolbox supports a flexible modeling language based on probabilistic graphical models with latent variables and temporal dependencies. The specified models can be learnt from large data s…
We propose Information-Theoretic Active Learning (ITAL), a novel batch-mode active learning method for binary classification, and apply it for acquiring meaningful user feedback in the context of content-based image retrieval. Instead of combining different heuristics such as uncertainty, diversity, or density, our met…
Normalization methods are a central building block in the deep learning toolbox. They accelerate and stabilize training, while decreasing the dependence on manually tuned learning rate schedules. When learning from multi-modal distributions, the effectiveness of batch normalization (BN), arguably the most prominent nor…
A monitoring procedure improves machine learning forecasts for digital platforms.
The paper studies early stopping methods in linear contextual bandits.
The gradient noise of SGD is considered to play a central role in the observed strong generalization abilities of deep learning. While past studies confirm that the magnitude and the covariance structure of gradient noise are critical for regularization, it remains unclear whether or not the class of noise distribution…
Neural networks fit fewer samples than their parameters suggest in practice.
Framework learns linear programs from optimal decisions.
Rule-based models are often used for data analysis as they combine interpretability with predictive power. We present RuleKit, a versatile tool for rule learning. Based on a sequential covering induction algorithm, it is suitable for classification, regression, and survival problems. The presence of a user-guided induc…
This paper proposes a voice conversion (VC) method using sequence-to-sequence (seq2seq or S2S) learning, which flexibly converts not only the voice characteristics but also the pitch contour and duration of input speech. The proposed method, called ConvS2S-VC, has three key features. First, it uses a model with a fully…
New research shows many batch selection methods for training work just as well as full batch training.
Determining the appropriate batch size for mini-batch gradient descent is always time consuming as it often relies on grid search. This paper considers a resizable mini-batch gradient descent (RMGD) algorithm based on a multi-armed bandit for achieving best performance in grid search by selecting an appropriate batch s…
We analyze how batch learning impacts bandit problems.
BatchGFN uses generative flow networks for efficient batch active learning.
CBN improves batch normalization for small mini-batch sizes.
Opt-BBAI identifies the best arm with minimal batches and pulls, optimizing both sample and batch complexity.
Batch Normalization (BN) uses mini-batch statistics to normalize the activations during training, introducing dependence between mini-batch elements. This dependency can hurt the performance if the mini-batch size is too small, or if the elements are correlated. Several alternatives, such as Batch Renormalization and G…
A new method designs batches for Bayesian optimization more efficiently.
Training deep neural networks with Stochastic Gradient Descent, or its variants, requires careful choice of both learning rate and batch size. While smaller batch sizes generally converge in fewer training epochs, larger batch sizes offer more parallelism and hence better computational efficiency. We have developed a n…
Pykg2vec is an open-source Python library for learning the representations of the entities and relations in knowledge graphs. Pykg2vec's flexible and modular software architecture currently implements 16 state-of-the-art knowledge graph embedding algorithms, and is designed to easily incorporate new algorithms. The goa…
Riemannian stochastic gradient descent converges faster with increasing batch size.
Polynomial-time algorithm for list-decodable linear regression with batches.
New findings show mini-batch SGD operates in a 'Edge of Stochastic Stability' regime.
Batch normalisation doesn't affect variational inference but fails for larger batch sizes.
A new scaling law predicts optimal batch size for training models.
E algorithm optimizes batched linear bandits with minimal regret and batches.
ABS dynamically adjusts batch size based on policy stability, improving RL performance.
SNGM improves large-batch training accuracy.
Paper proposes MABN to stabilize BN in small batch sizes.
We investigate two new strategies for the numerical solution of optimal stopping problems within the Regression Monte Carlo (RMC) framework of Longstaff and Schwartz. First, we propose the use of stochastic kriging (Gaussian process) meta-models for fitting the continuation value. Kriging offers a flexible, nonparametr…