Bayesian optimization has demonstrated impressive success in finding the optimum input x* and output f* = f(x*) = max f(x) of a black-box function f. In some applications, however, the optimum output f* is known in advance and the goal is to find the corresponding optimum input x*. In this paper, we consider a new sett…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,694 papers · 148 categories
Trend · papers per month
48 results for “performative optimum”
A new framework for performative prediction robust to distributional misspecification.
problem Performative prediction models can be influenced by their own predictions, leading to suboptimal outcomes.
method Introduces distributionally robust performative prediction (DRPO) to approximate the true performative optimum (PO) robustly.
result DRPO provides provable guarantees as a robust approximation to the true PO when the nominal distribution map is misspecified.
This paper compares three portfolio designs for Indian stocks.
problem Designing an optimum portfolio that balances return and risk.
method Three approaches: minimum risk, optimum risk, and Eigen portfolios.
result Optimum risk portfolios and Eigen portfolios identified for each sector.
Bayesian Optimization with a Prior for the Optimum (BOPrO) improves efficiency and accuracy.
problem Bayesian Optimization's standard priors are not intuitive for domain experts.
method BOPrO injects expert knowledge into the optimization process using priors about the optimum.
result BOPrO is 6.67x faster than state-of-the-art methods and achieves new state-of-the-art performance.
Bayesian optimisation is improved by incorporating expert prior through space warping.
problem Cold start phase in expensive function optimisation.
method Prior distribution warps the search space around high probability regions of function optimum.
result Improves optimisation performance through acquisition agnostic approach.
RWR converges to global optimum in certain settings.
problem Proving convergence of RWR to optimal policy.
method Iterative learning with return-weighted log-likelihood.
result RWR converges to global optimum under certain conditions.
Numerous empirical evidence has corroborated that the noise plays a crucial rule in effective and efficient training of neural networks. The theory behind, however, is still largely unknown. This paper studies this fundamental problem through training a simple two-layer convolutional neural network model. Although trai…
Local search algorithms applied to optimization problems often suffer from getting trapped in a local optimum. The common solution for this deficiency is to restart the algorithm when no progress is observed. Alternatively, one can start multiple instances of a local search algorithm, and allocate computational resourc…
Mixed membership factorization is a popular approach for analyzing data sets that have within-sample heterogeneity. In recent years, several algorithms have been developed for mixed membership matrix factorization, but they only guarantee estimates from a local optimum. Here, we derive a global optimization (GOP) algor…
Proposes qPO, a new acquisition strategy for batched Bayesian optimization that maximizes the probability of including the optimum.
problem Efficiently identifying top-performing compounds from a large chemical library.
method qPO (multipoint Probability of Optimality) acquisition strategy that maximizes the probability of including the true optimum.
result Empirical evidence shows that qPO is competitive with and complements other state-of-the-art methods in batched Bayesian optimization.
Optimum-statistical collaboration improves black-box optimization efficiency.
problem Improving black-box optimization efficiency through better statistical collaboration.
method Introducing optimum-statistical collaboration framework for hierarchical bandits-based optimization.
result Demonstrated improved regret bounds and better performance in experiments.
Inverse problems in imaging such as denoising, deblurring, superresolution (SR) have been addressed for many decades. In recent years, convolutional neural networks (CNNs) have been widely used for many inverse problem areas. Although their indisputable success, CNNs are not mathematically validated as to how and what …
In this paper, a new sequential surrogate-based optimization (SSBO) algorithm is developed, which aims to improve the global search ability and local search efficiency for the global optimization of expensive black-box models. The proposed method involves three basic sub-criteria to infill new samples asynchronously to…
For large scale on-line inference problems the update strategy is critical for performance. We derive an adaptive scan Gibbs sampler that optimizes the update frequency by selecting an optimum mini-batch size. We demonstrate performance of our adaptive batch-size Gibbs sampler by comparing it against the collapsed Gibb…
Novel method for high-dimensional BO using CMA to define local regions.
problem Challenges in applying BO to high-dimensional optimization problems.
method CMA strategy to learn search distribution and define local regions.
result Our method outperforms existing techniques on various benchmarks.
We study the problem of learning a latent variable model from a stream of data. Latent variable models are popular in practice because they can explain observed data in terms of unobserved concepts. These models have been traditionally studied in the offline setting. In the online setting, on the other hand, the online…
Postprocessing reduces Bayesian optimization steps for global optima.
problem Slow convergence in Bayesian optimization for high-dimensional problems.
method Prohibits duplicated samples in the dataset postprocessing method.
result Significantly reduces the number of sequential steps to find the global optimum.
Residual Network (ResNet) is undoubtedly a milestone in deep learning. ResNet is equipped with shortcut connections between layers, and exhibits efficient training using simple first order algorithms. Despite of the great empirical success, the reason behind is far from being well understood. In this paper, we study a …
Gradient descent with noise converges to a unique optimum in nonconvex matrix factorization.
problem Gradient descent with noise converges to a unique optimum in nonconvex matrix factorization.
method A perturbed form of gradient descent with arbitrary initialization.
result Gradient descent with noise converges to a unique optimum.
Community detection in graphs has been the subject of many algorithms. Recent methods want to optimize a modularity function which shows a maximum of relationships within communities and found a minimum of inter-community relations. these algorithms are applied to unipartite, multipartite and directed graphs. However, …
Machine learning techniques have been paramount throughout the last years, being applied in a wide range of tasks, such as classification, object recognition, person identification, and image segmentation. Nevertheless, conventional classification algorithms, e.g., Logistic Regression, Decision Trees, and Bayesian clas…
In-network distributed estimation of sparse parameter vectors via diffusion LMS strategies has been studied and investigated in recent years. In all the existing works, some convex regularization approach has been used at each node of the network in order to achieve an overall network performance superior to that of th…
This work simplifies SVM parameter selection using S&S ratio.
problem SVM parameter tuning for optimal performance.
method S&S ratio to model SVM performance; automatic RP, kernel, and parameter selection.
result Optimized SVM parameters with reduced computational complexity.
A new algorithm removes stale observations in dynamic Bayesian optimization.
problem Optimizing functions that change over time, keeping track of the optimum.
method Wasserstein distance-based criterion to quantify relevancy, removing stale observations.
result W-DBO maintains good predictive performance and high sampling frequency.
New findings on the max margin problem in neural networks.
problem Understanding the max margin problem in neural networks.
method Analyzing gradient flow and max margin problem in linear and ReLU networks.
result The KKT point is not always an optimum of the max margin problem.
Performance Evaluation of Equal-Weight Portfolio and Optimum Risk Portfolio on Indian Stocksq-fin.PM
This study evaluates different portfolio designs for Indian stocks.
problem Optimizing portfolio weights for risk and return in volatile stock markets.
method Three portfolio design approaches: risk minimization, risk optimization, and equal weighting. Historical data from 2017-2022 used.
result Equal-weight portfolios outperformed other designs in most sectors.
Community detection using both graphs and social networks is the focus of many algorithms. Recent methods aimed at optimizing the so-called modularity function proceed by maximizing relations within communities while minimizing inter-community relations. However, given the NP-completeness of the problem, these algorith…
Theoretical comparison of three invariance approaches in deep linear networks.
problem Understanding invariance in deep linear networks.
method Data augmentation, regularization, and hard-wiring approaches.
result Regularization introduces additional critical points, but they remain saddles except for the global optimum.
Bayesian optimization methods improved for min max optimization problems.
problem Min-max optimization for unknown functions.
method Extended Bayesian optimization to min-max problems with new acquisition functions.
result Improved acquisition functions lead to better solutions.
Study uses Bayesian Optimization to analyze noise effects in materials research.
problem Optimizing materials with many variables and experimental noise.
method Batch Bayesian Optimization with synthetic data analysis.
result Noise sensitivity varies by problem landscape, impacting optimization outcomes.
Proposes on-the-fly joint feature selection and classification for time-sensitive decisions.
problem Online feature selection and classification for time-sensitive decision making.
method Proposes a framework for joint feature selection and classification on-the-fly, optimizing feature usage and classification accuracy.
result Demonstrates superior performance over state-of-the-art methods on various datasets.
Optimizes expensive experiments by incorporating expert knowledge.
problem Expensive experiments require minimizing the number of trials.
method Bayesian optimization with posterior sampling of expert knowledge.
result Demonstrates significant efficiency gains in experiments and hyperparameter tuning.
The computational asymptotics of Gaussian variational inference and the Laplace approximationstat.CO
New algorithms improve likelihood of finding global optima in Bayesian inference.
problem Finding global optima in Bayesian inference is difficult due to nonconvexity.
method Developed two algorithms: consistent Laplace approximation (CLA) and consistent stochastic variational inference (CSVI).
result Both CSVI and CLA improve likelihood of obtaining global optima compared to standard methods.
Hyperalignment has been widely employed in Multivariate Pattern (MVP) analysis to discover the cognitive states in the human brains based on multi-subject functional Magnetic Resonance Imaging (fMRI) datasets. Most of the existing HA methods utilized unsupervised approaches, where they only maximized the correlation be…
Neural networks solve Knapsack problems with provable guarantees.
problem Solving the Knapsack Problem efficiently and with guarantees.
method Recurrent neural networks (RNNs) with rectified linear units applied iteratively to each item.
result An RNN of depth four and width proportional to the profit of an optimum solution finds optimal solutions.
Contemporary global optimization algorithms are based on local measures of utility, rather than a probability measure over location and value of the optimum. They thus attempt to collect low function values, not to learn about the optimum. The reason for the absence of probabilistic global optimizers is that the corres…
This paper analyzes popular time-nonseparable utility functions that describe "habit formation" consumer preferences comparing current consumption with the time averaged past consumption of the same individual and "catching up with the Joneses" (CuJ) models comparing individual consumption with a cross-sectional averag…
We consider a distributed learning setup where a sparse signal is estimated over a network. Our main interest is to save communication resource for information exchange over the network and reduce processing time. Each node of the network uses a convex optimization based algorithm that provides a locally optimum soluti…
Optimizes data collection for ranking and selection problems.
problem Identifying the best system from multiple solutions with limited data.
method Sequential sampling algorithm with MPB estimator and kernel ridge regression.
result OSAR achieves optimal sampling ratios almost surely in the limit.
In electroencephalogram (EEG) signal processing, finding the appropriate information from a dataset has been a big challenge for successful signal classification. The feature selection methods make it possible to solve this problem; however, the method selection is still under investigation to find out which feature ca…
In linear regression we wish to estimate the optimum linear least squares predictor for a distribution over -dimensional input points and real-valued responses, based on a small sample. Under standard random design analysis, where the sample is drawn i.i.d. from the input distribution, the least squares solution for…
Topology optimization by optimally distributing materials in a given domain requires non-gradient optimizers to solve highly complicated problems. However, with hundreds of design variables or more involved, solving such problems would require millions of Finite Element Method (FEM) calculations whose computational cos…
Bayesian optimization offers a flexible framework to optimize an objective function that is expensive to be evaluated. A Bayesian optimizer iteratively queries the function values on its carefully selected points. Subsequently, it makes a sensible recommendation about where the optimum locates based on its accumulated …
We introduce incremental variational inference and apply it to latent Dirichlet allocation (LDA). Incremental variational inference is inspired by incremental EM and provides an alternative to stochastic variational inference. Incremental LDA can process massive document collections, does not require to set a learning …
Multiple Kernel Learning, or MKL, extends (kernelized) SVM by attempting to learn not only a classifier/regressor but also the best kernel for the training task, usually from a combination of existing kernel functions. Most MKL methods seek the combined kernel that performs best over every training example, sacrificing…
Exact causal network discovery is polynomial for sparse networks.
problem Finding the optimal causal Bayesian network from data is computationally hard.
method Pruning the search space using network properties, combined with dynamic programming and shortest-path searches.
result Exact discovery is polynomial for sparse causal Bayesian networks.
When using deep, multi-layered architectures to build generative models of data, it is difficult to train all layers at once. We propose a layer-wise training procedure admitting a performance guarantee compared to the global optimum. It is based on an optimistic proxy of future performance, the best latent marginal. W…
We propose an optimum mechanism for providing monetary incentives to the data sources of a statistical estimator such as linear regression, so that high quality data is provided at low cost, in the sense that the sum of payments and estimation error is minimized. The mechanism applies to a broad range of estimators, in…