Paper proves existence and computation of Risk Budgeting portfolios.
problem Challenges to mean-variance framework sensitivity.
method Mathematical proofs and stochastic algorithms for risk measures.
result Existence and uniqueness of Risk Budgeting portfolios for various risk measures.
This work makes neural sequence models more efficient by controlling computation.
problem Fixed compute for all examples in neural networks.
method Conditional computation to adapt compute to example complexity.
result Conditional Computation Transformer (CCT) improves efficiency and performance.
Estimates model performance from compute budget for distillation.
problem Risk mitigation in large-scale distillation.
method Distillation scaling law based on compute budget allocation.
result Maximizes student performance with compute-optimal allocation.
Due to concerns about human error in crowdsourcing, it is standard practice to collect labels for the same data point from multiple internet workers. We here show that the resulting budget can be used more effectively with a flexible worker assignment strategy that asks fewer workers to analyze easy-to-label data and m…
Proposes a method to allocate time budgets in mixed criticality systems.
problem Managing execution time variability in mixed criticality systems.
method Quantifies execution time variability using statistical dispersion parameters and proposes a heuristic to allocate time budgets.
result The proposed heuristic reduces the probability of exceeding allocated budgets.
Paper uses Mirror Descent for efficient risk budgeting portfolios.
problem Computing optimal risk budgeting weights for various risk measures.
method Employed Mirror Descent algorithms in deterministic and stochastic settings.
result Established convergence and quantitative rate for averaged Mirror Descent algorithm.
EERO optimizes resource usage for efficient classification.
problem Managing computational resources in complex machine learning models.
method EERO uses multiple classifiers with a reject option to adaptively shorten processing paths.
result EERO effectively manages budget allocation and enhances accuracy in complex scenarios.
Estimates model performance based on compute budget and evaluates stability over time.
problem Understanding how model performance evolves with compute budget and stability over time.
method Large-scale observational evaluations, prescriptive scaling laws, smoothed quantile regression, I-optimal sampling algorithm.
result Estimates attainable accuracies and stability of model performance over time.
Monte Carlo algorithms simulate some prescribed number of samples, taking some random real time to complete the computations necessary. This work considers the converse: to impose a real-time budget on the computation, which results in the number of samples simulated being random. To complicate matters, the real time t…
Paper simplifies DP composition for adaptive privacy budgets, enabling better privacy and accuracy in deep learning.
problem Tension between efficiency and flexibility in DP composition theorems.
method Rényi Differential Privacy (RDP) for adaptive privacy budgets, proving simpler composition theorem with smaller constants.
result Practical DP composition for adaptive privacy budgets, enabling better privacy and accuracy in deep learning.
Quantum computing optimizes ESG portfolios efficiently.
problem Optimizing investment portfolios with risk, return, and ESG considerations.
method Formulated discrete Markowitz portfolio theory (DMPT) for quantum annealers, incorporating ESG ratings.
result Discrete portfolios converge to continuous solutions as budgets increase, outperforming traditional methods.
Framework ranks sectors influenced by Indian Union Budgets.
problem Real-time analysis of budgetary impacts on sector-specific equity performance.
method Fine-tuned embeddings and language models for sector identification and performance ranking.
result 0.997 NDCG score in predicting sector ranks based on post-budget performances.
We analyze general model selection procedures using penalized empirical loss minimization under computational constraints. While classical model selection approaches do not consider computational aspects of performing model selection, we argue that any practical model selection procedure must not only trade off estimat…
Study wSAA for contextual decisions, improving uncertainty quantification under computational constraints.
problem Uncertainty quantification limitations in wSAA for contextual stochastic optimization.
method Establish central limit theorems and asymptotic-normality-based confidence intervals for optimal costs.
result Over-optimizing can mitigate misspecification and preserve asymptotic normality, albeit at a slower convergence rate.
Deep learning models' architectures, including depth and width, are key factors influencing models' performance, such as test accuracy and computation time. This paper solves two problems: given computation time budget, choose an architecture to maximize accuracy, and given accuracy requirement, choose an architecture …
New OCBA procedures minimize PICS in robust R&S.
problem Selecting the best alternative under input uncertainty.
method Developed new asymptotically optimal OCBA procedures.
result Procedures minimize probability of incorrect selection.
Framework uses hindsight regret to audit marketing budget allocations.
problem Lack of principled way to assess strategic budget allocations.
method Hindsight regret framework based on constraint-faithful benchmark.
result Identifies practical trade-off between allocation flexibility and detectability.
New method optimizes privacy and compute trade-offs for deep learning.
problem Privacy and compute trade-offs in deep learning training.
method Decoupling privacy analysis and experimental behavior, using TAN and scaling laws for DP-SGD.
result Stronger privacy guarantees with significant reduction in computational budget.
New algorithm reduces clustering cost in bandit feedback.
problem Discovering hidden groupings in stochastic bandit feedback.
method ACB algorithm for active clustering with non-asymptotic lower bound.
result ACB algorithm's budget matches lower bound in most regimes.
Private cancer prediction model trained on federated genomic data.
problem Train a private cancer prediction model on federated genomic data.
method Differentially private federated learning (FL) for genomic cancer prediction.
result Ranked 3rd in a competition for private cancer prediction.
SODA optimizes data augmentation allocation for deep learning models.
problem Inefficient allocation of data augmentation budget in deep neural networks.
method Online learning to dynamically allocate data augmentation budget during training.
result Optimized data augmentation can save computation time and promote greener machine learning.
Framework detects and mitigates data-poisoning attacks in causal effect estimation.
problem Vulnerability to append-only attacks in observational causal analyses.
method Develops a data-poisoning audit for augmented inverse-probability-weighted estimation.
result Proposes a greedy scan to compute exact worst-case movement at every append budget.
Engineering problems often involve data sources of variable fidelity with different costs of obtaining an observation. In particular, one can use both a cheap low fidelity function (e.g. a computational experiment with a CFD code) and an expensive high fidelity function (e.g. a wind tunnel experiment) to generate a dat…
We present analytical investigations of a multiplicative stochastic process that models a simple investor dynamics in a random environment. The dynamics of the investor's budget, x(t), depends on the stochasticity of the return on investment, r(t), for which different model assumptions are discussed. The fat-tail d…
This paper extends financial theory to measure learnable market structure under computational constraints.
problem Understanding learnable market structure under bounded computational capacity.
method Introduces financial epiplexity as a measure of learnable market structure, extending classical information theory.
result Proves that equal entropy does not imply equal epiplexity and derives thresholds for useful regimes.
BNRE improves simulation-based inference by producing more conservative posteriors.
problem Overconfident posteriors from current simulation-based inference algorithms risk false inferences.
method Balanced Neural Ratio Estimation (BNRE) that produces more conservative posterior approximations.
result BNRE produces more conservative posterior surrogates on all tested benchmarks and simulation budgets.
This paper uses robust optimization to analyze supply chain resilience.
problem Supply chain resilience analysis of multi-modal logistics networks.
method Robust optimization with budget-of-uncertainty.
result Interactive effects of network size, disruption scale, and degree on resilience.
LinearAPT optimizes decision-making under resource constraints for a linear threshold problem.
problem Optimizing sequential decisions with a linear threshold under resource limitations.
method LinearAPT, an adaptive algorithm for fixed-budget TLB problem.
result LinearAPT achieves theoretical upper bounds and robust performance on various datasets.
Best-of-∞ improves LLM performance by efficiently allocating inference-time computation.
problem Achieving optimal performance in test-time LLM ensembling with infinite budget.
method Adaptive generation scheme and weighted ensembles of LLMs, formulated as mixed-integer linear program.
result Optimal ensemble weighting improves performance over individual models.
The aim of black-box optimization is to optimize an objective function within the constraints of a given evaluation budget. In this problem, it is generally assumed that the computational cost for evaluating a point is large; thus, it is important to search efficiently with as low budget as possible. Bayesian optimizat…
WSD schedule improves model training efficiency by adapting learning rates dynamically.
problem Fixed compute budgets limit training efficiency of language models.
method Introduces a WSD schedule that uses a constant learning rate followed by a rapid decay phase.
result WSD schedule generates a non-traditional loss curve with stable and decay phases.
In this paper we first identify a basic limitation in gradient descent-based optimization methods when used in conjunctions with smooth kernels. An analysis based on the spectral properties of the kernel demonstrates that only a vanishingly small portion of the function space is reachable after a polynomial number of g…
Memory bandwidth bottleneck is a major challenges in processing machine learning (ML) algorithms. In-memory acceleration has potential to address this problem; however, it needs to address two challenges. First, in-memory accelerator should be general enough to support a large set of different ML algorithms. Second, it…
Synthetic tabular data synthesis models balance utility and risk.
problem Generating synthetic tabular data for regulated domains.
method Latent flow models with various learning targets, paths, and sampling methods.
result Velocity and posterior matching objectives yield higher utility, while score and noise matching achieve lower risk.
This paper compares self-reflection and budget tuning for LLMs, revealing domain-specific performance gains.
problem Improving inference-time performance of LLMs without retraining, balancing quality, cost, and latency.
method Systematic comparison of self-reflection and budget tuning across mathematical reasoning and translation tasks, evaluating various LLMs and model families.
result Substantial domain-dependent variation in self-reflection effectiveness, with up to 220% performance gains in mathematical reasoning.
A new method for active learning works well across all label budgets.
problem Active learning methods perform poorly in both low and high label budgets.
method Uncertainty Herding: a simple, computationally fast method that optimizes uncertainty coverage.
result Uncertainty Herding nearly optimizes distribution-level coverage and performs well across various active learning tasks.
HAMLET optimizes algorithm selection for machine learning tasks.
problem Limited time budgets and computational resources make traditional bandit approaches ineffective for automated algorithm selection.
method HAMLET incorporates learning curve extrapolation and time-awareness to select machine learning algorithms.
result HAMLET variants outperform other bandit-based strategies in experiments with recorded hyperparameter tuning traces.
This paper analyzes OCBA algorithms' convergence rates for DEDS optimization.
problem Optimizing discrete-event dynamic systems with limited computing resources.
method Characterizes convergence rates of two OCBA algorithms under different performance measures.
result OCBA algorithms achieve optimal convergence rates under probability of correct selection and expected opportunity cost measures.
Georgia needs a new budget code to manage fiscal policies effectively.
problem Weak and incomplete law on Budget System hinders fiscal policy implementation.
method Develop and adopt a new Budget Code with equal force as the Tax Code.
result Effective correlation between state, regional, and local budgets is crucial for social-economic development.
New method predicts and optimizes test-time scaling for LLMs.
problem Lack of principled guidance on scaling LLMs efficiently.
method Tail-guided search to predict and allocate compute.
result SLG Search achieves higher rewards with less compute.
GSSBO reduces GP fitting time in Bayesian optimization.
problem High computational cost of fitting Gaussian process surrogate models in Bayesian optimization.
method Gradient-based sample selection to reduce the number of samples used in GP fitting.
result Sublinear regret bounds and significant reduction in computational cost.
The log-determinant of a kernel matrix appears in a variety of machine learning problems, ranging from determinantal point processes and generalized Markov random fields, through to the training of Gaussian processes. Exact calculation of this term is often intractable when the size of the kernel matrix exceeds a few t…
Automated Budget Constrained Training optimizes model training under time constraints.
problem Balancing model quality and computational cost in constrained time.
method Developed a hyperparameter optimisation algorithm that learns the relationship between hyperparameters, model quality, and computational cost.
result The algorithm optimally decides whether to terminate or continue training, and what hyperparameters to use.
Budgeted deferral framework reduces expert query costs in machine learning.
problem High cost of querying all experts in deferral algorithms.
method Introduces budgeted deferral framework for selective expert querying.
result Empirical results show substantial reduction in training costs without sacrificing accuracy.
We present a dual subspace ascent algorithm for support vector machine training that respects a budget constraint limiting the number of support vectors. Budget methods are effective for reducing the training time of kernel SVM while retaining high accuracy. To date, budget training is available only for primal (SGD-ba…
Efficiently simulates risk budgeting portfolios using novel algorithms.
problem Estimating risk contributions in portfolios efficiently.
method Cutting planes algorithm, specialised SGD for Expected Shortfall, numerical simulations.
result Outperforms standard convex optimisation solvers in estimating risk budgeting portfolios.
Bridges uplift modeling and sequential decision-making with online budget allocation.
problem Treatment allocation under budget constraints in digital advertising.
method Budget-Constrained Causal Bandits (BCCB) integrates learning, exploration, and budget pacing.
result Data-efficiency crossover: BCCB operates effectively from the first user, 3-5x lower performance variance.
A new algorithm reduces memory usage for deep learning models.
problem Training deep learning models requires significant memory.
method Dynamic Tensor Rematerialization (DTR) is a greedy online algorithm that dynamically plans recomputations.
result DTR achieves comparable performance to optimal static checkpointing with only a small memory budget.