Machine learning develops rapidly, which has made many theoretical breakthroughs and is widely applied in various fields. Optimization, as an important part of machine learning, has attracted much attention of researchers. With the exponential growth of data amount and the increase of model complexity, optimization met…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “machine learning optimization”
Optimal Transport enhances machine learning with new methods.
problem Comparing and manipulating probability distributions in machine learning.
method Probabilistic framework rooted in rich history and theory.
result New solutions in generative modeling and transfer learning.
This paper explores hyperparameter optimization for machine learning models.
problem Finding the best hyper-parameters for machine learning models.
method Introduces state-of-the-art optimization techniques and discusses their application.
result Comparison of different optimization methods on benchmark datasets.
Bayesian optimization outperformed random search in machine learning hyperparameter tuning challenge.
problem Optimizing hyperparameters of machine learning models using derivative-free methods.
method Bayesian optimization vs. random search on real datasets.
result Bayesian optimization significantly outperformed random search in held-out objective functions.
Quantum computers can speed up machine learning optimization problems.
problem Long computation times and high resource requirements for classical optimization algorithms in machine learning.
method Developed a mathematical model to leverage quantum parallelism for machine learning.
result Quantum machine learning applied to a 3D time-varying image demonstrated significant speedup.
Study uses machine learning to optimize power generation in electrical grids.
problem Optimizing power generation in electrical grids while respecting physical and engineering constraints.
method Two formulations of ACOPF as machine learning problems: direct prediction and constraint prediction.
result Validated machine learning approaches on two benchmark grids.
FanG-HPO optimizes machine learning models for fairness and low energy consumption.
problem Bias in machine learning models and high energy consumption in hyperparameter optimization.
method Combines multi-objective and multiple information source Bayesian optimization.
result FanG-HPO identifies fair and energy-efficient machine learning models.
Nonsmoothness in Machine Learning: specific structure, proximal identification, and applicationsmath.OC
This paper tackles nonsmooth optimization in machine learning.
problem Nonsmoothness in machine learning optimization problems.
method Identifying specific structures and leveraging them for practical applications.
result Compression, acceleration, and dimension reduction are possible with nonsmooth optimization.
OMLT combines ML and optimization for solving complex problems.
problem Solving complex decision-making problems in computer science and engineering.
method OMLT integrates neural networks and gradient-boosted trees into optimization problems using machine learning.
result OMLT seamlessly integrates with Pyomo and solves real-world problems.
This thesis explores optimization methods for high-dimensional machine learning problems.
problem High-dimensional optimization challenges in machine learning.
method Intuition and convergence proofs for stochastic gradient descent and momentum methods.
result Explanation of why common machine learning optimization methods are successful.
Quantum machine learning uses superposition to create a large ensemble of classifiers.
problem Improving machine learning efficiency on quantum computers.
method Using superposition to create an exponentially large ensemble of classifiers, trained with an optimization-free learning algorithm.
result Adding an optimization step improves the performance of quantum ensembles of classifiers.
This paper surveys the recent attempts, both from the machine learning and operations research communities, at leveraging machine learning to solve combinatorial optimization problems. Given the hard nature of these problems, state-of-the-art algorithms rely on handcrafted heuristics for making decisions that are other…
Amazon SageMaker AMT automates machine learning model tuning.
problem Challenging hyperparameter tuning for complex machine learning systems.
method Gradient-free optimization using random search or Bayesian optimization.
result AMT finds the best hyperparameter configurations for models.
Paper proposes a machine learning framework for VLSI mask optimization.
problem Costly VLSI mask optimization due to complex processes.
method Heterogeneous OPC framework using machine learning.
result Demonstrates efficiency and effectiveness of the proposed framework.
Machine learning factors outperform traditional portfolio optimization methods.
problem Comparing machine learning and traditional portfolio optimization methods.
method Examined machine learning and factor-based portfolio optimization using autoencoder neural networks and dimensionality reduction techniques.
result Minimum-variance portfolios using latent factors derived from autoencoders and sparse methods outperform simpler benchmarks in risk minimization.
Machine learning improves learning and memory retention by optimizing study sessions.
problem Improving learning and memory retention methods for factual material.
method Large-scale randomized controlled trial with machine learning optimization of study sessions.
result Study sessions optimized with machine learning lead to 67% longer retention and 50% higher return rate.
This paper introduces multi-objective hyperparameter optimization in machine learning.
problem Optimizing machine learning pipelines for multiple objectives, not just accuracy.
method Survey of optimization strategies and applications in multi-objective hyperparameter optimization.
result The importance and utility of multi-objective hyperparameter optimization in applied machine learning.
Enhances inverse design optimization with machine learning and reduced fidelity simulations.
problem Limited compute resources in inverse design optimization.
method Synergy of multi-fidelity simulations, machine learning, and search space reduction.
result Significant computational resource savings and improved optimization performance.
This paper reviews zeroth-order optimization in signal processing and machine learning.
problem Optimization problems without gradient information.
method Iterative steps: gradient estimation, descent direction computation, solution update.
result Demonstrates applications in robustness evaluation and black-box model explanations.
This paper improves combine harvester performance using ANN-PSO hybrid model.
problem Improving performance of combine harvesters to minimize waste and reduce maintenance.
method Proposes a hybrid machine learning model combining artificial neural networks and particle swarm optimization.
result Demonstrates higher accuracy and stability in predicting optimal performance of combine harvesters.
Examines challenges and proposes new approaches in machine learning theory.
problem Challenges in machine learning as a function approximation and optimization.
method Mathematical analysis of gradient descent, fixed network limitations, and RNNs.
result New insights and mathematical approaches to improve machine learning.
The paper connects optimization and generalization using a new gradient inequality.
problem Connecting optimization dynamics to generalization bounds in machine learning.
method The approach uses the Łojasiewicz gradient inequality to derive convergence rates and generalization bounds.
result The framework provides generalization estimates matching or extending previous results for various models.
Sherpa automates hyperparameter tuning for machine learning models.
problem Hyperparameter optimization for computationally expensive models.
method Powerful and interchangeable algorithms, interactive dashboard.
result Automates tedious aspects of model tuning.
We propose a clustering-based iterative algorithm to solve certain optimization problems in machine learning, where we start the algorithm by aggregating the original data, solving the problem on aggregated data, and then in subsequent steps gradually disaggregate the aggregated data. We apply the algorithm to common m…
Uber optimizes marketplace levers using machine learning to improve resource allocation efficiency.
problem Optimizing budget allocation for drivers and riders to maximize business value.
method End-to-end machine learning and optimization procedure using feature store, model training, and ADMM.
result Substantially improved Uber's resource allocation efficiency through high-dimensional optimization.
Deep learning models optimize protein sequences.
problem Optimizing protein properties through sequence design.
method Deep generative models guided by machine learning.
result Improved protein sequence generation from prior knowledge.
New approach to machine learning optimization using distribution space.
problem Optimization challenges in machine learning with non-convex constraints.
method Relaxation to convex optimization in distribution space, numerical algorithm based on mixture distributions.
result Approximate optimization in distribution space is consistent and effective.
L2O uses ML to optimize traditional optimization techniques.
problem Real-world optimization problems with shared structures.
method Exploiting shared structures to enhance optimization techniques.
result Better or faster solutions through machine learning integration.
I describe an optimal control view of adversarial machine learning, where the dynamical system is the machine learner, the input are adversarial actions, and the control costs are defined by the adversary's goals to do harm and be hard to detect. This view encompasses many types of adversarial machine learning, includi…
GAMA is a user-friendly AutoML system for machine learning pipeline optimization.
problem Empowering users to control and optimize machine learning pipelines.
method Modular AutoML system with three search algorithms and two post-processing steps.
result GAMA allows users to track and control the search process for optimal machine learning pipelines.
New regularizer for machine learning using private data.
problem Machine learning with private data.
method Distributionally-robust optimization with locally-differentially-private datasets.
result New regularizer for training linear regression models.
Adaptive methods optimize machine learning training faster.
problem Non-adaptive stochastic optimization requires tuning for each application.
method Develop adaptive stochastic optimization methods.
result Adaptive methods offer computational savings for large-scale systems.
Charities can increase donations by targeting optimal recipients.
problem Ineffective fundraising leads to lower resources for goods.
method Combines field experiment and causal machine-learning approach.
result Machine-learning-based optimal targeting increases donations significantly.
Surveying machine learning for solving graph optimization problems.
problem Solving combinatorial optimization problems on graphs requires algorithmic engineering.
method Surveying machine learning approaches for graph optimization.
result Machine learning offers new ways to solve graph optimization problems.
New approach to bilevel optimization for machine learning using functional methods.
problem Solving bilevel optimization problems in machine learning, especially with over-parameterized neural networks.
method Functional point of view, scalable and efficient algorithms for functional bilevel optimization.
result Demonstrates benefits of functional approach on instrumental regression and reinforcement learning tasks.
pyLOT library simplifies machine learning on 3D point clouds via linearized optimal transport.
problem Performing machine learning tasks on 3D point clouds.
method Linearized optimal transport (LOT) to embed distributions into Hilbert space, enabling linear machine learning.
result Downstream tasks on embedded representations are simplified to linear operations.
Graph machine learning lacks a balanced theory, focusing on expressive power and optimization.
problem Insufficient theoretical understanding of GNNs' generalization behavior.
method Develop a balanced theory focusing on expressive power, generalization, and optimization.
result Theoretical advancements need to align with practical success in graph machine learning.
Lecture notes on optimization for machine learning, derived from a course at Princeton University and tutorials given in MLSS, Buenos Aires, as well as Simons Foundation, Berkeley.
FiberNet integrates geometry into machine learning for clearer classification.
problem Lack of interpretability in traditional deep learning.
method Reformulates classification as geometric optimization on fiber bundles, introducing learnable Riemannian metrics and variational prototype optimization.
result Clear geometric interpretability and efficiency in classification.
Framework optimizes model performance and interpretability for tabular data.
problem Balancing model performance and interpretability in machine learning models.
method Model-agnostic multi-objective optimization framework with evolutionary algorithm.
result Framework generates diverse models that trade off performance and interpretability efficiently.
Green Machine Learning via Augmented Gaussian Processes and Multi-Information Source Optimizationcs.LG
A new method reduces energy consumption in machine learning by using multiple, less costly data sources.
problem High computational and energy costs in machine learning model training.
method Augmented Gaussian Process (AGP-MISO) with multi-source optimization.
result The AGP-MISO method reduces computational time and energy consumption compared to traditional approaches.
Survey of techniques for automating hyperparameter optimization in machine learning.
problem Finding optimal hyperparameters for machine learning algorithms.
method Review of various hyperparameter optimization techniques.
result Unified treatment of hyperparameter optimization techniques.
We propose an algorithm for a family of optimization problems where the objective can be decomposed as a sum of functions with monotonicity properties. The motivating problem is optimization of hyperparameters of machine learning algorithms, where we argue that the objective, validation error, can be decomposed as mono…
This study analyzes and optimizes hyperparameters for machine learning models.
problem Optimizing hyperparameters for machine learning models to improve performance.
method Experimental analysis of 30 hyperparameters from six machine learning algorithms using R and SPOT.
result A new consensus ranking method for analyzing results from multiple algorithms.
Automated machine learning has gained a lot of attention recently. Building and selecting the right machine learning models is often a multi-objective optimization problem. General purpose machine learning software that simultaneously supports multiple objectives and constraints is scant, though the potential benefits …
Bayesian Optimization improves machine learning for detecting network attacks.
problem Detecting network attacks to secure critical information.
method Bayesian Optimization for tuning machine learning algorithms (SVM-RBF, RF, k-NN).
result The proposed framework achieves high accuracy and low false alarms.
SMAC3 optimizes machine learning hyperparameters efficiently.
problem Optimizing hyperparameters for machine learning algorithms.
method Bayesian Optimization framework with facades for various use cases.
result Improves performance with minimal evaluations.
Myriad offers a testbed for integrating machine learning and trajectory optimization.
problem Real-world trajectory optimization challenges.
method JAX-based testbed with machine learning and trajectory optimization integration.
result End-to-end learning and planning with neural ODEs.