Fully automating machine learning pipelines is one of the key challenges of current artificial intelligence research, since practical machine learning often requires costly and time-consuming human-powered processes such as model design, algorithm development, and hyperparameter tuning. In this paper, we verify that au…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,051 papers · 148 categories
Trend · papers per month
48 results for “Gradient-based Search”
Gradient-based feature selection for large datasets.
problem Feature selection for large datasets with high-order correlations.
method Iterative mini-batch calculation, discrete-to-continuous relaxation.
result Efficiently finds higher-order feature correlations in both N > D and N < D regimes.
Novel BSG method for efficient stochastic optimization.
problem Efficient optimization of non-convex surfaces in stochastic settings.
method Binary search combined with first order gradient optimization.
result BSG produces more promising results and better generalization than other methods.
New method reduces NAS search time and complexity.
problem High computational cost and complexity in NAS.
method Differentiable search space with annealing and pruning.
result Achieves 1.68% error on CIFAR-10 with 0.2 GPU days.
DrNAS improves neural architecture search with Dirichlet distribution and progressive learning.
problem Efficiently search for neural architectures with improved generalization and exploration.
method Formulates architecture search as a distribution learning problem using Dirichlet distribution and gradient-based optimization. Introduces a progressive learning scheme to handle large-scale tasks.
result Achieves state-of-the-art results on CIFAR-10 and ImageNet, demonstrating improved generalization and exploration.
Paper proposes MCTSPO for better reinforcement learning policy optimization.
problem Local optima and saddle points in gradient-based methods and poor initialization in gradient-free methods.
method Monte-Carlo tree search combined with gradient-free optimization.
result Improved performance on reinforcement learning tasks with deceptive or sparse reward functions.
A Bayesian approach to neural architecture search improves efficiency and accuracy.
problem Improving the efficiency and accuracy of neural architecture search.
method Formulating NAS from a Bayesian perspective, explicitly estimating the joint posterior distribution over architectures and weights, using Variational Dropout, and posterior-guided sampling.
result Posterior-guided NAS (PGNAS) achieves a good trade-off between precision and speed of search.
Efficiently infers neural architectures for unseen tasks.
problem High cost of neural architecture search.
method Gradient-based framework sharing information across tasks.
result Quick identification of good candidate architectures for new tasks.
Paper proposes a method to test and verify control systems with machine learning components.
problem Testing and verifying control systems with machine learning components is challenging.
method Gradient-based method combined with randomized search to find adversarial samples.
result Method outperforms Simulated Annealing optimization in finding adversarial samples.
VINNAS uses variational inference to avoid mode collapse in neural architecture search.
problem Mode collapse in gradient-based NAS methods, leading to suboptimal architectures.
method Differentiable variational inference with variational dropout and automatic relevance determination.
result State-of-the-art accuracy with up to twice fewer non-zero parameters.
MTL-NAS combines NAS with GP-MTL for task-agnostic multi-task learning.
problem Designing architectures for diverse tasks with varying priors.
method Disentangled GP-MTL networks, hierarchical feature sharing, and gradient-based search.
result General-purpose model trained once can adapt to multiple tasks.
PDNAS optimizes GNN architectures for diverse datasets.
problem Inadequate adaptability and combinatorial search space in GNNs.
method Dual architecture search (micro- and macro-architectures) with gradient-based optimization.
result PDNAS finds deeper GNNs with better performance on diverse datasets.
Gradient-based method prunes large models to create transferable architectures.
problem Creating transferable architectures from large models with limited fine-tuning data.
method Gradient-based algorithm for architecture pruning and subset selection.
result Successfully retrain architectures on new tasks with few fine-tuning data.
A new framework improves graph construction for semi-supervised learning.
problem Improving graph construction for better semi-supervised classification accuracy.
method Parallel hyperparameter search with adaptive resource allocation for gradient-based optimization of edge weights.
result Significantly outperforms existing graph construction schemes in accuracy and scalability.
Paper introduces a privacy-preserving line search method for optimization.
problem Optimization performance depends on step size tuning, which is difficult and privacy-sensitive.
method Introduces a stochastic adaptive line search algorithm that satisfies differential privacy.
result The algorithm efficiently uses privacy budget and outperforms existing private optimizers.
Proposes a gradient-based bilevel optimization method for efficient hyperparameter tuning.
problem Efficiently tuning hyperparameters in machine learning models.
method Gradient-based bilevel optimization approach.
result The proposed method is multiple times faster than existing techniques.
Planning problems are among the most important and well-studied problems in artificial intelligence. They are most typically solved by tree search algorithms that simulate ahead into the future, evaluate future states, and back-up those evaluations to the root of a search tree. Among these algorithms, Monte-Carlo tree …
Trained SPENs with efficient search in reward function for structured prediction.
problem Expensive ground-truth labeling in structured output prediction.
method Efficient truncated randomized search in reward function for training SPENs.
result Local improvements and effective supervision for SPENs without labeled data.
Adaptor 'E' extends gradient-based optimizers to explore loss landscapes, improving generalization.
problem Finding lower and better-generalizing minima in deep learning.
method Proposes an adaptor 'E' to extend gradient-based optimizers, encouraging exploration along landscape valleys.
result Adapted optimizers increase test accuracy by an average of 2.5% in large-batch training tasks.
SNAS efficiently searches neural architectures using stochastic optimization.
problem Efficiently searching for optimal neural architectures.
method SNAS trains parameters of both neural operations and architecture distribution in a single round of backpropagation, using a novel search gradient and locally decomposable rewards.
result SNAS achieves state-of-the-art accuracy with fewer training epochs compared to other NAS methods.
DSA efficiently allocates sparsity across layers for budgeted pruning.
problem Efficiently distributing resources (sparsity) across layers in pruning under resource constraints.
method DSA uses differentiable pruning to find continuous layer-wise pruning ratios via gradient-based optimization.
result DSA achieves superior performance and significantly reduces the time cost of pruning.
MetaNAS improves few-shot learning by optimizing neural architectures with meta-learning.
problem Few-shot learning challenges due to limited data and compute time.
method MetaNAS integrates NAS with gradient-based meta-learning to adapt neural architectures to new tasks efficiently.
result MetaNAS achieves state-of-the-art results on few-shot classification benchmarks.
Develops a robust, fast, and widely-applicable neural architecture search method.
problem Inability of current NAS methods to be easily applied to new problems.
method Adaptive stochastic natural gradient method for simultaneous optimization of weights and architecture.
result Near state-of-the-art performances with low computational budgets.
Enhances neural architecture search efficiency and prevents performance collapse.
problem Improving memory efficiency and preventing performance collapse in neural architecture search.
method Employing continuous relaxation strategy and gradient-based optimization for over-parameterized BCNN construction, introducing Confident Learning Rate and partial channel connections.
result NAS-v2 delivers state-of-the-art search efficiency on CIFAR-10 and ImageNet.
A new NAS framework optimizes 3D medical image segmentation architectures.
problem Optimizing neural architectures for high-resolution 3D medical images.
method Stochastic sampling algorithm for scalable gradient-based optimization of neural connectivities and operation types in both encoder and decoder.
result Automatically designed architecture outperforms human-designed U-Net.
New hierarchical search algorithm improves neural architecture design across different operator sets.
problem DARTS's performance drops when search space changes due to operator correlation and optimization complexity.
method Operator clustering and optimization complexity matching in a hierarchical search algorithm.
result The algorithm consistently finds high-performance architectures across various search spaces, outperforming other methods.
Survey of techniques for automating hyperparameter optimization in machine learning.
problem Finding optimal hyperparameters for machine learning algorithms.
method Review of various hyperparameter optimization techniques.
result Unified treatment of hyperparameter optimization techniques.
Paper stabilizes DARTS algorithm for better neural architecture search.
problem Weak stability of DARTS algorithm leading to unreliable results.
method Amended gradient estimation method to bridge optimization gap.
result Significant improvement in search stability and larger search spaces explored.
New distance metric for neural architecture search reduces search space complexity.
problem Reducing the complexity of neural architecture search.
method Fisher task distance for measuring task similarity and online neural architecture search.
result Reduced search space complexity for task-specific architectures.
New algorithms improve neural architecture search with faster convergence.
problem Improving efficiency and accuracy of neural architecture search.
method Geometry-aware gradient algorithms to optimize continuous relaxation of discrete search spaces.
result Exceeds state-of-the-art results on CIFAR and ImageNet benchmarks.
FDS tackles long horizon hyperparameter optimization issues.
problem Memory scaling and gradient degradation in long horizon tasks.
method Forward-mode differentiation with sharing (FDS).
result Significantly outperforms greedy gradient-based alternatives.
New robustness attacks improve evaluation of neural networks.
problem Difficulty in evaluating robustness of neural networks.
method Gradient-based adversarial attacks that are more reliable and efficient.
result Developed attacks are more reliable and efficient than existing methods.
A new activation function k-WTA improves neural network defenses against adversarial attacks.
problem Improving neural network robustness against gradient-based adversarial attacks.
method Proposes k-Winners-Take-All activation function and analyzes its effectiveness.
result k-WTA activation significantly enhances neural network robustness against adversarial attacks.
New method learns DAGs from data using neural networks.
problem Learning directed acyclic graphs from observational data.
method Adapting a continuous constrained optimization formulation to neural networks.
result New method outperforms existing continuous methods on most tasks.
VaSST uses soft symbolic trees for probabilistic symbolic regression.
problem Efficiently recover symbolic expressions from noisy data.
method Variational inference with soft symbolic trees.
result Superior performance in structural recovery and predictive accuracy.
The paper proposes a gradient-based method for multi-penalty Ridge regression.
problem Optimizing multiple regularization hyperparameters for linear regression.
method Gradient-based optimization through matrix differential calculus.
result The method outperforms traditional regularization techniques like LASSO and Ridge.
Bayesian optimization improves policy search in reinforcement learning.
problem Finding optimal policies with high variance estimates from random samples.
method Develops an algorithm combining Bayesian optimization and policy gradients.
result Improves sample complexity and reduces variance in empirical evaluations.
Credit assignment in Meta-reinforcement learning (Meta-RL) is still poorly understood. Existing methods either neglect credit assignment to pre-adaptation behavior or implement it naively. This leads to poor sample-efficiency during meta-training as well as ineffective task identification strategies. This paper provide…
Introduces GAMPS for better model-based policy learning.
problem Misspecified model classes lead to poor policy estimates.
method Exploits current policy to learn approximate transition model, focusing on relevant parts of the environment.
result Empirically validated GAMPS on benchmark domains, demonstrating improved properties.
In this paper we study convex stochastic search problems where a noisy objective function value is observed after a decision is made. There are many stochastic search problems whose behavior depends on an exogenous state variable which affects the shape of the objective function. Currently, there is no general purpose …
In this paper, we propose to combine imitation and reinforcement learning via the idea of reward shaping using an oracle. We study the effectiveness of the near-optimal cost-to-go oracle on the planning horizon and demonstrate that the cost-to-go oracle shortens the learner's planning horizon as function of its accurac…
During recent years there has been an increased interest in stochastic adaptations of limited memory quasi-Newton methods, which compared to pure gradient-based routines can improve the convergence by incorporating second order information. In this work we propose a direct least-squares approach conceptually similar to…
A new hyperparameter optimization method reduces overfitting.
problem Overfitting in hyperparameter optimization.
method PAC-Bayes bound minimization using gradient-based algorithm.
result Significant reduction in out-of-sample error.
Interval attacks find more adversarial examples than existing methods.
problem Evaluating robustness of adversarially trained neural networks against unknown attacks.
method Symbolic interval propagation for bound over-approximation and gradient-guided attacks.
result Interval attacks find on average 47% more violations than state-of-the-art methods.
FrostNet improves INT8 quantization efficiency in mobile networks.
problem The importance of network architecture for optimal INT8 quantization.
method Quantization-aware training (QAT) with StatAssist and GradBoost, hardware-aware NAS.
result FrostNets achieve higher recognition accuracy with comparable latency when quantized.
Time series prediction with deep learning methods, especially long short-term memory neural networks (LSTMs), have scored significant achievements in recent years. Despite the fact that the LSTMs can help to capture long-term dependencies, its ability to pay different degree of attention on sub-window feature within mu…
The paper proposes a method to learn hyperparameters without validation sets, improving efficiency and accuracy.
problem Training large models on limited data to avoid overfitting and reduce validation set usage.
method Gradient-based learning of hyperparameters via a data-emphasized evidence lower bound (ELBO) objective.
result The data-emphasized ELBO reduces hyperparameter search time from 88+ hours to under 3 hours while maintaining comparable accuracy.
Square Attack efficiently attacks deep models with random updates.
problem Efficiently attacking deep learning models without gradient information.
method Randomized search of localized square updates.
result Significantly more query-efficient and higher success rate compared to state-of-the-art methods.