Lapse improves parameter servers by dynamically allocating parameters, achieving near-linear scaling.
problem Efficiently managing distributed training with reduced communication overhead.
method Integrate dynamic parameter allocation into parameter servers, proposing Lapse.
result Lapse provides near-linear scaling and can be orders of magnitude faster than existing parameter servers.
Combines NES and PPO to enhance exploration in various environments.
problem Improving exploration in reinforcement learning environments.
method Parameter transfer and parameter space noise methods for combining NES and PPO.
result PPO benefits from both NES methods in discrete and continuous control tasks.
Deep learning estimates time-varying Markov model parameters.
problem Estimating time-dependent parameters in Markov models.
method Reframes parameter estimation as an optimization problem using maximum likelihood.
result Real solution close to SDE with neural network-derived parameters under specific conditions.
Training-free model learns SDE dynamics without training, accelerating parameter studies.
problem High computational cost of simulating parameter-dependent SDEs.
method Training-free conditional diffusion model with joint kernel-weighted Monte Carlo estimator.
result Accurate approximation of conditional distributions across varying parameter values.
Dual Bayesian Affine Estimators for Wiener-type state-space models
problem Estimating parameters in Wiener-type state-space models
method Fixed-point architecture combining two affine estimators
result Dual basis-parameter estimator achieves comparable parameter MSE to purely affine estimator
New method achieves optimal performance without needing problem parameters.
problem Parameter-free stochastic optimization in non-convex and convex settings.
method Simple hyperparameter search technique for non-convex setting, and method with stochastic gradients for convex setting.
result Fully parameter-free methods can outperform state-of-the-art algorithms in both non-convex and convex settings.
A new method transfers parameters in ELM networks using projective model.
problem Parameter transfer in extreme learning machine networks.
method Projective model to bridge source and target model parameters, L2,1-norm penalty for joint feature selection and parameter transfer.
result Significantly outperforms non-transfer ELM networks and other methods.
Study shows how numerical discretization affects reconstructions and parameter distributions in nano metrology.
problem Impact of numerical discretization on parameter reconstructions and model parameter distributions.
method Bayesian target vector optimization, finite element model, Gaussian process, stochastic machine learning surrogate models, Markov chain Monte Carlo sampler.
result Numerical discretization parameters impact the accuracy and distribution of reconstructed model parameters.
Study examines the cost of tuning hyper-parameters in regression models.
problem Estimating optimal hyper-parameters for regression models.
method Established finite-sample oracle inequalities for hyper-parameter selection.
result Generalization error of selected models shrinks at nearly a parametric rate.
Develops a parameter-free SGD algorithm with optimal convergence rate.
problem Optimizing parameters in stochastic convex optimization.
method A novel parameter-free algorithm for SGD with high-probability guarantees and adaptive properties.
result Achieves optimal convergence rate with only a double-logarithmic factor increase compared to known-parameter settings.
Algorithm learns optimal parameters from infinite space for computational resource optimization.
problem Finding nearly-optimal parameters from an infinite space of tunable parameters.
method Learn a finite set of promising parameters from an infinite set using a data-independent discretization approach.
result Algorithm can help compile a configuration portfolio or select input to a configuration algorithm for finite parameter spaces.
Deep learning outperforms traditional methods in estimating OU process parameters.
problem Parameter estimation of the Ornstein-Uhlenbeck process is challenging.
method Used a multi-layer perceptron to estimate OU process parameters compared to traditional methods like Kalman filter and maximum likelihood estimation.
result Deep learning method outperforms traditional methods in parameter estimation of the OU process.
Bayesian classification and regression with high order interactions is largely infeasible because Markov chain Monte Carlo (MCMC) would need to be applied with a great many parameters, whose number increases rapidly with the order. In this paper we show how to make it feasible by effectively reducing the number of para…
Stage-based hyper-parameter optimization reduces GPU-hours and training time.
problem Efficiently executing hyper-parameter optimization for deep learning models.
method Stage-based execution strategy to remove redundant computations.
result Stage-based execution outperforms trial-based method by up to 6.60 times in GPU-hours and 4.13 times in training time.
Two approaches improve parameter learning in various mixture models.
problem Parameter learning in mixture models.
method Complex-analytic and algebraic-combinatorial methods.
result Improved sample sufficiency for parameter estimation in specific mixture models.
Paper proposes a reinforcement learning framework for efficient hyper-parameter tuning of stochastic optimization algorithms.
problem Efficient tuning of hyper-parameters for stochastic optimization algorithms.
method Modeling hyper-parameter tuning as a Markov decision process and using policy gradient algorithms.
result The proposed framework significantly reduces the time required for hyper-parameter tuning compared to Bayesian optimization.
FNFs model parameter-dependent densities by combining a fixed flow with a polynomial parameter-dependent transformation.
problem Learning a separate flow for every parameter configuration is intractable.
method Factorizable Normalizing Flows (FNFs) represent the parameter-dependent density as a fixed flow for a reference configuration and a learnable polynomial transformation factorized over parameters.
result FNFs enable the recovery of the combined effect of multiple parameters without sampling their joint space, providing a scalable and interpretable solution.
Generative Adversarial Networks optimize model parameters for image matching.
problem Optimizing model parameters for accurate image matching.
method Model-Assisted Generative Adversarial Network (GAN) to produce fake images matching true images.
result Best match model parameter values can minimize bias in image recognition.
ThriftyNet uses a single convolutional layer recursively to maximize parameter usage.
problem Maximizing the use of parameters in deep convolutional neural networks.
method A single convolutional layer is used recursively, with normalization, non-linearities, downsampling, and shortcuts to maintain model expressivity.
result ThriftyNet achieves competitive performance with significantly fewer parameters.
Adapters add few trainable parameters per task, improving NLP performance.
problem Parameter inefficiency in fine-tuning large pre-trained models for multiple downstream tasks.
method Adapter modules that add only a few trainable parameters per task, allowing for high parameter sharing and task extensibility.
result Adapters achieve near state-of-the-art performance with minimal additional parameters.
In this paper, we introduce a new parameter, the affine twist parameter for the affine deformation of a sphere with holes. We show that the affine deformation space can be parametrized by Margulis invariants and affine twist parameters. The affine twist parameter is canonically regarded as a correspondence to the Fench…
A new method estimates parameters of complex models using ordinary least squares.
problem Estimating parameters of nonlinear dynamic models from time series data.
method Physics-Informed Regression (PIR) using regularized ordinary least squares.
result PIR outperforms physics-informed neural networks (PINN) in parameter estimation.
Bayesian approach labels segment parameters for improved change detection.
problem Improving Bayesian change point detection by leveraging segment parameter patterns.
method Proposes a Bayesian mean-shift change point detection algorithm with a Dirichlet process prior for segment class labels.
result Enhanced performance in synthetic and real-world data.
A new method for multi-task learning by allocating parameters.
problem Sharing parameters between unrelated tasks can hurt performance.
method Learned binary variables to allocate components to tasks, encouraging sharing between related tasks.
result Achieves a 17% relative reduction of the error rate on Omniglot benchmark.
This paper proposes an efficient autoHPO method based on data-to-hyper-parameter mapping.
problem Manual hyper-parameter tuning is costly and dependent.
method The approach is based on mapping from data to hyper-parameters using a sophisticated network structure and effective construction algorithms.
result The proposed approach significantly outperforms state-of-the-art methods.
Empirical study shows removing neural parameter symmetries impacts model performance.
problem Understanding the impact of neural parameter symmetries on model performance.
method Developed two methods to reduce parameter space symmetries in neural networks.
result Removing parameter symmetries can lead to faster and more effective Bayesian neural network training.
Supervised learning is an active research area, with numerous applications in diverse fields such as data analytics, computer vision, speech and audio processing, and image understanding. In most cases, the loss functions used in machine learning assume symmetric noise models, and seek to estimate the unknown function …
The paper defines and studies canonical parameters on surfaces in 4D space.
problem Understanding surfaces in 4D space without minimal points.
method Defining and proving existence of canonical principal parameters.
result Surfaces in 4D space are uniquely determined by four functions satisfying partial differential equations.
Bayesian Optimization improves Neural Network parameter tuning for XOR function.
problem Optimizing model parameters for neural networks, especially for large models.
method Bayesian Optimization with Gaussian Process Priors.
result Achieved higher prediction accuracy for the XOR function.
New findings on Malgrange-Galois groupoid for Painlevé VI equation parameters.
problem Understanding transformations preserving specific forms for Painlevé VI equation.
method Computed Malgrange-Galois groupoid for Painlevé VI family with all parameters.
result Solutions of Painlevé VI do not satisfy new partial differential equations.
Deep neural networks with specific parameter sets can approximate smooth functions efficiently.
problem Approximating smooth functions with deep neural networks.
method Deep neural networks with ReLU activation and specific parameter sets {0,±21,±1,2} are used to approximate Cβ-smooth functions. result The constructed networks can approximate Cβ-smooth functions with parameters {0,±21,±1,2} efficiently, achieving the same convergence rate as sparse networks with parameters in [−1,1]. NanoFlow reduces parameter complexity in normalizing flows.
problem Efficient parameter complexity in flow-based models.
method Single neural density estimator with flow indication embedding.
result Sublinear parameter complexity achieved.
Paper proposes a new algorithm for efficient hyper-parameter optimization.
problem Efficient hyper-parameter tuning for machine learning models.
method Information geometric optimization with stochastic natural gradient for discrete search domains.
result The proposed algorithm achieves faster optimization than existing methods without manual tuning.
A new method quantizes LSTM gate parameters without performance loss.
problem Quantization loss in LSTM gate parameters without performance degradation.
method Lossy quantization of gate parameters during training, weight parameters adjust to offset quantization loss.
result F1 score decreased by only 0.7% on Named Entity Recognition dataset.
A new method assigns hidden parameters deterministically to improve learning efficiency.
problem Traditional learning methods struggle with high computational burden.
method Two-stage learning with deterministic assignment of hidden parameters.
result Deterministic assignment of hidden parameters almost matches traditional learning's generalization performance.
Study optimizes sensor placement for accurate parameter estimation in complex systems.
problem Challenges in parameter estimation with limited or noisy data.
method Physics-Informed Neural Networks (PINNs) for optimal sensor placement and parameter estimation.
result PINNs-based framework achieves higher accuracy in parameter estimation compared to random sensor placements.
New intervals improve confidence in selected parameters.
problem Deceptive optimism in reported uncertainties for selected parameters.
method Constructing simultaneous over the selected (SoS) error rate controlling confidence intervals.
result New intervals improve coverage probability for selected parameters.
Deep learning used for parameter estimation in hard-to-infer models.
problem Parameter estimation in intractable models like max-stable processes.
method Train deep neural networks on simulated data to estimate parameters.
result Deep learning provides accurate and faster parameter estimation.
Bayesian active learning tackles nuisance parameters, leading to bias and dilemmas.
problem Bayesian active learning with nuisance parameters leads to bias and dilemmas.
method Characterizes and mitigates negative interference by accurately estimating nuisance parameters.
result The extent of negative interference can be extremely large, and accurate estimation of nuisance parameters is critical.
Improves parameter selection for denoising with elastic net.
problem Challenges in selecting optimal parameters for denoising.
method Combines statistical learning theory and regularisation theory to approximate optimal elastic net parameters.
result Explicit error bounds on accuracy of approximated parameter and regularisation solution.
We address the problem of parameter estimation in models of systems biology from noisy observations. The models we consider are characterized by simultaneous deterministic nonlinear differential equations whose parameters are either taken from in vitro experiments, or are hand-tuned during the model development process…
Hippo optimizes deep learning hyper-parameters by reducing redundant trials.
problem Redundant hyper-parameter trials in hyper-parameter optimization.
method Hippo breaks down hyper-parameter sequences into stages and executes them in a tree structure.
result Hippo reduces GPU-hours and training time significantly compared to existing methods.
This work improves Bayesian Optimization for setting DNN hyper-parameters.
problem Manual setting of DNN hyper-parameters is error-prone and computationally expensive.
method Combines Bayesian Optimization with tuning rules to reduce search space and improve accuracy.
result Improves efficiency and accuracy of hyper-parameter tuning for deep neural networks.
We study the Euler-Lagrange equations for a parameter dependent G-invariant Lagrangian on a homogeneous G-space. We consider the pullback of the parameter dependent Lagrangian to the Lie group G, emphasizing the special invariance properties of the associated Euler-Poincaré equations with advected parameters.
Study on measuring vulnerability of neural network parameters via corruption.
problem Understanding the robustness and generalization of deep neural networks.
method Proposes an indicator to measure parameter robustness via parameter corruption and provides a gradient-based estimation.
result Demonstrates the effectiveness of the proposed indicator and training method in improving parameter robustness and accuracy.
We consider the bridge linear regression modeling, which can produce a sparse or non-sparse model. A crucial point in the model building process is the selection of adjusted parameters including a regularization parameter and a tuning parameter in bridge regression models. The choice of the adjusted parameters can be v…
Introduces a new stationary GE-process for gold price analysis.
problem Analyzing gold price data with a flexible stationary process.
method Developed a new stationary GE-process with three parameters. Analyzed synthetic and real gold price data.
result Maximum likelihood estimators can be obtained for the unknown parameters.
FlexEncoder analyzes DAE parameters' impact on recommender systems.
problem Varying parameter settings lead to different performance in DAE-based recommender systems.
method Built FlexEncoder with configurable parameters to analyze parameter influences.
result DAE parameters significantly affect prediction accuracy across datasets.