A new method transfers parameters in ELM networks using projective model.
problem Parameter transfer in extreme learning machine networks.
method Projective model to bridge source and target model parameters, L2,1-norm penalty for joint feature selection and parameter transfer.
result Significantly outperforms non-transfer ELM networks and other methods.
Deep learning estimates time-varying Markov model parameters.
problem Estimating time-dependent parameters in Markov models.
method Reframes parameter estimation as an optimization problem using maximum likelihood.
result Real solution close to SDE with neural network-derived parameters under specific conditions.
Deep learning used for parameter estimation in hard-to-infer models.
problem Parameter estimation in intractable models like max-stable processes.
method Train deep neural networks on simulated data to estimate parameters.
result Deep learning provides accurate and faster parameter estimation.
Federated learning improves with adaptive hyper-parameters and representation matching.
problem Heterogeneous client data leads to divergent local models in federated learning.
method Representation matching and adaptive hyper-parameters.
result Significant performance and robustness improvements in federated learning.
Two approaches improve parameter learning in various mixture models.
problem Parameter learning in mixture models.
method Complex-analytic and algebraic-combinatorial methods.
result Improved sample sufficiency for parameter estimation in specific mixture models.
This paper explores hyperparameter optimization for machine learning models.
problem Finding the best hyper-parameters for machine learning models.
method Introduces state-of-the-art optimization techniques and discusses their application.
result Comparison of different optimization methods on benchmark datasets.
Study models forest transitions with deep learning for parameter estimation.
problem Complex dynamics of forest, agricultural, and abandoned lands.
method Developed a stochastic differential equation model and used deep learning for parameter estimation.
result Deep learning approach estimates model parameters from time-series data.
Computational limitations require more model parameters for robust learning.
problem Computational constraints affect the number of parameters needed for robust learning.
method Analyzes computational limitations and their impact on model size for robust learning.
result Computational bounded learners need significantly more parameters for robust learning.
SPID-GAN learns bidirectional mappings in subsurface models.
problem Challenges in identifying and approximating causal structures in high-dimensional parameter spaces.
method Generative adversarial networks (GANs) for learning cross-domain mappings.
result SPID-GAN achieves satisfactory performance in identifying bidirectional state-parameter mappings.
Paper proposes a method to estimate scientific parameters in hybrid models without relying on model architecture.
problem Estimating unknown parameters in hybrid models combining machine learning and scientific models.
method Sharpness-aware minimization adapted for hybrid modeling, focusing on model simplicity.
result Demonstrates effectiveness of SAM-based hybrid model learning for scientific parameter estimation.
Study shows best hyper-parameters improve deep learning model's accuracy for IoT attack detection.
problem Improving accuracy of deep learning model for IoT attack detection.
method Examined three hyper-parameters' influence on model performance.
result Model's reported accuracy not achievable due to optimal hyper-parameters.
A method to reuse 98% of parameters for multi-task learning.
problem Improving efficiency in deep learning parameter usage.
method Learning model patches for each task, reusing pretrained network parameters.
result Significant improvement in transfer learning accuracy with fewer parameters.
The paper provides a method to find optimal machine learning model parameters with confidence.
problem Finding optimal machine learning model parameters that generalize well to the entire population.
method Constructs valid confidence sets for the optimal parameter using only training data.
result Valid confidence sets for optimal machine learning model parameters can be generated using bootstrapping techniques.
Training-free model learns SDE dynamics without training, accelerating parameter studies.
problem High computational cost of simulating parameter-dependent SDEs.
method Training-free conditional diffusion model with joint kernel-weighted Monte Carlo estimator.
result Accurate approximation of conditional distributions across varying parameter values.
Study shows not all ML models are uniquely identifiable from data.
problem Identifiability issues in machine learning models.
method Investigated through a case study on gait dynamics using a bipedal-spring mass model.
result Some parameters can be identified, but others remain unidentifiable.
Proposes a method to measure model parameter similarity for visual tasks.
problem Estimating relations between different visual tasks.
method LPS method using a second-order neural network to align model parameters and learn second-order similarity.
result Extensive experiments validate the effectiveness of the proposed method.
Proposes efficient model for continual learning that grows model over task-specific parameters.
problem Limited transfer learning ability and forgetting of earlier knowledge in existing methods.
method Filter and channel expansion method that grows model over previous task parameters.
result Better knowledge transfer and improved performance in task incremental learning.
Paper studies MCCR models with scale parameters tending to zero, revealing optimal learning rate and comparing robustness.
problem Analyzing MCCR models with scale parameters approaching zero.
method Investigates MCCR models with scale parameters tending to zero, revealing optimal learning rate and comparing robustness.
result Optimal learning rate of MCCR models is O(n−1) in the asymptotic sense. We apply variational inference to learn vehicle trajectory parameters from noisy data.
problem Learning parameters for vehicle trajectory estimation from noisy measurements.
method Gaussian variational inference with parameter learning in a motion and sensor model context.
result High-quality state estimates achieved even with outliers and false loop closures.
A new method for fitting mixture models using Boltzmann exploration.
problem Challenges in learning mixture models, especially with good initialisation.
method Boltzmann exploration expectation-maximisation (BEEM) algorithm.
result BEEM can escape local optima and is insensitive to parameter initialisation.
The paper studies how regularization parameters affect sparsity in deep neural networks.
problem Reducing the complexity of deep neural networks by promoting sparsity.
method Derives ℓ1-norm sparsity-promoting models, characterizes sparsity levels, and develops algorithms for selecting optimal regularization parameters. result Developed algorithms to select regularization parameters for desired sparsity levels in neural networks.
A new method quantizes LSTM gate parameters without performance loss.
problem Quantization loss in LSTM gate parameters without performance degradation.
method Lossy quantization of gate parameters during training, weight parameters adjust to offset quantization loss.
result F1 score decreased by only 0.7% on Named Entity Recognition dataset.
Stage-based hyper-parameter optimization reduces GPU-hours and training time.
problem Efficiently executing hyper-parameter optimization for deep learning models.
method Stage-based execution strategy to remove redundant computations.
result Stage-based execution outperforms trial-based method by up to 6.60 times in GPU-hours and 4.13 times in training time.
Adaptive multi-domain learning reduces parameter count for efficient deep learning.
problem Different domains have varying complexity, leading to inefficient model training.
method Proposes adaptive parameterization to reduce model complexity without sacrificing performance.
result Efficient multi-domain learning solutions with far fewer parameters.
Framework for multi-task learning with semiparametric models and nuisance parameters.
problem Improving parameter estimation from diverse, heterogeneous datasets.
method Late fusion multi-task learning framework with two-step process: individual task learning followed by adaptive aggregation.
result The method achieves faster convergence rates compared to individual task learning when tasks share similar parametric components.
Method learns all edges and link parameters globally for binary pairwise Markov models.
problem Learning sparse Ising models with sparsity assumption.
method l1-regularized logistic regression for simultaneous estimation of all edges and link parameters.
result Numerical experiments show the advantage of the simultaneous estimation method.
A new method for multi-task learning by allocating parameters.
problem Sharing parameters between unrelated tasks can hurt performance.
method Learned binary variables to allocate components to tasks, encouraging sharing between related tasks.
result Achieves a 17% relative reduction of the error rate on Omniglot benchmark.
Natural gradient optimization improves model parameter estimation in graphical models.
problem Estimating model parameters in graphical models.
method Reformulated as an information geometric optimization problem, introduced natural gradient descent strategy.
result Natural gradient strategy leads to optimal parameter learning without fitting an incorrect distribution.
New method uses machine learning to optimize Fourier pricing methods.
problem Difficulty in tuning parameters for Fourier pricing methods.
method Learning tuning parameters of Fourier methods using machine learning.
result Very fast algorithms with full error control.
Machine learning models estimate water parameters from hyperspectral data.
problem Estimating multiple water parameters from hyperspectral data.
method Regression framework using machine learning models, PCA preprocessing.
result Machine learning models can accurately estimate water parameters from hyperspectral data.
Machine learning speeds up LSM parameter optimization and uncertainty quantification.
problem Optimizing and assessing uncertainty of LSM parameters.
method Combining MCMC with Gaussian process regression.
result 50,000 times faster than direct MCMC application.
Automatically optimizes simulation parameters for machine learning.
problem Costly or hard-to-acquire training data for machine learning models.
method Reinforcement learning to control simulator parameters for data synthesis.
result Approach optimizes simulation parameters to maximize model accuracy.
ES for non-differentiable parameters scales to large models.
problem Learning non-differentiable parameters in large models.
method Hybrid approach combining ES for non-differentiable and gradient-based methods for differentiable parameters.
result Hybrid approach is competitive and allows training sparse models from the start.
CAVIA improves meta-learning by adapting context parameters at test time.
problem Meta-learning overfits and requires complex parallelisation.
method CAVIA splits model into context and shared parameters, updating only the former.
result CAVIA outperforms MAML across regression, classification, and reinforcement learning.
FiT combines transfer and meta-learning for efficient few-shot image classification.
problem Few-shot image classification in personalized and federated learning settings.
method Combines transfer learning and meta-learning with fixed pretrained backbones and fine-tuned FiLM adapter layers.
result Achieves state-of-the-art accuracy on VTAB-1k benchmark with fewer than 1% of updateable parameters.
Paper analyzes consistency of Bayesian and machine learning methods for hierarchical parameter estimation.
problem Learning hierarchical parameters in complex and real-world problems.
method Empirical Bayes and Kernel Flow approaches.
result Consistency results for Matérn-like model on the torus, and comparison of algorithms.
Deep learning predicts neural network parameters efficiently.
problem Optimizing neural network parameters remains inefficient.
method Used graph neural networks to predict parameters of unseen networks.
result Achieved surprisingly good performance on unseen networks.
Hippo optimizes deep learning hyper-parameters by reducing redundant trials.
problem Redundant hyper-parameter trials in hyper-parameter optimization.
method Hippo breaks down hyper-parameter sequences into stages and executes them in a tree structure.
result Hippo reduces GPU-hours and training time significantly compared to existing methods.
FURL improves model accuracy in FL by locally training user embeddings.
problem Improving prediction accuracy of neural-network-based models in Federated Learning.
method FURL divides model parameters into federated and private parameters, training private parameters locally.
result Significant performance improvement with 8% and 51% increases on two datasets.
The paper explores strong identifiability and parameter learning in regression models with heterogeneous responses.
problem Understanding heterogeneity in data populations through conditional distributions of a response variable.
method Investigation of strong identifiability, convergence rates, and posterior contraction behavior in finite mixture of regression models.
result Theoretical findings on conditions for strong identifiability and rates of convergence in regression mixture models.
SSRCA simplifies ABM sensitivity analysis using machine learning.
problem Hardness of performing sensitivity analysis for complex ABMs.
method Machine learning pipeline (Simulate, Summarize, Reduce, Cluster, Analyze) for ABMs.
result SSRCA identifies sensitive parameters and common output patterns for ABMs.
Learning models of artificial intelligence can nowadays perform very well on a large variety of tasks. However, in practice different task environments are best handled by different learning models, rather than a single, universal, approach. Most non-trivial models thus require the adjustment of several to many learnin…
Efficient model selection framework for online learning without parameter tuning.
problem Model selection in online learning without predefined parameters.
method Generic meta-algorithm framework for model selection in arbitrary Banach spaces under mild smoothness assumptions.
result First computationally efficient parameter-free algorithms in arbitrary Banach spaces.
Due to the growing ubiquity of unlabeled data, learning with unlabeled data is attracting increasing attention in machine learning. In this paper, we propose a novel semi-supervised kernel learning method which can seamlessly combine manifold structure of unlabeled data and Regularized Least-Squares (RLS) to learn a ne…
Paper presents an efficient method for selecting machine learning algorithms and hyper-parameters.
problem Efficient selection of machine learning algorithms and hyper-parameters is challenging for large datasets.
method Progressive sampling-based Bayesian optimization
result Significantly reduces search time, classification error rate, and error rate variability.
ANPyC combats forgetting by pruning and consolidating neural parameters.
problem Catastrophic forgetting in neural networks, especially with long-term tasks.
method Adversarial Neural Pruning and Synaptic Consolidation (ANPyC) to balance task-relevant and irrelevant parameters.
result ANPyC prevents forgetting while enabling efficient learning of multiple tasks.
Proposes a method to integrate learner models robustly against misspecifications.
problem Misspecifications in learner models and parameter sharing patterns degrade prediction accuracy.
method Sequentially incorporates additional learners based on user-specified parameter sharing patterns.
result Data-adaptively selects the most suitable way of parameter sharing to enhance predictive performance.
This work provides bounds on generalization error and privacy leakage in federated learning.
problem Bounding generalization error and privacy leakage in federated learning.
method Information-theoretic framework for classical, distributed, and federated learning.
result Upper and lower bounds on generalization error and privacy leakage.