The paper combines Bitcoin price models with expert corrections for better predictions.
problem Improving Bitcoin price predictions using statistical and expert insights.
method Linear regression models combined with expert corrections, utilizing Bayesian approach for fat-tailed distributions.
result Better price prediction results compared to using either model or expert opinion alone.
A novel framework for regression with multiple experts, addressing challenges in infinite and continuous label spaces.
problem Challenges in regression with multiple experts due to the infinite and continuous nature of the label space.
method Introduces a novel framework for regression with deferral, analyzing both single-stage and two-stage scenarios with new surrogate loss functions.
result Proves H-consistency bounds for both single-stage and two-stage methods, providing stronger guarantees than Bayes consistency. Developed shrinkage methods for Poisson regression models with experts to handle multicollinearity.
problem Multicollinearity in Poisson regression models with experts.
method Ridge and Liu-type shrinkage methods.
result Shrinkage methods offer more reliable estimates for coefficients in multicollinearity.
In regression tasks the distribution of the data is often too complex to be fitted by a single model. In contrast, partition-based models are developed where data is divided and fitted by local models. These models partition the input space and do not leverage the input-output dependency of multimodal-distributed data,…
A new method for sparse Gaussian process regression using correlated experts.
problem Sparse Gaussian process regression for large datasets with cubic computational complexity.
method Aggregating predictions from correlated experts to improve scalability and accuracy.
result Superior performance compared to state-of-the-art methods for synthetic and real-world datasets.
Consider a regression problem where there is no labeled data and the only observations are the predictions fi(xj) of m experts fi over many samples xj. With no knowledge on the accuracy of the experts, is it still possible to accurately estimate the unknown responses yj? Can one still detect the leas…
New method calibrates Gaussian product experts for better predictions.
problem Erratic predictions and uncalibrated uncertainty in Gaussian product experts.
method Calibration via tempered softmax and Wasserstein barycenter for predictions.
result Improved predictions with better mean and uncertainty quantification.
The paper proposes a new model to capture specialized brain regions using mixture of regression experts.
problem Learning a forward mapping that relates stimuli to brain activation assumes all regions respond similarly, ignoring brain specialization.
method Clustering brain regions, learning different linear regression models for each cluster, using a mixture of linear experts.
result The proposed model predicts brain activation more accurately than conventional models.
Adaptive Bayesian learning aggregates experts to improve performance.
problem Bayesian online learning's performance depends on inferential choices.
method Treat Bayesian update rules as experts and aggregate them based on sequential predictive losses.
result The aggregate competes with the best expert in hindsight at a low aggregation cost.
The paper presents a novel approach to multi-output regression using probabilistic circuits.
problem Capturing correlations between multiple output dimensions in large-scale regression problems.
method Employing a mixture of single-output Gaussian process experts encoded via a probabilistic circuit.
result The method can capture correlations between output dimensions and often outperforms other approaches.
A new method combines experts' opinions to train regression models with noisy labels.
problem Training regression models with noisy labels from multiple experts.
method Estimate each labeler's expertise and combine opinions using learned weights.
result Empirically outperforms existing techniques on simulated and real data.
Mixture of Experts (MoE) is a popular framework for modeling heterogeneity in data for regression, classification and clustering. For continuous data which we consider here in the context of regression and cluster analysis, MoE usually use normal experts, that is, expert components following the Gaussian distribution. …
Interactive IL algorithm queries noisy expert for feedback, reducing sample size.
problem Imitation learning with noisy expert feedback requires many samples.
method Selective sampling algorithm with online regression oracle.
result Interactive IL algorithm reduces query complexity and improves performance.
Meta-learning model divides tasks into sub-problems for efficient adaptation.
problem Training models to quickly adapt to new tasks.
method Hierarchical Expert Networks with information-theoretic partitioning and specialized experts.
result Specialized experts lead to efficient adaptation to new tasks.
MoE outperforms vanilla networks in detecting latent cluster structures in regression tasks.
problem Detecting latent cluster structures in regression tasks with gradient-based learning.
method Theoretical analysis of Mixture of Experts (MoE) using stochastic gradient descent (SGD).
result MoE can successfully divide the problem into subproblems, allowing each expert to recover simpler functions corresponding to individual clusters.
Improved text-conditioned regression using LLMs and diffusion-based neural processes.
problem Major error cascades and computational inefficiency in LLMs for short sequences.
method Combining LLM predictive densities with a diffusion-based neural process.
result Better-calibrated predictions and locally consistent trajectories.
A simple algorithm improves model generalization in expert advice settings.
problem Improving model generalization in expert advice settings.
method A naive aggregation algorithm for point estimations of high-dimensional nonlinear functions.
result Conditions for optimal parameter estimates through sequential mixing distribution strategies.
A new model predicts multivariate regression using similarities to data points.
problem Complex, high-dimensional input-output relationships.
method Bayesian mixture-of-experts with conditional Gaussian mixtures and variational Bayes.
result Outperforms competitors in high-dimensional settings.
Sigmoid gating is more sample efficient than softmax in mixture of experts.
problem Softmax gating leads to unnecessary competition among experts, causing representation collapse.
method Theoretical analysis of a regression framework with mixture of experts, identifying identifiability conditions and convergence rates.
result Sigmoid gating requires fewer samples to achieve the same expert estimation error as softmax gating.
New model handles complex non-linear relationships with hidden graph structures.
problem Modeling non-linear relationships with hidden graph-structured interactions.
method Block-diagonal localized mixture of polynomial experts (BLoMPE) regression model with penalized maximum likelihood selection criterion.
result Strong theoretical guarantee for finite-sample oracle inequality.
Prediction in a small-sized sample with a large number of covariates, the "small n, large p" problem, is challenging. This setting is encountered in multiple applications, such as precision medicine, where obtaining additional samples can be extremely costly or even impossible, and extensive research effort has recentl…
Mixture of Experts (MoE) is a popular framework in the fields of statistics and machine learning for modeling heterogeneity in data for regression, classification and clustering. MoE for continuous data are usually based on the normal distribution. However, it is known that for data with asymmetric behavior, heavy tail…
Bayesian framework integrates spectral deconvolution with expert reasoning for robust peak estimation.
problem Challenges in extracting meaningful peaks from noisy or complex spectra.
method Bayesian spectral deconvolution coupled with a physical-property regression layer.
result Recovery of weak peaks in poly(lactic acid) IR spectra related to degradation rates.
Introduces a rule-based Bayesian regression for better uncertainty quantification and expert knowledge integration.
problem Handling regression problems with uncertainty quantification and expert intuition.
method Combines Bayesian inference and rule-based systems for better model performance.
result Improves model performance with better uncertainty quantification and point predictions.
Mixture of Experts (MoE) is a popular framework for modeling heterogeneity in data for regression, classification, and clustering. For regression and cluster analyses of continuous data, MoE usually use normal experts following the Gaussian distribution. However, for a set of data containing a group or groups of observ…
Logistic regression can handle noisy labels effectively when labels are imperfectly assigned by multiple experts.
problem Label noise in supervised classification due to manual labelling by multiple experts.
method Using approximate posterior probabilities of class membership from multiple experts to train logistic regression models.
result Logistic regression can be robust to label noise when classification difficulty is the only source of errors.
A method to improve LLMs by automating the construction of a mixture of expert prompts.
problem Limitation of single instruction prompts in covering complex problem spaces.
method Divide the problem space into sub-regions, each governed by a specialized expert with both an instruction and demos. A two-phase process constructs these experts.
result Achieves an average win rate of 81% across major benchmarks.
The paper establishes convergence rates for MoE models in classification problems.
problem Understanding the behavior of MoE models in classification settings.
method Established convergence rates for density and parameter estimation in softmax gating multinomial logistic MoE models.
result Parameter estimation rates are significantly improved with a novel modified softmax gating function.
This thesis tackles learning with multi-class abstention and multi-expert deferral, improving model reliability and efficiency.
problem Improving model reliability and efficiency in large language models (LLMs) by leveraging multiple experts.
method Developed new surrogate losses and consistency guarantees for multi-class classification and regression with deferral.
result Strong consistency guarantees for surrogate losses in multi-class classification and regression with deferral.
In mixtures-of-experts (ME) model, where a number of submodels (experts) are combined, there have been two longstanding problems: (i) how many experts should be chosen, given the size of the training data? (ii) given the total number of parameters, is it better to use a few very complex experts, or is it better to comb…
Mixtures-of-Experts (MoE) are conditional mixture models that have shown their performance in modeling heterogeneity in data in many statistical learning approaches for prediction, including regression and classification, as well as for clustering. Their estimation in high-dimensional problems is still however challeng…
fMRI semantic category understanding using linguistic encoding models attempt to learn a forward mapping that relates stimuli to the corresponding brain activation. Classical encoding models use linear multi-variate methods to predict the brain activation (all voxels) given the stimulus. However, these methods essentia…
We propose a practical and scalable Gaussian process model for large-scale nonlinear probabilistic regression. Our mixture-of-experts model is conceptually simple and hierarchically recombines computations for an overall approximation of a full Gaussian process. Closed-form and distributed computations allow for effici…
We introduce a novel personalized Gaussian Process Experts (pGPE) model for predicting per-subject ADAS-Cog13 cognitive scores -- a significant predictor of Alzheimer's Disease (AD) in the cognitive domain -- over the future 6, 12, 18, and 24 months. We start by training a population-level model using multi-modal data …
EBBS integrates expert assessments into MIO best-subsets problem.
problem Identifying relevant features in statistical models.
method Expert-implied Bayesian approach using MIO.
result Reduces to classical best-subsets when experts are silent.
Investigates least squares estimation in deterministic MoE models.
problem Complexity and difficulty in analyzing MoE models.
method Least squares estimation under deterministic MoE models.
result Rates of estimating strongly identifiable experts are faster than polynomial experts.
We investigate a class of hierarchical mixtures-of-experts (HME) models where exponential family regression models with generalized linear mean functions of the form psi(ga+fx^Tfgb) are mixed. Here psi(...) is the inverse link function. Suppose the true response y follows an exponential family regression model with mea…
A deep learning model corrects precipitation bias without expert knowledge.
problem Precipitation bias in numerical predictions due to limited observation and models.
method Data-driven deep learning model with Denoising Autoencoder and Ordinal Regression blocks.
result The model achieves the best correcting performance and TS compared to classical methods.
The paper introduces a method to incorporate expert opinion on observable quantities into statistical models.
problem Tackling the challenge of integrating expert knowledge on observable quantities into statistical models.
method The approach involves updating a prior belief using a loss function that reflects expert opinion on observable quantities.
result The method allows for a flexible specification of expert opinion and is straightforward to implement.
A fast method combines deep mixtures of sparse GPs for flexible modeling.
problem Flexible modeling with changing output densities.
method Designing gating network with DNN for selecting sparse GPs, using CCR algorithm.
result The method outperforms competing methods in accuracy and uncertainty quantification.
Dropout is a very effective method in preventing overfitting and has become the go-to regularizer for multi-layer neural networks in recent years. Hierarchical mixture of experts is a hierarchically gated model that defines a soft decision tree where leaves correspond to experts and decision nodes correspond to gating …
CoCoAFusE fuses expert predictions to model complex patterns with interpretability and uncertainty.
problem Modeling complex patterns with interpretability and uncertainty quantification.
method Competitive/Collaborative Fusion of Experts (CoCoAFusE) that fuses expert distributions in addition to mixing.
result CoCoAFusE avoids multimodality artifacts and provides tighter credible bounds on the response variable.
New AMP algorithm estimates signals and latent variables in mixed regression models.
problem Estimating signals and latent variables in mixed regression models.
method Approximate Message Passing (AMP) algorithm for matrix GLM.
result State evolution recursion and optimal denoising functions for precise error minimization.
Fuzzy systems have achieved great success in numerous applications. However, there are still many challenges in designing an optimal fuzzy system, e.g., how to efficiently optimize its parameters, how to balance the trade-off between cooperations and competitions among the rules, how to overcome the curse of dimensiona…
Mixture of Experts (MoE) are successful models for modeling heterogeneous data in many statistical learning problems including regression, clustering and classification. Generally fitted by maximum likelihood estimation via the well-known EM algorithm, their application to high-dimensional problems is still therefore c…
Paper introduces deep structured mixtures of Gaussian processes for scalable GP approximations.
problem Scalability issues with Gaussian Processes (GPs).
method Deep structured mixtures of GP experts for scalable approximate inference.
result Deep structured mixtures provide better predictive uncertainties and competitive performance.
Study improves stock price prediction using adaptive Mixture of Experts framework.
problem Tackles diverse volatility regimes in stock price prediction.
method Combines RNN for high-volatility stocks and linear regression for stable stocks with a gating mechanism.
result Achieves up to 33% improvement in MSE for volatile assets and 28% for stable assets.
Heteroscedastic regression considering the varying noises among observations has many applications in the fields like machine learning and statistics. Here we focus on the heteroscedastic Gaussian process (HGP) regression which integrates the latent function and the noise function together in a unified non-parametric B…