In this work, we study the use of logistic regression in manufacturing failures detection. As a data set for the analysis, we used the data from Kaggle competition Bosch Production Line Performance. We considered the use of machine learning, linear and Bayesian models. For machine learning approach, we analyzed XGBoost…
Proposes efficient Bayesian logistic regression for large sparse datasets.
problem Infeasibility of theoretical Bayesian methods for large sparse feature sets.
method Low complexity analytical approximations for sparse online logistic and probit regressions.
result Empirical results show superior performance compared to more complex methods.
The article describe the model, derivation, and implementation of variational Bayesian inference for linear and logistic regression, both with and without automatic relevance determination. It has the dual function of acting as a tutorial for the derivation of variational Bayesian inference for simple models, as well a…
Improved variational inference for logistic regression and classification.
problem Intractability of Evidence Lower Bound in variational logistic regression.
method Introducing a new bound for the expectation of softplus function, applied to variational logistic regression and Gaussian process classification.
result The new bound results in a tighter posterior and faster computation compared to Monte-Carlo methods.
Improved VB algorithm for high-dimensional logistic regression with theoretical guarantees.
problem Sparse high-dimensional logistic regression model selection.
method Spike and slab variational Bayes approximation.
result Optimal convergence rates in ℓ2 and prediction loss for sparse truths. Efficiently selects important variables in high-dimensional logistic regression.
problem Variable selection in high-dimensional logistic regression with binary responses.
method Developed a variational empirical Bayes approach for efficient model space marginal distribution.
result The variational approximation inherits strong selection consistency from the posterior distribution.
Flexible Bayesian approach for generalized linear models, especially for sparse logistic regression.
problem Sparse logistic regression challenges in machine learning.
method Empirical Bayes approach with mean-field variational inference, tuning-free and scalable.
result Superior predictive performance in sparse logistic regression compared to existing methods.
New method for GLMs under DP provides private uncertainty quantification.
problem Private inference for GLMs with uncertainty quantification.
method Noise-aware DP Bayesian inference method for GLMs.
result Posterior uncertainty allows determination of statistically significant coefficients.
A new method for Bayesian neural networks using probabilistic backpropagation.
problem Approximating posterior distributions in Bayesian neural networks.
method Variational Expectation Propagation (VEP) with probabilistic backpropagation.
result Efficient algorithm for approximate integration over posterior distributions.
Efficient Bayesian variable selection for binomial and negative binomial data.
problem Computational challenges in Bayesian variable selection for complex models.
method Tempered Gibbs Sampling and MCMC scheme.
result Demonstrated effectiveness on cancer data with thousands of covariates.
In this paper, we analyze the Wisconsin Diagnostic Breast Cancer Data using Machine Learning classification techniques, such as the SVM, Bayesian Logistic Regression (Variational Approximation), and K-Nearest-Neighbors. We describe each model, and compare their performance through different measures. We conclude that S…
High-dimensional feature selection arises in many areas of modern science. For example, in genomic research we want to find the genes that can be used to separate tissues of different classes (e.g. cancer and normal) from tens of thousands of genes that are active (expressed) in certain tissue cells. To this end, we wi…
The paper uses regression models to predict COVID-19 spread and its stock market impact.
problem Predicting and understanding the impact of COVID-19 on stock markets.
method Logistic curve model with Bayesian regression for predictive analytics.
result Different crises have different impacts on the same stocks.
Bayesian method tackles variable selection in high-dimensional data.
problem Challenges in Bayesian variable selection with large P.
method Efficient MCMC scheme with sublinear cost per iteration, extended to generalized linear models.
result Demonstrated effectiveness on cancer and maize genomic data.
Hybrid machine learning improves gallstone risk prediction.
problem Complex gallstone disease risk factors and interactions.
method Adaptive LASSO for variable selection, BART for interactions, differential equations for interpretation.
result Enhanced prediction accuracy and actionable insights.
Mean-field variational methods are widely used for approximate posterior inference in many probabilistic models. In a typical application, mean-field methods approximately compute the posterior with a coordinate-ascent optimization algorithm. When the model is conditionally conjugate, the coordinate updates are easily …
SMP estimator improves density and logistic regression under misspecification.
problem Improper estimator for optimal excess risk in misspecified models.
method SMP minimizes a new excess risk bound for statistical learning.
result SMP achieves optimal excess risk of O((d+B2R2)/n) for logistic regression. New algorithms accelerate SVGD convergence using deep unfolding.
problem Improving the speed of SVGD convergence.
method Integrating deep unfolding into SVGD for parameter learning.
result Proposed algorithms achieve faster convergence in various tasks.
Bayesian logistic regression improves clinical risk prediction models over time.
problem Improving clinical risk prediction models after deployment to adapt to temporal shifts.
method Bayesian logistic regression (BLR) and Markov variant (MarBLR) for online recalibration and revision of prediction models.
result BLR and MarBLR consistently outperform static models and other online revision methods, improving average AUC and calibration index.
A new method improves active learning for large batch sizes.
problem Challenges in scaling Bayesian active learning to large batch sizes.
method Derives Partial Batch Label Sampling (ParBaLS) for EPIG algorithm.
result ParBaLS EPIG outperforms top-B selection and BatchBALD. Bayesian model uses simple functions to forecast macroeconomic data.
problem Forecasting large datasets in macroeconomics with complex nonlinear relationships.
method Sum of simple two-component location mixtures, logistic function threshold, conjugate priors.
result Accurate point and density forecasts in US macroeconomic aggregates.
We propose a betting strategy based on Bayesian logistic regression modeling for the probability forecasting game in the framework of game-theoretic probability by Shafer and Vovk (2001). We prove some results concerning the strong law of large numbers in the probability forecasting game with side information based on …
Proposes logistic-beta process for modeling dependent probabilities with beta marginals.
problem Limited work on flexible and computationally convenient stochastic process extensions for dependent random probabilities.
method Introduces logistic-beta process with logistic transformation and beta marginals, capable of modeling dependence in discrete and continuous domains.
result Logistic-beta processes enable effective posterior inference and design of computationally tractable dependent Bayesian nonparametric models.
Robust Bayesian models are appealing alternatives to standard models, providing protection from data that contains outliers or other departures from the model assumptions. Historically, robust models were mostly developed on a case-by-case basis; examples include robust linear regression, robust mixture models, and bur…
Generalized linear models (GLMs) -- such as logistic regression, Poisson regression, and robust regression -- provide interpretable models for diverse data types. Probabilistic approaches, particularly Bayesian ones, allow coherent estimates of uncertainty, incorporation of prior information, and sharing of power acros…
Algorithm improves variational inference in Wasserstein distance.
problem Improving variational inference methods for complex models.
method Wasserstein contraction analysis of coordinate ascent.
result General and sharp convergence guarantees for various models.
The paper provides guarantees for a tangent transform algorithm in logistic regression models.
problem Finding theoretical guarantees for statistical optimality and algorithmic convergence in non-conjugate models.
method Exploiting convex duality and minorizing the marginal likelihood, the paper derives non-asymptotic upper bounds and convergence guarantees for a tangent transform algorithm in logistic regression models.
result The tangent transform algorithm is shown to be locally asymptotically stable without assumptions on the data-generating process.
Bayesian framework for encoding uncertainty and inducing sparsity.
problem Handling uncertainty and inducing sparsity in statistical models.
method General Bayesian framework with explicit encoding of uncertainty and sparsity-inducing approach.
result Effective in linear and logistic regression, and Bayesian neural networks.
In this paper we introduce ZhuSuan, a python probabilistic programming library for Bayesian deep learning, which conjoins the complimentary advantages of Bayesian methods and deep learning. ZhuSuan is built upon Tensorflow. Unlike existing deep learning libraries, which are mainly designed for deterministic neural netw…
Paper finds a lower bound for estimating low-rank matrices in logistic regression.
problem Estimating low-rank coefficient matrices in logistic regression.
method Derives a minimax lower bound on the risk.
result The bound depends on matrix dimensions, rank, and sample size.
Noise-free sampling method using Wasserstein proximal for faster convergence.
problem Sampling from distributions governed by potential functions.
method Deterministic score-based MCMC with regularized Wasserstein proximal.
result Improved mixing time bounds for Gaussian distributions compared to ULA and MALA.
Novel bounds for logistic regression coreset construction and feature selection.
problem Efficiently summarize and reduce logistic regression inputs.
method Feature space sketching for logistic regression.
result Tight bounds for coreset construction and feature selection.
Unified framework for sparse logistic regression with nonconvex regularization.
problem Sparse logistic regression with nonconvex regularization.
method Unified framework, line search criteria for nonconvex terms.
result Effective classification and feature selection at lower computational cost.
The paper uses XAI to predict RFQ fulfillment accuracy.
problem Improving accuracy in predicting RFQ fulfillment for less liquid asset classes.
method Advanced algorithms like Logistic Regression, Random Forest, XGBoost, and Bayesian Neural Tree.
result Improved accuracy in RFQ fill rate predictions.
Dropout has recently emerged as a powerful and simple method for training neural networks preventing co-adaptation by stochastically omitting neurons. Dropout is currently not grounded in explicit modelling assumptions which so far has precluded its adoption in Bayesian modelling. Using Bayesian entropic reasoning we s…
Bayesian model improves classification performance with flexible uncertainty modeling.
problem Improving classification performance with flexible uncertainty modeling.
method Combines Gaussian process and Dirichlet process priors for latent function and link function, respectively.
result Outperforms standard logistic regression on simulated data.
Bayesian inference for neural networks improves uncertainty quantification.
problem Improving predictive uncertainty in neural networks.
method Ensemble Kalman filter extensions and interacting particle systems.
result Effective methods for quantifying predictive uncertainty in neural networks.
Prevalidated ridge regression simplifies logistic regression for high-dimensional data.
problem Efficient probabilistic classification in high-dimensional data with logistic regression.
method Developed a prevalidated ridge regression model that matches logistic regression's performance but is more computationally efficient.
result Prevalidated ridge regression achieves similar classification error and log-loss to logistic regression for high-dimensional data.
Improved sketching for logistic and ℓ1 regression with near-linear dimensions.
problem Efficiently approximate ℓ1 and logistic regression problems. method New sketching techniques achieving near-linear dimensions for both problems.
result Achieved near-linear sketching dimensions for ℓ1 and logistic regression. The use of Bayesian methods in large-scale data settings is attractive because of the rich hierarchical models, uncertainty quantification, and prior specification they provide. Standard Bayesian inference algorithms are computationally expensive, however, making their direct application to large datasets difficult or …
Identifying small subsets of features that are relevant for prediction and/or classification tasks is a central problem in machine learning and statistics. The feature selection task is especially important, and computationally difficult, for modern datasets where the number of features can be comparable to, or even ex…
New AMP algorithm detects change points in high-dimensional GLMs.
problem Detecting change points in high-dimensional GLMs.
method Approximate Message Passing (AMP) algorithm for estimating signals and change points.
result Characterization of AMP algorithm's performance in high-dimensional limit.
Safe screening rules reduce computation time in logistic regression with ℓ0−ℓ2 regularization.
problem Efficiently solving logistic regression with many features and regularization.
method Screening rules based on Fenchel dual lower bounds of strong conic relaxations.
result A high percentage of features can be safely removed before solving, leading to substantial speed-up.
Improves logistic regression performance with nonconvex programming.
problem Stochastic generalized linear regression with chance constraints.
method Nonconvex programming techniques, clustering, quantile estimation.
result Over 1 to 2 percent improvement in model performance.
We propose a new data-augmentation strategy for fully Bayesian inference in models with binomial likelihoods. The approach appeals to a new class of Polya-Gamma distributions, which are constructed in detail. A variety of examples are presented to show the versatility of the method, including logistic regression, negat…
Paper explores connections between loss functions and consistency in binary classification and regression.
problem Consistency in binary classification and regression applications.
method Characterization of conformable loss functions and derivation of a new Huber-type loss function.
result Margin-based loss functions are equivalent to loss functions of squared standardized logistic regression residuals.
Maximum likelihood estimator performance in logistic regression analyzed.
problem Performance of maximum likelihood estimator in logistic regression.
method Sharp non-asymptotic guarantees for existence and excess logistic risk.
result Sharp guarantees for the existence and excess risk of MLE in logistic regression.
We comment on the fact that gradient ascent for logistic regression has a connection with the perceptron learning algorithm. Logistic learning is the "soft" variant of perceptron learning.