Research reveals simplicity bias in random logistic map, impacting data analysis and forecasting.
problem Simplicity bias in dynamical systems and its impact on data analysis and prediction.
method Examined the logistic map and random logistic map, focusing on simplicity bias and noise effects.
result Simplicity bias is observable in the random logistic map, persisting even with small noise levels.
This paper describes how to convert a machine learning problem into a series of map-reduce tasks. We study logistic regression algorithm. In logistic regression algorithm, it is assumed that samples are independent and each sample is assigned a probability. Parameters are obtained by maxmizing the product of all sample…
City Logistics is characterized by multiple stakeholders that often have different views of such a complex system. From a public policy perspective, identifying stakeholders, issues and trends is a daunting challenge, only partially addressed by traditional observation systems. Nowadays, social media is one of the bigg…
L1 regularized logistic regression has now become a workhorse of data mining and bioinformatics: it is widely used for many classification problems, particularly ones with many features. However, L1 regularization typically selects too many features and that so-called false positives are unavoidable. In this pape…
Framework improves ML flood mapping generalization.
problem Improving machine learning models' ability to generalize to new conditions.
method Dimensionless, multi-scale features constrained by the Buckingham Π theorem.
result Model outperformed dimensional features, improving AUC in unmapped areas.
This manuscript studies statistical properties of linear classifiers obtained through minimization of an unregularized convex risk over a finite sample. Although the results are explicitly finite-dimensional, inputs may be passed through feature maps; in this way, in addition to treating the consistency of logistic reg…
Early stopping improves neural networks' performance on binary classification tasks.
problem Improving shallow ReLU networks' performance on binary classification tasks.
method Gradient descent with early stopping on binary classification data.
result Gradient descent with early stopping achieves population risk arbitrarily close to optimal.
Logistic Gaussian process (LGP) priors provide a flexible alternative for modelling unknown densities. The smoothness properties of the density estimates can be controlled through the prior covariance structure of the LGP, but the challenge is the analytically intractable inference. In this paper, we present approximat…
For the problem of multi-class linear classification and feature selection, we propose approximate message passing approaches to sparse multinomial logistic regression (MLR). First, we propose two algorithms based on the Hybrid Generalized Approximate Message Passing (HyGAMP) framework: one finds the maximum a posterio…
Introduces Fitzpatrick losses, tighter than Fenchel-Young losses.
problem Improving loss functions for machine learning.
method Introduces Fitzpatrick losses based on the Fitzpatrick function.
result Fitzpatrick losses are tighter than Fenchel-Young losses.
Modeling bank leverage dynamics to understand systemic risk in financial markets.
problem Understanding systemic risk in financial markets triggered by bank leverage dynamics.
method Developed a dynamical model of bank leverage, analyzing coupled dynamics in isolated and interconnected bank models.
result Identified a procyclical feedback loop between asset prices and leverage, leading to chaotic dynamics.
Maximum likelihood estimator performance in logistic regression analyzed.
problem Performance of maximum likelihood estimator in logistic regression.
method Sharp non-asymptotic guarantees for existence and excess logistic risk.
result Sharp guarantees for the existence and excess risk of MLE in logistic regression.
A latent space model for a family of random graphs assigns real-valued vectors to nodes of the graph such that edge probabilities are determined by latent positions. Latent space models provide a natural statistical framework for graph visualizing and clustering. A latent space model of particular interest is the Rando…
Paper finds a lower bound for estimating low-rank matrices in logistic regression.
problem Estimating low-rank coefficient matrices in logistic regression.
method Derives a minimax lower bound on the risk.
result The bound depends on matrix dimensions, rank, and sample size.
Fenrir efficiently estimates Bayesian MLN-DLMs for scalable inference.
problem Computational challenges in Bayesian MLN-DLMs for longitudinal count compositional data.
method Novel algorithm for MAP estimation and accurate posterior marginal approximation.
result Fenrir can be three orders of magnitude more efficient than Stan.
Revises logistic-softmax likelihood for Bayesian meta-learning in few-shot classification.
problem Inherent uncertainty in logistic-softmax leads to suboptimal performance in meta-learning.
method Redesigns logistic-softmax likelihood with a temperature parameter for better control of prior confidence.
result Achieves well-calibrated uncertainty estimates and comparable/superior performance on benchmark datasets.
Neural Shadow-Mapping uncovers causal links in dynamic systems.
problem Discovering causal structures in dynamic systems with mirage correlations.
method Neural network based method embedding high-dimensional data into a shadow representation for causal link estimation.
result Demonstrates performance in discovering causal links from video-representations of dynamic systems.
Novel bounds for logistic regression coreset construction and feature selection.
problem Efficiently summarize and reduce logistic regression inputs.
method Feature space sketching for logistic regression.
result Tight bounds for coreset construction and feature selection.
Unified framework for sparse logistic regression with nonconvex regularization.
problem Sparse logistic regression with nonconvex regularization.
method Unified framework, line search criteria for nonconvex terms.
result Effective classification and feature selection at lower computational cost.
Study explores geometric structure and prior for beta-logistic distribution.
problem Understanding the geometric structure and prior distributions of the beta-logistic distribution.
method Exploring dual geometric structure and uncovering α-parallel prior. result The beta-logistic distribution admits an α-parallel prior for any real number α. We comment on the fact that gradient ascent for logistic regression has a connection with the perceptron learning algorithm. Logistic learning is the "soft" variant of perceptron learning.
Paper explains learning property of logistic and softmax losses for balanced and imbalanced class data.
problem Understanding and optimizing loss functions for deep neural networks with class imbalances.
method Analyzing necessary conditions for convergence of logistic and softmax losses in CNNs.
result Proposes a novel reweighted logistic loss function that improves performance over softmax loss.
Improved sketching for logistic and ℓ1 regression with near-linear dimensions.
problem Efficiently approximate ℓ1 and logistic regression problems. method New sketching techniques achieving near-linear dimensions for both problems.
result Achieved near-linear sketching dimensions for ℓ1 and logistic regression. Safe screening rules reduce computation time in logistic regression with ℓ0−ℓ2 regularization.
problem Efficiently solving logistic regression with many features and regularization.
method Screening rules based on Fenchel dual lower bounds of strong conic relaxations.
result A high percentage of features can be safely removed before solving, leading to substantial speed-up.
Proposes logistic-beta process for modeling dependent probabilities with beta marginals.
problem Limited work on flexible and computationally convenient stochastic process extensions for dependent random probabilities.
method Introduces logistic-beta process with logistic transformation and beta marginals, capable of modeling dependence in discrete and continuous domains.
result Logistic-beta processes enable effective posterior inference and design of computationally tractable dependent Bayesian nonparametric models.
Classification is the most important process in data analysis. However, due to the inherent non-convex and non-smooth structure of the zero-one loss function of the classification model, various convex surrogate loss functions such as hinge loss, squared hinge loss, logistic loss, and exponential loss are introduced. T…
Paper introduces imprecise logistic regression for handling uncertain data.
problem Uncertainties in data prevent traditional logistic regression from being applied effectively.
method Develops imprecise logistic regression model using intervals of possible values.
result Clearly expresses epistemic uncertainty in predictions.
The MSPI predicts market stress with machine learning.
problem Estimating the probability of high market stress.
method L1-regularized logistic regression on stock fragility signals.
result MSPI tracks major stress episodes and improves accuracy.
Improved regret bounds for logistic bandits via novel confidence set construction.
problem Dependencies in parameter space for logistic bandits, especially when S≥d. method Regret-to-confidence-set conversion (R2CS) to construct convex confidence sets.
result Strict improvement in regret bound w.r.t. S in logistic bandits. Improved confidence bounds for linear logistic model with applications to bandits.
problem Improving confidence bounds for linear logistic model.
method Self-concordant analysis of the logistic loss to avoid dependence on worst-case variance.
result Significant improvement in confidence bounds, avoiding dependence on 1/κ. A new data-oblivious sketch for logistic regression reduces data size while maintaining approximation accuracy.
problem Efficiently solving logistic regression in one pass over a data stream.
method Data-oblivious sketching approach that reduces data size to poly(μdlog n) weighted points.
result Sketching reduces data size significantly and provides approximation guarantees.
The l1-regularized logistic regression (or sparse logistic regression) is a widely used method for simultaneous classification and feature selection. Although many recent efforts have been devoted to its efficient implementation, its application to high dimensional data still poses significant challenges. In this paper…
PIANO speeds up multinomial logistic regression solving.
problem Handling large datasets and many classes in logistic regression.
method Parallel iterative algorithm based on Majorization Minimization.
result PIANO converges to a stationary point of Multinomial and Sparse Multinomial Logistic Regression.
Self-attention optimizers converge to optimal weights, revealing bias patterns.
problem Understanding the implicit bias in self-attention mechanisms.
method Analysis of gradient-based optimization in self-attention layers.
result Adaptive step-size strategies can accelerate convergence to optimal weights.
Logistic regression is a widely used method in several fields. When applying logistic regression to imbalanced data, for which majority classes dominate over minority classes, all class labels are estimated as `majority class.' In this article, we use an F-measure optimization method to improve the performance of logis…
Develops first optimal algorithm for logistic bandits.
problem Pure exploration in logistic bandits.
method Logistic track-and-stop (Log-TS) algorithm.
result Asymptotically matches lower bound for expected sample complexity.
A new algorithm approximates logistic regression probabilities efficiently.
problem Efficiently approximating probabilities in logistic regression for large datasets.
method Randomized sampling-based algorithm with leverage scores.
result Accurate approximations to estimated probabilities with smaller sample sizes.
Paper explores connections between loss functions and consistency in binary classification and regression.
problem Consistency in binary classification and regression applications.
method Characterization of conformable loss functions and derivation of a new Huber-type loss function.
result Margin-based loss functions are equivalent to loss functions of squared standardized logistic regression residuals.
Prevalidated ridge regression simplifies logistic regression for high-dimensional data.
problem Efficient probabilistic classification in high-dimensional data with logistic regression.
method Developed a prevalidated ridge regression model that matches logistic regression's performance but is more computationally efficient.
result Prevalidated ridge regression achieves similar classification error and log-loss to logistic regression for high-dimensional data.
Convex optimization models predict outputs from inputs via optimization problems.
problem Predicting outputs from inputs using convex optimization models.
method Proposed a heuristic for learning parameters of convex optimization models from datasets.
result Demonstrated the effectiveness of the proposed method on three model classes.
Identifying small subsets of features that are relevant for prediction and/or classification tasks is a central problem in machine learning and statistics. The feature selection task is especially important, and computationally difficult, for modern datasets where the number of features can be comparable to, or even ex…
Study shows sample complexity for logistic regression with normal covariates.
problem Estimating parameters of logistic regression with normal design.
method Analyzes sample complexity in terms of dimension and inverse temperature.
result Shows two change-points in sample complexity curve based on inverse temperature.
Logistic regression gets a new, simpler uniform bound.
problem Finding a uniform bound for logistic regression's empirical risk.
method PAC-Bayes approach with second-order expansion and Rademacher-complexity bounds.
result Provides a dimension-free uniform concentration bound.
The paper reveals the hidden costs of digitizing commodity money and proposes a new stable-coin system.
problem Depreciation of banknotes due to high logistics costs after digitization.
method Analyzing the functions of money from a logistics perspective and comparing commodity money to digital currency.
result There is no honest money that is both a store of value and has negligible logistics costs.
Estimates future ICU demand using age-structured data and logistic modeling.
problem Projecting future ICU demand from low-quality data.
method Age-structured sampling, logistic fit, subnotification factor correction.
result Corrected logistic curve projects future ICU demand.
SGD converges globally to logistic loss minima for two-layer nets.
problem Global convergence of SGD for logistic loss on two-layer neural nets.
method Demonstrates existence of Frobenius norm regularized logistic loss functions as Villani functions, proving convergence and exponential rate.
result SGD converges globally to the global minima of appropriately regularized logistic empirical risk of depth 2 nets.
The paper proves asymptotic normality for multinomial logistic regression on null covariates.
problem Classical asymptotic normality results fail in high-dimensional multinomial logistic models.
method Developed asymptotic normality and chi-square results for multinomial logistic MLE on null covariates.
result Validated new methodology to test feature significance in high-dimensional classification problems.
Neural network outperforms traditional methods in chaotic dynamics classification.
problem Classifying chaotic and regular dynamics of the Chirikov standard map.
method Trained a convolutional neural network on finite-length trajectories compared to traditional Lyapunov exponent computation.
result Neural network outperforms traditional methods for short periods, converging faster and more robustly.