Near-optimal private tests for simple and MLR hypotheses developed under Gaussian differential privacy.
problem Developing private tests for simple and MLR hypotheses under Gaussian differential privacy.
method A private mean estimator with data-driven clamping bounds, constructing private test statistics.
result Private tests achieve the same asymptotic relative efficiency as non-private most powerful tests.
We implemented several multilabel classification algorithms in the machine learning package mlr. The implemented methods are binary relevance, classifier chains, nested stacking, dependent binary relevance and stacking, which can be used with any base learner that is accessible in mlr. Moreover, there is access to the …
Develops a fast algorithm for fitting multilevel factor models.
problem Fitting multilevel factor models with covariance structure.
method Novel expectation-maximization algorithm tailored for multilevel factor models.
result Shows efficient computation of inverse of positive definite MLR matrix.
Optimal Biweight kernel and computationally efficient Epanechnikov kernel for modal linear regression.
problem Finding the best kernel for modal linear regression.
method Refined analysis of asymptotic statistical behavior and IRLS algorithm convergence.
result Biweight kernel minimizes asymptotic mean squared error, Epanechnikov kernel guarantees IRLS convergence.
This paper tackles fitting multilevel low rank matrices by addressing three problems.
problem Fitting a given matrix by an MLR matrix in the Frobenius norm.
method Factor fitting, rank allocation, and hierarchical partitioning.
result The proposed methods can fit a given matrix by an MLR matrix in the Frobenius norm.
Algorithm learns mixtures of linear regressions in subexponential time.
problem Learning mixtures of linear regressions with high accuracy.
method Fourier moment descent method using univariate density estimation and low-degree moments of Fourier transforms.
result First algorithm for learning MLRs in subexponential time.
MLR-SNet learns flexible LR schedules for diverse tasks.
problem Adapting LR schedules for non-convex optimization problems.
method Parameterizes LR schedules with an explicit mapping formulation.
result Meta-learned LR schedules are transferable and adaptable.
New algorithm for estimating MLR parameters with non-Gaussian noise.
problem Estimating MLR parameters with non-Gaussian noise.
method Combining ADMM with EM algorithm idea.
result Our method outperforms EM algorithm in non-Gaussian noise case.
Proposes a new stochastic optimization method for MLR models.
problem Slow convergence of SGD in big data scenarios.
method Dual Stochastic Natural Gradient Descent (DNSGD) based on manifold optimization.
result DNSGD converges and has linear computational complexity.
Scaling multinomial logistic regression to datasets with very large number of data points and classes is challenging. This is primarily because one needs to compute the log-partition function on every data point. This makes distributing the computation hard. In this paper, we present a distributed stochastic gradient d…
Paper tackles MLR prediction error without assuming realizable models.
problem Prediction error in mixture of linear regressions without realizable assumptions.
method Developed algorithms for list-decoding MLR predictions and minimized empirical risk.
result Alternating minimization algorithm finds best fit lines in non-realizable settings.
We consider the problem of learning parameters of latent variable models from mixed (continuous and ordinal) data with missing values. We propose a novel Bayesian Gaussian copula factor (BGCF) approach that is consistent under certain conditions and that is quite robust to the violations of these conditions. In simulat…
Paper addresses global convergence of MLR estimation under weak data conditions.
problem Learning mixed linear regression models with general data conditions.
method Two-step recursive identification algorithm using least squares and EM principles.
result Global convergence and optimal clustering performance established under general data conditions.
Mix-IRLS solves imbalanced mixed linear regression problems efficiently.
problem Imbalanced mixed linear regression problems.
method Sequential robust regression approach.
result Mix-IRLS outperforms other methods on imbalanced mixtures and real-world datasets.
Wasserstein framework solves mixed linear regression problems.
problem Mixed linear regression with multi-modal distributions.
method Wasserstein distance minimization for nonconvex-concave minimax optimization.
result WMLR achieves global convergence and generalization guarantees for two linear models.
For the problem of multi-class linear classification and feature selection, we propose approximate message passing approaches to sparse multinomial logistic regression (MLR). First, we propose two algorithms based on the Hybrid Generalized Approximate Message Passing (HyGAMP) framework: one finds the maximum a posterio…
Paper addresses online identification and clustering for mixed linear regression models.
problem Online identification and clustering of mixed linear regression models.
method Introduces two online identification algorithms based on the EM principle, proving global convergence without i.i.d. data assumptions.
result Global convergence of the proposed algorithms for mixed linear regression models.
We present ADMM-Softmax, an alternating direction method of multipliers (ADMM) for solving multinomial logistic regression (MLR) problems. Our method is geared toward supervised classification tasks with many examples and features. It decouples the nonlinear optimization problem in MLR into three steps that can be solv…
Warfarin dosing remains challenging due to narrow therapeutic index and highly individual variability. Incorrect warfarin dosing is associated with devastating adverse events. Remarkable efforts have been made to develop the machine learning based warfarin dosing algorithms incorporating clinical factors and genetic va…
The paper tackles high-dimensional mixed linear regression with unknown parameters and proposes methods for estimation, confidence intervals, and hypothesis testing.
problem High-dimensional mixed linear regression with unknown parameters and covariance structure.
method Iterative high-dimensional EM algorithm for estimating regression vectors, debiased estimators for individual coordinates, and large-scale multiple testing procedure.
result Asymptotic normality of debiased estimators and FDR control for hypothesis testing.
Overparameterized MLR fits hyper-curves, improving model robustness.
problem Improper predictors degrade model generalizability.
method Parameterizing with a scalar and monomial basis, fitting hyper-curves.
result Hyper-curve approach yields robust predictions for noisy data.
Improved robustness in multivariate regression and classification with DRO under Wasserstein metric.
problem Outliers in covariates and responses.
method Distributionally Robust Optimization (DRO) with Wasserstein metric ambiguity set and regularization.
result Significant improvement in predictive error and robustness.
We consider {\em Mixed Linear Regression (MLR)}, where training data have been generated from a mixture of distinct linear models (or clusters) and we seek to identify the corresponding coefficient vectors. We introduce a {\em Mixed Integer Programming (MIP)} formulation for MLR subject to regularization constraints on…
Machine learning techniques are presented for automatic recognition of the historical letters (XI-XVIII centuries) carved on the stoned walls of St.Sophia cathedral in Kyiv (Ukraine). A new image dataset of these carved Glagolitic and Cyrillic letters (CGCL) was assembled and pre-processed for recognition and predictio…
Designs neural networks on matrix manifolds for improved performance in tasks like human action recognition.
problem Designing neural networks for tasks on non-Euclidean manifolds.
method Develops fully-connected and convolutional layers for SPD manifolds, and MLR on SPSD manifolds.
result Demonstrates improved performance in human action recognition and node classification tasks.
Machine learning predicts US and EuroZone business cycles with high accuracy.
problem Predicting the business cycle phases in US and EuroZone.
method Three machine learning approaches were compared: Multinomial Logistic Regression (MLR) achieved the best results.
result MLR achieved 65.25% accuracy for EuroZone and 75% for US in predicting business cycle phases.
Paper develops a consistent model selection framework for learning Hypotheses Space from data.
problem Avoiding overfitting in complex spaces with limited data.
method Develops a model selection framework based on Learning Spaces, selecting a Hypotheses Space from data.
result The method converges with probability one to a target Hypotheses Space, providing a consistent framework for model selection.
Paper tackles efficient learning of non-convex hypotheses in metric spaces.
problem Efficiently find consistent hypotheses for non-convex hypotheses composed of possibly several disconnected regions.
method Proposes a general domain-independent algorithm for finding consistent weakly convex hypotheses and proves sufficient conditions for its efficiency.
result Shows that consistent hypothesis finding problem can be solved in polynomial time for a broad class of weakly convex hypotheses over metric spaces.
Boosting combines weak hypotheses to create accurate predictions under bounded VC dimension.
problem How to combine weak hypotheses to achieve accurate predictions efficiently.
method Designing a novel boosting algorithm with complex aggregation rules for bounded VC dimension classes.
result The new boosting algorithm requires fewer weak hypotheses than classical lower bounds, provided they belong to a bounded VC class.
DivDis learns diverse hypotheses from underspecified data to improve robustness.
problem Learning from underspecified datasets leads to multiple equally viable solutions, causing out-of-distribution issues.
method DivDis framework: 1) learns diverse hypotheses using unlabeled test data, 2) selects one hypothesis with minimal additional supervision.
result DivDis finds robust features in image and natural language processing problems.
OpenML is an online machine learning platform where researchers can easily share data, machine learning tasks and experiments as well as organize them online to work and collaborate more efficiently. In this paper, we present an R package to interface with the OpenML platform and illustrate its usage in combination wit…
Machine Learning benefits from prior information and computational power for better performance and understanding.
problem Improper use of Machine Learning methods leads to lack of understanding and performance issues.
method Employing prior information and computational power to solve learning problems, emphasizing interpretability and performance.
result Combining prior information and computational power can lead to better understanding and performance in Machine Learning.
New method uses LLMs to generate detailed scientific hypotheses.
problem Generating detailed, actionable scientific hypotheses from coarse initial directions.
method Hierarchical search method that incrementally adds details to hypotheses.
result Hierarchical search method consistently outperforms strong baselines on expert-annotated hypotheses.
Sequential tests for nonparametric hypotheses using supermartingales.
problem Designing valid sequential tests for nonparametric null hypotheses.
method Using elicitable and identifiable functionals, nonnegative supermartingales, and Online Convex Optimization.
result Rigorous guarantees on asymptotic power for a wide range of alternative hypotheses.
In many practical applications of multiple hypothesis testing using the False Discovery Rate (FDR), the given hypotheses can be naturally partitioned into groups, and one may not only want to control the number of false discoveries (wrongly rejected null hypotheses), but also the number of falsely discovered groups of …
We propose a method to generate multiple diverse and valid human pose hypotheses in 3D all consistent with the 2D detection of joints in a monocular RGB image. We use a novel generative model uniform (unbiased) in the space of anatomically plausible 3D poses. Our model is compositional (produces a pose by combining par…
Hypothesis testing in singular models is fundamentally about identifiable vs. non-identifiable parameters.
problem Testing in singular models is inherently problematic due to non-identifiability and degeneracy of Fisher information.
method Formalized the overlap obstruction and showed that hypotheses over non-identifiable parameters are untestable, while those over identifiable parameters reduce to classical testing.
result Hypotheses over non-identifiable parameters are untestable, while those over identifiable parameters reduce to classical testing.
The paper uses RL to verify hypotheses, overcoming existing limitations.
problem Verifying hypotheses using reinforcement learning.
method Formulated hypothesis verification as an RL problem, exploiting hypothesis structure.
result RL agents can successfully verify hypotheses, even those not factorizable.
There is a significant literature on methods for incorporating knowledge into multiple testing procedures so as to improve their power and precision. Some common forms of prior knowledge include (a) beliefs about which hypotheses are null, modeled by non-uniform prior weights; (b) differing importances of hypotheses, m…
Attention-based encoder decoder network uses a left-to-right beam search algorithm in the inference step. The current beam search expands hypotheses and traverses the expanded hypotheses at the next time step. This traversal is implemented using a for-loop program in general, and it leads to speed down of the recogniti…
The paper analyzes different neural network architectures for 3D point cloud processing.
problem Understanding the effects of specific intermediate-layer network architectures on 3D point cloud processing.
method Designing five metrics to diagnose various types of DNNs and conducting comparative studies.
result The hypotheses on the effects of specific intermediate-layer network architectures on the representation capacity of DNNs are verified.
s-RBFN integrates multiple hypotheses for efficient and diverse prediction.
problem Integrating multiple hypotheses into learning models for regression.
method Structured Radial Basis Function Network (s-RBFN) using Voronoi tessellations and least-squares training.
result s-RBFN achieves superior generalization and efficiency compared to other models.
It was proved in 1998 by Ben-David and Litman that a concept space has a sample compression scheme of size d if and only if every finite subspace has a sample compression scheme of size d. In the compactness theorem, measurability of the hypotheses of the created sample compression scheme is not guaranteed; at the same…
Current statistical inference problems in areas like astronomy, genomics, and marketing routinely involve the simultaneous testing of thousands -- even millions -- of null hypotheses. For high-dimensional multivariate distributions, these hypotheses may concern a wide range of parameters, with complex and unknown depen…
Confirmation bias leads to biased estimates in noisy data analysis.
problem Confirmation bias affects scientific conclusions in noisy data environments.
method Investigation of confirmation bias in Gaussian mixture models using K-means and EM algorithms.
result Estimates from algorithms are biased and resemble initial hypotheses, not the noise.
The paper proves weaker conditions for global smoothings of special Lagrangian submanifolds with conical singularities.
problem Conditions for global smoothings of special Lagrangian submanifolds with isolated conical singularities.
method Proof of weaker conditions for global smoothings.
result Global smoothings are possible under weaker hypotheses than previously known.
Proposes Siegel neural networks for improved classification tasks.
problem Classification on Siegel spaces is underexplored.
method Uses quotient structure and vector-valued distance notation.
result Demonstrates state-of-the-art performance in radar and node classification.
We present mlrMBO, a flexible and comprehensive R toolbox for model-based optimization (MBO), also known as Bayesian optimization, which addresses the problem of expensive black-box optimization by approximating the given objective function through a surrogate regression model. It is designed for both single- and multi…