Improved Naive Bayes classifier with neural network models.
problem Limited complexity handling and independence assumption in Naive Bayes.
method Introducing Neural Naive Bayes and Neural Pooled Markov Chain models.
result Error rate reduced by 4.5 on IMDB dataset.
This work uses PAC-Bayes for structured prediction with ILE, yielding insights and algorithms.
problem Structured prediction with interdependent outputs and implicit loss embeddings.
method PAC-Bayes perspective applied to ILE framework, deriving generalization bounds and learning algorithms.
result Two learning algorithms derived from PAC-Bayes bounds, analyzed and implemented.
Bayes EMbedding refines embeddings from knowledge and behavior graphs.
problem Combining knowledge and behavior graphs for better entity embeddings.
method Bayesian framework integrating pre-trained embeddings from both graphs.
result BEM refines embeddings from both knowledge and behavior graphs.
We introduce Deep Variational Bayes Filters (DVBF), a new method for unsupervised learning and identification of latent Markovian state space models. Leveraging recent advances in Stochastic Gradient Variational Bayes, DVBF can overcome intractable inference distributions via variational inference. Thus, it can handle …
Improved music recommendations using hyperbolic embeddings and Bayesian estimation.
problem Generating personalized music recommendations.
method Hyperbolic Poincaré embeddings for hierarchies, empirical Bayes for link reliability.
result Significant performance improvement in A/B testing.
For graphs generated from stochastic blockmodels, adjacency spectral embedding is asymptotically consistent. Further, adjacency spectral embedding composed with universally consistent classifiers is universally consistent to achieve the Bayes error. However when the graph contains private or sensitive information, trea…
Inference for the stochastic blockmodel is currently of burgeoning interest in the statistical community, as well as in various application domains as diverse as social networks, citation networks, brain connectivity networks (connectomics), etc. Recent theoretical developments have shown that spectral embedding of gra…
A nonparametric approach for policy learning for POMDPs is proposed. The approach represents distributions over the states, observations, and actions as embeddings in feature spaces, which are reproducing kernel Hilbert spaces. Distributions over states given the observations are obtained by applying the kernel Bayes' …
AIM models explain deep learning in attention layers, offering solvable insights.
problem Understanding how deep learning models learn in attention layers.
method Statistical mechanics and random matrix theory.
result Closed-form predictions for Bayes-optimal generalization error and gradient descent performance.
A Hilbert space embedding for probability measures has recently been proposed, wherein any probability measure is represented as a mean element in a reproducing kernel Hilbert space (RKHS). Such an embedding has found applications in homogeneity testing, independence testing, dimensionality reduction, etc., with the re…
Paper proposes MMD-Bayes for robust Bayesian estimation in misspecified models.
problem Inconsistent estimates in Bayesian statistics due to model misspecification.
method Uses Maximum Mean Discrepancy (MMD) to create a robust pseudo-likelihood.
result MMD-Bayes posterior is consistent and robust to model misspecification.
Conditional kernel mean embeddings form an attractive nonparametric framework for representing conditional means of functions, describing the observation processes for many complex models. However, the recovery of the original underlying function of interest whose conditional mean was observed is a challenging inferenc…
Proposes PENNs for deep learning with missing data.
problem Deep learning with missing covariates in multivariate nonparametric regression.
method Pattern Embedded Neural Networks (PENNs) combining imputation and neural networks.
result PENNs achieve minimax rate of convergence for typical cases, improving standard neural networks.
IITK wins FinSim 2020 task on financial hypernym detection.
problem Classifying financial terms into hypernym concepts in an external ontology.
method Used context-dependent and context-independent word embeddings (Word2vec and BERT) for classification.
result Ranked 1st based on mean rank and accuracy metrics.
Naive Bayes spam filters are highly susceptible to data poisoning attacks. Here, known spam sources/blacklisted IPs exploit the fact that their received emails will be treated as (ground truth) labeled spam examples, and used for classifier training (or re-training). The attacking source thus generates emails that will…
Meta-learning method improves PU classification performance.
problem Improving binary classifiers from PU data in unseen tasks.
method Adapts model to PU data using related tasks and neural networks.
result Proposed method outperforms existing methods on synthetic and real-world datasets.
New tests for binary classification regression functions without distribution assumptions.
problem Testing regression functions in binary classification without distributional assumptions.
method Conditional kernel mean embeddings and resampling-based framework.
result Distribution-free hypothesis tests with exact type I error control.
New uniqueness concept for adversarial Bayes classifier.
problem Understanding adversarial Bayes classifiers in binary classification.
method Developed a new notion of uniqueness and analyzed it for a family of one-dimensional data distributions.
result Improved regularity of adversarial Bayes classifiers as perturbation radius increases.
Empirical Bayes rates via variational approximations and prior decomposition.
problem Nonparametric and high-dimensional inference convergence rates.
method Variational perspective and prior decomposition.
result Empirical Bayes posterior rates derived from variational Bayes.
The study compares clustering risk in Hidden Markov and i.i.d. models, showing the Bayes classifier is nearly optimal.
problem Comparing clustering risk in Hidden Markov and i.i.d. models.
method Analysis of Bayes risk, theoretical bounds, and simulations.
result The Bayes classifier is nearly optimal for clustering in both Hidden Markov and i.i.d. models.
This paper introduces an approach for detecting differences in the first-order structures of spatial point patterns. The proposed approach leverages the kernel mean embedding in a novel way by introducing its approximate version tailored to spatial point processes. While the original embedding is infinite-dimensional a…
Rodent hippocampal population codes represent important spatial information about the environment during navigation. Several computational methods have been developed to uncover the neural representation of spatial topology embedded in rodent hippocampal ensemble spike activity. Here we extend our previous work and pro…
Despite its simplicity, the naive Bayes classifier has surprised machine learning researchers by exhibiting good performance on a variety of learning problems. Encouraged by these results, researchers have looked to overcome naive Bayes primary weakness - attribute independence - and improve the performance of the algo…
This paper develops embeddings that preserve likelihood-based statistical inference.
problem Modern machine learning embeddings destroy the geometric structure required for likelihood-based inference.
method Developed a rigorous theory of likelihood-preserving embeddings and introduced the Likelihood-Ratio Distortion metric.
result Controlling the distortion Δn is necessary and sufficient for preserving inference. Researchers prove NP-hardness of learning parameter-bounded Bayes nets.
problem Learning parameter-bounded Bayes nets is computationally hard.
method Proved NP-hardness of learning parameter-bounded Bayes nets and a promise search variant.
result Proved NP-hardness of a promise search variant of LEARN.
Deep learning improves Bayes factor computation for likelihood-free models.
problem Computing Bayes factors for likelihood-free models is challenging.
method Proposes a deep learning estimator of Bayes factors using simulated data.
result Establishes consistency of the Deep Bayes Factor estimator.
Meta-learning bounds derived using PAC-Bayes theory for improved generalization.
problem Uncertainty in generalization performance for meta-learning with new tasks.
method PAC-Bayes relative entropy bounds and empirical risk minimization (ERM) method.
result Competitive generalization performance and rapid convergence with data-dependent prior.
The article analyzes LCE in Hilbert space, deriving new formulas and regularisation methods.
problem Analyzing conditional expectation in infinite-dimensional Hilbert space.
method Establishing analytical properties and regularisation for LCE in Hilbert space, deriving new formulas.
result Simple derivation and intuitive justification of conditional mean embedding formula.
PAC-Bayes framework fails on simple 1D linear classification task.
problem Proving the learnability of simple 1D linear classification tasks using PAC-Bayes bounds.
method Demonstrated a specific 1D linear classification task that PAC-Bayes cannot analyze.
result PAC-Bayes framework cannot prove learnability of simple 1D linear classification tasks.
DKULENOVO team improves speech diarization by 27.5% and 31.7% in DIHARD II.
problem Challenges in speech diarization, especially in distinguishing overlapping speakers.
method Used a combination of VAD, speaker embedding extraction, similarity scoring, clustering, and resegmentation techniques.
result Achieved 18.84% DER in Track1 and 27.90% DER in Track2, reducing baseline DERs by 27.5% and 31.7% respectively.
For random graphs distributed according to stochastic blockmodels, a special case of latent position graphs, adjacency spectral embedding followed by appropriate vertex classification is asymptotically Bayes optimal; but this approach requires knowledge of and critically depends on the model dimension. In this paper, w…
PAC-Bayes bound requires prior to place mass on high-performing predictors.
problem Explaining generalization in machine learning.
method Analyzing necessary conditions for PAC-Bayes bounds to provide meaningful generalization guarantees.
result Achieving a target generalisation level requires the prior to place sufficient mass on high-performing predictors.
New estimators outperform maximum likelihood without hyper-parameter estimation.
problem Improving system identification performance without hyper-parameter estimation.
method Developed generalized Bayes and closed-form biased estimators using excess MSE.
result New estimators have comparable performance to empirical-Bayes-based regularized estimator.
Paper introduces Generalized Naive Bayes for better data fitting.
problem Improving Naive Bayes for better data fitting.
method Developed new greedy and optimal algorithms for GNB.
result Proves GNB fits data at least as well as classical NB.
Smart Bayes integrates generative and discriminative features for improved classification.
problem Improving classification performance by combining generative and discriminative modeling.
method Integrates generative likelihood-ratio features into a logistic-regression-style classifier.
result Often outperforms logistic regression and Naive Bayes in simulations and real data.
New method reduces computational cost for estimating PAC-Bayes bounds.
problem High computational cost in estimating PAC-Bayes bounds.
method General alternative method that makes computational savings.
result Reduces computational cost on the order of the dataset size.
This work bounds classification error in machine learning for low Bayes error conditions.
problem Understanding the error mismatch between Bayes error and model-based classification error.
method Applying classification error bounds to study the relationship with Kullback-Leibler divergence and proposing a linear approximation for low Bayes error conditions.
result A linear approximation of the classification error bound for low Bayes error conditions is proposed.
Study calculates Bayes risk for semi-supervised learning with uncertain labels.
problem Uncertain labeling in semi-supervised classification.
method Gaussian mixture model, Bayes risk computation, comparison with algorithm performance.
result New insights into semi-supervised learning algorithm performance.
We extend a recently proposed 1-nearest-neighbor based multiclass learning algorithm and prove that our modification is universally strongly Bayes-consistent in all metric spaces admitting any such learner, making it an "optimistically universal" Bayes-consistent learner. This is the first learning algorithm known to e…
We propose a vector-valued regression problem whose solution is equivalent to the reproducing kernel Hilbert space (RKHS) embedding of the Bayesian posterior distribution. This equivalence provides a new understanding of kernel Bayesian inference. Moreover, the optimization problem induces a new regularization for the …
Unified derivation of PAC-Bayes and MI bounds for general VC classes with fast rates.
problem Generalization bounds for machine learning models with VC classes.
method Unified derivation of conditional PAC-Bayesian and mutual information bounds, including MAC-Bayesian bounds.
result Nontrivial bounds for general VC classes and faster rates for specific conditions.
We formalize and study the natural approach of designing convex surrogate loss functions via embeddings, for problems such as classification, ranking, or structured prediction. In this approach, one embeds each of the finitely many predictions (e.g.\ rankings) as a point in Rd, assigns the original loss val…
Bayes classifier cannot be learned from noisy labels without knowing noise distribution.
problem Learning a Bayes classifier from noisy labels when the noise distribution is unknown.
method Demonstrates the identifiability issues and proposes a simple algorithm for learning the Bayes decision rule.
result The Bayes decision rule is generally unidentified and cannot be learned without knowing the noise distribution.
This paper improves meta-learning by developing new PAC-Bayes bounds.
problem Meta-learning generalization gap across multiple tasks.
method Upper bounding convex functions linking environment and task-level losses.
result New PAC-Bayes bounds for meta-learning with improved algorithms.
New PAC-Bayes training method improves model generalization for unbounded loss.
problem Improving generalization of complex models under unbounded loss.
method Established new PAC-Bayes bound for unbounded loss, jointly training prior and posterior.
result Outperforms existing PAC-Bayes training algorithms and matches ERM accuracy.
A new method estimates Bayes error for deep networks, suggesting they may have reached the limit.
problem Evaluating the performance of deep learning models and detecting overfitting.
method A simple and direct Bayes error estimator based on uncertainty of class assignments.
result Deep networks may have reached the Bayes error limit for benchmark datasets.
The paper develops a method to accurately estimate the Bayes misclassification error rate.
problem Estimating the best achievable classifier performance without learning a Bayes-optimal classifier.
method Learning to benchmark using an ensemble of ε-ball estimators and Chebyshev approximation.
result The proposed method achieves an optimal mean squared error rate of O(N^(-1)) under a smoothness assumption.
Kernel Bayes' rule has been proposed as a nonparametric kernel-based method to realize Bayesian inference in reproducing kernel Hilbert spaces. However, we demonstrate both theoretically and experimentally that the prediction result by kernel Bayes' rule is in some cases unnatural. We consider that this phenomenon is i…