New clustering algorithms capture time-evolving clusters using Markov models.
problem Capturing time-evolving clusters in data.
method Small-variance asymptotic analysis of Markov chain mixture models.
result Two clustering algorithms (D-Means and SD-Means) outperform existing methods in accuracy and computational cost.
In this note we provide detailed derivations of two versions of small-variance asymptotics for hierarchical Dirichlet process (HDP) mixture models and the HDP hidden Markov model (HDP-HMM, a.k.a. the infinite HMM). We include derivations for the probabilities of certain CRP and CRF partitions, which are of more general…
Markov jump processes (MJPs) are used to model a wide range of phenomena from disease progression to RNA path folding. However, maximum likelihood estimation of parametric models leads to degenerate trajectories and inferential performance is poor in nonparametric models. We take a small-variance asymptotics (SVA) appr…
Topic models have emerged as fundamental tools in unsupervised machine learning. Most modern topic modeling algorithms take a probabilistic view and derive inference algorithms based on Latent Dirichlet Allocation (LDA) or its variants. In contrast, we study topic modeling as a combinatorial optimization problem, and p…
Develops fast inference for nonparametric Bayesian LFRM.
problem Inference in LFRM is challenging and slow.
method Small-variance asymptotics framework for nonparametric Bayesian LFRM.
result Deterministic inference algorithms are fast and competitive.
Bayesian hierarchical clustering (BHC) is an agglomerative clustering method, where a probabilistic model is defined and its marginal likelihoods are evaluated to decide which clusters to merge. While BHC provides a few advantages over traditional distance-based agglomerative clustering algorithms, successive evaluatio…
The classical mixture of Gaussians model is related to K-means via small-variance asymptotics: as the covariances of the Gaussians tend to zero, the negative log-likelihood of the mixture of Gaussians model approaches the K-means objective, and the EM algorithm approaches the K-means algorithm. Kulis & Jordan (2012) us…
New concentration inequality for U-statistics of Markov chains.
problem Proving a concentration inequality for U-statistics of order two in uniformly ergodic Markov chains.
method Inductive analysis using martingale techniques, uniform ergodicity, Nummelin splitting, and Bernstein's inequality.
result Recovery of convergence rate for U-statistics of independent random variables and canonical kernels, with improved results for dependent kernels.
This work provides a semi-analytic approximation method for decoupled forwardbackward SDEs (FBSDEs) with jumps. In particular, we construct an asymptotic expansion method for FBSDEs driven by the random Poisson measures with σ-finite compensators as well as the standard Brownian motions around the small-variance limit …
Method minimizes electricity procurement cost based on demand prediction errors.
problem Minimizing electricity procurement cost in spot markets.
method Formulate method to minimize procurement cost over two parameters.
result Minimizes total electricity cost with known unit prices and prediction errors.
Recently, there has been considerable progress on designing algorithms with provable guarantees -- typically using linear algebraic methods -- for parameter learning in latent variable models. But designing provable algorithms for inference has proven to be more challenging. Here we take a first step towards provable i…
DIFF2 improves differential privacy in nonconvex optimization with better utility bounds.
problem Improving differential privacy in nonconvex optimization with better utility bounds.
method DIFF2 constructs a differential private global gradient estimator using gradient differences.
result DIFF2 achieves a utility of \(\widetilde O(d^{2/3}/(n\varepsilon_{\mathrm{DP}})^{4/3})\), significantly better than \(\widetilde O(\sqrt{d}/(n\varepsilon_{\mathrm{DP}}))\).
We consider a stochastic bandit problem with infinitely many arms. In this setting, the learner has no chance of trying all the arms even once and has to dedicate its limited number of samples only to a certain number of arms. All previous algorithms for this setting were designed for minimizing the cumulative regret o…
The Dirichlet process mixture (DPM) is a ubiquitous, flexible Bayesian nonparametric statistical model. However, full probabilistic inference in this model is analytically intractable, so that computationally intensive techniques such as Gibb's sampling are required. As a result, DPM-based methods, which have considera…
We aim to design strategies for sequential decision making that adjust to the difficulty of the learning problem. We study this question both in the setting of prediction with expert advice, and for more general combinatorial decision tasks. We are not satisfied with just guaranteeing minimax regret rates, but we want …
Study evaluates uncertainty quantification for atomistic neural networks, revealing complex relationships between error and uncertainty.
problem Uncertainty quantification for predictions of atomistic neural networks.
method Modified PhysNet NN architecture, evaluated with various metrics, analyzed QM9 and tautomerization reaction databases.
result Error and uncertainty are not linearly related; redundancy and noise complicate predictions, especially for small changes.
In this paper, we propose a model-based clustering method (TVClust) that robustly incorporates noisy side information as soft-constraints and aims to seek a consensus between side information and the observed data. Our method is based on a nonparametric Bayesian hierarchical model that combines the probabilistic model …
Optimizes ranking from click feedback in a bandit setting.
problem Learning to rank from Bernoulli click feedback in a bandit setting.
method Variance-aware confidence sets derived from Bernstein and Chernoff bounds for optimal algorithms.
result Optimal algorithms for the case of small mean rewards, improving on previous suboptimal results.
DLNs dynamics change with variance, leading to saddle-to-saddle training phases.
problem Understanding the dynamics of DLNs with varying initialization variance.
method Analyzing the phase transition of DLNs' dynamics as variance changes.
result Gradient descent visits a sequence of saddles, reaching a sparse global minimum.
k-means derived from Gaussian mixture models with isotropic Gaussians.
problem Clustering with Gaussian mixture models.
method Truncated variational EM approximations applied to Gaussian Mixture Models.
result k-means is a special case of variational EM for Gaussian Mixture Models.
Study compares priors for ABNs to improve model accuracy.
problem Inadequate priors lead to model selection issues in ABNs.
method Simulation study with three priors: Gaussian, Student's t, and strongly informative Gaussian.
result Informative Student's t-prior performs best, mitigating Lindley's paradox.
Quantum codes linked to abelian varieties, providing mathematical rigor.
problem Quantum error correction through complex abelian varieties.
method Mathematical formulation of Gottesman-Kitaev-Preskill codes using abelian varieties.
result Asymptotic isometry of encoding, precise gate realizations, and failure probability optimization.
LMC algorithm converges to target in Chi-squared and Renyi divergence.
problem Sampling from target distribution using LMC with strong dissipativity and smoothness conditions.
method LMC algorithm with strong dissipativity and first-order smoothness, initialized with Gaussian.
result LMC reaches ε-neighborhood of target in Chi-squared and Renyi divergence in O(λ²dε⁻¹) steps.
ANT learns sparse embeddings for large vocabularies efficiently.
problem Lack of scalable methods for embedding large vocabularies in neural networks.
method Anchor & Transform (ANT) algorithm that learns a small set of anchor embeddings and a sparse transformation matrix.
result ANT achieves stronger performance with fewer parameters (up to 40x compression) compared to existing methods.
New algorithm samples neural network posteriors efficiently.
problem Challenges of sampling multimodal Bayesian posteriors for neural networks.
method Greedy Bayes method using log-concave coupling of posterior and auxiliary random variable.
result Log-concave coupling facilitates efficient sampling of neuron weights.
Efficient private matrix analysis algorithms for recent variants.
problem Private analysis of recent matrix updates.
method Identifying sufficient conditions on positive semidefinite matrices.
result First efficient differentially private algorithms for various matrix analysis tasks.
Lecture notes on analysis tools for X-ray tomography.
problem Understanding X-ray tomography using mathematical analysis.
method Overview of analysis tools and ideas, minimal assumptions.
result Broad overview of analysis tools for X-ray tomography.
The paper explores machine learning in mobile big data analysis.
problem Challenges in mobile big data analysis.
method Discussion and review of existing methods.
result Identification of main challenges and future directions.
This paper introduces compositional data analysis for financial ratios, improving industry-level analysis.
problem Statistical issues with standard financial ratios at industry level.
method Compositional data analysis techniques for financial ratios.
result Improved analysis of financial ratios using compositional data methods.
In this dissertation, the main goal is visualisation of financial time series. We expect that visualisation of financial time series will be a useful auxiliary for technical analysis. Firstly, we review the technical analysis methods and test our trading rules, which are built by the essential concepts of technical ana…
Paper combines geometry and time-series analysis for spatiotemporal data.
problem Multivariate time-series data from multiple sensors.
method Combines manifold learning, Riemannian geometry, and spectral analysis.
result Proposes Riemannian multi-resolution analysis (RMRA) for dynamic mode extraction.
In this paper the exact linear relation between the leading eigenvectors of the modularity matrix and the singular vectors of an uncentered data matrix is developed. Based on this analysis the concept of a modularity component is defined, and its properties are developed. It is shown that modularity component analysis …
Paper uses dynamic analysis to detect malware with PHMMs.
problem Malware detection using static and dynamic analysis techniques.
method Hidden Markov Models (HMMs) and Profile Hidden Markov Models (PHMMs) trained on API call sequences.
result PHMMs outperform HMMs in malware detection.
This paper simplifies complex geometry for shape analysis.
problem Understanding interactions between differential geometry and functional analysis.
method Provides an overview of infinite-dimensional Riemannian manifolds and metrics.
result Roadmap for beginners in computational anatomy and shape analysis.
Interactive DR framework for comparing datasets.
problem Limited flexibility in existing DR methods for comparative analysis.
method Unified linear comparative analysis (ULCA) with interactive optimization and visualization.
result ULCA and optimization algorithm improve comparative analysis efficiency and flexibility.
This paper reviews R packages for automating data analysis tasks.
problem Time-consuming Exploratory Data Analysis in large, noisy data sets.
method Systematic review of 12 R packages for autoEDA.
result Identifies automated tasks and areas for future development.
Improves transparency of deep neural networks through feature and consistency analysis.
problem Black-box nature of deep learning inference limits transparency for safety-critical systems.
method Structural and linguistic feature analysis, consistency analysis.
result 75% of human workers found input data and results consistent, 70% found inference and results consistent.
Proposes a method to optimize class mean preservation in kernel-based feature spaces.
problem Optimizing the selection of kernel subspace for better performance.
method Component analysis method for kernel-based dimensionality reduction that optimally preserves class mean distances.
result Discriminant analysis version of the proposed method provides insights into feature space properties.
Proposes a multivariate regression model for better analysis of multiple datasets.
problem Insufficient performance of single-dataset analysis in integrative studies.
method Sparse estimation for variable and group selection, alternating direction method of multipliers algorithm.
result Demonstrated improved performance through simulations and real data analysis.
Combines topological and geometric approaches to data analysis.
problem Understanding when and how geometric objects intersect.
method Connects topological and geometric concepts of curvature.
result Reconceptualizes curvature and links it to hyperconvexity.
New method uses topological data analysis to study stock market crashes.
problem Characterizing and predicting stock market crashes.
method Topological data analysis, persistence landscape, dynamic time series analysis.
result Demonstrates effectiveness of new method for Flash Crash characterization and prediction.
PARSEC compresses text for sentiment analysis with minimal loss in accuracy.
problem Compressing text data for sentiment analysis without losing accuracy.
method Uses Parts-of-Speech tags to compress text intelligently.
result Accurate compression is possible with minimal loss in sentiment classification accuracy.
A primer on navigating causal analysis problems.
problem Estimating causal effects from observational data.
method Four schools of thought for causal analysis.
result Conceptual map for causal analysis.
Novel sensitivity analysis for neural networks improves model interpretability.
problem Improving interpretability of complex probabilistic models.
method Bayesian neural networks with latent variables and sensitivity analysis.
result Increases interpretability of black-box probabilistic models.
Analyzes stock trends and e-commerce user behavior using Twitter data.
problem Understanding the relationship between stock prices, stock news, and e-commerce user behavior.
method Cross-domain analysis using Hadoop, Hive, and Tableau on three datasets.
result Identified correlations between stock sentiment, stock trends, and e-commerce user behavior.
Genetic programming optimizes Gaussian kernels for better sentiment analysis.
problem Improving accuracy of sentiment analysis in text.
method Genetic Programming applied to evolve more effective Gaussian kernels.
result The evolved kernels outperform traditional Gaussian Processes in sentiment analysis.
This work improves group data analysis using modified tensor decompositions.
problem Improving group data analysis models for better signal modeling.
method Introduces a new generalization of block tensor decomposition for group data analysis.
result Demonstrates improved performance in multilabel classification and clustering tasks.
Study analyzes Disney stock market performance using machine learning.
problem Forecasting stock market performance of Disney.
method Exploratory data analysis, feature engineering, model selection (linear regression).
result Linear regression model performed best.