The paper introduces a new volatility model for state heterogeneous financial markets using high-frequency data.
problem State heterogeneity in financial volatility processes.
method Developed a state heterogeneous GARCH-Ito (SG-Ito) model based on continuous Ito diffusion process.
result Empirical studies reveal various state heterogeneities in S&P 500 index volatility.
CCs learn high-dimensional distributions from heterogeneous data.
problem Learning high-dimensional distributions from heterogeneous data.
method Introducing characteristic circuits (CCs) that learn from data and use spectral domain.
result CCs outperform state-of-the-art density estimators on common benchmark data sets.
Generative models unify heterogeneous data for multimodal fusion.
problem Learning effective representations of mixed numerical and categorical data.
method Bayesian approach with exponential family distributions and Laplace-Bernstein approximation.
result Generative models enable fusion of multimodal data for various machine learning tasks.
VSAE learns from missing heterogeneous data by modeling latent dependencies.
problem Learning from partially-observed heterogeneous data with missingness.
method Variational selective autoencoder (VSAE) models joint distribution of observed, unobserved, and missing data.
result VSAE improves over state-of-the-art models in data generation and imputation tasks.
Scalable model learns from mixed data groups.
problem Heterogeneous data affecting predictive models and interpretability.
method Joint learning of feature distributions, regression models, and latent group labels.
result Effective in high dimensions, combining data reduction and re-weighting.
HL-VAE extends VAE for heterogeneous temporal and longitudinal data.
problem Handling heterogeneous data in temporal and longitudinal datasets.
method Proposes HL-VAE, an extension of existing VAEs for temporal and longitudinal data, incorporating likelihood models for various data types.
result HL-VAE achieves competitive performance in missing value imputation and predictive accuracy.
Flexible model captures varying scales in data clusters.
problem Real-world data often exhibits varying scales or intensities, violating the homogeneity assumption of classical Gaussian mixture models.
method Individual-heterogeneous sub-Gaussian mixture model with an efficient spectral method for exact recovery.
result The method provably achieves exact recovery of true cluster labels under mild separation conditions.
We consider the task of meta-analysis in high-dimensional settings in which the data sources are similar but non-identical. To borrow strength across such heterogeneous datasets, we introduce a global parameter that emphasizes interpretability and statistical efficiency in the presence of heterogeneity. We also propose…
FLAME auto-labels mobile data efficiently on diverse processors.
problem Accurately and efficiently labeling mobile data with unknown labels on heterogeneous processors.
method Self-adaptive auto-labeling system Flame that schedules and executes workloads on mobile processors.
result Flame achieves high labeling accuracy and performance on heterogeneous mobile processors.
UNTIE learns representations of coupled categorical data.
problem Challenges in learning from unlabeled categorical data with complex couplings.
method UNTIE approach for unsupervised representation learning of heterogeneous couplings.
result UNTIE significantly improves categorical data representations on 25 diverse datasets.
New bounds for LDP with heterogeneous privacy levels guaranteeing high probability of accuracy.
problem Statistical estimation under LDP with users having varying privacy levels.
method Developed finite sample upper bounds in ℓ_2-norm with high probability, complemented by lower bounds.
result Optimal guarantees for heterogeneous LDP in terms of probability and constants.
Proposes a robust estimator for high-dimensional data with heterogeneous treatment effects.
problem Estimating heterogeneous treatment effects with many more regressors than observations.
method Doubly robust two-stage semiparametric difference-in-difference estimator using machine learning for propensity score estimation.
result Valid inference for heterogeneous treatment effects with bias correction procedures.
A new method corrects weight values to improve treatment effect estimation.
problem Estimating heterogeneous treatment effects in high-dimensional data with sample selection bias.
method Differentiable Pareto-Smoothed Weighting (DPSW) framework.
result Our method outperforms existing methods in treatment effect estimation.
A novel transfer learning framework combines multiple data sources for PU learning.
problem Challenges in PU learning due to lack of negative labels and data scarcity.
method Model averaging of heterogeneous data sources, including binary labeled, semi-supervised, and PU data.
result Method outperforms other methods in predictive accuracy and robustness, especially under limited labeled data.
Efficiently solves heterogeneous QPs by reducing variables using instance-specific projections.
problem Solving high-dimensional quadratic programming problems efficiently.
method Data-driven framework with a graph neural network generating projections tailored to each QP instance.
result Produces high-quality solutions with reduced computation time, outperforming existing methods.
SUOD accelerates OD for large, diverse models.
problem Training and scoring new samples with many unsupervised, heterogeneous OD models.
method Data reduction, model approximation, and taskload optimization.
result SUOD accelerates OD for over 20 benchmark datasets and a real-world case.
Bayesian method learns causal orderings from heterogeneous data.
problem Learning causal structure from heterogeneous data.
method Order-based Bayesian framework for Gaussian DAG models.
result Causal ordering is identifiable up to two permutations.
Mixtures-of-Experts (MoE) are conditional mixture models that have shown their performance in modeling heterogeneity in data in many statistical learning approaches for prediction, including regression and classification, as well as for clustering. Their estimation in high-dimensional problems is still however challeng…
S-DIDML integrates structural DID with ML for causal inference in high-dimensional data.
problem Causal inference in high-dimensional observational panel data with confounding variables.
method Structural identification with high-dimensional estimation, Neyman orthogonality, cross-fitting, causal forests, semi-parametric models.
result Precision in identifying policy-sensitive groups and optimizing resource allocation.
Variational autoencoders (VAEs), as well as other generative models, have been shown to be efficient and accurate for capturing the latent structure of vast amounts of complex high-dimensional data. However, existing VAEs can still not directly handle data that are heterogenous (mixed continuous and discrete) or incomp…
New method stabilizes private LASSO for high-dimensional data with diverse covariate scales.
problem Privacy constraints and heterogeneity in covariate scales degrade LASSO stability and accuracy.
method Gram-based anisotropic objective perturbation to counteract covariate structure.
result Significantly improves convergence and statistical efficiency of private LASSO estimators.
Proposes BGNN for tumor heterogeneity prediction using graph neural networks.
problem Tumor classification limitations and heterogeneity assessment challenges.
method Artificial data generation, tumor heterogeneity estimation, and BGNN model development.
result BGNN achieves 89.67% accuracy in predicting tumor heterogeneity. Dynamic CBDT improves treatment effect estimation in clinical data.
problem Estimating heterogeneous treatment effects in observational data with high accuracy and interpretability.
method Dynamic Regularized Causal Boosted Decision Trees (CBDT) integrating variance regularization and calibration.
result Significantly improved estimation accuracy and reliable coverage of true treatment effects.
Proposes a transfer learning method for high-dimensional quantile regression.
problem Inadequate handling of heterogeneity and heavy tails in transfer learning.
method High-dimensional quantile regression framework with double transfer learning estimator.
result Established error bounds and valid confidence intervals for high-dimensional quantile regression coefficients.
Graphical model has been widely used to investigate the complex dependence structure of high-dimensional data, and it is common to assume that observed data follow a homogeneous graphical model. However, observations usually come from different resources and have heterogeneous hidden commonality in real-world applicati…
Model predicts drug overdose hotspots using EMS and toxicology data.
problem Predicting drug overdose hotspots to focus limited services.
method Spatial-temporal point process model integrating EMS and toxicology data.
result Model improves prediction accuracy by integrating heterogeneous data.
New method identifies nonstationary causal structures in time series data.
problem Identifying causal relationships in time series data that change over time.
method High-order Markov Switching Models for regime-dependent causal discovery.
result Scalable approach for estimating high-order regime-dependent causal structures.
When devising a course of treatment for a patient, doctors often have little quantitative evidence on which to base their decisions, beyond their medical education and published clinical trials. Stanford Health Care alone has millions of electronic medical records (EMRs) that are only just recently being leveraged to i…
New method corrects missing data bias in dimension reduction.
problem Missing data complicates high-dimensional data analysis.
method Developed a bias-corrected Gram matrix for heterogeneous missingness.
result Proposed method improves dimension reduction techniques significantly.
Single global merging boosts decentralized learning performance.
problem Limited communication in decentralized learning hinders performance.
method Scheduled communication, focusing on final step with global merging.
result Single global merging improves global test performance.
New framework discovers PDEs from sparse, noisy data.
problem Discovering PDEs with high-order derivatives and heterogeneous parameters.
method Combines deep-learning and integral form to handle sparse and noisy data.
result More robust and accurate compared to existing methods.
CRE discovers interpretable subgroups with heterogeneous treatment effects.
problem Identifying subgroups with notable treatment effect heterogeneity.
method Causal Rule Ensemble (CRE) using an ensemble-of-trees approach.
result CRE offers interpretable decision rules and high stability in subgroup discovery.
New methods improve distributed optimization on non-iid data.
problem Communication bottleneck in distributed machine learning models.
method Two types of distributed gradient compression methods (D-QSGD and D-EF-SGD) analyzed for non-iid data.
result D-EF-SGD performs better than D-QSGD on non-iid data but can still slow down with high data skewness.
A new method for clustering heterogeneous data using likelihood-adjusted SDP.
problem Clustering heterogeneous data with different cluster shapes and sizes.
method Iterative likelihood-adjusted semidefinite programming (iLA-SDP) method.
result iLA-SDP achieves lower mis-clustering errors compared to other methods.
Mixture of Experts (MoE) are successful models for modeling heterogeneous data in many statistical learning problems including regression, clustering and classification. Generally fitted by maximum likelihood estimation via the well-known EM algorithm, their application to high-dimensional problems is still therefore c…
CTSyn generates high-quality synthetic tabular data.
problem Challenges in generating high-quality synthetic tabular data.
method Diffusion-based generative foundation model with autoencoder and conditional latent diffusion.
result CTSyn outperforms existing table synthesizers on standard benchmarks.
Every design choice will have different effects on different units. However traditional A/B tests are often underpowered to identify these heterogeneous effects. This is especially true when the set of unit-level attributes is high-dimensional and our priors are weak about which particular covariates are important. How…
Develops HCQRF for estimating heterogeneous treatment effects with censored data.
problem Estimating heterogeneous treatment effects on censored responses with high-dimensional variables.
method Hybrid Censored Quantile Regression Forest (HCQRF) combining random forests and censored quantile regression.
result Demonstrates the effectiveness and stability of HCQRF through simulation studies and real-world application.
Paper proposes a novel MTL framework for personalized modeling of diverse individuals.
problem Personalized modeling of heterogeneous subpopulations with high-dimensional data.
method Low-rank tensor decomposition for multi-task learning.
result Superior performance compared to benchmarks in diverse subpopulation scenarios.
Securities markets are quintessential complex adaptive systems in which heterogeneous agents compete in an attempt to maximize returns. Species of trading agents are also subject to evolutionary pressure as entire classes of strategies become obsolete and new classes emerge. Using an agent-based model of interacting he…
New framework tackles stochastic latent subgroup heterogeneity in online decision-making.
problem Stochastic latent heterogeneity in online decision-making where individual responses vary with unobserved subgroups.
method Latent heterogeneous bandit framework using EM-greedy algorithm to learn subgroup probabilities and reward parameters.
result Achieves optimal estimation and classification guarantees, revealing a fundamental stochastic barrier in online decision-making.
We study the fundamental tradeoffs between statistical accuracy and computational tractability in the analysis of high dimensional heterogeneous data. As examples, we study sparse Gaussian mixture model, mixture of sparse linear regressions, and sparse phase retrieval model. For these models, we exploit an oracle-based…
Paper proposes inference method for high-dimensional censored quantile regression.
problem Identifying heterogeneous effects of high-dimensional genetic biomarkers on survival outcomes.
method Combines low-dimensional model estimates based on multi-sample splittings and variable selection.
result Proposed estimator is consistent and asymptotically follows a Gaussian process.
A new method for clustering functional data outperforms existing methods.
problem Clustering heterogeneous functional linear regression data.
method funWeightClust, a family of parsimonious models based on cluster weighted models.
result funWeightClust outperforms existing methods in simulations and real-world traffic analysis.
The paper improves classification accuracy by leveraging a shared signal across domains in high-dimensional classification.
problem Improving classification accuracy in high-dimensional data with shared signals across domains.
method Transfer learning for linear discriminant analysis, decomposing mean differences into common and domain-specific components.
result Deterministic limits for transfer performance, leading to optimal weights and corrections for bias.
Distance metric learning (DML) plays a crucial role in diverse machine learning algorithms and applications. When the labeled information in target domain is limited, transfer metric learning (TML) helps to learn the metric by leveraging the sufficient information from other related domains. Multi-task metric learning …
AdapDISCOM tackles high-dimensional multimodal data with missingness and errors, improving prediction and biomarker selection.
problem High-dimensional multimodal data with block-wise missingness and measurement errors.
method AdapDISCOM introduces modality-specific weighting schemes to address heterogeneity and error magnitudes.
result AdapDISCOM consistently outperforms existing methods under heterogeneous contamination and heavy-tailed distributions.
Paper introduces PHI to identify structurally distinct payment patterns in UK municipal procurement.
problem Vulnerability of public procurement to error, fraud, and corruption in high-volume transactions.
method Introduces Payment Heterogeneity Index (PHI) using Gaussian Mixture Model (GMM) and non-parametric statistics.
result Identifies a significant cohort with structurally distinct payment patterns, improving procurement oversight.