Develops new algorithms for QRF to handle mixed-frequency and longitudinal data.
problem Handling mixed-frequency and longitudinal data in quantile regression.
method Mixed-Frequency Quantile Regression Forest (MIDAS-QRF) and Finite Mixture Quantile Regression Forest (FM-QRF).
result Valid and flexible models for complex empirical settings in financial risk management and climate-change impact evaluation.
Paper introduces a new IV regression method for mixed-frequency data.
problem Estimating high-dimensional slope parameters in mixed-frequency data.
method Tikhonov-regularized estimator for high-dimensional linear IV regression.
result High-dimensional slope parameter can be accurately estimated using a low-frequency instrumental variable.
The paper proposes a mixed-frequency quantile regression model for VaR and ES forecasting.
problem Forecasting VaR and ES with mixed-frequency data.
method Mixed-frequency quantile regression model to estimate VaR and ES.
result The proposed model outperforms other models in VaR and ES backtesting tests.
Paper uses machine learning for nowcasting corporate earnings from mixed-frequency data.
problem Predicting corporate earnings for a large cross-section of firms with different frequency data.
method Structured machine learning regressions with sparse-group LASSO regularization for panel data.
result Machine learning models outperform traditional methods in nowcasting corporate earnings.
Bayesian machine learning methods improve nowcasting with mixed frequency data.
problem Handling frequency mismatches and predicting short-term economic indicators.
method Developed Gaussian process (GP) methods for MIDAS regressions.
result Gaussian process-MIDAS methods offer gains in predictive accuracy.
Bayesian econometrics improves nowcasting during pandemics.
problem Improving nowcasting during extreme economic events like pandemics.
method Bayesian econometric methods using non-parametric mixed frequency VARs with additive regression trees.
result Significant improvements in nowcasting performance compared to linear models.
LSTM models improve macroeconomic forecasting with mixed frequency data.
problem Improving accuracy of macroeconomic forecasts using mixed frequency data.
method Adapted LSTM model to mixed frequency data, using U-MIDAS scheme.
result Proposed LSTM models outperform conventional MIDAS models in out-of-sample predictive performance.
Transformer model with mixed-frequency data improves stock volatility prediction.
problem Improving stock volatility prediction using mixed-frequency data.
method Transformer model trained on mixed-frequency data (GARCH-MIDAS model for frequency alignment).
result Transformer model reduces mean square error from 1.00 to 0.86.
MPTE uses Transformer attention to estimate mixed-frequency factor models.
problem Estimating factor models in panel datasets with mixed frequencies and nonlinear signals.
method Mixed-Panels-Transformer Encoder (MPTE) with attention mechanisms.
result MPTE achieves competitive performance in nonlinear forecasting environments.
The paper forecasts corporate distress using a novel MIDAS logistic regression method.
problem Forecasting corporate distress with right-censored data, high-dimensional predictors, and mixed-frequency data.
method The paper introduces a novel high-dimensional censored MIDAS logistic regression method that handles censoring through inverse probability weighting and employs a sparse-group penalty for mixed-frequency predictors.
result The method achieves accurate estimation and superior performance in predicting financial distress of Chinese-listed firms.
HANET combines LSTM and attention mechanisms for better financial forecasting.
problem Lack of distinct macroeconomic regimes in financial datasets.
method Hierarchical Cross-Attention mechanism integrating long-run macro contexts with high-frequency market dynamics.
result HANET outperforms neural forecasters, especially during turbulent periods.
Paper develops a new estimator for high-dimensional panel data with common shocks.
problem Cross-sectionally dependent errors driven by common shocks in high-dimensional panel data.
method Factor-augmented sparse-group LASSO estimator combining MIDAS aggregation with latent factors.
result The estimator outperforms standard LASSO for prediction and estimation in settings with cross-sectional dependence.
The paper improves machine learning for heavy-tailed panel data.
problem Improving estimates for financial and economic data with fat tails.
method Sparse-group LASSO regularization and Fuk-Nagaev concentration inequality.
result Oracle inequalities for panel data estimators.
Proposes a deep spectral Q-learning for mobile health data.
problem Personalized treatment assignment for patients with time-varying covariates.
method Integrates PCA with deep Q-learning for mixed frequency data.
result Mean return converges to optimal under estimated optimal policy.
Granger causality reviewed and advanced for complex data.
problem Validity of inferring causal relationships from time series data.
method Recent advances in models for high-dimensional time series, accounting for nonlinear and non-Gaussian observations, and sub-sampled data.
result Improved computational tools for Granger causality.
Model predicts real-time job applicant numbers for regional economic analysis.
problem Real-time economic analysis using alternative data.
method Mixed-Frequency Aggregate Learning (MF-AGL) model.
result Model accurately predicts regional labor market conditions and economic status changes.
This paper reviews deep learning methods for state space models.
problem Analyzing temporal dynamics in dynamical systems.
method Selective review of deep neural network approaches for state space models.
result Unified perspective on discrete and continuous time SSMs.
Alternative wavelet analysis method for financial signals.
problem Analyzing oscillations in financial signals with noise.
method Modeling financial signals as isolated events producing ripples of various frequencies.
result Element analysis distinguishes between noise and logically matched generators.
Study uses non-parametric method to analyze EU ETS price determinants.
problem Understanding price determinants of EU ETS to inform policy.
method Non-parametric measure (Information Imbalance) to study variables.
result Commodity variables are most informative in Phase 3, while financial variables become more important in Phase 4.
Big data sets must be carefully partitioned into statistically similar data subsets that can be used as representative samples for big data analysis tasks. In this paper, we propose the random sample partition (RSP) data model to represent a big data set as a set of non-overlapping data subsets, called RSP data blocks,…
Prevents sensitive data generation in diffusion models using labeled and unlabeled data.
problem Generating sensitive data in diffusion models using unlabeled data.
method Positive-Unlabeled Diffusion Models, approximating ELBO with labeled and unlabeled data.
result Prevents the generation of sensitive data without compromising image quality.
Study reveals Data Shapley's inconsistent performance in data selection tasks.
problem Inconsistency of Data Shapley's performance in data selection across different settings.
method Hypothesis testing framework and identification of utility functions.
result Data Shapley's performance is no better than random selection without specific constraints.
PRRO generates synthetic tabular data that improves SL performance and class distribution.
problem Low SL utility of synthetic data due to class imbalance and overlooked data relationships.
method Data pruning and column reordering to optimize SL utility.
result Synthetic data generated with PRRO enhances predictive performance and class distribution.
Defines data science as a natural ecosystem with challenges and missions.
problem Challenges and missions in data science due to 5D complexities and data life cycle phases.
method Systemic and data-centric view of data science as a fusion of data universe and its challenges, formalizing a general-purpose architecture.
result Essential data science as a natural ecosystem integrating specific disciplines and high-impact applications.
Differences in data size per class, also known as imbalanced data distribution, have become a common problem affecting data quality. Big Data scenarios pose a new challenge to traditional imbalanced classification algorithms, since they are not prepared to work with such amount of data. Split data strategies and lack o…
Synthetic data enhances analytics but requires careful volume management.
problem Accuracy of statistical methods on synthetic data vs. raw data.
method Synthetic Data Generation for Analytics framework using tabular diffusion models.
result Error rate decreases with more synthetic data but may stabilize or increase.
New test ensures quality of shared data in machine learning.
problem Ensuring quality of external data in machine learning tasks.
method Distribution-free two-sample testing procedures grounded in conformal outlier detection.
result Identifies valuable external data agents for model personalization.
Differential privacy allows quantifying privacy loss resulting from accessing sensitive personal data. Repeated accesses to underlying data incur increasing loss. Releasing data as privacy-preserving synthetic data would avoid this limitation, but would leave open the problem of designing what kind of synthetic data. W…
Paper creates fair synthetic data ensuring equal predictions across sensitive attributes.
problem Ensuring fair predictions across sensitive attributes in synthetic data.
method Equalizing target probability distributions across sensitive attributes in synthetic data generation.
result Synthetic data provides strong fair predictions, equal across all thresholds.
DPA preserves data distribution in reduced dimensions.
problem Loss of data distribution in dimension reduction.
method DPA combines encoder and decoder to match data distribution.
result DPA successfully reconstructs data distribution.
Efficient synthetic data generation improves model performance on tabular data.
problem Improving model robustness and performance with scarce or low-quality data.
method Hardness characterization to identify high-value training points, generating synthetic data only from these points.
result Synthetic data generated from hardest points outperforms non-targeted methods on tabular datasets.
For most problems in science and engineering we can obtain data sets that describe the observed system from various perspectives and record the behavior of its individual components. Heterogeneous data sets can be collectively mined by data fusion. Fusion can focus on a specific target relation and exploit directly ass…
DAERNN models censored data using neural networks with data augmentation.
problem Handling censored data in expectile regression.
method Data augmentation based Expectile Regression Neural Networks (ERNNs).
result DAERNN outperforms existing censored ERNNs methods and achieves comparable predictive performance to fully observed data.
Data preprocessing techniques are devoted to correct or alleviate errors in data. Discretization and feature selection are two of the most extended data preprocessing techniques. Although we can find many proposals for static Big Data preprocessing, there is little research devoted to the continuous Big Data problem. A…
Data stream classification methods demonstrate promising performance on a single data stream by exploring the cohesion in the data stream. However, multiple data streams that involve several correlated data streams are common in many practical scenarios, which can be viewed as multi-task data streams. Instead of handli…
Data collection is a major bottleneck in machine learning and an active research topic in multiple communities. There are largely two reasons data collection has recently become a critical issue. First, as machine learning is becoming more widely-used, we are seeing new applications that do not necessarily have enough …
This paper quantifies uncertainty in Data Shapley using statistical inference.
problem Uncertainty in data valuation due to dynamic data distribution.
method Established relationship with U-statistics and quantified uncertainty using statistical inference.
result Confidence intervals for Data Shapley estimations are provided.
DCoM uses deep neural networks to detect semantic data types from raw column values.
problem Detecting semantic data types from dirty and unseen data.
method DCoM employs multi-input NLP-based deep neural networks trained on 686,765 data columns.
result DCoM outperforms existing methods significantly on 78 different semantic data types.
Causal inference from observational data is the goal of many data analyses in the health and social sciences. However, academic statistics has often frowned upon data analyses with a causal objective. The introduction of the term "data science" provides a historic opportunity to redefine data analysis in such a way tha…
Task-agnostic data valuation without validation requirements.
problem Valuing data without specific task assumptions.
method Estimating data diversity and relevance through queries without raw data.
result Estimates capture the diversity and relevance of seller's data for the buyer.
Data mining is about obtaining new knowledge from existing datasets. However, the data in the existing datasets can be scattered, noisy, and even incomplete. Although lots of effort is spent on developing or fine-tuning data mining models to make them more robust to the noise of the input data, their qualities still st…
New algorithm improves data imputation for complex multimodal data sets.
problem Artifacts in imputation methods for multimodal distributions.
method Combines kNN and KDE for probabilistic estimates. result Lower imputation errors and higher likelihood estimates.
New algorithms for clustering and synthetic data generation of heterogeneous tabular datasets.
problem Clustering and generating synthetic data from heterogeneous tabular datasets with hidden cluster structure.
method Developed MMM and MMMsynth algorithms for clustering and synthetic data generation.
result MMMsynth algorithm outperforms other literature tabular-data generators and approaches real data performance.
WeMix improves data augmentation by correcting bias in deep learning.
problem Data augmentation's effectiveness is limited by data bias.
method Developed AugDrop and MixLoss algorithms to correct data bias.
result WeMix improves data augmentation performance through bias correction.
A new method reduces data valuation variance for more trustworthy data trading.
problem Data valuation and trustworthy data trading in algorithmic prediction.
method Variance reduced Shapley value estimation using stratified sampling.
result VRDS method reduces estimation variance and improves data marketplace development.
Adapts data analysis for growing data, improving generalization guarantees.
problem Challenges of overfitting and statistical validity in adaptive workflows with growing data.
method Generalizes adaptive analysis on dynamic data, incorporating time-varying empirical accuracy bounds and mechanisms.
result First generalization bounds for adaptive analysis on dynamic data, matching prior works' improvement over data splitting.
This work redefines data-centric AI by unifying categorical and cochain notions.
problem The need to rethink data notions for data-centric AI.
method Proposes unifying principles from categorical and cochain notions of data.
result Unified definition of data impacts machine learning development, implementation, and utilization.
SMOTE-DP enhances synthetic data privacy without sacrificing utility.
problem Balancing privacy and utility in synthetic data generation.
method Integrating SMOTE with differential privacy mechanisms.
result SMOTE-DP produces synthetic data that is both private and useful.