This paper quantifies privacy loss in exploratory data analysis.
problem Privacy loss in exploratory data analysis is often overlooked in privacy budgets.
method Quantitative analysis of privacy loss for statistical functions.
result Privacy loss must be considered in calculating machine learning privacy budgets.
The increasing availability of large but noisy data sets with a large number of heterogeneous variables leads to the increasing interest in the automation of common tasks for data analysis. The most time-consuming part of this process is the Exploratory Data Analysis, crucial for better domain understanding, data clean…
Study uses big data to analyze quantum invariants.
problem Investigate structural properties of Jones polynomial.
method Exploratory and topological data analysis, including coloring, rank increase, categorification.
result Contrasts behavior of Jones polynomial under various enhancements.
Making sense of a dataset in an automatic and unsupervised fashion is a challenging problem in statistics and AI. Classical approaches for {exploratory data analysis} are usually not flexible enough to deal with the uncertainty inherent to real-world data: they are often restricted to fixed latent interaction models an…
We propose in this paper an exploratory analysis algorithm for functional data. The method partitions a set of functions into K clusters and represents each cluster by a simple prototype (e.g., piecewise constant). The total number of segments in the prototypes, P, is chosen by the user and optimally distributed am…
New method explains high-dimensional sphere data with latent factors.
problem Understanding intricate dependence structure in high-dimensional sphere data.
method Exploratory factor analysis of the projected normal distribution with a fast alternating expectation profile conditional maximization algorithm.
result Uniformly excellent results on various data types, including tweets, brain imaging, and cancer gene expression.
Study finds key investing characteristics for success in equity markets.
problem Understanding what traits lead to financial success in equity markets.
method Exploratory factor analysis and multiple linear regression on 403 respondents' data.
result Investing characteristics significantly impact individual investors' excess return.
ACA identifies and explains anomalies in data.
problem Explaining anomalies in non-supervised data analysis.
method Abnormal Component Analysis (ACA) using data depth.
result ACA provides a linear explanation for anomalies.
This study analyzes data science vocabulary changes over 13 years.
problem Understanding evolution of data science terms over time.
method Exploratory Data Analysis, Latent Semantic Analysis, Latent Dirichlet Analysis, N-grams Analysis.
result Identified new vocabulary and its incorporation into scientific literature.
Archetypal analysis helps understand binary data sets.
problem Explaining binary questionnaire data.
method Using archetypal analysis for binary observations.
result The approach contributes to understanding binary data sets.
This paper proposes a novel profile likelihood method for estimating the covariance parameters in exploratory factor analysis of high-dimensional Gaussian datasets with fewer observations than number of variables. An implicitly restarted Lanczos algorithm and a limited-memory quasi-Newton method are implemented to deve…
A new method clusters mixed-type data tables effectively.
problem Clustering data with mixed types (numerical and categorical).
method Two-step approach: binarize mixed data, then co-cluster.
result Shows improved clustering of mixed-type data compared to MCA.
Action chunking and data exploration improve behavior cloning in robotics.
problem Exponential errors in learning from demonstrations for continuous control tasks.
method Action chunking and exploratory data collection.
result Control-theoretic stability is key to improving imitation learning.
Paper introduces probabilistic methods to approximate archetypal analysis, reducing complexity.
problem Inherent computational complexity of archetypal analysis limits its practical applicability.
method Two preprocessing techniques: dimensionality reduction and representation cardinality reduction, using probabilistic geometry.
result The method effectively reduces scaling and provides near-optimal solutions for prediction errors.
This describes a statistical technique called "tonsuring" for exploratory data analysis in finance. Instead of rejecting "outlier" data that conflicts with the model, this strips out "inlier" data to get a clearer picture of how the market changes for larger moves.
Social media analytics allows us to extract, analyze, and establish semantic from user-generated contents in social media platforms. This study utilized a mixed method including a three-step process of data collection, topic modeling, and data annotation for recognizing exercise related patterns. Based on the findings,…
Biarchetype analysis identifies extreme instances of observations and features.
problem Representing complex data structures in a more interpretable form.
method Solves biarchetype analysis through an algorithm that identifies biarchetypes as mixtures of observations and features.
result Biarchetypes enhance interpretability of data structures compared to traditional methods.
CEDA improves understanding of data fit to models.
problem Real-world data often deviates from theoretical models.
method Categorical Exploratory Data Analysis (CEDA) to highlight deviations.
result CEDA reveals where and how data fits or deviates from models.
As data collections become larger, exploratory regression analysis becomes more important but more challenging. When observations are hierarchically clustered the problem is even more challenging because model selection with mixed effect models can produce misleading results when nonlinear effects are not included into…
Factor analysis aims to determine latent factors, or traits, which summarize a given data set. Inter-battery factor analysis extends this notion to multiple views of the data. In this paper we show how a nonlinear, nonparametric version of these models can be recovered through the Gaussian process latent variable model…
ClusterGraph visualizes and simplifies multidimensional data clusters for better understanding.
problem Lack of global structure information in clustering results.
method Combining clustering with Topological Data Analysis to provide global structure.
result ClusterGraph provides global layout information about clusters.
Study predicts adverse events in Afghanistan using time series data.
problem Predicting the number of negative events in Afghanistan's theater of war.
method Regression analysis on time series data, non-conventional aggregation of districts, machine learning models.
result Predictive models show reasonable performance on historical data, but other variables do not improve prediction quality.
Study analyzes Disney stock market performance using machine learning.
problem Forecasting stock market performance of Disney.
method Exploratory data analysis, feature engineering, model selection (linear regression).
result Linear regression model performed best.
Exploratory data analysis is crucial for developing and understanding classification models from high-dimensional datasets. We explore the utility of a new unsupervised tree ensemble called uncharted forest for visualizing class associations, sample-sample associations, class heterogeneity, and uninformative classes fo…
In this paper, we propose a new algorithm for exploratory projection pursuit. The basis of the algorithm is the insight that previous approaches used fairly narrow definitions of interestingness / non interestingness. We argue that allowing these definitions to depend on the problem / data at hand is a more natural app…
Marginal maximum likelihood (MML) estimation is the preferred approach to fitting item response theory models in psychometrics due to the MML estimator's consistency, normality, and efficiency as the sample size tends to infinity. However, state-of-the-art MML estimation procedures such as the Metropolis-Hastings Robbi…
Modern data is messy and high-dimensional, and it is often not clear a priori what are the right questions to ask. Instead, the analyst typically needs to use the data to search for interesting analyses to perform and hypotheses to test. This is an adaptive process, where the choice of analysis to be performed next dep…
Wasserstein t-SNE embeds hierarchical datasets considering within-unit distributions.
problem Exploring hierarchical datasets where units are compared based on means of sample distributions.
method Uses Wasserstein distance metric for 2D embeddings of units, approximating Gaussian distributions for efficiency.
result Demonstrates effective embedding of hierarchical datasets, uncovering meaningful structure.
Python package for functional data analysis.
problem Handling and analysis of functional data.
method Comprehensive tools for representation, preprocessing, and exploratory analysis of functional data.
result Scikit-fda package provides a comprehensive set of tools for functional data analysis.
The paper analyzes Lending Club's loan applicants to predict default risk.
problem Predicting default risk in loan applicants of Lending Club.
method Exploratory data analysis and machine learning (Logistic Regression, Random Forest) were used.
result A credit derivative based on Credit Default Swap was designed to hedge default risk.
Latent feature modeling allows capturing the latent structure responsible for generating the observed properties of a set of objects. It is often used to make predictions either for new values of interest or missing information in the original data, as well as to perform data exploratory analysis. However, although the…
New method tackles unknown unknowns in machine learning.
problem Unknown classes in training data misperceived as other labels.
method Exploratory machine learning with rejection model, feature exploration, and model cascade.
result The method discovers potentially hidden classes and improves model performance.
Proposes a flexible feature allocation model for sparse factor analysis.
problem Sparse data and rigid assumptions in traditional exploratory tools.
method Adaptive latent feature sharing with control over feature sparsity.
result Derives a novel adaptive Factor analysis (aFA) and aPPCA for flexible dimensionality reduction.
Performing diagnosis or exploratory analysis during the training of deep learning models is challenging but often necessary for making a sequence of decisions guided by the incremental observations. Currently available systems for this purpose are limited to monitoring only the logged data that must be specified before…
Visual exploration of high-dimensional real-valued datasets is a fundamental task in exploratory data analysis (EDA). Existing methods use predefined criteria to choose the representation of data. There is a lack of methods that (i) elicit from the user what she has learned from the data and (ii) show patterns that she…
cGAP visualizes high-dimensional categorical data with interpretable geometric structure.
problem Lack of visualization tools for high-dimensional categorical data.
method cGAP uses Homogeneity Analysis (HOMALS) to embed data in a 3D space and maps it to colors.
result cGAP reveals coherent clusters, outliers, and local-to-global structure in categorical data.
cGAP visualizes high-dimensional categorical data with interpretable geometric structure.
problem Lack of visualization tools for high-dimensional categorical data.
method cGAP uses Homogeneity Analysis (HOMALS) to embed data in a 3D space and maps it to colors for visualization.
result cGAP reveals coherent clusters, outliers, and local-to-global structure in categorical data.
Yarbus' claim to decode the observer's task from eye movements has received mixed reactions. In this paper, we have supported the hypothesis that it is possible to decode the task. We conducted an exploratory analysis on the dataset by projecting features and data points into a scatter plot to visualize the nuance prop…
Study speculative trading using RL with exploratory framework.
problem Sequential optimal stopping problem over entry and exit times with general utility function and price process.
method Formulated as a sequential optimal stopping problem, solved using Cox processes driven by bounded, non-randomized intensity controls. Characterized randomized control via probability measure over jump intensities and regularized objective function by Shannon's entropy. Established error estimates and convergence of RL objective to value function.
result Closed-form solutions for optimal policy and value function are derived.
Sampling strategies significantly affect feature approximations in ELA, impacting classifier accuracy.
problem The impact of sampling strategies on feature approximations in ELA.
method Analysis of feature approximations from different sampling strategies and sample sizes.
result Feature approximations from different sampling strategies do not converge, affecting classifier accuracy.
New Shapley values reveal non-linear feature dependencies.
problem Understanding non-linear dependencies in machine learning models.
method Model-independent Shapley values using non-parametric measures of dependence.
result Model-independent Shapley values can uncover non-linear dependencies.
This paper analyzes the quantitative relations between stock prices and quantities of tradable stock shares in Chinese stock markets at six time points by means of Exploratory Data Analysis (EDA) method. It is found the resulting formulae have the same structure but different parameters. This paper also uses these rela…
Advances neural tri-factorization for clustering and discordance analysis of multi-typed data.
problem Challenges in analyzing heterogeneous, multimodal relational data.
method Deep collective matrix tri-factorization for spectral clustering and cluster association learning.
result Demonstrates efficacy over previous non-neural approaches in clustering and discordance analysis.
A-DOGE embeds attributed graphs efficiently using density of states.
problem Efficiently represent node-attributed graphs with few numerical features.
method A-DOGE uses density of states to blend topology and attributes, leveraging efficient approximation algorithms.
result A-DOGE achieves competitive performance with modern supervised GNNs while being significantly faster.
Breaks the hardness conjecture for batch RL with a novel tournament-based approach.
problem Sample-efficient reinforcement learning from exploratory data.
method BVFT algorithm using pairwise comparison and state-action partition.
result Solves the learning problem in a setting previously thought impossible.
Study of entropy-regularized LQG MFGs with exploratory actions.
problem Optimizing multi-population mean field games with entropy regularization.
method Introduced exploratory actions and derived optimal action distributions.
result Optimal action distributions lead to ε-Nash equilibria in finite-population MFGs.
New method visualizes noisy data better than existing techniques.
problem Noisy data impairs data visualization methods.
method Functional Information Geometry (FIG) adapts EIG framework using functional data analysis.
result FIG outperforms EIG variant in capturing true structure, robustness, and speed.
The paper tackles confidence calibration for exploratory machine learning problems.
problem Difficulty in curating datasets and confusion about category validity.
method Introduces four new algorithms for category-specific confidence estimation, including kernel density ratios.
result Kernel density ratios provide a novel approach to confidence calibration, especially for exploratory problems.