This paper quantifies privacy loss in exploratory data analysis.
problem Privacy loss in exploratory data analysis is often overlooked in privacy budgets.
method Quantitative analysis of privacy loss for statistical functions.
result Privacy loss must be considered in calculating machine learning privacy budgets.
The increasing availability of large but noisy data sets with a large number of heterogeneous variables leads to the increasing interest in the automation of common tasks for data analysis. The most time-consuming part of this process is the Exploratory Data Analysis, crucial for better domain understanding, data clean…
Study uses big data to analyze quantum invariants.
problem Investigate structural properties of Jones polynomial.
method Exploratory and topological data analysis, including coloring, rank increase, categorification.
result Contrasts behavior of Jones polynomial under various enhancements.
Study finds key investing characteristics for success in equity markets.
problem Understanding what traits lead to financial success in equity markets.
method Exploratory factor analysis and multiple linear regression on 403 respondents' data.
result Investing characteristics significantly impact individual investors' excess return.
Making sense of a dataset in an automatic and unsupervised fashion is a challenging problem in statistics and AI. Classical approaches for {exploratory data analysis} are usually not flexible enough to deal with the uncertainty inherent to real-world data: they are often restricted to fixed latent interaction models an…
We propose in this paper an exploratory analysis algorithm for functional data. The method partitions a set of functions into K clusters and represents each cluster by a simple prototype (e.g., piecewise constant). The total number of segments in the prototypes, P, is chosen by the user and optimally distributed am…
New method explains high-dimensional sphere data with latent factors.
problem Understanding intricate dependence structure in high-dimensional sphere data.
method Exploratory factor analysis of the projected normal distribution with a fast alternating expectation profile conditional maximization algorithm.
result Uniformly excellent results on various data types, including tweets, brain imaging, and cancer gene expression.
ACA identifies and explains anomalies in data.
problem Explaining anomalies in non-supervised data analysis.
method Abnormal Component Analysis (ACA) using data depth.
result ACA provides a linear explanation for anomalies.
This paper proposes a novel profile likelihood method for estimating the covariance parameters in exploratory factor analysis of high-dimensional Gaussian datasets with fewer observations than number of variables. An implicitly restarted Lanczos algorithm and a limited-memory quasi-Newton method are implemented to deve…
Archetypal analysis helps understand binary data sets.
problem Explaining binary questionnaire data.
method Using archetypal analysis for binary observations.
result The approach contributes to understanding binary data sets.
This study analyzes data science vocabulary changes over 13 years.
problem Understanding evolution of data science terms over time.
method Exploratory Data Analysis, Latent Semantic Analysis, Latent Dirichlet Analysis, N-grams Analysis.
result Identified new vocabulary and its incorporation into scientific literature.
Action chunking and data exploration improve behavior cloning in robotics.
problem Exponential errors in learning from demonstrations for continuous control tasks.
method Action chunking and exploratory data collection.
result Control-theoretic stability is key to improving imitation learning.
Paper introduces probabilistic methods to approximate archetypal analysis, reducing complexity.
problem Inherent computational complexity of archetypal analysis limits its practical applicability.
method Two preprocessing techniques: dimensionality reduction and representation cardinality reduction, using probabilistic geometry.
result The method effectively reduces scaling and provides near-optimal solutions for prediction errors.
Deep learning speeds up IFA estimation for large datasets.
problem Slow MML estimation for large-scale IFA models.
method Importance-weighted autoencoder (IWAE) for fast VI.
result IWAE yields accurate estimates faster than MH-RM.
Biarchetype analysis identifies extreme instances of observations and features.
problem Representing complex data structures in a more interpretable form.
method Solves biarchetype analysis through an algorithm that identifies biarchetypes as mixtures of observations and features.
result Biarchetypes enhance interpretability of data structures compared to traditional methods.
A new method clusters mixed-type data tables effectively.
problem Clustering data with mixed types (numerical and categorical).
method Two-step approach: binarize mixed data, then co-cluster.
result Shows improved clustering of mixed-type data compared to MCA.
This describes a statistical technique called "tonsuring" for exploratory data analysis in finance. Instead of rejecting "outlier" data that conflicts with the model, this strips out "inlier" data to get a clearer picture of how the market changes for larger moves.
Social media analytics allows us to extract, analyze, and establish semantic from user-generated contents in social media platforms. This study utilized a mixed method including a three-step process of data collection, topic modeling, and data annotation for recognizing exercise related patterns. Based on the findings,…
As data collections become larger, exploratory regression analysis becomes more important but more challenging. When observations are hierarchically clustered the problem is even more challenging because model selection with mixed effect models can produce misleading results when nonlinear effects are not included into…
Study speculative trading using RL with exploratory framework.
problem Sequential optimal stopping problem over entry and exit times with general utility function and price process.
method Formulated as a sequential optimal stopping problem, solved using Cox processes driven by bounded, non-randomized intensity controls. Characterized randomized control via probability measure over jump intensities and regularized objective function by Shannon's entropy. Established error estimates and convergence of RL objective to value function.
result Closed-form solutions for optimal policy and value function are derived.
Sampling strategies significantly affect feature approximations in ELA, impacting classifier accuracy.
problem The impact of sampling strategies on feature approximations in ELA.
method Analysis of feature approximations from different sampling strategies and sample sizes.
result Feature approximations from different sampling strategies do not converge, affecting classifier accuracy.
A-DOGE embeds attributed graphs efficiently using density of states.
problem Efficiently represent node-attributed graphs with few numerical features.
method A-DOGE uses density of states to blend topology and attributes, leveraging efficient approximation algorithms.
result A-DOGE achieves competitive performance with modern supervised GNNs while being significantly faster.
Factor analysis aims to determine latent factors, or traits, which summarize a given data set. Inter-battery factor analysis extends this notion to multiple views of the data. In this paper we show how a nonlinear, nonparametric version of these models can be recovered through the Gaussian process latent variable model…
CEDA improves understanding of data fit to models.
problem Real-world data often deviates from theoretical models.
method Categorical Exploratory Data Analysis (CEDA) to highlight deviations.
result CEDA reveals where and how data fits or deviates from models.
Study of entropy-regularized LQG MFGs with exploratory actions.
problem Optimizing multi-population mean field games with entropy regularization.
method Introduced exploratory actions and derived optimal action distributions.
result Optimal action distributions lead to ε-Nash equilibria in finite-population MFGs.
The paper tackles confidence calibration for exploratory machine learning problems.
problem Difficulty in curating datasets and confusion about category validity.
method Introduces four new algorithms for category-specific confidence estimation, including kernel density ratios.
result Kernel density ratios provide a novel approach to confidence calibration, especially for exploratory problems.
Study analyzes Disney stock market performance using machine learning.
problem Forecasting stock market performance of Disney.
method Exploratory data analysis, feature engineering, model selection (linear regression).
result Linear regression model performed best.
ClusterGraph visualizes and simplifies multidimensional data clusters for better understanding.
problem Lack of global structure information in clustering results.
method Combining clustering with Topological Data Analysis to provide global structure.
result ClusterGraph provides global layout information about clusters.
Study predicts adverse events in Afghanistan using time series data.
problem Predicting the number of negative events in Afghanistan's theater of war.
method Regression analysis on time series data, non-conventional aggregation of districts, machine learning models.
result Predictive models show reasonable performance on historical data, but other variables do not improve prediction quality.
Exploratory data analysis is crucial for developing and understanding classification models from high-dimensional datasets. We explore the utility of a new unsupervised tree ensemble called uncharted forest for visualizing class associations, sample-sample associations, class heterogeneity, and uninformative classes fo…
TensorWatch enables real-time interactive analysis of deep learning training.
problem Challenges in diagnosing and exploring deep learning models during training.
method Modeling inspection and diagnostic tasks as streams using a map-reduce paradigm.
result Real-time interactive queries on live deep learning training processes.
In this paper, we propose a new algorithm for exploratory projection pursuit. The basis of the algorithm is the insight that previous approaches used fairly narrow definitions of interestingness / non interestingness. We argue that allowing these definitions to depend on the problem / data at hand is a more natural app…
Wasserstein t-SNE embeds hierarchical datasets considering within-unit distributions.
problem Exploring hierarchical datasets where units are compared based on means of sample distributions.
method Uses Wasserstein distance metric for 2D embeddings of units, approximating Gaussian distributions for efficiency.
result Demonstrates effective embedding of hierarchical datasets, uncovering meaningful structure.
In this paper, we build upon previous work on designing informative and efficient Exploratory Landscape Analysis features for characterizing problems' landscapes and show their effectiveness in automatically constructing algorithm selection models in continuous black-box optimization problems. Focussing on algorithm pe…
New method tackles unknown unknowns in machine learning.
problem Unknown classes in training data misperceived as other labels.
method Exploratory machine learning with rejection model, feature exploration, and model cascade.
result The method discovers potentially hidden classes and improves model performance.
Repository tackles fake health news in cancer research.
problem Spread of fake health news over the internet.
method Developed comprehensive FakeHealth repository with rich features and detailed explanations.
result Repository helps in understanding and validating health fake news datasets.
A framework for robust exploration in reinforcement learning under ambiguity.
problem Optimal stopping under ambiguity in reinforcement learning.
method Continuous-time robust reinforcement learning framework using g-expectation and backward stochastic differential equations. result Constructs a robust exploratory stopping time approximating the optimal stopping time under ambiguity.
The paper analyzes Lending Club's loan applicants to predict default risk.
problem Predicting default risk in loan applicants of Lending Club.
method Exploratory data analysis and machine learning (Logistic Regression, Random Forest) were used.
result A credit derivative based on Credit Default Swap was designed to hedge default risk.
Proposes a flexible feature allocation model for sparse factor analysis.
problem Sparse data and rigid assumptions in traditional exploratory tools.
method Adaptive latent feature sharing with control over feature sparsity.
result Derives a novel adaptive Factor analysis (aFA) and aPPCA for flexible dimensionality reduction.
DECE visualizes machine learning decisions with counterfactual explanations.
problem Making machine learning models transparent and explainable.
method Interactive visualization system supporting counterfactual explanations at instance- and subgroup-levels.
result DECE enables users to explore and understand machine learning model decisions.
Sampling one or more effective solutions from large search spaces is a recurring idea in machine learning, and sequential optimization has become a popular solution. Typical examples include data summarization, sample mining for predictive modeling and hyper-parameter optimization. Existing solutions attempt to adaptiv…
This paper analyzes the quantitative relations between stock prices and quantities of tradable stock shares in Chinese stock markets at six time points by means of Exploratory Data Analysis (EDA) method. It is found the resulting formulae have the same structure but different parameters. This paper also uses these rela…
Modern data is messy and high-dimensional, and it is often not clear a priori what are the right questions to ask. Instead, the analyst typically needs to use the data to search for interesting analyses to perform and hypotheses to test. This is an adaptive process, where the choice of analysis to be performed next dep…
Study on utility maximization with Tsallis entropy in reinforcement learning.
problem Exploring utility maximization with Tsallis entropy in reinforcement learning.
method Introducing Tsallis entropy regularizer to induce exploration, investigating specific examples, characterizing well-posedness, designing reinforcement learning algorithm.
result Characterized well-posedness and provided semi-closed-form solutions for specific examples, found distinct optimal strategies.
Latent feature modeling allows capturing the latent structure responsible for generating the observed properties of a set of objects. It is often used to make predictions either for new values of interest or missing information in the original data, as well as to perform data exploratory analysis. However, although the…
Python package for functional data analysis.
problem Handling and analysis of functional data.
method Comprehensive tools for representation, preprocessing, and exploratory analysis of functional data.
result Scikit-fda package provides a comprehensive set of tools for functional data analysis.
Neural model with parameterized algorithms improves graph CO problem solving.
problem Solving NP-hard graph combinatorial optimization problems efficiently and accurately.
method Combining neural models and parameterized algorithms to identify and handle hard and easy parts of CO instances.
result Framework produces superior solution quality and out-of-distribution generalization.
Analyzing and understanding the structure of complex relational data is important in many applications including analysis of the connectivity in the human brain. Such networks can have prominent patterns on different scales, calling for a hierarchically structured model. We propose two non-parametric Bayesian hierarchi…