New method discovers time series motifs in datasets with missing data.
problem Missing data hinders motif discovery in time series.
method Admissible time series motif discovery technique for datasets with missing data.
result Proves method is admissible, producing no false negatives.
New sampling method improves search efficiency in machine learning.
problem Efficiently sampling effective solutions from large search spaces.
method Developed a parameterized family of coverage-based designs and algorithms for effective synthesis.
result Consistently outperforms existing exploratory sampling methods in sample mining and hyper-parameter optimization.
ABDA automatically analyzes data without expert supervision.
problem Automatic exploratory data analysis for mixed data types.
method Automatic Bayesian Density Analysis (ABDA) for missing value estimation, data type and likelihood discovery, anomaly detection, and dependency structure mining.
result ABDA provides accurate density estimation and is suitable for mixed data types.
A popular tool for unsupervised modelling and mining multi-aspect data is tensor decomposition. In an exploratory setting, where and no labels or ground truth are available how can we automatically decide how many components to extract? How can we assess the quality of our results, so that a domain expert can factor th…
In the field of exploratory data mining, local structure in data can be described by patterns and discovered by mining algorithms. Although many solutions have been proposed to address the redundancy problems in pattern mining, most of them either provide succinct pattern sets or take the interests of the user into acc…
Improves PARAFAC tensor decomposition rank estimation for better interpretability and accuracy.
problem Estimating the optimal number of latent factors in PARAFAC tensor decomposition.
method Automatically determines the rank using Core Consistency Diagnostic (CORCONDIA) and explores the trade-off between interpretability and predictive accuracy.
result Striking a good balance between interpretability and accuracy benefits rank estimation.
Randomization helps verify if data mining results are due to inherent patterns.
problem Verify if data mining results are due to inherent patterns or coincidental findings.
method Metropolis sampling based on local swaps to randomize data while preserving discovered patterns.
result Randomized data often reveals that clustering results imply frequent pattern discovery.
Paper proposes a new framework to mine synergistic formulaic alphas for better stock trend forecasting.
problem Mining alphas separately ignores their combined performance, leading to suboptimal models.
method Proposes a reinforcement learning-based framework that optimizes the mining of synergistic formulaic alpha sets.
result Demonstrates higher returns in stock trend forecasting compared to previous approaches.
We propose a new class of metrics on sets, vectors, and functions that can be used in various stages of data mining, including exploratory data analysis, learning, and result interpretation. These new distance functions unify and generalize some of the popular metrics, such as the Jaccard and bag distances on sets, Man…
A-DOGE embeds attributed graphs efficiently using density of states.
problem Efficiently represent node-attributed graphs with few numerical features.
method A-DOGE uses density of states to blend topology and attributes, leveraging efficient approximation algorithms.
result A-DOGE achieves competitive performance with modern supervised GNNs while being significantly faster.
Mining itemsets that are the most interesting under a statistical model of the underlying data is a commonly used and well-studied technique for exploratory data analysis, with the most recent interestingness models exhibiting state of the art performance. Continuing this highly promising line of work, we propose the f…
This paper quantifies privacy loss in exploratory data analysis.
problem Privacy loss in exploratory data analysis is often overlooked in privacy budgets.
method Quantitative analysis of privacy loss for statistical functions.
result Privacy loss must be considered in calculating machine learning privacy budgets.
This paper reviews R packages for automating data analysis tasks.
problem Time-consuming Exploratory Data Analysis in large, noisy data sets.
method Systematic review of 12 R packages for autoEDA.
result Identifies automated tasks and areas for future development.
A new visualization tool MD plot discovers interesting structures in continuous features.
problem Identifying interesting structures in data distributions, especially with skewed, clipped, or multimodal distributions.
method Proposes a new visualization tool called the mirrored density plot (MD plot) that does not require adjusting density estimation parameters.
result The MD plot outperforms conventional methods in identifying structures in complex distributions.
Study finds key investing characteristics for success in equity markets.
problem Understanding what traits lead to financial success in equity markets.
method Exploratory factor analysis and multiple linear regression on 403 respondents' data.
result Investing characteristics significantly impact individual investors' excess return.
Data preprocessing improves data quality for robust data mining.
problem Noisy and incomplete data hinders data mining models.
method Overview of data cleaning, transformation, and preprocessing methods.
result Preprocessing significantly affects data mining model performance.
Study uses big data to analyze quantum invariants.
problem Investigate structural properties of Jones polynomial.
method Exploratory and topological data analysis, including coloring, rank increase, categorification.
result Contrasts behavior of Jones polynomial under various enhancements.
The paper extracts structured data from physician-patient conversations, reducing clerical burden.
problem Mining insights from physician-patient conversations for electronic health record documentation.
method Created a dataset of transcripts and summaries, extracted noteworthy utterances, and improved model performance.
result Extracting noteworthy utterances significantly boosts model performance for recognizing diagnoses and RoS abnormalities.
In this paper, we propose a new algorithm for exploratory projection pursuit. The basis of the algorithm is the insight that previous approaches used fairly narrow definitions of interestingness / non interestingness. We argue that allowing these definitions to depend on the problem / data at hand is a more natural app…
DataLearner simplifies data mining on Android devices.
problem Lack of general-purpose data-mining tools for mobile devices.
method Augments Weka engine with Charles Sturt University algorithms, providing 40 mining algorithms.
result Delivers classification accuracy similar to PCs/laptops with acceptable speed and battery life.
Action chunking and data exploration improve behavior cloning in robotics.
problem Exponential errors in learning from demonstrations for continuous control tasks.
method Action chunking and exploratory data collection.
result Control-theoretic stability is key to improving imitation learning.
Paper mines rank data patterns from rankings.
problem Mining rank data patterns from rankings.
method Proposes algorithms for frequent rankings and dependencies.
result Experimental validation of algorithms on synthetic and real data.
Agricultural research has been profited by technical advances such as automation, data mining. Today, data mining is used in a vast areas and many off-the-shelf data mining system products and domain specific data mining application soft wares are available, but data mining in agricultural soil datasets is a relatively…
Study compares PLS, Lasso, and RVM for soft sensor design, finding Lasso superior.
problem Improving predictive performance of soft sensors in high-dimensional data.
method Comparison of three regression techniques (PLS, Lasso, RVM) on real and simulated datasets.
result Lasso outperforms PLS in offline learning with large predictor sets, but PLS remains superior with small MW sizes.
Automates organizing diverse web data into a hierarchical topic model.
problem Manual classification of all scientific and popular scientific knowledge is impractical.
method Proposes an algorithm to aggregate multiple collections into a single hierarchical topic model.
result Demonstrates a web service for topical exploratory search.
We propose in this paper an exploratory analysis algorithm for functional data. The method partitions a set of functions into K clusters and represents each cluster by a simple prototype (e.g., piecewise constant). The total number of segments in the prototypes, P, is chosen by the user and optimally distributed am…
This paper categorizes and analyzes existing outlying aspect mining methods.
problem Finding unique features in data objects that differ from others.
method Grouping and analyzing existing outlying aspect mining approaches in three categories.
result Comparison of strengths, weaknesses, and time complexities of different techniques.
New method explains high-dimensional sphere data with latent factors.
problem Understanding intricate dependence structure in high-dimensional sphere data.
method Exploratory factor analysis of the projected normal distribution with a fast alternating expectation profile conditional maximization algorithm.
result Uniformly excellent results on various data types, including tweets, brain imaging, and cancer gene expression.
Data mining enhances a heuristic for the Minimum Latency Problem.
problem Finding optimal solutions for the Minimum Latency Problem efficiently.
method Combining GRASP with data mining to find frequent patterns in high-quality solutions.
result Improved solution quality and reduced computational time compared to existing methods.
Estimating the strength of dependency between two variables is fundamental for exploratory analysis and many other applications in data mining. For example: non-linear dependencies between two continuous variables can be explored with the Maximal Information Coefficient (MIC); and categorical variables that are depende…
A new tool, matrix profile, finds all pair similarities in time series data.
problem Finding all pair similarities in time series data.
method Near universal time series data mining tool called matrix profile.
result Matrix profile solves the all-pairs-similarity-search problem for time series subsequences.
Safe Pattern Pruning reduces pattern explosion in predictive pattern mining.
problem Exponential growth of patterns in structured data.
method Safe Pattern Pruning (SPP) method.
result Effective model building in practical data analysis.
Study speculative trading using RL with exploratory framework.
problem Sequential optimal stopping problem over entry and exit times with general utility function and price process.
method Formulated as a sequential optimal stopping problem, solved using Cox processes driven by bounded, non-randomized intensity controls. Characterized randomized control via probability measure over jump intensities and regularized objective function by Shannon's entropy. Established error estimates and convergence of RL objective to value function.
result Closed-form solutions for optimal policy and value function are derived.
RL solves large-scale MV portfolio allocation with high returns.
problem Large-scale mean-variance portfolio optimization.
method Continuous-time reinforcement learning with a multivariate Gaussian policy.
result Our method outperforms econometric and deep RL methods by significant margins.
Breaks the hardness conjecture for batch RL with a novel tournament-based approach.
problem Sample-efficient reinforcement learning from exploratory data.
method BVFT algorithm using pairwise comparison and state-action partition.
result Solves the learning problem in a setting previously thought impossible.
Peer-reviewed research and mined data predict stock returns similarly.
problem Predicting stock returns using research quality.
method Cross-sectional analysis of 29,000 accounting ratios with t-statistics > 2.0.
result Post-sample performance is largely independent of whether the predictor is peer-reviewed or mined.
This paper discusses issues in mining user behavioral rules for context-aware mobile apps.
problem Mining contextual behavioral rules from smartphone data.
method Addressing quality of data, relevancy of contexts, discretization, rule discovery, semantic understanding, and dynamic rule updating.
result Potential solutions for mining user behavioral rules for context-aware mobile apps.
The paper explores how mining costs, rewards, and blockchain security are interconnected.
problem Understanding the interdependencies between mining costs, mining rewards, and blockchain security.
method Theoretical derivation and empirical analysis using daily crypto market data and autoregressive distributed lag approach.
result Cryptocurrency price and mining rewards are intrinsically linked to blockchain security outcomes.
Data mining techniques predict breast cancer types with high accuracy.
problem Early detection of breast cancer to reduce mortality rates.
method Twelve classification algorithms applied to the Breast Cancer Wisconsin dataset.
result High accuracy in predicting malignant and benign breast cancer.
Study of entropy-regularized LQG MFGs with exploratory actions.
problem Optimizing multi-population mean field games with entropy regularization.
method Introduced exploratory actions and derived optimal action distributions.
result Optimal action distributions lead to ε-Nash equilibria in finite-population MFGs.
This paper introduces C-DSL to improve data mining outcomes by considering context.
problem Data collection ambiguities, data imbalance, hidden biases, lack of domain info, and data incompleteness.
method Developed Context-Driven Data Science Lifecycle (C-DSL) to address data quality issues.
result Tangible improvements to data mining outcomes were achieved through C-DSL.
FedSLIM optimizes compact pattern models across distributed databases without sharing raw data.
problem Privacy-preserving federated descriptive analytics for data silos.
method Federated MDL-based framework using SLIM principle.
result FedSLIM variants preserve high-quality compression structure and recover globally informative patterns.
Automatically mined rules from dependency parsing help neural models learn from less labeled data.
problem Lack of labeled data for aspect and opinion term extraction.
method Automatically mined rules from dependency parsing, applied to auxiliary data, combined with human-annotated data.
result Neural models achieve better performance than state-of-the-art with mined rules and auxiliary data.
Novel process model for metabolomics data analysis.
problem Analyzing complex metabolomics data.
method Data-driven and hypothesis-driven data mining approaches using various techniques.
result Demonstrated applicability and strengths of MeKDDaM model.
Uncharted Forest visualizes data associations for classification and provenance studies.
problem Exploratory data analysis in high-dimensional datasets.
method Unsupervised tree ensemble (uncharted forest) for partitioning and visualizing data.
result Visualizes class associations, sample associations, and class heterogeneity.
Confidence intervals improve evaluation of binary prediction rules in data mining.
problem Uncertainty in performance measures estimation from finite datasets.
method Asymptotic normal approximations for confidence intervals, with a blurring correction.
result Improved finite sample coverage probabilities and general performance measures inference.
A new method speeds up factor analysis for high-dimensional data.
problem Estimating covariance parameters in high-dimensional Gaussian data with limited observations.
method Matrix-free likelihood method using implicitly restarted Lanczos and limited-memory quasi-Newton algorithms.
result Our method is faster than EM without sacrificing accuracy.
This document describes an approach to the problem of predicting dangerous seismic events in active coal mines up to 8 hours in advance. It was developed as a part of the AAIA'16 Data Mining Challenge: Predicting Dangerous Seismic Events in Active Coal Mines. The solutions presented consist of ensembles of various pred…