CICLAD efficiently mines frequent closed itemsets from data streams with minimal memory usage.
problem Mining frequent closed itemsets from data streams is resource-intensive.
method CICLAD is an intersection-based sliding-window FCI miner that optimizes memory usage while maintaining performance.
result CICLAD achieves significantly lower memory footprint compared to existing methods.
Frequent Itemsets (FIs) mining is a fundamental primitive in data mining. It requires to identify all itemsets appearing in at least a fraction θ of a transactional dataset D. Often though, the ultimate goal of mining D is not an analysis of the dataset \emph{per se}, but the understanding of …
Enhances distributed Apriori-like frequent itemsets mining performance.
problem Improving performance of distributed Apriori-like frequent itemsets mining.
method Presented a new distributed approach considering Apriori algorithm's characteristics and distribution aspects.
result The proposed approach significantly enhances performance and achieves good scalability compared to a typical distributed Apriori algorithm.
This paper introduces the combinatorial Boolean model (CBM), which is defined as the class of linear combinations of conjunctions of Boolean attributes. This paper addresses the issue of learning CBM from labeled data. CBM is of high knowledge interpretability but naïve learning of it requires exponentially large compu…
Proposes SCR-Apriori for efficient mining of SCR-patterns.
problem Mining high-quality `Set of Contrasting Rules'-pattern (SCR-pattern) efficiently.
method Integrates SCR-pattern structure into Apriori algorithm to prune search space.
result Significantly reduces computational cost compared to state-of-the-art.
In binary-transaction data-mining, traditional frequent itemset mining often produces results which are not straightforward to interpret. To overcome this problem, probability models are often used to produce more compact and conclusive results, albeit with some loss of accuracy. Bayesian statistics have been widely us…
We present a novel algorithm, Westfall-Young light, for detecting patterns, such as itemsets and subgraphs, which are statistically significantly enriched in one of two classes. Our method corrects rigorously for multiple hypothesis testing and correlations between patterns through the Westfall-Young permutation proced…
The problem of multiple hypothesis testing arises when there are more than one hypothesis to be tested simultaneously for statistical significance. This is a very common situation in many data mining applications. For instance, assessing simultaneously the significance of all frequent itemsets of a single dataset entai…
A new method for selecting high quality itemsets from a large collection.
problem Selecting a small, high-quality subset of patterns from a larger collection of itemsets.
method Using decomposable families of itemsets and a junction tree structure.
result The method provides high quality results and is computationally efficient.
Estimates itemset significance using Maximum Entropy and Kullback-Leibler divergence.
problem Defining the significance of itemsets in data mining.
method Maximum Entropy estimation and Kullback-Leibler divergence for deviation measurement.
result Significance measure goes to zero for derivable itemsets and ranks as a statistical test.
Method detects interactions for better CTR prediction.
problem Predicting click-through rate with high-dimensional categorical features and time-varying interactions.
method Online Random Intersection Chains (ORIC) for detecting informative interactions.
result ORIC detects high-interpretability interactions that improve CTR prediction.
Proposes a new method for finding frequent closed patterns in transaction bases.
problem Frequent closed patterns in transaction bases.
method Partitioning the search space into subcontexts and updating frequent closed patterns with their minimal generators.
result Proposed approach called UFCIGs-DAC for efficient search of frequent closed itemsets.
This paper creates a tagging system for paintings using historical descriptions.
problem Tagging accuracy for paintings over time with varying expert descriptions.
method A neural network with frequent itemsets as tags, followed by clustering.
result Improved tagging accuracy for paintings over time.
Mining itemsets that are the most interesting under a statistical model of the underlying data is a commonly used and well-studied technique for exploratory data analysis, with the most recent interestingness models exhibiting state of the art performance. Continuing this highly promising line of work, we propose the f…
The paper improves itemset quality assessment by incorporating background knowledge.
problem Assessing the quality of discovered itemsets is challenging due to many patterns being explainable by background knowledge.
method The authors introduce a maximum entropy approach to efficiently infuse additional background knowledge such as row margins, lazarus counts, and bounds of ones.
result More sophisticated models that incorporate background knowledge fit the data better and improve frequency prediction of itemsets.
The problem of finding itemsets that are statistically significantly enriched in a class of transactions is complicated by the need to correct for multiple hypothesis testing. Pruning untestable hypotheses was recently proposed as a strategy for this task of significant itemset mining. It was shown to lead to greater s…
Pattern sampling has been proposed as a potential solution to the infamous pattern explosion. Instead of enumerating all patterns that satisfy the constraints, individual patterns are sampled proportional to a given quality measure. Several sampling algorithms have been proposed, but each of them has its limitations wh…
Algorithm extracts non-monotonic rules from statistical models using HUIM.
problem Extracting non-monotonic rules from statistical learning models.
method Reduces problem to HUIM, uses TreeExplainer for feature importance.
result Significant improvement in classification metrics and training time.
OMBA learns product and user representations for better online market basket analysis.
problem Limited ability to uncover rarely occurring and temporal associations in MBA.
method Jointly learns product and user representations, captures temporal dynamics, scalable online method.
result OMBA outperforms state-of-the-art methods by 21% on real-world datasets.
Significant pattern mining, the problem of finding itemsets that are significantly enriched in one class of objects, is statistically challenging, as the large space of candidate patterns leads to an enormous multiple testing problem. Recently, the concept of testability was proposed as one approach to correct for mult…
Scales gradual pattern discovery from imprecise data.
problem Managing large collections of gradual patterns from imprecise data.
method Introduces gradualness thresholds and considers attribute value distribution.
result Eliminates noise patterns, showing a smaller set of meaningful patterns.
The problem of frequent pattern mining has been studied quite extensively for various types of data, including sets, sequences, and graphs. Somewhat surprisingly, another important type of data, namely rank data, has received very little attention in data mining so far. In this paper, we therefore addresses the problem…
Improved ridge regression with Frequent Directions for large-scale tasks.
problem Improving performance of ridge regression for large-scale data.
method Combines Frequent Directions with iterative optimization schemes.
result Achieves high accuracy in estimating bias and variance for sketched ridge regression.
In deep reinforcement learning (RL) tasks, an efficient exploration mechanism should be able to encourage an agent to take actions that lead to less frequent states which may yield higher accumulative future return. However, both knowing about the future and evaluating the frequentness of states are non-trivial tasks, …
Sparse activations in neural models correlate with frequent words, suggesting sparsity is natural.
problem Interpretability and resource efficiency in neural language models.
method Used the Taxi-Euclidean norm to measure sparsity and analyzed gradients and activations of frequent words.
result Frequent input words are associated with sparse activations, while frequent target words are associated with dispersed activations.
Batch uniformization improves anomaly detection in sound data.
problem Anomaly scores for rare and frequent normal sounds are not uniform.
method Propose batch uniformization to minimize anomaly scores by weighting samples based on their density.
result Improves performance of unsupervised anomaly detection in sound data.
Study automates detection of visitation disruptions in ICU patients.
problem Difficulty in detecting frequent visitation disruptions in ICU patients.
method Used DensePose R-CNN model to count people in video frames, analyzed disruptions and patient outcomes.
result Automated method detects visitation disruptions, impacts on pain and length of stay examined.
Blockchain scaling reduces gas fees, allowing more frequent liquidity updates and concentration.
problem Adverse selection risk and high gas fees on decentralized exchanges.
method Instrumental variables analysis using blockchain scaling solutions (Arbitrum, Polygon) as instruments.
result Higher repositioning intensity and precision lead to greater liquidity concentration, benefiting small trades.
Proposes using frequent sequences to improve sequential recommendation models.
problem Combining user history and recent actions for personalized recommendations.
method Uses frequent sequences to identify relevant parts of user history, embedding items based on preferences and dynamics in a unified metric model.
result Outperforms state-of-the-art methods, especially on sparse datasets.
Some high-dimensional data.sets can be modelled by assuming that there are many different linear constraints, each of which is Frequently Approximately Satisfied (FAS) by the data. The probability of a data vector under the model is then proportional to the product of the probabilities of its constraint violations. We …
SGE learns symbolic node representations from relational data.
problem Mining insights from complex, real-world systems.
method SGE uses frequent pattern mining on a node's neighborhood to learn symbolic node representations.
result SGE outperforms shallow node embedding methods on a venue classification task.
Bayesian approach to portfolio selection reduces pessimism in frequent trading.
problem Tackling the challenge of estimating drift in Merton's portfolio selection model.
method Bayesian distributionally robust control with nonlinear Wasserstein projections.
result Reduced pessimism and improved performance in frequent rebalancing compared to existing methods.
Study reveals centralization in Bitcoin transactions involving retail users.
problem Centralization and bias in Bitcoin transaction data.
method Heuristic classification of Bitcoin users, weekly activity pattern analysis.
result Most real transactions involve Frequent Receivers, centralizing the ecosystem.
Less frequent retraining improves forecast accuracy in retail demand forecasting.
problem Balancing forecast accuracy and computational efficiency in global models.
method Analysis of ten machine learning and deep learning models across two large retail datasets with various retraining scenarios.
result Less frequent retraining strategies maintain forecast accuracy while reducing computational costs.
Two-step model estimates DLMO using both daily and frequent data.
problem Expensive and time-consuming DLMO measurement.
method Two-step framework combining daily and frequent data.
result Two-step model with two time-scale features has lower errors.
We construct a general stochastic process and prove weak convergence results. It is scaled in space and through the parameters of its distribution. We show that our simplified scaling is equivalent to time scaling used frequently. The process is constructed as an integral with respect to a Poisson random measure which …
Paper improves tensor approximation for streaming data.
problem Challenges in finding accurate low-tubal-rank tensor approximations in streaming settings.
method Extends Frequent Directions for efficient low-tubal-rank tensor approximation.
result The new algorithm achieves arbitrarily small approximation error with linear sketch size growth.
System states that are anomalous from the perspective of a domain expert occur frequently in some anomaly detection problems. The performance of commonly used unsupervised anomaly detection methods may suffer in that setting, because they use frequency as a proxy for anomaly. We propose a novel concept for anomaly dete…
This paper investigates how dataset properties affect GAN training outcomes.
problem GANs often fail to reach equilibrium due to instability or mode collapse.
method Experiments to identify patterns in dataset properties and their effects on GAN training.
result Patterns in dataset properties influence GAN training dynamics and outcomes.
IGML learns discriminative metrics for graph classification.
problem Defining an appropriate distance metric for graph data.
method Supervised distance metric learning in a subgraph-based feature space with sparsity-inducing penalty.
result IGML identifies important subgraphs for graph classification.
Sketchy reduces memory and compute requirements for adaptive regularization in deep learning.
problem Prohibitive memory and running time for adaptive regularization methods in deep learning.
method Low-rank sketching approach using Frequent Directions (FD) to reduce memory and compute requirements.
result Efficient interpolation between resource requirements and degradation in regret guarantees with rank k. In many applications that involve processing high-dimensional data, it is important to identify a small set of entities that account for a significant fraction of detections. Rather than formalize this as a clustering problem, in which all detections must be grouped into hard or soft categories, we formalize it as an i…
Study shows frequent 'stock' mentions on Twitter correlate with stock market declines.
problem Understanding how social media impacts financial markets during pandemics.
method Collected over 26 million tweets about COVID-19, analyzed using natural language processing.
result More frequent use of 'stock' in tweets predicts substantial declines in stock market indices.
Deterministic algorithm estimates ridge regression with minimal space.
problem Estimating ridge regression solutions efficiently.
method Deterministic space-efficient algorithm using Frequent Directions.
result First o(d2) space deterministic streaming algorithm with guaranteed error. Most sales applications are characterized by competition and limited demand information. For successful pricing strategies, frequent price adjustments as well as anticipation of market dynamics are crucial. Both effects are challenging as competitive markets are complex and computations of optimized pricing adjustments…
ETM identifies field-specific keywords in text classification.
problem Unsupervised text classification with field-specific keywords.
method Weighted Lasso penalty and pairwise Kullback-Leibler divergence penalty for topic separation.
result ETM improves topic coherence by 22% and 10% compared to LDA.
LDAO addresses imbalanced regression by learning local distribution structures.
problem Imbalanced regression with sparse target regions difficult for models.
method LDAO learns local distribution structures, models and samples from each, then merges.
result LDAO outperforms state-of-the-art methods on 45 imbalanced datasets.
Random Intersection Chains selects important interactions from categorical features.
problem Heavy computational burden in considering all interactions for categorical features.
method Randomly generates chains of intersections, estimates and selects frequent patterns.
result Selected patterns are the most frequent in the data set.