This study designs a financial risk control platform using big data and machine learning.
problem Traditional risk management models are inadequate for modern financial complexities.
method Big data mining, real-time streaming data processing, statistical analysis, and precise customer behavior mining.
result The platform effectively identifies and responds to potential risks in real-time.
Framework optimizes portfolios using big data from financial markets.
problem Optimizing investment decisions with structured and unstructured financial data.
method 5-stage methodology including DEA, text mining, clustering, ranking, and heuristics for portfolio optimization.
result Helps investors select, weight, and manage assets for informed investment decisions.
Motivation: In this paper we present the latest release of EBIC, a next-generation biclustering algorithm for mining genetic data. The major contribution of this paper is adding support for big data, making it possible to efficiently run large genomic data mining analyses. Additional enhancements include integration wi…
Nonnegative matrix factorization (NMF) is a powerful tool for data mining. However, the emergence of `big data' has severely challenged our ability to compute this fundamental decomposition using deterministic algorithms. This paper presents a randomized hierarchical alternating least squares (HALS) algorithm to comput…
Paper proposes a predictive maintenance system for solar plants using big data.
problem Fault prediction in photovoltaic plants to reduce downtime and maintenance costs.
method Data-driven approach with unsupervised clustering and Pattern Recognition Neural Network.
result Effective prediction of both generic and specific faults, up to 7 days in advance.
In an effort to overcome the data deluge in computational biology and bioinformatics and to facilitate bioinformatics research in the era of big data, we identify some of the most influential algorithms that have been widely used in the bioinformatics community. These top data mining and machine learning algorithms cov…
This paper surveys smart buildings using machine learning and big data.
problem Improving comfort and efficiency in smart buildings through data analysis.
method Survey of machine learning and big data techniques for smart buildings.
result Machine learning and big data are crucial for smart building services.
Deep learning applied to biological data mining.
problem Mining complex biological data from diverse sources.
method Artificial neural networks, deep learning architectures.
result Deep learning techniques improve pattern recognition in biological data.
The paper uses GIS data to predict urban sprawl.
problem Overgrowth and expansion of low-density areas with car dependency and segregation.
method Data mining algorithms (Apriori, J4.8) adapted for geospatial analysis using ArcGIS.
result Prototype spatial decision support system (SDSS) predicts urban sprawl and estimates impact variables.
When can reliable inference be drawn in the "Big Data" context? This paper presents a framework for answering this fundamental question in the context of correlation mining, with implications for general large scale inference. In large scale data applications like genomics, connectomics, and eco-informatics the dataset…
Big data comes in various ways, types, shapes, forms and sizes. Indeed, almost all areas of science, technology, medicine, public health, economics, business, linguistics and social science are bombarded by ever increasing flows of data begging to analyzed efficiently and effectively. In this paper, we propose a rough …
New clustering algorithm for mixed data improves applicability and efficiency.
problem Clustering large, mixed data with improved accuracy and efficiency.
method Developed a new clustering algorithm using peak-finding technique, reducing computational complexity.
result Algorithm detects outliers, clusters of lower density, and determines correct number of clusters.
System recommends disease treatments based on big data and cloud computing.
problem Inaccurate disease classification and treatment recommendations due to complex symptoms and multi-pathogenesis.
method DPCA for disease-symptom clustering, Apriori for D-D and D-T rules, parallel Apache Spark implementation.
result Effective disease-symptom clustering and accurate treatment recommendations for inexperienced doctors.
New challenges in causal inference with big data.
problem Extending causal inference to incrementally available observational data.
method Formal definition of continual treatment effect estimation, solutions to challenges.
result New methods for handling challenges in causal inference with observational data.
K-Metamodes clusters security data without converting categorical attributes.
problem Clustering heterogeneous security data sets with categorical and numerical attributes.
method Frequency-based distance function for ensemble-based k-modes clustering, adapted feature discretisation.
result Higher effectiveness compared to previous methods on public security data sets.
Interactive model analysis, the process of understanding, diagnosing, and refining a machine learning model with the help of interactive visualization, is very important for users to efficiently solve real-world artificial intelligence and data mining problems. Dramatic advances in big data analytics has led to a wide …
Tensor completion is a problem of filling the missing or unobserved entries of partially observed tensors. Due to the multidimensional character of tensors in describing complex datasets, tensor completion algorithms and their applications have received wide attention and achievement in areas like data mining, computer…
Distributed, online data mining systems have emerged as a result of applications requiring analysis of large amounts of correlated and high-dimensional data produced by multiple distributed data sources. We propose a distributed online data classification framework where data is gathered by distributed data sources and…
New algorithm detects and handles concept drift in data streams.
problem Handling concept drift in data streams for timely predictions.
method Hybrid Forest algorithm combining Hoeffding Trees and fast startup.
result The algorithm outperforms other methods in classification and regression tasks.
Data preprocessing improves data quality for robust data mining.
problem Noisy and incomplete data hinders data mining models.
method Overview of data cleaning, transformation, and preprocessing methods.
result Preprocessing significantly affects data mining model performance.
DataLearner simplifies data mining on Android devices.
problem Lack of general-purpose data-mining tools for mobile devices.
method Augments Weka engine with Charles Sturt University algorithms, providing 40 mining algorithms.
result Delivers classification accuracy similar to PCs/laptops with acceptable speed and battery life.
Agricultural research has been profited by technical advances such as automation, data mining. Today, data mining is used in a vast areas and many off-the-shelf data mining system products and domain specific data mining application soft wares are available, but data mining in agricultural soil datasets is a relatively…
Non-negative matrix factorization (NMF) is the problem of determining two non-negative low rank factors W and H, for the given input matrix A, such that A≈WH. NMF is a useful tool for many applications in different domains such as topic modeling in text mining, background separation in video analysis, …
Paper proposes a distributed algorithm for multi-label feature selection.
problem Maximizing diversity and quality in non-redundant feature selection.
method Greedy algorithm for distributed optimization of submodular plus diversity functions.
result Achieves constant factor approximation of optimal solution in big data settings.
The problem of frequent pattern mining has been studied quite extensively for various types of data, including sets, sequences, and graphs. Somewhat surprisingly, another important type of data, namely rank data, has received very little attention in data mining so far. In this paper, we therefore addresses the problem…
iGecco+ integrates multi-view data for better clustering.
problem Discovering common group structure in mixed multi-view data.
method Integrative Generalized Convex Clustering Optimization (iGecco) with adaptive feature selection.
result iGecco+ achieves superior clustering performance on high-dimensional mixed multi-view data.
This paper categorizes and analyzes existing outlying aspect mining methods.
problem Finding unique features in data objects that differ from others.
method Grouping and analyzing existing outlying aspect mining approaches in three categories.
result Comparison of strengths, weaknesses, and time complexities of different techniques.
A distributed version of ReliefF improves scalability for large datasets.
problem Scalability of traditional FS algorithms on large datasets.
method Distributed implementation of ReliefF on Spark.
result DiReliefF outperforms non-distributed version in processing large datasets.
Tensor decompositions are powerful tools for large data analytics as they jointly model multiple aspects of data into one framework and enable the discovery of the latent structures and higher-order correlations within the data. One of the most widely studied and used decompositions, especially in data mining and machi…
This article guides data scientists on avoiding discrimination in machine learning.
problem Machine learning systems can create or exacerbate societal disparities.
method Provides a taxonomy of practices and measures to mitigate discrimination.
result Data scientists should be intentional about modeling and reducing discriminatory outcomes.
InfDetect detects e-commerce insurance fraud using graph analysis.
problem Detecting fraudulent claims in e-commerce insurance with multiple parties involved.
method Developed a large-scale fraud detection system InfDetect using graph-based approaches.
result InfDetect successfully detected thousands of fraudulent claims and saved money daily.
Data mining enhances a heuristic for the Minimum Latency Problem.
problem Finding optimal solutions for the Minimum Latency Problem efficiently.
method Combining GRASP with data mining to find frequent patterns in high-quality solutions.
result Improved solution quality and reduced computational time compared to existing methods.
A new tool, matrix profile, finds all pair similarities in time series data.
problem Finding all pair similarities in time series data.
method Near universal time series data mining tool called matrix profile.
result Matrix profile solves the all-pairs-similarity-search problem for time series subsequences.
Safe Pattern Pruning reduces pattern explosion in predictive pattern mining.
problem Exponential growth of patterns in structured data.
method Safe Pattern Pruning (SPP) method.
result Effective model building in practical data analysis.
Peer-reviewed research and mined data predict stock returns similarly.
problem Predicting stock returns using research quality.
method Cross-sectional analysis of 29,000 accounting ratios with t-statistics > 2.0.
result Post-sample performance is largely independent of whether the predictor is peer-reviewed or mined.
This paper discusses issues in mining user behavioral rules for context-aware mobile apps.
problem Mining contextual behavioral rules from smartphone data.
method Addressing quality of data, relevancy of contexts, discretization, rule discovery, semantic understanding, and dynamic rule updating.
result Potential solutions for mining user behavioral rules for context-aware mobile apps.
The paper explores how mining costs, rewards, and blockchain security are interconnected.
problem Understanding the interdependencies between mining costs, mining rewards, and blockchain security.
method Theoretical derivation and empirical analysis using daily crypto market data and autoregressive distributed lag approach.
result Cryptocurrency price and mining rewards are intrinsically linked to blockchain security outcomes.
Data mining techniques predict breast cancer types with high accuracy.
problem Early detection of breast cancer to reduce mortality rates.
method Twelve classification algorithms applied to the Breast Cancer Wisconsin dataset.
result High accuracy in predicting malignant and benign breast cancer.
Big data sets must be carefully partitioned into statistically similar data subsets that can be used as representative samples for big data analysis tasks. In this paper, we propose the random sample partition (RSP) data model to represent a big data set as a set of non-overlapping data subsets, called RSP data blocks,…
This paper introduces C-DSL to improve data mining outcomes by considering context.
problem Data collection ambiguities, data imbalance, hidden biases, lack of domain info, and data incompleteness.
method Developed Context-Driven Data Science Lifecycle (C-DSL) to address data quality issues.
result Tangible improvements to data mining outcomes were achieved through C-DSL.
The tremendous growth of positioning technologies and GPS enabled devices has produced huge volumes of tracking data during the recent years. This source of information constitutes a rich input for data analytics processes, either offline (e.g. cluster analysis, hot motion discovery) or online (e.g. short-term forecast…
FedSLIM optimizes compact pattern models across distributed databases without sharing raw data.
problem Privacy-preserving federated descriptive analytics for data silos.
method Federated MDL-based framework using SLIM principle.
result FedSLIM variants preserve high-quality compression structure and recover globally informative patterns.
Spurred by the enthusiasm surrounding the "Big Data" paradigm, the mathematical and algorithmic tools of online optimization have found widespread use in problems where the trade-off between data exploration and exploitation plays a predominant role. This trade-off is of particular importance to several branches and ap…
Automatically mined rules from dependency parsing help neural models learn from less labeled data.
problem Lack of labeled data for aspect and opinion term extraction.
method Automatically mined rules from dependency parsing, applied to auxiliary data, combined with human-annotated data.
result Neural models achieve better performance than state-of-the-art with mined rules and auxiliary data.
Novel process model for metabolomics data analysis.
problem Analyzing complex metabolomics data.
method Data-driven and hypothesis-driven data mining approaches using various techniques.
result Demonstrated applicability and strengths of MeKDDaM model.
DPASF stream preprocesses Big Data streams efficiently.
problem Efficient preprocessing of streaming Big Data.
method Implemented six preprocessing algorithms in Apache Flink.
result Preprocessing improves data accuracy in streaming Big Data.
This article provides the role of big idea statisticians in future of Big Data Science. We describe the `United Statistical Algorithms' framework for comprehensive unification of traditional and novel statistical methods for modeling Small Data and Big Data, especially mixed data (discrete, continuous).
New problems on NNSC fill-ins for Bartnik data in high dimensions.
problem Conditions for (n−1)-dimensional Bartnik data to be NNSC-cobordant. method Formulating three problems related to nonnegative scalar curvature fill-ins.
result Conditions for (n−1)-dimensional Bartnik data to be NNSC-cobordant.