Machine Learning has been steadily gaining traction for its use in Anomaly-based Network Intrusion Detection Systems (A-NIDS). Research into this domain is frequently performed using the KDD~CUP~99 dataset as a benchmark. Several studies question its usability while constructing a contemporary NIDS, due to the skewed r…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,051 papers · 148 categories
Trend · papers per month
48 results for “KDD competition”
Winning solution for predicting player churn in a video game.
problem Predicting when players will stop playing a game.
method Long Short-Term Memory (LSTM) approach and conditional inference survival ensemble model.
result Models accurately predicted player churn and were robust to changing business models.
This paper uses β-VAE for unsupervised anomaly detection in NSL-KDD.
problem Unsupervised anomaly detection in network traffic.
method β-VAE with latent space structure and reconstruction error.
result Latent space exploitation is more effective for classification tasks.
Project compares KDDCup99 and NSL-KDD datasets using ML classifiers.
problem Quality and bias in IoT network intrusion detection datasets.
method Evaluation of various ML classifiers on both datasets.
result NSL-KDD dataset is of higher quality with lower bias.
Researchers tackle malaria control as a reinforcement learning problem with limited data.
problem Learning optimal malaria control policies with scarce data.
method Applied Genetic Algorithm, Bayesian Optimization, and Q-learning with sequence breaking.
result Q-Learning with sequence breaking achieved 7th place in the KDD Cup challenge.
Proposes an autonomous IDS using multiple learning techniques.
problem Autonomous and zero-day attack detection in IDS.
method Parallel use of GRU, CNN, and random forest; majority vote and OR logic combination.
result Achieved high accuracy (87.28%) on NSL-KDD's 'KDDTest+' dataset.
Chromatic Learning reduces feature dimensions for sparse datasets.
problem Sparse, high-dimensional data challenges traditional learning methods.
method Graph coloring over co-occurrence graph to create dense feature representation.
result Compresses sparse datasets significantly while maintaining model accuracy.
AutoIDS uses auto-encoders to detect network intrusions efficiently.
problem Detecting network intrusions with high accuracy and efficiency.
method Semi-supervised machine learning with cascading encoder-decoder neural networks.
result AutoIDS achieves 90.17% accuracy on NSL-KDD dataset, superior to other methods.
A novel resampling technique addresses class imbalance in imbalanced datasets.
problem Class imbalance in real-world datasets, especially in rare event detection.
method Developed two oversampling algorithms: G1Nos 1-Nearest Neighbour.
result Our oversampling algorithms outperform state-of-the-art methods in all metrics.
We describe our language-independent unsupervised word sense induction system. This system only uses topic features to cluster different word senses in their global context topic space. Using unlabeled data, this system trains a latent Dirichlet allocation (LDA) topic model then uses it to infer the topics distribution…
Markowitz Meets Bellman: Knowledge-distilled Reinforcement Learning for Portfolio Managementq-fin.CP
Hybrid approach combines Markowitz's theory with reinforcement learning for optimal portfolio management.
problem Optimizing investment portfolios while balancing returns and risks.
method Knowledge distillation for training reinforcement learning agents.
result Achieves highest yield and Sharpe ratio of 2.03, ensuring top profitability with low risk.
We combine fine-grained spatially referenced census data with the vote outcomes from the 2016 US presidential election. Using this dataset, we perform ecological inference using distribution regression (Flaxman et al, KDD 2015) with a multinomial-logit regression so as to model the vote outcome Trump, Clinton, Other / …
We present a system that enables rapid model experimentation for tera-scale machine learning with trillions of non-zero features, billions of training examples, and millions of parameters. Our contribution to the literature is a new method (SA L-BFGS) for changing batch L-BFGS to perform in near real-time by using stat…
Use Dimensionality Reduction and SVM Methods to Increase the Penetration Rate of Computer Networkscs.CR
This research improves network security by detecting attacks using SVM and dimension reduction.
problem Detecting and preventing attacks on computer networks.
method Combining SVM and SVD for anomaly detection.
result The proposed method outperforms other methods in detecting network attacks.
SynGAN generates synthetic network attacks to improve NIDS effectiveness.
problem Low effectiveness of NIDS due to high false positives and lack of continuous testing.
method SynGAN uses GANs to generate adversarial network attacks based on real attack traffic.
result SynGAN-generated attacks improve NIDS detection rates compared to real attacks.
In this paper, we study the Nystr{ö}m type subsampling for large scale kernel methods to reduce the computational complexities of big data. We discuss the multi-penalty regularization scheme based on Nystr{ö}m type subsampling which is motivated from well-studied manifold regularization schemes. We develop a theoretica…
Two novel dataset optimization strategies improve malware traffic detection accuracy.
problem Redundant and irrelevant information in network traffic datasets increases computational cost and noise.
method Feature selection and dimensional reduction techniques using mutual information and autoencoders.
result Optimized dataset leads to improved accuracy of Multi Layer Perceptron for malware detection.
Measuring the impact of scientific articles is important for evaluating the research output of individual scientists, academic institutions and journals. While citations are raw data for constructing impact measures, there exist biases and potential issues if factors affecting citation patterns are not properly account…
A prediscretisation of numerical attributes which is required by some rule learning algorithms is a source of inefficiencies. This paper describes new rule tuning steps that aim to recover lost information in the discretisation and new pruning techniques that may further reduce the size of rule models and improve their…
A new IDS method uses GANs to detect anomalies in network traffic.
problem Detecting network anomalies with limited labeled data.
method End-to-end adversarial learning using GANs for semi-supervised learning.
result The method outperforms state-of-the-art IDS approaches.
This paper studies adversarial examples in NIDS, revealing their vulnerability.
problem Vulnerability of machine learning-based NIDS to adversarial examples.
method Used evolutionary computation and deep learning to generate adversarial examples.
result Adversarial examples cause high misclassification rates in various machine learning models.
Anomaly detection is one of the frequent and important subroutines deployed in large-scale data processing systems. Even being a well-studied topic, existing techniques for unsupervised anomaly detection require storing significant amounts of data, which is prohibitive from memory and latency perspective. In the big-da…
BGRL learns graph representations without costly negative examples.
problem Efficient representation learning on large graphs without labels.
method Bootstrapped Graph Latents (BGRL) learns by predicting augmentations.
result BGRL achieves state-of-the-art performance with 2-10x memory savings.
Proposes CoPO, a new policy optimization method for competitive games.
problem Designing efficient optimization methods for competitive Markov decision processes.
method Competitive policy optimization (CoPO) approach that exploits game-theoretic nature of competitive games.
result Stable optimization, convergence to sophisticated strategies, and higher scores compared to baseline methods.
Kaggle competitions offer valuable insights for business forecasting.
problem Lack of attention to Kaggle competitions in academic forecasting studies.
method Review of results from six Kaggle competitions featuring real-life business forecasting tasks.
result Global ensemble models outperform local single models in Kaggle competitions.
Calibrated models can lead to miscalibrated aggregations in strategic interactions.
problem Miscalibration in aggregated predictions from multiple calibrated models.
method Analysis of strategic interactions between calibrated predictors, proving conditions for miscalibration and comparing VCG and Brier-score aggregation methods.
result VCG aggregation method outperforms Brier-score in strategic settings, providing robustness and comparable accuracy.
The origin of economic crises is a key problem for economics. We present a model of long-run competitive markets to show that the multiplicity of behaviors in an economic system, over a long time scale, emerge as statistical regularities (perfectly competitive markets obey Bose-Einstein statistics and purely monopolist…
New framework promotes reproducible, domain-agnostic reinforcement learning algorithms.
problem Domain-specific, compute-resource-maximizing, and non-reproducible participant solutions in reinforcement learning competitions.
method Submission retraining, domain randomization, desemantization through domain obfuscation, and compute/environment-sample budget limitation.
result Participant submissions are reproducible, non-specific to the competition environment, and sample/resource efficient.
The real estate is a pillar industry of China's national economy. Due to changes in policy and market conditions, the real estate companies are facing greater pressures to survive in a competitive environment. They must improve their financial competitiveness. Based on the conceptual framework of financial competitiven…
First ABAW 2020 Competition analyzes affective behavior tasks.
problem Automatic analysis of valence-arousal, basic expressions, and action units in real-world scenarios.
method Provided Aff-Wild2 database, described Challenges, evaluation metrics, and top-performing systems.
result Demonstrated the feasibility of automatic affective behavior analysis in real-world settings.
Study examines machine learning competitions' impact on AI development.
problem Fostering innovation and skill development in AI.
method Analysis of major competition platforms, workflows, and participant demographics.
result MLCs promote collaboration, reproducibility, and continuous innovation in AI.
Bayesian rating system for large competitions improves prediction and efficiency.
problem Rating systems for large, competitive events like online programming contests.
method Developed a Bayesian rating system for many participants, proving robustness and runtime.
result The system outperforms existing systems in accuracy and computation speed.
Retrospective Analysis of the 2019 MineRL Competition on Sample Efficient Reinforcement Learningcs.LG
MineRL Competition reduced reinforcement learning sample needs.
problem Sample inefficiency in reinforcement learning.
method Human demonstrations and imitation learning integrated into reinforcement learning algorithms.
result Top solutions used deep reinforcement learning and imitation learning.
This paper evaluates financial competitiveness of Indian real estate companies using entropy method.
problem Improving financial competitiveness of Indian real estate companies in a competitive market.
method Financial competitiveness evaluation index system using key financial ratios and a scoring system.
result Companies with high scores have strong profitability and operational capacity, while those with lower scores struggle with solvency and working capital.
Paper tackles online optimization with memory and competitive control.
problem Minimizing hitting and switching costs in online optimization problems.
method Optimistic Regularized Online Balanced Descent algorithm.
result Achieves a constant, dimension-free competitive ratio.
How to Host a Data Competition: Statistical Advice for Design and Analysis of a Data Competitionstat.AP
This paper offers advice for designing and analyzing data competitions.
problem Overfitting in data competitions can limit the learning outcomes.
method Strategic design of data sets and post-competition analysis using GLMs.
result Richer and more informative summaries enhance the interpretation of results.
In this paper, the optimal pricing strategy in Avellande-Stoikov's for a monopolistic dealer is extended to a general situation where multiple dealers are present in a competitive market. The dealers' trading intensities, their optimal bid and ask prices and therefore their spreads are derived when the dealers are info…
Study competitive agents' optimal consumption and investment strategies with relative performance criteria.
problem Optimizing consumption and investment strategies for multiple agents with relative performance considerations.
method Derived a closed-form solution for an -player game and mean field game, analyzing the impact of risk tolerance and competitiveness parameters.
result Unique equilibria found, showing nonlinear and non-monotone dependence on agents' risk tolerance and competitiveness parameters.
Model predicts growth competition on curved surfaces.
problem Growth dynamics of two subsets on Riemannian manifolds.
method Modeling growth rates on spherically symmetric Riemannian manifolds.
result Conditions for bounded or unbounded growth on different manifolds.
Market competition depends on computational complexity, P != NP makes it impossible.
problem Competitive market outcomes require computational intractability.
method Analyzes the computational hardness of collusion detection in markets.
result If P != NP, collusion detection is computationally infeasible, making collusion unstable.
The M5 competition tackles overdispersed retail sales forecasting with GAMLSS.
problem Overdispersed and zero-inflated retail sales data.
method Distributional forecasting using GAMLSS framework.
result GAMLSS provides better probabilistic forecasting for count data.
DoPa detects various physical adversarial attacks on CNNs.
problem Vulnerability of CNNs to physical adversarial attacks.
method Interprets CNN's vulnerability, adds self-verification stage.
result Achieves 90% success rate for image attacks and 92% for audio attacks.
Model predicts competition between similar products in sales.
problem Predicting cannibalization between similar products in sales.
method Developed a neural network model that computes a 'competitiveness' function based on product features.
result The model outperforms traditional methods in predicting market share.
Improved forecasting in daily time series competition using a correlator method.
problem Forecasting daily time series with data leakage issues.
method Ensemble of five statistical forecasting methods and a correlator method.
result The correlator method was responsible for most of the gains over naive forecasting.
The NIPS 2018 Adversarial Vision Challenge is a competition to facilitate measurable progress towards robust machine vision models and more generally applicable adversarial attacks. This document is an updated version of our competition proposal that was accepted in the competition track of 32nd Conference on Neural In…
We show that for the problem of testing if a matrix has rank at most , or requires changing an -fraction of entries to have rank at most , there is a non-adaptive query algorithm making queries. Our algorithm works for any field . This improves upon the previous…
Politicians world-wide frequently promise a better life for their citizens. We find that the probability that a country will increase its {\it per capita} GDP ({\it gdp}) rank within a decade follows an exponential distribution with decay constant . We use the Corruption Perceptions Index (CPI) and the Global …
Bose-Einstein Condensation in Competitive Processescond-mat.stat-mech
We introduce an irreversible discrete multiplicative process that undergoes Bose-Einstein condensation as a generic model of competition. New players with different abilities successively join the game and compete for limited resources. A player's future gain is proportional to its ability and its current gain. The the…