MLSys aims to bridge ML and systems research.
problem Designing ML systems for real-world deployment is challenging.
method Foster a new conference and research community.
result MLSys conference focuses on intersection of systems and ML.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,982 papers · 148 categories
Trend · papers per month
48 results for “systems and ML”
MLPerf benchmarks ML inference systems across diverse hardware.
problem Challenges in evaluating ML inference systems due to variety and complexity.
method MLPerf prescribes a set of rules and best practices for comparability.
result First call for submissions yielded over 600 reproducible measurements.
This paper tackles hidden technical debts in fair ML systems for Fintech.
problem Building fair machine learning systems in financial services.
method Examining key stages of ML system development and deployment.
result Technical debts exist in deploying fair ML systems in Fintech.
Industry lacks tools to secure ML systems, study finds.
problem Insufficient security tools for ML systems in industry.
method Interviews with 28 organizations to identify gaps.
result Need revised Security Development Lifecycle for ML.
Paper proposes using creative ML for game design.
problem Lack of creative ML in game design.
method Leverage existing creative ML systems for game content creation.
result Illustrates how creative ML can inform new game design systems.
The rise of Big Data has led to new demands for Machine Learning (ML) systems to learn complex models with millions to billions of parameters, that promise adequate capacity to digest massive datasets and offer powerful predictive analytics thereupon. In order to run ML algorithms at such scales, on a distributed clust…
Community-based system dynamics improves ML fairness by involving excluded stakeholders.
problem Bias in ML system development during problem formulation.
method Community-based system dynamics (CBSD) for stakeholder participation.
result CBSD facilitates deeper problem understanding and bias mitigation.
Paper bridges AI/ML and causal modeling to reduce bias.
problem Difficulty in combining methods from different assumptions.
method Integrates system dynamics and structural equation modeling.
result Unified mathematical framework for AI/ML and causal modeling.
Study uses ML techniques to reveal quantum-like features in classical systems.
problem Understanding the intuition behind unsupervised ML in physical systems.
method Three ML techniques applied to adjacency matrices of 2D particulate systems.
result ML techniques reveal quantum-like features in classical systems.
Paper proposes counterfactual explanations for ML on multivariate time series data.
problem Lack of user trust and difficulty in debugging ML frameworks using multivariate time series data.
method Proposes a novel explainability technique for providing counterfactual explanations.
result Outperforms state-of-the-art explainability methods in metrics like faithfulness and robustness.
Fair ML systems can be safe ML systems by considering uncertainty.
problem Safety of data-driven decision systems is often neglected.
method Viewing ML systems as socio-technical, uncertainty-aware modeling.
result Fair models should be uncertainty-aware, e.g. through distributional regression.
Develops fair feature importance scores for tree-based models to interpret fairness.
problem Ensuring fairness in machine learning models, especially tree-based ones.
method Inspired by decision trees, proposes a novel fair feature importance score based on mean decrease in group bias.
result Valid interpretations of fairness for tree-based ensembles and surrogates of other ML systems.
A framework for developing ML systems from ML primitives.
problem Complexity and fragmentation in ML systems development.
method Unified API, ML primitives, AutoML strategies, and pipelines.
result General-purpose, multi-task AutoML system for various data modalities and problem types.
Many of today's machine learning (ML) systems are not built from scratch, but are compositions of an array of {\em modular learning components} (MLCs). The increasing use of MLCs significantly simplifies the ML system development cycles. However, as most MLCs are contributed and maintained by third parties, their lack …
Improves ML efficiency for vast, rapidly growing data.
problem Low latency and cost in ML with distributed, growing data.
method Designs ML systems exploiting ML characteristics, data structures, and data distribution.
result Improves ML latency and cost by 1-2 orders of magnitude.
Interactive tool helps understand ML model performance.
problem Understanding ML model performance across various inputs.
method Open-source What-If Tool for interactive probing and analysis.
result Allows testing performance in hypothetical situations and measuring fairness.
AI methods are energy-intensive, but efficiency alone isn't enough for sustainability.
problem AI methods are energy-intensive and contribute to climate change.
method Critically examines the limitations of efficiency in improving environmental sustainability of AI.
result Efficiency alone is insufficient to address the environmental impacts of AI.
A novel ML verification technique using manifold learning.
problem Ensuring trust in machine learning systems.
method Variational autoencoder for extracting a low-dimensional manifold from high-dimensional training data.
result The manifold provides diverse test data, fault-revealing test cases, and independent runtime trust assessment.
Fueled by massive amounts of data, models produced by machine-learning (ML) algorithms, especially deep neural networks, are being used in diverse domains where trustworthiness is a concern, including automotive systems, finance, health care, natural language processing, and malware detection. Of particular concern is …
What is a systematic way to efficiently apply a wide spectrum of advanced ML programs to industrial scale problems, using Big Models (up to 100s of billions of parameters) on Big Data (up to terabytes or petabytes)? Modern parallelization strategies employ fine-grained operations and scheduling beyond the classic bulk-…
In distributed ML applications, shared parameters are usually replicated among computing nodes to minimize network overhead. Therefore, proper consistency model must be carefully chosen to ensure algorithm's correctness and provide high throughput. Existing consistency models used in general-purpose databases and moder…
Paper proposes a dataset quality process for ML systems.
problem Inadequate standards for ML datasets in safety-critical systems.
method Proposes a dataset specification and verification process.
result Demonstrates the process on a railway signal recognition system.
Recently software development companies started to embrace Machine Learning (ML) techniques for introducing a series of advanced functionality in their products such as personalisation of the user experience, improved search, content recommendation and automation. The technical challenges for tackling these problems ar…
Over the last several years, the use of machine learning (ML) in neuroscience has been rapidly increasing. Here, we review ML's contributions, both realized and potential, across several areas of systems neuroscience. We describe four primary roles of ML within neuroscience: 1) creating solutions to engineering problem…
System detects overfitting in ML apps, improving quality and efficiency.
problem Overfitting in ML applications during continuous development.
method ease. ml/meter system for automated overfitting detection and measurement.
result Probabilistic overfitting signals for developers to take actions.
Survey examines challenges of ML in avionic systems certification.
problem Challenges in current certification standards for ML in avionic systems.
method Literature review focusing on robustness and explainability of ML results.
result Current certification standards do not support ML in avionic systems.
Recent efforts in Machine Learning (ML) interpretability have focused on creating methods for explaining black-box ML models. However, these methods rely on the assumption that simple approximations, such as linear models or decision-trees, are inherently human-interpretable, which has not been empirically tested. Addi…
Paper robustifies reinforcement learning agents against action space perturbations.
problem Vulnerability of reinforcement learning agents to action space perturbations (e.g. actuator attacks).
method Adversarial training to robustify DRL agents against perturbations.
result DRL agents can be robustified against action space perturbations through adversarial training.
When machine learning systems fail because of adversarial manipulation, how should society expect the law to respond? Through scenarios grounded in adversarial ML literature, we explore how some aspects of computer crime, copyright, and tort law interface with perturbation, poisoning, model stealing and model inversion…
MLDemon monitors ML systems post-deployment, improving reliability with real-time performance estimates and expert labels.
problem Ensuring reliability of machine learning systems post-deployment, especially when user inputs differ from training data.
method Integrates unlabeled and on-demand labeled data to monitor ML model performance in real-time, deciding when to acquire expert labels.
result Outperforms existing approaches in temporal datasets with diverse distribution drifts, providing theoretical optimality for distribution drifts.
Combining data assimilation and machine learning to infer unresolved scale parametrisationphysics.comp-ph
Combines ML and DA to infer unresolved scale parametrisation from noisy data.
problem Training ML-based parametrisations from realistic, noisy and sparse observations.
method Two-step process: DA for state estimation, ML for model error prediction.
result Hybrid model produces better forecasts and attractor representation.
Framework classifies machine learning failures into intentional and unintentional.
problem Understanding and preventing failures in machine learning systems.
method Developed a taxonomy of machine learning failure modes.
result Stakeholders found the framework useful for discussing machine learning failures.
Survey examines data quality challenges in edge ML.
problem Data quality issues in edge ML due to limited resources and decentralized data.
method Provides a comprehensive survey of existing literature on data quality in edge ML.
result No comprehensive survey of data quality in edge ML exists.
MLPerf benchmarks ML training to drive performance improvements.
problem Unique challenges in ML training benchmarks.
method Developed MLPerf to overcome ML training's specific challenges.
result Quantitatively evaluated MLPerf's effectiveness.
Survey of methods for ML assurance across its lifecycle stages.
problem Ensuring safety of ML in safety-critical applications.
method Comprehensive survey of methods at each ML lifecycle stage.
result Identification of open challenges in ML assurance.
The study examines backward compatibility issues in ML systems, especially with noisy data.
problem Backward compatibility challenges in ML systems, especially with noisy data.
method Empirical analysis of ML systems across different architectures and datasets, focusing on data shifts and noise.
result Backward compatibility issues arise even without data shift due to optimization stochasticity and training on large-scale noisy datasets can significantly decrease compatibility.
Sage platform protects ML models trained on sensitive data from leakage.
problem Protecting sensitive data in machine learning models exposed to untrusted domains.
method Develops block composition for privacy accounting and privacy-adaptive training to manage privacy budget and utility tradeoff.
result Enables continuous training of models on sensitive data streams while maintaining global DP guarantees.
XSP profiles ML models across hardware and software stacks.
problem Challenges in profiling ML model performance across different layers of the stack.
method XSP uses distributed tracing to aggregate data from various sources and introduces a leveled, iterative measurement approach.
result XSP provides insights into ML model execution not easily discernible otherwise.
Machine Learning (ML) has been successfully applied to a wide range of domains and applications. One of the techniques behind most of these successful applications is Ensemble Learning (EL), the field of ML that gave birth to methods such as Random Forests or Boosting. The complexity of applying these techniques togeth…
A new ML-based filter improves data assimilation for nonlinear systems.
problem Improving data assimilation for nonlinear systems using ensemble methods.
method Developed a machine learning-based conditional mean filter (ML-EnCMF) integrating ANN and linear functions.
result ML-EnCMF outperforms EnKF and likelihood-based EnCMF in nonlinear systems.
Defends against ML inference attacks using adversarial examples.
problem Automated inference attacks using ML classifiers pose privacy and security threats.
method Turns ML classifier vulnerabilities into defenses by adding adversarial noise to public data.
result Adversarial examples can mislead ML classifiers and protect private data.
The DoD needs a robust process to evaluate AI/ML model performance and robustness.
problem AI/ML models are brittle and nonrobust, posing risks in national security.
method Reviews AI/ML development process and best practices for evaluation.
result Recommendations for DoD evaluators to ensure robust AI/ML capabilities.
Bayesian methods improve adversarial machine learning robustness.
problem Improving machine learning systems' security against adversarial manipulation.
method Integrating Bayesian perspectives to relax unrealistic assumptions in game-theoretic models.
result Bayesian approach provides more robust inferences about opponent's beliefs and interests.
Integrating Scientific Knowledge with Machine Learning for Engineering and Environmental Systemsphysics.comp-ph
Paper integrates ML with physics models for engineering and environmental challenges.
problem Complex science and engineering problems require new methodologies combining physics-based models and ML.
method Structured overview of integrating physics-based models with ML techniques.
result Taxonomy of existing techniques and potential research gaps identified.
Security for Machine Learning-based Systems: Attacks and Challenges during Training and Inferencecs.LG
The exponential increase in dependencies between the cyber and physical world leads to an enormous amount of data which must be efficiently processed and stored. Therefore, computing paradigms are evolving towards machine learning (ML)-based systems because of their ability to efficiently and accurately process the eno…
New ML fairness measure excludes subjective opinions.
problem Lack of adaptation of ML models to specific applications.
method Formalised new notion of unfairness, quantified and visualised.
result Insights into ML systems' functioning to mitigate unfairness.
Framework improves ML performance by identifying high-quality data.
problem Poor data quality hampers ML performance.
method Intelligent data-centric evaluation framework combining quality measurements and unsupervised learning.
result Framework improves ML system performance in real-world use case.
Proposes a novel Out-of-Bag anomaly detection method for ML systems.
problem Challenges of detecting data anomalies in real-world datasets.
method Model-based approach decomposing unsupervised problem into ensemble models using Out-of-Bag estimates.
result Demonstrates state-of-the-art performance and improved accuracy in ML systems.