This papers presents a deep learning-based framework to predict crowdsourced service availability spatially and temporally. A novel two-stage prediction model is introduced based on historical spatio-temporal traces of mobile crowdsourced services. The prediction model first clusters mobile crowdsourced services into r…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “Mobile Crowdsourcing”
Mobile sensing is an emerging technology that utilizes agent-participatory data for decision making or state estimation, including multimedia applications. This article investigates the structure of mobile sensing schemes and introduces crowdsourcing methods for mobile sensing. Inspired by social network, one can estab…
Mobile crowdsourcing has become easier thanks to the widespread of smartphones capable of seamlessly collecting and pushing the desired data to cloud services. However, the success of mobile crowdsourcing relies on balancing the supply and demand by first accurately forecasting spatially and temporally the supply-deman…
ADRL improves participant selection in MCS systems.
problem Designing a participant selection algorithm for different MCS systems with multiple goals.
method Auxiliary-task based deep reinforcement learning (ADRL) using transformers and pointer networks.
result ADRL outperforms other baselines in various MCS settings.
Heteroscedastic Gaussian processes for uncertainty modeling in large-scale crowdsourced traffic datastat.ML
Accurately modeling traffic speeds is a fundamental part of efficient intelligent transportation systems. Nowadays, with the widespread deployment of GPS-enabled devices, it has become possible to crowdsource the collection of speed information to road users (e.g. through mobile applications or dedicated in-vehicle dev…
Federated learning (FL) rests on the notion of training a global model in a decentralized manner. Under this setting, mobile devices perform computations on their local data before uploading the required updates to improve the global model. However, when the participating clients implement an uncoordinated computation …
Biodiversity monitoring using audio recordings is achievable at a truly global scale via large-scale deployment of inexpensive, unattended recording stations or by large-scale crowdsourcing using recording and species recognition on mobile devices. The ability, however, to reliably identify vocalising animal species is…
Detects adversaries in crowdsourcing to improve accuracy.
problem Adversaries can skew results in crowdsourced classification.
method Analyzes second-order moments of annotator responses to identify and mitigate adversaries.
result The approach can identify and mitigate adversaries in crowdsourcing tasks.
Driven by the tremendous technological advancement of personal devices and the prevalence of wireless mobile network accesses, the world has witnessed an explosion in crowdsourced live streaming. Ensuring a better viewers quality of experience (QoE) is the key to maximize the audiences number and increase streaming pro…
Mosquitoes are a major vector for malaria, causing hundreds of thousands of deaths in the developing world each year. Not only is the prevention of mosquito bites of paramount importance to the reduction of malaria transmission cases, but understanding in more forensic detail the interplay between malaria, mosquito vec…
Bayesian algorithms improve crowdsourcing with label and instance constraints.
problem Efficiently labeling large datasets with additional human annotator information.
method Developed Bayesian algorithms for semi-supervised crowdsourced classification under label and instance constraints.
result Improved performance compared to unsupervised crowdsourcing on various datasets.
There is a rapidly increasing interest in crowdsourcing for data labeling. By crowdsourcing, a large number of labels can be often quickly gathered at low cost. However, the labels provided by the crowdsourcing workers are usually not of high quality. In this paper, we propose a minimax conditional entropy principle to…
Allowing members of the crowd to propose novel microtasks for one another is an effective way to combine the efficiencies of traditional microtask work with the inventiveness and hypothesis generation potential of human workers. However, microtask proposal leads to a growing set of tasks that may overwhelm limited crow…
Integrates multiple datasets to solve open set crowdsourcing problems.
problem Crowdsourcing with unknown label space and unfamiliar tasks.
method Integrates multiple crowdsourced datasets, weights them based on category correlation, and uses open set transfer learning.
result Proves OSCrowd solves open set crowdsourcing problems and outperforms related solutions.
Crowdsourcing utilizes the wisdom of crowds for collective classification via information (e.g., labels of an item) provided by labelers. Current crowdsourcing algorithms are mainly unsupervised methods that are unaware of the quality of crowdsourced data. In this paper, we propose a supervised collective classificatio…
We consider the problem of optimal budget allocation for crowdsourcing problems, allocating users to tasks to maximize our final confidence in the crowdsourced answers. Such an optimized worker assignment method allows us to boost the efficacy of any popular crowdsourcing estimation algorithm. We consider a mutual info…
Bayesian approach improves crowdsourcing predictions.
problem Estimating continuous labels from crowdsource workers.
method Variational Bayesian technique for worker noise models.
result Bayesian approaches significantly outperform non-Bayesian methods.
In this paper, we analyze PAC learnability from labels produced by crowdsourcing. In our setting, unlabeled examples are drawn from a distribution and labels are crowdsourced from workers who operate under classification noise, each with their own noise parameter. We develop an end-to-end crowdsourced PAC learning algo…
Crowdsourced data helps detect incidents faster, balancing accuracy and practicality.
problem Detecting incidents from crowdsourced data is challenging due to noise and uncertainty.
method CROME (Crowdsourced Multi-objective Event Detection) uses CNN and Pareto optimization.
result The approach outperforms existing methods in incident detection and practicality.
Optimizes crowdsourced preference-based subjective evaluation with online learning.
problem Large-scale evaluation of generative media using crowdsourcing due to combinatorial explosion.
method Automatic optimization of pair combination selections and evaluation volumes with online learning.
result Optimizes evaluation by reducing pair combinations and allocating optimal evaluation volumes.
NeuCrowd creates high-quality samples from crowdsourced labels to improve representation learning.
problem Limited and inconsistent crowdsourced labels for representation learning.
method Unified framework that generates high-quality n-tuplet samples and learns a neural sampling network.
result NeuCrowd outperforms state-of-the-art baselines in prediction accuracy and AUC.
Adapts AUM to identify ambiguous tasks in crowdsourced learning, improving generalization.
problem Discerning ambiguous tasks in crowdsourced labels to prevent mislabeling.
method Introduces Weighted Areas Under the Margin (WAUM) to average AUMs weighted by task-specific scores.
result Improves generalization performance by discarding ambiguous tasks.
Learning representation has been proven to be helpful in numerous machine learning tasks. The success of the majority of existing representation learning approaches often requires a large amount of consistent and noise-free labels. However, labels are not accessible in many real-world scenarios and they are usually ann…
Crowdsourced labeling recovers task types with minimal queries.
problem Labeling tasks accurately with minimal queries.
method Worker clustering, skill estimation, weighted majority voting.
result Achieves any targeted recovery accuracy with minimum queries.
Allocation strategies improve the efficiency of crowdsourcing by decreasing the work needed to complete individual tasks accurately. However, these algorithms introduce bias by preferentially allocating workers onto easy tasks, leading to sets of completed tasks that are no longer representative of all tasks. This bias…
Crowdsourcing has become widely used in supervised scenarios where training sets are scarce and difficult to obtain. Most crowdsourcing models in the literature assume labelers can provide answers to full questions. In classification contexts, full questions require a labeler to discern among all possible classes. Unfo…
Unlearn What You Have Learned: Adaptive Crowd Teaching with Exponentially Decayed Memory Learnersstat.ML
With the increasing demand for large amount of labeled data, crowdsourcing has been used in many large-scale data mining applications. However, most existing works in crowdsourcing mainly focus on label inference and incentive design. In this paper, we address a different problem of adaptive crowd teaching, which is a …
We proposed a probabilistic approach to joint modeling of participants' reliability and humans' regularity in crowdsourced affective studies. Reliability measures how likely a subject will respond to a question seriously; and regularity measures how often a human will agree with other seriously-entered responses coming…
A key challenge in crowdsourcing is inferring the ground truth from noisy and unreliable data. To do so, existing approaches rely on collecting redundant information from the crowd, and aggregating it with some probabilistic method. However, oftentimes such methods are computationally inefficient, are restricted to som…
Inferring the correct answers to binary tasks based on multiple noisy answers in an unsupervised manner has emerged as the canonical question for micro-task crowdsourcing or more generally aggregating opinions. In graphon estimation, one is interested in estimating edge intensities or probabilities between nodes using …
Analysis of Minimax Error Rate for Crowdsourcing and Its Application to Worker Clustering Modelstat.ML
While crowdsourcing has become an important means to label data, there is great interest in estimating the ground truth from unreliable labels produced by crowdworkers. The Dawid and Skene (DS) model is one of the most well-known models in the study of crowdsourcing. Despite its practical popularity, theoretical error …
In the last years, crowdsourcing is transforming the way classification training sets are obtained. Instead of relying on a single expert annotator, crowdsourcing shares the labelling effort among a large number of collaborators. For instance, this is being applied to the data acquired by the laureate Laser Interferome…
Traffic speed data imputation is a fundamental challenge for data-driven transport analysis. In recent years, with the ubiquity of GPS-enabled devices and the widespread use of crowdsourcing alternatives for the collection of traffic data, transportation professionals increasingly look to such user-generated data for m…
Paper tackles representation learning with limited labels from crowdsourced data.
problem Limited labeled data and inconsistent crowdsourced labels.
method Grouping-based deep neural network and Bayesian confidence estimator.
result Framework learns effective representations from limited data with crowdsourced labels.
Crowdsourcing platforms provide marketplaces where task requesters can pay to get labels on their data. Such markets have emerged recently as popular venues for collecting annotations that are crucial in training machine learning models in various applications. However, as jobs are tedious and payments are low, errors …
Outlier detection is a crucial part of robust evaluation for crowdsourceable assessment of Quality of Experience (QoE) and has attracted much attention in recent years. In this paper, we propose some simple and fast algorithms for outlier detection and robust QoE evaluation based on the nonconvex optimization principle…
Deep learning predicts RNA degradation from crowdsourced data.
problem Predicting RNA degradation to improve thermostability.
method Crowdsourced machine learning competition on Kaggle.
result 41% of predictions matched experimental data, and models generalized to longer RNA molecules.
OpinionRank uses graph-based ranking to improve unreliable crowdsourced labels.
problem Improving trustworthiness of crowdsourced labels for machine learning.
method Graph-based spectral ranking to integrate unreliable labels.
result OpinionRank outperforms conventional algorithms in reliability and scalability.
Enhanced nearest neighbor improves accuracy in crowdsourced data.
problem Noise in crowdsourced labels degrades classification accuracy.
method Developed two algorithms to estimate worker quality and an enhanced nearest neighbor classifier.
result Proposed methods achieve the same regret as oracle version based on expert data.
Unified deep learning from noisy crowds using BP and MF.
problem Inference and learning from noisy crowdsourced data.
method Neural-powered Bayesian framework with deepMF and deepBP.
result deepBP is more robust against wrong priors and feature overfitting.
The unprecedented demand for large amount of data has catalyzed the trend of combining human insights with machine learning techniques, which facilitate the use of crowdsourcing to enlist label information both effectively and efficiently. The classic work on crowdsourcing mainly focuses on the label inference problem …
Crowdsourcing has become a primary means for label collection in many real-world machine learning applications. A classical method for inferring the true labels from the noisy labels provided by crowdsourcing workers is Dawid-Skene estimator. In this paper, we prove convergence rates of a projected EM algorithm for the…
Proposes a new method for deep ensembles that improves accuracy and calibration.
problem Improving accuracy and calibration of deep ensembles.
method Estimates confusion matrices of ensemble members and weighs them according to their inferred performance.
result Empirically shows superiority of soft Dawid Skene over ensemble averaging.
Integrating Crowdsourcing and Active Learning for Classification of Work-Life Events from Tweetscs.CL
Crowdsourcing and active learning reduce manual annotation in social media event classification.
problem Manual annotation is time-consuming and resource-intensive in social media event classification.
method Crowdsourcing pipeline combined with active learning strategies.
result Active learning strategies help reduce the number of tweets needed for classification.
In many machine learning applications, crowdsourcing has become the primary means for label collection. In this paper, we study the optimal error rate for aggregating labels provided by a set of non-expert workers. Under the classic Dawid-Skene model, we establish matching upper and lower bounds with an exact exponent …
The paper tackles optimal level set estimation in crowdsourcing and tournaments.
problem Deciphering small entries in a partially observed matrix of expert correctness.
method Constructs an efficient polynomial-time algorithm for recovering level sets up to a precision.
result The algorithm is minimax optimal for the classification problem, contrasting with existing literature.
Method teaches students without teachers, estimating true labels from crowdsourcing.
problem Teaching without access to true labels.
method Apply crowdsourcing techniques to estimate true labels and student models for iterative teaching.
result Teaching performance is particularly effective for low-level students.
Paper recovers top-two answers and confusion probability in multi-choice crowdsourcing.
problem Recovering top-two answers and confusion probability in multi-choice crowdsourcing tasks.
method Proposes a two-stage inference algorithm based on a model quantifying task difficulty and worker reliability.
result Achieves minimax optimal convergence rate and outperforms other algorithms in synthetic and real data experiments.