Evaluating surgeon skill has predominantly been a subjective task. Development of objective methods for surgical skill assessment are of increased interest. Recently, with technological advances such as robotic-assisted minimally invasive surgery (RMIS), new opportunities for objective and automated assessment framewor…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “Skill Assessment”
Machine Learning Distinguishes Neurosurgical Skill Levels in a Virtual Reality Tumor Resection Taskcs.LG
Background: Virtual reality simulators and machine learning have the potential to augment understanding, assessment and training of psychomotor performance in neurosurgery residents. Objective: This study outlines the first application of machine learning to distinguish "skilled" and "novice" psychomotor performance du…
Paper learns skill distributions from game outcomes, proving minimax optimality.
problem Learning skill distributions from noisy pairwise game outcomes.
method Proposes a simple algorithm using kernel density estimation and minimax techniques.
result Near-optimal minimax mean squared error scaling for skill density estimation.
Multimodal analysis assesses job interview performance and provides feedback.
problem Assessing candidate performance in interviews for professional roles.
method Multimodal analytical framework using video, audio, and text data.
result The proposed methodology achieved promising results in predicting behavioral cues.
Perinatal stroke (PS) is a serious condition that, if undetected and thus untreated, often leads to life-long disability, in particular Cerebral Palsy (CP). In clinical settings, Prechtl's General Movement Assessment (GMA) can be used to classify infant movements using a Gestalt approach, identifying infants at high ri…
An Exploratory Analysis of the Latent Structure of Process Data via Action Sequence Autoencoderstat.ML
Computer simulations have become a popular tool of assessing complex skills such as problem-solving skills. Log files of computer-based items record the entire human-computer interactive processes for each respondent. The response processes are very diverse, noisy, and of nonstandard formats. Few generic methods have b…
Investigates the use of Information Coefficient as a stock selection model performance measure.
problem The adequacy and effectiveness of Information Coefficient (IC) for evaluating stock selection models is unclear.
method Simulation and simple statistical modeling to examine IC behavior statically and dynamically.
result Proposes two practical procedures for IC-based ongoing performance monitoring of stock selection models.
ReSkill reconciles RL skill creation with policy optimization.
problem RL policies lack reusable strategies across tasks.
method Integrates skill creation into RL loop with three mechanisms.
result Consistently outperforms existing methods, especially on unseen tasks.
Accurate and reliable forecasting of total cloud cover (TCC) is vital for many areas such as astronomy, energy demand and production, or agriculture. Most meteorological centres issue ensemble forecasts of TCC, however, these forecasts are often uncalibrated and exhibit worse forecast skill than ensemble forecasts of o…
Human players in professional team sports achieve high level coordination by dynamically choosing complementary skills and executing primitive actions to perform these skills. As a step toward creating intelligent agents with this capability for fully cooperative multi-agent settings, we propose a two-level hierarchica…
Although compelling assessments have been examined in recent years, more studies are required to yield a better understanding of the several methods where assessment techniques significantly affect student learning process. Most of the educational research in this area does not consider demographics data, differing met…
Employers actively look for talents having not only specific hard skills but also various soft skills. To analyze the soft skill demands on the job market, it is important to be able to detect soft skill phrases from job advertisements automatically. However, a naive matching of soft skill phrases can lead to false pos…
Sharpe ratio (sometimes also referred to as information ratio) is widely used in asset management to compare and benchmark funds and asset managers. It computes the ratio of the (excess) net return over the strategy standard deviation. However, the elements to compute the Sharpe ratio, namely, the expected returns and …
The main objective of exams consists in performing an assessment of students' expertise on a specific subject. Such expertise, also referred to as skill or knowledge level, can then be leveraged in different ways (e.g., to assign a grade to the students, to understand whether a student might need some support, etc.). S…
Elo rating outperforms complex models in skill estimation despite model misspecification.
problem Elo rating's reliability in skill estimation is questioned due to model misspecification.
method Interpreted Elo as online gradient descent and conducted synthetic experiments on non-BT models.
result Elo's superior performance in win rate prediction compared to complex models is explained through its sparsity and ranking effectiveness.
Bayesian nonparametrics enables dynamic skill discovery from expert data.
problem Fixed K for offline skill discovery in reinforcement learning.
method Variational inference, continuous relaxations, Bayesian nonparametrics.
result Nonparametric model with dynamically-changing number of options.
We present a novel solution to the problem of simulation-to-real transfer, which builds on recent advances in robot skill decomposition. Rather than focusing on minimizing the simulation-reality gap, we learn a set of diverse policies that are parameterized in a way that makes them easily reusable. This diversity and p…
Long-range climate forecasts use integrated assessment models to link the global economy to greenhouse gas emissions. This paper evaluates an alternative economic framework outlined in part 1 of this study (Garrett, 2014) that approaches the global economy using purely physical principles rather than explicitly resolve…
We introduce the Adaptive Skills, Adaptive Partitions (ASAP) framework that (1) learns skills (i.e., temporally extended actions or options) as well as (2) where to apply them. We believe that both (1) and (2) are necessary for a truly general skill learning framework, which is a key building block needed to scale up t…
We present a novel algorithm that predicts the probability that the time derivative of the horizontal component of the ground magnetic field exceeds a specified threshold at a given location. This quantity provides important information that is physically relevant to Geomagnetically Induced Currents (GIC), whic…
Deep reinforcement learning has demonstrated increasing capabilities for continuous control problems, including agents that can move with skill and agility through their environment. An open problem in this setting is that of developing good strategies for integrating or merging policies for multiple skills, where each…
The paper explains how language models acquire complex skills through scaling laws and statistical analysis.
problem Understanding how language models acquire new skills as their size and training data increase.
method Statistical framework and mathematical analysis of scaling laws.
result Language models can learn complex skills efficiently due to a strong inductive bias.
Researchers study how skills are learned in neural networks using physics principles.
problem Understanding how skills are sequentially learned in neural networks.
method Abstract and simplify the problem using physics principles, proposing three models: Geometry, Resource, and Domino.
result Models reveal insights into neural scaling laws, learning dynamics, and the benefits of modularity.
Misspecifying non-compensatory as compensatory IRT: analysis of estimated skills and variancestat.ME
The paper examines skill estimation and variance under model misspecification in IRT.
problem Underestimation and overestimation of skills when non-compensatory model is misspecified as compensatory.
method Theoretical approach to analyze underestimation and overestimation of skills and variance.
result Overestimation of skills occurs around the origin and asymptotic variance differs under model misspecification.
Acquiring new robot motor skills is cumbersome, as learning a skill from scratch and without prior knowledge requires the exploration of a large space of motor configurations. Accordingly, for learning a new task, time could be saved by restricting the parameter search space by initializing it with the solution of a si…
Humans are able to perform a myriad of sophisticated tasks by drawing upon skills acquired through prior experience. For autonomous agents to have this capability, they must be able to extract reusable skills from past experience that can be recombined in new ways for subsequent tasks. Furthermore, when controlling com…
ABC method improves subseasonal weather forecasting by 60-90%.
problem Improving subseasonal temperature and precipitation forecasting accuracy.
method Combines dynamical forecasts with machine learning-based bias correction.
result Significant improvement in temperature and precipitation forecasting skills.
Simulation-to-real transfer is an important strategy for making reinforcement learning practical with real robots. Successful sim-to-real transfer systems have difficulty producing policies which generalize across tasks, despite training for thousands of hours equivalent real robot time. To address this shortcoming, we…
A planning approach learns skills from interactions, balancing exploration and exploitation.
problem Learning robust high-level skills in noisy environments with unknown pre-conditions.
method Formulates skills as high-level policies, learns plans via bandit problems, balances exploration and exploitation.
result A planner capable of learning robust high-level skills in high-dimensional state spaces.
Spaced repetition is among the most studied learning strategies in the cognitive science literature. It consists in temporally distributing exposure to an information so as to improve long-term memorization. Providing students with an adaptive and personalized distributed practice schedule would benefit more than just …
Integrates skills and world models for efficient task solving and transfer.
problem Quickly solve new tasks in complex environments using reusable knowledge.
method Leverages partial amortization for fast adaptation and online skill planning.
result Improved sample efficiency in single tasks and transfer between tasks.
New algorithm trains neural nets on simple skills to learn complex tasks faster.
problem Learning complex tasks through simple imitation.
method Train neural networks on simple, easy-to-learn skills to accelerate learning of complex, hard-to-learn tasks.
result Consistently outperforms state-of-the-art baseline in training speed and performance.
EDL discovers state-covering skills without relying on task rewards.
problem Discovering skills in reinforcement learning without a task-oriented reward function.
method EDL optimizes information-theoretic objective using different machinery to address coverage problem.
result EDL discovers state-covering skills more effectively than existing methods.
Purpose: Manual feedback from senior surgeons observing less experienced trainees is a laborious task that is very expensive, time-consuming and prone to subjectivity. With the number of surgical procedures increasing annually, there is an unprecedented need to provide an accurate, objective and automatic evaluation of…
Sloth predicts LLM performance using latent skills across families.
problem Variations in benchmark performance due to differences in training configurations and data processing across model families.
method Sloth uses publicly available benchmark data and assumes LLM performance is driven by latent skills influenced by model size and training tokens. It exploits correlations across benchmarks to provide accurate predictions.
result Sloth predicts LLM performance accurately and offers insights into scaling behaviors for complex tasks.
The paper presents a method for personalized exercise recommendations that improves learner skill gain.
problem Adapting to individual needs in large, diverse groups of learners in digital environments.
method Contextual Thompson Sampling to select exercises that advance learner skill.
result The method recommends exercises associated with greater skill improvement and adapts to learner differences.
New methods for skill rating in sports using state-space models.
problem Improving skill rating in competitive sports.
method State-space models, sequential Monte Carlo, discrete hidden Markov models.
result Advantages of state-space models for time-varying player skills.
We introduce a method for constructing skills capable of solving tasks drawn from a distribution of parameterized reinforcement learning problems. The method draws example tasks from a distribution of interest and uses the corresponding learned policies to estimate the topology of the lower-dimensional piecewise-smooth…
A Neural Approach to Ordinal Regression for the Preventive Assessment of Developmental Dyslexiacs.LG
Neural model detects DD risk in 5-year-olds, predicting 2 years ahead.
problem Early detection of developmental dyslexia for preventive teaching.
method Mixed neural model using auto-encoder and ordinal regression.
result System predicts DD risk 2 years before phonological processing is assessed.
Hierarchical reinforcement learning is a promising approach to tackle long-horizon decision-making problems with sparse rewards. Unfortunately, most methods still decouple the lower-level skill acquisition process and the training of a higher level that controls the skills in a new task. Leaving the skills fixed can le…
For robots to coexist with humans in a social world like ours, it is crucial that they possess human-like social interaction skills. Programming a robot to possess such skills is a challenging task. In this paper, we propose a Multimodal Deep Q-Network (MDQN) to enable a robot to learn human-like interaction skills thr…
New method boosts skill learning by encouraging optimistic exploration.
problem Intrinsic reward for exploration is inherently pessimistic.
method Derive an information gain auxiliary objective involving an ensemble of discriminators and rewarding policy disagreement.
result Improves skill learning in grid worlds and Atari games.
We consider worker skill estimation for the single-coin Dawid-Skene crowdsourcing model. In practice, skill-estimation is challenging because worker assignments are sparse and irregular due to the arbitrary and uncontrolled availability of workers. We formulate skill estimation as a rank-one correlation-matrix completi…
Paper learns versatile balancing and recovery motions for humanoid robots.
problem Training humanoid robots to handle unexpected perturbations.
method Hierarchical Deep Reinforcement Learning in a physics simulator.
result Learned skills comparable to preprogrammed controllers but more adaptable.
Conventionally, model-based reinforcement learning (MBRL) aims to learn a global model for the dynamics of the environment. A good model can potentially enable planning algorithms to generate a large variety of behaviors and solve diverse tasks. However, learning an accurate model for complex dynamical systems is diffi…
Multiplayer Online Battle Arena (MOBA) games are among the most played digital games in the world. In these games, teams of players fight against each other in arena environments, and the gameplay is focused on tactical combat. Mastering MOBAs requires extensive practice, as is exemplified in the popular MOBA Defence o…
New method automates asymmetric choice for better skill transfer in reinforcement learning.
problem Improving sample efficiency and transferability of reinforcement learning agents.
method Attentive Priors for Expressive and Transferable Skills (APES) using hierarchical KL-regularization.
result APES automates asymmetric choice, leading to better skill transfer across sequential tasks.
The composition of elementary behaviors to solve challenging transfer learning problems is one of the key elements in building intelligent machines. To date, there has been plenty of work on learning task-specific policies or skills but almost no focus on composing necessary, task-agnostic skills to find a solution to …