Skill-Based Key Performance Indicators

The following is an exclusive excerpt from the book NSCA’s Essentials of Sport Science, published by Human Kinetics. All text and images provided by Human Kinetics.

Skill-based KPIs are relatively easy to determine in acrobatic and aesthetic sports (e.g., gymnastics, figure skating, synchronized swimming) in that a clear points system exists according to how specific skills are executed. In these sports, the sport scientist should be familiar with the details of how high scores are obtained and help develop the assessment framework to appraise these skills routinely in training and provide the athlete and the coach with a measure of progress. This type of analysis relies mostly on a combination of qualitative and quantitative measurements. The qualitative assessments should be carried out by technical experts (e.g., coaches and judges) and, ideally, in combination with video analysis. Quantitative assessments should rely more on wearable technology, motion capture systems, video analysis (with appropriate procedures typical of biomechanical assessments), and specialized equipment (e.g., force platforms, electromyography).

The key for each type of skill assessment is to provide useful information that can affect subsequent coaching interventions. For this to happen, the main criteria are data collection procedures that minimize disruption to training and competition, ease of access to information, rapid reporting, and, most of all, valid and reliable assessments that can be used to determine the degree of progression.

Acrobatic sports require very precise assessment of the skills required to succeed and are therefore better suited to the use of advanced biomechanical techniques with specific instrumentation in order to increase accuracy and possibly develop modeling approaches that predict the outcome of technique changes. In particular, since elite performers seem to demonstrate less variability of technique in key aspects (22), it is important to have measurement systems that are sensitive to small changes; therefore the use of 2D or 3D video analysis and a combination of wearable sensors are often the preferred route.

Key performance indicators for skill-based analysis should be clearly defined according to a reference framework, and then assessed against it. The terms of reference should be the optimal execution of the skill according to the point system of the sport, as well as past performances of top performers for benchmarking. For this activity, a model template should be drawn that identifies the high-scoring form for each phase of movement; the next step is to perform a comparative analysis on the athlete using video analysis or a combination of wearable technology. In such assessments, joint angles in key positions, relationships between different joint angles, segmental velocities, and other biomechanical parameters tend to be the KPIs of interest; therefore, particular care should be taken to collect data correctly using appropriate techniques and procedures to make sure the data can actually be used to assess the athlete. Indeed, even in today’s sporting landscape, video is often poorly collected, with no reference frame and incorrect camera angles, and resolutions that lead to errors in measurement and incorrect interpretations or assumptions made.

In sports in which skills are not the outcome but are part of the athlete’s toolbox, things become more complicated. Therefore the analysis of complex sport skills is usually conducted to improve technical and tactical outcomes (e.g., assessing the takeoff and running of a long jumper, the kicking technique of a Taekwondo athlete, or throwing techniques in a quarterback), to reduce injury risk (e.g., landing strategies, analysis of side-stepping techniques), or to assess the effectiveness of equipment (e.g., shoes, skis). Such approaches involve assessing a movement sequence relevant to the specific sporting context, and therefore care should be taken in developing the correct assessment procedure, the definition of what constitutes a KPI that can be tracked effectively, and, most of all, how the KPI can be used to produce effective interventions. While technology is helping to move the measurement gold standard from the laboratory to the field, inherent variability occurs when sporting movements are assessed within the live sporting context. Consequently, while some level of validity and reliability can be obtained in some sports (e.g., athletics events, weightlifting) and appropriate KPIs can be developed and tracked (e.g., running speed, takeoff angle, speed of the barbell), more challenges arise when open skills need to be assessed in chaotic sports. For example, a large body of evidence has been produced to understand cutting and change-of-direction maneuvers, which are common skills in team sports and invasion games. Most of the work has been conducted via simulating game conditions in the laboratory (19, 45, 52), and clear guidelines and reference data are available. However, only rarely has the use of opponents been introduced in this type of assessment (30). Unsurprisingly, these data show increased lower limb movements and forces, demonstrating clearly that simulations in a laboratory setting may not reflect what happens in the context of the actual sporting event. The most important aspects to consider when defining KPIs are always related to the validity and reliability of the measurement. Without a valid and reliable measure, such efforts should be discarded. All sports require the application of cognitive, perceptual, and motor skills to varying degrees, and the current consensus seems to be that aside from aesthetic sports or sports with closed skills, challenges lie in defining usable skill-based KPIs in team sports (1).

Finally, consideration should be given to the use of technical models that are particularly popular in many sports, including track and field athletics. While the best techniques have been analyzed and reported for years (and, apart from the technique change in the high jump introduced by Dick Fosbury, have not changed much), these should be used only as a general guideline to drive the training process and qualitatively assess the athlete. Researchers have discussed the possibility of identifying optimal techniques as a reference for athletes (17), further strengthening the view that individual assessments should be implemented that recognize the large role that movement variability, stress of competition, environmental conditions, and other factors play when athletes perform skills and techniques in competition.

The purpose of assessing KPIs in technical and skill-based sports is to identify areas that improve performance, reduce errors, minimize the risks of injury, and optimize the technical execution of a skill. This is represented in figure 5.3.

Skill assessment should be conducted using a holistic approach in which both the perceptual-cognitive aspects and technical motor skills are routinely evaluated, given the reciprocal nature of the relationship between perception and action and the need to intervene to improve both aspects in the practical sporting setting. Most of the scientific literature has so far focused on assessing the skills progression by measuring the outcome of the skill performed rather than the contributing factors. This is possibly due to the limited technology and methods available to conduct such assessments outside of biomechanics laboratories. Some attempts have been made in assessing generic motor skills, with mixed success. However, work describing the Canadian Agility and Movement Skill Assessment has suggested that this approach is valid and reliable (28) if appropriately trained individuals conduct the assessments following the protocol details.

Researchers (15) also suggest the need to assess skills within a periodization framework similar to the assessment of other training and performance-specific KPIs. However, while theoretically sound, the real applicability of this approach is currently limited and the time requirements to conduct a thorough skill assessment are among the main reasons coaches do not spend much time conducting regular quantitative skill assessments. Some examples of well-defined sport skills are published in the literature, with relatively simple data-gathering procedures (e.g., tennis serve [27], table tennis [24], fencing [51]). The equipment of choice still remains the use of video analysis, and thanks to the lower costs of high-speed cameras and access to some advanced applications, it is now possible to conduct detailed analysis of sport skills even with relatively low-cost solutions. Miniature wearable sensors are now permeating the market and in the near future may improve the ability to routinely assess sport skills using body sensor networks with immediate feedback—which may ultimately increase chances to infer meaningful approaches to improve performance. However, while some sports have plenty of examples of well-conducted studies (albeit mostly performed in noncompetitive situations in laboratories) to use as a reference, many sports, including “new” sports, have limited information. Therefore, it is the goal of each practitioner involved to develop an assessment framework that can provide the coaching staff with meaningful information to then correct and enhance skill execution during competitions.

NSCA’s Essentials of Sport Science provides the most contemporary and comprehensive overview of the field of sport science and the role of the sport scientist. It is a primary preparation resource for the Certified Performance and Sport Scientist^® (CPSS^®) certification exam. The book is available in bookstores everywhere, as well as online at the NSCA Store.