Non-Technical Skills for Surgeons (NOTSS)


What NOTSS is:

The aim of the NOTSS project was to develop and test an educational system for assessment and training of non-technical skills in the intraoperative phase of surgery. NOTSS is a behaviour rating system based on a skills taxonomy that allows valid and reliable observation and assessment of four categories of surgeons' non-technical skill: situation awareness, decision making, communication & teamwork, and leadership. These are the essential non-technical skills surgeons need to perform safely in the OR and NOTSS allows measurement of several ACGME competencies, including professionalism, interpersonal and communication skills, and systems-based practice. The skills taxonomy can be used to structure training and assessment in this emerging area of surgical competence.


How NOTSS was developed:

A multidisciplinary team of psychologists, surgeons and anaesthesiologists developed NOTSS under funding from The Royal College of Surgeons of Edinburgh and NHS Education for Scotland since 2003. Psychologists at the University of Aberdeen in Scotland led the development. A systematic process was used to develop NOTSS, involving subject matter experts (attending surgeons) and replicating methods of systems design used for developing methods of assessing behaviour in other high demand professions such as civil aviation and nuclear power. An adapted model of systems design was used to guide the iterative development of NOTSS through three phases of work from task analysis, through system design, to evaluation . Empirical papers documenting this process are in the surgical, educational and psychology literature. Subject matter experts (consultant/ attending surgeons) were involved at all stages of design and a steering group chaired by applied psychologists facilitated the process. Design criteria ensured that the resulting system was designed by surgeons, for surgeons and was written in surgical language, free of technical or psychological jargon. The resulting system is based on a skills taxonomy which makes the required skills explicit (see table 1). A set of indicative 'good' and 'poor' behaviours were written for each element and are used for training assessors.


Table 1. NOTSS skills taxonomy

Category Element

Situation Awareness

Gatering information
Understanding information
Projecting and anticipating future state

Decision Making

Considering options
Selecting and communicating option
Implementing and reviewing decisions


Setting and maintaining standards
Supporting others
Coping with pressure

Communication and Teamwork

Exhanging information
Establishing a shared understanding
Co-ordinating team


The inter-rater reliability and sensitivity of the system was established in a study of 44 consultant/ attending surgeons who were trained for 3 hours in the use of the NOTSS behaviour rating tool (3). Subsequent phases of work have used the NOTSS system for debriefing trainees after surgery and for coaching non-technical skills in surgical simulation. NOTSS has also been subject to an independent trial of workplace assessment systems along with PBA (Procedure Based Assessment) and OSATS (Objective Structured Assessment of Technical Skill) tools with encouraging results in the operating room.


How NOTSS is used:

The system is in surgical language for suitably trained assessors to observe, rate and provide feedback on non-technical skills in a structured manner. Surgeons have been using NOTSS in the OR and in simulated settings. The Royal College of Surgeons of Edinburgh has been successfully running a NOTSS Masterclass in observing and rating behaviour for attending surgeons since 2006. Faculty development has also occurred for groups in North America, Japan and Australia. NOTSS has been adopted by the Royal Australasian College of Surgeons as part of their competence assessment and recommended by the ACGME (Accreditation Council for General Medical Education) for workplace assessment. In the UK, it is proposed that all surgical trainees will be assessed using NOTSS; two online training resources NOTSS in a box (for senior trainees and consultants) and NOTSS for Trainees (for early stage trainees) have been developed by RCSEd and discussions regarding implementation methods are ongoing.


Applying NOTSS to surgical training:

NOTSS has a number of applications to structure the training and assessment of surgeons. The skills taxonomy could be used to develop a curriculum for residents with behaviours expected at each level of training. For formative assessment, NOTSS has been successfully trialled as a method of coaching residents in simulation. NOTSS is ideally placed to measure ACGME competencies and provide evidence for milestones or targets in surgical training. Ultimately, NOTSS could be used as a summative assessment tool for progression through surgical training and beyond.