MRI grading of knee osteoarthritis

The aim and objective of the "Surgeons-in- Training" section of the Journal is to encourage surgeons who are in BST or SPR posts to submit ongoing work that has so far been unpublished but is of a sufficient maturity to warrant publication.Only one author may not be a surgeon in training, and the leading author responsible for the majority of the work must be a trainee. The guidelines for submissions require manuscripts to be tightly disciplined in production but hopefully will provide a design and format for article writing that trainees will find useful in other contexts. The section will provide feedback for submitting authors whether or not a manuscript is accepted, and referees' comments regarding published articles will be provided as a learning tool for readers and other authors. Guidelines are included in the general Notes for Authors, and fuller details can be accessed from the College website at http://www.rcsed.ac.uk

If you are interested in submitting a Surgeons-in-Training manuscript to the Journal or have any queries regarding the SiT section, please contact the Associate Editor for the section, Professor David Rowley, via the Editorial Office.

Observer agreement analysis of MRI grading of knee osteoarthritis

M.J. MCNICHOLAS*, A.J. BROOKSBANK* and C.M. WALKER#
*University Department of Orthopaedic and Trauma Surgery, Dundee Royal Infirmary; #Department of Radiology, Dundee Royal Infirmary

The aim of the study was to use Kappa statistical analysis to assess the inter and intraobserver agreement of the interpretation of the osteoarthritis changes observed on a FISP sequence, 1.0 Tesla, MRI analysis of the articular cartilage in 30 knees. The images were read on two separate occasions by three observers. The best agreement was seen in the patello-femoral compartment. The most experienced of the assessors produced the more consistent results. The results of the interobserver agreement had mainly "slight" or "fair" agreement. Our results are disappointing and accordingly, we have reservations about the use of MRI in the assessment of osteoarthritis of the knee.

Keywords: magnetic resonance imaging (MRI), knee, osteoarthritis, gonarthrosis, osteoarthrosis, intraobserver agreement, interobserver agreement, grading

Magnetic resonance imaging (MRI) has been in widespread clinical use since 1985 and within this relatively short period there has been a revolution in musculoskeletal imaging. For meniscal lesions, correlation of image diagnosis with arthro-scopic findings was at the 86% level for the "expert" knee "imager", but only 75% for the "non-expert".¹

MRI can demonstrate chondral defects as small as 2 mm in diameter and accurate imaging of hyaline cartilage thickness with differentiation of cartilage from fluid is reported.² The MRI appearance of normal and abnormal articular cartilage, and the various sequences that can be used to evaluate chondral lesions have been described. The exact role of MRI in such evaluation remains undefined.^3-5

The aim of the study was to assess the inter and intraobserver agreement of MRI analysis of articular cartilage of the knee using an established scoring system ⁶

MRI scans of 30 knees were read on two separate occasions by three observers (one Consultant Radiologist: CW; and two orthopaedic trainees: MMcN and AB). In cases of dispute a consensus score was given. The observers were blinded from other results, and the readings were at least a week apart. In order to familiarise ourselves with the grading system, an initial reading was carried prior to data collection. The medial, lateral and patello-femoral compartments were scored separately.

The hyaline cartilage changes in each of the 3 compartments were graded from zero to three. Grade 0 was assigned to normal cartilage. Grade 1 changes were reserved for cartilage that showed minor surface irregularity, due to early fissuring, but no loss of thickness of the cartilage. Definite cartilage loss due to deeper fissuring, but without full thickness loss indicated grade 2 changes. Areas of full thickness cartilage loss were assigned grade 3.⁶

Imaging was performed on a Siemens 1.0 T MRI scanner using a dedicated surface coil to give a field of view of approximately 16cm. The patient was positioned supine, with the knee extended and externally rotated by 20 degrees. The sequence employed was FISP 3D TR 40, TE 15, flip angle 40 degrees, with a slab thickness of 100mm, matrix 256 x 256, with one acquisition in sagittal orientation.

Rather than comparing percentage observer agreement, the "Kappa" statistic⁷ was calculated, which acts to eliminate agreement by chance. A value of 1.0 means that both assessors agree on every single case, a value of zero means that any agreement has been by chance alone, negative results imply worse agreement than that expected by chance. Values of

0.00-0.20 represent slight agreement, 0.21-0.40 fair agreement, 0.41-0.60 moderate agreement, 0.61-0.80 substantial agreement, and above 0.8 is considered to be almost perfect.⁸

Kappa statistical analysis and grading of these data defined the intra and interobserver agreements for the three assessors in their interpretation of the OA changes on the MR images.

The levels of intraobserver agreement for the three different compartments were variable (see Table 1); the best agreement was seen in the patello-femoral compartment. The most experienced of the assessors produced the more consistent results. The results of the interobserver agreement were generally poor, the majority having only "slight" or "fair" agreement (see Table 2).

Table 1: Intraobserver agreement analysis for MRI analysis of articular cartilage

Observer studied	Lateral compartment Kappa score (group)	Medial compartment Kappa score (group)	Patello-femoral compartment Kappa score (group)
MMcN	()	0.19 (slight)	0.33 (fair)
AB	0.6 (moderate)	()	0.51 (moderate)
CW	0.17 (slight)	0.32 (fair)	0.73 (substantial)

* denotes inability to compute a kappa statistic for a feature due to the table for that particular feature containing row values that did not equal column values. MMcN = MJ McNicholas, AB = AJ Brooksbank, CW = CM Walker

Table 2: Interobserver agreement analysis for MRI analysis of articular cartilage

Observer compared	Lateral compartment Kappa score (group)	Medial compartment Kappa score (group)	Patello-femoral compartment Kappa score (group)
MMcN v AB	0.38 (fair)	()	0.0 (chance)
MMcN v CW	0.29 (slight)	0.22 (slight)	0.33 (fair)
MMcN v Con	0.38 (fair)	0.06 (slight)	0.19 (slight)
AB v CW	0.16 (slight)	()	0.17 (slight)
AB v Con	0.35 (fair)	()	0.10 (slight)
CW v Con	0.32 (fair)	0.17 (slight)	0.73 (substantial)

These results are disappointing, and we must therefore consider the potential reasons for this. The scoring system described by Heron has some limitations. In her actual study the correlation between MRI and arthroscopy was best seen in Grades 2 and 3 cartilage loss, i.e. when there was actual loss of height of articular cartilage. In MRI sequences there is difficulty in defining what in a series of 4 to 6 slices actually represents the articular surface, the problems are compounded by partial volume effects confusing the point at which the intercondylar notch actually begins. What is the normal MRI anatomy of the articular cartilage? For example a commonly seen area of thinner articular cartilage beneath the anterior horn of the meniscus may be confused with pathological thinning. The "magic angle effect" which means assessment of the superior and inferior pole of the patellar articular cartilage is not possible; the definition by an observer of what is perceived as artefact, as opposed to actual localised cartilage loss is unknown.

The most experienced observer was only able to produce consistent results for the patello-femoral compartment. This may reflect, at least in part the relatively small area being analysed in a smaller number of slices. Most clinical measures fall into the "moderate" group.⁹ Our results are disappointing and accordingly, we have reservations about the use of MRI in the assessment of osteoarthritis of the knee. We plan to compare the performance of MRI assessment of OA with that of radio-graphic analysis, and the correlation of these scores with functional data. There is clearly a need for the identification of early defects of hyaline cartilage, but the reproducibility of MRI assessment using this scoring system has been disappointing.

We are grateful to Mrs T McLeay, Superintendent Radiographer of MRI and CT, Department of Radiology, and to Mr DW McGurty, research statistician, University department of Orthopaedic and Trauma Surgery, Dundee Royal Infirmary for their expert advice and assistance with imaging and statistical analysis which form the basis of this paper.

Correspondence: Mr M.J. McNicholas, Department of Orthopaedic and Trauma Surgery, University of Dundee, Dundee, Scotland, UK.

� 1998 The Royal College of Surgeons of Edinburgh, J.R.Coll.Surg.Edinb., 44, February, 31-33