Reliability of coders

 

Intra class correlation

 

To calculate the reliability of the coders in the Classic Expression research group their Intra Class Correlation (ICC) is calculated.

 

This is to know if coders score consistent and similar using a coding construct.

 

It is not an absolute number, but a percentage, based on the relative similarity between coders.

 

It means for example that the highest score of coder 1 is the same child as the highest score of coder 2.

 

ICC is poor/good if:

 

-many scores are (not) the same

 

-the differences are smaller/bigger

 

-the order in grading is different/similar

 

The coders of the Classic Expression coding group that have passed the last test in August 2021 have rates between 82 and 92.

 

These are considered good to excellent by the guidelines for interpretation of ICC by Cicchetti and Koo and Li.

 

Cicchetti (1994):

 

Less than 0.40—poor.

 

Between 0.40 and 0.59—fair.

 

Between 0.60 and 0.74—good.

 

Between 0.75 and 1.00—excellent

 

 

Koo and Li (2016):

 

below 0.50: poor

 

between 0.50 and 0.75: moderate

 

between 0.75 and 0.90: good

 

above 0.90: excellent

 

 

One coder did not yet pass the test yet and will not code in the research.