Validating test items with spss
Validating test items with spss - Cam2cam adult video chat
The restriction is straightforward: you must have the same number of ratings for every case rated.The questions are more complicated, and their answers are based upon how you identified your raters, and what you ultimately want to do with your reliability estimate.
For example, if you had 2000 ratings to make, you might assign your 10 research assistants to make 400 ratings each – each research assistant makes ratings on 2 ratees (you always have 2 ratings per case), but you counterbalance them so that a random two raters make ratings on each subject.For example, in our Facebook study, we want to know both.First, we might ask “what is the reliability of our ratings?For example, if someone reported the reliability of their measure was .8, you could conclude that 80% of the variability in the scores captured by that measure represented the construct, and 20% represented random variation.The more uniform your measurement, the higher reliability will be.This means that the raters in your task are the only raters anyone would be interested in.
This is uncommon in coding, because theoretically your research assistants are only a few of an unlimited number of people that could make these ratings.
An estimate of interrater reliability will tell me what proportion of their ratings is “real”, i.e.
represents an underlying construct (or potentially a combination of constructs – there is no way to know from reliability alone – all you can conclude is that you are measuring consistently).
Here are the first two questions: If your answer to Question 1 is no, you need ICC(1).
In SPSS, this is called “One-Way Random.” In coding tasks, this is uncommon, since you can typically control the number of raters fairly carefully.
If you’re coding for research, you’re probably going to use the mean rating.