Psychological - W2 - Chapter 5 - Reliability (DN) Part 4
This deck covers key concepts from Chapter 5 on reliability in psychological testing, focusing on generalizability theory, variance, error variance, and methods for estimating reliability.
60 universe
Key Terms
universe score
variance
a statistic useful in describing SOURCES of test score variability equal to the MEAN of the SQUARES of the DIFFERENCES between SCORES in a distribu...
What is the main challenge of a test creator?
to MAXIMIZE the proportion of TOTAL VARIANCE that is TRUE VARIANCE and to MINIMIZE the proportion that is ERROR VARIANCE - p.147
What are four main SOURCES of ERROR VARIANCE?
1) TEST CONSTRUCTION item sampling/content sampling 2) TEST ADMINISTRATION test environment; testtaker variables; examiner related variable. 3) TES...
What are some methods of measuring INTERNAL CONSISTENCY of a test's items?
1) SPEARMAN-BROWN FORMULA p.153-4 2) KUDER-RICHARDSON FORMULAS p.155-6 3) COEFFICIENT ALPHA p.157 4) AVERAGE PROPORTIONAL DISTANCE (APD) p.157
Related Flashcard Decks
Study Tips
- Press F to enter focus mode for distraction-free studying
- Review cards regularly to improve retention
- Try to recall the answer before flipping the card
- Share this deck with friends to study together
| Term | Definition |
|---|---|
60 universe | in GENERALIZABILITY THEORY
the TOTAL CONTEXT of a particular test situation
including ALL the FACTORS that lead to an individual testtakers score
p.167 |
universe score | in GENERALIZABILITY THEORY
a test score corresponding to the PARTICULAR UNIVERSE being assessed or evaluated
p. 167 |
variance | a statistic useful in describing SOURCES of test score variability equal to the MEAN of the SQUARES of the DIFFERENCES between SCORES in a distribution and THEIR MEAN calculated by SQUARING & SUMMING all the DEVIATION SCORES then DIVIDING by the total number of scores p.95, 146 |
What is the main challenge of a test creator? | to MAXIMIZE the proportion of TOTAL VARIANCE that is TRUE VARIANCE and to MINIMIZE the proportion that is ERROR VARIANCE - p.147 |
What are four main SOURCES of ERROR VARIANCE? | 1) TEST CONSTRUCTION item sampling/content sampling 2) TEST ADMINISTRATION test environment; testtaker variables; examiner related variable. 3) TEST SCORING and INTERPRETATION scorers; scoring systems). 4) OTHER SOURCES OF ERROR sampling error methodological error - researchers not trained, ambiguous wording, item biases) p. 147 - 149 |
What are some methods of measuring INTERNAL CONSISTENCY of a test's items? | 1) SPEARMAN-BROWN FORMULA p.153-4 2) KUDER-RICHARDSON FORMULAS p.155-6 3) COEFFICIENT ALPHA p.157 4) AVERAGE PROPORTIONAL DISTANCE (APD) p.157 |
How is obtaining estimates of ALTERNATE-FORMS reliability & PARALLEL FORMS reliability SIMILAR? | 1) Two test administrations with the SAME GROUP are required 2) Test scores between tests may be AFFECTED by factors such as MOTIVATION, FATIGUE, or INTERVENING EVENTS (practise, learning or therapy) - although not as much as if the EXACT SAME test had been administered twice |
What is an INHERENT source of ERROR-VARIANCE when computing an ALTERNATE or PARALLEL-FORMS reliability coefficient? | ITEM SAMPLING ERROR | p. 152 |
What are the THREE steps of computation of a COEFFICIENT of SPLIT-HALF RELIABILITY? | Step 1 - Divide the test into EQUIVALENT HALVES Step 2 - calculate a Pearson r between scores on the TWO HALVES of the test STEP 3 - adjust the HALF-TEST reliability using the SPEARMAN-BROWN FORMULA (p.152-153) |
Contrast the Coefficient alpha & Pearson r | Ca - 0-1 Pr - -1 to +1 Ca - gauging how SIMILAR data sets are PR - dealing with SIMILARITY & DISSIMILARITY |
What is the DIFFERENCE between the FOCUS of Average proportional distance (APD) and SPLIT-HALF methods & CRONBACH's ALPHA? | APD - focus is on the DEGREE of DIFFERENCE between item scores SH & CA - focus is on SIMILARITIES between item scores p.157 |
What are the 3 approaches to ESTIMATING RELIABILITY? | 1) test-retest 2) alternate or parallel forms 3) internal or inter-item consistency method chosen will depend on a number of factors - e.g., the PURPOSE, NATURE for obtaining the measure p. 160 |
How do we decide which RELIABILITY COEFFICIENT to CHOOSE (use)? | the method chosen will depend on a number of factors - e.g., the PURPOSE, NATURE for obtaining the measure NOTE: the various RELIABILITY COEFFICIENTS DO NOT all reflect the same SOURCES of ERROR VARIANCE see pg. 161 (impt to understand why each test is selected, also refer to Table 5-4) |
What are the 3 ASSUMPTIONS made when using IRT? | 1) Unidimensionality 2) Local Independence 3) Monotonicity p. 170 |