IOP 3701 EXAM PACK
written by
mrsNURSE01
www.stuvia.com
Downloaded by: mrsNURSE01 |
Distribution of this document is illegal
, Stuvia.com - The Marketplace to Buy and Sell your Study Material
IOP 3701 EXAM PACK-2017
PREPARED BY CHIPO MAREGA
MAY/JUNE 2016
1. Motivate the use of intra scorer and inters corer types of reliability as opposed to other
types of reliability (5)
Inter-scorer (rater)
Inter-scorer (rater) reliability is used to measure the degree to which different raters/observers
give consistent approximate scores/measures of the same observable fact. Examiner
inconsistency is a possible source of error variance. Inter-scorer (or inter-rater) reliability can be
determined by having all the test-takers' test procedures scored by two assessment practitioners.
The correlation coefficient between these two sets of scores reveals the inter-scorer reliability
coefficient.
Advantages of inter scorer validity
Imposes level of objectivity.
Provides validation of evaluation results.
Certify skills competency
Assign the appropriate “grade”
Increase confidence that people are following the guidelines in like manner.
Intra-scorer (rater)
The intra-scorer reliability coefficient refers to the consistency of ratings for a single rater.
Repeated ratings or scores by the same rater would give an indication of the degree of error
variance between such ratings for that particular rater.
Advantages of intra-scorer validity
Improve single marker consistency in evaluating candidates’ performance.
Eliminating external factors that can compromise marker’s evaluation.
Increase the confidence in marker’s ability.
Increase the marker’s competence in repeated occurrence.
Downloaded by: mrsNURSE01 |
Distribution of this document is illegal
, Stuvia.com - The Marketplace to Buy and Sell your Study Material
2. Discuss the importance of item difficulty and discrimination indices in the analysis of
items during the test development process.
Determining item difficulty – p value
This is proportion/percentage of individuals who answer correctly
Number of people who answered correctly divided by number of people who took the
measure
The higher correct responses the easier the item
The smaller the percentages of correct responses, the more difficult
No info about characteristics of item
P value provides uniform measure of difficulty & is used when selecting final items for
measure
Determining discriminating power
One of purpose of item analysis is to discover which items best measures the construct or
content domain that the measure aim to assess
Good items consistently measure the same aspect that the total test is measuring. Individuals
do well in answering a good item correctly, whilst those who do poorly would answer a good
item incorrectly
The discriminating power of an item can be determined by discriminating index & item total
correlation
A positive D value indicates an item that discriminates btw the extreme groups and a negative
D value indicates an item with poor discriminating power
3.. Test results represent only one source of information and this information is predominantly
approximate in nature. Explain this statement (5)
Important issues
Test results represent only one source of information in the assessment process:
- One 1 source of info in the assessment process
- Assessment measures are objective measures scientific and unbiased thus practitioner
begin 2 value them above their own option/judgment
- Thus obscuring the view because they are one sided
- Thus subjectivity should be included
Need to recognize the approximate nature of assessment results
Downloaded by: mrsNURSE01 |
Distribution of this document is illegal
written by
mrsNURSE01
www.stuvia.com
Downloaded by: mrsNURSE01 |
Distribution of this document is illegal
, Stuvia.com - The Marketplace to Buy and Sell your Study Material
IOP 3701 EXAM PACK-2017
PREPARED BY CHIPO MAREGA
MAY/JUNE 2016
1. Motivate the use of intra scorer and inters corer types of reliability as opposed to other
types of reliability (5)
Inter-scorer (rater)
Inter-scorer (rater) reliability is used to measure the degree to which different raters/observers
give consistent approximate scores/measures of the same observable fact. Examiner
inconsistency is a possible source of error variance. Inter-scorer (or inter-rater) reliability can be
determined by having all the test-takers' test procedures scored by two assessment practitioners.
The correlation coefficient between these two sets of scores reveals the inter-scorer reliability
coefficient.
Advantages of inter scorer validity
Imposes level of objectivity.
Provides validation of evaluation results.
Certify skills competency
Assign the appropriate “grade”
Increase confidence that people are following the guidelines in like manner.
Intra-scorer (rater)
The intra-scorer reliability coefficient refers to the consistency of ratings for a single rater.
Repeated ratings or scores by the same rater would give an indication of the degree of error
variance between such ratings for that particular rater.
Advantages of intra-scorer validity
Improve single marker consistency in evaluating candidates’ performance.
Eliminating external factors that can compromise marker’s evaluation.
Increase the confidence in marker’s ability.
Increase the marker’s competence in repeated occurrence.
Downloaded by: mrsNURSE01 |
Distribution of this document is illegal
, Stuvia.com - The Marketplace to Buy and Sell your Study Material
2. Discuss the importance of item difficulty and discrimination indices in the analysis of
items during the test development process.
Determining item difficulty – p value
This is proportion/percentage of individuals who answer correctly
Number of people who answered correctly divided by number of people who took the
measure
The higher correct responses the easier the item
The smaller the percentages of correct responses, the more difficult
No info about characteristics of item
P value provides uniform measure of difficulty & is used when selecting final items for
measure
Determining discriminating power
One of purpose of item analysis is to discover which items best measures the construct or
content domain that the measure aim to assess
Good items consistently measure the same aspect that the total test is measuring. Individuals
do well in answering a good item correctly, whilst those who do poorly would answer a good
item incorrectly
The discriminating power of an item can be determined by discriminating index & item total
correlation
A positive D value indicates an item that discriminates btw the extreme groups and a negative
D value indicates an item with poor discriminating power
3.. Test results represent only one source of information and this information is predominantly
approximate in nature. Explain this statement (5)
Important issues
Test results represent only one source of information in the assessment process:
- One 1 source of info in the assessment process
- Assessment measures are objective measures scientific and unbiased thus practitioner
begin 2 value them above their own option/judgment
- Thus obscuring the view because they are one sided
- Thus subjectivity should be included
Need to recognize the approximate nature of assessment results
Downloaded by: mrsNURSE01 |
Distribution of this document is illegal
