IOP4861 ASSIGNMENT 1 2021.

IOP4861 ASSIGNMENT 1 2021. Industrial Psychological Assessment. Define reliability and the different types thereof. Furthermore, elaborate on the sources of error influencing reliability Reliability refers to the consistency with which measures whatever it measures (Moerdyk, 2015, Chapter 4). It is important to remember that every assessment will have some error component. The theory of measurement states that when an attribute is measures, we can never obtain a 100% accuracy, this is due to the error component in the measurement. It is the result which is an observed score and consists of a true component which is the aspect we intend to measure, and the error component which is the aspects of the score we do not intent to measure. When the error component is smaller it is said that the reliability of the measure is then greater. Reliability is also expressed through the standard error of measurement (SEM). When a test is not reliable it is therefore not valid. The different types of reliability also referred to as strategies which are applied to tests to ensure the reliability of the measure are: 1.1 Test-retest reliability This is the exact same test, but it is administered at different time under the same standardised conditions to the same people. The coefficient of stability is consistency over time, the results should be consistent in order to show the reliability. 1.2 Parallel or alternate form reliability This is when two equal forms of the same tests are administered to the same group on two different occasions, this should result similar (parallel) tests 2 yielding similar results. The correlations obtained from the two sets of scores represent the reliability coefficient. Meaning they should have the same number of items. It is important the test is equivalent on all levels such as number of items, item content, administration and so forth. Some theorists argue that the development of a parallel or alternate form of any test is time consuming and expensive and therefore not recommended (Wolfaard & Roodt, 2009, P. 48). This method is said to be time consuming and expensive. 1.3 Internal consistency Also known as split-half reliability is when a measure is split into two equal halves after a single administration test. Correlate different parts of the measure to one another, e.g. split the measure in half and calculate the correlation of the results – this is the coefficient of internal consistency. The scores of the two equal halves are correlated to be able to identify and assess if they provide similar measures i.e. the coefficient of internal consistency. Another measure of internal consistency is the coefficient alpha, this is based on the consistency of the item responses. This method compares the responses to the items, it is the average of all the possible split-half reliabilities. 1.4 Inter-score or Inter-rater reliability This is a method used to assess reliability by focusing on the raters of the assessment, such as the open-ended questions, projective techniques etc. Its main purpose it the extent of which two or more raters, observers or judges are able to agree about what is being observed. The determination of inter- scorer reliability is when correlating scores of the two independent assessors of the same test then resulting in the interscorer reliability coefficient. This method can be determined by making sure that all the test takers tests protocols which are then scored by two assessment practioniers. Inter-score is also referred to consistency of rating between raters. 3 2. Errors influencing reliability According to Laher and Cockcroft (2013), there are five types of reliability, namely, test-retest, alternate - form, split-half, inter-item as well as inter- and intra-scorer reliability. 2.1 Sources of error 2.1.1 The test itself Errors can occur for reasons such as the test itself. This is when the person may not understand the way in which the test has been designed or not understanding the items on the test, the item alternatives may also be poor. 2.1.2 Test administration This is when there is a failure in adhering to the time limits set out, poor rapport between the administration and the test taker which will have negative effects on the results as well as various distractions such as noise and temperature 2.1.3 Test scoring This can also affect the reliability of the test results should the markers be too strict or lenient, poor data capturing and scoring procedures which will cause errors. 2.1.4 Test takers This might be that the respondent may be taking the test in a language that is not their home language and are not fluent in the language the test is administered in, the different moods of the test takers can also greatly affect their responses. 1. Critically discuss test moderators that need to be taken into account when testing in a multicultural environment South Africa has a diverse culture in order to be fair it is vital factors such as test moderators are taken into account. During test development there are many factors which can influence the test results. Test moderators as defined by Moerdyk (2015), are variables that influence test results and may include variables before, during and after the assessment. According to Laher and Cockcroft (2013), a person's performance in one administration of a measure does not reflect with complete accuracy the true amount of the trait that the person possesses. Moerdyk (2015) 4 defines fairness as the lack of random error or systematic bias in the assessment technique and/or the interpretation thereof, while discrimination refers to treating people differently from others, due to their race, gender, culture, socio-economic background etc. These Factors that may influence test results and need to be taken into account when testing in a multicultural environment to ensure tests are reliable, valid and fair. 1.1. Language Language is generally regarded as the most important single moderator of performance on assessment measures (Maree, 2009). Miscommunication and misunderstanding can occur if a test taker is not proficient in the language the test is being administered in. This results in the difficulty of being able to determine if the test takers poor performance on their test was due to the language or communication difficulties or if it was a result of a low level of the construct being measured. The language the test is being administered in needs to be the language of the test taker is proficient in. Test scores are impacted greatly by language use and reading ability, the fact that measures administered in languages or for cultures of which other than those for which the test has been standardised. If a test taker is not able to read test items correctly, they will not be able to understand the contents of the test items and this will result in the test taker responding inaccurately or poorly. Test adaptation is also based on retaining the original meaning but refers to that process of making a measure more applicable to a specific context while using the same language. In adapted tests, the language remains the same but the words, context. In view of the differences that exist between various cultural and language groups with respect to their traditions, customs, values and different world views, the same construct could be interpreted and understood in very different ways in various cultural and language groups (Hambleton, 1994). Important to note is that language has been proven to be the one of the most important primary influencers of intelligence test performances and as such it is said that the negative impact when a test is administered to a test taker is a 5 second or third language has a significant negative effect. Children from environmentally disadvantaged backgrounds who are assessed in a language other than their home language perform significantly lower than those who have been assessed in their home language (mother tongue). 1.2. Culture It needs to be kept in mind that the South African society has a diversity of cultures in which appreciation for the culture of origin exists alongside variations in acculturation towards a Western norm (Claassen, 1997). Culture has a pervasive influence on the way we learn, think about things, and behave. An assessment tool is biased when the constructs that are measured do not have the same meaning in different cultures and therefore valid comparisons cannot be achieved between the different test takers from different cultures on the basis of their test results. Constructs have different meanings in for each culture and experiences across cultures differ and thus culture bias can occur and is extremely relevant in personality testing for this reason. Without measures that have cultural relevant content and appropriate norms, fair testing practices may then be compromised as a result. Culture is therefore one of the most important test moderators in test performance as it significantly affects the behaviours and construct being measured. If a test does not take into consideration the culture of the test takers the test will not be reliable or valid, results will not be a true reflection of the test takers results. 1.3. Test-wiseness Moerdyk (2015) describes test wiseness as the person gets used to being assessed, this anxiety level reduces. This is known as test wiseness or test sophistication. People who were previously disadvantaged, little to no schooling, living in communities where they were isolated are not test wise. Which means they are do not have the ability to understand the necessary requirements in order to perform successfully on the test and thus they are not able to understand the motivation for doing the assessment measure and are not familiar with the ethic of working fast and accurate. This is a significant disadvantage for these 6 illiterate adults and the test scores will not be a true reflection of the test performance results. Those that are test wise versus those that are not test wise will not yield the same results from their tests and the test results will not be a true reflection of those that are not test wise. 1.4. Urbanisation Cognitive performance in children from urban areas compared to children in rural areas have shown a significant difference in the tests of cognitive performance. This is due to the fact that urban children’s environments allow for the children to be more stimulated in aspects of cognition which is what is usually assessed by formal psychological measures. From the study done by Addams (2017), Children’s representations of nature using photovoice and community mapping: perspectives from South Africa from previously disadvantaged communities has suggested that overcrowding negatively influences children’s cognitive milieus by, limiting and interrupting exploratory behaviour, decreasing the number of intimate child-caretaker exchanges, and increasing noise levels. Urbanisation is known to be associated with parents that have high levels of education, whereas those from rural areas parents do not have the same educational backgrounds as those from the urban areas. This results in those mothers whom are educated are seen to provide more stimulating home environments which are believed to be beneficial to the children’s cognitive development which include things such as interactive playing with their children, using various techniques to improve their children’s cognitive abilities such as teaching them to learn nursery rhymes and songs as well as allowing them to explore, taking their kids on adventures (outings).