AptitudekTestkConsultation
Contentsk
1 Background 3
1.1 Proposedkalteration 3
1.2 Background,krationalekandkproblemskwithkthekcurrentksystem 3
1.3 AptitudekandkIELTS 7
1.4 PossiblekAdversekEffects 8
1.5 Aim 9
2 ThekFirstkPilotkJank–kAugk2010
2.1 Resultskofkthekfirstkpilot 10
2.2 Needkforkakfurtherkpilot 10
3 ThekSecondkpilotkNovemberk2010k–kNovemberk2011 11
3.1 Hypothesis 11
3.2 Method 11
3.3 ScorekDistribution 14
3.4 PredictingkPerformance 14
3.5 DemographickScorekComparisons 17
4 DiscussionkPoints 19
4.1 Cutkscores 19
4.2 Alternatives 22
5 ThekRegulatorykObjectiveskandkBetterkregulationkprinciples 24
5.1 ThekRegulatorykObjectives 24
5.2 ThekBetterkRegulationkPrinciples 26
6 Evaluation 29
6.1 Constraints 29
6.2 OperationalkIssues 30
Pagek1kofk138
,7 Conclusions 34
7.1 Mainkfindings 34
7.2 DesiredkOutcomekandkRecommendations 34
8 ConsultationkQuestions 36
Appendicesk
1 CommentskreceivedkduringkBVCkReviewk&kConsultationk2007-2008 37
2 IndependentkConsultantkReportkofkthekfirstkpilot 40
3 IndependentkConsultantkReportksecondkpilot,kReportk1:kMonitoring 54
4 IndependentkConsultantkReportksecondkpilot,kReportk2:kInitialkValidation 66
5 IndependentkConsultantkReportksecondkpilot,kReportk3:kFinalkValidation 91
6 CommentskonkthekDewberrykReport 137
7 EqualitykImpactkAssessment (2009,kupdatedk2010kandk2011) 140
Informationkonkthekformatkofkthekproposedktestkandkexamplekquestionskcankbekfoundkatkhttp://w
ww.barstandardsboard.org.uk/media/1344440/watsonglaser_form_ab_example_questions.p
dfk
ResponsesktokthekconsultationkQuestionsk(pagek39)kshouldkbeksubmittedkbyk
29kFebruaryk2012kto:k
AptitudekTestkConsultation
BarkStandardskBoard
289-293kHighkHolborn
LondonkWC1Vk7HZ
Orkbykemailkto:
Responseskarekwelcomekfromkallkthosekwhokmaykhavekviewskorkevidencekrelatingktokiss
ueskraisedkinkthiskpaper.kItkwouldkbekhelpful;kifkresponseskcouldkbekfullykpresentedkwithk
detailedkreasonskgivenkforkcomments,kaskwellkaskanykunderpinningkevidence.k
ThekBSBkwillksummarisekthekresponseskreceivedkandkwillknormallykpublishkresponsesko
nkitskwebsite.kIfkyoukdoknotkwishkyourkresponsektokbekpublished,kpleasekmakekthatkclearki
nkyourkresponse.
Pagek2kofk138
,1 Background
1.1 Proposedkalteration
Itkiskproposedkthat,kinkadditionktokexistingkentrykrequirementskaskspecifiedkinkthekBarkTrainingkR
egulationsk(BTRs1),kapplicantskforkthekpostgraduatekBarkProfessionalkTrainingkCoursek(BPTC)
,kformerlykknownkaskthekBarkVocationalkCoursek(BVC),kbekrequiredktokattainkakminimumkpasskth
resholdkonkthekBarkCoursekAptitudekTest2k(BCAT),kwhichkhaskbeenkcarefullykdevelopedkandkpil
otedkspecificallykforkthiskpurpose.
Itkiskproposedkthatkthiskchangekshouldkcommencekwithkthekcohortkofkcandidateskapplyingkfromk
Novemberk2012ktokstartkthekcoursekinkSeptemberk2013.
Itkiskproposedkthatkthosektakingkthektestkwillkbekallowedkankunlimitedknumberkreksits.kCandidates
kwithkadequatekskillskbutkonek(orkmore)kunsuccessfulkattemptskatkthekBCATkwillknotkbekprevente
dkfromkundertakingkthekBPTCkoncektheykhavekachievedkthekrequisitekscore.
k
ItkiskproposedkthatkthektestkwillkbekrunkbykPearsonkVuekandkthereforekavailablektokbektakenkatkan
ykPearsonkVuektestingkcentre,kofkwhichktherekarekhundredskworldwide.
1.2 Background,krationalekandkproblemskwithkthekcurrentksystem
DuektokpressurekfromkthekOFT,kthekmonopolykofkthekInnskofkCourtkSchoolkofkLawkinkdeliveringkt
hekBarkCoursekendedkand,kfollowingkakvalidationkexercise,keightkinstitutionskinkEnglandkandkWa
leskwerekapprovedktokdeliverkthekcourse.kThekperiodk1997-
2006ksawkaksteadykincreasekinkstudentknumbers,kakwideningkgapkbetweenktheknumberkofkgrad
uateskandkpupillageskavailablekandkakhighkratekofkfailurekonkthekcourse.kConcernskaboutktheksta
ndardskonkthekBarkVocationalkCoursekandkthekmismatchkbetweenkthekgrowingknumberkofkBVCk
graduateskandkreducingknumberskofkPupillageskhadkincreasedksignificantlykbykthektimekofktheke
stablishmentkofkthekBarkStandardskBoard.kAkmajorkReviewkofkthekBVCkwaskthereforekcommissi
onedkbykthekBSBkwithkakWorkingkGroupkChairedkbykDerekkWoodkQC,kwhichkproducedkrecomm
endationskafterkworkingkonkthekprojectkbetweenkOctoberk2007kandkJulyk2008.kk
Duektokconcernskexpressedkatkthekstandardkandkapparentkunsuitabilitykofkmanykstudentskonkth
ekcourse,kthekBSBkadoptedkthekrecommendationk(amongstkothers)kofkthekWoodkReviewkthatkst
andardskshouldkbekraisedkonkentrykandkexitktokthekBarkCourse,kwhichkwaskrenamedkthekBPTC.k
Thiskincludedkthekrecommendationkthatkthekentrykrequirementskforkthekcoursekbekamendedktoki
ncludekankAptitudekTestkthatkwouldkbekfairktokallkapplicantskandktestkthekrequisitekskillsk(ratherkt
han,kforkexample,kexcludingksomeksuitablekcandidateskbykraisingkacademickentrykrequirements
).kkItkwaskthereforekproposedkthat,kinkadditionktokexistingkrequirementskaskspecifiedkbykthekCons
olidatedkRegulations3,ksuchkaktestkshouldkbekdevelopedkandkthatkthisktestkshouldkbekcompulsor
ykforkallkstudentsktokpasskbeforektheykarekablektokstartkakBarkCoursek(theknewkBPTC).k
1
kCankbekfoundkatkhttp://www.barstandardsboard.org.uk/media/1344498/bartrainingregulations-1092011.pdfk
2
kThekAptitudekTestkiskbasedkonkthekestablishedkandkrecognisedkWatsonkGlaserkCriticalkThinkingkTestkwhichkiskusedkbyksomekl
awkfirmskinkrecruitmentkassessmentkdayskandkbykthekGraduatekManagementkAdmissionskCouncil.kThiskiskdiscussedkinkmorekde
tailklaterkinkthiskreport.
3
kSupersededkbykthekBarkTrainingkRegulations(BTRs)kfromk1kSeptemberk2009
Pagek3kofk138
, Thiskrecommendationkwaskmadekinkorderktokaddresskconcernskaboutkthekabilitieskofkstudentsku
ndertakingkthekcourse.kThesekconcernskaroseknotkonlykfromkaksignificantkbodykofkanecdotalkevi
dence,kbutkalsokthekexperiencekofkpanelkmemberskwhokhadkvisitedkthekBarkVocationalkCoursek
Providerskandkfromkspeakingktokinterestkgroupskaskdiscussedklaterkinkthiskreport.
Akthresholdkrequirementkorkpassklevelkshouldkbekidentifiedktokensurekthatkonlyksuitablekcandida
teskshouldkbekadmittedktokthekcourse.kAnykstudentkunablektokachievekthekpasskthresholdkinkthekt
estkwouldknotkbekeligiblektokstartkthekcoursekbutkankunlimitedknumberkofkreksitskwouldkbekpermitt
edksokthatkcandidatesksubsequentlykattainingkthekrequiredkthresholdkwouldkthenkbekadmitted.k
Akdetailedkprogrammekofkdevelopmentkandkimplementationkthenkfollowed:
Octoberk2007k–kJulyk2008 WoodkReviewkincludingkconsultation
Septemberk2008k–kJulyk2009
Developmentkofkspecificationkforkthektest,ktenderingkproc
esskforkaksuitablekproviderkandkindependentkexternalkcon
sultant/statisticiank
Septemberk2009k–kJulyk2010
Firstkpilotk(ck200kBVCkstudentsktookkthektestktokdetermine
basickviability,kapproachktokassessmentkandkvaliditykaskak
k
methodkofkidentifyingksuitablekcandidateskforkthekBPTC)
Septemberk2010k–kJulyk2011
Secondkpilotk(ck1500kBPTCkstudentsktookkthektestktokesta
blishkitskvalidity,ktestkthekassessmentskthemselveskandkde
terminekthekpasskthresholdkork‘cutkscore’)
Julyk2011k–kNovemberk2011 Analysiskofkfindingskandkrecommendationsk
Decemberk2011 BSBkformalkconsideration
Decemberk2011k–kFebruaryk2012k Furtherkconsultationkperiod
ItkwaskdeterminedkthatkthektestkwouldkbekrunkbykPearsonkVue,kavailablektokbektakenkatkanykofkth
eirktestingkcentres,kofkwhichktherekarekoverk150kinkthekUKkandkotherskink165kcountrieskworldwid
e.kThekBSBkalsokdulykengagedkankindependentkconsultantkwhokwaskemployedktokassesskwhet
herkthekchosenktest waskappropriatekforkusekandkalsokcontractedkwithkPearsonkVuektokrunktwokp
k
ilotskofkthektest,kaskdetailedkabove.k
1.2.1 StandardskandkFailurekrateskonkthekBarkCoursek(BVC)
ThekWorkingkGroupkreportedkthatkthekexistingkentrykrequirementskhadkbeenkfoundktokbekinsuffi
cientkinktheirkpresentkformktokmaintainktheknecessarykstandardskonkentrykforkthekBarkCourse:k“T
hekstudentkbodykincludeskgraduateskwhokareksokfarklackingkinkthekqualitieskneededkforksuccessf
ulkpracticekatkthekBar....kthatktheykwouldkneverkobtainkpupillage,khoweverkmanykpupillageskwere
kavailable.”kThiskiskbornekoutkbykstatisticalkevidencekof klowkstandardskonkthekcourse.kInkfact,kfork
Pagek4kofk138
Contentsk
1 Background 3
1.1 Proposedkalteration 3
1.2 Background,krationalekandkproblemskwithkthekcurrentksystem 3
1.3 AptitudekandkIELTS 7
1.4 PossiblekAdversekEffects 8
1.5 Aim 9
2 ThekFirstkPilotkJank–kAugk2010
2.1 Resultskofkthekfirstkpilot 10
2.2 Needkforkakfurtherkpilot 10
3 ThekSecondkpilotkNovemberk2010k–kNovemberk2011 11
3.1 Hypothesis 11
3.2 Method 11
3.3 ScorekDistribution 14
3.4 PredictingkPerformance 14
3.5 DemographickScorekComparisons 17
4 DiscussionkPoints 19
4.1 Cutkscores 19
4.2 Alternatives 22
5 ThekRegulatorykObjectiveskandkBetterkregulationkprinciples 24
5.1 ThekRegulatorykObjectives 24
5.2 ThekBetterkRegulationkPrinciples 26
6 Evaluation 29
6.1 Constraints 29
6.2 OperationalkIssues 30
Pagek1kofk138
,7 Conclusions 34
7.1 Mainkfindings 34
7.2 DesiredkOutcomekandkRecommendations 34
8 ConsultationkQuestions 36
Appendicesk
1 CommentskreceivedkduringkBVCkReviewk&kConsultationk2007-2008 37
2 IndependentkConsultantkReportkofkthekfirstkpilot 40
3 IndependentkConsultantkReportksecondkpilot,kReportk1:kMonitoring 54
4 IndependentkConsultantkReportksecondkpilot,kReportk2:kInitialkValidation 66
5 IndependentkConsultantkReportksecondkpilot,kReportk3:kFinalkValidation 91
6 CommentskonkthekDewberrykReport 137
7 EqualitykImpactkAssessment (2009,kupdatedk2010kandk2011) 140
Informationkonkthekformatkofkthekproposedktestkandkexamplekquestionskcankbekfoundkatkhttp://w
ww.barstandardsboard.org.uk/media/1344440/watsonglaser_form_ab_example_questions.p
dfk
ResponsesktokthekconsultationkQuestionsk(pagek39)kshouldkbeksubmittedkbyk
29kFebruaryk2012kto:k
AptitudekTestkConsultation
BarkStandardskBoard
289-293kHighkHolborn
LondonkWC1Vk7HZ
Orkbykemailkto:
Responseskarekwelcomekfromkallkthosekwhokmaykhavekviewskorkevidencekrelatingktokiss
ueskraisedkinkthiskpaper.kItkwouldkbekhelpful;kifkresponseskcouldkbekfullykpresentedkwithk
detailedkreasonskgivenkforkcomments,kaskwellkaskanykunderpinningkevidence.k
ThekBSBkwillksummarisekthekresponseskreceivedkandkwillknormallykpublishkresponsesko
nkitskwebsite.kIfkyoukdoknotkwishkyourkresponsektokbekpublished,kpleasekmakekthatkclearki
nkyourkresponse.
Pagek2kofk138
,1 Background
1.1 Proposedkalteration
Itkiskproposedkthat,kinkadditionktokexistingkentrykrequirementskaskspecifiedkinkthekBarkTrainingkR
egulationsk(BTRs1),kapplicantskforkthekpostgraduatekBarkProfessionalkTrainingkCoursek(BPTC)
,kformerlykknownkaskthekBarkVocationalkCoursek(BVC),kbekrequiredktokattainkakminimumkpasskth
resholdkonkthekBarkCoursekAptitudekTest2k(BCAT),kwhichkhaskbeenkcarefullykdevelopedkandkpil
otedkspecificallykforkthiskpurpose.
Itkiskproposedkthatkthiskchangekshouldkcommencekwithkthekcohortkofkcandidateskapplyingkfromk
Novemberk2012ktokstartkthekcoursekinkSeptemberk2013.
Itkiskproposedkthatkthosektakingkthektestkwillkbekallowedkankunlimitedknumberkreksits.kCandidates
kwithkadequatekskillskbutkonek(orkmore)kunsuccessfulkattemptskatkthekBCATkwillknotkbekprevente
dkfromkundertakingkthekBPTCkoncektheykhavekachievedkthekrequisitekscore.
k
ItkiskproposedkthatkthektestkwillkbekrunkbykPearsonkVuekandkthereforekavailablektokbektakenkatkan
ykPearsonkVuektestingkcentre,kofkwhichktherekarekhundredskworldwide.
1.2 Background,krationalekandkproblemskwithkthekcurrentksystem
DuektokpressurekfromkthekOFT,kthekmonopolykofkthekInnskofkCourtkSchoolkofkLawkinkdeliveringkt
hekBarkCoursekendedkand,kfollowingkakvalidationkexercise,keightkinstitutionskinkEnglandkandkWa
leskwerekapprovedktokdeliverkthekcourse.kThekperiodk1997-
2006ksawkaksteadykincreasekinkstudentknumbers,kakwideningkgapkbetweenktheknumberkofkgrad
uateskandkpupillageskavailablekandkakhighkratekofkfailurekonkthekcourse.kConcernskaboutktheksta
ndardskonkthekBarkVocationalkCoursekandkthekmismatchkbetweenkthekgrowingknumberkofkBVCk
graduateskandkreducingknumberskofkPupillageskhadkincreasedksignificantlykbykthektimekofktheke
stablishmentkofkthekBarkStandardskBoard.kAkmajorkReviewkofkthekBVCkwaskthereforekcommissi
onedkbykthekBSBkwithkakWorkingkGroupkChairedkbykDerekkWoodkQC,kwhichkproducedkrecomm
endationskafterkworkingkonkthekprojectkbetweenkOctoberk2007kandkJulyk2008.kk
Duektokconcernskexpressedkatkthekstandardkandkapparentkunsuitabilitykofkmanykstudentskonkth
ekcourse,kthekBSBkadoptedkthekrecommendationk(amongstkothers)kofkthekWoodkReviewkthatkst
andardskshouldkbekraisedkonkentrykandkexitktokthekBarkCourse,kwhichkwaskrenamedkthekBPTC.k
Thiskincludedkthekrecommendationkthatkthekentrykrequirementskforkthekcoursekbekamendedktoki
ncludekankAptitudekTestkthatkwouldkbekfairktokallkapplicantskandktestkthekrequisitekskillsk(ratherkt
han,kforkexample,kexcludingksomeksuitablekcandidateskbykraisingkacademickentrykrequirements
).kkItkwaskthereforekproposedkthat,kinkadditionktokexistingkrequirementskaskspecifiedkbykthekCons
olidatedkRegulations3,ksuchkaktestkshouldkbekdevelopedkandkthatkthisktestkshouldkbekcompulsor
ykforkallkstudentsktokpasskbeforektheykarekablektokstartkakBarkCoursek(theknewkBPTC).k
1
kCankbekfoundkatkhttp://www.barstandardsboard.org.uk/media/1344498/bartrainingregulations-1092011.pdfk
2
kThekAptitudekTestkiskbasedkonkthekestablishedkandkrecognisedkWatsonkGlaserkCriticalkThinkingkTestkwhichkiskusedkbyksomekl
awkfirmskinkrecruitmentkassessmentkdayskandkbykthekGraduatekManagementkAdmissionskCouncil.kThiskiskdiscussedkinkmorekde
tailklaterkinkthiskreport.
3
kSupersededkbykthekBarkTrainingkRegulations(BTRs)kfromk1kSeptemberk2009
Pagek3kofk138
, Thiskrecommendationkwaskmadekinkorderktokaddresskconcernskaboutkthekabilitieskofkstudentsku
ndertakingkthekcourse.kThesekconcernskaroseknotkonlykfromkaksignificantkbodykofkanecdotalkevi
dence,kbutkalsokthekexperiencekofkpanelkmemberskwhokhadkvisitedkthekBarkVocationalkCoursek
Providerskandkfromkspeakingktokinterestkgroupskaskdiscussedklaterkinkthiskreport.
Akthresholdkrequirementkorkpassklevelkshouldkbekidentifiedktokensurekthatkonlyksuitablekcandida
teskshouldkbekadmittedktokthekcourse.kAnykstudentkunablektokachievekthekpasskthresholdkinkthekt
estkwouldknotkbekeligiblektokstartkthekcoursekbutkankunlimitedknumberkofkreksitskwouldkbekpermitt
edksokthatkcandidatesksubsequentlykattainingkthekrequiredkthresholdkwouldkthenkbekadmitted.k
Akdetailedkprogrammekofkdevelopmentkandkimplementationkthenkfollowed:
Octoberk2007k–kJulyk2008 WoodkReviewkincludingkconsultation
Septemberk2008k–kJulyk2009
Developmentkofkspecificationkforkthektest,ktenderingkproc
esskforkaksuitablekproviderkandkindependentkexternalkcon
sultant/statisticiank
Septemberk2009k–kJulyk2010
Firstkpilotk(ck200kBVCkstudentsktookkthektestktokdetermine
basickviability,kapproachktokassessmentkandkvaliditykaskak
k
methodkofkidentifyingksuitablekcandidateskforkthekBPTC)
Septemberk2010k–kJulyk2011
Secondkpilotk(ck1500kBPTCkstudentsktookkthektestktokesta
blishkitskvalidity,ktestkthekassessmentskthemselveskandkde
terminekthekpasskthresholdkork‘cutkscore’)
Julyk2011k–kNovemberk2011 Analysiskofkfindingskandkrecommendationsk
Decemberk2011 BSBkformalkconsideration
Decemberk2011k–kFebruaryk2012k Furtherkconsultationkperiod
ItkwaskdeterminedkthatkthektestkwouldkbekrunkbykPearsonkVue,kavailablektokbektakenkatkanykofkth
eirktestingkcentres,kofkwhichktherekarekoverk150kinkthekUKkandkotherskink165kcountrieskworldwid
e.kThekBSBkalsokdulykengagedkankindependentkconsultantkwhokwaskemployedktokassesskwhet
herkthekchosenktest waskappropriatekforkusekandkalsokcontractedkwithkPearsonkVuektokrunktwokp
k
ilotskofkthektest,kaskdetailedkabove.k
1.2.1 StandardskandkFailurekrateskonkthekBarkCoursek(BVC)
ThekWorkingkGroupkreportedkthatkthekexistingkentrykrequirementskhadkbeenkfoundktokbekinsuffi
cientkinktheirkpresentkformktokmaintainktheknecessarykstandardskonkentrykforkthekBarkCourse:k“T
hekstudentkbodykincludeskgraduateskwhokareksokfarklackingkinkthekqualitieskneededkforksuccessf
ulkpracticekatkthekBar....kthatktheykwouldkneverkobtainkpupillage,khoweverkmanykpupillageskwere
kavailable.”kThiskiskbornekoutkbykstatisticalkevidencekof klowkstandardskonkthekcourse.kInkfact,kfork
Pagek4kofk138
