A Formative Assessment Toolkit:

Initial LNA:

In the new world of Specialist Training should we have new methods of Learning Needs Assessment? Pippa Stables reviews some new and old assessment tools:

The first step hasn’t changed. Make time early on to get to know the Specialist Trainee, their aspirations and motivations, their background and home situation, and their medical education and experience so far. Now that doctors often don’t have CVs, constructing an Education Time Line can be useful, and reveal the gaps.

For an up to date curriculum list and confidence rating scales consider using the ‘Condensed Curriculum Guide’  available from http://www.rcgp.org.uk/acatalog/GP_Curriculum_Essentials.html

Although it is long, it is complete. You may need to tackle it in smaller chunks, starting with areas the ST thinks they need to review. There is space to write a 1 to 5 confidence rating. We all know that this is personality dependant, but it does help to prioritise. The book also offers many up to date learning resources, mainly on line.

If you like to use MCQs, consider the college AKT trial questions: http://www.rcgp.org.uk/the_gp_journey/nmrcgp/akt/sample_questions.aspx
There are no correct answers given. You could construct your own answer sheet.

The RCGP nPEP offers 150 questions that are MRCGP AKT in style. They can be purchased for £75 for an associate in training, £100 for a college member. http://www.rcgp.org.uk/councils__faculties/rcgp_scotland/products__services/pep_ekit.aspx

The Cornwall Trainers MCQ is remarkably not out of date as it tests quite ‘traditional knowledge’ rather than recent guidelines.

Consider using a COT and a CbD very early on, to get a baseline measurement

And other time honoured tools:

Learning styles assessment can be a useful component of the initial getting to know each other exercise, enabling conversations about learning method preferences.

Learning Styles questionnaire to buy your own copy of this questionnaire go to the Peter honey website, link below, which has many useful resources at a price. http://www.peterhoney.com/
VARK guide to learning styles (Visual, Aural, Reading/writing, Kinesthetic)
Also, see BMJ article on Learning Needs Analysis

Older curriculum lists are more succinct, but not ‘the real thing’ and dated in places.
Cornwall Trainers Trainee Initial Self-Assessment List (on their website)
Northumbria Vocational Training Scheme Rating Scale (on their website)

Regular reviews:

A review should have the following stages:

Agree the agenda, the tasks to be completed and the time available

Ask if there are any burning issues (such as a complaint)

Ask some open questions to gauge how the registrar is feeling

Discuss the main areas of work and learning- surgeries, OOH, tutorials, day release etc

Look at the PDP and summary of evidence, discussing issues as they arise

Offer feedback and agree a plan.

Update the PDP

The Educational Supervisor’s 6 monthly reviews on the e portfolio are checked by a Deanery panel, and the minimum assessment requirements can be found on:

http://www.rcgp.org.uk/the_gp_journey/nmrcgp/wpba_and_eportfolio/minimum_evidence.aspx

The 6 monthly meeting does require a lot of box ticking, which can detract from the process. Make sure you have read the summary pages, all the assessments and the PDP  in advance, and make time before or afterwards to fill out the bulk of the boxes, so that you can ask and listen at the review. The emphasis is not on collecting massive of detailed ‘evidence’ across the curriculum, but on your impressions about the learner’s progress, performance and planning.

Formative assessment and feedback is a continuous process. The assessment tools, COT and CbD can be spread over the 6 months. Additional assessment tools may be used according to the trainees particular needs, but the older ones are probably not necessary now and it is better to become familiar with the newer tools.

Older assessment tools:

Modified Manchester Rating Scales (available on London Deanery website within formative assessment toolbox) I find the majority of registrars score so well initially that it is less useful these days.
Barnet Assessment Form
Mid-year report (adopted by some Bristol trainers)

Week to week:

Weekly documentation of reflection on the tutorial, and mutual feedback could be kept on the e portfolio under the curriculum headings. Some would do this together at the end of the tutorial, other STs may prefer to reflect after the event, then flag their notes for sharing with the trainer. If you still prefer pen and paper then the old reflection sheets may have a place, and could still be saved as a template or scanned for attachment. Keeping a hand written reflective diary is an alternative.

Reflection sheets used in the past:

Week ending sheet
Trainer & GPR end of week reflection

Problem Case Analysis and Random Case Analysis remain the back bone of teaching in the practice. CbD is the formal assessment derived from these.

Other ideas for formative assessment include:

PACT: prescribing analysis of the ST3 as a separate prescriber is not currently possible via e PACT. Auditing the learners prescribing currently should be done via RCA.
Referrals (SAIL…Sheffield Assessment Instrument for Letters)
Admission outcomes

PUNS & DENS
Discomfort log (ref: Education for General Practice (2002), 13, 490-6)
Coffee/lunch break discussions
Ethical/moral issues (attitudinal questionnaire)

Feedback from non-trainer individuals:

The Multi Source Feedback and Patient Satisfaction questionnaire derived from the e portfolio have succeeded older tools. I have not removed the old ones from the website because you may use them in other contexts such as practice nurse training.

Staff feedback
Well Close Square Consultation Satisfaction questionnaire
Partners questionnaire
Surgery staff questionnaire (eg STAR…Staff Trainee Assessment Rating Scale, available from Well Close Sq website)
PCHT staff questionnaire
Patients satisfaction questionnaires eg DISQ…Doctors Interpersonal Skills Questionnaire, originally formulated by Australian GP dept, ref: Education for General Practice (1999), 10, 256-264)
Small group facilitator
OOH clinical supervisor
Informal feedback log (odd comments to trainer eg from pt, reception)

Other:

Well Close Square Tutorial Assessment This one may be useful to do together to facilitate a discussion about tutorial usefulness
Tutorial Appraisal Form This is a simple registrar to trainer tutorial feedback form

Complaints/significant events

Useful Websites:

http://www.gp-training.net

http://gppro.co.uk/vts/index.htm

www.cornwalltrainers.co.uk