Microsoft word - auditool.doc

AUDIT ASSESSMENT SHEET FOR TRAINERS & REGISTRARS Drs. Ramesh Mehay & Maggie Eisner
(based on 8-point COGPED criteria) Bradford VTS Course Organisers

Reproduced by kind permission of Drs Ramesh Mehay & Maggie Eisner, VTS Course Organisers in Bradford. This excellent Audit Assessment Sheet is a development of the COGPED 8 point scale audit marking schedule used by Summative Assessors. It was developed by Drs Ramesh Mehay & Maggie Eisner, VTS Course Organisers in Bradford. They developed this tool so that GP Registrars – and their trainers - can assess their own audits. Not all the box points in this tool are on the COGPED marking schedule. Drs Mehay and Eisner reviewed other guidance on audit and derived a consensus amongst colleagues in Bradford to write the document. Michael Harris Associate Director for General Practice, Bath 16 October 2003 PDF created with FinePrint pdfFactory trial version AUDIT ASSESSMENT SHEET FOR TRAINERS & REGISTRARS Drs. Ramesh Mehay & Maggie Eisner
(based on 8-point COGPED criteria) Bradford VTS Course Organisers

The audit report must satisfactorily cover all main areas to pass.
Title PASS/FAIL
Often useful to phrase the title as a question derived from the criterion eg “Have all our diabetics had a HBA1C in the last 3 months?” Reason For Choice of Audit PASS/FAIL
Should indicate potential for change Team members agreeing on audit area Relevant to General Practice (particularly important for those doing an audit while in SHO posts) You need to state why your chosen subject is important, and then relate it to practice level and your personal experience. If your audit results from a problem you've noticed in the practice, document it in your write-up; it suggests that there is a potential for change. Criterion/Criteria Chosen PASS/FAIL
Criterion clearly stated and unambiguous Relevant to audit subject Has criterion been justified (ie evidence based), eg. current literature At least two references (preferably more) to justify criterion References properly quoted? Not auditing too many criteria (1 or 2 only) ie criteria not complicated or over-ambitious? Does the statement fit the following phrase: “Patients with xxxx should have a xxxxx every xx years” More than one criterion which the registrar needs to set out separately and clearly A criterion is a hallmark of good practice ( a gold standard) Ensure that the criterion is measurable – “asthmatics should have yearly PFs” is difficult to measure (how many years will you go back?); so re-phrase as “asthmatics should have a PF recorded in the past year” is more practical. For quickest results, make sure that what you are doing is fairly easy to measure, e.g.’ is Read-coded, though don’t let other data-gathering methods put you off if you you’re really interested in the subject • Think about how reliably al the points in your criteria are coded. In the second example above, how confident can you be that all your asthmatics are coded? You may need to do some cross-checking – for example, searching for patients using inhalers who aren’t coded as being asthmatic. PDF created with FinePrint pdfFactory trial version AUDIT ASSESSMENT SHEET FOR TRAINERS & REGISTRARS Drs. Ramesh Mehay & Maggie Eisner
(based on 8-point COGPED criteria) Bradford VTS Course Organisers

Standards Set PASS/FAIL
Targets towards a standard with a suitable time scale?
Standard set for EACH criterion
The standard follows on directly from the criterion – for example,
“Patients on thyroxine should have had TFTs done in the past year; this should have happened in at least 90% of patients”. Standard derived from reasonable sources (hierarchy of evidence : research, national guidelines, hot topics in reputed journals, local guidelines ,discussion with consultants and trainer/partners, PCT/PMS targets, NICE guidelines) – ie how you decided on the figure say 85% and relate to local circumstances The standard reflects the clinical and medico-legal significance of the criterion (ie does the level set seem reasonable to the issue under investigation?) There is more than one standard which the registrar needs to set out separately and clearly A standard is a target which should be realistic and achievable NOT the ideal (which is the criterion)
Some criteria are so important that they need 100% standard (but rare!). 100% standards are unusual – patients or circumstances usually conspire against perfection and the standard needs to reflect that. If a certain practise is being carried out really badly, then you could start with a standard of 50% in the interests of realism (although a higher standard should be aimed for in the long term). Your literature search should give you an idea of what standards others have managed to reach. Make sure that you can justify your standard. If you can't find any literature evidence to back it up, explain why you chose the percentage that you did. Preparation and Planning PASS/FAIL
Evidence of teamwork - This may include the GPs and Nurses who will have to implement any changes, as well as office staff who can help you doing a computer search. Again, you will need to document this. Evidence of delegation Adequate discussion where appropriate Mentioning the use of : computer registers , review of contents of medical records, questionnaires – patients, staff or GPs, data collection sheets , Read codes COMMENTS : Hot Tips Try and include a timetable for planning events/deadlines What Read codes did you use? (note – Read code – capital R after Dr. J Read who founded them) PDF created with FinePrint pdfFactory trial version AUDIT ASSESSMENT SHEET FOR TRAINERS & REGISTRARS Drs. Ramesh Mehay & Maggie Eisner
(based on 8-point COGPED criteria) Bradford VTS Course Organisers

Data Collection 1 PASS/FAIL
All patients should be seen within 15 minutes of their Use of simple graphs/charts – results set out clearly? Graphs/charts grouped for each criterion (ie for clarity) Numbers add up? (Statistics appropriate) Reasons cited as to why standards not met? Do they seem reasonable? Compared to standard? – standard must be included COMMENTS : Hot Tips Reasons why standards not met : think in terms of - Practice reasons: Results having been put down as free text on computer, rather than coded; · Opportunistic rather than formal recall system in use; - Doctor reasons: Not all GPs were aware of the practice policy; · Not all partners agreed with the policy; - Patient reasons: Patients refusing to have tests done; · Patients on holiday when tests due You may need to use flow diagrams to illustrate results clearly and effectively eg for an audit regarding proton pump inhibitor prescribing, results could be displayed as Maintenance (15mg daily) High Dose PPI Documented Reasoning No Documented Reason 1 benign stricture 5 Barretts oesophagus 17 refused to try BUT YOU WOULD STILL NEED A GRAPH/TABLE TO COMPARE THE RESULTS TO THE STANDARD.
Changes to be evaluated PASS/FAIL
(= Implementing Change) = the most difficult part of audit
Emphasise what has been achieved and any deficiencies How can we correct any deficiencies? - Actual examples described Changes must be practical! (ie how are you actually going to make the changes?)
For each example described
what needs to be done who’s going to do it when and how PDF created with FinePrint pdfFactory trial version AUDIT ASSESSMENT SHEET FOR TRAINERS & REGISTRARS Drs. Ramesh Mehay & Maggie Eisner
(based on 8-point COGPED criteria) Bradford VTS Course Organisers

Consider bold-type to highlight each change.
Just telling people to do things better won't result in change. You need to write up in some detail how the changes will take place. Try and state who will be doing what. Try and include a time table of events/deadlines eg “all to be achieved before the next audit in 6 months time” Example: "The GPs agreed to do a serum rhubarb on any patient that they see who is on Viagra" - fail - there is no system to help them remember. "(a) The GPs were given a prompt card that they could stick on their computer screen as a reminder to do a serum rhubarb on any patient that they see who is on Viagra; (b) the secretary will search every three months for patients who are overdue for their serum rhubarb, and flag it as an active problem on the computer system" - pass Data Collection 2 PASS/FAIL
Compared with data collection 1 Compared with standard – standard must be included for comparison! Results set out clearly? Graphs/charts need to be re-grouped for each criterion (ie for clarity) Discussion of results were standards met the second time round reasons if still not met. Possible sources of bias. “The effects can be clearly seen” is a non statement. State the effects. Beware – too many registrars include in the discussion what should be in the conclusion. Conclusions PASS/FAIL
Summary of main issues learned Comment on any improvements that have resulted. How well did your proposals for change work? Factors enhancing and impeding change (?whole team approach, partner resistance) If you again didn't reach the standard that you set, why not? Future suggested method(s) of improving – must be specific. Possible sources of bias. If you did, should you be aiming higher next time, or look at something else e.g. whether abnormal serum rhubarbs have actually been acted on? Where should the practice go from here? Mention of re-auditing? “This audit highlighted an important clinical and educational area” is again a non statement! “This audit cycle has proved a successful tool in the implementation of change” is a bland statement. You need to explain how. NICE and other guidelines are very hard to implement. Implementing change takes a lot of time and effort and needs a committed lead person. • A whole team approach may be the key to enhancing that change. PDF created with FinePrint pdfFactory trial version AUDIT ASSESSMENT SHEET FOR TRAINERS & REGISTRARS Drs. Ramesh Mehay & Maggie Eisner
(based on 8-point COGPED criteria) Bradford VTS Course Organisers

References PASS/FAIL
Should be evidence based Peer reviewed respectable journals References are properly quoted (authors, year, journal/book, volume, pages etc) Minimum 6 references COMMENTS :
OTHER COMMENTS (Tick active points registrar needs to consider, delete comments which do not apply)
Registrar needs to restate criteria & reasons for choice Registrar needs to restate standards & reason for choice Registrar needs to use subheadings which are identical to marking criteria on the schedule and Registrar needs to smarten up their writing style Shorter & Clearer – ie registrar uses a lot of words to say very little! Active sentences using ordinary language Are appendices referred to in the main text – ie they have not been added just to add bulk to the project!
List other specific/general comments not covered in the headings above :

Hot Tips (SENTENCE CONSTRUCTION)
WRITING STYLE
Try to use ordinary language when you can. Use short words eg start rather than “commence” Read sentences out loud to yourself to make sure they are clear and sensible. Try to make each point only once, and make sure you actually have a point to make! PASSIVE & ACTIVE SENTENCES – “These patients had only paper notes available. These were individually read through to cover the previous 12 months” is an example of a passive sentence which reduces readability significantly. “I read through these individual y” is better. Another example : “I chose to…” not “it was chosen to ….” SIMPLIFYING SENTENCES – “One year was chosen to achieve these standards as it allowed the practice to implement change and discuss and allow reorganisation within the practice. It was also a good time frame to judge the effectiveness of the changes that were implemented” can be simplified to: “One year was chosen to achieve these standards as it gave time for the practice to implement change and to judge their effectiveness” – easy peasy lemon squeezy! APOSTROPHES – remember, plurals do not need apostrophes eg proton pump inhibitors NOT proton pump inhibitor’s ABBREVIATIONS - the very first time you use them, write them in full with the abbreviation in parenthesis – like journals do. LAYOUT
•
Use bold type discreetly to highlight important words or phrases. Do not bold type big chunks of paragraphs Use bullet points to clarify separate issues USE OF REFERENCES
Back assertions with references whenever possible Mention names of authors & what kind of study it was (“In a review article of the use of PPIs in the UK general practice in 1999, Bloggs & Doodah showed that…….”)
Sources
8-point COGPED criteria (NOSA) – www.nosa.org.uk
Bath Dept of General Practice - http://www.mharris.eurobell.co.uk
PDF created with FinePrint pdfFactory trial version

Source: http://gppro.co.uk/resource/audit/auditool.pdf