Quality Assurance

In ISAAC Phases One and Three, tools were developed to assist the ISAAC International Data Centre (IIDC) Research Manager to undertake quality assurance processes.

In ISAAC Phase One, to enable centre methodology to be checked, the Steering Committee developed a five page centre report [Phase One Centre Report]. This was sent to the Principal Investigators (PI’s) when they submitted their centre data to the IIDC which they completed and sent back. This documented aspects of the fieldwork and centre methodology, which were considered important to record and enabled checks to be made against aspects of the data. Close communication with the PI’s was vital whilst undertaking the checks.

When the ISAAC Phase One data and methodology checks had been completed, the centre report was evaluated by several Steering Committee members to ensure it was suitable for use in Phase Three, particularly for those with English as a second language. The evaluation identified that some areas of the report were difficult to interpret. Subsequently the report was redesigned for use in Phase Three. The Phase Three centre report retained the same information but simplified the questions and in some cases a single question was changed and became several questions to ensure its meaning would be understood. Collaborators found this new Phase Three centre report an easier document to complete [Phase Three Centre Report].

In addition, this report was sent to the PI’s at the time they registered, so that they could complete it when the fieldwork was being undertaken rather than completing it retrospectively as in Phase One. For the centres that were new to Phase Three, the centre report enabled checks to be made against the data as in Phase One. For the Phase Three centres that had also completed Phase One, the Phase Three centre report was checked against the Phase One centre report to ensure PI’s had used the same methodology as in Phase One. All deviations between Phase One and Three were documented and these were categorised: major deviations (centres excluded from the analyses); minor deviations (deviations identified by the use of footnotes in the published tables) and; very minor deviations (deviations accepted and not identified in the publication tables).

This information has been collated and a manuscript on “The challenge in replicating the methodology between Phase One and Three of ISAAC” will be submitted for publication in April 2011. From the 112 centre reports for the adolescent group (13-14 year olds) and 70 for the children (6-7 year olds)  that were submitted, six centres for the adolescent group and four for the children had major deviations and were excluded. There were 35 minor deviations for the adolescents and 20 for the children which were identified in the publications by the use of a footnote and there were 92 very minor deviation for the adolescents and 51 for the children that were accepted and not identified. We also found that a change in PI between phases did not adversely affect the methodology (odds ratios 0.80 [95% CI 0.36, 1.81] for adolescents and 0.91 [95% CI 0.32, 2.62] for children).

We concluded that with attention to detail and careful recording of methodology, repeated, cross-sectional, epidemiological multicentre studies using the same methodology such as Phases One and Three in ISAAC are feasible and can be achieved throughout the world by people with diverse cultural backgrounds and research experience. The IIDC is very appreciative of the commitment of the ISAAC collaboration to their attention to detail which has produced such a high standard of methodology in Phase One and Phase Three.