The ISAAC Toolbox
Early in the development of the ISAAC study it became apparent that there were no widely accepted tools for international surveys of asthma prevalence in children. Although many questionnaires and measurement tools had been used successfully for epidemiological studies within a single centre, they were often heavily dependent upon the local patterns of clinical diagnosis and were therefore poorly suited for international comparisons. From the beginning, therefore, ISAAC developed its own epidemiological instruments, initially for asthma, but soon extending to questionnaires for other allergic conditions, and to objective measurements of bronchial hyperresponsiveness, allergic sensitisation and atopic eczema. This section charts the development of these tools, which are now widely used both within ISAAC and by other investigators.
The written questionnaire for asthma and wheezing
(David Strachan, Ross Anderson, Colin Robertson)During 1990, an informal meeting was held at St George’s Hospital Medical School, London, between Ross Anderson, David Strachan and Colin Robertson. We discussed ideas for developing an asthma questionnaire which would include measures of severity which were relevant in children. Colin’s interest and expertise related to the long standing follow-up study of wheezy children in Melbourne originated by Howard Williams in the 1960s. We drew on ideas from the surveys that had been completed in Croydon, south London, during 1978 and which we planned to repeat in 1991 to obtain information on time trends in our area.
Two key concepts guided the development of specific questions. Firstly, the principle of using symptoms rather than diagnosis for international comparisons, and secondly, the recognition that there were several dimensions to asthma severity: frequency of attacks, intensity of attacks, and persistence of symptoms. Thus, a child could have only one or two attacks in a year, but if these were intense enough to result in admission to hospital, that would be considered severe. On the other hand, a child might be persistently troubled by wheezing but they or their parent might not recognise discrete episodes or attacks of symptoms.
At this stage, we had very little independent validation of specific questions, so there was a natural tendency to promote questions that we had used in our own surveys. Frequency of attacks of wheezing had been used in surveys in Auckland, and speech-limiting wheeze and sleep disturbance due to wheezing had been included in the Croydon surveys. The question about wheeze that caused difficulty with speaking had been validated in terms of its correlation with hospital admissions for asthma among 8-9-year-olds in Croydon.
It is important to note that the questions were selected because of prior experience with their use in English-speaking populations, with children mainly of primary-school age and with responses by the parent rather than by the child. Their adoption within the ISAAC core questionnaires led to their use in teenage children (self-reporting their own symptoms) and translation into many languages. Uncertainties about how reliably the written questions would perform in a different age group and in different languages and cultures led to the interest in developing a video questionnaire for administration alongside the written one. This is described in the next section.
The video questionnaire for asthma and wheezing
(Julian Crane)The idea for the video questionnaire and for using video sequences in questionnaires came from my early research in the Tokelau Atolls. We conducted a couple of surveys in Tokelau on 2 out of the 3 atolls and during development of the questionnaires and their translation and back translation from English to Tokelauan and Tokelauan to English I was struck by problems around the term wheezing. It turned out, as in other languages, that the term for wheezing also had many other meanings in Tokelauan including shortness of breath and coughing. This led to quite a lot of difficulty of questionnaire preparation.
When I was actually in Tokelau we conducted our surveys through the schools and as Tokelauan was an entirely oral language and in fact while we were on the atolls was in the process of having its first grammar developed by a Norwegian linguistic anthropological group of researchers (that’s another story), it was evident that in the school environment there were relatively few books but video tape recorders and many video recordings and indeed amongst the general, few books in anyone’s houses but many videos and it was clear that much of the information that Tokelauans collected and learnt came from video recordings. That first gave me the idea that a recording showing wheezing with a good soundtrack might be useful.
We thus started to film asthmatic adults and children initially in the wards and in outpatients and occasionally in Accident and Emergency and started to develop sequences which included exercise, nocturnal coughing and wheezing and then more severe cases and then as you know we developed an initial version in which we did some validation work.
This was a group effort to validate the questions both against the written questions that had been developed for ISAAC and also against airway hyper responsiveness to methacholine. This process was actually led by Robyn Shaw, whom I am sure you will know, now a paediatrician at the Hutt Hospital, who was working with us as a Research Fellow at the time and I think she incorporated some of this into her MD thesis as well as the publications.
We undertook the validation here in schools and subsequently published that. Other people have of course validated various other aspects of the video questionnaire along the way. The video questionnaire, I think, at some stage won the TSANZ poster prize with some confusion as to whether it was a poster or an audiovisual presentation as we presented it on video screens at the poster site.
Subsequently when it was begun to be thought about to use it internationally there were issues of, for example having men and women exercising together which wouldn’t be acceptable in the Islamic community and also that it was very European ‘white’ based. That led us to ask for sequences from various people. We received one from India, some from Chris Lai in Hong Kong and others and put together the final series of sequences that were used in the ISAAC study although as you know the European ISAAC was used first and then later we used the international version with the first sequence retained throughout all videos.
A number of groups around the world have used the video often in conjunction with the written ISAAC questionnaire and I did make two attempts to have funded a much wider development of the video concept given that I think using audiovisual presentation of symptoms could actually be very valuable in many different circumstances where the visual and or audio presentation might be most useful. For example, breathlessness I thought would be one, possibly distinguishing fits and faints and also in the assessment of sleep apnoea and snoring amongst others, however we never managed to achieve funding for this. I think this was a shame as video questionnaires would have been very useful for international studies.
The ISAAC video has been an interesting exercise and has provided some additional novelty in relation to the ISAAC studies.
Asthma alone, or asthma and allergies?
(David Strachan)Originally, the Auckland proposal had focused mainly on asthma, whereas the German group had been asked by their Federal Ministry to develop a study of time trends and determinants of asthma and allergic disorders. A broader perspective was also considered to be relevant to the Auckland grant application, since allergen sensitisation, indoor allergen exposure and infant eczema were putative determinants of asthma occurrence and severity. At the second Bochum meeting, in December 1991, the aims of ISAAC were agreed to include studies of the prevalence and severity of asthma, allergic rhinitis and atopic eczema.
Development of the rhinitis questionnaire
(David Strachan and Bonnie Sibbald)Bonnie Sibbald, who was working at that time as an epidemiologist with Ross Anderson and David Strachan at St George’s, London, had completed a population survey of rhinitis symptoms in south west London, with follow-up interviews and skin prick tests to define seasonal and perennial rhinitis and their relationship to aeroallergen sensitisation. The combination of rhinitis symptoms (sneezing, blocked or runny nose) with itching or watering of the eyes had the best predictive value for skin prick test positivity, and the questionnaire was therefore developed to include all these symptoms.
This was a study of adult respondents and some concerns were expressed about the generalizability of the results to children. However, at that time there was no comparable study of rhinoconjunctivitis in younger children, and it was felt that results for adults could be applied with caution at least to the teenage group. Two further issues which were resolved by discussion at the 1991 workshop were related to seasonality and severity of symptoms.
Whereas hay fever or other forms of seasonal allergic rhinitis have a characteristic monthly pattern within any given country, it was clear that for international comparisons it would be difficult to refer to a specific season without loss of standardisation in the questionnaire. Therefore, it was decided to enquire about the occurrence of rhinitis symptoms in each of the 12 months of the year, and leave the task of defining seasonality to the statistical analysts. In practice, little use has been made of the seasonality responses in the main ISAAC papers, partly because of concerns that they may be biased by the month of fieldwork, and partly because many of the centres who eventually took part in ISAAC were from tropical countries. On reflection, perhaps more use could be made of these data.
As there were no widely accepted measures of severity of rhinitis (allergic or otherwise), there was a lively discussion at the 1991 workshop about how to assess this. Neither sleep disturbance nor night waking were felt to be appropriate, and school absence due to allergic rhinitis was considered to be rare and probably culture-dependent. There was some debate about whether quality of life measures should be included at all, but in the end, it was agreed that a single question on interference with daily activities was justified as a simple severity marker. It was recognised, however, that there were no supporting data, in either adults or children, to validate this choice.
Development of the eczema questionnaire
(David Strachan and Hywel Williams)Hywel Williams, a trainee dermatologist, who was working at that time as a Wellcome Trust Clinical Epidemiology Fellow at St Thomas’s Hospital in London, did not attend the 1991 workshop but had been in close liaison with David Strachan over the preceding year as they collaborated on analysis of eczema data from 1958 British birth cohort study. Hywel was planning to do a study to look at the differences between eczema prevalence between London-born Afro-Caribbean children and those living in Kingston, Jamaica, as the main focus of his PhD, but found a complete absence of valid diagnostic criteria that could be used for epidemiological studies. Prior attempts at developing diagnostic criteria for eczema, such as the Hanifin and Rajka criteria, were based on empirical grounds by a group of experts working in secondary care settings. Whilst many of the proposed criteria had good face validity, many were poorly defined (eg tendency towards infections) and some were uncommon (keratoconus), and trying to determine the presence or absence of 33 minor features in an epidemiological survey was going to be quite impractical. It was this topic that was to become to the main focus of Hywel’s PhD. He set up a UK working party of 16 leading dermatologists to try and identify a minimum list of reliable discriminators for diagnosing a typical case of eczema. The aim was to develop a much smaller list of diagnostic features that could be reliably ascertained by trained nurses in epidemiological studies, rather than criteria that were the exclusive domain of clinician experts in an outpatient setting.
The UK Working Party study included direct inspection of the skin among its diagnostic criteria, and although many aspects of skin examination were tested for in the national case control study, it became clear at an early stage that historical features such as a history of involvement of the skin creases was just as, if not more important than physical signs. Thus, the combination of itchy rash with flexural involvement had both high sensitivity and specificity for atopic eczema in the age range planned for ISAAC, and the specificity could be improved if the age at onset was less than 5 years. Although the age range of this UK study was wider than that proposed for ISAAC, this was the most relevant information available to guide the construction of an ISAAC questionnaire for eczema. The age at onset question was only included for the younger age group (parentally completed questionnaires), as it was considered unreasonable for teenagers to recall the age at which they had first suffered from itchy rash. Although the questions contained in the final set of UK diagnostic criteria for atopic eczema that appeared in1994 were not identical to those used in ISAAC, they were very similar. Around that time, empirical questions for another epidemiological study - the Avon Longitudinal Study of Parents and Children (ALSPAC) had been suggested by Dr. Clive Archer (a dermatologist at Bristol with an interest in eczema) in discussion with Hywel Williams, which included an element of chronicity of flexural symptoms. The establishment of the ALSPAC questionnaire at around the same time influenced the inclusion of the phrase “which was coming and going for at least six months” in relation to an itchy rash.
German allergologists and dermatologists who were present at the 1991 meeting were concerned that there were multiple expressions for “eczema” in German and therefore that translation might prove to be a problem. However, this argued in favour of standardising on a symptom-based definition. There was also concern that scabies could be a common cause of itchy rash in developing countries and had not been included in the UK case control study, a point that was subsequently partially addressed in later validation studies.
Night waking, which had been agreed as a measure of severity for wheezing and asthma, was a natural choice of severity marker for eczema given the frequent sleep disturbance to child and family due to scratching at night. In addition, a single question was included to enquire whether the itchy rash had ever cleared completely in the last year as a possible indicator of less severe or more transient eczema. However, this second question has not been used widely in ISAAC publications, which tend to concentrate on sleep disturbance as a measure of eczema severity.
The 1991 workshop also discussed briefly the development of a skin examination module for inclusion in objective surveys of atopic eczema. Subsequently, the development of the ISAAC Phase Two skin examination protocol followed closely the recommendations of the UK Working Party and further validation work by Hywel Williams, who joined the ISAAC Steering Committee in 1992.
In retrospect, it is reassuring to note the results of ISAAC Phase Two surveys have shown a high correlation between the prevalence of eczema determined by ISAAC core questionnaire and by skin examination, at least in 9-11-year-old children. This suggests that the idea of measuring eczema by questionnaire alone, which was controversial in the early 1990s, has a sound epidemiological basis.
Bronchial responsiveness
(Innes Asher)In Phase Two objective measures related to asthma and allergies were developed. In December 1991 it was decided to have a module on bronchial responsiveness. At this time several methods were being used, and there were pros and cons to each. In December 1993 an ISAAC Airway Responsiveness Satellite Workshop was held in Auckland, New Zealand to choose a method for Phase Two.
Attendees at the workshop were Sandra Anderson (Sydney), Innes Asher (Auckland), Tadd Clayton (Auckland), Julian Crane (Wellington), Philippa Ellwood (Auckland), Ed Mitchell (Auckland), Philip Pattemore (Christchurch), Jenny Peat (Sydney), Colin Robertson (Melbourne), Robyn Shaw (Wellington), Alistair Stewart (Auckland), Erika von Mutius (Munich). Apologies were received from Pierre Ernst (Quebec), Michelle Haby (Sydney), Alison Leversha (Auckland), David Strachan (London).
The meeting developed criteria on which to judge challenge tests. The greatest priority was given to ease of standardisation between centres, countries and cultures, reproducibility, safety, specificity, and availability of dose response information. Of a lesser priority were acceptance by parents, feasibility in all participating countries, costs, co-operation by children, state of knowledge concerning tests, and legality in participating countries.
The methods considered were cold air, PEFR variability, exercise challenge, hyperosmolar challenge, methacholine, and histamine.
Cold air challenge (-15 degrees C), 5% CO2 did not result in a dose-response, and equipment costs were high. Isocapnic hyperventilation was replacing this kind of test, but still no dose response.
Histamine was illegal for use with humans in Australia. There were probably insurmountable regulatory obstacles to the use of histamine and methacholine in New Zealand, USA and Germany.
PEFR variability had a number of disadvantages relating to effort, and use of asthma drugs.
Exercise challenge. There is a large experience with this test, which met most of the criteria. However the response to exercise challenge would be altered by variations in ambient humidity It is the absolute water content of the air that is the concern, given that a water content of >10mg/L will eliminate the exercise response.
The hyperosmolar challenge using hypertonic saline. This acts by rapidly altering the osmolarity of the airway surface liquid which causes mast cells to degranulate.A good test for current asthma. In school studies in Melbourne this test compared well with an exercise test. The ultrasonic nebulisation was not ideal, as time consuming and a bit cumbersome.
There was considerable discussion on the pros and cons of each method as a tool for ISAAC. The preferred option recommended was hypertonic (4.5%) saline challenge. The exercise test according to the Haby method was ‘highly recommended’. If histamine or methacholine challenges were to be used for ISAAC then the Yan method was preferred.
Post script 2011: In future epidemiological studies dry powder mannitol would be a consideration – it is an hyperosmolar challenge which is safe and reliable, with individual capsules for dry powder inhalation. The only equipment needed is a laptop spirometer, the dry powder device and the capsules.
Measuring eczema by questionnaire and examination
(Hywel Williams)Whilst questionnaires were attractive in their ease of application and low cost for making broad prevalence comparisons involving many thousands of children across many countries, concerns arose about the complete reliance on questionnaires due to potential problems in translating or understanding the concepts contained in the questions. Whilst many elements of the questions had been shown to have reasonable sensitivity and specificity when validated in UK community settings, it was possible that their performance was less good in other countries, where other itchy conditions such as scabies was more prevalent in the ages studied. ISAAC Phase Two, with its child contact module, therefore provided an ideal opportunity to examine the skin in a standardised manner.
One of the advantages of skin disease is that it is immediately visible - it is either present or not present, although visible eczema at one point in time in a condition that classically comes and goes will only provide a point prevalence which is typically lower than a 1-year period prevalence. By the time the ISAAC Phase Two modules were being developed, the UK Working Party’s refinement of the Hanifin and Rajka diagnostic criteria for atopic eczema was completed in terms of development and community validation. The UK criteria included the one physical sign of “visible flexural dermatitis” which had been carefully constructed to only include those flexural sites that were helpful discriminators from other skin diseases such as contact dermatitis or lichen planus or scabies.
For the first time ever, a photographic protocol defining what eczema looks like, which sites could be deemed as being flexural at different ages, and what size or pattern of involvement could be deemed as a positive response was developed. The visible flexural dermatitis protocol was shown to be a reliable tool when used by trained nurses, and a set of training photographs and a set of quality control photographs were also developed for use in the public domain. Given concerns about the validity of questionnaire-ascertained eczema, it could be argued that the visible flexural dermatitis protocol used in ISAAC Phase Two offers the most objective method for standardising the eczema phenotype for a host of scientific studies given its freedom form language and or interpretation bias.
Development of Phase Two modules
(David Strachan)The concept of developing objective tests for asthma and allergy was discussed in outline at the first Bochum workshop (December 1990) and at the second workshop, one year later, it was decided that a series of self-contained “modules” would be desirable. Bronchial hyper-responsiveness, serum IgE, skin prick tests, skin examination, and a questionnaire about medications & health services were discussed at this time. The following year, at the Steering Committee in London, the list was extended to include supplementary questionnaires about risk factors and other respiratory symptoms.
Work on the modules continued through 1993 to 1995, including a special workshop on bronchial responsiveness testing, described in a separate section. As much of the attention within ISAAC at this time was on the expansion of Phase One, the main stimulus to finalise the measurement protocols came from the Germans (Stephan Weiland and Erika von Mutius) who planned comparative studies of 9-11-year-old children in former West Germany (Munich) and former East Germany (Leipzig and Dresden) starting in 1995.
Although the risk factor questionnaire underwent some changes after the German fieldwork, the protocols were essentially fixed at this time, and after their successful implementation in Germany the modules were printed as a bound volume in 1998. The measurement modules were summarised in the Phase Two methods paper, published in 2004. By the time of that publication, 30 centres in 22 centres had used combinations of the modules and contributed data to the ISAAC Phase Two Data Centre, which moved with Stephan Weiland from Münster to Ulm in 2002.
The Phase Three risk factor questionnaire
(Ed Mitchell)The aim of the Phase Three risk factor questionnaire (often referred to as the "Environmental questionnaire") was to achieve individual level data which would be novel because of the scope of the questions and the scope of Phase Three. A list of variables of potential interest was developed in June 2000. This list included variables that could not be obtained by questionnaire, but were included to ensure that the item was explicitly considered. The main focus of the questionnaire was on the 6-7 year age group (completed by parents) with a briefer version for the 13-14 year age group (self completed).
The list of variables and the potential questions was discussed at the ISAAC Steering Committee meeting in October 2000 in Auckland. It was decided that a core questionnaire would be developed. Each question was developed to address a specific hypothesis. This was led by Ed Mitchell with Neil Pearce, Ross Anderson and Colin Robertson.
A month later the document "Environmental Questionnaire Instructions and Hypotheses" was produced for each age group (view 6-7 year document | view 13-14 year Document). The format was to provide a referenced argument for the specific hypothesis, the source of the question used (in many cases the question was developed by the Steering Committee), and a brief outline of the analysis (key outcome variable and potential confounders).
The questionnaire was produced in record time and was used by some centres early in the following year.