- Research article
- Open Access
- Open Peer Review
Test-retest repeatability of child’s respiratory symptoms and perceived indoor air quality – comparing self- and parent-administered questionnaires
BMC Pulmonary Medicine volume 18, Article number: 32 (2018)
Questionnaires can be used to assess perceived indoor air quality and symptoms in schools. Questionnaires for primary school aged children have traditionally been parent-administered, but self-administered questionnaires would be easier to administer and may yield as good, if not better, information. Our aim was to compare the repeatability of self- and parent-administered indoor air questionnaires designed for primary school aged pupils.
Indoor air questionnaire with questions on child’s symptoms and perceived indoor air quality in schools was sent to parents of pupils aged 7–12 years in two schools and again after two weeks. Slightly modified version of the questionnaire was administered to pupils aged 9–12 years in another two schools and repeated after a week. 351 (52%) parents and 319 pupils (86%) answered both the first and the second questionnaire. Test-retest repeatability was assessed with intra-class correlation (ICC) and Cohen’s kappa coefficients (k).
Test-retest repeatability was generally between 0.4–0.7 (ICC; k) in both self- and parent-administered questionnaire. In majority of the questions on symptoms and perceived indoor air quality test-retest repeatability was at the same level or slightly better in self-administered compared to parent-administered questionnaire. Agreement of self- and parent administered questionnaires was generally < 0.4 (ICC; k) in reported symptoms and 0.4–0.6 (ICC; k) in perceived indoor air quality.
Children aged 9–12 years can give as, or even more, repeatable information about their respiratory symptoms and perceived indoor air quality than their parents. Therefore, it may be possible to use self-administered questionnaires in future studies also with children.
Indoor air quality problems can be caused by inadequate ventilation, unpleasant room temperature, dampness and moisture damages , gaseous indoor air pollutants and particulate matter . At group level, for instance at work places or schools, indoor air quality problems can be assessed with indoor air questionnaires. These questionnaires typically contain questions about perceived symptoms and indoor air quality [3, 4]. However, questionnaires alone cannot verify the source or presence of indoor air problems . Questionnaires can, however, point to the cause of the problem, for instance high room temperature or problems in ventilation, and therefore can be used as an aid in the problem management accompanied by adequate building and structural inspections. Indoor air questionnaires used today are validated with adult workers, but not with children  and are intended to be parent-administered when for example used among pupils in schools.
Questionnaires concerning child respiratory symptoms have traditionally relied on parent-report. These questionnaires have been shown to have fairly good repeatability e.g. [5,6,7]. However, in children’s health related quality of life questionnaires (HRQL) self-report has been increasingly used . Furthermore, it has been suggested that children aged 6–11 years have basic understanding about concepts of health and have also capacity to associate health to environmental factors . Questionnaires are easier to administer to pupils in schools than to their parents, which often leads to better response rates.
To our knowledge, repeatability of perceived symptoms and indoor air quality collected with self-administered questionnaires from primary school aged children has not been studied earlier. We therefore assessed the test- retest repeatability of two types of questionnaires, self- and parent-administered, on reported symptoms and perceived indoor air quality among primary school children and their parents. If it would be possible to collect reliable and repeatable information about perceived symptoms and environmental quality from the children themselves, self-administered questionnaires could be more widely used in epidemiologic studies also in younger children.
Study population and data collection
Present study is part of a larger study on indoor air, which was conducted in ten primary schools (age 7–12 years) in five municipalities in Finland between March 17 and April 4, 2014. All 71 regional environmental health units in Finland were approached via e-mail, asking for willingness to participate in the study and for suggestions on schools with current indoor air problems and control schools without indoor air problems. Of twelve suggestions received, five case-control school pairs were included in this study. More detailed criteria for selection of the schools are descripted previously . A parent-administered questionnaire on indoor air quality was sent to all parents of the pupils in the ten schools. The research plan was approved by the Institutional Review Board (IRB) of National Institute for Health and Welfare (THL) (IRB 00007085, FWA 00014588). The study participation was voluntary and parents of the pupils were able to deny participation in advance.
Test-retest repeatability of parent-administered questionnaire was studied with a two week interval in two schools (case and control) and 351 parents (52%) returned both questionnaires. Test-retest repeatability of self-administered questionnaire was studied only among 3rd to 6th graders (age 9–12 years) with a one week interval in another two schools (case and control) and 319 pupils (86%) answered both the first and the second questionnaire. In the schools, which repeatability of self-report was tested, we had data at the baseline also from a parent -administered questionnaire, which made it possible to study also agreement between the self- and parent-administered questionnaires. However, these questionnaires were not administered exactly at the same time, as the interval between questionnaires was about 2 weeks.
Parent-administered indoor air questionnaire for primary school pupils (7–12 years) was developed to study the prevalence of self-reported symptoms and perceived indoor air quality at school. The questionnaire included two sections on symptoms in the last four weeks, categorized as respiratory symptoms (8 items) and other symptoms (17 items). All items had four response options (no; yes, sometimes; yes, every week; yes, almost every day). Every item was followed by the question “if yes, does the symptom get worse at school” with response options “no”, “yes” and “don’t know”. Symptoms were selected based on previous indoor air questionnaires used in Finland (National institute for Health and Welfare and Finnish Institute of Occupational Health) and Sweden (MM-40). The questionnaire included also questions about perceived indoor air quality at school in the last four weeks (14 items). All items had five response options (never; yes, sometimes; yes, every week; yes, almost every day; don’t know). Prevalence of asthma in the study population was determined with the question “Do your child have had following diseases in the past 12 months? Asthma?” with response options “no” and “yes”. The parents were instructed to fill in the questions on symptoms together with the child, and 89% of them reported doing so for some or most of the symptoms.
Self-administered indoor air questionnaire was slightly modified version from parent-administered questionnaire. The questionnaire included respiratory symptoms (5 items) and other symptoms (7 items) in the last two weeks. All items had three response options (no; yes, sometimes; yes, almost every day) and every item was followed by the question “Does the symptom get worse at school” with response options “no/no symptom”, “yes” and “don’t know”. The questionnaire included also questions about perceived indoor air quality and other environmental characteristics at school in the last two weeks (7 items). All items had four response options (no; yes, sometimes; yes, almost every day; don’t know). Prevalence of asthma in the study population was determined with the question “Do you have asthma?” with response options “no”, “yes” and “don’t know”. Asked symptoms and environmental characteristics in schools used in both parent- and self-administered questionnaire are show in Figs. 1 and 2.
Intra-class correlation (ICC) and Cohen’s kappa coefficients (k) and percentages of total, negative and positive agreement  were used to assess test-retest repeatability of self- and parent- administered questionnaire and agreement between self- and parent-administered questionnaires. As the response categories in self- and parent-administered questionnaire varied, the parent-administered answers were categorized to 3 classes (“no”, “yes, sometimes” or “yes, every week” and “yes almost every day”) for these analyses, so that the number of categories would be the same. To calculate Cohen’s kappa coefficients and percentages of total, negative and positive agreement, responses were dichotomized to “no” or “yes” (includes all of the different “yes” answer options). All of the analyses were done with SPSS.
Test-retest repeatability of parent-administered questionnaire was assessed with 351 pupils aged 7–12 years and from these pupils 49% were boys and 51% girls. Similarly, test-retest repeatability of self-administered questionnaire was assessed with 319 pupils aged 9–12 years and from these pupils 44% were boys and 56% girls. Prevalence of the symptoms, perceived indoor air quality and missing information in these two populations are show in Additional file 1: Table S1 and S2. Prevalence of missing information in symptoms and perceived indoor air quality was low, generally around 1 %, and highest percentage of missing information observed was 2.7% (wheezing in self-administered questionnaire). Highest prevalence of don’t know answers was 13.5% in self-administered questionnaire (dust and dirt) and 7.5% (unpleasant odor) in parent-administered questionnaire, but generally lower in both self- and parent- administered questionnaires. Missing information and don’t know answers were excluded from further analysis.
Repeatability of the self-administered questionnaire
Test-retest repeatability of questions on symptoms and perceived indoor air quality in schools from self-administered questionnaires were generally within 0.4–0.7 (ICC, k) (Figs. 1 and 2). Only question with repeatability below 0.4 was temperature over 37.5 C in past 2 weeks (0.33 ICC; k). For sensitivity analysis, we stratified the analysis by pupil’s grade in schools, but no major differences in test-retest repeatability in self-administered questionnaire for symptoms and perceived indoor air quality between the younger or older pupils were observed (data not shown). Percentages of observed total agreement and proportions of negative and positive agreement of question from self-administered questionnaire are shown in Additional file 1: Table S3.
Repeatability of the parent-administered questionnaire
Test-retest repeatability of reported symptoms and perceived indoor air quality in schools from parent-administered questionnaires were generally at the same level or slightly lower than in self-administered questionnaire and generally was within 0.4–0.7 (ICC, k) in parent-administered questionnaire also (Figs. 1 and 2). The questions with test-retest repeatability below 0.4 were runny nose (0.38 ICC; 0.36 k), hoarseness (0.35 ICC; 0.33 k), and cough (0.47 ICC; 0.39 k). Only symptom where test-retest repeatability was distinctly higher in parent-administered questionnaire was wheezing (0.68 vs. 0.48 ICC; 0.64 vs. 0.42 k). Percentages of observed total agreement and proportions of negative and positive agreement of question from parent-administered questionnaire are shown in Additional file 1: Table S4.
Agreement between the self- and parent administered questionnaires
Agreement between self- and parent administered questionnaires at baseline was generally within 0.2–0.4 (ICC; k) in all reported symptoms expect headache (0.51 ICC; 0.45 k) (Fig. 3). Compared to reported symptoms, agreement between parent- and self-administered questionnaires in perceived indoor air quality was higher, as the intra-class correlation was between 0.47–0.60 and Cohen’s kappa coefficient between 0.38–0.58 (Fig. 4). Percentages of observed total agreement and proportions of negative and positive agreement of question between self- and parent administered questionnaires are shown in Additional file 1: Table S5.
Prevalence of asthma was 8.8% in both self-administered questionnaires and Cohen’s kappa coefficient between self-administered questionnaires was 0.98. In first self-administered questionnaire prevalence of don’t know answers was 7.6% and missing information was 0.6%. In second self-administered questionnaire prevalence of don’t know answers was 8.8% and missing information was 0.6%.
Prevalence of asthma was 9.3% in the first parent-administered questionnaire and 8.6% in the second parent-administered questionnaire. Cohen’s kappa coefficient between parent-administered questionnaires was 0.87 in reported asthma. In first parent-administered questionnaire prevalence of missing information was 2.3%. In second parent-administered questionnaire prevalence missing information was 3.4%. Cohen’s kappa coefficient between parent- and self-administered questionnaires (parent-child agreement) was 0.87 in reported asthma.
Present study shows that primary school aged children can give repeatable information about their symptoms and perceived indoor air quality at schools. In perceived indoor air quality and with majority of symptoms the repeatability was at the same level or higher in self-administered than parent-administered questionnaire. Agreement between the parent and child was generally poor for symptoms, but better for perceived indoor air quality.
Primary school aged children in this study could give as, or even more, repeatable information about their symptoms and perceived indoor air quality as their parents. To our knowledge, there are no previous studies comparing repeatability among parents and children. In a study by Olson et al. asthmatic children even down to 7 years old could give repeatable information with interview-administered questionnaires about their health . Adolescents have also been shown to give repeatable information about their respiratory symptoms with self-administered questionnaires [13, 14]. Furthermore, it has been reported that children aged 11 years can give repeatable information about their physical and social environment at home and in neighborhood . In our study, only in questions about temperature over 37.5 C and wheezing the test-retest repeatability was distinctly lower in self-administered than in parent-administered questionnaire. Wheezing is a difficult concept even for parents to understand [16, 17], therefore it is not surprising that repeatability of self-reported wheeze was poor in children. Thus, questions should be quite descriptive or should be visualized to be more understandably for children . However, based on this and previous studies, children aged 9–12 years appear to be able to give repeatable information about their symptoms and perceived indoor air quality.
In this study parent-child agreement on symptoms was low, which is in line with previous studies [18,19,20]. It is possible that this is partly explained by characteristics of the child or the parent, which may lead to under- or over reporting of symptoms. It has been shown that mothers self-reported health is strongly associated with how she reports on the health of the child . Parent’s lifestyle factors (e.g. smoking) may also affect their reporting on child’s symptoms . Furthermore, psychosocial factors are known to affect environmental complaints and perceived symptoms . However, agreement on perceived indoor air quality was better than for symptoms in the present study.
The validity of the information given by the children cannot be assessed with the present results and study design. Previous studies have suggested that in child perception of asthma symptoms are associated with objective measurements of the lung function and the correlation between symptoms and lung function may be better or at the same level than parent’s perception of the symptoms [24, 25] but dependent on age of the child . In this study, child-parent agreement on asthma was excellent which is in line with previous studies [18, 20, 27]. This gives some suggestive indication also for validity of the questionnaire.
The test-retest repeatability of reported symptoms and perceived indoor air quality in both self- and parent-administered questionnaire was generally 0.4–0.7 (ICC, k) in this study. This level of repeatability is generally considered to be acceptable for this type of questionnaires. However, when test-retest repeatability is assessed by using kappa values, the dependence of kappa values on marginal prevalence’s (i.e. amount of change in agreement) should be acknowledged. The dependence results directly from definition of kappa .
The few limitations of the study should be acknowledged. The self-administered questionnaire was slightly different from the parent-administered questionnaire. There were fewer questions, questions had one less response category, and the time period used was different (2 weeks in self-reported vs. 4 weeks in parent-reported questionnaire). Furthermore, the time interval in which the test-retest was performed was not the same (1 week in self-reported vs. 2 weeks in parent-reported questionnaire). These factors can affect differences between test-retest repeatability of self- and parent-administered questionnaires and agreement between self- and parent- reported symptoms. It should also be noted that the repeatability of self and parent-administered questionnaires were tested in different schools, since otherwise the children would have had to take part in filling of four questionnaires in total, two self-administered and two parent-administered, as parent-administered questionnaires were instructed to be filled in together with the child. Furthermore, the agreement of the reported symptoms between self- and parent administered questionnaire can be partly affected by the fact that questionnaires were not administered exactly at the same time.
Children aged 9–12 years can give as, or even more, repeatable information about their symptoms and perceived indoor air quality than their parents. Therefore, in future it may be possible to use self-administered questionnaires in epidemiologic studies on perceived indoor air quality and symptoms also among primary school age children. If indoor air questionnaires could be administered to pupils during school day, this could lead to better response rates and even more reliable results.
Cohen’s kappa coefficients
WHO Guidelines. For indoor air quality: dampness and Mould. Geneva: World health Organization; 2009.
Bernstein JA, Alexis N, Bacchus H, Bernstein IL, Fritz P, Horner E, Li N, Mason S, Nel A, Oullette J, Reijula K, Reponen T, Seltzer J, Smith A, Tarlo SM. The health effects of non-industrial indoor air pollution. J Allergy Clin Immunol. 2008;121(3):585–91.
Reijula K, Sundman-Digert C. Assessment of indoor air problems at work with a questionnaire. Occup Environ Med. 2004;61(1):33–8.
Andersson K. Epidemiological approach to indoor air problems. Indoor Air. Supplement: 4 Pages: 32–39 Published: 1998.
Strippoli MP, Silverman M, Michel G, Kuehni CE. A parent-completed respiratory questionnaire for 1-year-old children: repeatability. Arch Dis Child. 2007;92(10):861–5.
Faniran AO, Peat JK, Woolcock AJ. Measuring persistent cough in children in epidemiological studies: development of a questionnaire and assessment of prevalence in two countries. Chest. 1999;115(2):434–9.
Brunekreef B, Groot B, Rijcken B, Hoek G, Steenbekkers A, de Boer A. Reproducibility of childhood respiratory symptom questions. Eur Respir J. 1992;5(8):930–5.
Upton P, Lawford J, Eiser C. Parent-child agreement across child health-related quality of life instruments: a review of the literature. Qual Life Res. 2008;17(6):895–913.
Bevans KB, Riley AW, Moon J, Forrest CB. Conceptual and methodological advances in child-reported outcomes measurement. Expert Rev Pharmacoecon Outcomes Res. 2010;10(4):385–96.
Ung-Lanki S, Lampi J, Pekkanen J. Analyzing symptom data in indoor air questionnaires for primary schools. Indoor Air. 2017 Sep;27(5):900–8.
Cicchetti DV, Feinstein AR. High agreement but low kappa: II. Resolving the paradoxes. J Clin Epidemiol. 1990;43(6):551–8.
Olson LM, Radecki L, Frintner MP, Weiss KB, Korfmacher J, Siegel RM. At what age can children report dependably on their asthma health status? Pediatrics. 2007;119(1):e93–102.
Shaw RA, Crane J, Pearce N, Burgess CD, Bremner P, Woodman K, Beasley R. Comparison of a video questionnaire with the IUATLD written questionnaire for measuring asthma prevalence. Clin Exp Allergy. 1992;22(5):561–8.
Braun-Fahrländer C, Gassner M, Grize L, Minder CE, Varonier HS, Vuille JC, Wüthrich B, Sennhauser FH. Comparison of responses to an asthma symptom questionnaire (ISAAC core questions) completed by adolescents and their parents. SCARPOL-Team Swiss Study on Childhood Allergy and Respir Symptoms with respect to Air Pollut Pediatr Pulmonol. 1998;25(3):159–66.
Hume C, Ball K, Salmon J. Development and Reliability of a self-report questionnaire to examine children's perceptions of the physical activity environment at home and in the neighbourhood. Int J Behav Nutr Phys Act. 2006;3:16.
Cane RS, Ranganathan SC, McKenzie SA. What do parents of wheezy children understand by "wheeze"? Arch Dis Child. 2000;82(4):327–32.
Cane RS, McKenzie SA. Parents' interpretations of children's respiratory symptoms on video. Arch Dis Child. 2001;84(1):31–4.
Sweeting H, West P. Health at age 11: reports from schoolchildren and their parents. Arch Dis Child. 1998;78(5):427–34.
Hoek G, Wypij D, Brunekreef B. Self-reporting versus parental reporting of acute respiratory symptoms of children and their relation to pulmonary function and air pollution. Int J Epidemiol. 1999;28(2):293–9.
Wong TW, Yu TS, Liu JL, Wong SL. Agreement on responses to respiratory illnesses questionnaire. Arch Dis Child. 1998;78(4):379–80.
Waters E, Doyle J, Wolfe R, Wright M, Wake M, Salmon L. Influence of parental gender and self-reported health and illness on parent-reported child health. Pediatrics. 2000;106(6):1422–8.
Dales RE, White J, Bhumgara C, McMullen E. Parental reporting of childrens' coughing is biased. Eur J Epidemiol. 1997;13(5):541–5.
Lahtinen M, Sundman-Digert C, Reijula K. Psychosocial work environment and indoor air problems: a questionnaire as a means of problem diagnosis. Occup Environ Med. 2004;61(2):143–9.
Yu TS, Wong TW. Can school children provide valid answers about their respiratory health experiences in questionnaires? Implications for epidemiological studies. Pediatr Pulmonol. 2004;37(1):37–42.
Lara M, Duan N, Sherbourne C, Lewis MA, Landon C, Halfon N, Brook RH. Differences between child and parent reports of symptoms among Latino children with asthma. Pediatrics. 1998;102(6):E68.
Guyatt GH, Juniper EF, Griffith LE, Feeny DH, Ferrie PJ. Children and adult perceptions of childhood asthma. Pediatrics. 1997;99(2):165–8.
Andersen A, Krølner R, Holstein BE, Due P, Hansen EH. Medicine use among 11- and 13-year-olds: agreement between parents' reports and children's self-reports. Ann Pharmacother. 2007;41(4):581–6.
Vach W. The dependence of Cohen's kappa on the prevalence does not matter. J Clin Epidemiol. 2005;58(7):655–61.
This work was partly funded by a grant from the Ministry of Social Affairs and Health, Finland.
Availability of data and materials
The data supporting our findings is presented within the article and additional file.
Ethics approval and consent to participate
The research plan was approved by the Institutional Review Board (IRB) of National Institute for Health and Welfare (THL) (IRB 00007085, FWA 00014588). The study participation was voluntary and parents of the pupils were able to deny participation in advance.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.