Epidemiologic Reviews Advance Access originally published online on June 1, 2006
Epidemiologic Reviews 2006 28(1):27-40; doi:10.1093/epirev/mxj001
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
ARTICLES |
Measuring Immunization Coverage among Preschool Children: Past, Present, and Future Opportunities
1 Department of Epidemiology and Health Policy Research, College of Medicine, University of Florida, Gainesville, FL
2 Institute for Vaccine Safety, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD
3 Department of International Health, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD
4 Immunization Services Division, National Immunization Program, Centers for Disease Control and Prevention, Atlanta, GA
Reprint requests to Dr. Daniel A. Salmon, Department of Epidemiology and Health Policy Research, College of Medicine, University of Florida, Gainesville, FL 32608 (e-mail: das{at}ehpr.ufl.edu).
accepted for publication March 20, 2006.
 |
ABSTRACT |
|---|
Control of vaccine-preventable diseases depends on maintaining high levels of immunization coverage. Immunization coverage among preschool children remains suboptimal in some areas and sociodemographic subgroups, as well as for more recently introduced vaccines, leaving susceptible young children vulnerable to complications from vaccine-preventable diseases. This paper reviews approaches historically used to measure immunization coverage among preschool children in the United States. The strengths and weaknesses of various approaches to measuring immunization coverage among preschool children are explored, with emphasis on the current means to measure national immunization coverage—the National Immunization Survey. Methods for measuring immunization coverage among preschool children at local and state levels are also evaluated. Future opportunities and challenges for measuring immunization coverage at the local, state, and national levels are explored.
child health services • child, preschool • communicable disease control • comparative study • immunization programs • immunization schedule • vaccination
CDC, Centers for Disease Control and PreventionNIPRCS, National Immunization Provider Record Check Study
The development and use of vaccines have been among the greatest achievements in medicine and public health during the past century, substantially contributing to reduced morbidity and mortality among infants and children (1, 2). Control of vaccine-preventable diseases depends on maintaining high levels of immunization coverage. School immunization requirements have helped to achieve and maintain overall high immunization coverage among school-aged children nationally for most vaccines universally recommended for preschool children (3). However, immunization coverage among preschool children remains suboptimal in some areas and sociodemographic subgroups, as well as for more recently introduced vaccines, leaving susceptible young children vulnerable to complications from vaccine-preventable diseases. Immunization coverage among preschool children should be monitored to ensure that young children in all areas and sociodemographic subgroups are protected from vaccine-preventable diseases, and to ensure rapid and equitable uptake of new vaccines. Vaccine recommendations are provided by the Advisory Committee on Immunization Practice, the American Academy of Pediatrics, and the American Academy of Family Physicians (4). The vaccines and their recommended numbers of doses typically used for measuring immunization coverage are as follows: diphtheria and tetanus toxoids and acellular pertussis vaccine, four doses; Haemophilus influenzae type b vaccine, three doses; poliovirus vaccine (polio), three doses; measles-containing vaccine, one dose; hepatitis B vaccine, three doses; varicella vaccine, one dose; and pneumococcal conjugate vaccine (5), three doses.
Both age-appropriate and up-to-date vaccination status are often measured among preschool children, although up-to-date vaccination measures can overlook windows of vulnerability to disease (6). Up-to-date vaccination status does not consider the age when the vaccine series was initiated and completed (6). Measures of immunization coverage usually require vaccine administration data from medical records, immunization cards, or data obtained directly from health-care providers, as several studies have shown that parental recall can be inaccurate for measures of immunization coverage (7–11). The age at which preschool immunization coverage is measured varies, typically at 12 months, 24 months or 36 months, or 19–35 months.
It is useful to measure immunization coverage at local levels, as immunization coverage is not homogenous within states and localities. Geographic pockets of low coverage occur in many areas that are vulnerable to outbreaks of vaccine-preventable diseases, and special intervention strategies are needed to improve local coverage. For example, during the 1989–1991 measles resurgence, state coverage levels were high, but areas in several major cities had very low levels of immunization coverage, resulting in large measles outbreaks among unvaccinated preschool-aged children (12). The 1989–1991 measles resurgence resulted in more than 55,000 measles cases and at least 120 deaths (13). In Milwaukee, vaccination coverage for measles at 24 months ranged from 43 percent to 90 percent among 207 census tracts. Measles outbreaks disproportionately impacted census tracts with low vaccine coverage (14).
This paper will review approaches historically used to measure immunization coverage among preschool children in the United States. The strengths and weaknesses of various approaches to measuring immunization coverage among preschool children will be explored, with emphasis on the current means to measure national immunization coverage—the National Immunization Survey. Methods for measuring immunization coverage among preschool children at local and state levels will also be evaluated.
|
MEASURING IMMUNIZATION COVERAGE—STATE AND LOCAL METHODS |
|---|
 TOPABSTRACT MEASURING IMMUNIZATION COVERAGE-... NATIONAL SURVEILLANCE OF VACCINE...MEASURING IMMUNIZATION COVERAGE-...FUTURE OPPORTUNITIES AND...FUTURE OPPORTUNITIES AND...CONCLUDING REMARKSREFERENCES |
|---|
Retrospective school surveys
There are several methods that have been used at the state or local level to measure immunization coverage among preschool children. Retrospective school surveys have been used in many states and in some regions, such as in Texas, Kansas, Colorado, Massachusetts, and Los Angeles, California. Retrospective school surveys involve selecting a sample of public and sometimes private schools and examining the school immunization record for dates of vaccine administration and the child's birth date. Algorithms are created that retrospectively measure immunization coverage at different ages, most commonly at age 2 years or at 19–35 months, to make direct comparisons with the National Immunization Survey (15–18). Schools are often selected by a sampling method such that the probability of a school's being selected is proportional to the school's size, ensuring that all children have an equal probability of being selected. Once a school is selected, investigators examine the immunization records of kindergarten students, first grade students, or new school entrants. School immunization records include the date of vaccine administration for all vaccines required by state law for school entry. Consequently, retrospective school surveys often are able to measure only the immunization coverage for antigens required by state law, although some states, such as Texas, include all antigens on school immunization records. In addition, school records may be incomplete even for vaccines and doses required by law, and completeness may vary by state and by school within states (14–17). In 44 states, school immunization records are completed by health-care providers; six states (Indiana, Minnesota, Nebraska, Oregon, Washington, and Wisconsin) permit parents to complete school immunization cards (19). Parental reporting may be problematic because of the increased probability of inaccurate reporting of immunization history.
Retrospective school surveys capitalize on existing infrastructure (immunization histories already collected by schools), include children who have migrated into the study area, and typically have high completion (response) rates in areas where school immunization laws are uniformly enforced (17). The utilization of existing infrastructure makes retrospective school surveys resource efficient. Retrospective school surveys do not require expensive screening of a large population to identify households with children in the target population. The primary weakness of retrospective school surveys is timeliness, as the data collected are several years old, depending on the age group for which coverage is measured. For example, children entering school (kindergarten or first grade) are typically 3–6 years of age. Thus, in the measurement of immunization coverage among children 24 months of age by use of retrospective school surveys, data are 1–4 years old when collected, and additional time (usually about 1 year) is required to enter and analyze data. In 2003, 4.7 percent of children aged 3–4 years and 69.0 percent of US children aged 5 years attended kindergarten (20). Fourteen states require children to attend kindergarten (21). By the age of 6 years, nearly all US children are in school (19). Additionally, retrospective school surveys do not capture children born in the study area but who migrated out of the study area before school entry. The immunization status of some homeschooled children cannot be measured by retrospective surveys in some states. Nationally, 1–2 percent of children are homeschooled (22), while there are some data to suggest that the rate of homeschooling is increasing (23, 24). Some states measure immunization coverage among all homeschooled children (14 states in the 2003–2004 school year). Children who are homeschooled may still attend some classes in schools, such as science and gymnasium activities. These children are often subject to school immunization requirements. Consequently, the immunization histories for many homeschooled children are available in schools and can be included in retrospective school surveys. Others strengths and weaknesses of retrospective school surveys are described in table 1.
|
Birth certificate follow-back surveys
Birth certificate follow-back surveys have been used in many states, including Georgia (25), Oregon (26), and Puerto Rico (27). These surveys typically involve randomly selecting children born during a particular time period (e.g., January 2003) on the basis of birth certificates, finding these children, and collecting immunization histories (28). This approach requires considerable effort to track children. Methods for finding children include searching Medicaid data sets, post office address information, driver's licenses, and telephone directories. Births can also be followed prospectively, although this approach is unusual.
Birth certificate follow-back surveys are extremely resource intensive, as it can be difficult to retrospectively track (locate/find) children, particularly in areas with high mobility. Tracking can be particularly difficult if one wants to include children who migrated out of the study area. Birth certificate follow-back surveys do not include children who immigrated into the study area after birth. Difficulty in finding children can lead to low completion rates with potential for bias. Completion rates are often 80 percent or less, creating the potential for bias as the children not included may be at increased risk for underimmunization. For example, the failure to track children by use of birth certificate follow-back surveys is related to family mobility. Highly mobile children may lack continuity of primary care and may be less likely to be up-to-date for vaccines. Typically, birth certificate follow-back surveys take 1–2 years to select eligible children, locate the children, obtain immunization histories, and calculate immunization coverage.
Household cluster surveys
Household cluster surveys have been used infrequently by states/localities because of the intensive effort to conduct such studies. Some localities, such as regions of Washington State, used household cluster surveys periodically for many years. The methodology of household cluster surveys has been outlined by the World Health Organization's Expanded Program on Immunization and is widely used throughout the developing world (29). Regions are divided into a complete set of nonoverlapping subpopulations usually defined in the United States by geographic boundaries. These clusters are sampled with probabilities proportional to size, often 30 clusters, and then households are contacted in a proscribed manner until a sufficient number of subjects (often seven) are identified within each cluster. Typically, units are on the household level, and all eligible children (based on age criteria) are included in the sample. Often, only the first household is randomly selected, and subsequent households are those that are closest to the last selected household.
Household cluster surveys are an efficient means for measuring immunization coverage. This method is particularly useful for densely populated areas; the costs can be much higher for rural areas. Vaccine coverage estimates can be valid for individual clusters (30). The "30 by 7" method is designed to provide vaccine coverage estimates within ±10 percentage points of the true proportion of vaccination coverage and to provide a low-cost, standardized method for comparing changes in immunization coverage over time. Immunization coverage can be compared from one region to another if the "30 by 7" method is conducted in each of the regions to be compared. The validity of immunization coverage measures using household cluster surveys in the United States is questionable if immunization histories are not verified by health-care providers. Parental report may be inaccurate, parents may not have shot records available, and shot records may be incomplete. Typically, household cluster surveys take 1–2 years to select eligible children, obtain immunization histories, and calculate immunization coverage. Household cluster surveys may take less time to complete than birth certificate follow-back surveys, as identifying children is done prospectively rather than retrospectively, yet the time of coverage measure since age 24 months may add additional time, depending on the age criteria. Although the "30 by 7" method is a low-cost, standardized method to compare immunization coverage, the random selection of only the first household in the cluster, as well as every eligible child in the household being selected, can lead to some bias if immunization coverage is spatially clustered or if a household is more or less prone to get their children vaccinated. A more costly alternative, yet superior statistically, is to apply a selection process based on probability samples, such as simple random sampling, list/systematic sampling, or grid sampling. These types of sample procedures provide better estimates of coverage, as well as more accurate measures of variance (31).
|
NATIONAL SURVEILLANCE OF VACCINE DISTRIBUTION |
|---|
|
TOPABSTRACTMEASURING IMMUNIZATION COVERAGE-...NATIONAL SURVEILLANCE OF VACCINE...MEASURING IMMUNIZATION COVERAGE-...FUTURE OPPORTUNITIES AND...FUTURE OPPORTUNITIES AND...CONCLUDING REMARKSREFERENCES |
|---|
The Centers for Disease Control and Prevention (CDC) has passively monitored immunization uptake since 1962 through voluntary vaccine manufacturer reports of the total number of vaccine doses distributed. The manufacturers' data were used in the early 1980s to watch for a potential decrease in use of diphtheria/pertussis/tetanus in favor of diphtheria/tetanus amidst public fears over the safety of the pertussis component. An additional passive monitoring system for immunization purchase involves the Biologics Surveillance System, which monitors the quantity of government vaccine purchased as part of the Vaccines for Children program, 317 grants, and state purchases. However, these vaccine purchase mechanisms include approximately 55 percent of all childhood vaccines purchased in the United States, so these data have limited utility in generating population estimates of vaccine coverage (32). These numbers do not account for unused or wasted vaccine that is not returned to manufacturers, age of recipient, and overimmunization (people who receive more than the recommended number of vaccine doses) (33, 34).
|
MEASURING IMMUNIZATION COVERAGE—NATIONAL SURVEYS |
|---|
|
TOPABSTRACTMEASURING IMMUNIZATION COVERAGE-...NATIONAL SURVEILLANCE OF VACCINE...MEASURING IMMUNIZATION COVERAGE-...FUTURE OPPORTUNITIES AND...FUTURE OPPORTUNITIES AND...CONCLUDING REMARKSREFERENCES |
|---|
In 1957, the Census Bureau began to supplement the Current Population Survey with questions regarding receipt of polio vaccine. Known as the "United States Immunization Survey," this was the first national immunization survey in the United States. The United States Immunization Survey was conducted via personal household interviews from 1957 to 1970 and over the telephone from 1970 to 1985 (32). When the United States Immunization Survey was discontinued in 1985, questions had been expanded to include pneumonia and influenza vaccines for all respondents. For children under the age of 19 years, questions regarding the receipt of polio, diphtheria, tetanus, pertussis, measles, mumps, and rubella vaccines, as well as prior history of measles and mumps diseases, were included. In its final year, the United States Immunization Survey covered 60,000 households and included data on the child's age, ethnicity, and gender (35). The United States Immunization Survey was unable to generate subnational coverage estimates (26). Additionally, the validity of this survey was limited by relying primarily on respondent recall; in 1985, less than one fourth of the responses was based on the respondent's referring to the immunization record (36). United States Immunization Survey estimates based on respondent recall were up to 23 percent lower than were estimates based on responses where the immunization record was used, according to data from one state (26).
From 1986 to 1991, no national data were systematically collected on preschool immunization coverage in the United States. In 1990, CDC conducted retrospective surveys of school immunization records for first and fifth grade students in eight urban areas to assess 2-year-old measles immunization status after seeing an increase in measles cases in preschool children. These studies found that lower measles immunization rates in preschool children were associated with an increased incidence of measles in this age group during the 1980s (36). In response to the measles resurgence of 1989–1991, CDC's Immunization Division recommended from 1990 to 1994 that all states conduct retrospective school surveys (17).
In 1991, the National Health Interview Survey began collecting information on vaccination coverage. The National Health Interview Survey consists of household interviews of the noninstitutionalized population in the United States to obtain information on a broad range of health topics. In 1991, immunization information was collected on one sample child under 18 years of age in the household. In 1992 and 1993, this changed to include one child under the age of 6 years; in 1994, to include one child under the age of 6 years and all other children between 19 and 35 months of age; from 1997 to 2002, to include one sample child under 18 years of age and all additional children 12–35 months of age; and in 2003, the additional sample of children aged 12–35 months was eliminated, and information was obtained on only one sample child under 18 years of age. Immunization information was removed from the National Health Interview Survey in 2004. In 2005, questions on receipt of influenza vaccination were added to the child core of the survey, asked of one randomly selected child aged less than 18 years in each family. The National Health Interview Survey cannot obtain state estimates of coverage in children because of limited sample size; not all households surveyed contained an eligible child, and obtaining a large enough sample to determine state-level coverage estimates is prohibited by cost. From 1994 to 2001, the National Immunization Provider Record Check Study (NIPRCS) increased the accuracy of the National Health Interview Survey by verifying household immunization coverage reports by use of the vaccination provider's immunization record. National Health Interview Survey/NIPRCS data were not reported until 12–18 months after data collection was complete. The Behavioral Risk Factor Surveillance System collects influenza vaccine uptake data and calculates coverage based upon self-report (37, 38).
In 1993, the Childhood Immunization Initiative (39) was established for the following reasons: 1) to improve the delivery of vaccines to children; 2) to reduce the cost of vaccines for eligible parents; 3) to enhance awareness, partnerships, and community participation; 4) to improve vaccinations and their use; and 5) to monitor vaccination coverage and occurrences of disease. Subsequently, the Healthy People 2000 (40) and 2010 (41) objectives established the goal of having at least 90 percent of children aged 2 years fully vaccinated with each of the recommended schedules of vaccines separately and 80 percent fully vaccinated with the complete series. To fulfill the Childhood Immunization Initiative mandate of monitoring vaccination coverage and marking progress toward achieving the Healthy People 2000 and 2010 goals, the National Immunization Survey was implemented by the National Immunization Program and the National Center for Health Statistics, CDC.
The target population of the National Immunization Survey is children 19–35 months of age living in US households at the time of the interview. Official coverage estimates from the National Immunization Survey give up-to-date rates with respect to the recommended numbers of doses of seven vaccines (42). Official coverage estimates for influenza vaccination of children aged 6–23 months are also reported for each influenza season, starting with the 2002–2003 season (43). Vaccination coverage estimates are published by CDC for each calendar year.
Since the second quarter of 1994, the National Immunization Survey has conducted quarterly surveys in 78 Immunization Action Plan areas, consisting of the 50 states, District of Columbia, and 27 other large urban areas. This design has made it possible to produce annual estimates of vaccination coverage levels for each of the 78 Immunization Action Plan areas. In 2005, five new areas were added, and five original Immunization Action Plan areas rotated off the National Immunization Survey. Rotation of Immunization Action Plan areas will allow monitoring of immunization coverage in new areas with possibly low coverage. Further, by using the same methodology across all Immunization Action Plan areas, the National Immunization Survey produces estimates that are comparable among Immunization Action Plan areas and over time. The National Immunization Survey is designed to obtain an average annual sample size of 400–500 completed household interviews in each of the 78 Immunization Action Plan areas.
The National Immunization Survey uses two phases of data collection to obtain vaccination information for a large national probability sample of young children: 1) a list-assisted, random digit dialing survey designed to identify households with children 19–35 months of age, followed by 2) the National Immunization Survey Provider Record Check survey, which obtains provider-reported vaccination histories for these children. When a household with an age-eligible child is identified, the random digit dialing interviewer collects demographic information about each age-eligible child in the household, demographic information about the child's mother, and sociodemographic information about the household. At the end of the random digit dialing interview, consent is requested to contact the children's medical care providers. If consent is given, all vaccination providers named by the National Immunization Survey random digit dialing respondent are contacted by mail to obtain the children's vaccination record from the providers. Provider-reported vaccination histories are used to evaluate sampled children's vaccination status. The Provider Record Check survey was not instituted for all children until 1995; in 1994, provider records were obtained only for children whose respondent did not report up-to-date coverage for diphtheria/tetanus/pertussis, measles/mumps/rubella, polio, and H. influenzae type B when the child's immunization record was available (44, 45).
Data from the Provider Record Check yield the numbers of doses for each of the vaccines reported. These counts are then compared with the number of doses recommended by the vaccination schedule to determine whether the child is up-to-date. These data, along with sampling weights and National Immunization Survey design information, are used to estimate vaccination coverage rates.
Prior to the 2002 revisions, the National Immunization Survey generated statistically similar coverage estimates as did the NIPRCS (46), probably because of, in part, the use of a statistical adjustment to the survey weights that calibrated the National Immunization Survey to NIPRCS. This adjustment was referred to as "modified post-stratification" and was used at that time to compensate for households that did not have telephones (47).
Smith et al. (48) and Barker et al. (49), in reviewing application of bootstrap methods to the National Immunization Survey, found that the confidence intervals for state rankings were quite wide, underscoring the importance of reporting the uncertainty in state ranks. For example, for the 4:3:1:3 series in 2003–2004, Oklahoma ranked 46th by the point estimates but could potentially have been ranked as high as second when considering the upper and lower bounds of the 95 percent confidence intervals (50).
The National Immunization Survey has provided continual monitoring of immunization coverage rates in all 78 Immunization Action Plan areas. The CDC has published coverage estimates for all vaccines recommended by the Advisory Committee on Immunization Practice/American Academy of Pediatrics/American Academy of Family Physicians annually based on National Immunization Survey coverage estimates (51–66). The National Immunization Survey is also used to estimate uptake of pneumococcal conjugate vaccine and other specific antigens, including influenza, hepatitis A, and rotavirus (41, 51, 65–69). The National Immunization Survey has also been used to look at differences in immunization coverage by race/ethnicity (70–76), income, and other socioeconomic and demographic factors (71, 77–84). The National Immunization Survey has examined other important issues associated with vaccine coverage, such as breastfeeding (85, 86); participation in the Special Supplemental Nutrition Program for Women, Infants, and Children (known as the "WIC program") (87); the impact of child-care immunization requirements (88, 89); associations between physician distribution and immunization rates (90); where children were vaccinated (public, private, and mixed sites and enrollment in the Vaccines for Children program) (91); number of provider visits (92, 93); impact of invalid immunization doses (84, 94); and vaccine safety beliefs (87, 95). Additionally, National Immunization Survey data have shown that diphtheria and tetanus toxoids and acellular pertussis vaccine coverage can be used as a proxy for the 4:3:1:3 series to measure the immunization status of children in nonmedical settings (74) and that immunization coverage estimates can be used to assess the association between Medicaid managed care and immunization coverage (96). The National Immunization Survey sampling frame has also been used to conduct immunization surveys of older adults (97) and other health surveys (98, 99).
The National Immunization Survey has been described as "the backbone of the immunization assessment system" (100, p. 85). The National Immunization Survey's standardized state coverage estimates allow for comparing vaccination rates across states that, according to Cordero and Orenstein, "has resulted in informal competition among immunization programs, and has provided positive motivation" (100, p. 84). State vaccination coverage measured by the National Immunization Survey is used by CDC to determine the level of funding for states; higher National Immunization Survey coverage results in greater funding, and areas with lower estimated vaccination coverage rates are targeted for technical assistance. Data from the National Immunization Survey are also used to determine national vaccine needs for the Vaccines for Children program. The diversity and quantity of issues explored through the National Immunization Survey exemplify the utility of having infrastructure such as the National Immunization Survey to measure immunization coverage among preschool children.
Studies based on the National Immunization Survey and program and policy decisions based on these studies are dependent upon the validity of National Immunization Survey data. There are a few limitations to the National Immunization Survey. Most notably, the survey potentially represents only a minority of the eligible households that are intended to be surveyed. The proportion of the population intended to be sampled who are actually sampled, and for whom adequate data are collected, is the product of three proportions: 1) the proportion of children 19–35 months of age who live in a household with a telephone that is reachable by the list-assisted random digit dialing survey (92.8 percent in 2000; more recent measure not available); 2) the overall response rate (101, 102) of households with children 19–35 months of age to the random digit dialing portion of the National Immunization Survey (48.8 percent in 2004); and 3) the proportion of households with completed random digit dialing interviews for whom adequate data are returned from the provider in the Provider Record Check (70.6 percent in 2002).
In 2004, the product of these three proportions was 32.0 percent. The overall response rate has been decreasing from 1995 to 2004 (figure 1). The proportion of households with completed random digit dialing interviews for whom adequate data are returned from the provider in the Provider Record Check saw some initial increases in 1995–1997 but has more recently remained fairly constant (figure 1). An alternative approach for describing nonresponse proportions at each stage of the National Immunization Survey is presented in table 5 of Smith et al. (103).
|
The noncoverage of the National Immunization Survey sampling frame for households without telephones, as well as cascading points of nonresponse as the survey unfolds, results in a relatively low proportion of the population intended to be surveyed for which adequate data are collected, even with relatively high response rates at most points. To minimize potential bias, several steps are taken in estimation of vaccination coverage, including adjustment for households without telephones and poststratification to estimated prevalence of children aged 19–35 months in each Immunization Action Plan area. These adjustments include variables known to be associated with vaccination coverage. A key underlying assumption of these adjustments is that, within subgroups defined by combinations of factors associated with coverage, sampled children for whom vaccination histories are obtained are similar to those for whom vaccination histories are not obtained. It is difficult to collect data needed to assess these assumptions. Further research is needed to assess the potential overall direction and level of bias, if any, in estimates of vaccination coverage from the National Immunization Survey.
Another approach for assessing potential bias in National Immunization Survey coverage data is to compare coverage estimates with estimates measured through other surveys. Table 2 provides coverage measured by retrospective school surveys, birth certificate follow-back surveys, and household cluster surveys compared with National Immunization Survey data. Care should be taken in drawing conclusions from such comparisons for several reasons: there is difficulty in ensuring that the populations measured are adequately comparable; studies being compared with the National Immunization Survey may have their own bias (e.g., retrospective school surveys may underestimate coverage because of incomplete school records of immunization; household surveys may use parental report of vaccination only); and table 2 is a sample to provide examples and is in no way a comprehensive list of coverage surveys. A more comprehensive review would be useful.
|
Even if some bias exists in a survey, valid comparisons can be made over time or across subgroups if the bias is constant over time and across subgroups. As in other random digit dialing surveys, the Council of American Survey Research Organizations response rate for the National Immunization Survey declined from 1995 through 2003, while there has been a slight increase in 2004 (figure 1) (104). The expected increased prevalence of households that use cell and broadband telephones exclusively may contribute to lower future response rates. The top broadband phone company, Vonage, reported more than one million subscribers in September 2005, an increase from 240,000 in September 2004; none of their numbers are listed unless reported by the homeowner (105). As a result, these numbers are unreachable by the National Immunization Survey. Data from the 2003 National Health Interview Survey show that 3.7 percent of all households in the United States are "cell-phone only" households, representing 62 percent of all households without a landline telephone in the United States. These data also showed variation in the rate of cell-phone households by race, education, and poverty status, which have been associated with variations in immunization coverage. Furthermore, these data suggest that the prevalence of cell-phone only households is increasing (106).
Another limitation of the National Immunization Survey is that the sample size provides data on immunization coverage by state or Immunization Action Plan area only, not local communities. Because of the complexity and labor-intensive nature of the National Immunization Survey, within-state sampling is cost prohibitive. The ongoing use of retrospective school surveys, birth certificate follow-back surveys, and household cluster surveys by states and localities is reflective of the need to have county-level immunization coverage data. Small-area estimation methods are currently being applied to National Immunization Survey data as an approach to meet this need.
|
FUTURE OPPORTUNITIES AND CHALLENGES FOR MEASURING STATE AND LOCAL IMMUNIZATION COVERAGE |
|---|
Population-based retrospective school surveys offer one alternative for measuring state and local immunization coverage by taking advantage of new technologies. In nearly every school in the United States, an immunization record for each student is filed in cabinets and occasionally in a variety of school-level databases. Each child's birth date and dates of vaccination are collected and reviewed to ensure that the child has met the school immunization requirements or filed an appropriate exemption. These records must be maintained at the school in order to exclude unimmunized children during a disease outbreak. Modern technology permits automated entry of this information into databases. Current school immunization forms could be converted to permit electronic data entry. Schools would need to maintain copies of all such forms, but the dates of vaccine administration and the children's birth dates could be forwarded to health departments for electronic data entry or scanning.
Population-based retrospective school surveys can provide immunization histories for nearly entire cohorts of children. The most attractive property of using retrospective surveys of school entrants to measure immunization coverage is that this process utilizes already existing infrastructure. The cost of such a system would be reasonably modest given that it builds upon already existing infrastructure and should be able to capture data from nearly the entire school-entry cohort. Population-based retrospective school surveys would provide immunization coverage by school building and consequently by community. This information would be extremely useful to public health agencies and immunization programs. Interventions to improve immunization coverage could be targeted to communities with low coverage, and subsequent cohorts of schoolchildren could be used to measure the impact of such interventions.
While feasible, the practicality of population-based retrospective school surveys must be demonstrated. A statewide retrospective survey was successfully accomplished in Minnesota (16), although electronic means of data entry were not used, and stratified by region in Colorado (107). The validity and completeness of school immunization forms require additional study, particularly in states that permit parents to complete these forms and in states with suboptimal enforcement of school immunization requirements. It is also important to develop and test quality-control measures to ensure the adequacy of using school immunization forms. Lack of timeliness of these surveys for assessing coverage by 2 years of age remains a major limitation, even for community-level assessment. Use of these surveys for assessing and comparing coverage at the state level may also be limited because of differences in required doses by state, the methods used, and completeness of immunization records across schools.
Immunization registries offer another possible method for measuring immunization coverage at local and state levels. As defined by the National Vaccine Advisory Committee, immunization registries are "confidential, computerized information systems that contain information about immunizations and children" (108, p. 2). Immunization registries can be used for measuring vaccine coverage, generating reminders when immunizations are due and recalls when immunizations are overdue, identifying pockets of need for targeted interventions, improving vaccine safety by reducing overimmunization and calculating accurate denominators important for postlicensure vaccine safety studies, and facilitating vaccine inventory, supply, and management (106).
Healthy People 2010 includes an objective of 95 percent participation of children under 6 years of age with two or more immunizations in an immunization registry. The federal government, states, and nonprofit organizations such as the Robert Wood Johnson Foundation's All Kids Count Program have invested substantial resources into the development of immunization registries. The Childhood Immunization Initiative Act of 1993 was originally introduced with language to establish a national registry, although this language was changed to appropriate funding for states and communities to establish their own registries (109–111). Despite over 10 years of registry development, enrollment in registries is still limited. In 2000, only 24 percent of children under the age of 6 years were participating in an immunization registry (112). In 2003, participation rates for children under the age of 6 years had increased to 44 percent according to a survey of immunization programs conducted by CDC (113). Twenty-seven among 56 grantees had participation rates above 64 percent in 2003; these grantees represent 33 percent of all US children under the age of 6 years, and an estimated 43 percent of these children aged under 6 years were enrolled in a registry. This estimate was similar to an estimate based on National Immunization Survey data that 44 percent of children aged 19–35 months had vaccinations reported to a registry. However, in the National Immunization Survey, 40 percent of children participating in a registry had incomplete records, and estimates of vaccination coverage based on National Immunization Survey provider-verified data were higher than were estimates based on registry data (114). Participation in registries, the proportion of public and private providers reporting to registries, and the completeness of immunization histories in registries vary by registry. In 2003, 25 registries reported that greater than 95 percent of public provider sites submitted data, and five registries reported that greater than 95 percent of private provider sites submitted data (114). The completeness of registry data, even in states such as these with high participation rates, requires further study.
Registries can be used to generate real-time coverage estimates for immunization coverage of children at any specified age within the catchment area of the registry, only if all, nearly all, or a representative sample of children have their complete immunization history included in the registry or if vaccination histories in registries can be adjusted statistically to account for underascertainment of children's vaccination histories. As immunization registries improve, they may become increasingly useful for measuring immunization coverage.
|
FUTURE OPPORTUNITIES AND CHALLENGES FOR MEASURING IMMUNIZATION COVERAGE NATIONALLY |
|---|
The National Immunization Survey will continue as the primary method for assessing immunization coverage in the United States until a viable, affordable alternative is identified that uses a consistent methodology across states and over time to validly monitor immunization coverage.
The primary priority for national coverage assessment is to monitor immunization coverage of preschool children, incorporating new vaccines as they are recommended and providing an evaluation tool for the Vaccines for Children program. An emerging priority is assessment of adolescents, particularly during the recommended adolescent preventive services visit for children 11–12 years of age, with recent licensure and recommendations for combined tetanus, diphtheria, and pertussis vaccines and meningococcal vaccine, as well as pending licensure of human papillomavirus vaccine. A pilot adolescent study was conducted in 2004. CDC plans to conduct annual adolescent coverage surveys starting in 2006, using the National Immunization Survey sampling frame and including a provider record check. Another priority is to maintain assessment of urban areas and to expand assessment to other urban and rural areas. This is being accomplished by rotation of urban areas, allowing assessment of new areas, and by use of small-area statistical methods to estimate coverage at the county level. Another priority is to maintain or improve response rates and to adapt to changes in telephone use in the United States. To address declining response rates, efforts are underway, including the offering of incentives to certain cases, cognitive interviewing to better understand the interview process and ways to maintain cooperation throughout the interview, experiments with caller identification, and analysis of call-history data to determine more efficient call patterns and times. To address the increasing prevalence of households with only wireless phone service, research is underway to estimate potential bias and to identify methods to correct for potential bias. Future research will consider use of dual landline and wireless telephone sampling frames. As response rates decrease, the costs of maintaining target sample sizes increase, a major limiting factor for the future of random digit dialing surveys.
A significant portion of the National Immunization Survey cost is consumed by identifying the 3–4 percent of households with children aged 19–35 months. Further research is needed on the use of dual sample frames, which include the usual random digit dialing sample frame in conjunction with another sampling frame containing a higher proportion of households with young children. Use of Immunization Information Systems should be considered, as they are typically populated using birth certificates and can provide contact information for households specifically with age-eligible children. Another major potential limiting factor is the increased amount of bias in vaccination coverage estimates arising from increasing nonresponse and noncoverage of households without landline telephones. In addition to the approaches mentioned above to improve response rates, to incorporate wireless-only households, and to improve the efficiency of survey operations, nonresponse bias analysis becomes increasingly important.
|
CONCLUDING REMARKS |
|---|
A variety of methods have been used for measuring immunization coverage among preschool children in the United States. Currently, the National Immunization Survey is the primary means of measuring coverage nationally. The declining trend in response rates to random digit dialing surveys signals the needs to carefully evaluate potential bias in National Immunization Survey vaccination coverage estimates, to collect additional data to further estimate and adjust for potential bias, to develop innovative ways to improve response rates and cost-efficiency, and to consider modifications to the survey sampling design (e.g., use of dual sampling frames). Many states and localities continue to use retrospective school surveys, birth certificate follow-back surveys, and household cluster surveys for local- and state-level assessments. Increased use of population-based retrospective school surveys, with evaluation of completeness of school immunization records and examination of methods to minimize bias and improve timeliness, would provide valuable community-level data. Use of immunization registries for vaccination coverage assessment deserves further attention and consideration. Further development and improvement of these vaccination coverage assessment systems are crucial for guiding local and national efforts to maximize the protection of children against vaccine-preventable diseases.
|
ACKNOWLEDGMENTS |
|---|
The findings and conclusions in this report are those of the authors and do not necessarily represent the views of the Centers for Disease Control and Prevention in Atlanta, Georgia.
Conflict of interest: none declared.
|
REFERENCES |
|---|
- Ten great public health achievements—United States 1900–1999. MMWR Morb Mortal Wkly Rep (1999) 48:241–3.[Medline]
- Impact of vaccines universally recommended for children—United States 1900–1998. MMWR Morb Mortal Wkly Rep (1999) 48:243–8.[Medline]
- Orenstein WA and Hinman AR. (1999) The immunization system in the United States—the role of school immunization laws. Vaccine 17:suppl 3, S19–24.
- Recommended childhood and adolescent immunization schedule—United States, 2005. MMWR Morb Mortal Wkly Rep (2005) 53:Q1–3.
- Notice to readers: limited supply of pneumococcal conjugate vaccine. MMWR Morb Mortal Wkly Rep (2003) 52:1234.
- Luman ET, Barker LE, Shaw KM, et al. (2005) Timeliness of childhood vaccinations in the United States: days undervaccinated and number of vaccines delayed. JAMA 293:104–11.
[Free Full Text] - Bolton P, Holt E, Ross A, et al. (1998) Estimating vaccination coverage using parental recall, vaccination cards, and medical records. Public Health Rep 113:521–6.[ISI][Medline]
- Lyratzopoulos G, Aston R, Bailey K, et al. (2002) Accuracy of routine data on MMR vaccination coverage and validity of parental recall of vaccination. Commun Dis Public Health 5:305–10.[Medline]
- Goldstein KP, Kviz FJ, Daum RS. (1993) Accuracy of immunization histories provided by adults accompanying preschool children to a pediatric emergency department. JAMA 270:2190–4.[Abstract]
- Suarez L, Simpson DM, Smith DR. (1997) Errors and correlates in parental recall of child immunizations: effects on vaccination coverage estimates. (Electronic article). Pediatrics 99:e3.
[Abstract/Free Full Text] - Czaja C, Crossette L, Metlay JP. (2005) Accuracy of adult reported pneumococcal vaccination status of children. Ann Epidemiol 15:253–6.[CrossRef][ISI][Medline]
- Atkinson WL and Orenstein WA. (1992) The resurgence of measles in the United States, 1989–1990. Annu Rev Med 43:451–63.[CrossRef][ISI][Medline]
- Lee B, Ying M, Papania MJ, et al. (2004) Measles hospitalization, United States, 1985–2002. J Infect Dis 189:suppl 1, S210–15.[Medline]
- Hutchins SS, Baughman AL, Orr M, et al. (2004) Vaccination levels associated with lack of measles transmission among preschool-aged populations in the United States, 1989–1991. J Infect Dis 189:suppl 1, S108–15.[Medline]
- Rodewald LE, Roghmann KJ, Szilagyi PG, et al. (1993) The school-based immunization survey: an inexpensive tool for measuring vaccine coverage. Am J Public Health 83:1749–51.
[Abstract/Free Full Text] - Ehresmann KR, White KR, Hedberg CW, et al. (1998) A statewide survey of immunization rates in Minnesota school age children: implications for targeted assessment and prevention strategies. Pediatr Infect Dis J 17:711–16.[CrossRef][ISI][Medline]
- Zell ER, Dietz V, Stevenson J, et al. (1994) Low vaccination levels of US preschool and school-age children. Retrospective assessments of vaccination coverage, 1991–1992. JAMA 271:833–9.[Abstract]
- Bastis DG, Jackson JF, Roche RA. (2001) Comparison of vaccine coverage levels from the National Immunization Survey and the Texas Retrospective Immunization Survey. J Public Health Manag Pract 7:65–8.[Medline]
- Omer SB. State policies for ensuring compliance with school immunization requirements: associations with incidence of pertussis. Presented at the 40th National Immunization Conference, Atlanta, Georgia, March 2006.
- Shin HB. School enrollment—social and economic characteristics of students: October 2003. Population characteristics. Washington, DC: US Census Bureau, 2005. (Publication no. 20-554) (http://www.census.gov/prod/2005pubs/p20–554.pdf).
- Education Commission of the States. Kindergarten, Denver, CO: Education Commission of the States, 2006, (http://www.ecs.org/html/issue.asp?subIssueID=0&issueID=77).
- Lines PM. (1998) Homeschoolers: estimating numbers and growth. (Office of Educational Research and Improvement, US Department of Education, Washington, DC).
- Henke RR and Kaufman P. (2000) Issues related to estimating the home-schooled population in the United States with national household survey data. (National Center for Education Statistics, US Department of Education, Washington, DC) (NCES publication no. 2000-311) (http://nces.ed.gov/pubs2000/2000311.pdf).
- Bauman KJ. (2001) Home schooling in the United States: trends and characteristics (US Census Bureau, Washington, DC).
- Hoban CA and Thompson R. (2005) Georgia Immunization Study, 2004 final report. (Georgia Department of Human Resources, Atlanta, GA) (http://health.state.ga.us/pdfs/publications/reports/gaimmunizationstudy.04.pdf).
- Wu L and Priedeman M. (2000) 1999 immunization survey of two year olds. (Oregon Immunization Program and Acute and Communicable Disease Epidemiology Program, Portland, OR) (http://www.oregon.gov/DHS/ph/imm/docs/s2yrolds.pdf).
- Impact of vaccine shortage on diphtheria and tetanus toxoids and acellular pertussis vaccine coverage rates among children aged 24 months—Puerto Rico, 2002. MMWR Morb Mortal Wkly Rep (2002) 51:667–8.[Medline]
- Pregnancy Risk Assessment Monitoring System (PRAMS): methodology. (2006) (Division of Reproductive Health, Centers for Disease Control and Prevention, Atlanta, GA) (http://www.cdc.gov/PRAMS/methodology.htm).
- Hoshaw-Woodard S. (2001) Description and comparison of the methods of cluster sampling and lot quality assurance sampling to assess immunization coverage. (Department of Vaccine and Biologicals, World Health Organization, Geneva, Switzerland) (http://www.who.int/vaccines-documents/DocsPDF01/www592.pdf).
- Pfeffermann D, Skinner CJ, Holmes DJ, et al. (1998) Weighting for unequal selection probabilities in multilevel models (with discussion). J R Stat Soc (B) 60:23–56.[CrossRef]
- Kish L. (1995) Survey sampling. Wiley Classics Library ed. (John Wiley & Sons, Inc, New York, NY).
- Simpson DM, Ezzati-Rice TM, Zell ER. (2001) Forty years and four surveys: how does our measuring measure up? Am J Prev Med 20:6–14.[ISI][Medline]
- Orenstein WA, Douglas RG, Rodewald LE, et al. (2005) Immunizations in the United States: success, structure, and stress: a complex collaboration involving government, industry, providers, academe, professional societies, and third-party payers. Health Aff 24:599–611.
[Abstract/Free Full Text] - Chen RT and Orenstein WA. (1996) Epidemiologic methods in immunization programs. Epidemiol Rev 18:99–117.
[Free Full Text] - Bureau of the Census. Current population survey, September 1985: United States Immunization and Smoking Survey. (Computer file). Washington, DC: US Department of Commerce (producer) and Ann Arbor, MI: Inter-university Consortium for Political and Social Research (distributor), 1989. (ICPSR study no. 9133).
- Measles vaccination levels among selected groups of preschool-aged children—United States. MMWR Morb Mortal Wkly Rep (1991) 40:36–9.[Medline]
- Estimated influenza vaccination coverage among adults and children—United States, September 1, 2004–January 31, 2005. MMWR Morb Mortal Wkly Rep (2005) 54:304–7.[Medline]
- Mokdad AH, Stroup DF, Giles WH, et al. (2003) Public health surveillance for behavioral risk factors in a changing environment. Recommendations from the Behavior Risk Factor Surveillance Team. MMWR Recomm Rep 52:RR-91–12.[Medline]
- Reported vaccine-preventable diseases—United States, 1993, and the Childhood Immunization Initiative. MMWR Morb Mortal Wkly Rep (1994) 43:57–60.[Medline]
- National Center for Health Statistics. (1999) Healthy People 2000 review, 1998–99. (Public Health Service, Hyattsville, MD) (DHHS publication no. (PHS) 99-1256) (http://www.cdc.gov/nchs/data/hp2000/hp2k99.pdf).
- US Department of Health and Human Services. (2000) Healthy People 2010: conference edition. (US Department of Health and Human Services, Washington, DC) Vols . 1 and 2:.
- Notice to readers: recommended childhood immunization schedule—United States, 2000. MMWR Morb Mortal Wkly Rep (2000) 49:35–8 47.[Medline]
- Childhood influenza-vaccination coverage—United States, 2002–03 influenza season. MMWR Morb Mortal Wkly Rep (2004) 53:863–6.[Medline]
- National vaccination coverage levels among children aged 19–35 months—United States, 1998. MMWR Morb Mortal Wkly Rep (1999) 48:829–30.[Medline]
- National, state, and urban vaccination coverage levels among children aged 19–35 months—United States, July 1994–June 1995. MMWR Morb Mortal Wkly Rep (1996) 45:508–9.[Medline]
- Bartlett DL, Ezzati-Rice TM, Stokley S, et al. (2001) Comparison of NIS and NHIS/NIPRCS vaccination coverage estimates. National Immunization Survey. National Health Interview Survey/National Immunization Provider Record Check Study. Am J Prev Med 20:25–7.[CrossRef][ISI][Medline]
- Battaglia MP, Malec DJ, Spencer BD, et al. (1995) Adjusting for noncoverage of nontelephone households in the National Immunization Survey. In: Proceedings of the section on survey research methods (American Statistical Association, Alexandria, VA) pp. 678–83.
- Smith PJ, Hoaglin DC, Battaglia MP, et al. (2003) Implementation and applications of bootstrap methods for the National Immunization Survey. Stat Med 22:2487–502.[CrossRef][ISI][Medline]
- Barker LE, Smith PJ, Gerzoff RB, et al. (2005) Ranking states' immunization coverage: an example from the National Immunization Survey. Stat Med 24:605–13.[CrossRef][ISI][Medline]
- Centers for Disease Control and Prevention. (2004) Estimated vaccination coverage with individual vaccines and selected vaccination series among children 19–35 months of age by state. US National Immunization Survey. (Excel spreadsheet). (Centers for Disease Control and Prevention, Atlanta, GA) (http://www.cdc.gov/nip/coverage/nis/03–04/tab03_antigen_state_0304.xls).
- National, state, and urban area vaccination coverage among children aged 19–35 months—United States, 2003. MMWR Morb Mortal Wkly Rep (2004) 53:658–61.[Medline]
- National, state, and urban area vaccination coverage levels among children aged 19–35 months—United States, July 1996–June 1997. MMWR Morb Mortal Wkly Rep (1998) 47:108–16.[Medline]
- National vaccination coverage levels among children aged 19–35 months—United States, 1998. MMWR Morb Mortal Wkly Rep (1999) 48:829–30.[Medline]
- National, state, and urban area vaccination coverage levels among children aged 19–35 months—United States, 1999. MMWR Morb Mortal Wkly Rep (2000) 49:585–89.[Medline]
- National, state, and urban area vaccination coverage levels among children aged 19–35 months—United States, 2000. MMWR Morb Mortal Wkly Rep (2001) 50:637–41.[Medline]
- National, state, and urban area vaccination coverage levels among children aged 19–35 months—United States, July 1994–June 1995. MMWR Morb Mortal Wkly Rep (1996) 45:508–13.[Medline]
- National, state, and urban area vaccination coverage levels among children aged 19–35 months—United States, April 1994–March 1995. MMWR Morb Mortal Wkly Rep (1996) 45:145–50.[Medline]
- Vaccination coverage levels among children aged 19–35 months—US, April 1994–March 1995. Ann Pharmacother (1996) 30:564–5.[Medline]
- National, state, and urban area vaccination coverage levels among children aged 19–35 months—United States, January–December 1995. MMWR Morb Mortal Wkly Rep (1997) 46:176–82.[Medline]
- National, state, and urban area vaccination coverage levels among children aged 19–35 months—United States, 1997. MMWR Morb Mortal Wkly Rep (1998) 47:547–54.[Medline]
- Vaccination coverage levels among children aged 19–35 months—United States, April–June 1994. MMWR Morb Mortal Wkly Rep (1995) 44:396–8.[Medline]
- National, state, and urban area vaccination levels among children aged 19–35 months—United States, 2002. MMWR Morb Mortal Wkly Rep (2003) 52:728–32.[Medline]
- Barker L, Luman E, Zhao Z, et al. (2002) National, state, and urban area vaccination coverage levels among children aged 19–35 months—United States, 2001. MMWR Morb Mortal Wkly Rep 51:664–6.[Medline]
- Barker LE and Luman ET. (2001) Changes in vaccination coverage estimates among children aged 19–35 months in the United States, 1996–1999. Am J Prev Med 20:28–31.[CrossRef][ISI][Medline]
- National, state, and urban area vaccination coverage among children aged 19–35 months—United States, 2004. MMWR Morb Mortal Wkly Rep (2005) 54:717–21.[Medline]
- Hepatitis A vaccination coverage among children aged 24–35 months—United States, 2003. MMWR Morb Mortal Wkly Rep (2005) 54:141–4.[Medline]
- Smith PJ, Schwartz B, Mokdad A, et al. (2003) The first oral rotavirus vaccine, 1998–1999: estimates of uptake from the National Immunization Survey. Public Health Rep 118:134–43.[ISI][Medline]
- Yusuf HR, Coronado VG, Averhoff FA, et al. (1999) Progress in coverage with hepatitis B vaccine among US children, 1994–1997. Am J Public Health 89:1684–9.
[Abstract/Free Full Text] - Yusuf H, Daniels D, Mast EE, et al. (2001) Hepatitis B vaccination coverage among United States children. Pediatr Infect Dis 20:suppl, S30–3.
- Vaccination coverage by race/ethnicity and poverty level among children aged 19–35 months—United States, 1996. MMWR Morb Mortal Wkly Rep (1997) 46:963–9.[Medline]
- Vaccination coverage by race/ethnicity and poverty level among children aged 19–35 months—United States, 1997. MMWR Morb Mortal Wkly Rep (1998) 47:547–54.[Medline]
- Chu SY, Barker LE, Smith PJ. (2004) Racial/ethnic disparities in preschool immunizations: United States, 1996–2001. Am J Public Health 94:973–7.
[Abstract/Free Full Text] - Daniels D, Jiles RB, Klevens RM, et al. (2001) Undervaccinated African-American preschoolers: a case of missed opportunities. Am J Prev Med 20:61–8.[ISI][Medline]
- Herrera GA, Zhao Z, Klevens RM. (2001) Variation in vaccination coverage among children of Hispanic ancestry. Am J Prev Med 20:69–74.[CrossRef][ISI][Medline]
- Strine TW, Mokdad AH, Barker LE, et al. (2003) Vaccination coverage of American Indian/Alaska Native children aged 19 to 35 months: findings from the National Immunization Survey, 1998–2000. Am J Public Health 93:2046–9.
[Free Full Text] - Vaccination coverage levels among Alaska Native children aged 19–35 months—National Immunization Survey, United States, 2000–2001. MMWR Morb Mortal Wkly Rep (2003) 52:710–13.[Medline]
- Luman ET, McCauley MM, Shefer A, et al. (2003) Maternal characteristics associated with vaccination of young children. Pediatrics 111:1215–18.
[Abstract/Free Full Text] - Strine TW, Luman ET, Okoro CA, et al. (2003) Predictors of age-appropriate receipt of DTaP dose 4. Am J Prev Med 25:45–9.[ISI][Medline]
- Klevens RM and Luman ET. (2001) U.S. children living in and near poverty: risk of vaccine-preventable diseases. Am J Prev Med 20:41–6.[CrossRef][ISI][Medline]
- Strine TW, Barker LE, Mokdad AH, et al. (2002) Vaccination coverage of foreign-born children 19 to 35 months of age: findings from the National Immunization Survey, 1999–2000. (Electronic article). Pediatrics 110: e15.
- Stokley S, Smith PJ, Klevens RM, et al. (2001) Vaccination status of children living in rural areas in the United States: are they protected? Am J Prev Med 20:55–60.<