Skip Navigation


Epidemiologic Reviews Advance Access originally published online on May 17, 2006
Epidemiologic Reviews 2006 28(1):54-70; doi:10.1093/epirev/mxj004
This Article
Right arrow Abstract Freely available
Right arrow FREE Full Text (PDF) Freely available
Right arrow All Versions of this Article:
28/1/54    most recent
mxj004v1
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in ISI Web of Science
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Add to My Personal Archive
Right arrow Download to citation manager
Right arrow Search for citing articles in:
ISI Web of Science (10)
Right arrowRequest Permissions
Google Scholar
Right arrow Articles by Ompad, D. C.
Right arrow Articles by Vlahov, D.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Ompad, D. C.
Right arrow Articles by Vlahov, D.
Social Bookmarking
Add to CiteULike   Add to Connotea   Add to Del.icio.us  
What's this?

Epidemiologic Reviews Copyright © 2006 by the Johns Hopkins Bloomberg School of Public Health All rights reserved; printed in U.S.A.

ARTICLES

Distribution of Influenza Vaccine to High-Risk Groups

Danielle C. Ompad1, Sandro Galea1,2 and David Vlahov1

1 Center for Urban Epidemiologic Studies, New York Academy of Medicine, New York, NY
2 Department of Epidemiology, University of Michigan School of Public Health, Ann Arbor, MI

Correspondence to Dr. Danielle C. Ompad, Center for Urban Epidemiologic Studies, New York Academy of Medicine, 1216 Fifth Avenue, New York, NY 10029 (e-mail: dompad{at}nyam.org).

accepted for publication March 20, 2006.


   ABSTRACT
TOP
 ABSTRACT
INTRODUCTION
 REVIEW OF THE LITERATURE
 FINDINGS
 LIMITATIONS OF THE CURRENT...
 SUMMARY AND FUTURE RESEARCH
 REFERENCES
 
Vaccine distribution programs have historically targeted individuals at high risk of complications due to influenza. Despite recommendations from the Advisory Committee on Immunization Practices, vaccination coverage among high-risk populations has been generally low. This review systematically summarizes the recent literature evaluating programs in different settings, from within medical settings to venue-based and community-based approaches, in an effort to identify successful program components. The published literature was identified by using the MEDLINE database from 1990 to 2006 covering studies that reported on interventions or programs aimed at vaccinating high-risk populations. The authors reviewed 56 studies. In the United States, the Healthy People 2010 goals included 90% vaccination coverage for adults aged ≥65 years and 60% for high-risk adults aged 18–64 years. Only a handful of the studies reviewed managed to meet those goals. Interventions that increased vaccination coverage to Healthy People 2010 goals included advertising, provider and patient mailings, registry-based telephone calls, patient and staff education, standing orders coupled with standardized forms, targeting of syringe exchange customers, and visiting nurses. Few studies evaluated the impact of vaccination programs by race/ethnicity and socioeconomic status. Few studies targeted individuals outside of the health-care and social services sectors. Given the growing disparities in health and health-care access, understanding the way in which interventions can remedy disparities is crucial.

adult • aged • child • immunization • influenza, human • review • vaccination


   INTRODUCTION
TOP
 ABSTRACT
 INTRODUCTION
REVIEW OF THE LITERATURE
 FINDINGS
 LIMITATIONS OF THE CURRENT...
 SUMMARY AND FUTURE RESEARCH
 REFERENCES
 
Influenza is associated with significant morbidity and mortality in the United States. Between the 1990–1991 and 1998–1999 influenza seasons, the rate of influenza-related primary respiratory and circulatory hospitalizations ranged from 62.6 to 159.1 per 100,000 person-years; the rate for primary pneumonia and influenza hospitalizations specifically ranged from 31.0 to 71.4 per 100,000 person-years (1). During the same interval, the influenza-related morality rates were 19.6 per 100,000 person-years for all causes, 13.8 for underlying respiratory and circulatory deaths, and 3.1 for underlying pneumonia and influenza deaths (2). The highest rates of influenza-related mortality are seen among individuals aged 65 years or older: 132.5 per 100,000 person-years for all-cause deaths, 98.3 for underlying respiratory and circulatory deaths, and 22.1 for underlying pneumonia and influenza deaths (2).

Vaccine distribution programs have historically targeted individuals at high risk of complications due to influenza (3). Mathematical modeling suggests that focusing influenza control efforts on high-risk groups (high risk in terms of both susceptibility and contagiousness) can offer a considerable advantage in terms of reduction of infectious disease compared with interventions that target more general populations (4). Most current recommendations suggest prioritizing vaccination for individuals at high risk of complications due to influenza virus infection. The Advisory Committee on Immunization Practices classified high-risk groups to include, but not be limited to, adults aged 65 years or older, children aged 6–23 months, pregnant women, and individuals with chronic medical conditions, especially pulmonary or cardiovascular disorders (5).

Given the higher risk of morbidity and mortality among high-risk groups, the Healthy People 2010 initiative has set an influenza vaccination goal of 90 percent coverage for noninstitutionalized adults aged 65 years or older and 60 percent for noninstitutionalized high-risk adults aged 18–64 years (6). However, data from the 2003 Behavioral Risk Factor Surveillance System study suggest that vaccination coverage is lower: coverage among adults aged 65 years or older was 69.9 percent and among adults aged 18–64 years was 34.0 percent for those with asthma and 49.0 percent for those with diabetes (7). During the 2004–2005 influenza season, when there was a significant vaccine shortage, vaccination coverage was estimated to be 62.7 percent among adults aged 65 years or older and 25.5 percent among adults aged 18–64 years with a high-risk condition (8). Furthermore, influenza vaccination coverage is not evenly distributed. Data from the National Health Interview Study revealed significant differences in influenza vaccination coverage among elderly adults by race/ethnicity, with rates being higher among Whites, followed by Hispanics and Blacks (66 percent vs. 50 percent and 46 percent, respectively) (9).

Rates of vaccination have been low despite recommendations. A variety of settings and approaches have been utilized in the effort to increase vaccination rates among individuals at high risk of complications due to influenza infection. This review systematically summarizes the recent literature evaluating programs in different settings, from within medical settings to venue-based and community-based approaches, in an effort to identify the features of such programs that are most successful and that may guide efforts to increase vaccination rates nationwide. We also considered limitations in the current body of literature, focusing on areas for further research and program development.


   REVIEW OF THE LITERATURE
TOP
 ABSTRACT
 INTRODUCTION
 REVIEW OF THE LITERATURE
FINDINGS
 LIMITATIONS OF THE CURRENT...
 SUMMARY AND FUTURE RESEARCH
 REFERENCES
 
The potential scope of the literature that may be considered relevant to the distribution of influenza vaccination is vast. In light of recent reviews examining immunization delivery methods that have included much of the older literature (10–12), we chose to limit the focus of our review to studies conducted after 1990. Given the large numbers of studies that allude to vaccine delivery methods, we limited our review to studies that were explicitly, at least in part, concerned with evaluating an intervention or program to either distribute or increase uptake of influenza vaccine among individuals at high risk.

The published literature was identified by using the MEDLINE database (National Library of Medicine, Bethesda, Maryland) from 1990 through early 2006, covering both US and international studies that reported on interventions or programs aimed at vaccinating high-risk populations. We considered randomized controlled trials, pre- and postintervention evaluation studies, and simple evaluations with and without controls. The search was limited to English-language studies in biomedical research. Keywords and terms used for the search included primarily the following: influenza, vaccination, intervention, program, distribution, elderly, children, pregnant, diabetes, cardiovascular, cardiovascular disease, chronic obstructive pulmonary disease, asthma, chronic lung disease, and HIV.


   FINDINGS
TOP
 ABSTRACT
 INTRODUCTION
 REVIEW OF THE LITERATURE
 FINDINGS
LIMITATIONS OF THE CURRENT...
 SUMMARY AND FUTURE RESEARCH
 REFERENCES
 
In this review, we move from the medical setting out into community settings and discuss influenza vaccine distribution in five different contexts: 1) hospital/tertiary-care settings, 2) primary-care settings, 3) venue-based (e.g., nursing homes) targeted delivery, 4) large-scale regional programs, and 5) community-based distribution programs. Table 1 provides details for each study, including year of publication, location, setting, sample size and study type (i.e., randomized controlled trial, pre-post evaluation, and simple evaluations with and without controls), intervention content, and study findings.


View this table:
[in this window]
[in a new window]
  		TABLE 1. Key studies that evaluated programs and interventions aimed at increasing influenza vaccination coverage, 1990–2005

 
Hospital/tertiary-care settings

Hospital- and tertiary-care-based programs for increasing influenza vaccination coverage generally focused on the provider and included standing orders for influenza vaccination and reminders to hospital staff. Two studies evaluated the impact of standing orders on vaccination coverage. One study of 761 elderly patients in a tertiary-care hospital that implemented a standing order for influenza vaccination reported that vaccination coverage increased from 45 percent prior to admission to 67 percent after the inpatient intervention; almost 40 percent of unvaccinated patients were vaccinated (13). In a 10-year evaluation of a vaccination program in a veterans' hospital that implemented standing orders and standardized forms, vaccination coverage increased from 79 percent to 86 percent (14).

Reminder systems have also been evaluated. Dexter et al. (15) assessed a computerized reminder system by using a randomized controlled trial among 6,371 patients representing all patients discharged from six general medicine wards in an urban hospital during an 18-month period. Four general medicine teams were randomized to the intervention team and four were randomized to the control team. Vaccination coverage was 51.4 percent among eligible patients in the intervention group compared with 1.0 percent in the control group. Bloom et al. (16) evaluated fax reminders to primary care physicians encouraging vaccination before discharge among 103 physicians serving 153 patients aged 65 years or older; the vaccination rate before discharge was less than 2 percent. Common reasons physicians gave for not vaccinating before discharge included the following: patient vaccinated prior to hospitalization (85 percent), patient discharged prior to completion of treatment (47 percent), and acute-care setting not appropriate to give vaccinations (40 percent). Claims data did not corroborate the reports of past vaccination; for only 24 percent of patients whom physicians identified as previously being vaccinated for pneumococcus or influenza was there a claim in the system of notation in their hospital medical record. However, the authors noted that claims data may have been lagged or physicians may have billed for vaccinations separately and thus no claims were submitted. Despite the limitations of the claims data, the authors concluded it was unlikely that the rate of influenza vaccination prior to admissions was as high as the physician reports suggested.

The structured setting of hospitals and tertiary-care facilities provides an important opportunity for vaccination. Standing orders and reminders appear to improve vaccination rates, but there are two problems. First, while the rates were higher in the intervention groups, they still did not meet the Healthy People 2010 goals, with one exception (14). Second, and perhaps more importantly, these approaches were limited to people hospitalized during the influenza season. In 2003, 13.2 million people aged 65 years or older were discharged from short-stay hospitals (17), representing approximately 38 percent of the US population aged 65 years or older. These facilities have limited ability to reach high-risk individuals.

Primary-care settings

Primary care was the most common setting for studies of mostly multicomponent vaccination programs for high-risk populations, and interventions were directed at the patient, provider, and organization levels. Of 30 studies evaluating interventions in primary-care settings, patient-level components were included in more than 75 percent of interventions. Patient mailings were among the most frequent patient-level program components either alone (18–21) or in combination with other intervention components (14, 22–31). A variety of other patient-level interventions have been evaluated, including telephone reminders (23, 26, 32–35), patient education (36–39), home visits (27, 29, 40, 41), clinic-based advertising (23, 38, 42), vaccination offers during office visits (28), and case management (43). Twelve studies included components aimed at improving provider performance. Provider-level components included provider reminders (22–24, 44), feedback to physicians and staff (22, 44, 45), provider education (23, 28, 36, 37, 43–47), provider mailings (42), and provider e-mails (23). Nine studies used organizational components including nurse-administered vaccines with physicians' order (24), standing orders (14, 25, 37, 46), standardized documentation (14, 45), vaccination clinics (14, 36, 39), and patient registries (43).

Because most programs included multiple components, disentangling the independent effect of specific components is challenging. In studies with a comparison group (e.g., pre-post evaluation or controlled trial), interventions that included only patient mailings increased vaccination coverage by 10–17 percent (18, 20, 21). For example, one randomized controlled trial examined the impact of mailings to parents of children with high-risk conditions that strongly encouraged influenza vaccination for their child (21). For those children who were not vaccinated after the initial letter, a reminder letter was mailed 4 weeks later and a postcard 4 weeks after that. Overall, vaccine coverage was 42 percent among the intervention group and 25 percent among controls (p < 0.001). In studies with a comparison group, vaccination coverage improved by 10–18 percent with standing orders (25), by 7 percent with provider education (47), and by 38 percent with a visiting nurse (41). Three studies evaluated interventions with telephone reminders alone (33–35) or in combination with patient mailings (26, 32), which resulted in 4–6 percent and 10–27 percent increases in vaccination coverage over the comparison groups, respectively. One study examined the impact of home health assessments, where vaccination was offered, and found that those persons who accepted the home health assessment were significantly more likely to have been vaccinated than those who refused the home health assessment (63.7 percent vs. 34.9 percent, p = 0.001) (40).

Most studies contained two or more intervention components. For example, Ahmed et al. (31) used up to two mailed reminders to managed-care organization patients and provided an influenza tool kit to employers, which included educational messages to employers, advertising material targeted to staff (e.g., flyers, newsletter articles, payroll stuffers), and tips and a checklist for employer-sponsored vaccination clinics at work sites. The authors observed only a 4 percent increase in vaccination coverage for two postcards compared with one among adults aged 50–64 years.

Although significant improvements were observed in 24 studies, one exception is noteworthy. Patel et al. (43) evaluated a multidisciplinary asthma disease management program that included a patient registry, asthma therapy assessment, case management, and physician education. A written home treatment plan was provided, which included reminders for influenza vaccination. Although not a primary target of the intervention, influenza vaccination was recommended for individuals with chronic lung disease. The study demonstrated reductions in hospitalizations and emergency department visits (the primary goals); however, vaccination coverage significantly decreased from 24.2 percent at baseline to 15.0 percent at follow-up (p < 0.001). No explanation was provided for the decrease.

Venue-based targeted delivery

An efficient method for vaccinating individuals at high risk of influenza is to target venues frequented by high-risk groups. While there is some overlap with respect to nursing homes, we distinguish these venues from the previously discussed primary- or tertiary-care settings based on location. Primary- and tertiary-care settings were either hospital based or situated in clinics or physician practices. Venues frequented by high-risk groups included nursing homes, which are specialized tertiary-care facilities, as well as senior centers and community-based organizations such as syringe exchange programs. Five such studies were published (30, 48–51) between 1990 and 2006, four of which focused specifically on long-term-care facilities and senior centers (30, 48–50). In a study among the elderly, influenza vaccination was offered to all residents in a nursing home, resulting in an increase in vaccination coverage from 14 percent to 52 percent (50). Kreiger et al. (30) reported on an intervention that utilized a senior center in Seattle, Washington. When patient mailings, telephone calls to the unvaccinated by peers, and computerized vaccination tracking were used, 50 percent of the intervention group that had not been vaccinated the previous year was vaccinated compared with 23 percent of the control group. Patient advertising, including a videotape about the benefits of the influenza vaccination featuring a Colorado First Lady, and patient education increased vaccination coverage from 85 percent to 89 percent in long-term-care facilities (48).

Although not explicitly designated as a group at high risk for influenza complications according to the Advisory Committee on Immunization Practices recommendations, injection drugs users are generally at high risk for a variety of chronic conditions. One study evaluated influenza vaccine distribution at a syringe exchange program in New York City (51). Among 181 people eligible for influenza vaccine, 86 percent accepted and received vaccination. Forty-eight people reported a chronic medical condition, and 87 percent of them were vaccinated.

Large-scale regional programs

Fourteen studies have evaluated large-scale regional vaccination interventions in different populations using a variety of approaches alone or in combination. Thirteen focused on the elderly, and one focused on the elderly and adults with high-risk conditions. Settings included regional public health programs (52–54), targeting of Medicare beneficiaries in a specific region (46, 55–61), and pharmacy-based vaccine distribution (62, 63). In terms of approaches, six studies have used mailings to the target population either alone (53, 57, 60, 64) or in combination with an educational insert or brochure (55, 58) or a media campaign (46, 54). Letters alone increased vaccination coverage 5–33 percent in those studies that had a comparison group (46, 53, 55, 57–60); however, mailed reminders were not always effective in improving coverage (58, 59).

Multicomponent programs were among the most successful at vaccinating high-risk individuals. One study reported on an effort to expand an existing influenza vaccination program by adding urban outreach clinics as well as open clinics at proprietary homes and senior nutrition centers and by enrolling area nursing homes, local hospitals, local home health agencies, and primary-care physicians (56). Special arrangements were made with the local health maintenance organization and individual practice association to enable participation. This demonstration project resulted in an immunization rate of 74.3 percent among 88,881 Medicare beneficiaries aged 65 years or older.

Two studies examined the effectiveness of pharmacy-based vaccination programs in the United States. Grabenstein et al. (63) compared urban chain pharmacies in Washington, where pharmacists could legally administer influenza vaccines, with similar pharmacies in Oregon, where pharmacists could not vaccinate. Although no significant differences were reported, vaccination coverage in Oregon remained stable and increased by 4.7 percent in Washington between 1997 and 1998. Steyer et al. (62) compared vaccination coverage in eight states where pharmacists could administer vaccine with eight states where they could not. Between 1995 and 1999, vaccine coverage increased by 10.7 percent in states where pharmacists could administer vaccine and 3.5 percent in states where they could not. The difference between years was significant, as was the difference between states in 1999.

One study evaluated the impact of home visits to the elderly by a public health nurse on vaccination coverage (52). There were no significant differences in vaccine uptake among the intervention group versus the control group (56.1 percent vs. 56.6 percent). However, significantly more people in the intervention group reported talking about immunization with the public health nurse (42.2 percent vs. 18.2 percent of controls).

Immunization distribution programs involving active community engagement

Program descriptions thus far have been medicine or public health directed. Another approach involves community–academic–local health department partnerships. Partnerships between researchers and community representatives that facilitate participation in defining the research problem, interpreting the data, and applying the findings can help address trust issues and translation of research into practice and policy (65). Four studies have used community engagement techniques to implement vaccination programs (66–69). In their program aimed at increasing influenza vaccination coverage among indigenous adults in Australia, Hanna et al. (66) involved key stakeholders in early planning and promotion and recruited indigenous public health workers to promote the program and develop materials.

Community-wide programs are less commonly reported. Weatherill et al. (67) evaluated a large-scale vaccination "blitz" in an inner-city neighborhood of 10 square blocks with 12,000 injection drug users who lived or spent time there. The researchers held an informational stakeholder meeting, which included local agencies, physicians, clinic representatives, and large businesses, to apprise the community of the program. Resident volunteers were recruited for distribution. Distribution sites included single room occupancy hotels, soup kitchens, food banks, community agencies, needle exchanges, drop-in centers, pubs, medical clinics, jails or pretrial centers, parks, streets, and alleys. Influenza vaccines were distributed to 8,043 people in 1999, 3,718 in 2000, 5,175 in 2001, and 4,131 in 2002. A decrease in emergency department visits for pneumonia was noted in 2000, although this reduction was confounded by differences in the timing and magnitude of the 1999 and 2000 influenza epidemics.

Two studies used a more community-directed approach. Working with faith-based neighborhood health centers, Zimmerman et al. (69) evaluated interventions aimed at vaccinating inner-city adults aged 50 years or older. The researchers offered a selection of intervention components derived from the US Task Force on Community Preventive Services (70) to two faith-based centers. Centers chose components that best suited their particular circumstances. Both centers selected examination room posters, free or low-cost vaccines for indigent adults, staff education, chart reminders, and standing orders. One center (center A) also chose to use mailings, while the other (center B) chose community advertisement and off-site vaccination clinics. There were no significant differences in vaccination rates before and after the intervention. Vaccination rates were 59 percent for center A and 49 percent for center B; this difference was not statistically significant. Adults aged 65 years or older were significantly more likely to be vaccinated compared with those aged 50–64 years (65 percent vs. 47 percent). In a similar study, Zimmerman et al. (68) worked with urban health centers to target children less than 2 years of age for the recommended initial two doses of influenza vaccine. The researchers again asked their community partners to choose intervention components that best suited their needs. Postintervention vaccination coverage increased from 6.5 percent to 38.5 percent for the first dose and from 1.9 percent to 13.2 percent for the second dose compared with preintervention. Including influenza vaccine did not delay receipt of other childhood vaccines.


   LIMITATIONS OF THE CURRENT LITERATURE
TOP
 ABSTRACT
 INTRODUCTION
 REVIEW OF THE LITERATURE
 FINDINGS
 LIMITATIONS OF THE CURRENT...
SUMMARY AND FUTURE RESEARCH
 REFERENCES
 
In evaluating interventions to vaccination, several methodological issues, limitations, and gaps should be considered. Methodologically, we note that enumerating the target group and collecting data on vaccination rates is challenging. In epidemiologic studies, clear enumeration of the target group is necessary for estimating vaccination coverage. Studies conducted in health-care settings, or with the use of governmental lists (e.g., Medicare recipients and municipal records) are able to accurately determine vaccination coverage insofar as these lists are complete. However, a number of important subpopulations may not be reflected in these enumerations, including the homeless and undocumented immigrants. Studies that use community settings are often challenged by the need to precisely enumerate their target populations.

Another methodological challenge is accurate documentation of vaccination rates. For studies involving medical claims data, these data may be lagged and physicians may bill for vaccinations separately, as reported earlier (16). Other studies have relied on patient or parent self-report. Zimmerman et al. (71) demonstrated that these methods can be problematic: sensitivity was 98 percent and specificity was 38 percent for self-reported influenza vaccination among elderly outpatients and was 85.7 percent and 66 percent, respectively, among parents of children aged 6–23 months (72).

The current literature is limited with respect to comprehensive coverage of high-risk groups and key subpopulations. Of 56 studies examined, 40 (71.4 percent) targeted the elderly, 14 (25.0 percent) were directed at adults with high-risk conditions, and five (8.9 percent) targeted children. More than half of the studies (n = 30) occurred in primary-care settings, 14 (25.0 percent) were large-scale regional programs, four (7.1 percent) were in tertiary-care facilities or hospitals, four (7.1 percent) were targeted to nursing homes or long-term-care facilities, three (5.4 percent) included active community engagement, and two (3.6 percent) were targeted to specific communities. Thus, most studies examined vaccination within the context of primary-care settings or large-scale regional programs. An important limitation of these types of approaches is their inability to reach those people not engaged in the health-care system. Given the number of Americans without health insurance, the need for alternative methods for providing influenza vaccinations is clear. Beyond lack of health insurance, a variety of other personal and structural barriers may limit appropriate vaccination coverage, including mistrust of modern medicine or the government, availability, convenience, and concern about side effects (73–77).

Targeted efforts to deliver influenza vaccine to high-risk groups have resulted in higher influenza vaccination rates among persons who are most at risk of influenza compared with the general population; however, vaccination rates in these target groups are not optimal. There are substantial disparities between and within high-risk groups. For example, data from the 1999 National Health Interview Survey reveal that Black adults with high-risk conditions such as diabetes, chronic heart disease, and cancer, compared with White adults with the same conditions, had a significantly lower prevalence of influenza vaccination (78). Although increasing attention has been given to racial/ethnic and socioeconomic status disparities in health and health-care access, only seven of 54 studies reviewed considered race/ethnicity or socioeconomic status characteristics with respect to vaccination uptake. One study presented influenza immunization data for the United States and for Louisiana by race but did not examine it as a correlate of vaccine uptake in their intervention (57). Three studies reported no difference in vaccination status by race (62, 69, 79), and one study showed a decreased likelihood for vaccination among Blacks (62). Weaver et al. (42) found that vaccine recipients were significantly more likely to be White, but possible explanations for the differences were not reported. Zimmerman et al. (68) reported differences in vaccine uptake by race/ethnicity in their study of inner-city children, but they were cautious in drawing conclusions because race/ethnicity was not available for a substantial number of study participants.

Only one study examined socioeconomic status in terms of vaccination uptake. Spaulding and Kugler (18) found that enlisted military beneficiaries were less likely than officers to be vaccinated. Our review of the literature suggests that there is a substantial gap in the literature with respect to reporting vaccination rates for interventions by race/ethnicity and/or socioeconomic status. Given that disparities in vaccination continue to be documented, understanding how programs address, or do not address, the needs of these groups is essential.

Low vaccination coverage among minorities and persons living in and near poverty is a persistent problem. Although there is a paucity of empiric evidence in this regard, previous infectious disease epidemics have been accompanied by concerns about the possible spread of disease through populations with relatively low vaccination rates. In the last smallpox epidemic in the United States (1901–1903 in Boston, Massachusetts), there were 1,596 recorded cases of smallpox (with a case fatality rate of 17 percent). A concerted effort by public health officials to forcibly vaccinate all homeless persons in Boston, while controversial, preceded resolution of the smallpox epidemic at the time (80). Vaccination rates for most vaccine-preventable diseases are particularly low among marginalized, difficult-to-reach (or "hidden") populations (e.g., injection drug users, elderly shut-ins) within disadvantaged urban communities (81–83). Few studies have made concerted attempts to vaccinate difficult-to-reach populations for influenza, such as the homeless, substance users, elderly shut-ins, and undocumented immigrants. With few notable exceptions, the interventions we reviewed engaged people already formerly connected to health-care or social services. Two studies stand out: Stancliff et al. (51) offered vaccine at a syringe exchange program, and Weatherill et al. (67) offered vaccine in a variety of nontraditional settings.

Most interventions that we reviewed were not community based but relied on programs that were professionally directed and administered. However, community-based participatory research is emerging as one of the more effective methods for addressing health disparities (84). Recent reviews of intervention studies have overwhelmingly concluded that population-level intervention research should include an expanded appreciation of social, environmental, and health policy components of health promotion (85, 86). New directions that have been suggested for community intervention studies consist of involving a diverse team including community members in program planning and implementation (87). We reviewed five studies that involved the community, all of which occurred after 2000. Moving forward, we anticipate that more programs will begin to use community-based participatory research as a powerful tool to improve health.


   SUMMARY AND FUTURE RESEARCH
TOP
 ABSTRACT
 INTRODUCTION
 REVIEW OF THE LITERATURE
 FINDINGS
 LIMITATIONS OF THE CURRENT...
 SUMMARY AND FUTURE RESEARCH
REFERENCES
 
Between 1990 and 2006, a number of studies examined interventions to increase influenza vaccination among high-risk populations. In the United States, a goal has been set to ensure 90 percent vaccination coverage for adults aged 65 years or older and 60 percent for high-risk adults aged 18–64 years (6). Only a handful of the studies we reviewed managed to meet that goal for their target populations. Interventions that increased vaccination rates to the goals outlined by Healthy People 2010 included advertising (42, 48), provider (42) and patient (14, 42) mailings, registry-based telephone calls (35), patient and staff education (48), standing orders coupled with standardized forms (14), targeting of syringe exchange customers (51), and visiting nurses (41).

During the 2004–2005 influenza season, coverage was estimated at 62.7 percent for people aged 65 years or older and 25.5 percent for people aged 18–64 years with a high-risk condition (8), which was lower than for the 2003 influenza season, where coverage was 69.9 percent among adults aged 65 years or older and, among adults aged 18–64 years, 34.0 percent for those with asthma and 49.0 percent for those with diabetes (7). Although there was a substantial influenza vaccine shortage in 2004–2005, those coverage rates highlight the effort still needed to meet national goals.

This review has focused on those at high risk of complications from influenza; however, those at high risk of transmitting influenza are also an important target group for vaccination programs. For example, epidemiologic models (88) and several Japanese studies have suggested that vaccinating schoolchildren for influenza can reduce morbidity and mortality among adults (89, 90). The Advisory Committee on Immunization Practices does not currently recommend influenza vaccination for healthy schoolchildren (5).

There are two primary areas in which future research can make a valuable contribution. First, there is a paucity of research that evaluates the impact of vaccination programs by race/ethnicity and socioeconomic status. Given the growing disparities in health and health-care access among racial/ethnic minorities and individuals with low socioeconomic status attainment, understanding the way in which interventions can remedy these differences is crucial. Second, few studies have targeted individuals outside of the health-care and social services sectors. To reach those who do not have health insurance and are not connected to social services, interventions that move away from these settings are needed.


   ACKNOWLEDGMENTS
 
This study was supported, in part, by a grant from the National Institute on Drug Abuse (DA017004).

Conflict of interest: none declared.


   REFERENCES
TOP
 ABSTRACT
 INTRODUCTION
 REVIEW OF THE LITERATURE
 FINDINGS
 LIMITATIONS OF THE CURRENT...
 SUMMARY AND FUTURE RESEARCH
 REFERENCES
 

  1. Thompson WW, Shay DK, Weintraub E, et al. (2004) Influenza-associated hospitalizations in the United States. JAMA 292:1333–40.[Abstract/Free Full Text]
  2. Thompson WW, Shay DK, Weintraub E, et al. (2003) Mortality associated with influenza and respiratory syncytial virus in the United States. JAMA 289:179–86.[Abstract/Free Full Text]
  3. Dowdle WR, Coleman MT, Gregg MB. (1974) Natural history of influenza type A in the United States, 1957–1972. Prog Med Virol 17:91–135.[ISI][Medline]
  4. Koopman JS, Simon CP, Riolo CP. (2005) When to control endemic infections by focusing on high-risk groups. Epidemiology 16:621–7.[CrossRef][ISI][Medline]
  5. Harper SA, Fukuda K, Uyeki TM, et al. (2005) Prevention and control of influenza: Recommendations of the Advisory Committee on Immunization Practices (ACIP). MMWR Recomm Rep 54:1–40.[Medline]
  6. US Department of Health and Human Services. (2000) Healthy people 2010: understanding and improving health. 2nd ed. (US Government Printing Office, Washington, DC).
  7. Influenza and pneumococcal vaccination coverage among persons aged > or =65 years and persons aged 18–64 years with diabetes or asthma—United States, 2003. MMWR Morb Mortal Wkly Rep (2004) 53:1007–12.[Medline]
  8. Estimated influenza vaccination coverage among adults and children—United States, September 1, 2004–January 31, 2005. MMWR Morb Mortal Wkly Rep (2005) 54:304–7.[Medline]
  9. Rangel MC, Shoenbach VJ, Weigle KA, et al. (2005) Racial and ethnic disparities in influenza vaccination among elderly adults. J Gen Intern Med 20:426–31.[CrossRef][ISI][Medline]
  10. Gyorkos TW, Tannenbaum TN, Abrahamowicz M, et al. (1994) Evaluation of the effectiveness of immunization delivery methods. Can J Public Health 85:suppl_1S14–30.
  11. McKibben LJ, Stange PV, Sneller VP, et al. (2000) Use of standing orders programs to increase adult vaccination rates. MMWR Recomm Rep 49:15–16.[Medline]
  12. Szilagyi P, Vann J, Bordley C, et al. (2002) Interventions aimed at improving immunization rates. Cochrane Database Syst Rev CD003941.
  13. Lawson F, Baker V, Au D, et al. (2000) Standing orders for influenza vaccination increased vaccination rates in inpatient settings compared with community rates. J Gerontol A Biol Sci Med Sci 55:M522–M526.[Abstract/Free Full Text]
  14. Nichol KL. (1998) Ten-year durability and success of an organized program to increase influenza and pneumococcal vaccination rates among high-risk adults. Am J Med 105:385–92.[CrossRef][ISI][Medline]
  15. Dexter PR, Perkins S, Overhage JM, et al. (2001) A computerized reminder system to increase the use of preventive care for hospitalized patients. N Engl J Med 345:965–70.[Abstract/Free Full Text]
  16. Bloom HG, Wheeler DA, Linn J. (1999) A managed care organization's attempt to increase influenza and pneumococcal immunizations for older adults in an acute care setting. J Am Geriatr Soc 47:106–10.[ISI][Medline]
  17. DeFrances CJ, Hall MJ, Podgornik MN. (2005) 2003 National Hospital Discharge Survey: advance data from vital and health statistics. (National Center for Health Statistics, Hyattsville, MD)1–20 (Report no. 359). (http://www.cdc.gov/nchs/data/ad/ad359.pdf).
  18. Spaulding SA and Kugler JP. (1991) Influenza immunization: the impact of notifying patients of high-risk status. J Fam Pract 33:495–8.[ISI][Medline]
  19. Hutchinson HL and Norman LA. (1995) Compliance with influenza immunization: a survey of high-risk patients at a family medicine clinic. J Am Board Fam Pract 8:448–51.[Medline]
  20. Satterthwaite P. (1997) A randomised intervention study to examine the effect on immunisation coverage of making influenza vaccine available at no cost. N Z Med J 110:58–60.[ISI][Medline]
  21. Daley MF, Barrow J, Pearson K, et al. (2004) Identification and recall of children with chronic medical conditions for influenza vaccination. Pediatrics 113: e26–e33.
  22. Barton MB and Schoenbaum SC. (1990) Improving influenza vaccination performance in an HMO setting: the use of computer-generated reminders and peer comparison feedback. Am J Public Health 80:534–6.[Abstract/Free Full Text]
  23. Nowalk MP, Lin CJ, Zimmerman RK, et al. (2005) Tailored interventions to introduce influenza vaccination among 6- to 23-month-old children at inner-city health centers. Am J Manag Care 11:717–24.[ISI][Medline]
  24. Nichol KL, Korn JE, Margolis KL, et al. (1990) Achieving the national health objective for influenza immunization: success of an institution-wide vaccination program. Am J Med 89:156–60.[CrossRef][ISI][Medline]
  25. Margolis KL, Nichol KL, Wuorenma J, et al. (1992) Exporting a successful influenza vaccination program from a teaching hospital to a community outpatient setting. J Am Geriatr Soc 40:1021–3.[ISI][Medline]
  26. Gaglani M, Riggs M, Kamenicky C, et al. (2001) A computerized reminder strategy is effective for annual influenza immunization of children with asthma or reactive airway disease. Pediatr Infect Dis J 20:1155–60.[CrossRef][ISI][Medline]
  27. Arthur AJ, Matthews RJ, Jagger C, et al. (2002) Improving uptake of influenza vaccination among older people: a randomised controlled trial. Br J Gen Pract 52:717–18 720–2.[ISI][Medline]
  28. Birchmeier M, Favrat B, Pécoud A, et al. (2002) Improving influenza vaccination rates in the elderly. J Fam Pract 51:856.[Medline]
  29. Nuttall D. (2003) The influence of health professionals on the uptake of the influenza immunization. Br J Community Nurs 8:391–6.[Medline]
  30. Krieger JW, Castorina JS, Walls ML, et al. (2000) Increasing influenza and pneumococcal immunization rates: a randomized controlled study of a senior center-based intervention. Am J Prev Med 18:123–31.[CrossRef][ISI][Medline]
  31. Ahmed F, Friedman C, Franks A, et al. (2004) Effect of the frequency of delivery of reminders and an influenza tool kit on increasing influenza vaccination rates among adults with high-risk conditions. Am J Manag Care 10:698–702.[ISI][Medline]
  32. Kellerman RD, Allred CT, Frisch LE. (2000) Enhancing influenza immunization. Postcard and telephone reminders and the challenge of immunization site shift. Arch Fam Med 9:368–72.[Abstract/Free Full Text]
  33. Crawford AG, Sikirica V, Goldfarb N, et al. (2005) Interactive voice response reminder effects on preventive service utilization. Am J Med Qual 20:329–36.[Abstract/Free Full Text]
  34. Hull S, Hagdrup N, Hart B, et al. (2002) Boosting uptake of influenza immunisation: a randomised controlled trial of telephone appointing in general practice. Br J Gen Pract 52:712–16.[ISI][Medline]
  35. Kempe A, Daley MF, Barrow J, et al. (2005) Implementation of universal influenza immunization recommendations for healthy young children: results of a randomized, controlled trial with registry-based recall. Pediatrics 115:146–54.[Abstract/Free Full Text]
  36. Knoell KR and Leeds AL. (1991) Influenza vaccination program for elderly outpatients. Am J Hosp Pharm 48:256–9.[Abstract]
  37. Herman CJ, Speroff T, Cebul RD. (1994) Improving compliance with immunization in the older adult: results of a randomized cohort study. J Am Geriatr Soc 42:1154–9.[ISI][Medline]
  38. Van Amburgh JA, Waite NM, Hobson EH, et al. (2001) Improved influenza vaccination rates in a rural population as a result of a pharmacist-managed immunization campaign. Pharmacotherapy 21:1115–22.[CrossRef][ISI][Medline]
  39. Pappano D, Humiston S, Goepp J. (2004) Efficacy of a pediatric emergency department-based influenza vaccination program. Arch Pediatr Adolesc Med 158:1077–83.[Abstract/Free Full Text]
  40. Arthur AJ. (2001) The effect of health assessments by practice nurses on uptake of influenza vaccination among older people in the UK. J Clin Nurs 10:716–17.[CrossRef][ISI][Medline]
  41. Dalby DM, Sellors JW, Fraser FD, et al. (2000) Effect of preventive home visits by a nurse on the outcomes of frail elderly people in the community: a randomized controlled trial. CMAJ 162:497–500.[Abstract/Free Full Text]
  42. Weaver FM, Goldstein B, Evans CT, et al. (2003) Influenza vaccination among veterans with spinal cord injury: part 2. Increasing vaccination rates. J Spinal Cord Med 26:210–18.[ISI][Medline]
  43. Patel PH, Welsh C, Foggs MB. (2004) Improved asthma outcomes using a coordinated care approach in a large medical group. Dis Manag 7:102–11.[CrossRef][Medline]
  44. Humair JP, Buchs CR, Stalder H. (2002) Promoting influenza vaccination of elderly patients in primary care. Fam Pract 19:383–9.[Abstract/Free Full Text]
  45. Jans MP, Schellevis FG, Van HW, et al. (2000) Improving general practice care of patients with asthma or chronic obstructive pulmonary disease: evaluation of a quality system. Eff Clin Pract 3:16–24.[Medline]
  46. Ohmit SE, Furumoto-Dawson A, Monto AS, et al. (1995) Influenza vaccine use among an elderly population in a community intervention. Am J Prev Med 11:271–6.[ISI][Medline]
  47. Kerse NM, Flicker L, Jolley D, et al. (1999) Improving the health behaviours of elderly people: randomised controlled trial of a general practice education programme. BMJ 319:683–7.[Abstract/Free Full Text]
  48. Schluter WW, Ralston DL, Delaney RJ, et al. (1999) Increasing influenza and pneumococcal vaccination and tuberculosis screening among residents of Colorado long-term care facilities. Eval Health Prof 22:466–83.[Abstract/Free Full Text]
  49. Bardenheier BH, Shefer A, McKibben L, et al. (2005) Factors predictive of increased influenza and pneumococcal vaccination coverage in long-term care facilities: The CMS-CDC Standing Orders Program Project. J Am Med Dir Assoc 6:291–9.[CrossRef][Medline]
  50. Hutt E, Reznickova N, Morgenstern N, et al. (2004) Improving care for nursing home-acquired pneumonia in a managed care environment. Am J Manag Care 10:681–6.[ISI][Medline]
  51. Stancliff S, Salomon N, Perlman DC, et al. (2000) Provision of influenza and pneumococcal vaccines to injection drug users at a syringe exchange. J Subst Abuse Treat 18:263–5.[CrossRef][ISI][Medline]
  52. Black ME, Ploeg J, Walter SD, et al. (1993) The impact of a public health nurse intervention on influenza vaccine acceptance. Am J Public Health 83:1751–3.[Abstract/Free Full Text]
  53. Honkanen PO, Keistinen T, Kivela SL. (1997) The impact of vaccination strategy and methods of information on influenza and pneumococcal vaccination coverage in the elderly population. Vaccine 15:317–20.[CrossRef][ISI][Medline]
  54. Christenson B, Lundbergh P, Hedlund J, et al. (2001) Effects of a large-scale intervention with influenza and 23-valent pneumococcal vaccines in adults aged 65 years or older: a prospective study. Lancet 357:1008–11.[CrossRef][ISI][Medline]
  55. Increasing influenza vaccination rates for Medicare beneficiaries—Montana and Wyoming, 1994. MMWR Morb Mortal Wkly Rep (1995) 44:744–6.[Medline]
  56. Bennett NM, Lewis B, Doniger AS, et al. (1994) A coordinated, communitywide program in Monroe County, New York, to increase influenza immunization rates in the elderly. Arch Intern Med 154:1741–5.[Abstract]
  57. Kumar S, Deichmann RE, Sarkar I. (1999) Effect of physician-specific mailouts aimed at increasing influenza immunization rates. J La State Med Soc 151:558–65.[Medline]
  58. Smith DM, Zhou XH, Weinberger M, et al. (1999) Mailed reminders for area-wide influenza immunization: a randomized controlled trial. J Am Geriatr Soc 47:1–5.[ISI][Medline]
  59. Chan L, MacLehose RF, Houck PM. (2002) Impact of physician reminders on the use of influenza vaccinations: a randomized trial. Arch Phys Med Rehabil 83:371–5.[CrossRef][ISI][Medline]
  60. McCaul KD, Johnson RJ, Rothman AJ. (2002) The effects of framing and action instructions on whether older adults obtain flu shots. Health Psychol 21:624–8.[CrossRef][ISI][Medline]
  61. Hannah KL, Schade CP, Cochran R, et al. (2005) Promoting influenza and pneumococcal immunization in older adults. Jt Comm J Qual Patient Saf 31:286–93.[Medline]
  62. Steyer TE, Ragucci KR, Pearson WS, et al. (2004) The role of pharmacists in the delivery of influenza vaccinations. Vaccine 22:1001–6.[CrossRef][ISI][Medline]
  63. Grabenstein JD, Guess HA, Hartzema AG, et al. (2001) Effect of vaccination by community pharmacists among adult prescription recipients. Med Care 39:340–8.[CrossRef][ISI][Medline]
  64. Luthi JC, Mean F, Ammon C, et al. (2002) Evaluation of a population-based prevention program against influenza among Swiss elderly people. Swiss Med Wkly 132:592–7.[Medline]
  65. Leung MW, Yen IH, Minkler M. (2004) Community based participatory research: a promising approach for increasing epidemiology's relevance in the 21st century. Int J Epidemiol 33:499–506.[Abstract/Free Full Text]
  66. Hanna JN, Young DM, Brookes DL, et al. (2001) The initial coverage and impact of the pneumococcal and influenza vaccination program for at-risk indigenous adults in Far North Queensland. Aust N Z J Public Health 25:543–6.[ISI][Medline]
  67. Weatherill SA, Buxton JA, Daly PC. (2004) Immunization programs in non-traditional settings. Can J Public Health 95:133–7.[ISI][Medline]
  68. Zimmerman RK, Hoberman A, Nowalk MP, et al. (2004) Feasibility of influenza immunization for inner-city children aged 6 to 23 months. Am J Prev Med 27:397–403.[CrossRef][ISI][Medline]
  69. Zimmerman RK, Nowalk MP, Raymund M, et al. (2003) Tailored interventions to increase influenza vaccination in neighborhood health centers serving the disadvantaged. Am J Public Health 93:1699–705.[Abstract/Free Full Text]
  70. Task Force on Community Preventive Services. (2000) Recommendations regarding interventions to improve vaccination coverage in children, adolescents, and adults. Am J Prev Med 18:92–6.[CrossRef][ISI][Medline]
  71. Zimmerman RK, Raymund M, Janosky JE, et al. (2003) Sensitivity and specificity of patient self-report of influenza and pneumococcal polysaccharide vaccinations among elderly outpatients in diverse patient care strata. Vaccine 21:1486–91.[CrossRef][ISI][Medline]
  72. Nowalk MP, Zimmerman RK, Lin CJ, et al. (2005) Parental perspectives on influenza immunization of children aged 6 to 23 months. Am J Prev Med 29:210–14.[CrossRef][ISI][Medline]
  73. Qureshi AM, Hughes NJ, Murphy E, et al. (2004) Factors influencing uptake of influenza vaccination among hospital-based health care workers. Occup Med (Lond) 54:197–201.[Medline]
  74. Telford R and Rogers A. (2003) What influences elderly peoples' decisions about whether to accept the influenza vaccination? A qualitative study. Health Educ Res 18:743–53.[Abstract/Free Full Text]
  75. Shui I, Kennedy A, Wooten K, et al. (2005) Factors influencing African-American mothers' concerns about immunization safety: a summary of focus group findings. J Natl Med Assoc 97:657–66.[Medline]
  76. Burnett M, Genao I, Wong WF. (2005) Race, culture, and trust: why should I take a shot if I'm not sick? Ethn Dis 15:2 suppl. 313–16.
  77. Salmon DA, Moulton LH, Omer SB, et al. (2005) Factors associated with refusal of childhood vaccines among parents of school-aged children: a case-control study. Arch Pediatr Adolesc Med 159:470–6.[Abstract/Free Full Text]
  78. Egede LE and Zheng D. (2003) Racial/ethnic differences in influenza vaccination coverage in high-risk adults. Am J Public Health 93:2074–8.[Abstract/Free Full Text]
  79. Sorvillo FJ and Nahlen BL. (1995) Influenza immunization for HIV-infected persons in Los Angeles. Vaccine 13:377–80.[CrossRef][ISI][Medline]
  80. Albert MR, Ostheimer KG, Breman JG. (2001) The last smallpox epidemic in Boston and the vaccination controversy, 1901–1903. N Engl J Med 344:375–9.[Free Full Text]
  81. Seal KH, Ochoa KC, Hahn JA, et al. (2000) Risk of hepatitis B infection among young injection drug users in San Francisco: opportunities for intervention. West J Med 172:16–20.[CrossRef][ISI][Medline]
  82. Levine OS, Vlahov D, Koehler J, et al. (1995) Seroepidemiology of hepatitis B virus in a population of injecting drug users. Association with drug injection patterns. Am J Epidemiol 142:331–41.[Abstract/Free Full Text]
  83. Solomon L, Frank R, Vlahov D, et al. (1991) Utilization of health services in a cohort of intravenous drug users with known HIV-1 serostatus. Am J Public Health 81:1285–90.[Abstract/Free Full Text]
  84. Israel BA, Schulz AJ, Parker EA, et al. (1998) Review of community-based research: assessing partnership approaches to improve public health. Annu Rev Public Health 19:173–202.[CrossRef][ISI][Medline]
  85. Fortmann SP, Flora JA, Winkleby MA, et al. (1995) Community intervention trials: reflections on the Stanford Five-City Project Experience. Am J Epidemiol 142:576–86.[Abstract/Free Full Text]
  86. Sorensen G, Emmons K, Hunt MK, et al. (1998) Implications of the results of community intervention trials. Annu Rev Public Health 19:379–416.[CrossRef][ISI][Medline]
  87. Lindenberg CS, Solorzano RM, Vilaro FM, et al. (2001) Challenges and strategies for conducting intervention research with culturally diverse populations. J Transcult Nurs 12:132–9.[Abstract/Free Full Text]
  88. Longini IM Jr and Halloran ME. (2005) Strategy for distribution of influenza vaccine to high-risk groups and children. Am J Epidemiol 161:303–6.[Abstract/Free Full Text]
  89. Reichert TA, Sugaya N, Fedson DS, et al. (2001) The Japanese experience with vaccinating schoolchildren against influenza. N Engl J Med 344:889–96.[Abstract/Free Full Text]
  90. Reichert TA. (2002) The Japanese program of vaccination of schoolchildren against influenza: implications for control of the disease. Semin Pediatr Infect Dis 13:104–11.[CrossRef][Medline]
  91. Barker WH, Bennett NM, LaForce FM, et al. (1999) "McFlu". The Monroe County, New York, Medicare vaccine demonstration. Am J Prev Med 16:118–27.[CrossRef][ISI][Medline]

Add to CiteULike CiteULike   Add to Connotea Connotea   Add to Del.icio.us Del.icio.us    What's this?


This article has been cited by other articles:


Home page
 Am. J. Public HealthHome page
M. H. Coady, S. Galea, S. Blaney, D. C. Ompad, S. Sisco, D. Vlahov, and for the Project Viva Intervention Working Group
Project VIVA: A Multilevel Community-Based Intervention to Increase Influenza Vaccination Rates Among Hard-to-Reach Populations in New York City
Am J Public Health, July 1, 2008; 98(7): 1314 - 1321.
[Abstract] [Full Text] [PDF]

Home page
 Eur J Public HealthHome page
A. Lopez-de-Andres, P. Carrasco-Garrido, V. Hernandez-Barrera, A. G. de Miguel, and R. Jimenez-Garcia
Coverages and factors associated with influenza vaccination among subjects with chronic respiratory diseases in Spain
Eur J Public Health, April 1, 2008; 18(2): 173 - 177.
[Abstract] [Full Text] [PDF]

Home page
 Epidemiol RevHome page
M. A. Ibrahim
Editorial: Vaccines and Public Health
Epidemiol. Rev., August 1, 2006; 28(1): 1 - 2.
[Full Text] [PDF]

Home page
 Am J EpidemiolHome page
A. R. Hinman, W. A. Orenstein, and L. E. Rodewald
Vaccines--Victories and Challenges
Am. J. Epidemiol., August 1, 2006; 164(3): 197 - 199.
[Full Text] [PDF]

This Article
Right arrow Abstract Freely available
Right arrow FREE Full Text (PDF) Freely available