Mary Patricia Nowalk, PhD, RD

Together, influenza and pneumonia are the sixth leading cause of death. Each year, influenza causes approximately 20,000 deaths, and pneumococcus causes approximately 500,000 cases of pneumonia, 50,000 cases of bacteremia, and 3000 cases of meningitis.^1,2 African Americans experience approximately twice the rate of invasive pneumococcal disease as whites.^3,4

Despite the burden of disease and the availability of vaccine guidelines,^1,2,5-7 vaccination rates are only modest, though they are slowly rising. In 1997, only 65% and 45% of persons 65 years or older reported receiving influenza and pneumococcal vaccines, respectively.⁸ Influenza vaccination rates were lower for persons of Hispanic (58%) and non-Hispanic black (50%) origin than for non-Hispanic whites (67%). Pneumococcal vaccination rates were 34% for Hispanics, 30% for non-Hispanic blacks, and 47% for non-Hispanic whites.⁸

The low rates are surprising, given that there is an abundance of literature reporting successful interventions for increasing immunization rates.^9,10 In a meta-analysis, system-oriented interventions (eg, standing orders for nurses) resulted in pooled rate increases of 39% and 45% for influenza and pneumococcal vaccines, respectively.⁹ Patient-oriented strategies (eg, postcard reminders that influenza or pneumococcal vaccine was due) resulted in increases of 12% and 75%, respectively. Provider-oriented strategies (eg, chart reminders) resulted in increases of 18% and 8%, respectively, for influenza and pneumococcal vaccines.⁹

Causes of low immunization rates

Although immunization rates are slowly increasing, why have they not risen more, given the evidence of effective interventions? We believe that there are 4 primary reasons why many clinical practices have not successfully applied research findings to improve adult vaccination rates.

First, offices are complex systems with idiosyncratic organization structures and values. They tend to accept changes when they are congruent with the organization’s goals and culture. Most previous efforts at intervention treated practices uniformly, with a “one size fits all” approach.^2,11 However, data from the Direct Observation of Primary Care Study reveals a wide variety of patients and problems.^12-14 Several authors have recommended tailoring interventions to match the organizational structure, office culture, and individual physician philosophies and practices as a means of increasing the likelihood of success.^11,12,15,16

Also, patient beliefs about adult vaccination are varied and include racial and ethnic diversity as well. Significant percentages of the elderly report lack of awareness of the need for immunizations (19% for influenza vaccination and 57% for pneumococcal vaccination).^17-19 Among racial groups, non-Hispanic blacks were least aware of the need for these vaccines, followed by Hispanics and non-Hispanic whites.¹⁷ Concern that vaccination may actually cause disease^17,20 and fear of the pain of injection and/or needles^17,21,22 lead many to decline vaccination. Although serious adverse events due to vaccination are rare, media attention to them increases public awareness of their occurrence and may contribute to fear of adverse reactions.^17,20-23

Time pressures on physicians also distract attention from prevention. Zyzanski and colleagues²⁴ found that physicians seeing high volumes of patients, in comparison to those with low volumes, had visits that were 30% shorter, scheduled fewer patients for well-care visits, delivered fewer preventive screenings, and gave fewer immunizations.

Finally, the responsibility for adult immunization has not been definitively assigned, resulting in fewer programmatic efforts. Many groups have an interest in adult immunization; however, coordination is limited, causing immunization messages to become diffuse. As a result, many providers caring for adults do not see vaccination as their responsibility.

The cumulative effect of these factors is that, despite access to medical care, many of the adults at high risk for vaccine-preventable diseases remain unvaccinated.²⁵

Research Questions

• Several research questions emerge from this scenario:
• What are the influenza and pneumococcal vaccination rates among persons 65 years and older of both majority and minority populations?
• What are the internal structure and office culture of various medical practices, and how do they facilitate or inhibit adult immunizations?
• What are providers’ attitudes, knowledge, and practices regarding adult immunizations?
• What are patients’ attitudes, knowledge, and beliefs regarding influenza and pneumococcal immunizations?
• What are the relationships among patient and provider knowledge, attitudes, beliefs, and practices and their impact on adult influenza and pneumococcal immunization rates?
• To answer these questions, a large multicomponent study with a variety of physician practice types and patient populations is required. Also, both quantitative and qualitative data need to be collected. In this article, we will describe the development of the methods used to answer these research questions.

Methods

Theoretical Framework and Models

To design our questionnaires, we used data from the literature, observations of the investigators, and 2 theoretical models: the Awareness to Adherence physician decision-making model and the Triandis consumer decision-making model.

The Awareness to Adherence model was developed to understand how physicians comply with new national practice guidelines for hepatitis B.²⁶ It was chosen because it is perhaps the only theoretical model that has both been designed and tested to explain the vaccination behavior of clinicians. This model includes 4 sequential cognitive and behavioral steps: awareness, agreement, adoption, and adherence. It is similar to the Stages of Change model of precontemplation, contemplation, preparation, action, and maintenance.²⁷ Shortly after national recommendations for hepatitis B vaccination of all infants, 98% of physicians were aware of them; 70% agreed with them; 55% adopted them; and 30% adhered to them.²⁶ Interventions to improve compliance with any given recommendation can fail if the specific problem of either awareness, agreement, adoption, or adherence is not identified and addressed. For example, efforts at further dissemination are the most common type of intervention to increase compliance, but in the case of hepatitis B vaccinations for children, 98% of physicians were aware of the guidelines. Therefore, further attempts to increase physicians’ awareness would be unlikely to increase vaccination rates.

The Triandis model has been used to understand consumer decision making and is based on the theory of reasoned action. We chose it for several reasons. First, The Triandis model as used for influenza immunization is internally consistent (Chronbach a = .91) and has been externally validated.²⁸ Second, it is broader than earlier models in that it accounts not only for beliefs, but also for values, social networks, habits, and physician influence on patients. Third, the Triandis model is able to predict behavior in a variety of cultural and economic situations.^28-31

Although these 2 models capture behavioral and educational issues related to health practices, they miss systemwide interventions such as standing orders that have had a major impact in raising immunization rates. Thus, we sought a larger framework that was comprehensive, would allow us to incorporate behaviorally oriented models as well as system interventions, and would facilitate the development of interventions. We chose the PRECEDE-PROCEED framework, a systematic process to evaluate health problems and design intervention programs.³² PRECEDE, an acronym for the Predisposing, Reinforcing, and Enabling Constructs in Educational Diagnosis and Evaluation, is an educational diagnosis model developed in the 1970s. PROCEED, an acronym for Policy, Regulatory, and Organizational Constructs in Educational and Environmental Development, was added to the model in 1991.³² PRECEDE-PROCEED offers specific guidelines for analysis of target populations so that the appropriateness of specific interventions can be determined.³² Although not a theory itself, PRECEDE-PROCEED provides a framework for applying theories. A key element of this framework is participation by the population in defining its problems and goals (Phase 1). In Phase 2, an epidemiologic diagnosis sets priorities for the community’s health problems so that resources can be applied to interventions that will have the most impact. Phase 3 is the behavioral and environmental diagnosis that helps planners determine risk factors for a particular problem and which of those risk factors are amenable to change. Phase 4, the educational and organizational diagnosis, enables planners to determine the predisposing, reinforcing, and enabling factors that influence the likelihood that behavioral and environmental change will occur. Those factors within an organization that have the capacity to facilitate or hinder the implementation of a program are determined in Phase 5 Figure 1. Phase 6 is the implementation phase, and Phases 7 to 9 comprise the evaluations of the process, impact, and outcome of the intervention program.

Triangulation

We also employed triangulation, a process that assesses the problem from multiple vantage points using multiple data collection techniques and multiple data sources Table 1.³³ The vantage points from which we collected data were the patient, the health care provider, and the health care organization. Our data collection techniques included focus groups, face-to-face and telephone interviews, self-administered surveys, site visits, participant observation, and medical record review. These methods provided data that are both quantitative (eg, immunization histories, demographics, surveys) and qualitative (eg, participant observations and focus group findings). Table 2 shows the relationships of the theoretical models and the specific research questions.

Conducting a study that collects both quantitative and qualitative methods requires the expertise of a multidisciplinary team. Our team included members from the disciplines of family medicine, preventive medicine, public health, internal medicine, medical sociology, medical anthropology, geriatrics, epidemiology, survey research, and biostatistics. This diversity in research backgrounds further broadens the perspective of the project.

Nested Sampling Design

The barriers to and facilitators of immunizations likely vary by characteristics of the patient population, by the mission of the health care facility, by the beliefs of the physicians, and by its internal operations and policies. We selected 4 types of facilities as our 4 strata to ensure access to a broad spectrum of patients, facilities, and policies, including: (1) inner-city neighborhood health centers serving economically disadvantaged populations with a high proportion of African American patients, (2) clinics in a Veterans Administration facility that also provides care for the underserved and which has an institutionwide program for increasing influenza and pneumococcal vaccination rates, (3) rural practices in a network, and (4) urban/suburban practices in a network.

A 2-stage stratified random cluster sampling was conducted to select participants. In stage 1, a stratified random cluster sample of 60 primary care clinicians (physicians, physician assistants, or nurse practitioners) was selected, 15 in each of the 4 strata. In stage 2, a randomly selected list of patients 66 years and older and seen in the office on or after October 1, 1998, was developed for each clinician. A random sample of 22 patients was then selected from each of these lists, with a target of 15 completed patient interviews per clinician. A total of 900 (60*15) patient interviews was the goal. This design allowed us to assess relationships among patient beliefs and behaviors, clinician beliefs and behaviors, and office systems and immunization records.

IRB Approval

This study was reviewed and approved by the Institutional Review Board of the University of Pittsburgh and the Human Use Subcommittee of the Institutional Review Board of the Veterans Affairs Healthcare System of Pittsburgh.

Data Collection

Seven survey instruments were used, 4 (1 each for physicians, nurses, office managers, and those patients followed by the anthropologists) were self-administered questionnaires of 19 to 59 items, primarily using scales and other quantitative measures. Three (1 each for physicians, nurses, and patients) were questionnaires used in face-to-face or telephone interviews that included open-ended questions.

All instruments were developed in a lengthy process of internal review and revision. The final drafts were piloted locally—the provider instruments on practicing primary care physicians, nurse practitioners, and nurses, and the patient instrument on visitors to a local senior citizen center. Subsequently, revisions were made.

Between July 1999 and December 1999, members of the research team visited each of the participating practices to further explain the project, photograph the office physical environment, collect floor plans, collect patient immunization related materials, distribute self-administered questionnaires, and complete as many face-to-face interviews as feasible.

On completion of the provider surveys and office visits, the patient telephone survey was initiated. To encourage patient participation, an endorsement letter from the clinician on practice stationery was mailed to patients, and they were offered a $20 honorarium. The patient questionnaire was programmed for computer-assisted telephone interviewing (CATI). CATI permits direct data entry during the interview, manages the sample of persons to be contacted, directs the sequence of questions, eliminates unintentionally skipped questions, and provides automatic range and logic checks.³⁴ Subsequently, medical records were reviewed using a standard form to collect information on immunization and other preventive services and to verify patient reported immunization status.

Participant/Practice Observation

On a subsample of the large study, we pursued a participant/practice observation study, in which 8 of the 24 practices were recruited, 2 from each of the 4 strata. Within the strata, an attempt was made to select 2 diverse practices, based on the number of clinicians, clinician sex, and clinician-to-patient ratio. An additional site was selected for pilot testing of the methodology.

Selected practices were divided between 2 trained anthropologists who spent 2 full workdays at each of their assigned sites. Following a prescribed protocol, their task was to observe the office practice, the interpersonal styles of the clinicians, and the office environment and culture. They also observed provider-patient interactions for 3 to 5 patients per practice and interviewed the patients following their examinations to assess their overall satisfaction and experience with the clinic. Also, the team reviewed information obtained from the face-to-face interviews of clinicians and nurses.

A systematic process of review of all observations occurred throughout the data collection to insure the trustworthiness of these data based on techniques used by Silverman and coworkers.³⁵ The first level of review was to discuss all observations among the 3 anthropologists following the collection of observation data of each site. This served to identify gaps in the data collection or areas where observations required some confirmation for accuracy and interpretation. The second level of review was through discussions at the weekly research team meetings for data verification and clarification of interpretation. Finally, the 8 sites were classified into types based on similarities in organizational structure, operational characteristics, and physician and staff philosophy.

Focus Groups

Two focus groups were held with senior adults who were not among the patients participating in the study. One group consisted of 14 African Americans, and the other consisted of 10 whites; both sexes were represented. Participants were recruited through a local senior center, where the focus groups were conducted. Discussion addressed issues of barriers to and facilitators of immunization and recommendations for improving immunization rates.

Statistical Methods

Qualitative data analysis methods such as the creation of code books was used to categorize the provider and staff responses to the interviewer questions and participant observations. These categorized responses were then used in the quantitative analysis in both bivariate and logistic regression models.³⁶

The quantitative analysis of the data must take into account the cluster-correlated nature of the data which results from a complex multistage stratified clustered sampling strategy. For this reason, we used statistical software that can compute standard errors, regression coefficients, and other statistics in accordance with the sample design.³⁷ Frequencies of patient responses to questions were computed using both quantitative and coded qualitative items. Bivariate relationships were examined, followed by logistic regression modeling, with receipt versus nonreceipt of vaccines as the dependent variable. Models were developed for each of the influenza and pneumococcal vaccines as dependent variables.

Discussion

Relevance of This Methodology for Future Interventions

The application of the PRECEDE-PROCEED framework and Awareness to Adherence and Triandis models to the study of provider and patient attitudes, knowledge, beliefs, and practices regarding adult immunizations is ideal. Using these models combined with a multidisciplinary research team and triangulation of data collection, we hope to gain insight into factors associated with adult immunizations. Authorities have suggested that a tailored approach that accounts for the core values, structure, and internal operations of practices, is more likely to raise immunization rates than using the same approach for all practices.^11,15,16 This unique study design allows for simultaneous examination of patient, provider, and office culture factors and their relative impact on adult immunization rates. This in turn will facilitate the development of tailored intervention plans to improve those rates.

Acknowledgments

This publication/project was funded by HS09874-01A1 from the Agency for Healthcare Research and Quality. The authors wish to acknowledge Michael J. Fine, MD; Edmund M. Ricci, PhD; Seymour Grufferman, MD; Ilene K. Jewell, MS Hyg; and Mahlon Raymund, PhD, for their significant contributions to the design and implementation of this project or paper.

Together, influenza and pneumonia are the sixth leading cause of death. Each year, influenza causes approximately 20,000 deaths, and pneumococcus causes approximately 500,000 cases of pneumonia, 50,000 cases of bacteremia, and 3000 cases of meningitis.^1,2 African Americans experience approximately twice the rate of invasive pneumococcal disease as whites.^3,4

Despite the burden of disease and the availability of vaccine guidelines,^1,2,5-7 vaccination rates are only modest, though they are slowly rising. In 1997, only 65% and 45% of persons 65 years or older reported receiving influenza and pneumococcal vaccines, respectively.⁸ Influenza vaccination rates were lower for persons of Hispanic (58%) and non-Hispanic black (50%) origin than for non-Hispanic whites (67%). Pneumococcal vaccination rates were 34% for Hispanics, 30% for non-Hispanic blacks, and 47% for non-Hispanic whites.⁸

The low rates are surprising, given that there is an abundance of literature reporting successful interventions for increasing immunization rates.^9,10 In a meta-analysis, system-oriented interventions (eg, standing orders for nurses) resulted in pooled rate increases of 39% and 45% for influenza and pneumococcal vaccines, respectively.⁹ Patient-oriented strategies (eg, postcard reminders that influenza or pneumococcal vaccine was due) resulted in increases of 12% and 75%, respectively. Provider-oriented strategies (eg, chart reminders) resulted in increases of 18% and 8%, respectively, for influenza and pneumococcal vaccines.⁹

Causes of low immunization rates

Although immunization rates are slowly increasing, why have they not risen more, given the evidence of effective interventions? We believe that there are 4 primary reasons why many clinical practices have not successfully applied research findings to improve adult vaccination rates.

First, offices are complex systems with idiosyncratic organization structures and values. They tend to accept changes when they are congruent with the organization’s goals and culture. Most previous efforts at intervention treated practices uniformly, with a “one size fits all” approach.^2,11 However, data from the Direct Observation of Primary Care Study reveals a wide variety of patients and problems.^12-14 Several authors have recommended tailoring interventions to match the organizational structure, office culture, and individual physician philosophies and practices as a means of increasing the likelihood of success.^11,12,15,16

Also, patient beliefs about adult vaccination are varied and include racial and ethnic diversity as well. Significant percentages of the elderly report lack of awareness of the need for immunizations (19% for influenza vaccination and 57% for pneumococcal vaccination).^17-19 Among racial groups, non-Hispanic blacks were least aware of the need for these vaccines, followed by Hispanics and non-Hispanic whites.¹⁷ Concern that vaccination may actually cause disease^17,20 and fear of the pain of injection and/or needles^17,21,22 lead many to decline vaccination. Although serious adverse events due to vaccination are rare, media attention to them increases public awareness of their occurrence and may contribute to fear of adverse reactions.^17,20-23

Time pressures on physicians also distract attention from prevention. Zyzanski and colleagues²⁴ found that physicians seeing high volumes of patients, in comparison to those with low volumes, had visits that were 30% shorter, scheduled fewer patients for well-care visits, delivered fewer preventive screenings, and gave fewer immunizations.

Finally, the responsibility for adult immunization has not been definitively assigned, resulting in fewer programmatic efforts. Many groups have an interest in adult immunization; however, coordination is limited, causing immunization messages to become diffuse. As a result, many providers caring for adults do not see vaccination as their responsibility.

The cumulative effect of these factors is that, despite access to medical care, many of the adults at high risk for vaccine-preventable diseases remain unvaccinated.²⁵

Research Questions

• Several research questions emerge from this scenario:
• What are the influenza and pneumococcal vaccination rates among persons 65 years and older of both majority and minority populations?
• What are the internal structure and office culture of various medical practices, and how do they facilitate or inhibit adult immunizations?
• What are providers’ attitudes, knowledge, and practices regarding adult immunizations?
• What are patients’ attitudes, knowledge, and beliefs regarding influenza and pneumococcal immunizations?
• What are the relationships among patient and provider knowledge, attitudes, beliefs, and practices and their impact on adult influenza and pneumococcal immunization rates?
• To answer these questions, a large multicomponent study with a variety of physician practice types and patient populations is required. Also, both quantitative and qualitative data need to be collected. In this article, we will describe the development of the methods used to answer these research questions.

Methods

Theoretical Framework and Models

To design our questionnaires, we used data from the literature, observations of the investigators, and 2 theoretical models: the Awareness to Adherence physician decision-making model and the Triandis consumer decision-making model.

The Awareness to Adherence model was developed to understand how physicians comply with new national practice guidelines for hepatitis B.²⁶ It was chosen because it is perhaps the only theoretical model that has both been designed and tested to explain the vaccination behavior of clinicians. This model includes 4 sequential cognitive and behavioral steps: awareness, agreement, adoption, and adherence. It is similar to the Stages of Change model of precontemplation, contemplation, preparation, action, and maintenance.²⁷ Shortly after national recommendations for hepatitis B vaccination of all infants, 98% of physicians were aware of them; 70% agreed with them; 55% adopted them; and 30% adhered to them.²⁶ Interventions to improve compliance with any given recommendation can fail if the specific problem of either awareness, agreement, adoption, or adherence is not identified and addressed. For example, efforts at further dissemination are the most common type of intervention to increase compliance, but in the case of hepatitis B vaccinations for children, 98% of physicians were aware of the guidelines. Therefore, further attempts to increase physicians’ awareness would be unlikely to increase vaccination rates.

The Triandis model has been used to understand consumer decision making and is based on the theory of reasoned action. We chose it for several reasons. First, The Triandis model as used for influenza immunization is internally consistent (Chronbach a = .91) and has been externally validated.²⁸ Second, it is broader than earlier models in that it accounts not only for beliefs, but also for values, social networks, habits, and physician influence on patients. Third, the Triandis model is able to predict behavior in a variety of cultural and economic situations.^28-31

Although these 2 models capture behavioral and educational issues related to health practices, they miss systemwide interventions such as standing orders that have had a major impact in raising immunization rates. Thus, we sought a larger framework that was comprehensive, would allow us to incorporate behaviorally oriented models as well as system interventions, and would facilitate the development of interventions. We chose the PRECEDE-PROCEED framework, a systematic process to evaluate health problems and design intervention programs.³² PRECEDE, an acronym for the Predisposing, Reinforcing, and Enabling Constructs in Educational Diagnosis and Evaluation, is an educational diagnosis model developed in the 1970s. PROCEED, an acronym for Policy, Regulatory, and Organizational Constructs in Educational and Environmental Development, was added to the model in 1991.³² PRECEDE-PROCEED offers specific guidelines for analysis of target populations so that the appropriateness of specific interventions can be determined.³² Although not a theory itself, PRECEDE-PROCEED provides a framework for applying theories. A key element of this framework is participation by the population in defining its problems and goals (Phase 1). In Phase 2, an epidemiologic diagnosis sets priorities for the community’s health problems so that resources can be applied to interventions that will have the most impact. Phase 3 is the behavioral and environmental diagnosis that helps planners determine risk factors for a particular problem and which of those risk factors are amenable to change. Phase 4, the educational and organizational diagnosis, enables planners to determine the predisposing, reinforcing, and enabling factors that influence the likelihood that behavioral and environmental change will occur. Those factors within an organization that have the capacity to facilitate or hinder the implementation of a program are determined in Phase 5 Figure 1. Phase 6 is the implementation phase, and Phases 7 to 9 comprise the evaluations of the process, impact, and outcome of the intervention program.

Triangulation

We also employed triangulation, a process that assesses the problem from multiple vantage points using multiple data collection techniques and multiple data sources Table 1.³³ The vantage points from which we collected data were the patient, the health care provider, and the health care organization. Our data collection techniques included focus groups, face-to-face and telephone interviews, self-administered surveys, site visits, participant observation, and medical record review. These methods provided data that are both quantitative (eg, immunization histories, demographics, surveys) and qualitative (eg, participant observations and focus group findings). Table 2 shows the relationships of the theoretical models and the specific research questions.

Conducting a study that collects both quantitative and qualitative methods requires the expertise of a multidisciplinary team. Our team included members from the disciplines of family medicine, preventive medicine, public health, internal medicine, medical sociology, medical anthropology, geriatrics, epidemiology, survey research, and biostatistics. This diversity in research backgrounds further broadens the perspective of the project.

Nested Sampling Design

The barriers to and facilitators of immunizations likely vary by characteristics of the patient population, by the mission of the health care facility, by the beliefs of the physicians, and by its internal operations and policies. We selected 4 types of facilities as our 4 strata to ensure access to a broad spectrum of patients, facilities, and policies, including: (1) inner-city neighborhood health centers serving economically disadvantaged populations with a high proportion of African American patients, (2) clinics in a Veterans Administration facility that also provides care for the underserved and which has an institutionwide program for increasing influenza and pneumococcal vaccination rates, (3) rural practices in a network, and (4) urban/suburban practices in a network.

A 2-stage stratified random cluster sampling was conducted to select participants. In stage 1, a stratified random cluster sample of 60 primary care clinicians (physicians, physician assistants, or nurse practitioners) was selected, 15 in each of the 4 strata. In stage 2, a randomly selected list of patients 66 years and older and seen in the office on or after October 1, 1998, was developed for each clinician. A random sample of 22 patients was then selected from each of these lists, with a target of 15 completed patient interviews per clinician. A total of 900 (60*15) patient interviews was the goal. This design allowed us to assess relationships among patient beliefs and behaviors, clinician beliefs and behaviors, and office systems and immunization records.

IRB Approval

This study was reviewed and approved by the Institutional Review Board of the University of Pittsburgh and the Human Use Subcommittee of the Institutional Review Board of the Veterans Affairs Healthcare System of Pittsburgh.

Data Collection

Seven survey instruments were used, 4 (1 each for physicians, nurses, office managers, and those patients followed by the anthropologists) were self-administered questionnaires of 19 to 59 items, primarily using scales and other quantitative measures. Three (1 each for physicians, nurses, and patients) were questionnaires used in face-to-face or telephone interviews that included open-ended questions.

All instruments were developed in a lengthy process of internal review and revision. The final drafts were piloted locally—the provider instruments on practicing primary care physicians, nurse practitioners, and nurses, and the patient instrument on visitors to a local senior citizen center. Subsequently, revisions were made.

Between July 1999 and December 1999, members of the research team visited each of the participating practices to further explain the project, photograph the office physical environment, collect floor plans, collect patient immunization related materials, distribute self-administered questionnaires, and complete as many face-to-face interviews as feasible.

On completion of the provider surveys and office visits, the patient telephone survey was initiated. To encourage patient participation, an endorsement letter from the clinician on practice stationery was mailed to patients, and they were offered a $20 honorarium. The patient questionnaire was programmed for computer-assisted telephone interviewing (CATI). CATI permits direct data entry during the interview, manages the sample of persons to be contacted, directs the sequence of questions, eliminates unintentionally skipped questions, and provides automatic range and logic checks.³⁴ Subsequently, medical records were reviewed using a standard form to collect information on immunization and other preventive services and to verify patient reported immunization status.

Participant/Practice Observation

On a subsample of the large study, we pursued a participant/practice observation study, in which 8 of the 24 practices were recruited, 2 from each of the 4 strata. Within the strata, an attempt was made to select 2 diverse practices, based on the number of clinicians, clinician sex, and clinician-to-patient ratio. An additional site was selected for pilot testing of the methodology.

Selected practices were divided between 2 trained anthropologists who spent 2 full workdays at each of their assigned sites. Following a prescribed protocol, their task was to observe the office practice, the interpersonal styles of the clinicians, and the office environment and culture. They also observed provider-patient interactions for 3 to 5 patients per practice and interviewed the patients following their examinations to assess their overall satisfaction and experience with the clinic. Also, the team reviewed information obtained from the face-to-face interviews of clinicians and nurses.

A systematic process of review of all observations occurred throughout the data collection to insure the trustworthiness of these data based on techniques used by Silverman and coworkers.³⁵ The first level of review was to discuss all observations among the 3 anthropologists following the collection of observation data of each site. This served to identify gaps in the data collection or areas where observations required some confirmation for accuracy and interpretation. The second level of review was through discussions at the weekly research team meetings for data verification and clarification of interpretation. Finally, the 8 sites were classified into types based on similarities in organizational structure, operational characteristics, and physician and staff philosophy.

Focus Groups

Two focus groups were held with senior adults who were not among the patients participating in the study. One group consisted of 14 African Americans, and the other consisted of 10 whites; both sexes were represented. Participants were recruited through a local senior center, where the focus groups were conducted. Discussion addressed issues of barriers to and facilitators of immunization and recommendations for improving immunization rates.

Statistical Methods

Qualitative data analysis methods such as the creation of code books was used to categorize the provider and staff responses to the interviewer questions and participant observations. These categorized responses were then used in the quantitative analysis in both bivariate and logistic regression models.³⁶

The quantitative analysis of the data must take into account the cluster-correlated nature of the data which results from a complex multistage stratified clustered sampling strategy. For this reason, we used statistical software that can compute standard errors, regression coefficients, and other statistics in accordance with the sample design.³⁷ Frequencies of patient responses to questions were computed using both quantitative and coded qualitative items. Bivariate relationships were examined, followed by logistic regression modeling, with receipt versus nonreceipt of vaccines as the dependent variable. Models were developed for each of the influenza and pneumococcal vaccines as dependent variables.

Discussion

Relevance of This Methodology for Future Interventions

The application of the PRECEDE-PROCEED framework and Awareness to Adherence and Triandis models to the study of provider and patient attitudes, knowledge, beliefs, and practices regarding adult immunizations is ideal. Using these models combined with a multidisciplinary research team and triangulation of data collection, we hope to gain insight into factors associated with adult immunizations. Authorities have suggested that a tailored approach that accounts for the core values, structure, and internal operations of practices, is more likely to raise immunization rates than using the same approach for all practices.^11,15,16 This unique study design allows for simultaneous examination of patient, provider, and office culture factors and their relative impact on adult immunization rates. This in turn will facilitate the development of tailored intervention plans to improve those rates.

Acknowledgments

This publication/project was funded by HS09874-01A1 from the Agency for Healthcare Research and Quality. The authors wish to acknowledge Michael J. Fine, MD; Edmund M. Ricci, PhD; Seymour Grufferman, MD; Ilene K. Jewell, MS Hyg; and Mahlon Raymund, PhD, for their significant contributions to the design and implementation of this project or paper.

Together, influenza and pneumonia are the sixth leading cause of death. Each year, influenza causes approximately 20,000 deaths, and pneumococcus causes approximately 500,000 cases of pneumonia, 50,000 cases of bacteremia, and 3000 cases of meningitis.^1,2 African Americans experience approximately twice the rate of invasive pneumococcal disease as whites.^3,4

Despite the burden of disease and the availability of vaccine guidelines,^1,2,5-7 vaccination rates are only modest, though they are slowly rising. In 1997, only 65% and 45% of persons 65 years or older reported receiving influenza and pneumococcal vaccines, respectively.⁸ Influenza vaccination rates were lower for persons of Hispanic (58%) and non-Hispanic black (50%) origin than for non-Hispanic whites (67%). Pneumococcal vaccination rates were 34% for Hispanics, 30% for non-Hispanic blacks, and 47% for non-Hispanic whites.⁸

The low rates are surprising, given that there is an abundance of literature reporting successful interventions for increasing immunization rates.^9,10 In a meta-analysis, system-oriented interventions (eg, standing orders for nurses) resulted in pooled rate increases of 39% and 45% for influenza and pneumococcal vaccines, respectively.⁹ Patient-oriented strategies (eg, postcard reminders that influenza or pneumococcal vaccine was due) resulted in increases of 12% and 75%, respectively. Provider-oriented strategies (eg, chart reminders) resulted in increases of 18% and 8%, respectively, for influenza and pneumococcal vaccines.⁹

Causes of low immunization rates

Although immunization rates are slowly increasing, why have they not risen more, given the evidence of effective interventions? We believe that there are 4 primary reasons why many clinical practices have not successfully applied research findings to improve adult vaccination rates.

First, offices are complex systems with idiosyncratic organization structures and values. They tend to accept changes when they are congruent with the organization’s goals and culture. Most previous efforts at intervention treated practices uniformly, with a “one size fits all” approach.^2,11 However, data from the Direct Observation of Primary Care Study reveals a wide variety of patients and problems.^12-14 Several authors have recommended tailoring interventions to match the organizational structure, office culture, and individual physician philosophies and practices as a means of increasing the likelihood of success.^11,12,15,16

Also, patient beliefs about adult vaccination are varied and include racial and ethnic diversity as well. Significant percentages of the elderly report lack of awareness of the need for immunizations (19% for influenza vaccination and 57% for pneumococcal vaccination).^17-19 Among racial groups, non-Hispanic blacks were least aware of the need for these vaccines, followed by Hispanics and non-Hispanic whites.¹⁷ Concern that vaccination may actually cause disease^17,20 and fear of the pain of injection and/or needles^17,21,22 lead many to decline vaccination. Although serious adverse events due to vaccination are rare, media attention to them increases public awareness of their occurrence and may contribute to fear of adverse reactions.^17,20-23

Time pressures on physicians also distract attention from prevention. Zyzanski and colleagues²⁴ found that physicians seeing high volumes of patients, in comparison to those with low volumes, had visits that were 30% shorter, scheduled fewer patients for well-care visits, delivered fewer preventive screenings, and gave fewer immunizations.

Finally, the responsibility for adult immunization has not been definitively assigned, resulting in fewer programmatic efforts. Many groups have an interest in adult immunization; however, coordination is limited, causing immunization messages to become diffuse. As a result, many providers caring for adults do not see vaccination as their responsibility.

The cumulative effect of these factors is that, despite access to medical care, many of the adults at high risk for vaccine-preventable diseases remain unvaccinated.²⁵

Research Questions

• Several research questions emerge from this scenario:
• What are the influenza and pneumococcal vaccination rates among persons 65 years and older of both majority and minority populations?
• What are the internal structure and office culture of various medical practices, and how do they facilitate or inhibit adult immunizations?
• What are providers’ attitudes, knowledge, and practices regarding adult immunizations?
• What are patients’ attitudes, knowledge, and beliefs regarding influenza and pneumococcal immunizations?
• What are the relationships among patient and provider knowledge, attitudes, beliefs, and practices and their impact on adult influenza and pneumococcal immunization rates?
• To answer these questions, a large multicomponent study with a variety of physician practice types and patient populations is required. Also, both quantitative and qualitative data need to be collected. In this article, we will describe the development of the methods used to answer these research questions.

Methods

Theoretical Framework and Models

To design our questionnaires, we used data from the literature, observations of the investigators, and 2 theoretical models: the Awareness to Adherence physician decision-making model and the Triandis consumer decision-making model.

The Awareness to Adherence model was developed to understand how physicians comply with new national practice guidelines for hepatitis B.²⁶ It was chosen because it is perhaps the only theoretical model that has both been designed and tested to explain the vaccination behavior of clinicians. This model includes 4 sequential cognitive and behavioral steps: awareness, agreement, adoption, and adherence. It is similar to the Stages of Change model of precontemplation, contemplation, preparation, action, and maintenance.²⁷ Shortly after national recommendations for hepatitis B vaccination of all infants, 98% of physicians were aware of them; 70% agreed with them; 55% adopted them; and 30% adhered to them.²⁶ Interventions to improve compliance with any given recommendation can fail if the specific problem of either awareness, agreement, adoption, or adherence is not identified and addressed. For example, efforts at further dissemination are the most common type of intervention to increase compliance, but in the case of hepatitis B vaccinations for children, 98% of physicians were aware of the guidelines. Therefore, further attempts to increase physicians’ awareness would be unlikely to increase vaccination rates.

The Triandis model has been used to understand consumer decision making and is based on the theory of reasoned action. We chose it for several reasons. First, The Triandis model as used for influenza immunization is internally consistent (Chronbach a = .91) and has been externally validated.²⁸ Second, it is broader than earlier models in that it accounts not only for beliefs, but also for values, social networks, habits, and physician influence on patients. Third, the Triandis model is able to predict behavior in a variety of cultural and economic situations.^28-31

Although these 2 models capture behavioral and educational issues related to health practices, they miss systemwide interventions such as standing orders that have had a major impact in raising immunization rates. Thus, we sought a larger framework that was comprehensive, would allow us to incorporate behaviorally oriented models as well as system interventions, and would facilitate the development of interventions. We chose the PRECEDE-PROCEED framework, a systematic process to evaluate health problems and design intervention programs.³² PRECEDE, an acronym for the Predisposing, Reinforcing, and Enabling Constructs in Educational Diagnosis and Evaluation, is an educational diagnosis model developed in the 1970s. PROCEED, an acronym for Policy, Regulatory, and Organizational Constructs in Educational and Environmental Development, was added to the model in 1991.³² PRECEDE-PROCEED offers specific guidelines for analysis of target populations so that the appropriateness of specific interventions can be determined.³² Although not a theory itself, PRECEDE-PROCEED provides a framework for applying theories. A key element of this framework is participation by the population in defining its problems and goals (Phase 1). In Phase 2, an epidemiologic diagnosis sets priorities for the community’s health problems so that resources can be applied to interventions that will have the most impact. Phase 3 is the behavioral and environmental diagnosis that helps planners determine risk factors for a particular problem and which of those risk factors are amenable to change. Phase 4, the educational and organizational diagnosis, enables planners to determine the predisposing, reinforcing, and enabling factors that influence the likelihood that behavioral and environmental change will occur. Those factors within an organization that have the capacity to facilitate or hinder the implementation of a program are determined in Phase 5 Figure 1. Phase 6 is the implementation phase, and Phases 7 to 9 comprise the evaluations of the process, impact, and outcome of the intervention program.

Triangulation

We also employed triangulation, a process that assesses the problem from multiple vantage points using multiple data collection techniques and multiple data sources Table 1.³³ The vantage points from which we collected data were the patient, the health care provider, and the health care organization. Our data collection techniques included focus groups, face-to-face and telephone interviews, self-administered surveys, site visits, participant observation, and medical record review. These methods provided data that are both quantitative (eg, immunization histories, demographics, surveys) and qualitative (eg, participant observations and focus group findings). Table 2 shows the relationships of the theoretical models and the specific research questions.

Conducting a study that collects both quantitative and qualitative methods requires the expertise of a multidisciplinary team. Our team included members from the disciplines of family medicine, preventive medicine, public health, internal medicine, medical sociology, medical anthropology, geriatrics, epidemiology, survey research, and biostatistics. This diversity in research backgrounds further broadens the perspective of the project.

Nested Sampling Design

The barriers to and facilitators of immunizations likely vary by characteristics of the patient population, by the mission of the health care facility, by the beliefs of the physicians, and by its internal operations and policies. We selected 4 types of facilities as our 4 strata to ensure access to a broad spectrum of patients, facilities, and policies, including: (1) inner-city neighborhood health centers serving economically disadvantaged populations with a high proportion of African American patients, (2) clinics in a Veterans Administration facility that also provides care for the underserved and which has an institutionwide program for increasing influenza and pneumococcal vaccination rates, (3) rural practices in a network, and (4) urban/suburban practices in a network.

A 2-stage stratified random cluster sampling was conducted to select participants. In stage 1, a stratified random cluster sample of 60 primary care clinicians (physicians, physician assistants, or nurse practitioners) was selected, 15 in each of the 4 strata. In stage 2, a randomly selected list of patients 66 years and older and seen in the office on or after October 1, 1998, was developed for each clinician. A random sample of 22 patients was then selected from each of these lists, with a target of 15 completed patient interviews per clinician. A total of 900 (60*15) patient interviews was the goal. This design allowed us to assess relationships among patient beliefs and behaviors, clinician beliefs and behaviors, and office systems and immunization records.

IRB Approval

This study was reviewed and approved by the Institutional Review Board of the University of Pittsburgh and the Human Use Subcommittee of the Institutional Review Board of the Veterans Affairs Healthcare System of Pittsburgh.

Data Collection

Seven survey instruments were used, 4 (1 each for physicians, nurses, office managers, and those patients followed by the anthropologists) were self-administered questionnaires of 19 to 59 items, primarily using scales and other quantitative measures. Three (1 each for physicians, nurses, and patients) were questionnaires used in face-to-face or telephone interviews that included open-ended questions.

All instruments were developed in a lengthy process of internal review and revision. The final drafts were piloted locally—the provider instruments on practicing primary care physicians, nurse practitioners, and nurses, and the patient instrument on visitors to a local senior citizen center. Subsequently, revisions were made.

Between July 1999 and December 1999, members of the research team visited each of the participating practices to further explain the project, photograph the office physical environment, collect floor plans, collect patient immunization related materials, distribute self-administered questionnaires, and complete as many face-to-face interviews as feasible.

On completion of the provider surveys and office visits, the patient telephone survey was initiated. To encourage patient participation, an endorsement letter from the clinician on practice stationery was mailed to patients, and they were offered a $20 honorarium. The patient questionnaire was programmed for computer-assisted telephone interviewing (CATI). CATI permits direct data entry during the interview, manages the sample of persons to be contacted, directs the sequence of questions, eliminates unintentionally skipped questions, and provides automatic range and logic checks.³⁴ Subsequently, medical records were reviewed using a standard form to collect information on immunization and other preventive services and to verify patient reported immunization status.

Participant/Practice Observation

On a subsample of the large study, we pursued a participant/practice observation study, in which 8 of the 24 practices were recruited, 2 from each of the 4 strata. Within the strata, an attempt was made to select 2 diverse practices, based on the number of clinicians, clinician sex, and clinician-to-patient ratio. An additional site was selected for pilot testing of the methodology.

Selected practices were divided between 2 trained anthropologists who spent 2 full workdays at each of their assigned sites. Following a prescribed protocol, their task was to observe the office practice, the interpersonal styles of the clinicians, and the office environment and culture. They also observed provider-patient interactions for 3 to 5 patients per practice and interviewed the patients following their examinations to assess their overall satisfaction and experience with the clinic. Also, the team reviewed information obtained from the face-to-face interviews of clinicians and nurses.

A systematic process of review of all observations occurred throughout the data collection to insure the trustworthiness of these data based on techniques used by Silverman and coworkers.³⁵ The first level of review was to discuss all observations among the 3 anthropologists following the collection of observation data of each site. This served to identify gaps in the data collection or areas where observations required some confirmation for accuracy and interpretation. The second level of review was through discussions at the weekly research team meetings for data verification and clarification of interpretation. Finally, the 8 sites were classified into types based on similarities in organizational structure, operational characteristics, and physician and staff philosophy.

Focus Groups

Two focus groups were held with senior adults who were not among the patients participating in the study. One group consisted of 14 African Americans, and the other consisted of 10 whites; both sexes were represented. Participants were recruited through a local senior center, where the focus groups were conducted. Discussion addressed issues of barriers to and facilitators of immunization and recommendations for improving immunization rates.

Statistical Methods

Qualitative data analysis methods such as the creation of code books was used to categorize the provider and staff responses to the interviewer questions and participant observations. These categorized responses were then used in the quantitative analysis in both bivariate and logistic regression models.³⁶

The quantitative analysis of the data must take into account the cluster-correlated nature of the data which results from a complex multistage stratified clustered sampling strategy. For this reason, we used statistical software that can compute standard errors, regression coefficients, and other statistics in accordance with the sample design.³⁷ Frequencies of patient responses to questions were computed using both quantitative and coded qualitative items. Bivariate relationships were examined, followed by logistic regression modeling, with receipt versus nonreceipt of vaccines as the dependent variable. Models were developed for each of the influenza and pneumococcal vaccines as dependent variables.

Discussion

Relevance of This Methodology for Future Interventions

The application of the PRECEDE-PROCEED framework and Awareness to Adherence and Triandis models to the study of provider and patient attitudes, knowledge, beliefs, and practices regarding adult immunizations is ideal. Using these models combined with a multidisciplinary research team and triangulation of data collection, we hope to gain insight into factors associated with adult immunizations. Authorities have suggested that a tailored approach that accounts for the core values, structure, and internal operations of practices, is more likely to raise immunization rates than using the same approach for all practices.^11,15,16 This unique study design allows for simultaneous examination of patient, provider, and office culture factors and their relative impact on adult immunization rates. This in turn will facilitate the development of tailored intervention plans to improve those rates.

Acknowledgments

This publication/project was funded by HS09874-01A1 from the Agency for Healthcare Research and Quality. The authors wish to acknowledge Michael J. Fine, MD; Edmund M. Ricci, PhD; Seymour Grufferman, MD; Ilene K. Jewell, MS Hyg; and Mahlon Raymund, PhD, for their significant contributions to the design and implementation of this project or paper.