Building research capacity in family medicine: Evaluation of the Grant Generating Project

In 1995, in recognition of the need for trained family medicine researchers,^1-6 the North American Primary Care Research Group (NAPCRG) Committee on Building Research Capacity launched the Family Medicine Grant Generating Project (GGP) to help family medicine researchers successfully apply for their first major research grant. Between 1995 and the time of the evaluation, 23 faculty participants from 21 family medicine departments enrolled in this year-long “fellowship without walls” designed to permit participants to remain housed in their home department while working on their research proposal. Participants attended 3 GGP sessions held in conjunction with the annual meeting of the Society of Teachers of Family Medicine (STFM), and the NACPRG and the Primary Care Research Methods and Statistics meetings over the course of 1 year. Additionally, participants worked via e-mail, telephone, and postal mail with GGP faculty and research mentors during the year. Many participants met in person with mentors.

Although the program is referred to as a “fellowship” and participants as “fellows,” this is neither a traditional family medicine “fellowship” in which participants are housed for a designated time period (usually 2 years), nor an innovative fellowship in which individuals spend concentrated time at another institution. Traditional fellowships usually offer courses for academic credit in a variety of topics such as statistics and epidemiology and typically result in a master’s degree. GGP, however, is designed to train fellows in the design and writing of competitive external grant proposals: it does not offer courses or a degree. Over the fellowship year, fellows attend sessions that focus on the various tools, techniques, and methods required to produce a competitive proposal. These sessions include time devoted to concept paper development, conducting a literature review, development of specific aims, and elaboration of various research methodologies. GGP uses the term “fellowship” to identify the program as a special 1-year period with at least 25% of the time devoted to developing “grantsmanship skills” and producing a competitive proposal. This program is the only one of its kind in family medicine and therefore cannot be compared with more traditional fellowships programs. This article reports on the evaluation results.

Methods

From December 1999 through April 2000 we mailed surveys to 23 former GGP participants. Respondents were asked to evaluate the GGP’s components (statistics, concept paper, grant writing, mentoring) on a scale from “not at all helpful” to “very helpful.” Several open-ended questions were asked about their program expectations; what they considered GGP’s strengths and weaknesses; how participation contributed to their academic career and helped increase research capacity; the barriers encountered in achieving their research goals; the skills learned; suggestions for improving the program; and advice they would give to future applicants. Respondents were asked to indicate the grants/contracts that they submitted after their fellowship. In addition, demographic information was obtained on age, employment position, date of initiation of GGP, the year they graduated from medical school, and the year they participated in the GGP program.

Eighteen (78%) of the participants returned a completed survey. Reasons for not completing the survey, given either by the participants who chose not to complete the survey, or by their departments, included the participant now working outside research; the participant on leave; the participant in the middle of writing a research grant (R01) and therefore not having time to complete the survey; and lack of response (only 2 participants did not respond after 3 attempts to contact them). Of those who did not complete the survey, 1 was from the first-year class, 2 were from the second-year class, and 2 were from the third-year class.

Results

The average age of respondents was 43.8 years (range, 37-53 years). Most were at the assistant professor level. An additional 4 were at the associate professor level, 2 were research directors, 1 was a professor, 1 was a clinician investigator, and 1 was an interim chair. A total of 12 of the respondents were physicians, 5 had a PhD, and 1 had a Doctor of Public Health degree; 4 of the physicians also had a Masters in Public Health and 2 had a PhD. The average number of years since obtaining their terminal degree was 15.1 years (range, 4-26 years). Most respondents (11) had their terminal degree 15 years or more.

Evaluation of research activities

A total of 58 grants/contracts were submitted by the respondents, representing approximately US$19.3 million. These figures represent an underestimate because 1 first-year class respondent only mentioned having submitted more than 10 grants over the last 3 years ranging from $50,000 to more than $1 million, but did not elaborate further. Due to this lack of specification, this information could not be entered into the subsequent analysis. Seven (12%) of those submitted were training grants, representing approximately $1.7 million. Most (32, or 55%) of the submitted grants/contracts were to nonfederal organizations, whereas 19 (or -33%) were to federal agencies. Although 22 (38%) of the submitted grants/contracts were for less than $50,000, 11 (19%) (excluding training grants) were for more than $300,000. Nineteen (33%) of the submitted grants/contracts were funded including 4 training grants, representing approximately $1 million excluding training grants ($2.3 million including training grants).

Although 22 (38%) of the grant submissions were not funded (representing approximately $6.2 million), another 17 (29%) were pending, representing approximately $10.8 million, including training grants. Based on prior experience, approximately 46% of these pending grants may receive funding, which could result in an additional $4.8 million.

Table 1 shows the number of grants/contracts submitted, funded, not funded, and pending as Principal Investigator (PI) or Co-PI by class year of the GGP fellowship. Most (42, 72%) of the grants/contracts submitted were by the first- or second-year class fellows with 12 grants pending and 16 funded. The Figure 1 shows the number of grants/contracts submitted by submission date and fellowship class year. The number of submissions increased within 1 to 2 years of the fellowship.

Although the sample size was too small for statistical evaluation, some trends are noted. Only 3 of the 18 respondents indicated that they had not submitted a grant or contract as PI or Co-PI, whereas 1 respondent indicated being an evaluator or research director on 6 funded grants. The average number of grants submitted by participants was 4.4. Participants who had submitted a grant tended to be older and to have held their terminal degree for a longer time. Thirty-seven of the total grant/contract submissions were by respondents older than the mean age of 43 years, and 41 of the total grant/contract submissions were by respondents who had received their terminal degree more than 15 years ago. The latter respondents also accounted for 15 of the 19 funded grants. No difference was noted in the rate of grant submission by terminal degree type.

Several respondents also indicated the following grant roles: qualitative analyst on 2 grants; research writer, administrator, project director, or evaluator on 8 grants; co-investigator on 2 grants; and consultant on 4 grants. Twelve of these grants were funded, with 4 training grants representing approximately $1.7 million and 8 research grants representing approximately $1.3 million.

Evaluation of GGP components

Table 2 shows the evaluation of each GGP component. Most respondents indicated that the concept paper and the grant-writing components were very helpful. Additional components indicated as very helpful included networking, mock reviews, consultant mentoring, and peer groups. Compared with the concept paper and grant-writing components, the statistics and the mentoring components were rated as less helpful. Expectations of the GGP

Respondents were asked to describe their major expectations of the GGP. Most respondents replied that they expected to receive help in grant writing, to better understand the grant application process, and to develop helpful relationships with experienced researchers.

All of the respondents indicated that most of their expectations were met. However, respondents were mixed regarding the mentoring process. Although 8 indicated that the mentoring was very helpful, some respondents had difficulty in identifying a mentor or were never successful in finding a mentor, whereas others indicated problems with long distance mentoring.

Strengths of the GGP

Respondents indicated that program strengths include enhancing the value of family medicine research, receiving guidance and feedback, learning from the experiences of their peers, and developing grant-writing skills. Respondents specifically mentioned the focus on the NIH grant process and the mock study section as strengths.

With respect to how the GGP contributed to their career, 7 of the respondents said that the program helped them connect with other researchers and gave them confidence in grant writing. In other instances, the GGP program helped respondents to achieve a more secure position, obtain a promotion, start a career, or change academic departments/institutions. As one respondent mentioned, “I think more critically and more daringly . . . I [was] promoted after that from assistant to associate.”

Most respondents indicated that the GGP program enabled them to mentor, teach, and encourage other faculty members in their department in their grant-writing efforts. As one respondent mentioned, the GGP experience ”. . . has helped me to bridge the gap between clinicians and researchers.”

Challenges of the GGP

The most specific challenge of the GGP program mentioned by respondents was the variable commitment of mentors and the lack of having a mentor with expertise in a respondent’s area of research. The major barriers to respondents toward achieving their research goals were (1) the lack of a supportive research environment including a lack of financial support, few peers engaged in research, a lack of local mentoring, and infrastructural and administrative barriers (eg, no links to statisticians, confused priorities within department, and lack of diversity and foresight in department); (2) the lack of time to do research while participating in clinical duties and academic activities; and (3) the lack of information about funding opportunities at their institution. One respondent mentioned not having any external barriers but that the GGP experience “. . . did cause me to reevaluate and ultimately redefine my research goals.”

When asked what skills they learned during the GGP that helped them address these barriers, several respondents indicated (1) time management, including how to realistically organize the research process; (2) the ability to make connections with outside mentors and consultants; and (3) confidence development, such as contacting a program specialist despite poor information from their institution. However, respondents who mentioned the barrier of a nonsupportive research environment indicated that skills to address this barrier were not taught in the GGP program. Recommendations for improving the GGP

Suggestions for improving the GGP program included having 2 GGP entry levels (for beginning researchers and experienced researchers), continuing networking opportunities for “graduates,” addressing the need for female mentors, following up with consultations and mentoring, and having a mentor with expertise in participants’ area of research.

With respect to advice to future applicants, respondents mentioned being more aggressive in working with a mentor, having reasonable expectations, having long-term support from the department, using time efficiently to network, and reflecting on one’s motivation toward an intensive research career. As one participant said, “I would support the application 100% and notify them that it is one of the best investments of time they can make for their academic career.”

Discussion

The respondents’ comments made clear that the GGP is an “excellent place to start a career.” Most respondents appeared to regard the program as well worth the time and effort invested. Aside from some adjustments, such as consistency in mentoring and accommodating the research focus and skill level of participants, it is “. . . very important to continue [the] effort to train new researchers.” Interestingly, the GGP had a substantial impact on the extent to which many participants are now teaching and mentoring others to help develop the skills necessary for meaningful research.

Moreover, within 1 to 2 years of the program participants achieved a remarkable track record of grant submissions. Although for some participants “. . . breaking in [getting grant awards] still seems mysterious and forbidding,” they appreciated the opportunity to learn how to write a coherent concept paper, how to apply “good methods,” the importance of a track record, and to learn from experienced researchers.

As a result in large part of this evaluation, funding for the GGP has been continued with support from NAPCRG, the American Academy of Family Physicians, the STFM, the College of Family Physicians of Canada, and the home departments of participating academic departments and community-based residency programs.

Family medicine is poised to generate new knowledge through the development of its own research agenda-one that focuses on integrated, relationship-centered patient, family, and community health care.⁷ Young researchers, however, need to be nurtured, mentored, and organizationally supported.⁸ Research initiatives such as the GGP represent a collective enterprise, a building of coalitions toward the development of a culture of family medicine research. Continued research success requires commitments from both individuals and departments if family medicine research is to grow and prosper.

In 1995, in recognition of the need for trained family medicine researchers,^1-6 the North American Primary Care Research Group (NAPCRG) Committee on Building Research Capacity launched the Family Medicine Grant Generating Project (GGP) to help family medicine researchers successfully apply for their first major research grant. Between 1995 and the time of the evaluation, 23 faculty participants from 21 family medicine departments enrolled in this year-long “fellowship without walls” designed to permit participants to remain housed in their home department while working on their research proposal. Participants attended 3 GGP sessions held in conjunction with the annual meeting of the Society of Teachers of Family Medicine (STFM), and the NACPRG and the Primary Care Research Methods and Statistics meetings over the course of 1 year. Additionally, participants worked via e-mail, telephone, and postal mail with GGP faculty and research mentors during the year. Many participants met in person with mentors.

Although the program is referred to as a “fellowship” and participants as “fellows,” this is neither a traditional family medicine “fellowship” in which participants are housed for a designated time period (usually 2 years), nor an innovative fellowship in which individuals spend concentrated time at another institution. Traditional fellowships usually offer courses for academic credit in a variety of topics such as statistics and epidemiology and typically result in a master’s degree. GGP, however, is designed to train fellows in the design and writing of competitive external grant proposals: it does not offer courses or a degree. Over the fellowship year, fellows attend sessions that focus on the various tools, techniques, and methods required to produce a competitive proposal. These sessions include time devoted to concept paper development, conducting a literature review, development of specific aims, and elaboration of various research methodologies. GGP uses the term “fellowship” to identify the program as a special 1-year period with at least 25% of the time devoted to developing “grantsmanship skills” and producing a competitive proposal. This program is the only one of its kind in family medicine and therefore cannot be compared with more traditional fellowships programs. This article reports on the evaluation results.

Methods

From December 1999 through April 2000 we mailed surveys to 23 former GGP participants. Respondents were asked to evaluate the GGP’s components (statistics, concept paper, grant writing, mentoring) on a scale from “not at all helpful” to “very helpful.” Several open-ended questions were asked about their program expectations; what they considered GGP’s strengths and weaknesses; how participation contributed to their academic career and helped increase research capacity; the barriers encountered in achieving their research goals; the skills learned; suggestions for improving the program; and advice they would give to future applicants. Respondents were asked to indicate the grants/contracts that they submitted after their fellowship. In addition, demographic information was obtained on age, employment position, date of initiation of GGP, the year they graduated from medical school, and the year they participated in the GGP program.

Eighteen (78%) of the participants returned a completed survey. Reasons for not completing the survey, given either by the participants who chose not to complete the survey, or by their departments, included the participant now working outside research; the participant on leave; the participant in the middle of writing a research grant (R01) and therefore not having time to complete the survey; and lack of response (only 2 participants did not respond after 3 attempts to contact them). Of those who did not complete the survey, 1 was from the first-year class, 2 were from the second-year class, and 2 were from the third-year class.

Results

The average age of respondents was 43.8 years (range, 37-53 years). Most were at the assistant professor level. An additional 4 were at the associate professor level, 2 were research directors, 1 was a professor, 1 was a clinician investigator, and 1 was an interim chair. A total of 12 of the respondents were physicians, 5 had a PhD, and 1 had a Doctor of Public Health degree; 4 of the physicians also had a Masters in Public Health and 2 had a PhD. The average number of years since obtaining their terminal degree was 15.1 years (range, 4-26 years). Most respondents (11) had their terminal degree 15 years or more.

Evaluation of research activities

A total of 58 grants/contracts were submitted by the respondents, representing approximately US$19.3 million. These figures represent an underestimate because 1 first-year class respondent only mentioned having submitted more than 10 grants over the last 3 years ranging from $50,000 to more than $1 million, but did not elaborate further. Due to this lack of specification, this information could not be entered into the subsequent analysis. Seven (12%) of those submitted were training grants, representing approximately $1.7 million. Most (32, or 55%) of the submitted grants/contracts were to nonfederal organizations, whereas 19 (or -33%) were to federal agencies. Although 22 (38%) of the submitted grants/contracts were for less than $50,000, 11 (19%) (excluding training grants) were for more than $300,000. Nineteen (33%) of the submitted grants/contracts were funded including 4 training grants, representing approximately $1 million excluding training grants ($2.3 million including training grants).

Although 22 (38%) of the grant submissions were not funded (representing approximately $6.2 million), another 17 (29%) were pending, representing approximately $10.8 million, including training grants. Based on prior experience, approximately 46% of these pending grants may receive funding, which could result in an additional $4.8 million.

Table 1 shows the number of grants/contracts submitted, funded, not funded, and pending as Principal Investigator (PI) or Co-PI by class year of the GGP fellowship. Most (42, 72%) of the grants/contracts submitted were by the first- or second-year class fellows with 12 grants pending and 16 funded. The Figure 1 shows the number of grants/contracts submitted by submission date and fellowship class year. The number of submissions increased within 1 to 2 years of the fellowship.

Although the sample size was too small for statistical evaluation, some trends are noted. Only 3 of the 18 respondents indicated that they had not submitted a grant or contract as PI or Co-PI, whereas 1 respondent indicated being an evaluator or research director on 6 funded grants. The average number of grants submitted by participants was 4.4. Participants who had submitted a grant tended to be older and to have held their terminal degree for a longer time. Thirty-seven of the total grant/contract submissions were by respondents older than the mean age of 43 years, and 41 of the total grant/contract submissions were by respondents who had received their terminal degree more than 15 years ago. The latter respondents also accounted for 15 of the 19 funded grants. No difference was noted in the rate of grant submission by terminal degree type.

Several respondents also indicated the following grant roles: qualitative analyst on 2 grants; research writer, administrator, project director, or evaluator on 8 grants; co-investigator on 2 grants; and consultant on 4 grants. Twelve of these grants were funded, with 4 training grants representing approximately $1.7 million and 8 research grants representing approximately $1.3 million.

Evaluation of GGP components

Table 2 shows the evaluation of each GGP component. Most respondents indicated that the concept paper and the grant-writing components were very helpful. Additional components indicated as very helpful included networking, mock reviews, consultant mentoring, and peer groups. Compared with the concept paper and grant-writing components, the statistics and the mentoring components were rated as less helpful. Expectations of the GGP

Respondents were asked to describe their major expectations of the GGP. Most respondents replied that they expected to receive help in grant writing, to better understand the grant application process, and to develop helpful relationships with experienced researchers.

All of the respondents indicated that most of their expectations were met. However, respondents were mixed regarding the mentoring process. Although 8 indicated that the mentoring was very helpful, some respondents had difficulty in identifying a mentor or were never successful in finding a mentor, whereas others indicated problems with long distance mentoring.

Strengths of the GGP

Respondents indicated that program strengths include enhancing the value of family medicine research, receiving guidance and feedback, learning from the experiences of their peers, and developing grant-writing skills. Respondents specifically mentioned the focus on the NIH grant process and the mock study section as strengths.

With respect to how the GGP contributed to their career, 7 of the respondents said that the program helped them connect with other researchers and gave them confidence in grant writing. In other instances, the GGP program helped respondents to achieve a more secure position, obtain a promotion, start a career, or change academic departments/institutions. As one respondent mentioned, “I think more critically and more daringly . . . I [was] promoted after that from assistant to associate.”

Most respondents indicated that the GGP program enabled them to mentor, teach, and encourage other faculty members in their department in their grant-writing efforts. As one respondent mentioned, the GGP experience ”. . . has helped me to bridge the gap between clinicians and researchers.”

Challenges of the GGP

The most specific challenge of the GGP program mentioned by respondents was the variable commitment of mentors and the lack of having a mentor with expertise in a respondent’s area of research. The major barriers to respondents toward achieving their research goals were (1) the lack of a supportive research environment including a lack of financial support, few peers engaged in research, a lack of local mentoring, and infrastructural and administrative barriers (eg, no links to statisticians, confused priorities within department, and lack of diversity and foresight in department); (2) the lack of time to do research while participating in clinical duties and academic activities; and (3) the lack of information about funding opportunities at their institution. One respondent mentioned not having any external barriers but that the GGP experience “. . . did cause me to reevaluate and ultimately redefine my research goals.”

When asked what skills they learned during the GGP that helped them address these barriers, several respondents indicated (1) time management, including how to realistically organize the research process; (2) the ability to make connections with outside mentors and consultants; and (3) confidence development, such as contacting a program specialist despite poor information from their institution. However, respondents who mentioned the barrier of a nonsupportive research environment indicated that skills to address this barrier were not taught in the GGP program. Recommendations for improving the GGP

Suggestions for improving the GGP program included having 2 GGP entry levels (for beginning researchers and experienced researchers), continuing networking opportunities for “graduates,” addressing the need for female mentors, following up with consultations and mentoring, and having a mentor with expertise in participants’ area of research.

With respect to advice to future applicants, respondents mentioned being more aggressive in working with a mentor, having reasonable expectations, having long-term support from the department, using time efficiently to network, and reflecting on one’s motivation toward an intensive research career. As one participant said, “I would support the application 100% and notify them that it is one of the best investments of time they can make for their academic career.”

Discussion

The respondents’ comments made clear that the GGP is an “excellent place to start a career.” Most respondents appeared to regard the program as well worth the time and effort invested. Aside from some adjustments, such as consistency in mentoring and accommodating the research focus and skill level of participants, it is “. . . very important to continue [the] effort to train new researchers.” Interestingly, the GGP had a substantial impact on the extent to which many participants are now teaching and mentoring others to help develop the skills necessary for meaningful research.

Moreover, within 1 to 2 years of the program participants achieved a remarkable track record of grant submissions. Although for some participants “. . . breaking in [getting grant awards] still seems mysterious and forbidding,” they appreciated the opportunity to learn how to write a coherent concept paper, how to apply “good methods,” the importance of a track record, and to learn from experienced researchers.

As a result in large part of this evaluation, funding for the GGP has been continued with support from NAPCRG, the American Academy of Family Physicians, the STFM, the College of Family Physicians of Canada, and the home departments of participating academic departments and community-based residency programs.

Family medicine is poised to generate new knowledge through the development of its own research agenda-one that focuses on integrated, relationship-centered patient, family, and community health care.⁷ Young researchers, however, need to be nurtured, mentored, and organizationally supported.⁸ Research initiatives such as the GGP represent a collective enterprise, a building of coalitions toward the development of a culture of family medicine research. Continued research success requires commitments from both individuals and departments if family medicine research is to grow and prosper.

In 1995, in recognition of the need for trained family medicine researchers,^1-6 the North American Primary Care Research Group (NAPCRG) Committee on Building Research Capacity launched the Family Medicine Grant Generating Project (GGP) to help family medicine researchers successfully apply for their first major research grant. Between 1995 and the time of the evaluation, 23 faculty participants from 21 family medicine departments enrolled in this year-long “fellowship without walls” designed to permit participants to remain housed in their home department while working on their research proposal. Participants attended 3 GGP sessions held in conjunction with the annual meeting of the Society of Teachers of Family Medicine (STFM), and the NACPRG and the Primary Care Research Methods and Statistics meetings over the course of 1 year. Additionally, participants worked via e-mail, telephone, and postal mail with GGP faculty and research mentors during the year. Many participants met in person with mentors.

Although the program is referred to as a “fellowship” and participants as “fellows,” this is neither a traditional family medicine “fellowship” in which participants are housed for a designated time period (usually 2 years), nor an innovative fellowship in which individuals spend concentrated time at another institution. Traditional fellowships usually offer courses for academic credit in a variety of topics such as statistics and epidemiology and typically result in a master’s degree. GGP, however, is designed to train fellows in the design and writing of competitive external grant proposals: it does not offer courses or a degree. Over the fellowship year, fellows attend sessions that focus on the various tools, techniques, and methods required to produce a competitive proposal. These sessions include time devoted to concept paper development, conducting a literature review, development of specific aims, and elaboration of various research methodologies. GGP uses the term “fellowship” to identify the program as a special 1-year period with at least 25% of the time devoted to developing “grantsmanship skills” and producing a competitive proposal. This program is the only one of its kind in family medicine and therefore cannot be compared with more traditional fellowships programs. This article reports on the evaluation results.

Methods

From December 1999 through April 2000 we mailed surveys to 23 former GGP participants. Respondents were asked to evaluate the GGP’s components (statistics, concept paper, grant writing, mentoring) on a scale from “not at all helpful” to “very helpful.” Several open-ended questions were asked about their program expectations; what they considered GGP’s strengths and weaknesses; how participation contributed to their academic career and helped increase research capacity; the barriers encountered in achieving their research goals; the skills learned; suggestions for improving the program; and advice they would give to future applicants. Respondents were asked to indicate the grants/contracts that they submitted after their fellowship. In addition, demographic information was obtained on age, employment position, date of initiation of GGP, the year they graduated from medical school, and the year they participated in the GGP program.

Eighteen (78%) of the participants returned a completed survey. Reasons for not completing the survey, given either by the participants who chose not to complete the survey, or by their departments, included the participant now working outside research; the participant on leave; the participant in the middle of writing a research grant (R01) and therefore not having time to complete the survey; and lack of response (only 2 participants did not respond after 3 attempts to contact them). Of those who did not complete the survey, 1 was from the first-year class, 2 were from the second-year class, and 2 were from the third-year class.

Results

The average age of respondents was 43.8 years (range, 37-53 years). Most were at the assistant professor level. An additional 4 were at the associate professor level, 2 were research directors, 1 was a professor, 1 was a clinician investigator, and 1 was an interim chair. A total of 12 of the respondents were physicians, 5 had a PhD, and 1 had a Doctor of Public Health degree; 4 of the physicians also had a Masters in Public Health and 2 had a PhD. The average number of years since obtaining their terminal degree was 15.1 years (range, 4-26 years). Most respondents (11) had their terminal degree 15 years or more.

Evaluation of research activities

A total of 58 grants/contracts were submitted by the respondents, representing approximately US$19.3 million. These figures represent an underestimate because 1 first-year class respondent only mentioned having submitted more than 10 grants over the last 3 years ranging from $50,000 to more than $1 million, but did not elaborate further. Due to this lack of specification, this information could not be entered into the subsequent analysis. Seven (12%) of those submitted were training grants, representing approximately $1.7 million. Most (32, or 55%) of the submitted grants/contracts were to nonfederal organizations, whereas 19 (or -33%) were to federal agencies. Although 22 (38%) of the submitted grants/contracts were for less than $50,000, 11 (19%) (excluding training grants) were for more than $300,000. Nineteen (33%) of the submitted grants/contracts were funded including 4 training grants, representing approximately $1 million excluding training grants ($2.3 million including training grants).

Although 22 (38%) of the grant submissions were not funded (representing approximately $6.2 million), another 17 (29%) were pending, representing approximately $10.8 million, including training grants. Based on prior experience, approximately 46% of these pending grants may receive funding, which could result in an additional $4.8 million.

Table 1 shows the number of grants/contracts submitted, funded, not funded, and pending as Principal Investigator (PI) or Co-PI by class year of the GGP fellowship. Most (42, 72%) of the grants/contracts submitted were by the first- or second-year class fellows with 12 grants pending and 16 funded. The Figure 1 shows the number of grants/contracts submitted by submission date and fellowship class year. The number of submissions increased within 1 to 2 years of the fellowship.

Although the sample size was too small for statistical evaluation, some trends are noted. Only 3 of the 18 respondents indicated that they had not submitted a grant or contract as PI or Co-PI, whereas 1 respondent indicated being an evaluator or research director on 6 funded grants. The average number of grants submitted by participants was 4.4. Participants who had submitted a grant tended to be older and to have held their terminal degree for a longer time. Thirty-seven of the total grant/contract submissions were by respondents older than the mean age of 43 years, and 41 of the total grant/contract submissions were by respondents who had received their terminal degree more than 15 years ago. The latter respondents also accounted for 15 of the 19 funded grants. No difference was noted in the rate of grant submission by terminal degree type.

Several respondents also indicated the following grant roles: qualitative analyst on 2 grants; research writer, administrator, project director, or evaluator on 8 grants; co-investigator on 2 grants; and consultant on 4 grants. Twelve of these grants were funded, with 4 training grants representing approximately $1.7 million and 8 research grants representing approximately $1.3 million.

Evaluation of GGP components

Table 2 shows the evaluation of each GGP component. Most respondents indicated that the concept paper and the grant-writing components were very helpful. Additional components indicated as very helpful included networking, mock reviews, consultant mentoring, and peer groups. Compared with the concept paper and grant-writing components, the statistics and the mentoring components were rated as less helpful. Expectations of the GGP

Respondents were asked to describe their major expectations of the GGP. Most respondents replied that they expected to receive help in grant writing, to better understand the grant application process, and to develop helpful relationships with experienced researchers.

All of the respondents indicated that most of their expectations were met. However, respondents were mixed regarding the mentoring process. Although 8 indicated that the mentoring was very helpful, some respondents had difficulty in identifying a mentor or were never successful in finding a mentor, whereas others indicated problems with long distance mentoring.

Strengths of the GGP

Respondents indicated that program strengths include enhancing the value of family medicine research, receiving guidance and feedback, learning from the experiences of their peers, and developing grant-writing skills. Respondents specifically mentioned the focus on the NIH grant process and the mock study section as strengths.

With respect to how the GGP contributed to their career, 7 of the respondents said that the program helped them connect with other researchers and gave them confidence in grant writing. In other instances, the GGP program helped respondents to achieve a more secure position, obtain a promotion, start a career, or change academic departments/institutions. As one respondent mentioned, “I think more critically and more daringly . . . I [was] promoted after that from assistant to associate.”

Most respondents indicated that the GGP program enabled them to mentor, teach, and encourage other faculty members in their department in their grant-writing efforts. As one respondent mentioned, the GGP experience ”. . . has helped me to bridge the gap between clinicians and researchers.”

Challenges of the GGP

The most specific challenge of the GGP program mentioned by respondents was the variable commitment of mentors and the lack of having a mentor with expertise in a respondent’s area of research. The major barriers to respondents toward achieving their research goals were (1) the lack of a supportive research environment including a lack of financial support, few peers engaged in research, a lack of local mentoring, and infrastructural and administrative barriers (eg, no links to statisticians, confused priorities within department, and lack of diversity and foresight in department); (2) the lack of time to do research while participating in clinical duties and academic activities; and (3) the lack of information about funding opportunities at their institution. One respondent mentioned not having any external barriers but that the GGP experience “. . . did cause me to reevaluate and ultimately redefine my research goals.”

When asked what skills they learned during the GGP that helped them address these barriers, several respondents indicated (1) time management, including how to realistically organize the research process; (2) the ability to make connections with outside mentors and consultants; and (3) confidence development, such as contacting a program specialist despite poor information from their institution. However, respondents who mentioned the barrier of a nonsupportive research environment indicated that skills to address this barrier were not taught in the GGP program. Recommendations for improving the GGP

Suggestions for improving the GGP program included having 2 GGP entry levels (for beginning researchers and experienced researchers), continuing networking opportunities for “graduates,” addressing the need for female mentors, following up with consultations and mentoring, and having a mentor with expertise in participants’ area of research.

With respect to advice to future applicants, respondents mentioned being more aggressive in working with a mentor, having reasonable expectations, having long-term support from the department, using time efficiently to network, and reflecting on one’s motivation toward an intensive research career. As one participant said, “I would support the application 100% and notify them that it is one of the best investments of time they can make for their academic career.”

Discussion

The respondents’ comments made clear that the GGP is an “excellent place to start a career.” Most respondents appeared to regard the program as well worth the time and effort invested. Aside from some adjustments, such as consistency in mentoring and accommodating the research focus and skill level of participants, it is “. . . very important to continue [the] effort to train new researchers.” Interestingly, the GGP had a substantial impact on the extent to which many participants are now teaching and mentoring others to help develop the skills necessary for meaningful research.

Moreover, within 1 to 2 years of the program participants achieved a remarkable track record of grant submissions. Although for some participants “. . . breaking in [getting grant awards] still seems mysterious and forbidding,” they appreciated the opportunity to learn how to write a coherent concept paper, how to apply “good methods,” the importance of a track record, and to learn from experienced researchers.

As a result in large part of this evaluation, funding for the GGP has been continued with support from NAPCRG, the American Academy of Family Physicians, the STFM, the College of Family Physicians of Canada, and the home departments of participating academic departments and community-based residency programs.

Family medicine is poised to generate new knowledge through the development of its own research agenda-one that focuses on integrated, relationship-centered patient, family, and community health care.⁷ Young researchers, however, need to be nurtured, mentored, and organizationally supported.⁸ Research initiatives such as the GGP represent a collective enterprise, a building of coalitions toward the development of a culture of family medicine research. Continued research success requires commitments from both individuals and departments if family medicine research is to grow and prosper.