Skip to main content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Provider Retention in High Need Areas

Publication Date
Dec 21, 2014

Final Report
Sebastian Negrusa, Projesh Ghosh and John T. Warner

Prepared for: Assistant Secretary for Planning and Evaluation

Submitted by: The Lewin Group, Inc.

 

Acknowledgements

Sebastian Negrusa, PhD, John Warner, PhD and Projesh Ghosh, PhD contributed to this report. We benefited greatly from discussions with members of our technical expert panel on the recruiting of providers into the National Health Services Corps (NHSC) program and their retention in under-served areas. We also gratefully acknowledge the input from Caroline Taplin, the ASPE Project Officer. This study was conducted by The Lewin Group under contract number HHSP23320095639WC, task order number HHSP23337023T, with the HHS’s Office of the Assistant Secretary for Planning and Evaluation. The authors take full responsibility for the accuracy of material presented herein. The views expressed are those of the authors and should not be attributed to ASPE or HHS.

Executive Summary

Since 1972, the National Health Services Corps (NHSC) has provided scholarships and loan repayment incentives for a large number of primary care, mental health, and dental health clinicians to work in Health Professional Shortage Areas (HPSAs) across the country. The NHSC deploys annually almost 9,000 health care professionals to thousands of sites across the country. These clinicians include physicians, dentists, certified nurse practitioners, certified nurse-midwives, physician assistants, registered dental hygienists, health service psychologists, licensed clinical social workers, psychiatric nurse specialists, marriage and family therapists, and licensed professional counselors. The objective of this study was to examine short- and long-term retention in high-need areas of providers who participated in the NHSC’s Loan Repayment Program (LRP) and Scholarship Program (SP) and compare their retention with retention of non-participants. We also conducted multivariate regression analyses to provide additional insights into the individual-level factors and local area characteristics that are associated with retention of program participants in HPSAs.

For this study, we constructed two provider-level analytic datasets based on NHSC administrative datasets and other publicly available and proprietary datasets. The first analytic dataset was constructed using NHSC data, Provider360 data (a proprietary dataset from OptumInsight Corporation, including detailed information on virtually the entire population of medical providers) and Medicare provider data. Using a number of individual-level variables, we matched the NHSC administrative files with Provider360 data and then with a Medicare provider file, obtained by rolling up Medicare claims at the provider level. The resulting analytic file contains annual information on over 1 million non-NHSC providers as well as on 8,973 NHSC participants (out of the total of 22,703 participants from NHSC data). The resulting dataset allows us to track the annual location of participants and non-participants (at the zip code level) over the timeframe between 2005 and 2011. The main advantage of this analytic dataset stems from the fact that it allows us to track providers after they complete their NHSC service.

The second analytic dataset relied mainly on NHSC and Provider360 data and its main feature was that it provided information on the NHSC providers’ location in two points in time: the year of NHSC program termination and December 2013 (i.e., the time when Provider360 information was recorded). The number of NHSC participants in this dataset increases to about 18,500. Nonetheless, once we compared the HPSA retention rates of participants after program completion from the first analytic dataset with the retention rates obtained with the second analytic dataset, we found them to be virtually indistinguishable from each other for primary care and mental health providers.1 We concluded that the first dataset provided a sufficiently representative picture of the entire sample of NHSC participants, and in addition it provided us with the opportunity to track providers’ location yearly. As a result, most of our quantitative analyses were conducted using this dataset.

It is also important to note that we dropped from the retention analyses the NHSC participants who left service in 2013, the last year of our timeframe. Of the initial sample of 8,973 participants we identified in the first analytic dataset, we ended up using a number of 6,296 participants, while of 18,500 participants in the second analytic dataset, we ended up using a number of 11,210 participants.

To guide the interpretation of our empirical analyses, we constructed a conceptual framework in which we modeled the location choices of providers across HPSA and non-HPSA locations. This framework yielded a sharp prediction – retention of NHSC participants in HPSAs after the completion of their obligations can never be higher than the retention of providers who choose to locate in those areas without participating in the program. The most important ingredients in our theoretical model were the concepts of being ‘fit’ for the program and the individual provider’s ‘preference’ for a HPSA location (both depending on individual characteristics that are unobservable in the data available for this project). The correlation between these two variables has a direct implication for the retention of participants in HPSAs. In the limiting case where acceptance into the NHSC program is based solely on preferences for being in a HPSA location, the program selects individuals who would have served in high-need areas even in the absence of the program, and therefore retention differences between participants and non-participants are nil. As the correlation between location preferences and program fit weakens, the program tends to select - at least to some extent - individuals who would not have gone to high-need areas in the absence of the program. In this case, participant retention after program completion will tend to be lower than non-participant retention. Somewhat counterintuitively, a lower retention rate for participants is a signal of the program’s success (not failure) in attracting to high-need areas providers who would not have located there in the absence of the program. As we show in a model simulation in Chapter VIII, the number of provider-years in HPSAs the NHSC obtains from participants after program completion is highest when the correlation between fit for the program and HPSA preference is zero.

We defined four measures of retention at the provider level:

  • Serving in the same HPSA and in the same county (‘same HPSA – same county’). This variable takes the value of 1 if the NHSC provider remains in the same county as the one where he or she served in the NHSC, and 0 otherwise.

  • Serving in a HPSA in another county (‘HPSA – other county’). This variable takes the value of 1 if the NHSC provider remains in a HPSA that is located in a different county than the one in which he or she served while in NHSC service, and 0 otherwise.

  • Serving in a non-HPSA from the same county (‘non-HPSA – same county’). This variable takes the value of 1 if the NHSC provider moves to a non-HPSA area from the county where he or she served while in the NHSC, and 0 otherwise.

  • Serving in a non-HPSA in another county (‘non-HPSA – another county’). This variable takes the value of 1 if the NHSC provider moves to a non-HPSA area from another county than the county he or she served while in the NHSC, and 0 otherwise.

Combining the first two measures, we also constructed an ‘any HPSA’ measure, taking the value of 1 if the provider remains in any HPSA and 0 otherwise. To ensure comparability across all providers, we defined these measures for non-participants as well. While in the case of participants, the reference point in time was the end of the NHSC service, for non-participants we chose the first year (“start year”) that the non-participants appear in Medicare data as their reference point-in-time. Calculating these measures for each cohort, we were able to construct retention rates one year after separation from NHSC, two years after separation from the NHSC and so on. In the case of non-participants the annual retention rates were calculated as one year since start year, two years since start year and so on for each cohort.

Using the first analytic dataset, we found that about 49% of NHSC primary care participants were located in the same HPSA one year after obligation completion, and 82% were located in any HPSA location (Figure ES.1). By the 6th year after obligation completion, 35% of participants were located in the same HPSA where they served during NHSC service, and 72% of them were in any HPSA location. Consistent with the main prediction of our theoretical model, non-participant retention in HPSAs is higher, with the difference being much bigger for retention in the same HPSA than retention in any HPSA location. Another important finding was that much of the geographic mobility of participants was from one HPSA location to another HPSA location. Also, after an initial higher mobility, participants have better retention in HPSAs than non-participants.

Figure ES. 1: Retention Rates of NHSC Participants and Non-Participants—Primary Care

Retention Rates of NHSC Participants and Non-Participants—Primary Care

Figure ES. 2: Retention Rates of NHSC Participants and Non-Participants—Mental Health

Retention Rates of NHSC Participants and Non-Participants—Mental Health

Similar to primary care HPSAs, non-participants in mental health HPSAs were much more likely to stay in the same HPSA than participants (Figure ES.2). Their retention rate declined by 3-4 percentage points each year since the start year, while the retention rate in any HPSA declined at a lower rate, about 2-3 percentage points. Also, the retention rates in any mental health HPSA was very similar across participants and non-participants, especially in the further out years.

Next, we estimated multivariate regression models at the provider level in which we modeled the ‘same HPSA’ and ‘any HPSA’ outcomes as a function of participation in NHSC programs, a set of individual-level characteristics (age, gender, provider type), Census division indicator variables and local area characteristics at the zip code level (average family income, poverty rate, percent White, percent Black, fraction of the population over 25 years of age with a high school degree and percent of the population over the age of 65).

As shown in Figure ES.3, in the first year since separation/start year, NHSC participants are 37.0% less likely to remain in the same HPSA relative to non-participants. This difference was obtained by netting out the impact of other (observable) individual socio-demographic and local area characteristics. Given that the unadjusted difference in the retention rate in the same HPSA in the first separation/start year is 42.8 percentage points (=83.5-40.7, from Figure ES.1), it follows that 86.4% (=37/42.8) of the observed difference in primary care ‘same HPSA’ retention is explained by NHSC participation. A similar fraction, 85.6% (=13.7/16.0), in the observed retention difference was explained by NHSC participation in the case of primary care ‘any HPSA’ in the first separation/start year. The other ratios between adjusted and unadjusted retention differences in retention between participants and non-participants remained similar for the other further out separation/start years, for both primary care ‘same HPSA’ and ‘any HPSA’ measures.

Figure ES. 3: Differences in the Participants’ Retention Probability Relative to Non-Participants—Primary Care

Differences in the Participants’ Retention Probability Relative to Non-Participants—Primary Care

Figure ES.4 presents the regression-adjusted retention differentials by NHSC participation for mental health HPSAs. The retention differentials are lower across the board for the ‘same HPSA’ measure than in the case of primary care HPSAs. As shown in Figure ES.2, the unadjusted retention in ‘any HPSA’ was higher for non-participants in the first separation/start years than that of participants. Nonetheless, after accounting for individual-level and local area characteristics, there was no statistically significant difference between the retention of participants and non-participants in mental health HPSAs for any of the separation/start years.

Figure ES. 4: Differences in the Participants’ Retention Probability Relative to Non-Participants—Mental Health

Differences in the Participants’ Retention Probability Relative to Non-Participants—Mental Health

Other findings from the regression analyses include the following. First, HPSA retention rises with age and local characteristics, but differences by gender, discipline, and Census division are small. Second, as reflected by regression estimates showing that providers have higher retention in poorer and less educated communities, providers select into HPSAs based on their preferences for serving underserved populations.

1 This finding holds despite the fact that pediatricians are under-represented in Medicare data.

Introduction

The National Health Service Corps (NHSC) is administered by the Bureau of Health Workforce (BHW) in the Health Resources and Services Administration (HRSA). The NHSC was originally designed to address geographic maldistribution of the health care workforce by increasing the number of health care professionals in areas designated by HRSA to be Health Professional Shortage Areas (HPSAs). Under Assistant Secretary for Planning and Evaluation (ASPE) leadership in coordination with HRSA, The Lewin Group is focusing on two programs within the NHSC: the Loan Repayment Program (LRP) and the Scholarship Program (SP). While the LRP has expanded significantly since 2009, the SP is constrained by budget policy. In FY 2014, individuals participating in LRP could receive up to $30,000 for a 2-year commitment in a HPSA with scores between 0 and 13 and up to $50,000 for a 2-year commitment in a HPSA with scoresof14 or higher. However, the statute allows for up to $35,000 per year (or $70,000 over two years). The Scholarship Program offers funding for a maximum of four years for: school tuition, required fees, education costs and a monthly support stipend. In exchange, SP participants agree to provide primary health care in a HPSA for a number of years that is equal to the number of years they were scholarship recipients.

The NHSC deploys almost 9,000 health care professionals to thousands of sites across the country annually (National Health Services Corps, 2012). In the FY 2015 Department of Health and Human Services (DHHS) budget, President Obama proposes boosting the National Health Services Corps to 15,000 a year over the next five years (DHHS Budget, 2014). These clinicians include primary care physicians, primary care certified nurse practitioners, certified nurse-midwives, primary care physician assistants, dentists, registered dental hygienists, health service psychologists, licensed clinical social workers, psychiatric nurse specialists, marriage and family therapists, and licensed professional counselors. These professionals deliver critical medical, dental, and mental health services in geographic areas, facilities, and populations that have limited access to health care services (Health Resources and Services Administration, 2013). In exchange for a service commitment (typically 2-5 years) the NHSC provides students with medical education incentives through the LRP and SP programs. Upon the conclusion of an initial service obligation period, NHSC providers may apply for additional loan repayment funding in return for further service.

In September 2013, ASPE awarded The Lewin Group a contract to examine short- and long-term retention in high-need areas of providers who participated in the LRP and SP programs and compare their retention with retention of non-participants working in those areas. Important questions for ASPE and HRSA is how many providers who participate in the NHSC loan repayment and scholarship programs remain in high need areas once they have completed their contract obligations and how their retention compares with the retention of providers in high need areas who did not participate in the program. This study addresses these questions using data from the period 2000-2013. Several past studies have addressed these questions, but with now often dated data obtained mostly by small scale surveys that have focused almost exclusively on physicians. In addition to physicians, this study will examine retention of non-physician providers, including nurse practitioners, physician assistants, mental health and dental care clinicians.

We start with a detailed survey of the literature about the NHSC program, including studies of how participant retention in high-needs areas compares with non-participant retention. Next, our empirical approach encompasses various data methods to evaluate the retention of NHSC providers beyond their initial service obligation. We utilize several large-scale administrative databases to track providers over time and compare the NHSC participant retention with the retention of comparable non-NHSC providers in the same HPSAs. In this document we produce detailed statistics on the number of years providers remain in the same HPSA after completion of their initial contract, whether they locate in a different HPSA, whether they remain in the same area but outside of HPSAs, and whether and when they move to other geographical areas (HPSAs or non-HPSAs). These statistics are broken down by provider type, age and other relevant characteristics. Next, we compare these trends with the retention and migration trends of non-NHSC providers that have similar characteristics and who at some point serve in the same HPSAs as the NHSC participants.2

In this study we also specify a formal economic model of individual geographic location decisions and apply that model to the NHSC programs, with an emphasis on the LRP. The model isolates the key factors influencing geographic location decisions, and it explains why some individuals might choose to locate in areas that others avoid. The model also explains when geographic mobility will be high and when it will be low. The general model of location decisions is modified to account for the essential features of the NHSC program. In particular, we show that the retention of participants in high-need areas after they complete their obligations depends crucially on the way that NHSC selects participants into the program.

To measure this relationship between enrollment, retention and monetary value of the NHSC programs we conduct econometric analyses to estimate the effect of LRP and SP on enrollment in the NHSC workforce and retention of participants in health care shortage areas. In these analyses we attempt to control for individual socio-demographic characteristics as well as for the multiple unobservable factors that are associated with the individual provider’s decisions to: enroll in NHSC; continue with NHSC under a new contract; stay in the same location after NHSC service completion; move to another HPSA after service completion; or move to a non-HPSA after service completion. Some of these unobservable characteristics are the preference for serving in rural/underserved areas, financial constraints or factors that may make a provider more likely to prefer to work in a certain location regardless of participation in NHSC programs (e.g., being close to one’s place of birth).

Although NHSC continues to achieve gains in the recruitment and retention of clinicians, further progress may be desirable. Today, 21 percent of the nation’s population resides in 5,800 Federally-designated shortage areas, and this proportion is climbing (NCSL, 2011; HRSA, 2013). Shortages of primary care services in underserved areas are likely to be further exacerbated, with the Affordable Care Act (ACA) coverage expansion expected to affect underserved areas disproportionally. While rural communities do not necessarily equate to underserved areas, the discrepancy between clinician need and supply in rural areas is sizeable—nearly 21 percent of the US population resides in rural areas, but only 9 percent of the physician workforce practices in such settings (Health Resources and Services Administration, 2011(a)).

A detailed survey of the existing literature relating to the NHSC scholarship and loan repayment programs is contained in Appendix A. The main body of the report is organized as follows. Chapter II provides a detailed discussion of our data sources and main measures. Chapter III presents summary statistics obtained with these data sources, including measures of the retention and geographic mobility of NHSC participants and non-participants. Chapter IV develops an economic model of location choices and uses that model to derive predictions about how the retention of NHSC participants in high-need areas will compare with the retention of non-participants. Chapter V uses the data to derive estimates of retention differences between participants and non-participants that control for a host of observable factors, including the provider’s medical discipline, age and gender as well as other factors such as local area median income and percentage of the population in poverty. Finally, Chapter VI simulates the economic model constructed in Chapter IV under different assumptions about the key parameters in the model. The empirical findings are interpreted in light of these simulations. Chapter VII concludes the report.

2Holmes (2005) pointed out that relating NHSC participation to HPSA designation may pose some problems, as, for instance, the areas that may be most underserved may not apply for HPSA status. An alternative approach would be to explore the enrollment and retention of NHSC providers in areas where the provider to patient ratios indicate shortages of primary care providers. However, this is beyond the scope of the current study.

Data Sources and Main Measures

To accomplish the goals of this study, we constructed two provider-level analytic datasets based on administrative datasets and other publicly available and proprietary datasets. We describe each data source in detail below and then we discuss the construction of the two analytic datasets.

Data Sets

NHSC Administrative File

The NHSC administrative file provided to us by HRSA represents a panel of 22,703 participants who entered the NHSC programs over the period 2000-2013. The NHSC administrative file contains information on where each participant was located each year during the participant’s program (down to the zip code level). Participants are tracked annually from their entry year until 2013, but only while they are in the program. Of the total number of providers, 10,123 are physicians, 6,850 are nurse practitioners (NP) or physician assistants (PA) and the remaining 5,730 individuals represent other providers, such as dental or behavioral health providers. This database contains information on participant demographics and award/service characteristics, including: age, gender, race/ethnicity, award year, entry year, type of award, length of initial service obligation, funds received, practice type, provider type, location, and separation year. Table II.1 provides a breakdown of the number of NHSC enrollees by entry year, provider type, gender, program type, average age on entry, and race/ethnicity.

Table II. 1: Select Socio-Demographic Characteristics of NHSC Enrollees by Entry Year

Entry Year

2000

2001

2002

2003

2004

2005

2006

2007

2008

2009

2010

2011

2012

2013

Total

918

1,062

1,462

2,395

3,005

3,412

3,367

3,209

3,142

4,081

6,710

9,650

9,424

8,275

Type

MDs

591

684

872

1,362

1,680

1,989

2,005

1,951

1,945

2,331

3,274

4,331

4,205

3,730

NP/PAs

227

254

391

636

807

863

798

764

725

1,044

2,008

2,975

2,827

2,410

Other

100

124

199

397

518

560

564

494

472

706

1,428

2,344

2,392

2,135

Gender

Females

520

619

910

1,504

1,876

2,167

2,166

2,085

2,070

2,769

4,740

6,932

6,823

6,002

Males

398

443

552

891

1,129

1,245

1,201

1,124

1,072

1,312

1,970

2,718

2,601

2,273

Program

LRP

807

880

1,185

2,057

2,620

2,952

2,839

2,632

2,563

3,475

6,148

9,125

8,892

7,752

SP

111

182

277

338

385

460

528

577

579

606

562

525

532

523

Age

36.5

36.8

37.3

37.4

37.4

37.3

37.4

37.3

37.3

37.1

37.4

37.8

37.9

38.0

Race/Ethnicity

White

89

250

475

918

1,245

1,408

1,375

1,332

1,331

1,774

3,622

5,876

5,976

5,290

Black

18

35

56

99

129

183

186

175

183

228

554

972

1,031

975

Hispanic

801

759

881

1,283

1,492

1,657

1,663

1,553

1,455

1,802

1,735

1,581

1,258

1,013

Other

10

18

50

95

139

164

143

149

173

277

799

1,221

1,159

996

The number of new NHSC participants has increased substantially over the last few years, with most of this increase reflected in an expansion in the number of women, non-physicians and White providers. In Appendix Table B.1 we present the distribution of the NHSC workforce by provider discipline.

In Table II.2 we present the number of records on NHSC providers by entry cohort.

Table II. 2: Number of Records on NHSC Providers by Entry Cohort

Entry Year

Years Served in NHSC

1

2

3

4

5

6

7

8

9

10

11

12

13

14

2000

918

645

351

221

117

65

41

24

22

19

15

13

11

8

2001

438

438

251

161

90

43

25

16

13

10

7

4

3

 

2002

714

714

363

219

121

63

40

23

14

13

9

6

 

 

2003

1,237

1,237

584

313

181

108

66

45

35

22

15

 

 

 

2004

1,151

1,151

545

318

156

101

55

36

23

17

 

 

 

 

2005

1,342

1,342

611

341

178

107

64

38

24

 

 

 

 

 

2006

1,071

1,071

563

363

172

101

59

28

 

 

 

 

 

 

2007

1,032

1,032

571

339

178

90

44

 

 

 

 

 

 

 

2008

989

989

622

391

193

103

 

 

 

 

 

 

 

 

2009

1,849

1,849

1,212

647

324

 

 

 

 

 

 

 

 

 

2010

3,425

3,425

1,779

1,127

 

 

 

 

 

 

 

 

 

 

2011

4,009

4,009

1,835

 

 

 

 

 

 

 

 

 

 

 

2012

2,382

2,382

 

 

 

 

 

 

 

 

 

 

 

 

2013

2,146

 

 

 

 

 

 

 

 

 

 

 

 

 

Total

22,703

 

 

 

 

 

 

 

 

 

 

 

 

 

The main disadvantage of the NHSC administrative data file is that it does not include information on where participants were located after program completion. Also, it does not contain the participant’s National Provider Identification (NPI) number, or other unique identifiers which would enable us to track their location after program completion. We therefore rely on additional data sources to identify the participants’ NPI and then subsequently determine where they were located after program completion. These four data sources, which are also used to identify non-participant providers, include:

  • AMA Physician Master Files of various years since 2000;

  • Medicare Providers File developed from Medicare claims data over the period 2005-2011; and

  • Optum Corporation’s Provider 360 File, a proprietary file containing comprehensive information on most medical providers in the United States.

Finally, we employ data from HRSA with information on HPSA designations, types, disciplines, sites and scores. We use the most recent file of these data, which was compiled in December 31, 2013.

Below we discuss these files in turn and highlight their advantages and limitations for the current project.

AMA Physician Master Files

The AMA Master file was begun in 1986 and has been tracking physicians since then. Multiple years of AMA data were extracted to obtain information about provider location over multiple years. Each year of the AMA data file contains current information on the physicians’ locations and practices, along with information on their training. The AMA is the single most comprehensive data source on physicians practicing in the US. In theory, linking NHSC administrative data with the AMA data, it is possible to track the practice location and retention information of NHSC enrollees over the years. Each observation includes location data (such as zip code), medical school, specialty, graduation year, birth date, race, and gender. We make use of complete AMA Physician Master Files for the following years: 2000, 2007, 2008, 2010-2012, and partial files (containing only data on primary care providers) for 2003 and 2005. The number of records increased steadily from about 1.0 million physicians in 2000 to about 1.3 million in the 2011 file. However, the main limitations of these files are that they include only physicians and that the providers’ location is updated infrequently, at intervals that are often longer than one year.

Provider360

Provider360 (P360) is a comprehensive provider database developed and maintained by OptumInsight, the Lewin Group’s parent company. Optum first developed Provider360 in the early 2000s and since then has updated it on a monthly basis. Optum links numerous private and public databases to create P360, and it gathers information about each provider’s demographics, education and training, NPI and DEA numbers, and location. Importantly, Provider360 contains both physician and non-physician providers, including PAs, NPs, and mental health and dental care clinicians.

This dataset covers virtually all currently active providers and includes most of the socio-demographic variables that we observe in the NHSC data, like provider demographics or provider type, as well as additional useful variables, like the provider’s NPI, medical education and practice affiliation. The large number of common variables appearing both in the NHSC files and P360, like name, birthdate, gender and others, allows for a link between the two datasets with a good match rate.

The main limitation of P360 is that it was not designed to provide panel information about providers, so we can tell where providers are located now but not each year in the past. However, even without historical P360 data retention patterns of NHSC providers can be constructed by comparing their locations upon NHSC program completion with their current locations in P360. For instance, we can look at the cohort of students finishing their medical training in, say, 2006 and determine how many of them are still serving in HPSAs as of the current year. We return to this point in the next section of this chapter. Other useful variables available in P360 include the provider’s degree, specialty, medical school attended, residency institution, license, DEA number, hospital affiliation, practice type, practice NPI, and sanctions against the provider.

Medicare Providers File

In order to track participants and non-participants over time, we constructed a provider level dataset using the Medicare claims of providers who billed Medicare between 2005 and 2011. In this dataset the unit of observation is the provider-year, meaning that each Medicare provider is observed annually along with his or her geographical location at the zip code level. The provider’s location in a given year is based on the most frequent zip code associated with that provider’s claims during that year. The dataset includes the provider’s NPI, which allows us to identify NHSC participants and track their location in the years after their service completion.

The Medicare provider file contains 5,757,405 observations on 1,099,836 unique providers. The breakdown of unique providers by type is: 626,836 physicians, 123,223 NP/PA’s, 54,168 DO’s and 295,615 other providers.3 Table II.3 shows the number of providers by type in each year.

Table II. 3: Number of Providers by Type in Each Year (Medicare Provider Data)

Year

MDs

NP/PAs

DOs

Other

Total

2005

428,574

48,018

34,220

140,748

651,560

2006

455,974

55,222

36,941

151,324

699,461

2007

502,661

69,647

41,424

191,968

805,700

2008

521,927

78,950

43,808

205,540

850,225

2009

535,191

86,033

45,953

214,118

881,295

2010

549,627

93,847

47,948

223,727

915,149

2011

564,707

102,335

50,115

236,858

954,015

As in the case of another provider data, available to the Lewin team known as the de-Identified Normative Health Information (dNHI) data, Medicare providers also appear to be well distributed across all states during the period between 2005 and 2011 (Appendix Table B.2).4

3 These providers include both primary care practitioners and non-primary care practitioners.

4 We performed the same exercise with the OptumInsight’s medical claims database called the De-Identified Normative Health Information (dNHI) as we did with the Medicare claims data. The dNHI database includes all claims submitted to United Healthcare, as well as claims that are processed for a number of other insurers. Although it contains information on a large number of providers, the dNHI database is arguably not representative for the entire population of US providers. Also, since most providers in dNHI work in private practices, there was a risk of identifying in this dataset a disproportionate fraction of NHSC participants who work in non-HPSAs with patients that are typically not underserved. Moreover, the number of NHSC participants we identified in dNHI in addition to those identified in the Medicare data was less than 1,000. We therefore decided not to use the dNHI data for the current project.

Data on HPSA Designations and HPSA Scores

HRSA also provided Lewin with a detailed file containing information on all HPSA sites as of December 2013. As of December 2013, there were 5,976 primary care HPSAs, 4,758 dental care HPSAs and 3,876 mental health HPSAs. This file includes information on each HPSA: ID, name, status, type, discipline, HPSA score and detailed geographic identifiers. In Table II.4 below we present the number of HPSA sites by HPSA disciplines (i.e., primary care, dental health and mental health) and by the main HPSA types (facility HPSAs, single county HPSAs, Census tract HPSAs and minor civil division HPSAs).5

Table II. 4: Distribution of HPSA Sites by Disciplines and Types

Discipline

Facility

Single County

Census Tract

Minor Civil Division

Total

Primary Care

Count

3,270

1,746

9,981

3,384

18,381

Percent

17.79

9.50

54.30

18.41

100

Dental Health

Count

2,676

1,695

7,334

1,083

12,788

Percent

20.93

13.25

57.35

8.47

100

Mental Health

Count

2,715

2,408

4,401

394

9,918

Percent

27.37

24.28

44.37

3.97

100


 

5 A minor civil division is a term used by the Census to designate the primary governmental or administrative divisions of a county, such as a civil township, precinct, or magisterial district.

Construction of Analytic Datasets

The first analytic dataset was constructed using NHSC data, P360 data and Medicare provider data. Using the name, birthdate, gender and a number of other variables, we matched the NHSC administrative file with the P360 data. This way we were able to uniquely identify about 18,500 of the 22,703 NHSC participants in the P360.6 This match gives us the participants’ NPIs as well as other important information including their location’s zip code in 2013. Almost all of these matches were associated with a valid NPI (i.e., for about 17,900 NHSC participants). The matches without an NPI correspond to NHSC participants that do not have an NPI.

The next step was to link the merged P360-NHSC file with the Medicare Provider file by NPI. The analytic file contains annual information on over 1 million non-NHSC providers as well as on 8,973 NHSC participants (out of the 17,900 participants with a valid NPI). The dataset allows us to track the annual locations at the zip code level for each year individuals are in the Medicare Provider data (i.e., over the timeframe between 2005 and 2011). Figure II.1 below summarizes the main steps we took to create the first analytic dataset.

Figure II. 1: Steps to Create the First Analytic Dataset on NHSC Providers

Steps to Create the First Analytic Dataset on NHSC Providers

As our chosen level of geography was the zip code, the next step was to determine whether the zip code associated with each provider’s location was part of a HPSA or not. For this purpose we used the dataset we received from HRSA containing information on all HPSAs and constructed an algorithm to determine whether a provider’s zip code was part of a single-county HPSA, Census tract HPSA, Census division HPSA, or a facility HPSA. Some Census tract and Census division HPSAs may straddle two or more zip codes, in which case we determined that if more than half of the zip codes’ population was within the bounds of a Census tract or Census minor civil division HPSA, those zip codes would be flagged as HPSAs. Also, we flagged the HPSA zip codes as primary care, mental health and dental HPSAs, using HRSA’s classification of HPSAs by type. An inherent limitation of this approach is that we only used data on HPSAs as of December 2013. To the extent various areas lose or gain HPSA status over our period of analysis, we may erroneously place providers in HPSAs (or non-HPSAs) in the years prior to 2013.

To ensure a consistent classification of NHSC providers and non-NHSC providers by their medical discipline, we mapped the detailed health care occupation (available in P360) of each non-NHSC participant into the corresponding NHSC discipline. The list of NHSC disciplines is presented in Table II.5 along with the distribution of providers by these disciplines in the overall population of NHSC participants and in the sample of participants we identified in the first analytic dataset. Table II.5 also provides the average age of providers (at entry in NHSC service) and the distribution by gender and HPSA type.

Table II. 5: Comparison between NHSC Providers in the Overall Population and in the First Analytic Dataset

Discipline

All NHSC Providers

NHSC in First Data Set

Allopathic Physician

4,465

2,960

Osteopathic Physician

1,382

1,010

Chiropractor

14

10

Certified Nurse Midwife

582

227

Dentist

2,508

36

Health Service Psychologist

1,768

597

Licensed Clinical Social Worker

2,208

635

Licensed Prof Counselor

1,988

66

Marriage and Family Therapist

348

14

Nurse Practitioner

3,735

1,836

Pharmacist

26

0

Physician Assistant

3,115

1,483

Psychiatric Nurse Specialist

81

44

Registered Dental Hygienist

483

19

TOTAL

22,703

8,937

Age (at Entry)

36.9

37.7

Gender

Males

6,820

3,249

Females

15,883

5,688

HPSA Type

Primary Care

12,452

6,985

Mental Health

7,260

1,897

Dental

2,991

55

TOTAL

22,703

8,973

We completed the construction of the first analytic dataset by eliminating the following groups of non-NHSC participant providers: (i) those who did not fall under any one of the NHSC discipline types; (ii) those who did not serve in HPSAs; (iii) those who served in HPSAs where no NHSC participant served over the 2005-2011 period; and (iv) specialists (cardiologists, dermatologists etc.). The purpose of these data restrictions was to ensure a degree of comparability between NHSC participants and non-NHSC participants who served in HPSAs. We ended up with a total of 202,999 non-participant providers serving in primary care HPSAs and a total of 19,304 non-participants serving in mental health HPSAs. One limitation of the analytic dataset is that it has very few dentists and pediatricians, since these providers are unlikely to file Medicare claims and thus appear in the Medicare data. The main characteristics of non-participants are presented in Table II.6.

Table II. 6: Non-NHSC Providers Serving in Primary Care and Mental Health HPSAs in the First Analytic Dataset

Characteristics

Primary Care HPSA

Mental Health HPSA

Discipline

Non-NHSC Providers

Percent

Non-NHSC Providers

Percent

Physician

168,620

83.1

9,034

46.8

Certified Nurse Midwife

572

0.3

--

--

Dentist

5,409

0.6

--

--

Health Service Psychologist

--

--

4,426

22.9

Licensed Clinical Social Worker

--

--

4,848

25.1

Licensed Professional Counselor

--

--

130

0.7

Marriage and Family Therapist

--

--

2

0.0

Nurse Practitioner

24,632

12.1

864

4.5

Physician Assistant

9,175

4.5

 

 

TOTAL

202,999

100.0

19,304

100.0

Age (at Entry)7

45.8

--

51.7

--

Gender

 

 

 

 

Males

130,046

64.1

9,427

48.8

Females

72,953

35.9

9,877

51.2

TOTAL

202,999

100.0

19,304

100.0

In Table B.3 of Appendix B we present the distribution of providers serving in NHSC in each state over the timeframe of the first analytic data.

Finally, in the case of providers who changed their location from one year to the next we constructed variables measuring the distance of the move by calculating the distance in miles between the zip code centroid of the initial location and the zip code centroid of the next year’s location.

The main advantage of the first analytic dataset stems from the fact that it allows us to track the location of providers after they complete their NHSC service. However, the number of NHSC providers is potentially limited. We therefore constructed a second analytic dataset relying only on NHSC data, P360 data and HPSA data. The main feature of this dataset is that provides information on the NHSC providers’ location in two points in time: the year of program termination and December 2013 (i.e., the time when P360 information is recorded).

The number of NHSC participants in this second dataset increases to about 18,500 (of which 17,983 have a valid NPI). As expected, the distribution of NHSC providers by discipline resembles the distribution from the overall NHSC population more closely (Table II.7). Also, the other characteristics of participants from the second dataset are more similar to the characteristics from overall population of participants. In the next chapter we compare the retention patterns of participants from the first dataset with the retention patterns of participants from the second dataset.

Table II. 7: Comparison between NHSC Providers in the Overall Population and in the Second Analytic Dataset

Discipline

All NHSC Providers

NHSC in Second Data Set

Allopathic Physician

4,465

4,151

Osteopathic Physician

1,382

1,288

Chiropractor

14

12

Certified Nurse Midwife

582

481

Dentist

2,508

1,842

Health Service Psychologist

1,768

1,272

Licensed Clinical Social Worker

2,208

1,573

Licensed Prof Counselor

1,988

1,342

Marriage and Family Therapist

348

257

Nurse Practitioner

3,735

3,058

Pharmacist

26

11

Physician Assistant

3,115

2,476

Psychiatric Nurse Specialist

81

67

Registered Dental Hygienist

483

153

TOTAL

22,703

17,983

Age (at Entry)

36.9

40.0

Gender

Males

6,820

5,775

Females

15,883

12,207

HPSA Type

Primary Care

12,452

10,729

Mental Health

7,260

5,259

Dental

2,991

1,995

TOTAL

22,703

17,983

NOTE: The total number of matched NHSC providers presented in the above table is limited to providers with a valid NPI.

Figure II.2 summarizes the main steps needed to construct the second analytic dataset.

Figure II. 2: Steps to Create the Second Analytic Dataset on NHSC Providers

Steps to Create the Second Analytic Dataset on NHSC Providers

6 The number of unique matches could have been higher, but in the case of about 1,500 participants their name was exactly the same as the name of multiple individuals in P360. As other information on these participants was unavailable in the two datasets, we were unable to uniquely identify these participants in the P360 data.

7 The average age of non-participant providers when they first appear in the Medicare data.

Retention Measures

Using the first analytic dataset we constructed four retention measures at the provider level:

  • Serving in the same HPSA and in the same county. This variable takes the value of 1 if the NHSC provider remains in the same county as the one where he or she served in NHSC, and 0 otherwise.

  • Serving in a HPSA in another county. This variable takes the value of 1 if the NHSC provider remains in a HPSA that is located in a different county than the one in which he or she served while in NHSC service, and 0 otherwise.

  • Serving in a non-HPSA from the same county. This variable takes the value of 1 if the NHSC provider moves to a non-HPSA area from the county where he or she served while in NHSC, and 0 otherwise.

  • Serving in a non-HPSA in another county. This variable takes the value of 1 if the NHSC provider moves to a non-HPSA area from another county than the county he or she served while in NHSC, and 0 otherwise.

To ensure comparability across all providers, we defined these measures for non-participants as well. While in the case of participants, the reference point in time was the end of the NHSC service, for non-participants we chose the first year (“start year”) that the non-participants appear in Medicare data as their reference point-in-time. Calculating these measures for each cohort, we were able to construct retention rates one year after separation from NHSC, two years after separation from NHSC and so on. In the case of non-participants the annual retention rates were calculated as one year since start year, two years since start year and so on for each cohort.

Geographic Mobility of Participants and Non-Participants

Retention Measures Using Longitudinal Data (First Analytic Dataset)

In Figure III.1 we start with retention rates in primary care HPSAs by years elapsed since separation from service (for participants) and by years since start year (for non-participants).

Using data from the first analytic dataset, we find that about 82% of the NHSC participants serve in primary care HPSAs one year after completion of their NHSC service. More than half of participants who are still in primary care HPSAs one year after separation are actually in the same county as the one in which they served while in service (i.e., 49% of participants). There is a fairly steep decline in the retention rate in years 2 and 3 after separation (by about 7 and 3 percentage points, respectively), followed by a leveling off thereafter.

Focusing on non-participants, we note that their retention rates in primary care HPSAs are always higher than the retention rates of participants, both in terms of retention in the same HPSA as well as in terms of retention in a different HPSA. Their retention also drops after we first observe them in the data (their ‘start’ year), but in contrast with participants the move rate out of HPSAs is relatively constant over time. One year after we first observe non-participants in HPSAs, 95% of them are still in HPSAs. Moreover, a very large fraction of them (91% of all non-participants) remain in the same HPSAs where they were first observed. The retention rates in any HPSAs decline by about 3-4 percentage points every year thereafter, while the retention rates in the same HPSA decline at a slightly faster rate (about 5-6 percentage points per year). These rates indicate that once non-participant providers serve in a HPSA, they tend to remain in those areas, and to some extent they migrate from one HPSA to another.

Figure III. 1: Retention Rates of NHSC Participants and Non-Participants—Primary Care

Retention Rates of NHSC Participants and Non-Participants—Primary Care

It is important to note in Figure V.1 that the participants’ retention appears to increase after more than 6 years, while the non-participants’ retention declines at a higher rate than in the previous years. However, these findings should be viewed with caution, because the retention rates for more than 6 years are constructed using only P360 data for 2013, as the timeframe afforded by the Medicare data is only six years (2005-2011).8

In Figure III.2 we present the retention rates in mental health HPSAs. The retention rates of participants are lower in mental health HPSAs than in the case of primary care HPSAs. Also, the decline in retention rates (or the move rate out of HPSAs) is much lower than for primary care providers. For instance, the fraction of participants serving in the same HPSA as during the program is about the same in the first two years since separation (37-38%), declines to 36% and then remains relatively constant at 32-33% thereafter. The retention rate of participants in any mental health HPSA is relatively constant over the years, hovering around 64% and 68%.

Similar to primary care HPSAs, non-participants in mental health HPSAs are much more likely to stay in the same HPSA than participants. Their retention rate declines by 3-4 percentage points each year since the start year, while the retention rate in any HPSA declines at a lower rate, about 2-3 percentage points. It is important to note that the retention rate in any mental health HPSA is very similar across participants and non-participants, especially in the further out years.

Figure III. 2: Retention Rates of NHSC Participants and Non-Participants—
Mental Health

Retention Rates of NHSC Participants and Non-Participants— Mental Health

It is important to note that we dropped from the retention analyses in Figures III.1 and III.2 the NHSC participants who left service in 2013, the last year of our timeframe. Of the initial sample of 8,973 participants we identified in the first analytic dataset, we ended up using a number of 6,296 participants, of which 4,995 are primary care providers and 1,301 are mental health providers.

8 The same caveat applies to Figure III.2.

Retention Measures Using the Second Analytic Dataset

So far, we have discussed retention statistics using only the first analytic dataset, which is based on NHSC, Medicare and P360 data. While this analytic dataset is very useful in that it tracks the participants’ and non-participants’ location over time, the number of participants identified in this dataset may be arguably viewed as small (around 9,000 providers of the total of 22,703). We address this issue by presenting in Tables III.1-III.3 the retention rates of NHSC participants using the second analytic dataset. However, despite the fact that we observe about 18,500 participants in that dataset, we can only construct retention rates as of December 2013, the time when the P360 data was recorded. As in the case of the first analytic dataset, we dropped participants who appear in our data only in 2013, and thus, of the initial 18,500 participants we matched with P360 data, we ended up using a number of 11,210 participants for the retention analyses we present below.

Table III. 1: Retention Rates of NHSC Participants as of December 2013—Primary Care

Year of Exit
from NHSC

HPSA & same county

HPSA & other county

Non-HPSA & same county

Non-HPSA & other county

Total matched
in P360 data

2000

35

54

23

39

151

2000

23.18%

35.76%

15.23%

25.83%

 

2001

34

48

15

43

140

2001

24.29%

34.29%

10.71%

30.71%

 

2002

36

65

11

35

147

2002

24.49%

44.22%

7.48%

23.81%

 

2003

79

88

23

43

233

2003

33.91%

37.77%

9.87%

18.45%

 

2004

149

158

50

93

450

2004

33.11%

35.11%

11.11%

20.67%

 

2005

196

205

45

103

549

2005

35.70%

37.34%

8.20%

18.76%

 

2006

216

188

57

117

578

2006

37.37%

32.53%

9.86%

20.24%

 

2007

184

170

49

94

497

2007

37.02%

34.21%

9.86%

18.91%

 

2008

188

162

42

89

481

2008

39.09%

33.68%

8.73%

18.50%

 

2009

184

158

45

74

461

2009

39.91%

34.27%

9.76%

16.05%

 

2010

229

195

56

90

570

2010

40.18%

34.21%

9.82%

15.79%

 

2011

499

418

128

172

1217

2011

41.00%

34.35%

10.52%

14.13%

 

2012

640

524

131

163

1458

2012

43.90%

35.94%

8.98%

11.18%

 

Total

2,669

2,433

675

1,155

6,932

Total

38.50%

35.10%

9.74%

16.66%

 

The rates in Tables III.1-III.3 indicate the fraction of participants who remain in the same HPSA or any HPSA between the time of service completion and December 2013. As a result, the fraction remaining in HPSAs for the cohort of providers exiting NHSC in 2012 represents a 1-year retention rate, for those exiting in 2011 it is a 2-year rate and so on. Although the rates in Table V.1 are not directly comparable to the rates plotted in Figure V.1, they nonetheless paint a similar picture regarding the retention of participants in HPSAs after service completion. For instance, the one-year retention rate in same HPSA same county of primary care participants who left service in 2012 was 43.9% (Table III.1), compared to 48.5% in Figure III.1. Similarly, the two-year retention rate was 41.0% (Table III.1), while the two-year retention rate was 4.07% in Figure III.1.

Given that the retention rates in Tables III.1-III.3 are inherently cohort-specific, we constructed cohort-specific retention rates using the first analytic dataset (Tables B.4 and B.5 in Appendix B). Tables A.4 and A.5 allow for a direct comparison between the retention rates from the first analytic dataset and the retention rates from the second analytic dataset (in Tables III.1 and III.2). As the retention rates from the first analytic dataset are virtually indistinguishable from the retention rates from the second analytic dataset, we conclude that the first dataset provides a comprehensive and representative picture of the entire sample of NHSC participants.

Table III. 2: Retention Rates of NHSC Participants as of December 2013—
Mental Health

Year of Exit
from NHSC

HPSA & same county

HPSA & other county

Non-HPSA & same county

Non-HPSA & other county

Total matched
in P360 data

2000

16

15

3

4

38

2000

42.1%

39.5%

7.9%

10.5%

 

2001

19

13

2

8

42

2001

45.2%

31.0%

4.8%

19.0%

 

2002

19

20

3

9

51

2002

37.3%

39.2%

5.9%

17.6%

 

2003

41

30

5

21

97

2003

42.3%

30.9%

5.2%

21.6%

 

2004

68

60

14

28

170

2004

40.0%

35.3%

8.2%

16.5%

 

2005

69

58

18

25

170

2005

40.6%

34.1%

10.6%

14.7%

 

2006

107

67

11

28

213

2006

50.2%

31.5%

5.2%

13.1%

 

2007

83

79

16

35

213

2007

39.0%

37.1%

7.5%

16.4%

 

2008

72

76

18

25

191

2008

37.7%

39.8%

9.4%

13.1%

 

2009

81

62

24

25

192

2009

42.2%

32.3%

12.5%

13.0%

 

2010

111

89

26

27

253

2010

43.9%

35.2%

10.3%

10.7%

 

2011

289

195

71

76

631

2011

45.8%

30.9%

11.3%

12.0%

 

2012

398

285

71

108

862

2012

46.2%

33.1%

8.2%

12.5%

 

Total

1,373

1,049

282

419

3,123

Total

44.0%

33.6%

9.0%

13.4%

 

Another advantage of the second analytic dataset was that it allowed us to construct retention rates for providers in Dental Health HPSAs. These rates could not have been constructed with data from the first analytic dataset, as the sample size of dental health providers billing Medicare was extremely limited. As can be noticed in Table III.3, the retention rates of dental health participants were similar to the retention rates of providers in primary care and mental health HPSAs.

Table III. 3: Retention Rates of NHSC Participants as of December 2013—
Dental Health

Year of Exit
from NHSC

HPSA & same county

HPSA & other county

Non-HPSA & same county

Non-HPSA & other county

Total matched
in P360 data

2000

9

10

3

3

25

2000

36.0%

40.0%

12.0%

12.0%

 

2001

11

12

4

10

37

2001

29.7%

32.4%

10.8%

27.0%

 

2002

13

14

5

11

43

2002

30.2%

32.6%

11.6%

25.6%

 

2003

19

16

11

17

63

2003

30.2%

25.4%

17.5%

27.0%

 

2004

30

27

9

23

89

2004

33.7%

30.3%

10.1%

25.8%

 

2005

22

38

13

22

95

2005

23.2%

40.0%

13.7%

23.2%

 

2006

30

44

18

16

108

2006

27.8%

40.7%

16.7%

14.8%

 

2007

21

31

9

25

86

2007

24.4%

36.0%

10.5%

29.1%

 

2008

19

22

9

13

63

2008

30.2%

34.9%

14.3%

20.6%

 

2009

22

26

10

24

82

2009

26.8%

31.7%

12.2%

29.3%

 

2010

29

27

9

14

79

2010

36.7%

34.2%

11.4%

17.7%

 

2011

66

53

16

32

167

2011

39.5%

31.7%

9.6%

19.2%

 

2012

96

82

12

28

218

2012

44.0%

37.6%

5.5%

12.8%

 

Total

387

402

128

238

1,155

Total

33.5%

34.8%

11.1%

20.6%

 

Finally, Tables B.6 and B.7 in Appendix B show no substantial variation in the retention rates of providers by provider type. Primary care participants are slightly more likely to stay in the same HPSA same county and less likely to stay in HPSAs in other counties than NP/PAs, while mental health physicians are in general less likely than NP/PAs to remain in HPSAs. We will return to this issue in Chapter V to determine whether differences in retention rates remain after we adjust for the provider’s age, gender and other individual level and local area characteristics.

Distances of Providers’ Moves

As described in the previous chapter, we constructed a variable measuring the distance of the provider’s move from one location to another by calculating the distance (in miles) between the zip code centroids of the initial and final location. In Table III.4, we present the mean distance of moves by the four retention metrics and by years since separation from NHSC service.

Table III. 4: Average Distance (in Miles) between NHSC Service Location and Current Location, by Retention Metrics and Years since Separation from NHSC

Years Since Separation

HPSA &
same county

HPSA &
other county

Non-HPSA &
same county

Non-HPSA &
other county

Primary Care

1

5.0

228.5

6.4

309.2

2

5.1

280.2

6.1

411.1

3

4.9

349.6

6.2

394.0

4

4.7

370.8

5.7

431.9

5

4.6

398.6

5.5

362.2

6

5.2

400.6

5.4

373.8

>6

4.8

388.8

5.0

378.4

Mental Health

1

3.8

201.4

5.5

301.7

2

3.7

182.2

6.3

292.7

3

4.5

237.6

6.4

345.9

4

3.3

294.0

5.9

384.6

5

3.7

254.1

6.5

318.3

6

3.0

275.6

8.4

503.6

>6

2.9

250.9

7.0

333.5

NHSC participants who remained in the same HPSA same county were, not surprisingly, located on average within 3 to 5 miles from their original NHSC service location, while those who moved to a non-HPSA in the same county were located on average within 5 to 8 miles from their initial NHSC location. Providers who moved to a HPSA outside the county where they served while in NHSC tended to be located further and further away as time since separation went by. For instance, primary care providers moved on average 229 miles away within one year since NHSC separation, 371 miles 4 years since separation and 401 miles 6 years since separation. This trend was less pronounced for mental health providers, who moved within shorter distances to HPSAs in other counties. Finally, all providers going to non-HPSAs in other counties had moves at greater distances than the providers who moved to HPSAs in other counties.

Retention of Participants by Place of NHSC Service

We also analyzed the retention patterns of NHSC providers by the place of service to understand whether the institution type they work in is associated with their propensity to remain in HPSAs after service completion.

Since about half of all NHSC participants work in FQHCs, we investigated the retention of providers by whether they were working in an FQHC at the time of NHSC service completion. Using the second analytic dataset, we found that about 88% of the 8,760 participants who worked in FQHCs were still in the same zip code as of December 2013. In contrast, participants who were in non-FQHCs while in service are substantially less likely to be in the same location at the end of our observation period.

Table III. 5: Comparison of Retention Trends by NHSC Place of Service (FQHC vs non-FQHC locations)

Separation Year

In Same Zip code

In Different Zip code

Total

% in Same Zip code

In FQHC

2000

109

15

124

87.9%

2001

100

15

115

87.0%

2002

117

14

131

89.3%

2003

174

30

204

85.3%

2004

324

38

362

89.5%

2005

390

49

439

88.8%

2006

410

74

484

84.7%

2007

340

46

386

88.1%

2008

321

43

364

88.2%

2009

316

44

360

87.8%

2010

422

54

476

88.7%

2011

886

110

996

89.0%

2012

998

137

1,135

87.9%

2013

2,710

384

3,094

87.6%

Total

7,617

1,053

8,670

87.9%

Not In FHQC

2000

21

73

94

22.3%

2001

16

91

107

15.0%

2002

17

95

112

15.2%

2003

38

160

198

19.2%

2004

75

288

363

20.7%

2005

84

317

401

20.9%

2006

83

366

449

18.5%

2007

76

360

436

17.4%

2008

78

312

390

20.0%

2009

93

300

393

23.7%

2010

100

349

449

22.3%

2011

233

841

1,074

21.7%

2012

329

1,141

1,470

22.4%

2013

768

2,609

3,377

22.7%

Total

2,011

7,302

9,313

21.6%

Economic Model of Location Choices with Application to NHSC Programs

The retention patterns we observed in Figures III.1 and III.2 appear to indicate that participants have lower HPSA retention than non-participants. There may be a suite of factors in addition to NHSC participation that influence the providers’ decisions to first locate in a HPSA and then move out of a HPSA. In this chapter we provide a theoretical model to provide insights into the impact of these additional factors on the observed retention patterns.

Specifically, in this model we aim to: (i) isolate the key factors influencing providers’ location decisions; (ii) explain why some providers locate in areas that others avoid; and (iii) explain when geographic mobility is high or low. We start with a general location choice model and then expand the model to incorporate the incentives offered by the NHSC programs. A technical version of this model is available in Appendix B.

An Economic Model of Location Choice

In general, in any given time period an individual calculates the value (or utility) of each possible location and chooses the location offering the highest value. The value of each location depends on three main factors:

  1. The value that the individual i places on the non-pecuniary factors associated with living in location j (climate, environment, local amenities, spousal employment opportunities, etc.), which is assumed to be time-invariant (denoted by the symbol ).

  2. The expected present value of money wages if the individual chooses location j in period t. This expected present value is the sum of:

  • the wage available in location j in period t, (); and

  • the discounted value of expected future utility if the individual chooses location j in period t (ρΕ(Vt+1)(where ρ is a one-period discount factor. Expected future utility depends on the value of all future wages in all possible locations.9

  1. Finally, a completely random location shock that is unrelated to the individual’s preference for location j in any given period t (denoted by the symbol ). This random shock accounts for unobservable factors that might induce an individual to choose a location she might dislike in period t, or leave a location she likes in period t.

Mathematically, the utility of location j at time t can be written as

                (1)

In this model, an individual will choose location j if its utility () exceeds the utilities associated with all other possible locations. Clearly, an individual who has strong non-pecuniary preferences for a particular location is more likely to choose it over other locations. That is to say, the probability of choosing to locate in location j initially, or remaining in location j if the individual is already there, increases with . But dislike for a particular location can be overcome if wages in that location are high enough. That is to say, even if an individual does not like a particular location as given by a negative value of , she may still choose to locate there if the pecuniary advantage of locating in the area, as measured by the value of the current wage plus the expected present value of future wages, is high enough. Given the values of the pecuniary and non-pecuniary factors associated with different locations, an individual’s propensity to move from one location to another is governed by the size of the location-specific random shocks. If wages were stable and random shocks did not exist, an individual would select his or her best (i.e., utility-maximizing) location in the first period and remain there forever.

Consider now aggregate (population average) probabilities of choosing a particular location and the aggregate probabilities of remaining in that location. These average probabilities are simply weighted averages of individual probabilities of selecting a location or remaining in it. The weights on which the aggregate averages are based are the fractions of the population with different values of . For example, if there were 5 different values of in the population and each value occurred with equal frequency, each value would receive a 1/5 weight in the calculation of aggregate probabilities. In general, the aggregate probabilities depend on the frequency distribution (probability density) of preferences (the ) in the population as well as the frequency distribution of the random shocks (the ). The parameters of these distributions (means and standard deviations) affect the aggregate probabilities and their sensitivities to changes in wages. We may show that, all else constant:

  • a smaller standard deviation of the random shock (denoted σε reduces the probability of an individual move from location j and increases the expected number of periods an individual stays in the initial location j;

  • the smaller is σε, the smaller is the frequency of moves in a cohort of individuals;

  • a smaller average preference for location  j (denoted ) results in a smaller fraction of individuals choosing a location or remaining in it;

  • higher current or future pay in location j increases the fraction of the population choosing to locate there and remain in it;

  • a larger standard deviation of in the population (denoted σθ) decreases the impact of pay changes.

Stated alternatively, the last proposition says that the more heterogeneous people are in their preferences for different locations, the less influence wage changes will have on their location choices. Conversely, if all individuals placed the same non-pecuniary value on each location, there exists a single set of wages across locations that would make individuals indifferent among locations. In other words, supposing location-specific random shocks are zero, wages would be the most important determinant of location choices. If wages were insufficiently high in locations with low non-pecuniaries, no one would choose those locations. Heterogeneous preferences ensure that most, if not all, locations will attract or retain some people, even when the average value preferences for those locations (i.e., their μj) are low or when wages are low.


9 Refer to Appendix B for a discussion of how Ε(Vt+1) is constructed.

Location Decisions in the Presence of the NHSC LRP

Consider now NHSC’s loan repayment program (LRP). A unique feature of this program is that an individual who applies for the LRP must have an NHSC-approved job in a HPSA and also have outstanding student debt in order to qualify for the program. Applicants are screened by the NHSC and not all applicants are accepted into the program. Importantly, in deciding whether to approve an applicant for the program, NHSC makes a determination regarding the applicant’s fit for the program and for the position the individual accepted. NHSC strives to choose the ‘most qualified’ applicants, but during the acceptance process it gives weight to an applicant’s fit for a particular position. An implication is that NHSC may select an applicant judged to be a ‘good fit’ over other applicants with better academic records. In addition to individual qualifications, the main driver of acceptance into the program is the severity of the shortage of health care providers in a particular area as measured by the HPSA score. Individuals applying for approval of a position in an area with a high HPSA score may have a better chance of approval than individuals applying in an area with a low HPSA score. Prior to 2009, approved individuals received funding only if the HPSA had a score of 14 or above. After the expansion in 2009, all approved individuals were funded regardless of HPSA score. Because of program funding constraints, not all applicants receive approval, even when applying for positions in high HPSA score areas.10

If an individual is accepted into the program and qualifies for the loan repayment amount Lj, the utility of location j is given by

       (2)

The individual prefers to participate in the program if there exists at least one HPSA location j for which the utility associated with that location is higher than the utilities associated with any other location. If the location that maximizes utility is not a HPSA, the individual of course does not apply. The attractiveness of a given HPSA depends on the loan amount Lj, so that the probability of choosing location j increases with the amount Lj. Some providers who do not participate in NHSC may still locate in a HPSA. This group will include: (1) individuals without student debt (and therefore ineligible to apply for NHSC program); (2) individuals who have student debt but did not apply (perhaps due to a low expected probability of approval, or lack of knowledge of the program); and (3) individuals who did apply and were not accepted.


10 In fiscal years 2012 and 2013 the admission rate into NHSC programs was around one third of applicants in those years.

LRP Participation

LRP participation is a joint outcome of application and acceptance. But this process is unobservable, and we only observe the outcomes of participation and non-participation. We denote the unobservable factors related to admission in the population of applicants with γi, where i represents the individual program participant. The term γi can be viewed as the individual’s ‘fit’ for the program. Fit for the program depends in part on observable factors such as academic background. Fit may also be related to the strength of an individual’s preference for a particular NHSC location (i.e., θj). We assume that NHSC ranks applicants in order of fit for the program and then fills all available spaces in the program. Rejection occurs when there are more applicants than spaces.

Exactly who is selected into the program depends on the weight NHSC places on factors other than preferences (e.g., academic background) during the selection process and the weight it gives to preferences. If fit for the program were based only on preferences, preferences would receive all of the weight and the rank-order of fit for the program would be identical to the rank-order of individuals’ location preferences. In this case, preferences and fit would be perfectly correlated and the program would select the same individuals who would have the strongest preferences for service in high-need areas in the absence of the program. At the other extreme, if preferences received no weight in the selection process and other factors such as academic ability received all of the weight, preferences and fit would have no correlation. In this case, the program would tend to attract the highest number of individuals who would be unlikely to serve in an underserved area in the absence of the program.

Retention

For simplicity, assume there are only two location types: HPSAs and non-HPSAs, and each individual has a certain preference for locating in HPSAs, θi. The correlation between the unobservable factors that affect admission to the LRP program and choice of geographic location (i.e., the correlation between γi and θi) has implications for observed retention patterns of NHSC participants and non-participants.

Under normal circumstances, there will be a number of participants with negative values for θi who will locate in HPSAs only because of the program. As a result of these providers going to HPSAs, the average preference of participants is lower than the average preference of non-participants and therefore the retention of participants in HPSAs will always be lower than the retention of non-participants. The difference between the retention rates is larger when the correlation between γi and θi is lower. As discussed above, the limiting case is when the correlation is zero, meaning that preferences for HPSAs play no role in the acceptance process. In this case, the number of NHSC participants with negative preferences for HPSAs will be the highest and therefore, the average retention rate in the population of participants will be at its lowest point relative to any other scenario when the correlation between γi and θi is strictly positive.11

In the unlikely case when there are more individuals with positive values for θi than the number of available program positions and the correlation between γi and θi is positive, all program participants will be individuals who have a preference for locating in HPSAs. Such individuals will have a higher average preference for HPSAs than program non-participants who were attracted to HPSAs anyway. Because they have a higher average preference for HPSAs than non-participants, program participants will tend to have higher retention in HPSAs after completion of their service obligations than non-participants. However, given that not all candidates are accepted into NHSC, this case cannot be encountered in reality.

Finally, the magnitude of the program effect on HPSA provider supply also depends on how sensitive location choices are to the presence of the program and the LRP amount. As in the general case, the impact depends on the mean preferences for various locations (the μj), the standard deviation of preferences (σθ), and the standard deviation of the random shocks (σε). Choices are more sensitive the more homogeneous location preferences are (i.e., the smaller is σθ) and the smaller the role of random shocks to location decisions (i.e., the smaller isσε).


11 A case of negative correlation between and would mean that the NHSC purposefully selects into the program providers who do not want to there.

Summary

The theoretical model gives a sharp prediction – retention of NHSC participants in HPSAs after the completion of their obligations can never be higher than the retention of providers who choose to locate in those areas without participating in the program. In the limiting case where selection into the program is based solely on preferences, the program selects individuals who would have tended to serve in high-need areas in the absence of the program and retention differences between participants and non-participants are nil. As the correlation between location preferences and program fit weakens, the program tends to select individuals who would not have gone to high-need areas in the absence of the program. In this case, participant retention after program completion will tend to be lower than non-participant retention. Somewhat counterintuitively, a lower retention rate for non-participants is a signal of the program’s success in attracting to high-need areas providers who would not have located there in the absence of the program.

Determinants of Provider Retention in HPSAs

One of our main goals in this study was to estimate the impact of the NHSC programs on HPSA retention.  In this chapter we present the main findings from our empirical analysis of providers’ retention in HPSAs.

Basic Econometric Model

For this purpose, we used the data on NHSC participants and the data on non-participants from the first analytic dataset to estimate regression models in which we control for observable characteristics on each individual (Χᵢ), local area characteristics (Ζᵢ) and an indicator for program participation (Prog i)

                     (1)

We estimated model (1) by using the ‘same HPSA’ and the ‘any HPSA’ indicator variables as the dependent variable, respectively.  The coefficient of interest, α, indicates the impact of NHSC program participation on the number of years served in a HPSA. The X vector included individual-level characteristics like age, gender and provider type, while the Z vector included Census division indicator variables and local area characteristics (at the zip code level), such as: the family income, poverty rate, percent White, percent Black, fraction of the population over 25 years of age with a high school degree and percent of the population over the age of 65.  These variables helped control for factors that retain or induce providers to leave from their initial place of service.

Main Results

We estimated model (1) using a logit regression specification, separately for primary care and mental health HPSAs, and by using the ‘same HPSA’ and the ‘any HPSA’ indicators as dependent variables.  The coefficient estimates are shown in Tables V.1-V.4.  In each of these tables we present 7 models, each estimated by the number of years elapsed since separation from service (for participants) and by the number of years since start year (for non-participants).

In all models of Table V.1 we estimated a lower probability of remaining in the same primary care HPSA for participants relative to non-participants.  Female providers were slightly less likely to remain in HPSAs, and older providers were more likely to remain in the same HPSA over time.  We detected virtually no differences in retention by provider type and, with the exception of the first two years since separation/start year, found no differences in retention by Census divisions.  Providers leaving service or appearing for the first time in the data in the earlier years (2005 or 2006) had a higher probability of remaining in the same HPSA relative to those whose separation/start year was beyond 2008.

It is important to note that the coefficient estimates on the local area characteristics are in many cases statistically significant, indicating that providers are more likely to remain in HPSAs where the poverty rate is higher, the fraction of older population is higher and the ratio of individuals over the age of 25 with a high school degree is lower.  Estimating the models separately on the population of participant providers and then on the population of non-participants, we found that the size of these effects are somewhat larger for non-participants than for participants.

Variable

1 Year

2 Years

3 Years

4 Years

5 Years

6 Years

> 6 Years

NHSC Participant

-2.253***

-1.531***

-1.326***

-1.212***

-1.080***

-1.083***

-0.868***

(0.081)

(0.071)

(0.077)

(0.081)

(0.091)

(0.108)

(0.093)

Female

0.010

-0.044***

-0.039**

-0.076***

-0.066***

-0.081***

-0.048**

(0.021)

(0.017)

(0.016)

(0.018)

(0.018)

(0.018)

(0.019)

Age 36 to 45

0.265***

0.259***

0.290***

0.323***

0.291***

0.249***

0.082

(0.023)

(0.023)

(0.022)

(0.024)

(0.033)

(0.044)

(0.064)

Age 46 to 55

0.581***

0.608***

0.680***

0.762***

0.749***

0.727***

0.447***

(0.033)

(0.030)

(0.030)

(0.030)

(0.037)

(0.048)

(0.064)

Age 56 to 65

0.771***

0.782***

0.898***

0.977***

0.977***

0.981***

0.705***

(0.044)

(0.038)

(0.038)

(0.036)

(0.043)

(0.051)

(0.064)

Age Over 65

1.051***

1.045***

1.189***

1.254***

1.265***

1.279***

0.991***

(0.069)

(0.059)

(0.056)

(0.051)

(0.056)

(0.055)

(0.070)

Medical Doctor

-0.199

-0.202

-0.241*

-0.241

-0.116

-0.331**

0.061

(0.176)

(0.134)

(0.139)

(0.147)

(0.128)

(0.150)

(0.177)

Nurse Practitioner

-0.032

-0.017

-0.043

-0.036

0.027

-0.175

0.253

(0.182)

(0.135)

(0.142)

(0.152)

(0.141)

(0.154)

(0.179)

Physician Assistant

-0.202

-0.180

-0.222

-0.221

-0.075

-0.355**

0.105

(0.188)

(0.139)

(0.147)

(0.156)

(0.151)

(0.159)

(0.196)

Start Year 2005

0.360***

1.038***

0.920***

0.852***

0.811***

0.530***

0.177***

(0.039)

(0.032)

(0.040)

(0.040)

(0.036)

(0.028)

(0.025)

Start Year 2006

0.157***

0.558***

0.520***

0.473***

0.404***

  

(0.044)

(0.040)

(0.044)

(0.044)

(0.039)

  

Start Year 2007

-0.091**

0.472***

0.512***

0.464***

   

(0.042)

(0.035)

(0.046)

(0.040)

   

Start Year 2008

-0.034

0.392***

0.369***

    

(0.041)

(0.040)

(0.049)

    

Middle Atlantic

-0.249

-0.217

0.074

0.068

0.049

0.020

-0.142

(0.225)

(0.225)

(0.244)

(0.258)

(0.258)

(0.253)

(0.234)

East North Central

-0.586***

-0.514***

-0.137

-0.147

-0.095

-0.139

-0.279

(0.145)

(0.141)

(0.190)

(0.193)

(0.191)

(0.190)

(0.184)

West North Central

-0.321*

-0.264

0.105

0.091

-0.004

-0.022

-0.210

(0.166)

(0.171)

(0.212)

(0.218)

(0.224)

(0.223)

(0.216)

South Atlantic

-0.406***

-0.386***

-0.030

-0.026

-0.008

-0.042

-0.057

(0.146)

(0.146)

(0.189)

(0.194)

(0.190)

(0.187)

(0.174)

East South Central

-0.770***

-0.674***

-0.273

-0.249

-0.180

-0.209

-0.252

(0.171)

(0.161)

(0.207)

(0.212)

(0.211)

(0.206)

(0.190)

West South Central

-0.818***

-0.688***

-0.266

-0.266

-0.223

-0.253

-0.216

(0.229)

(0.235)

(0.284)

(0.287)

(0.292)

(0.288)

(0.283)

Mountain

-0.131

-0.129

0.214

0.173

0.174

0.114

0.031

(0.141)

(0.138)

(0.187)

(0.193)

(0.193)

(0.187)

(0.181)

Pacific

-0.269*

-0.247

0.051

0.056

0.050

0.088

-0.032

(0.156)

(0.163)

(0.207)

(0.213)

(0.210)

(0.202)

(0.197)

Log Family Income

-0.095

-0.151

-0.192

-0.235

-0.265

-0.304

-0.648***

(0.220)

(0.224)

(0.227)

(0.235)

(0.231)

(0.225)

(0.219)

Poverty Rate

0.035***

0.033***

0.032***

0.031***

0.030***

0.030***

0.026***

(0.006)

(0.006)

(0.006)

(0.007)

(0.006)

(0.006)

(0.006)

Percent White

-0.299

-0.376

-0.621*

-0.679*

-0.633*

-0.450

0.054

(0.377)

(0.354)

(0.371)

(0.371)

(0.367)

(0.367)

(0.361)

Percent Black

0.559

0.500

0.489

0.435

0.460

0.555

0.887**

(0.385)

(0.357)

(0.374)

(0.368)

(0.370)

(0.368)

(0.369)

Pct HS Grads Over 25 Yrs

-3.886***

-4.140***

-3.899***

-3.401***

-3.850***

-3.744***

-4.858***

(0.935)

(0.958)

(0.994)

(0.982)

(1.007)

(0.981)

(1.037)

Pct Population Over 65

3.218**

3.047**

4.128***

3.914***

3.980***

3.676**

3.353**

(1.280)

(1.279)

(1.438)

(1.513)

(1.540)

(1.432)

(1.387)

Intercept

4.201*

3.706

3.355

3.340

3.675

4.333*

8.132***

(2.375)

(2.407)

(2.419)

(2.554)

(2.499)

(2.396)

(2.390)

Observations

195,189

191,713

178,925

165,474

149,059

141,902

147,725

 

NOTE: Robust standard errors, clustered by HPSA, in parentheses.  *, ** and ***: significant at 10%, 5% and 1%.  The base group is defined as providers who are: male, age 26-35, non-physician and non-NP/PA, having a start year of 2009 or later, and serving in a location in the Northeast.  The coefficient on any of the dummy variables included in these models shows the estimated difference relative to the corresponding excluded category.

Estimating model (1) with the ‘any HPSA’ indicator as the dependent variable, we found very similar patterns in the retention of providers in primary care HPSAs (Table V.2).   The main difference is that the coefficients on family income variable become statistically significant and negative, indicating that providers are less likely to stay in HPSAs where the average family income is increasing.  Along with the estimates on the other local area characteristics (that are directionally similar to the estimates from Table V.1), these findings are in line with our hypothesis that providers serving in HPSAs have a preference to serve underserved populations.

Table V. 2: Logit Models of Retention in Any HPSA by Years since Separation—Primary Care

Variable

1 Year

2 Years

3 Years

4 Years

5 Years

6 Years

> 6 Years

NHSC Participant

-1.432***

-0.829***

-0.667***

-0.564***

-0.444***

-0.418***

-0.301***

(0.096)

(0.079)

(0.083)

(0.089)

(0.092)

(0.117)

(0.102)

Female

-0.032

-0.077***

-0.075***

-0.107***

-0.101***

-0.113***

-0.074***

(0.029)

(0.024)

(0.022)

(0.022)

(0.022)

(0.023)

(0.023)

Age 36 to 45

0.233***

0.199***

0.228***

0.234***

0.195***

0.193***

-0.045

(0.032)

(0.028)

(0.025)

(0.029)

(0.035)

(0.050)

(0.076)

Age 46 to 55

0.577***

0.569***

0.552***

0.609***

0.564***

0.563***

0.206***

(0.045)

(0.039)

(0.037)

(0.037)

(0.043)

(0.055)

(0.078)

Age 56 to 65

0.799***

0.786***

0.812***

0.820***

0.791***

0.792***

0.420***

(0.061)

(0.045)

(0.045)

(0.043)

(0.048)

(0.059)

(0.080)

Age Over 65

1.071***

0.961***

1.044***

1.072***

0.990***

1.011***

0.606***

(0.095)

(0.082)

(0.068)

(0.062)

(0.064)

(0.066)

(0.086)

Medical Doctor

0.098

0.040

-0.213

-0.095

0.016

-0.291

0.239

(0.231)

(0.158)

(0.177)

(0.184)

(0.162)

(0.198)

(0.218)

Nurse Practitioner

0.182

0.127

-0.116

0.003

0.051

-0.234

0.332

(0.243)

(0.164)

(0.178)

(0.201)

(0.180)

(0.206)

(0.219)

Physician Assistant

0.031

0.045

-0.156

0.027

0.144

-0.209

0.385

(0.255)

(0.175)

(0.192)

(0.203)

(0.184)

(0.211)

(0.240)

Start Year 2005

0.337***

0.955***

0.786***

0.666***

0.619***

0.400***

0.101***

(0.055)

(0.043)

(0.048)

(0.046)

(0.043)

(0.036)

(0.031)

Start Year 2006

0.203***

0.497***

0.471***

0.366***

0.309***

  

(0.064)

(0.048)

(0.054)

(0.048)

(0.045)

  

Start Year 2007

-0.055

0.473***

0.513***

0.422***

   

(0.053)

(0.044)

(0.059)

(0.048)

   

Start Year 2008

0.026

0.406***

0.392***

    

(0.056)

(0.049)

(0.061)

    

Middle Atlantic

-0.078

-0.119

0.181

0.152

0.155

0.159

-0.012

(0.378)

(0.375)

(0.385)

(0.395)

(0.395)

(0.390)

(0.327)

East North Central

-0.480**

-0.420*

-0.021

-0.031

0.022

-0.009

-0.166

(0.225)

(0.222)

(0.271)

(0.265)

(0.264)

(0.260)

(0.233)

West North Central

0.020

-0.005

0.392

0.363

0.214

0.242

0.187

(0.272)

(0.271)

(0.311)

(0.301)

(0.316)

(0.312)

(0.277)

South Atlantic

-0.035

-0.040

0.364

0.339

0.352

0.327

0.293

(0.250)

(0.244)

(0.280)

(0.275)

(0.275)

(0.265)

(0.228)

East South Central

-0.556*

-0.460

-0.056

-0.057

0.056

0.028

0.088

(0.309)

(0.289)

(0.335)

(0.330)

(0.330)

(0.320)

(0.276)

West South Central

-0.568

-0.460

0.013

0.022

0.091

0.079

0.073

(0.361)

(0.369)

(0.433)

(0.430)

(0.431)

(0.431)

(0.394)

Mountain

0.558*

0.543*

0.922***

0.909***

0.869**

0.810**

0.648**

(0.329)

(0.316)

(0.349)

(0.337)

(0.350)

(0.330)

(0.286)

Pacific

0.027

-0.011

0.323

0.351

0.344

0.403

0.228

(0.256)

(0.267)

(0.311)

(0.303)

(0.302)

(0.290)

(0.282)

Log Family Income

-0.524

-0.612*

-0.743**

-0.839**

-0.906**

-0.927***

-1.297***

(0.355)

(0.354)

(0.360)

(0.365)

(0.359)

(0.356)

(0.338)

Poverty Rate

0.075***

0.071***

0.064***

0.060***

0.056***

0.063***

0.057***

(0.014)

(0.013)

(0.014)

(0.014)

(0.013)

(0.014)

(0.013)

Percent White

-0.145

-0.056

-0.291

-0.367

-0.234

0.092

0.561

(0.677)

(0.584)

(0.601)

(0.593)

(0.564)

(0.560)

(0.519)

Percent Black

0.807

0.846

0.995

0.957

1.003

1.097

1.372**

(0.785)

(0.689)

(0.727)

(0.714)

(0.687)

(0.680)

(0.660)

Pct HS Grads Over 25 Yrs

-5.306***

-5.511***

-5.305***

-4.667***

-5.256***

-4.831***

-5.655***

(1.638)

(1.633)

(1.621)

(1.582)

(1.601)

(1.582)

(1.582)

Pct Population Over 65

9.565***

9.253***

10.301***

10.193***

10.024***

9.671***

9.738***

(3.042)

(2.872)

(3.043)

(2.976)

(2.932)

(2.729)

(2.586)

Intercept

8.221**

8.188**

9.194**

9.835**

10.623***

10.745***

14.696***

(4.075)

(3.975)

(3.994)

(4.074)

(4.004)

(3.939)

(3.808)

Observations

195,189

191,713

178,925

165,474

149,059

141,902

147,725

 

NOTE: Robust standard errors, clustered by HPSA, in parentheses. *, ** and ***: significant at 10%, 5% and 1%. The base group is defined as providers who are: male, age 26-35, non-physician and non-NP/PA, having a start year of 2009 or later, and serving in a location in the Northeast.  The coefficient on any of the dummy variables included in these models shows the estimated difference relative to the corresponding excluded category.

In the case of mental health HPSAs, male and female providers did not differ in their retention probability in the same HPSA (Table V.3).  Similar to primary care providers, the older the providers the more likely they were to remain in the same mental health HPSA.   Also, providers with a separation/start year at the beginning of our timeframe were in general more likely to remain in the same HPSA.  In terms of differences in retention relative to primary care HPSAs, we estimated a lower probability to remain in the same mental health HPSA for medical doctors and nurse practitioners relative to other mental health workers, with the effect being stronger in the further out separation/start years.  We also estimated more variation in same HPSA retention by Census divisions.  Providers in the East South Central and South Atlantic divisions were least likely to remain in the same mental health HPSA.  Finally, although the estimates on poverty rate and percent of people over 25 who are high school graduates are directionally the same as in the case of primary care models, we found that a higher proportion of Blacks at the local level increases the retention probability of mental health providers.

Table V. 3: Logit Models of Retention in the Same County-Same HPSA by Years since Separation—Mental Health

Variable

1 Year

2 Years

3 Years

4 Years

5 Years

6 Years

> 6 Years

NHSC Participant

-1.215***

-0.801***

-0.765***

-0.968***

-0.727***

-0.849***

-0.377*

(0.174)

(0.166)

(0.173)

(0.195)

(0.205)

(0.238)

(0.213)

Female

0.018

0.027

0.011

-0.004

-0.018

0.019

0.030

(0.042)

(0.043)

(0.043)

(0.044)

(0.045)

(0.046)

(0.042)

Age 36 to 45

0.128*

0.231***

0.201**

0.243**

0.317*

0.516**

-0.773*

(0.075)

(0.075)

(0.100)

(0.117)

(0.186)

(0.263)

(0.450)

Age 46 to 55

0.066

0.262***

0.354***

0.462***

0.539***

0.786***

-0.464

(0.077)

(0.082)

(0.104)

(0.122)

(0.193)

(0.260)

(0.442)

Age 56 to 65

0.173**

0.389***

0.437***

0.515***

0.638***

0.875***

-0.340

(0.079)

(0.092)

(0.109)

(0.124)

(0.190)

(0.266)

(0.440)

Age Over 65

0.153

0.463***

0.533***

0.635***

0.715***

0.971***

-0.165

(0.105)

(0.109)

(0.121)

(0.131)

(0.194)

(0.271)

(0.440)

Medical Doctor

-0.087

-0.239***

-0.344***

-0.420***

-0.407***

-0.430***

-0.409***

(0.073)

(0.068)

(0.065)

(0.064)

(0.061)

(0.062)

(0.061)

Nurse Practitioner

-0.080

-0.151

-0.128

-0.261**

-0.374***

-0.389***

-0.371***

(0.114)

(0.108)

(0.105)

(0.120)

(0.107)

(0.121)

(0.115)

Start Year 2005

0.185***

0.564***

0.739***

0.679***

0.581***

0.435***

0.067

(0.067)

(0.091)

(0.098)

(0.082)

(0.089)

(0.058)

(0.075)

Start Year 2006

0.100

0.314***

0.516***

0.407***

0.337**

  

(0.101)

(0.121)

(0.130)

(0.113)

(0.131)

  

Start Year 2007

0.153*

0.395***

0.581***

0.483***

   

(0.080)

(0.101)

(0.107)

(0.094)

   

Start Year 2008

0.116

0.373***

0.371***

    

(0.083)

(0.086)

(0.088)

    

Middle Atlantic

0.388

0.260

0.437

0.411

0.350

0.297

0.170

(0.447)

(0.352)

(0.299)

(0.304)

(0.290)

(0.274)

(0.267)

East North Central

-0.619*

-0.593**

-0.255

-0.281

-0.299

-0.436**

-0.407**

(0.349)

(0.262)

(0.213)

(0.207)

(0.197)

(0.186)

(0.202)

West North Central

-0.332

-0.331

-0.111

-0.114

-0.149

-0.268

-0.336

(0.388)

(0.307)

(0.257)

(0.259)

(0.234)

(0.222)

(0.220)

South Atlantic

-0.706*

-0.655**

-0.483**

-0.397*

-0.464**

-0.483**

-0.547**

(0.385)

(0.298)

(0.246)

(0.235)

(0.226)

(0.214)

(0.219)

East South Central

-1.561***

-1.434***

-1.180***

-1.107***

-1.093***

-1.306***

-1.175***

(0.495)

(0.432)

(0.398)

(0.402)

(0.393)

(0.354)

(0.354)

West South Central

-0.897**

-0.900**

-0.455

-0.457

-0.485

-0.643**

-0.758**

(0.439)

(0.362)

(0.321)

(0.312)

(0.304)

(0.304)

(0.331)

Mountain

-0.332

-0.223

0.058

-0.007

0.103

-0.006

0.008

(0.399)

(0.319)

(0.276)

(0.266)

(0.259)

(0.245)

(0.242)

Pacific

-0.283

-0.230

0.008

-0.015

-0.020

-0.062

-0.100

(0.454)

(0.356)

(0.301)

(0.288)

(0.273)

(0.251)

(0.242)

Log Family Income

0.005

0.075

0.158

0.180

0.115

0.066

-0.164

(0.330)

(0.290)

(0.278)

(0.272)

(0.281)

(0.246)

(0.229)

Poverty Rate

0.053***

0.054***

0.055***

0.056***

0.052***

0.052***

0.053***

(0.016)

(0.013)

(0.013)

(0.012)

(0.011)

(0.011)

(0.010)

Percent White

-0.218

-0.211

-0.197

-0.173

-0.281

-0.221

-0.228

(0.728)

(0.595)

(0.622)

(0.553)

(0.568)

(0.562)

(0.528)

Percent Black

2.068**

1.752**

1.737**

1.770***

1.755***

1.548**

1.441**

(0.915)

(0.734)

(0.711)

(0.652)

(0.675)

(0.648)

(0.629)

Pct HS Grads Over 25 Yrs

-4.369**

-4.435**

-3.875**

-4.037**

-3.707**

-3.489**

-3.971***

(2.106)

(1.815)

(1.697)

(1.712)

(1.653)

(1.694)

(1.518)

Pct Population Over 65

2.645

2.090

2.868

3.673

2.831

1.848

2.100

(3.139)

(2.605)

(2.572)

(2.402)

(2.337)

(2.341)

(2.009)

Intercept

1.749

0.391

-1.495

-1.898

-1.196

-0.755

3.288

(4.054)

(3.608)

(3.454)

(3.357)

(3.435)

(3.054)

(2.916)

Observations

19,046

18,609

16,841

15,111

13,123

12,758

13,547

 

NOTE: Robust standard errors, clustered by HPSA, in parentheses. *, ** and ***: significant at 10%, 5% and 1%.  The base group is defined as providers who are: male, age 26-35, non-physician and non-NP/PA, having a start year of 2009 or later, and serving in a location in the Northeast.  The coefficient on any of the dummy variables included in these models shows the estimated difference relative to the corresponding excluded category.

As shown in Table V.4, the difference in retention between participants and non-participants disappeared in the case of mental health ‘same HPSA’.  Other important features in Table V.4 are that the differences in retention by age group, provider type, separation/start year and Census division were less pronounced or virtually non-existent, while the differences by local area characteristics are similar to those from Table V.3.

Table V. 4: Logit Models of Retention in Any HPSA by Years since Separation—Mental Health

Variable

1 Year

2 Years

3 Years

4 Years

5 Years

6 Years

> 6 Years

NHSC Participant

0.007

0.162

0.143

-0.007

0.267

0.051

0.280

(0.198)

(0.162)

(0.176)

(0.191)

(0.192)

(0.208)

(0.196)

Female

0.028

0.045

0.014

0.011

-0.009

0.002

-0.004

(0.044)

(0.046)

(0.044)

(0.044)

(0.049)

(0.043)

(0.045)

Age 36 to 45

0.123

0.127

0.120

0.047

0.416**

0.322

-0.493

(0.080)

(0.083)

(0.101)

(0.119)

(0.174)

(0.243)

(0.523)

Age 46 to 55

0.023

0.113

0.165

0.127

0.512***

0.471*

-0.354

(0.081)

(0.089)

(0.106)

(0.125)

(0.176)

(0.245)

(0.515)

Age 56 to 65

0.118

0.203**

0.196*

0.109

0.575***

0.501**

-0.355

(0.083)

(0.097)

(0.106)

(0.122)

(0.175)

(0.239)

(0.518)

Age Over 65

0.143

0.276**

0.321***

0.285**

0.641***

0.595**

-0.205

(0.113)

(0.116)

(0.117)

(0.131)

(0.184)

(0.240)

(0.517)

Medical Doctor

0.104

0.047

-0.048

-0.094

-0.070

-0.081

-0.118*

(0.080)

(0.075)

(0.072)

(0.066)

(0.068)

(0.068)

(0.071)

Nurse Practitioner

0.015

0.092

0.151

0.043

0.032

0.061

-0.121

(0.121)

(0.115)

(0.106)

(0.127)

(0.114)

(0.126)

(0.114)

Start Year 2005

0.055

0.227***

0.395***

0.329***

0.175**

0.219***

0.003

(0.071)

(0.084)

(0.099)

(0.083)

(0.079)

(0.061)

(0.085)

Start Year 2006

0.037

0.070

0.259*

0.122

-0.005

  

(0.100)

(0.122)

(0.135)

(0.123)

(0.131)

  

Start Year 2007

0.120

0.210**

0.413***

0.344***

   

(0.079)

(0.096)

(0.111)

(0.102)

   

Start Year 2008

0.093

0.183**

0.276***

    

(0.088)

(0.088)

(0.094)

    

Middle Atlantic

0.477

0.359

0.515

0.490

0.451

0.420

0.293

(0.496)

(0.403)

(0.360)

(0.366)

(0.355)

(0.332)

(0.316)

East North Central

-0.605

-0.580*

-0.289

-0.303

-0.265

-0.324

-0.320

(0.377)

(0.297)

(0.247)

(0.234)

(0.233)

(0.218)

(0.213)

West North Central

-0.325

-0.316

-0.104

-0.024

0.066

-0.057

-0.046

(0.422)

(0.344)

(0.295)

(0.294)

(0.277)

(0.260)

(0.241)

South Atlantic

-0.655

-0.584*

-0.446

-0.343

-0.247

-0.286

-0.199

(0.420)

(0.341)

(0.289)

(0.274)

(0.272)

(0.250)

(0.241)

East South Central

-1.623***

-1.493***

-1.182***

-1.148***

-1.132***

-1.200***

-1.150***

(0.527)

(0.471)

(0.436)

(0.440)

(0.434)

(0.394)

(0.384)

West South Central

-0.981**

-0.983**

-0.553

-0.574

-0.599*

-0.661*

-0.653*

(0.483)

(0.420)

(0.373)

(0.368)

(0.364)

(0.365)

(0.361)

Mountain

-0.342

-0.155

0.128

0.142

0.275

0.253

0.339

(0.437)

(0.376)

(0.341)

(0.341)

(0.340)

(0.322)

(0.302)

Pacific

-0.294

-0.200

0.013

0.054

0.067

0.056

0.027

(0.492)

(0.408)

(0.351)

(0.338)

(0.327)

(0.305)

(0.288)

Log Family Income

-0.180

-0.165

-0.182

-0.275

-0.296

-0.436

-0.573**

(0.351)

(0.319)

(0.314)

(0.319)

(0.316)

(0.298)

(0.262)

Poverty Rate

0.055***

0.060***

0.061***

0.057***

0.056***

0.057***

0.063***

(0.019)

(0.017)

(0.016)

(0.016)

(0.015)

(0.015)

(0.014)

Percent White

-0.077

-0.086

0.116

0.221

0.213

0.328

0.165

(0.809)

(0.745)

(0.731)

(0.657)

(0.672)

(0.653)

(0.649)

Percent Black

2.602**

2.535**

2.595***

2.966***

2.847***

2.678***

2.346***

(1.119)

(1.009)

(0.931)

(0.890)

(0.874)

(0.843)

(0.837)

Pct HS Grads Over 25 Yrs

-4.535*

-4.455**

-4.196**

-4.594**

-4.109**

-3.905**

-3.717**

(2.344)

(2.110)

(1.958)

(1.972)

(1.913)

(1.943)

(1.760)

Pct Population Over 65

3.866

4.571

5.366*

6.069**

6.130**

4.602

6.726**

(3.511)

(3.165)

(3.102)

(2.983)

(3.027)

(2.839)

(2.666)

Intercept

3.730

3.020

2.357

3.490

3.149

4.765

6.871**

(4.359)

(4.060)

(3.942)

(3.976)

(3.925)

(3.703)

(3.420)

Observations

19,046

18,609

16,841

15,111

13,123

12,758

13,547

 

NOTE: Robust standard errors, clustered by HPSA, in parentheses. *, ** and ***: significant at 10%, 5% and 1%.  The base group is defined as providers who are: male, age 26-35, non-physician and non-NP/PA, having a start year of 2009 or later, and serving in a location in the Northeast.  The coefficient on any of the dummy variables included in these models shows the estimated difference relative to the corresponding excluded category.



In the first year since separation/start year, NHSC participants are 37.0% less likely to remain in the same HPSA relative to non-participants.  This difference represents a regression-adjusted difference, obtained by netting out the impact of other (observable) individual socio-demographic and local area characteristics.  Given that the unadjusted difference in the retention rate in the same HPSA in the first separation/start year is 42.8 percentage points (=83.5-40.7, from Figure VI.1), it follows that 86.4% (=37/42.8) of the observed difference in primary care ‘same HPSA’ retention is explained by NHSC participation.  A similar fraction, 85.6% (=13.7/16.0), in the observed retention difference was explained by NHSC participation in the case of primary care ‘any HPSA’ in the first separation/start year.  The other ratios between adjusted and unadjusted retention differences in retention between participants and non-participants remained about the same for the other further out separation/start years, for both primary care ‘same HPSA’ and ‘any HPSA’ measures.

Figure V.2 presents the regression-adjusted retention differentials by NHSC participation for mental health HPSAs.  The retention differentials are lower across the board for the ‘same HPSA’ measure than in the case of primary care HPSAs.  For the ‘any HPSA’ measure the differences in retention between participants and non-participants were practically zero, as shown by the statistically insignificant coefficients in Table V.2.

Figure V. 2:  Differences in the Participants’ Retention Probability Relative to Non Participants—Mental Health

Differences in the Participants’ Retention Probability Relative to Non Participants—Mental Health

As shown in Figure V.1, the unadjusted retention in ‘any HPSA’ was higher for non-participants in the first separation/start years than that of participants.  Nonetheless, after accounting for individual-level and local area characteristics, there was no statistically significant difference between the retention of participants and non-participants in mental health HPSAs for in any of the separation/start years.

Retention by NHSC Program

We also estimated variants of model (1) in which we tested for whether participation in SP vs LRP has an impact on the retention of providers in HPSAs.  As shown in Table B.8 in Appendix B, we find that participants in SP are less likely to remain in HPSAs after service completion.  All the differences we report in Table B.8 are statistically significant at significance levels of less than 1%.  The models in Table B.8 are obtained from regression models that were estimated only the subsample of physicians serving in primary care HPSAs. .  We could not estimate models using data for mental health physicians or non-physician providers because the number of scholarship program participants in these groups was severely limited.

Multinomial Logit Models

A more complex model to estimate the providers’ probability to move over the observed period is the multinomial logit.  The advantage of this model is that it simultaneously considers the entire locations choice set available to providers in each year.  Specifically, after controlling for the same characteristics from above, this model provides estimates of the probability of each of four mutually exclusive outcomes: (1) the probability of choosing to remain in the same HPSA and same county; (2) the probability of moving to a non-HPSA location within the same county; (3) the probability to move to another HPSA in another county; and (4) the probability of moving to a non-HPSA in another county.

Inclusion of the NHSC participation variable in the multinomial logit model allows for the estimation of the difference in migration probabilities between participants and non-participants to any of the four HPSA/non-HPSA type locations.  In Figure V.3 we present the marginal effects associated with these migration probability differences for primary care HPSAs, while in Figure V.4 we present the marginal effects for mental health HPSAs.  The actual coefficients from the multinomial logit models are presented in Tables B.9 and B.10 in Appendix B.

Figure V. 3:  Differences in the Participants’ Migration Probabilities Relative to Non-Participants—Primary Care

Differences in the Participants’ Migration Probabilities Relative to Non-Participants—Primary Care

According to our estimates, participants serving in primary care HPSAs are 36.6% less likely than non-participants to remain in the same HPSA in the first separation/start year.  At the same time, they are 25.2 % more likely to move to a HPSA in another county in the first separation/start year, meaning that overall, they are 11.4 percentage points (=-36.6+25.2) less likely to remain in any primary care HPSA.  The ‘same HPSA’ and ‘any HPSA’ estimates from Figure V.3 are in line with our estimate from Figure V.1, but the advantage of the multinomial logit model is that it also shows that participants are 6.3% more likely than non-participants to move to non-HPSAs in the same county and 5.2% to move to non-HPSAs in another county.

As shown in Figure V.3, as the retention window after program completion increases, the differences in migration probabilities between participants and non-participants become lower and lower.

Figure V. 4:  Differences in the Participants’ Migration Probabilities Relative to Non-Participants—Mental Health

Differences in the Participants’ Migration Probabilities Relative to Non-Participants—Mental Health

In addition, the multinomial logit model provides an explanation for our previous finding that mental health participants show no difference in retention in any HPSAs relative to non-participants.  As shown in Figure V.4, in the first separation/start year participants are 30.6% less likely to remain in the same county same HPSA than non-participants, but 31.1% more likely to move to HPSAs in another county.  When these differences are added up they yield an estimate of virtually zero difference between participants and non-participants in terms of retention in any mental health HPSA.

Effect of NHSC Programs on Retention

It is tempting to interpret the adjusted retention differences by NHSC participation as causal effects of participation in the NHSC programs.  However, that would be the case only if (1) NHSC participation is random and (2) participants and non-participants have the same underlying characteristics.  As we explained in the previous chapter, this is unlikely to be the case, as participants are a self-selected sample of all providers.  Moreover, the sample of non-participants that we constructed may not be representative for the general population of medical providers, given that they may have a strong preference to be in an underserved area and/or serve underserved populations.  A more sophisticated empirical approach (which is beyond the scope of this study) would have to control for these currently unobservable characteristics in order to be able to attach a causal interpretation to the estimate on the NHSC indicator variable.

As discussed earlier, the crucial insight of the theoretical model from Chapter VI is that, in most circumstances, retention of NHSC participants in high-need areas will be less than the retention of non-participants.  The reason is that the NHSC programs will tend to attract individuals who have a lower average non-pecuniary preference for HPSA locations than individuals who choose the location without the inducement of a loan repayment.  Because they have a lower preference for the location, the NHSC participants tend to stay in the HPSA locations at a lower rate after program completion.  The countervailing force is that the NHSC may screen applicants on the basis of the strength of their preferences for HPSA locations.  The more the NHSC selects applicants on the basis of their location preferences, the smaller the retention differences between participants and non-participants will be.  At the limit, retention differences disappear when selection into the program is completely on the basis of preferences.

In light of our theoretical model, all estimates presented in this Chapter provide strong evidence that participation in the NHSC programs is not entirely based on selection by preferences.  The implication is that the program attracts, at least to some extent, individuals who would not have gone to HPSAs in the absence of the program.  It follows that, somewhat counterintuitively, the NHSC has a bigger effect on person-years of service in high-need areas than in a case when selection into the program is based only on preferences (and as a result there are no differences in retention between participants and non-participants).  Furthermore, another implication is that a larger difference between the retention in a HPSA of NHSC participants and non-participants is a sign of program success, not failure.

Model Simulations

Generic Two-Location Choice Model

Table VI.1 shows the key assumed values of variables or distribution parameters for the two sets of simulations.  We endow each of 20,000 hypothetical individuals with a preference for location 1 drawn from a normal distribution with a mean of -10,000 and a standard deviation of 7,500 and a preference for location 2 drawn from a normal distribution with a mean of 0 and a standard deviation of 7,500.  Location 1 is obviously less desirable on average than location 2.  But the relatively large standard deviation of each preference factor indicates that there is significant heterogeneity of preferences in the population.  Wages are set at 30,000 in both locations, and the LRP amount is set at 5,000.  Simulation set 1 assumes a relatively low standard deviation of random shocks to decisions in each period (1,000).  Simulation set 2 doubles this standard deviation.

Table VI. 1: Assumed Values of Variables or Distribution Parameters

Variable or Distribution Parameter

Simulation Set 1

Simulation Set 2

Location 1 Wage (W1)

30,000

30,000

Location 2 Wage (W2)

30,000

30,000

Loan Repayment Amount (L)

5,000

5,000

Mean Preference Location 1 (μ1)

-10,000

-10,000

Mean Preference Location 2 (μ2)

0

0

Standard Deviation of Preferences (σθ)

7,500

7,500

Standard Deviation of Random Shocks (σε)

1,000

2,000

The model is first run under the assumption of no NHSC program.  In simulation set 1, 17.7% of the individuals choose location 1 (the HPSA location).  But although the individuals in the cohort of 20,000 have an average net preference for location 1 of -10,000, the 17.7% that actually select location 1 in period 1 have an average net preference for the location of 5,300.  Because the average preference of those choosing location 1 to begin with is so high, 90.2% of these individuals chose to remain in the location in period 2; the other 9.8% received a location shock that induced them to move to location 2.  Finally, note that 2.2% of the individuals who chose location 2 in period 1 chose to move to location 1 in period 2.

Table VI. 2: Means of Key Outcomes without a NHSC Program

Migration Patterns

Simulation Set 1

Simulation Set 2

% Choosing Location 1 in Period 1

0.177

0.185

% Retained in Location 1 in Period 2

0.902

0.826

% Moving from Location 2 to 1 in Period 2

0.022

0.037

Average Δθ | Location 1 in Period 1

5,300

4,417

Table VI.2 indicates that a larger standard deviation of random shocks (Simulation Set 2) increases the fraction of the initial cohort that choose location 1 in period 1.  Intuitively, larger shocks induce more individuals to choose location 1 even though they have a negative preference for it.  But larger random shocks also induce more individuals to move away from location 1 in period 2; larger shocks also induce more movement in period 2 from location 2 to location 1.

Two-Location Choice Model with NHSC Programs

We now turn to simulations of choices under an NHSC LRP.  As discussed above, there are four distinct groups to consider:

  1. Group 1 – individuals with outstanding loans who apply for NHSC and are accepted;
  2. Group 2 -- individuals with outstanding loans who apply and are rejected;
  3. Group 3 -- individuals with outstanding loans who do not apply and
  4. Group 4 – individuals without outstanding loans and not eligible to apply.

To determine these groups, we randomly assign half of the 20,000 cohort to have outstanding debt and thus be eligible for application for LRP.  Group 4 thus contains 10,000 individuals.  Each of the 10,000 individuals with outstanding debt decides to apply or not apply depending upon whether the inequality in equation (9) holds.  The NHSC then evaluates the individual’s application.  According to equation (11), the applicant’s unobservable value to the program is based on his (standardized) net preference for location 1 and a random shock that is uncorrelated with preferences.  Once a random shock is generated and the applicant’s value to the program (A) is computed, the NHSC is assumed to rank-order applicants and select half of them for acceptance into the program.  Accepted applicants thus form Group 1 and rejected applicants form Group 2.  In all simulations, there were 1,644 accepted applicants and 1,643 rejected applicants.  The remaining are 6,713 individuals with outstanding debt form Group 3.

The following two tables (Tables VI.3 and Table VI.4) contain means of key outcome variables in the simulations for groups 1, 2 and 4.  Group 3 outcomes are ignored because everyone in this group goes to location 2 and stays there in period 2.  Since groups 2 and 4 together represent all non-participants who located in the HPSA, the table shows rates and means for these two groups combined.  Each table contains five scenarios that vary the correlation between preferences and other factors that influence selection into NHSC.  The correlations range from 0 to 0.98.  In the first scenario, selection into the program is independent of location preferences.  In the last scenario, applicants are selected into the program almost completely on the basis of the strength of their location preferences.

First inspect scenario 1 in Table VI.3.  Obviously, everyone in group 1 must go to location 1, the HPSA, in period 1.  After completing their NHSC service, however, only 53.4% decide to remain in the HPSA for period 2.  This retention rate is much lower than the retention rates of groups 2 and 4.  About 90% of both groups choose to remain in the HPSA for period 2.  These much higher retention rates are explained by the fact that the individuals in the latter two groups did not receive an inducement to locate in the HPSA (in the form of LRP); consequently the smaller percentages that did locate there had much higher average net preferences for the location than the NHSC participants.  Measured by their average preferences for the HPSA, the two groups of non-participants who locate in the HPSA appear relatively homogeneous.  As we shall see, these groups become more heterogeneous the more the NHSC selects applicants on the basis of their location preferences.

Table VI. 3: Means of Key Outcomes with NHSC Program in Five Scenarios (Simulation Set 1)

Migration Patterns

Group

Overall

1

2

4

2 & 4

Scenario 1: Corr(ua, ub)=0

% Choosing Location 1 in Period 1

1.000

0.548

0.174

0.227

0.214

% Retained in Location 1 in Period 2

0.534

0.897

0.907

0.906

0.762

% Moving from Location 2 to 1

 

0.140

0.022

0.038

0.018

Average Δθ | Location 1 in Period 1

1,636

5,308

5,197

5,213

3,854

Scenario 2: Corr(ua, ub)=0.447

% Choosing Location 1 in Period 1

1.000

0.474

0.174

0.216

0.208

% Retained in Location 1 in Period 2

0.609

0.872

0.907

0.902

0.783

% Moving from Location 2 to 1

0.000

0.126

0.022

0.036

0.018

Average Δθ | Location 1 in Period 1

2,878

4,129

5,197

5,046

4,081

Scenario 3: Corr(ua, ub)=0.707

% Choosing Location 1 in Period 1

1.000

0.408

0.174

0.207

0.203

% Retained in Location 1 in Period 2

0.679

0.842

0.907

0.898

0.804

% Moving from Location 2 to 1

0.000

0.112

0.022

0.034

0.018

Average Δθ | Location 1 in Period 1

3,859

3,066

5,197

4,896

4,302

Scenario 4: Corr(ua, ub)=0.894

% Choosing Location 1 in Period 1

1.000

0.324

0.174

0.195

0.196

% Retained in Location 1 in Period 2

0.766

0.795

0.907

0.892

0.833

% Moving from Location 2 to 1

0.000

0.095

0.022

0.032

0.018

Average Δθ | Location 1, Period 1

4,855

1,851

5,197

4,725

4,598

Scenario 5: Corr(ua, ub)=0.98

% Choosing Location 1 in Period 1

1.000

0.216

0.174

0.180

0.187

% Retained in Location 1 in Period 2

0.880

0.685

0.907

0.876

0.874

% Moving from Location 2 to 1

0.000

0.078

0.022

0.029

0.017

Average Δθ | Location 1, Period 1

5,857

103

5,197

4,478

5,003

Now let us examine collectively the other scenarios.  Notice, as the NHSC selects applicants more and more on the basis of their net preferences for service in the HPSA, the retention rate among NHSC participants in period 2 increases.  In fact, by scenario 5 the retention of the participants converges to the retention of the non-participants without student loans (group 4) and the average retention of all non-participants (groups 2 & 4 combined).  As we move from scenario 1 to scenario 5, however, the retention of group 2 declines significantly.  This decline is due to the fact that, as the NHSC selects applicants more on the basis of preferences, the pool of rejected applicants consists more of individuals with weaker net preferences for location 1.

An important take-away from these scenarios is that retention of NHSC participants in a HPSA is never likely to exceed the retention of non-participants and will likely be less than the retention of non-participants unless applicants are selected into the program almost exclusively on the basis of preferences for service in a HPSA.

Table VI.4 repeats the five scenarios assuming a larger standard deviation of shocks to location decisions.  The overall pattern of outcomes remains the same as in Table VI.1, but (as discussed above) larger shocks have the effect of inducing a higher rate of selection of location 1 in period 1 but higher rates of movement out of and into location 1 in period 2. The lower overall retention in location 1 is also explained by the fact that the average net preference for location 1 in period 1 among the people choosing that location in period 1 is smaller in simulation set 2 with the higher variance in the random shock.

Table VI. 4: Means of Key Outcomes with NHSC Program in Five Scenarios (Simulation Set 2)

Migration Patterns

Group

Overall

1

2

4

2 & 4

Scenario 1: Corr(ua, ub)=0

% Choosing Location 1 in Period 1

1.000

0.578

0.184

0.184

0.220

% Retained in Location 1 in Period 2

0.546

0.816

0.832

0.832

0.723

% Moving from Location 2 to 1

 

0.175

0.038

0.038

0.030

Average Δθ

1,012

1,068

-9,979

-9,979

-10,008

Average Δθ | Location 1, Period 1

1,012

4,274

4,476

4,476

3,152

Scenario 2: Corr(ua, ub)=0.447

% Choosing Location 1 in Period 1

1.000

0.507

0.184

0.229

0.410

% Retained in Location 1 in Period 2

0.626

0.776

0.832

0.824

0.743

% Moving from Location 2 to 1

0.000

0.147

0.038

0.053

0.030

Average Δθ

2,426

-347

-9,979

-8,620

-10,008

Average Δθ | Location 1, Period 1

2,426

3,039

4,476

4,273

3,421

Scenario 3: Corr(ua, ub)=0.707

% Choosing Location 1 in Period 1

1.000

0.450

0.184

0.221

0.210

% Retained in Location 1 in Period 2

0.688

0.730

0.832

0.817

0.758

% Moving from Location 2 to 1

0.000

0.136

0.038

0.052

0.030

Average Δθ

3,533

-1,454

-9,979

-8,776

-10,008

Average Δθ | Location 1, Period 1

3,533

1,752

4,476

4,092

3,635

Scenario 4: Corr(ua, ub)=0.894

% Choosing Location 1 in Period 1

1.000

0.370

0.184

0.210

0.203

% Retained in Location 1 in Period 2

0.766

0.667

0.832

0.808

0.781

% Moving from Location 2 to 1

0.000

0.124

0.038

0.050

0.030

Average Δθ

4,697

-2,619

-9,979

-8,940

-10,008

Average Δθ | Location 1, Period 1

4,697

185

4,476

3,871

3,928

Scenario 5: Corr(ua, ub)=0.98

% Choosing Location 1 in Period 1

1.000

0.297

0.184

0.200

0.197

% Retained in Location 1 in Period 2

0.835

0.593

0.832

0.798

0.804

% Moving from Location 2 to 1

0.000

0.113

0.038

0.049

0.030

Average Δθ

5,552

-3,475

-9,979

-9,061

-10,008

Average Δθ | Location 1, Period 1

5,552

-1,474

4,476

3,636

4,191

Implication of NHSC LRP on Provider Person-Years

Finally, we examine person-years of service across simulation sets and scenarios (Table VI.5).  Over both time periods and locations, there are 40,000 person-years of service.  Table 5 shows that in all scenarios person-years of service increase in period 1 compared to the case of no program.  That is to say, the program will increase person-years of service in high-need areas.  However, the magnitude of the increase diminishes as the strength of selection into the NHSC program on the basis of preferences increases.  That is to say, the more the program selects on the basis of preferences, the more it attracts individuals who would have chosen to locate in a HPSA in the absence of the program.  Less selection on preferences allows the program to attract individuals who would not have chosen to serve in a HPSA in the absence of the program.  As we explained in more detail in Chapter IV, paradoxically, the program is most successful in increasing person-years of service when it attracts individuals who are least likely to stay after completing their obligations.14

Table VI. 5: Person-Years of Service in Each Location

Scenario

Location 1

Location 2

Period 1

Period 2

Total

Period 1

Period 2

Total

Simulation Set 1

No Program

3,534

3,549

7,083

16,466

16,451

32,917

Scenario 1

4,284

3,549

7,833

15,716

16,451

32,167

Scenario 2

4,163

3,549

7,712

15,837

16,451

32,288

Scenario 3

4,054

3,549

7,603

15,946

16,451

32,397

Scenario 4

3,917

3,549

7,466

16,083

16,451

32,534

Scenario 5

3,739

3,549

7,288

16,261

16,451

32,712

Simulation Set 2

No Program

3,703

3,653

7,356

16,297

16,347

32,644

Scenario 1

4,406

3,653

8,059

15,594

16,347

31,941

Scenario 2

4,290

3,653

7,943

15,710

16,347

32,057

Scenario 3

4,197

3,653

7,850

15,803

16,347

32,150

Scenario 4

4,068

3,653

7,721

15,932

16,347

32,279

Scenario 5

3,949

3,653

7,602

16,051

16,347

32,398

Notice that within a given simulation set, person-years of service in period 2 are always the same.  This is because, in period 2, individuals always face the same incentives (wage differential) and have the same preferences and random shocks; they therefore make the same period 2-choice in all scenarios.


14 To be clear, the person-year gain results from the fact that, when selection is based less on preferences and more on other factors, many high-preference applicants who are rejected still choose to locate in the HPSA.

Conclusions

Combining data on NHSC program participants from administrative sources with Provider360 and Medicare data, we examined the retention trends in HPSA locations of participants in comparison to non-participants serving in HPSAs.  We found that about 49% of NHSC Primary Care (PC) participants were located in the same HPSA one year after obligation completion and 82% were located in any HPSA.  Also, by the 6th year after obligation, 35% of participants were located in same the HPSA and 72% of them in any HPSA.

In comparison, non-participant primary care retention in HPSAs is higher, but the difference was much bigger for retention in same HPSA than retention in any HPSA locations.  We found that 91 % of the non-participants working in primary care HPSAs remain in the same county same HPSA in the first year after they are first observed in a HPSA.  The retention rate declines constantly over the years, such that the retention rate reaches 69% in the 6th year since start year.  The rates are higher in the case of ‘any HPSA’, ranging from 95% in the first year since start year to 82% in the 6th year since start year.

These findings indicate that much of the geographic mobility of participants that occurs after program completion is from one HPSA to another.  Moreover, the convergence of retention rates over the longer run implies that after an initially higher mobility, NHSC participants have better retention in HPSAs than non-participants.  Also, participants are only slightly more likely than non-participants to relocate within a county to a non-HPSA zip code or to relocate to a non-HPSA county.  The findings are of similar magnitude for providers serving in mental health HPSAs.

Next, controlling for demographic characteristics, cohort, calendar year, and local area economic characteristics in a multivariate regression framework reduces the mean retention differences discussed above.  The regression-adjusted difference between participants and non-participants’ probability to remain in the same HPSA same county location is -37% in the first year since separation/start and it is -21% in the 6th year since separation/start year in the case of primary care providers.  The similarly calculated difference for the ‘any HPSA’ outcome shows a -11% difference in the first year and a -8% difference in the 6th year since separation/start year.

In addition, the regression analysis revealed that retention rises significantly with age, but differences by gender, provider type and Census division are small.  Also retention is significantly related to zip code-level economic characteristics such as the poverty rate and other local area characteristics (at the zip code level), like percent White, percent Black or percent individuals with a high school degree that are ages 25 or above.  These effects suggest that providers select into underserved areas based on the strength of their preferences for serving underserved populations.

Finally, our findings are consistent with an economic model predicting higher non-participant retention in HPSAs due to their self-selection into HPSAs without the financial inducement of NHSC programs.  As we show in Chapters VI and VIII, the model predicts that NHSC programs increase total provider-years in HPSAs when the HPSA retention among participants is lower than that of non-participants.

Appendix A: Previous Literature

In this appendix we provide an extensive literature survey focused on the retention metrics, policies and practices concerning NHSC programs. We also take into account literature providing estimates of the effects of other federal, state, or institutional programs on the number of NHSC providers.15 In essence, our review includes a discussion of the following components:

  • Designations of Health Professional Shortage Areas (HPSA) and other definitions for underserved areas

  • Definitions of retention and retention metrics used for NHSC enrollees

  • Retention metrics used elsewhere in the health sector and in the military

  • Previous estimates of the effect of NHSC programs on recruiting and retention

  • NHSC enrollment and retention under the Patient Protection and Affordable Care Act (ACA) and American Recovery and Reinvestment Act (ARRA) funding expansion

  • Variation in health workforce retention in different health networks

  • Strategies to increase provider retention in underserved areas and the impact of these strategies

Our literature review indicates that clinician retention in the NHSC programs has been a frequently studied topic over the past several decades. Although there appears to be substantial variation in the definitions of retention, with respect to provider’s location after completion of the service obligation or time frame over which retention is assessed, all currently available empirical estimates suggest that retention is heavily influenced by a number of common factors. These factors include: being motivated primarily by the nature of the work rather than by financial incentives; having prior experience in an underserved area; or having educational and employment opportunities for other family members. Findings from analogous programs run by states, institutions, and the Armed Forces reach similar conclusions.

Overall, NHSC has had success in recruiting clinicians into both programs over the years, particularly following the supplemental funding from the 2009 American Recovery and Reinvestment Act (ARRA), which allowed NHSC to place practitioners at more service sites as well as extend many service contracts. Retaining these clinicians after they complete their service requirements is essential to the program’s lasting impact on communities of need. The program’s impact is essentially doubled when clinicians supported with an initial two-year loan repayment award choose to remain in their service sites for an additional two years beyond their service term (Pathman and Konrad, 2012). Clinicians who eventually leave their original service sites but continue serving at-risk populations at other sites also contribute to the NHSC’s goal of impacting underserved communities.

Results of a 2012 retention assessment survey conducted by NHSC show that over the past decade short-term and long-term retention rates, measured as the fraction of providers remaining for up to one year beyond service term and remaining for ten years beyond service term, respectively, have increased (National Health Services Corps, 2012). Short-term retention increased by 28 percent between the two iterations of the survey, conducted in 2000 and 2012. Over 82 percent of NHSC alumni continue to practice in underserved communities for up to a year after service completion, while 55 percent of alumni remain in these areas ten years after completing their service obligation. Long-term retention has increased by 6 percent between 2000 and 2012, and is markedly higher for those who served in rural communities.


15 The current literature review discusses the State Loan Repayment Program (SLRP), a grant program administered by HRSA providing costing-sharing grants to over 30 states for the purpose of operating state loan repayment programs for primary care clinicians working in Health Professional Shortage Areas (HPSAs). However, the analysis in this project does not attempt to determine how the SLRP directly affects NHSC retention, and does not evaluate potential spillover effects of the program.

Definitions and Measures

Definition of HPSAs

The Health Professional Shortage Areas (HPSA) designation and scores are managed by the Health Resources and Services Administration’s (HRSA) Office of Shortage Designation. They identify geographic areas, facilities, and population groups within the United States that are experiencing a shortage of primary medical care, dental, or mental health providers. HPSA designations are determined based on the entities’ application to the Bureau of Health Workforce.

Within each discipline (primary care, dental, or mental health), there are three types of designations (Health Resources and Services Administration, 2013(c); HRSA, 2013 (d); HRSA, 2013(e)):

  • Geographic designations, which depend on the ratio between full-time equivalent (FTE) clinical providers and the patient population within a given area;

  • Population designations, that depend on the ratio between FTE clinical providers who serve a particular underserved population group and the size of the underserved population; and

  • Facility designations, which are based on criteria relating to one of three types of facilities, including federal and state correctional institutions and youth detention facilities, public or nonprofit private facilities, and state and county mental hospitals. Facilities must be serving patients from a previously designated population or geographic area and must prove that available capacity is insufficient to meet the level of need.

Geographic and population designations are based on characteristics of health care delivery within specific geographic areas or population groups. A service area can be a single county, group of counties, a partial-county, or a group of partial-counties. Partial-county geographic designations must comprise neighborhood or community areas whose characteristics are sufficiently different from surrounding areas (e.g., low income, high concentration of homeless, existence of isolating physical barriers, and extremely rural or remote areas).

Regions that are contiguous to geographic or population HPSAs must not contain adequate primary care resources that are readily accessible by the population residing within the HPSA. Resources in these nearby areas must be prohibitively distant or over-utilized.

NHSC-approved HPSA service sites include (National Health Services Corps, 2013):

  1. Federally Qualified Health Centers (FQHC);

  2. FQHC Look-Alikes;

  3. Rural Health Clinics (RHC);

  4. Hospital-affiliated Primary Care Outpatient Clinics;

  5. Indian Health Service, Tribal Clinic, and Urban Indian Health Clinics;

  6. State or Federal Correctional Facilities;

  7. Private Practices (Solo/Group); and

  8. Other Health Facilities, including:

    • Community Mental Health Facilities;

    • Community Outpatient Facilities;

    • Critical Access Hospitals (CAHs);

    • Free Clinics;

    • Immigration and Customs Enforcement (ICE) Health Services Corps;

    • Mobile Units;

    • School-based Health Programs; and

    • State and County Department Health Clinics.

These sites provide comprehensive outpatient, ambulatory, and primary health care services, while inpatient services may be provided by NHSC-associated CAHs. FQHCs, FQHC Look-Alikes, and Indian Health Service sites are automatically approved as NHSC sites so long as they meet all NHSC site requirements and the requirements of their respective facility type.

NHSC-approved sites of all types are rated on the degree of shortage, which is used in part to determine priorities for the placement of NHSC clinicians. Scores range from 1 to 25 for primary care and mental health facilities, and from 1 to 26 for dental care facilities, with 1 indicative of “lowest priority”. Automatic facility designations may receive a HPSA designation of 0 (HRSA Shortage Designation, 2013). Variables determining a HPSA score include the service area’s percent of population below poverty, an Infant Health index, and travel distance to the nearest accessible care outside of the HPSA. . HPSA scoring for all primary care facilities depends on four factors: population-to-primary care physician ratio, percent of the population with incomes below 100 percent of poverty level, infant mortality rate or low birth weight rate (whichever score is higher), and travel time or distance to nearest available source of care (whichever score is higher).

Currently, HPSA designations are used to define workforce shortages in primary care,, dental, and mental health HPSAs. Workforce shortage designations were developed in the mid-1960s (Salinsky, 2010), concurrent with the implementation of loan repayment programs and scholarship programs that were predecessors of modern NHSC programs. In the past, a similar designation was also used to characterize shortages in other health professions, including pharmacy, podiatry, and veterinary medicine (Salinsky, 2010).

In the past few years, several updates to the HPSA definition and NHSC recruitment policies have had effects on recruitment and retention. We review below three of these most important updates.

  1. The Health Care Safety Net Amendment of 2002 included a provision for the automatic designation of certain types of facilities as HPSAs. All FQHCs, FQHC-Look-Alikes, IHS sites, and certain Rural Health Clinics that provide care regardless of the patients’ ability to pay are automatically designated as HPSAs on the date when they are approved as one of these provider types. HPSA designation is required to be updated on a regular basis, but auto HPSA scores are only updated upon request.

  2. In 2009, as a result of the surge in funding from ARRA, NHSC cancelled the HPSA score floor requirement. The NHSC proceeded to allow clinics in all HPSAs to apply to become NHSC sites. This, in turn, nearly doubled the number of service sites, creating thousands of additional openings for NHSC clinicians (Pathman et al., 2012(a)).

  3. NHSC identified priority HPSAs to ensure adequate assignment to areas with the highest levels of need. As of January 2014, the NHSC LRP offers two levels of funding based on HPSA score. Initial award amounts for clinicians who serve at sites with HPSA scores of 0-13 may receive up to $30,000 for a two-year full-time commitment, or up to $15,000 for a two-year half-time commitment. Those who serve at sites with HPSA score of 14 or more may receive up to $50,000 or up to $25,000 for a two-year, full-time or half-time commitment, respectively (NHSC, 2014). NHSC Scholars are required to serve in HPSAs of greatest need, a classification that is determined each year. From July 1, 2013 through June 30, 2014, NHSC SP recipients must serve at NHSC-approved sites with a HPSA score of 16 or above for their discipline (NHSC, 2014).

Other Designations for Underserved Areas

Alternative methods for designating shortage areas are used within HRSA and other programs. Medically Underserved Areas (MUAs) is one such designation and was originally developed to assess a region’s primary care capacity and determine grantee eligibility for community health center grants. The HPSA and MUA designations depend on similar factors, but differ substantially in terms of the assessment procedure. While the shortage of health care providers is the primary factor in the designation of a HPSA, the MUA designation is based largely on the health needs of an area’s vulnerable population, measured in terms of the ratio of primary medical care physicians to 1,000 population, infant mortality rate, percentage of the population with incomes below poverty level, or percentage of the population age 65 or over (HRSA, 2013(g)).

States use a variety of shortage area designations for financial incentive programs that are similar to those of NHSC. In a study of 69 state-based programs operating in 40 states in 1996 (including loan repayment programs, scholarship programs, loan programs, direct incentive programs, and resident support programs), only 10 percent of programs used HPSA or MUA criteria without modification, while 35 percent of programs used these federal designations in conjunction with state-devised measures.16 More than half (55 percent) used unique criteria specific to their states sometimes specified within their state legislation (Pathman et al., 2004). More recent national estimates are currently not available. In the past, the Government Accountability Office (GAO) has criticized the methodology of both designations for their exclusion of certain types of physicians from provider counts, thus potentially inflating provider shortages (GAO, 1995). Additionally, neither the HPSA nor MUA designation calculation currently includes providers such as nurse practitioners, potentially overstating primary care provider shortages17. Some research also suggests that groups of sub-county service areas are often shaped for the purpose of securing a workforce shortage designation (Ricketts et al., 2007). Holmes (2005) further points out that because HPSA designation is self-initiated, communities with fewer resources may not apply.


16 In 1996, the 69 eligible programs were still relatively new and small, with median workforce size of 11 physicians (Pathman et al., 2004).

17 In 2011, the Negotiated Rulemaking Committee on the Designation of Medically Underserved Populations and Health Professional Shortage Areas proposed that nurse practitioners, physician assistants, and certified nurse midwives be included in the provider count for the development of the population-to-provider ratio (Health Resources and Services Administration, 2011(b)). To date, this proposal has not been acted upon.

Retention Metrics

The definition of retention may be approached in several ways. Retention metrics may be linked to the length of retention and the times at which retention is measured. For example, retention may be measured from the first day of a clinician’s service contract obligation, the day that the clinician concludes his or her initial service obligation, or the day that he or she completes the final service contract if a renewal contract was signed. Additionally, retention may be defined by location—whether the clinician remains in the same specific service site as under the service contract, remains within the same community but at a different practice location, remains within the same county or geographic region, or serving at any practice that prioritizes primary care for underserved communities (Pathman et al., 2012(b)).

NHSC typically assesses retention on the basis of the number of years remaining within either the original service site or within the same HPSA. The point in time from which retention is measured may vary, but it is worth noting that HRSA’s Bureau of Health Workforce (BHW) in its 2012 nationwide retention study of NHSC providers (Pathman et al., 2012(a)) opted to measure retention as the time between completion of the last service contract and the date of departure from either the index site or an alternative underserved site. The outcome of interest was the percentage of NHSC clinicians who were “still working in their NHSC service sites and within the broader set of practices that focus on care for the underserved at specific points in time after they had completed their NHSC service terms.” For clinicians who apply for and are granted renewal of Loan Repayment contracts after completing their initial Scholarship Program or Loan Repayment Program term, retention was calculated from the end of their last renewal contract (Pathman et al., 2012(a)).

Retention metrics used in federal, state, academic, and other types of programs vary widely with respect to length of assessment period, and degree of geographic inclusion in the retention criteria.

NHSC Retention Metrics

For primary medical care providers and primary oral care providers, retention in NHSC was evaluated under different criteria in the currently available studies. Table A.1 below summarizes the retention metrics that were previously used in the literature.

Retention Metric Characteristics

Time Frame

Study/Source

Population

Location

Short-term

Medium-term

Long-term

Pathman et al., 1992

NHSC SP physicians

  1. Remained in index site

  2. Remained in index community

  3. In practice in any rural county

-

Three years after initial date of employment

Eight years after initial date of employment

Konrad et al., 2000

All NHSC LRP and SP clinicians

  1. Index site retention

  2. Any underserved site retention

One month beyond period of obligation

One year beyond service obligation

Over four years beyond service obligation

Holmes, 2004

Physicians with a self-declared primary care specialty

  1. High underserved community

  2. Moderate underserved

  3. Non-US, not contiguous

  4. Non-US, contiguous

  5. Non-US metropolitan

-

Three to five years after graduation from medical school

Eight to ten years after graduation from medical school

Holmes, 2005

NHSC LRP and SP physicians

  1. Remained in initial community

  2. Practiced in any underserved location

-

Five years from initial placement

-

NHSC, 2012

NHSC primary mental and behavioral health providers

Continued to practice in a HPSA

-

Four years after service completion

-

Pathman and Konrad, 2012

All NHSC LRP and SP clinicians

Continuing to practice in a HPSA

Up to one year

-

Ten years and beyond

Pathman et al., 2012(a)

All NHSC LRP and SP clinicians

Remained in Index site

Worked in other practices that focus on care for the underserved

One month to one year after service completion

Two to five years after service completion

Seven to twelve years after service completion

In 2008 and 2012, extensive retention surveys were administered to NHSC providers. The 2012 survey commissioned by BCRS assessed retention in the short-term (1 month to 1 year after service terms are completed), mid-term (2-5 years), and long-term (7-12 years) (Pathman et al, 2012(a)). By contrast, NHSC employs slightly different criteria in its assessment of primary mental and behavioral health care providers. Retention for these groups of clinicians is measured as the percent continuing to practice in a HPSA four years after completion of the service term (National Health Services Corps, 2012).

In the BCRS survey, retention was assessed with respect to remaining within the same practice where the clinician served during his or her NHSC contract term, but also with respect to working in other practices that focus on serving underserved populations. This is similar to the approach taken by Holmes (2004) who, instead of defining retention solely on the basis of index site retention, as was common and remains a widespread practice, used a holistic definition of retention in underserved locations by studying both index site retention and retention in any underserved location after leaving the original service practice.

In the 2012 BCRS survey, “remaining within the last NHSC service site” was calculated as the number of months from the self-reported date of service completion until the date the clinician reported leaving the site at which he or she last served when completing the last NHSC service contract. “Working in practices that were focused on care for the underserved” was calculated from alumni’s self-reported information of working in practices that focused on care for the underserved at a given point in time. Those who reported working in non-clinical positions, were in training positions, were not working at the point in time, were not working in a practice that they indicated as focused on providing care for the underserved, and were not still working at their last NHSC service site, were considered to be not working in a “practice that focused on care for the underserved (Pathman et al., 2012(a)).”

The researchers who led the BCRS survey suggest that future studies of retention of NHSC clinicians would benefit from a more “clearly and consistently defined measure of the location of sites that qualify as successful retention outcomes (e.g., same site, any underserved site, any rural site, any ‘safety net’ employer, high reliance on Medicaid, etc.) as well as more focus on a consistent and meaningful measure of duration of retention.” Further, the researchers suggested that more attention be given to the definition and benchmarking of “success,” as well as the suitability of using various comparison groups (e.g., health professionals recruited to communities of similar circumstances but without a service obligation, or those working in similar settings under a state-based or other type of service obligation) (Pathman et al., 2012(b)).

In response to suggestions from the early 1990s that NHSC could enhance retention if it accepted only applicants from primary care-oriented schools, a research team led by Donald Pathman sought to determine, separately for NHSC scholars and physicians not affiliated with NHSC, whether retention in rural practice may be longer for physicians who graduated from public medical schools, were trained in a community hospital-based residences, or participated in rural training programs as medical students or residents. The indices of retention were percentage of physicians who continued working in nonmetropolitan areas, and the percentage of physicians who remained in their index practices (Pathman et al., 1992).

A distinction between retention at primary sites versus secondary sites has yet to be studied. Satellite clinical practice locations are generally located in areas apart from the main clinic in an effort to expand clinical access to patients in more remote areas. Typically, these sites have less patient volume than the main practice site, which is ultimately reflected in staffing. Retention at original service site or within the same geographic region is not known to take into account this distinction.

Military Clinician Retention Metrics

Certain elements of the medical military commitment requirements are similar to the service obligation requirements facing clinicians in NHSC’s Loan Repayment Program and Scholarship Program. Characteristics of the military medical program that are unlike NHSC requirements include:

  • Long obligation to repay subsidized medical training;

  • Military retirement eligibility only occurs after 20 years of service; and

  • Availability of physician retention bonus payments set by the military, based on medical specialty.

The accession and retention of military physicians is an integral part of maintaining an effective and sufficient medical workforce, particularly during periods of frequent deployments overseas. Currently, physicians enter the Army, Navy, and Air Force through one of four primary channels: the Uniformed Services University of the Health Sciences (USUHS), the Health Professions Scholarship Program (HPSP), the Financial Assistance Program (FAP), and direct commissioning of fully trained physicians (Edgar, 2009).

Retention among military physicians generally refers to an extension of the physician’s active duty service commitment after their initial service commitment has expired. All else equal, retention of skilled physicians is more cost-effective than recruiting and training new physicians. Because the skill-to-cost ratio is higher for retention than it is for recruitment, it makes sense to focus on retention efforts. The significant time lag between an individual’s commitment to serve and the actual provision of services further emphasizes the importance of reducing attrition of clinicians whose service commitments are about to expire.

Several studies evaluated the efficacy of current retention efforts in the Air Force Medical Corps and in the military at-large. For example, Keating et al. (2009) has shown that in virtually every accession category in the Air Force Medical Corps, the majority of entering physicians do not stay beyond their minimum service requirement.18 In 2009, the Senate Armed Services Committee (SASC) issued a directive to the Department of Defense (DOD) to evaluate medical workforce planning efforts, assess the existence of gaps between need and fill rate among certain specialties, and document challenges that contribute to clinician shortfalls. Retention was broadly defined as “retaining service members with the necessary skills and experience”

(GAO, 2009). For fiscal years 2001 through 2008, year-to-year medical personnel retention rates in the military’s eight branches ranged between 77 percent and 95 percent (GAO, 2009). The retention rate was calculated by first subtracting the number of medical officers who left service in a given fiscal year from the beginning total for that fiscal year. The difference was then divided by the total number of medical officers serving at the beginning of the fiscal year (GAO, 2009). Within this time frame, annual retention rate among active component military nurses was higher than that of military physicians and military dentists (GAO, 2009). Three main factors are cited as likely to affect retention: the pay differential between military and civilian physician positions; length of deployment; and opportunities for military physicians to maintain and improve clinical skills (Mundell, 2010). Most evidence suggests that increasing salaries for military physicians would lead to increased physician retention. The Multiyear Specialty Pay (MSP) program allows eligible physicians and dentists to make two-, three-, or four-year commitments to additional service in exchange for supplementary annual payments. Increasing MSP levels among Air Force physicians also increases the percentage of physicians who choose to accept contract renewal payments rather than leave service (Keating et al., 2009).

An analysis of dentist retention by Keating et al. (2009) found that between 1976 and 2007over 60 percent of dentists entering the Corps at the beginning of the study period stayed for longer than three years, while more than 40 percent stayed longer than seven years, and over 20 percent stayed longer than 19 years. However, retention over the study period fell across all experience levels. Additionally, while direct accession was the primary supply of dentists during the beginning of the study period, this trend eventually reversed, allowing HPSP graduates to become the most common accession source. While the cause for the decline in retention is not certain, factors differentiating Dental Corps participants and Medical Corps participants abound. Dentists inherently have fewer specialties from which to choose, and the promotion trajectory for dentists is much more predictable. Interestingly, eligible dentists accept MSP nearly four times the rate that physicians accept it (52 percent and 13 percent, respectively). The study does not analyze nurse practitioners, registered nurses or physician assistants.


18 Includes direct accessions, and accession through the Health Professions Scholarship Program (HPSP) and Uniformed Services University of Health Sciences (USUHS) medical school.

Other Program Retention Metrics

Many state-issued, institutional, or other types of programs tend to use their own retention metrics that may be more relevant to local conditions than federal metrics. Generally, the differences lie in the length of time assessed and geographic inclusion criteria.

For example, in evaluating retention in state programs, Pathman et al. (2004(a)) tracked retention at two-year intervals to measure the percentage of clinicians remaining in the index practice at discrete points in time. Of the 69 state programs operating in 1996, the average level of assistance did not differ significantly across the five types of programs reviewed (scholarships, service-option loans, loan repayment, direct financial incentives, and resident support). Four-year and eight-year retention in index practice were used as the primary metrics of retention in the study. Average minimum service term was 12 months in resident support programs, 18 months in scholarship programs, 10 months in service option loan programs, 29 months in loan repayment programs, and 36 months in direct incentive programs (Pathman et al., 2004(a)).

To assess long-term retention of graduates of the Physician Shortage Area Program (PSAP) of Jefferson Medical College, Rabinowitz (2013) defines retention for this program as the percentage of individual graduates practicing in the same rural areas in 2011 as they were initially, 20 to 25 years after they first began practice. Practice location was considered to be the same area if it was in the same rural county or an adjacent county as when the graduate was first located (Rabinowitz et al., 2013). ‘Rural’ was defined as counties that are not designated as standard metropolitan areas.

Other programs aimed at increasing the supply and retention of rural clinicians define retention in varied ways. For example, the term rural is sometimes used without further explanation, or it may refer to a non-Standard Metropolitan Statistical Area county (non-SMSA), or communities of population below a certain threshold. Oftentimes, studies use population under 25,000 or under 50,000 as a benchmark (Rabinowitz et al., 2008). The length of retention may vary across studies as well, ranging from 1 year to 25 years.

Key Findings in the Literature

NHSC Enrollment and Retention

Since 1972, NHSC has enabled health care facilities in underserved communities across the nation to compete with private medical practices, health systems, and hospitals for community-responsive and culturally competent clinicians. Experience has shown that the misdistribution of clinicians does not remedy by itself (NHSC, 2000). According to Pathman and Konrad (2012) among all federal initiatives implemented since the 1960s to address the medical workforce shortage and mal-apportionment, the NHSC is a key resource.

Prior to the 2012 NHSC retention analysis commissioned by BCRS, the last large-scale evaluation of NHSC retention was conducted through a 1998 survey of NHSC clinicians and alumni from the 1980s and early 1990s (Konrad et al., 2000). This evaluation used survival analysis to demonstrate that over the years, retention rates were higher among clinicians who had completed the Loan Repayment Program rather than the Scholarship Program (57.2 percent of LRP clinicians remained at the service site at least one month after service completion, compared to only 20.7 percent of those in the Scholarship Program). Those in the Scholarship Program were also found to be less likely to be working in any practice that focused on care for the underserved. The hazard ratio of leaving the original service site over time for LRP participants ranged between 0.63 and 0.72 for “any underserved site.” These findings align with the results of retention rates of state loan repayment programs (Pathman et al., 2004(a))19. When measured from the date clinicians began serving within their programs, the hazard ratio of leaving one’s original service site over time was 1.96 for scholarship program participants relative to loan repayment program and direct incentive program participants (Pathman et al., 2004(a)).

While some studies report that program participants of loan repayment and scholarship programs are more likely to serve in underserved areas, retention rates from these programs may suffer from self-selection bias and therefore may not be indicative of the true program effect on retention (Rittenhouse et al. 2008). This bias arises as those who choose to apply for NHSC enrollment may be different from non-applicants along various unobserved characteristics. Using AMA Masterfile Data to follow three different physician cohorts from 1976 to 1996, Holmes (2004, 2005) accounts for selection bias while estimating the transition probabilities of NHSC enrollees and non-enrollees moving from period 1 (3-5 years after graduation) to period 2 (8-10 years after graduation). For many enrollees, this is the transition from the initial service obligation to the post-service period. He finds that enrollees are less likely than non-enrollees to stay in their initial place of practice and that the decrease in enrollees is not directly attributable to community characteristics or the nature of being an underserved area, but rather due to the fact that the enrollees’ initial location preferences are constrained, as some of these locations are not approved by the NHSC. This selection effect can be shown in a dynamic programming model that simulates the choice of individual providers regarding location, program enrollment, and retention, in response to policy incentives, socio-demographic variables, random shocks, and other unobserved variables. This dynamic programming model is described briefly in the Appendix. In Table A.2 below we summarize the transition rates estimated by Holmes (2005), across underserved and non-underserved areas for NHSC enrollees and non-enrollees at the end of period 1.

Table A.2: Estimated Transition Probabilities for NHSC Enrollees and non-Enrollees

 

HUS

MUS

Other

NOTES: HUS = highly underserved area; MUS = moderately underserved area; Other = aggregation of four non-underserved areas, the largest of which is metropolitan areas. Source: Table 2 of Holmes (2005).

Non-Enrollees

HUS

0.25

0.20

0.55

MUS

0.03

0.31

0.65

Other

0.01

0.03

0.96

Total

0.01

0.06

0.93

Enrollees

HUS

0.19

0.22

0.60

MUS

0.02

0.13

0.85

Other

0.02

0.08

0.90

Total

0.03

0.10

0.87

In the long run approximately 3 percent of physicians supported by NHSC are retained in highly underserved areas and 10 percent in moderately underserved areas (Holmes, 2004). He estimates that if NHSC tuition subsidies were to be increased by $5,000 per enrollee, the funds would yield a 1.7 percent increase in the long term (post-service) physician supply over current physician supply in highly underserved communities. Assuming a current repayment amount of $30,000/year, the estimated elasticity is 0.1 with respect to the loan repayment amount, which indicates that a 10 percent increase in the NHSC LRP would yield a 1 percent increase in long term post-service physician supply over current physician supply in highly underserved communities. Holmes also estimates that a $1,000 rise in tuition costs increases the likelihood of NHSC enrollment by 0.36 percentage points.

General retention of the medical workforce in rural, urban and frontier sites has also become more balanced over the years. Among clinicians serving in 2005, retention rates did not differ significantly for those across these three types of communities. In 1998, however, retention at all points in time was higher for those who served in rural practices (Pathman et al., 2012(a)). A common perception is that retention is shorter in rural areas because shortages are generally more prevalent and more critical in rural areas. However, Pathman et al. (2004) demonstrated that physician retention in any area is similar—shortage areas arise because of lower recruitment rates rather than lower retention rates. Pathman, Konrad, and Ricketts (1992) examined whether there is an association between characteristics of a physician’s training and the amount of time that he or she chooses to stay in rural practice. The results indicated that among NHSC Scholarship Program physicians, no retention differences existed for those who trained or have lived in rural areas previously. Additionally, public school graduates in the NHSC were found to remain in rural areas for shorter periods of time than private school graduates. For rural physicians, only the type of medical school predicted retention. At the time of study, NHSC physicians were also substantially less likely than non-NHSC physicians to be working in their index practices after eight years of employment (13 percent versus 44 percent) and in nonmetropolitan counties (25 percent versus 52 percent). Long-term retention rates of NHSC clinicians in their original practices have not improved significantly over the years. Among 2005 alumni, 46 percent remained in their index practice for at least two years, while only 26.4 percent remained after four years (Pathman et al., 2012(a)).

It has also been shown that the presence of NHSC participants in underserved areas increases the supply of nonparticipating physicians in those areas on average by 6 percent (Pathman et al., 2006). From 1981 to 2001, rural single-county HPSAs staffed by NHSC enrollees saw an average increase of 1.4 non-NHSC primary care physicians per 10,000 population, compared to a 0.57 mean increase in counties without NHSC enrollees, a finding which remained statistically significant even after adjusting for baseline county demographics and health care resources. The workforce growth in NHSC-supported HPSA counties was due in part to initial differences in the availability of primary care physicians and hospitals relative to other counties – which lead to flows of both NHSC and non-NHSC providers into those counties-, and in part possibly due to factors not incorporated within the study, such as stronger leadership and community organization in NHSC-staffed counties. Researchers also suggest that this growth coincided with the emergence of NHSC’s loan repayment program and the expansion of state-run scholarship and loan repayment programs. The increase in non-NHSC physicians may have come from NHSC alumni, physicians who were serving obligations to state programs, or unobligated physicians who were attracted to the local medical communities that were improved by NHSC staffing (Pathman et al. 2006).

During the past two decades, NHSC renewed its programmatic focus on retaining providers beyond their service terms. These efforts included shifting resources toward the expansion of the LRP, which was found to be more effective at promoting retention than SP. The GAO reported in 1995 that the cost-per-LRP recipient was 37 percent lower than the cost-per-SP recipient, when adjusted for the time-value of money and defaults (USGAO, 1995).20 In recent years, substantial funding for LRP and SP programs has come from the American Recovery and Reinvestment Act (ARRA), and more recently from the Affordable Care Act (ACA), which has allowed extending service contracts of physicians, NPs, and PAs.


19 Some state loan repayment programs are financed in part by HRSA-administered programs, such as the State Loan Repayment Program (SLRP). SLRP is a federally-funded grant program that provides cost-sharing grants to assist states and territories in operating their own state-based medical education loan repayment programs for primary care providers working in HPSAs in the state. As mentioned above, since the focus of the current study is NHSC retention, we ignore the potential spillover effects that SLRP might have on NHSC programs.

20 More recent studies of the cost efficiency of NHSC programs have not been performed.

American Recovery and Reinvestment Act (ARRA) and Patient Protection and Affordable Care Act (ACA) funding expansion

The American Recovery and Reinvestment Act (ARRA) was an economic stimulus package to save and create jobs and invest in infrastructure, education, health, and renewable energy. The Act designated $300 million to expand the NHSC, which was intended to add over 4,000 clinicians to the NHSC workforce over a two-year period (ending in February 2011). In total, NHSC saw its workforce rise from 3,017 to 7,713 over this period—an increase of 156 percent (Pathman and Konrad, 2012).

Due to increased funding, the estimated number of people receiving care from NHSC clinicians rose from 4 million to 9 million during this timeframe (Pathman and Konrad, 2012). This study documents the effects of ARRA funds on the NHSC’s workforce size, composition, and location during the first two years of increased funding. During this period, NHSC made several programmatic changes that facilitated expansion. It abolished its HPSA score floor requirement and extended Loan Repayment Program eligibility to include sites located within all HPSAs. In addition, it added a half-time service option and increased maximum loan repayment amounts from $50,000 to $60,000 for an initial 2-year contract. It also revamped its application system and streamlined its assignment process to efficiently facilitate additional enrollment. NHSC interchangeably funded clinicians with both ARRA funds and regular annual appropriations. Pathman and Konrad (2012) document the following changes in enrollment between 2009 and 2011:

  • NHSC LRP clinicians increased from 2,474 in March 2009 to 7,187 in February 2011

  • LRP primary care clinicians increased from 1,449 to 4,095

  • LRP dental care providers increased from 341 to 975

  • LRP mental health clinicians rose from 684 to 2,117

  • NHSC SP awards declined from 543 in March 2009 to 526 in February 2011

Clinician composition also changed significantly during the ARRA funding period. Notably, among primary care clinicians in the Loan Repayment Program, physician enrollment grew substantially less over the two-year period (114 percent) than among nurse practitioners (367 percent), physician assistants (199 percent), and nurse midwives (175 percent). The proportion of physicians among all Loan Repayment Program participants fell from 31.3 percent to 23.1 percent over the two year period. The highest rate of overall growth occurred among nurse practitioners, who increased from 10.1 percent to 16.0 percent of the entire NHSC workforce. Significant growth was also seen among mental health providers—licensed professional counselors increased by 389 percent. Pathman and Konrad, 2012 note that the highest rates of growth were seen in disciplines with lower average income (i.e., licensed professional counselors, dental hygienists, etc.), in which the loan repayment sum constitutes a higher proportion of salary.

The additional ARRA funding did not bring about an immediate change in the urban/rural distribution of the NHSC workforce. Prior to the ARRA period, 44.7 percent of clinicians served in rural areas defined by the Rural Urban Commuting Area (RUCA) classification. In February 2011, 42.0 percent were serving in rural areas. A 2.5 percent decrease was seen in the proportion of NHSC clinicians serving in very small towns (<2,500 population) over this time period. Interestingly, the study found that workforce growth rates varied significantly across states. During the ARRA period, NHSC practitioners rose 291 percent in states with the lowest NHSC clinicians per 100,000 people below poverty rate, compared to a 111 percent growth in states with the highest rates.

Despite growth in NHSC LRP enrollment, the real efficacy of the federal incentives through ARRA will be reflected in the changes in short and long-term retention of NHSC clinicians in underserved areas after the completion of their service obligation period, and whether the program buys more person-years. Another important consideration will be the cost at which more person-years are added. Despite the 2.5 fold growth in total clinicians gained through the supplemental funding, the total workforce still meets less than one-third of the 34,000 physician-need in HPSAs across the United States.

ACA appropriated $1.5 billion in new funding for NHSC over a five-year period beginning in FY 2011. The law contained new provisions to support the recruitment and retention of providers in underserved communities by increasing the value of LRP awards, instituting a half-time service option and allowing for limited teaching and other non-clinical work. This is particularly important now and in the coming years because the rural physician shortage may be exacerbated due to increases in insured patients and pent-up demand for health care, as was seen in Massachusetts after its health care expansion (Massachusetts Medical Society, 2012).

A 2012 study that sought to assess medium and long-term NHSC retention found that clinicians are now remaining in service sites for longer periods of time (Pathman et al., 2012(a)). Many who leave those sites are moving to other sites that are focused on providing care to the underserved. Among NHSC clinicians who finished their final service terms in 1998 and 2005, retention rate at sites focusing on care for the underserved were the highest for physicians and the lowest for physician assistants. In the 2005 cohort, the highest retention was seen for mental health practitioners and the lowest for dentists.21 In 2012, long-term retention rate for dentists and dental hygienists (defined by NHSC as practicing within a HPSA ten years after the completion of the service requirement) was 48.1 percent (NHSC, 2012). The retention rate for mental health and behavioral health care providers (including health service psychologists, licensed clinical social workers, psychiatric nurse specialists, marriage and family therapists, and licensed professional counselors), measured as the percentage of clinicians who remain in a HPSA four years after service obligation completion, was 61.1 percent (NHSC, 2012).


21 It is important to note, however, that the mental/behavioral health disciplines participate only in the Loan Repayment Program.

Other characteristics associated with a higher NHSC Retention

The Multi-State NHSC Retention Collaborative, a consortium of 11 state Primary Care Offices, funded a study on the retention of clinicians within their states who serve in NHSC or in similar state-based programs (Pathman et al. 2012(b)). Using data from two key surveys—the 2012 national NHSC medium and long-term retention study and the survey of clinicians in NHSC and state programs in 11 states participating in the Collaborative, this evaluation sought to identify the circumstances and characteristics of clinicians, service sites, and service experiences that contribute to longer retention. Examining the current and recent NHSC and state-program participants’ plans to remain in their original service sites for one, two, and up to ten years, the authors find that of the 1,558 NHSC and state service program participants surveyed, 69 percent remained or anticipated remaining in their service sites for at least one year beyond their service terms, 48 percent anticipated remaining at least three years, and 20 percent anticipated remaining at least ten years. Consistent with previous studies, a significantly higher proportion of NHSC Loan Repayment Program participants anticipate remaining in service sites beyond contractual terms (70 percent compared to 36 percent at one year, 35 percent compared to 13 percent at five years, and 19 percent compared to 2 percent at 10 years, respectively). Within the NHSC Loan Repayment Program, anticipated retention rates are similar across the eleven states.

After simultaneously controlling for the clinicians’ disciplines and demographics, factors relating to principal reason for service and type of practice account for 16.3 percent of variation across clinicians in their plans to remain in their service sites for at least two years beyond their service term, and for 18.7 percent of the variation at five years (Pathman et al., 2012(b)). The factors that have a positive effect on retention include:

  • Being a physician (as opposed to a nurse practitioner, physician assistant, dentist or mental and behavioral health practitioner);

  • Being age 30 or over, non-Hispanic White race/ethnicity, having children, and serving in a state where one grew up and where one trained;

  • Principally motivated by the chance to work with underserved populations rather than for the programs’ financial support; and

  • Serving in a rural health facility, mental health or substance abuse treatment facility, a prison, or “other” type of facility.

Factors relating to clinicians’ satisfaction with work and practice, family integration into the broader community, and overall assessment of their service program explain 28.6 percent of variation across clinicians in their plans to remain in their service sites for at least two years beyond their service term and for 27.1 percent at five years (Pathman et al., 2012(b). These factors include:

  • Feeling of belonging and safety within the community for the clinician and clinician’s family;

  • Satisfaction with the program administrator, salary, the assessment of the practice overall, and access to specialist consultation; and

  • Overall satisfaction with the service program and the program staff support.

Other findings from the study regarding socio-demographic characteristics that contribute to retention are summarized as follows (Pathman et al., 2012(b)):

Characteristics of clinicians

  • Physicians and mental health clinicians are more likely to remain in their service sites than nurse practitioners, physician assistants, or dentists at five and ten years beyond the service term agreement. The anticipated retention rate among dentists increases over time relative to other disciplines, and matches that of physicians and mental health practitioners 10 years post service terms.

  • Clinicians who are older than 29 years, non-Hispanic White, have children, and grew up and/or trained in the state where they serve are more likely to anticipate remaining in their service sites than younger, minority, childless, and out-of-state clinicians. Anticipated retention was not found to be associated with gender or marital status.

  • Clinicians primarily motivated by the financial support aspect of program are less likely to anticipate remaining in their service sites over time. Those in the NHSC LRP are more likely to rate their desire to serve underserved populations as an equal or higher motivation than financial assistance, compared to SP or state program participants.

  • Variation in satisfaction with the NHSC Loan Repayment Program exists among states. Participants in California and Kentucky are more likely to report having their expectations exceeded, while those in North Carolina, North Dakota, and Nebraska are more likely to report that their expectations were not met.

Rural versus urban county setting

  • Rural versus urban location of practice site was not statistically significantly associated with anticipated retention among participants. However, rural/urban differences in retention were found for specific states. For example, NHSC LRP participants serving in rural counties of Kentucky and Nebraska show higher anticipated retention than LRP clinicians in urban counties of their states.

  • Also, anticipated retention rates among rural participants of New Mexico’s state loan repayment program are higher than for peers serving in the urban counties of that state. Conversely, anticipated retention in many other state programs, such as the Alaska support-for-service program, is higher in urban counties than rural counties.

Clinician experiences

  • The following factors are consistently associated with higher anticipated retention rates: overall satisfaction with the practice, having a satisfactory relationship with the practice administrator, salary and income, and access to specialist consultation for patients.

  • Retention is more likely among clinicians who report a greater sense of clinician/family fit with the community.

  • NHSC Loan Repayment Program clinicians report higher average satisfaction with their work and practices than NHSC Scholarship Program participants. Minimal variation of this finding was seen across the 11 states studied.

Type of organization

  • Rates of anticipated retention are higher for those serving in rural health centers, prisons, and mental health and substance abuse facilities than in FQHCs, Indian Health Service sites, or tribal sites.

  • Higher proportions of clinicians working in hospital-based clinics anticipate remaining in their sites over time than those working in any other type of site.

Retention of Military Clinicians

The Armed Services currently offer a variety of incentive plans to bolster retention, including accession bonuses and other pay incentives, scholarship programs, special pays, loan repayment programs, and retention bonuses. Findings from evaluations of these programs may inform our understanding of retention in analogous programs in NHSC.

An evaluation of the Multiyear Specialty Program (MSP) was commissioned by the Air Force in 2009 to assess the tendencies of physicians and dentists in the Air Force Medical Corps and Dental Corps to accept MSP. The study (Keating et al., 2009) differentiates between retention among two cohorts of MSP eligibility. The first “peak” of eligibility occurs after three or four years, when physicians who completed a civilian residency fulfill their initial service obligation (termed ‘early eligibles’). The second eligibility peak occurs after seven to eight years, when military residency-completing physicians fulfill their initial service term (‘later eligibles’). ‘Later eligibles’ accept MSP at much higher rates than ‘early eligible’ physicians.

Several findings from this study are consistent with other evidence suggesting that prior indication of commitment to serving a particular population or organization is correlated with longer retention. Although retention in the Air Force Medical Corps is overall low, retention among USUHS graduates is notably high. Graduates were just as likely to stay for over 20 years as they were to leave after serving their minimum service obligation. It is suggested that members of this group, in particular, have a greater inclination for military service. For recipients of the Health Professions Scholarship program, the largest accession group, completing a military residency was associated with nearly a three-fold increase in the likelihood of remaining for more than 20 years, relative to those who completed a civilian residency. One hypothesis is that a military residency is indicative of greater commitment to military service and/or, the military residency allowed physicians additional time to become acclimated to military culture (Keating et al., 2009).

Overall, accession source and whether the physician completed a military residency (for HPSP entrants) were the factors most strongly associated with retention in the Air Force Medical Corps. Short-term retention (retention within the first seven years of entrance) for HPSP entrants and long-term retention (20 years or longer) has not changed consistently or considerably between entering cohorts from 1979 to 2000. Within this time period, short-term retention for HPSP entrants with military residencies was approximately 55 percent of all entrants and 5-15 percent for physicians with civilian residencies. Long-term retention was 12-18 percent for scholarship program entrants who completed military residencies, compared to 3-8 percent for those who completed civilian residencies.

Recent research by Gray et al. (2012) found that military retention is highly sensitive to civilian-military compensation differences in the first year of obligated service and substantially less sensitive after the initial year. Gray et al. (2012) found that a 10 percent increase in total compensation is associated with an approximate 18.7 percent increase in retention during the first unobligated year, but just a 0.2 percent average increase in yearly retention thereafter, up to the point of retirement eligibility at 20 years of service. Further, while studies have demonstrated the effect of deployment length on retention, little research has been performed on deployment length, which can vary significantly among practitioners and may impact morale and how clinicians view their deployment tours (Edgar, 2009). Administrative burden is also commonly cited as a major deterrent, particularly regarding the Armed Forces Health Longitudinal Technology Application (AHLTA) electronic medical record system.

While the impact of pecuniary rewards on retention is relatively well documented, albeit variable, the link between the ability to maintain or improve clinical skills and retention is less well understood. In exchange for additional service, physicians may receive a MSP, a permanent change of station, or opportunities to pursue further training. Research has shown that physicians are more likely to stay in the Air Force Medical Corps when offered higher MSP levels (Keating et al., 2009). An earlier study by Daubert (1985) analyzed the retention of direct-accession physicians into the Air Force during 1980s and found the lack of parity between civilian and military wages and lack of professional development opportunities were key reasons for low retention.

Mundell (2010) estimated the magnitude of the effect of “practice opportunities,” on the retention of critical care specialists in the Army, Navy, and Air Force. Using data from the Defense Manpower Data Center, the study sought to determine the effects of practice opportunities (using facility type as proxy) on the military’s ability to retain physicians. The estimates, obtained from duration models, suggest that a hypothetical programmatic shift of assigning physicians to large medical centers between June 1996 and June 2009 would have led to an increase in average active duty duration of approximately 0.62 months across all three services (Army, Air Force, and Navy) in the first 5.5 years of post-residency service. Implementing a required military residency would have led to an increase of approximately 1.8 months across all three services in the first 5.5 years of post-residency service. Furthermore, the results suggest that increasing deployment length or frequency during a physician’s initial service obligation decreases retention, while increases in the same two dimensions after the clinician has completed the initial service term do not have a negative effect on retention and may even have a positive effect in certain cases. This is consistent with the hypothesis that those with a dis-preference for deployment leave after completing their initial obligation.22


22 Other findings for non-medical military personnel suggest a positive correlation between deployment and retention. Though longer deployment (beyond 11 months) has negative effects on retention, higher reenlistment bonuses have helped the Army retain its personnel in spite of longer deployment (Hosek and Martorell, 2009).

Variation of Retention in Large Health Networks

Provider retention is a critical issue for health plans and service delivery networks because quality clinicians are a health plans’ most valuable resource. Retention is particularly important in Medicaid managed care, where low reimbursement rates make recruiting and retaining providers very challenging. Research indicates that there are significant costs associated with losing a network physician. A Colorado Permanente Medical Group study showed that patients whose primary care physician had left the medical group had more hospital admissions, emergency room and specialty visits, and laboratory and x-ray tests (Magrid et al., 2001). Dissatisfaction with care may also lead to higher rates of patient disenrollment.

The Association for Community Affiliated Plans, which represents 18 Medicaid-focused health plans and one integrated service delivery network across 11 states, conducted a comprehensive study of four of its member health plans in 2004 to assess the actions and policies that may inhibit adequate provider recruitment and retention. The top four challenges for recruitment and retention reported by plan executives were: 1) low payment rates; 2) clinicians’ preference for private patients; 3) scarcity of providers, particularly in rural regions; and 4) frustration with referral and pre-authorization processes (The Commonwealth Fund, 2005). The health plans emphasized the importance of two key modes of retention—sustaining relationships with providers through regular meaningful communication and keeping up to date with technology offerings to ease administrative burden.

Health plan leaders achieved the greatest success in improving provider relations in the following five areas:

  1. Payment practices, payment incentives, and financial assistance—providing prompt payment for care or offering incentives for high-quality care

  2. Utilization management practices—updating referral processes, improving authorization practices, improvements to customer service

  3. Communications and provider outreach practices—conducting site visits and engaging in frequent written communication, holding provider recognition events, and updating provider outreach processes

  4. Practices to simplify administrative burdens—includes simplifying the eligibility verification process, the credentialing process, and encounter data submission process

  5. Enabling service practices—health plans may provide certain services to ease burden on provider. Services may include transportation services, child care, interpreter services, and providing phone service to facilitate contact between patient and care manager

Kaiser Permanente, a national leader in provider retention, offered a similar set of best practices centered on enculturation and mentoring, as well as demonstrating strong leadership qualities such as setting expectations, providing feedback, providing recognition, and active listening (King and Speckart, 2006).

Strategies to Increase the Supply of Primary Care Services

There are numerous strategies that federal or state programs and medical institutions use to increase the retention of the national state or local clinical workforce. There are several types of programs that provide assistance to medical students and residents in the primary care disciplines. With the exception of some resident-support programs, nearly all programs require a service commitment in exchange for financial support. The remainder of this section highlights these initiatives, including recent programmatic changes by NHSC, state-level incentive programs, programs funded by Title VII and Title VIII, institutional programs, and recent movements to increase the attractiveness of primary care.

The primary types of incentive programs include:

Pre-service programs

  1. Scholarship programs, which obligate physicians early in their medical training to complete a service requirement in the future or face heavy financial penalties

  2. Service-option loan programs, which offer medical students a choice of performing service or repaying loan funds at standard interest rates

  3. Resident-support programs (RSP), which are employee assistance programs designed specifically for residents

Post-training programs

  1. Loan repayment programs which commit students near the end of their residency training to provide a period of service in exchange for assistance in repaying traditional education loans

  2. Direct financial incentive programs, which commit students near the end of their residency to provide a period of service in exchange for unrestricted funds

Most medical students fund their education with traditional loans that do not carry service obligations. However, past research has indicated that given the chance, a large proportion of students would change to a primary care specialty if adjustments in income, hours worked, or loan repayment availability occurred (Rosenthal et al., 1994). This is relevant because participants in financial incentive programs are more likely to serve in underserved areas than non-participants (Holmes, 2005).

Overall, the literature suggests loan repayment programs and direct financial incentive programs are more successful than scholarship programs, loan programs, and resident-support programs in affecting clinician supply (NHSC, 1995; Pathman et al., 2004). These programs are also the most prevalent nationwide, and make up the largest share of aggregate funding in the NHSC program.

Pre-service programs often suffer from lower enrollment completion rates because student interest often changes over the several years of training. Post-training programs have been shown to be much more important for students with more than $50,000 in debt than for students with less debt (Rosenthal et al., 1994).

Appendix B: Detailed Tables from the Analytic Datasets

Table B.1: Distribution of the NHSC Workforce by Provider Discipline

Year

2000

2001

2002

2003

2004

2005

2006

2007

2008

2009

2010

2011

2012

2013

Allopathic Physician

294

341

418

665

807

977

947

927

898

998

1,353

1,786

1,712

1,531

Osteopathic Physician

78

95

125

189

253

290

303

331

330

363

470

612

607

540

Chiropractor

0

0

0

7

8

10

8

3

1

1

1

1

1

1

Certified Nurse Midwife

27

26

44

73

98

102

90

84

84

101

174

202

164

147

Dentist

172

181

214

306

341

385

393

364

377

507

774

1,099

1,128

1,039

Health Service Psychologist

48

68

117

204

280

337

362

329

340

464

678

834

758

620

Licensed Clinical Social Worker

55

67

95

158

196

207

217

193

180

255

514

857

911

778

Licensed Prof Counselor

0

7

27

73

113

155

171

142

129

240

507

889

950

872

Marriage and Family Therapist

11

10

10

26

37

33

33

35

37

44

80

132

139

134

Nurse Practitioner

117

141

216

304

370

396

391

362

335

512

1,070

1,648

1,563

1,312

Pharmacist

0

3

3

24

23

5

0

0

0

0

0

0

0

0

Physician Assistant

110

113

175

332

437

467

407

402

390

532

938

1,327

1,264

1,098

Psychiatric Nurse Specialist

2

4

6

9

9

8

6

6

10

10

14

33

35

27

Registered Dental Hygienist

5

7

14

27

34

40

39

31

31

55

138

230

192

176

 

Table B.2: Distribution of Providers by State and Year in Medicare Provider Data

State

Year

2005

2006

2007

2008

2009

2010

2011

Unknown

2,604

2,774

3,467

3,091

2,924

2,865

2,751

Total

651,560

699,461

805,700

850,225

881,295

915,149

954,015

 

AK

1,582

1,759

2,057

2,194

2,269

2,410

2,521

AL

8,951

9,645

10,762

11,165

11,357

11,713

12,113

AR

5,064

5,593

6,380

6,631

6,801

6,989

7,228

AZ

11,115

12,136

14,347

15,473

16,334

17,293

17,940

CA

54,668

57,871

66,901

71,741

74,686

78,946

84,831

CO

9,550

10,184

12,362

13,263

14,008

14,696

15,405

CT

10,578

11,351

13,144

13,741

14,085

14,396

14,876

DC

2,177

2,286

2,603

3,310

3,556

3,807

3,573

DE

2,140

2,307

2,706

2,877

3,003

3,140

3,294

FL

36,038

38,874

45,224

48,247

49,943

51,884

54,105

GA

15,483

16,975

19,619

20,598

21,207

22,142

23,256

HI

2,597

2,935

3,459

3,630

3,670

3,801

3,873

IA

7,389

7,858

8,825

9,237

9,498

9,821

10,303

ID

3,006

3,245

3,776

4,066

4,249

4,413

4,625

IL

26,440

28,555

32,803

34,451

35,493

36,888

38,608

IN

12,569

13,754

15,935

16,593

17,160

17,776

18,432

KS

6,616

7,154

8,184

8,496

8,669

9,008

9,385

KY

8,871

9,690

11,362

11,933

12,406

12,841

13,365

LA

8,828

9,140

10,569

11,202

11,697

12,176

12,781

MA

23,272

24,986

28,877

30,383

31,519

32,708

33,617

MD

13,932

14,989

16,990

17,422

18,204

18,992

20,315

ME

4,597

4,958

5,842

6,134

6,266

6,487

6,668

MI

24,678

26,492

30,206

31,716

32,414

33,568

35,012

MN

15,946

17,049

19,466

20,514

21,228

21,916

22,711

MO

13,402

14,348

16,518

17,258

17,702

18,410

19,298

MS

4,832

5,178

5,868

6,224

6,373

6,705

7,128

MT

2,719

2,904

3,378

3,518

3,610

3,764

3,861

NC

18,560

20,232

24,056

25,586

26,840

28,144

29,163

ND

2,217

2,374

2,674

2,775

2,915

3,044

3,167

NE

4,374

4,761

5,534

5,795

5,958

6,182

6,456

NH

4,043

4,249

4,950

5,142

5,377

5,612

5,758

NJ

20,286

21,947

25,204

26,457

27,446

28,326

29,546

NM

3,996

4,318

5,047

5,265

5,406

5,670

5,883

NV

3,811

4,161

4,829

5,132

5,291

5,434

5,680

NY

51,267

54,530

62,871

66,062

68,130

69,456

72,182

OH

26,411

28,242

31,736

32,860

34,008

35,396

36,511

OK

6,776

7,344

8,423

8,878

9,250

9,501

9,843

OR

8,121

8,774

10,257

11,231

11,904

12,493

13,047

PA

33,298

35,472

40,509

42,951

44,619

45,920

47,569

PR

4,500

4,784

5,719

5,976

5,831

5,909

5,981

RI

3,255

3,440

3,840

3,973

4,136

4,283

4,396

SC

8,205

8,891

10,212

10,690

11,118

11,646

12,185

SD

2,295

2,454

2,804

2,981

3,105

3,221

3,339

TN

14,096

15,022

17,036

17,978

18,633

19,126

20,040

TX

37,491

40,265

45,952

48,799

51,248

53,682

55,986

UT

4,773

5,139

5,901

6,295

6,648

6,859

7,071

VA

14,928

16,068

18,644

19,957

20,756

21,460

22,263

VT

2,166

2,270

2,636

2,764

2,842

2,919

2,974

WA

15,968

17,205

19,925

21,098

22,250

23,003

23,830

WI

15,697

16,787

18,659

19,456

19,981

20,711

21,523

WV

4,240

4,527

5,188

5,431

5,584

5,812

5,902

WY

1,142

1,215

1,464

1,585

1,688

1,785

1,845

Table B.3: Distribution of NHSC Providers by State and Year in the First Analytic Dataset

State

Year

Total

2000

2001

2002

2003

2004

2005

2006

2007

2008

2009

2010

2011

2012

2013

 

Other

0

0

0

0

1

2

2

2

1

1

3

2

1

3

18

Total

440

526

739

1,180

1,508

1,745

1,685

1,629

1,576

1,886

2,883

3,904

3,407

2,460

25,568

AK

3

2

3

7

11

13

9

12

14

23

27

29

27

20

200

AL

12

10

12

18

23

25

27

33

28

23

41

46

33

22

353

AR

4

6

7

15

18

21

22

26

27

21

23

46

45

32

313

AZ

7

17

21

36

51

55

58

51

42

43

73

115

119

100

788

CA

19

23

27

41

59

82

89

91

88

113

161

221

176

120

1,310

CO

16

23

25

37

44

42

42

47

45

56

84

94

63

37

655

CT

4

6

10

13

17

19

19

25

21

15

31

50

45

38

313

DC

1

1

2

5

11

15

12

9

16

17

15

25

24

23

176

DE

2

3

3

3

4

5

4

5

3

2

4

7

6

3

54

FL

18

25

43

60

70

86

91

83

74

71

100

128

117

85

1,051

GA

15

18

21

31

45

47

43

37

29

34

58

76

70

55

579

HI

1

1

1

8

9

9

8

13

15

14

19

24

20

19

161

IA

12

8

10

15

21

28

27

18

14

21

36

59

61

40

370

ID

7

10

14

19

34

37

33

35

37

63

87

112

97

76

661

IL

20

22

25

48

57

56

54

63

54

70

120

157

126

100

972

IN

7

8

10

12

17

24

26

25

32

35

40

60

48

25

369

KS

9

14

26

27

31

28

28

29

28

36

54

74

70

52

506

KY

10

10

12

20

22

26

29

32

31

43

66

84

56

29

470

LA

1

3

5

7

11

15

19

21

27

33

38

54

53

39

326

MA

11

13

16

27

30

42

44

41

51

65

102

133

103

61

739

MD

1

3

10

16

24

29

30

29

31

38

41

65

65

54

436

ME

9

8

10

13

22

32

22

20

22

31

51

52

40

34

366

MI

21

22

29

47

60

56

44

49

54

67

128

173

135

92

977

MN

10

10

20

30

36

48

53

40

40

56

85

115

108

87

738

MO

18

27

45

65

88

97

84

79

82

87

125

191

194

149

1,331

MS

7

5

9

19

22

18

11

10

11

20

37

60

48

37

314

MT

5

8

14

21

19

27

21

19

32

37

57

90

79

55

484

NC

16

18

20

35

43

56

48

39

34

36

57

93

93

69

657

ND

6

8

8

12

12

12

14

12

17

22

29

42

38

27

259

NE

4

3

3

6

4

5

3

7

7

13

20

29

21

12

137

NH

0

1

3

8

8

4

6

5

3

5

14

17

13

9

96

NJ

3

4

7

10

9

15

16

16

14

11

15

22

19

16

177

NM

5

7

13

21

23

27

30

30

38

40

52

70

63

47

466

NV

3

4

4

7

9

13

18

13

8

8

13

16

19

13

148

NY

31

38

58

72

77

89

83

79

66

75

142

223

205

144

1,382

OH

11

6

6

11

20

26

28

33

29

28

52

66

55

37

408

OK

3

4

6

9

15

15

11

14

13

20

24

37

43

33

247

OR

8

9

11

29

42

52

44

36

27

44

70

85

80

63

600

PA

15

15

16

26

39

42

43

36

37

39

67

84

65

51

575

PR

1

2

2

0

1

2

1

1

1

3

4

4

6

4

32

RI

1

2

2

6

10

12

19

15

17

18

22

26

23

11

184

SC

10

14

20

34

35

37

24

32

33

42

46

54

54

45

480

SD

6

7

10

14

15

11

11

9

7

18

21

23

21

16

189

TN

5

5

15

29

44

59

64

64

58

64

99

112

81

54

753

TX

11

15

22

44

72

92

83

59

48

52

93

124

107

73

895

UT

17

17

14

34

35

34

31

37

38

47

51

53

44

34

486

VA

7

5

6

14

18

28

28

28

22

26

49

66

54

33

384

VI

0

0

1

1

0

0

0

0

0

0

0

0

5

5

12

VT

1

1

3

4

3

8

10

8

8

6

8

18

15

7

100

WA

10

16

28

47

57

54

47

39

36

58

121

163

136

88

900

WI

4

9

12

20

24

21

23

30

39

46

62

74

70

51

485

WV

2

3

6

8

16

24

27

24

13

20

33

41

31

21

269

WY

10

7

13

19

20

23

22

19

14

10

13

20

17

10

217

Table B.4: Retention Rates of NHSC Participants as of December 2013 – Primary Care (First Analytic Dataset)

Year of Exit from NHSC

HPSA &
same county

HPSA & other county

Non-HPSA & same county

Non-HPSA & other county

Total NHSC providers in Medicare data

Total

1,905

1,746

493

851

4,995

38.1%

35.0%

9.9%

17.0%

 

2000

23

37

12

28

100

23.0%

37.0%

12.0%

28.0%

2001

25

35

11

33

104

24.0%

33.7%

10.6%

31.7%

2002

26

56

9

28

119

21.8%

47.1%

7.6%

23.5%

2003

58

66

19

34

177

32.8%

37.3%

10.7%

19.2%

2004

116

117

35

70

338

34.3%

34.6%

10.4%

20.7%

2005

143

152

35

76

406

35.2%

37.4%

8.6%

18.7%

2006

166

138

47

85

436

38.1%

31.7%

10.8%

19.5%

2007

136

128

38

71

373

36.5%

34.3%

10.2%

19.0%

2008

143

126

36

68

373

38.3%

33.8%

9.7%

18.2%

2009

128

111

37

59

335

38.2%

33.1%

11.0%

17.6%

2010

160

130

44

62

396

40.4%

32.8%

11.1%

15.7%

2011

338

304

85

133

860

39.3%

35.3%

9.9%

15.5%

2012

443

346

85

104

978

45.3%

35.4%

8.7%

10.6%

Table B.5: Retention Rates of NHSC Participants as of December 2013 – Mental Health (First Analytic Dataset)

Year of Exit from NHSC

HPSA &
same county

HPSA & other county

Non-HPSA & same county

Non-HPSA & other county

Total NHSC providers in Medicare data

Total

568

485

86

162

1,301

43.7%

37.3%

6.6%

12.5%

 

2000

11

8

1

1

21

52.4%

38.1%

4.8%

4.8%

2001

13

11

2

3

29

44.8%

37.9%

6.9%

10.3%

2002

12

9

3

4

28

42.9%

32.1%

10.7%

14.3%

2003

16

12

3

10

41

39.0%

29.3%

7.3%

24.4%

2004

36

32

5

13

86

41.9%

37.2%

5.8%

15.1%

2005

34

35

7

16

92

37.0%

38.0%

7.6%

17.4%

2006

53

35

4

12

104

51.0%

33.7%

3.8%

11.5%

2007

41

36

5

22

104

39.4%

34.6%

4.8%

21.2%

2008

34

41

6

12

93

36.6%

44.1%

6.5%

12.9%

2009

39

33

11

9

92

42.4%

35.9%

12.0%

9.8%

2010

45

35

7

9

96

46.9%

36.5%

7.3%

9.4%

2011

106

88

17

15

226

46.9%

38.9%

7.5%

6.6%

2012

128

110

15

36

289

44.3%

38.1%

5.2%

12.5%

Table B.6: Retention Rates of NHSC Participants by Provider Type as of December 2013 — Primary Care (Second Analytic Dataset)

Year of Exit from NHSC

HPSA &
same county

HPSA &
other county

Non-HPSA & same county

Non-HPSA & other county

Total Matched in P360

Total

983

775

287

436

2,481

39.6%

31.2%

11.6%

17.6%

Physicians

2000

11

18

11

15

55

20.0%

32.7%

20.0%

27.3%

2001

24

22

6

20

72

33.3%

30.6%

8.3%

27.8%

2002

14

25

7

15

61

23.0%

41.0%

11.5%

24.6%

2003

32

27

11

15

85

37.6%

31.8%

12.9%

17.6%

2004

55

49

20

30

154

35.7%

31.8%

13.0%

19.5%

2005

78

77

26

37

218

35.8%

35.3%

11.9%

17.0%

2006

94

62

23

49

228

41.2%

27.2%

10.1%

21.5%

2007

83

66

28

38

215

38.6%

30.7%

13.0%

17.7%

2008

89

63

23

44

219

40.6%

28.8%

10.5%

20.1%

2009

73

63

21

34

191

38.2%

33.0%

11.0%

17.8%

2010

90

64

18

41

213

42.3%

30.0%

8.5%

19.2%

2011

148

118

45

47

358

41.3%

33.0%

12.6%

13.1%

2012

192

121

48

51

412

46.6%

29.4%

11.7%

12.4%

Year of Exit from NHSC

HPSA &
same county

HPSA &
other county

Non-HPSA & same county

Non-HPSA & other county

Total Matched in P360

Total

1,271

1,267

285

518

3,341

38.0%

37.9%

8.5%

15.5%

NP/PA's

2000

21

25

7

18

71

29.6%

35.2%

9.9%

25.4%

2001

6

20

6

21

53

11.3%

37.7%

11.3%

39.6%

2002

17

31

3

14

65

26.2%

47.7%

4.6%

21.5%

2003

33

47

7

22

109

30.3%

43.1%

6.4%

20.2%

2004

65

84

24

42

215

30.2%

39.1%

11.2%

19.5%

2005

85

97

15

48

245

34.7%

39.6%

6.1%

19.6%

2006

93

89

25

42

249

37.3%

35.7%

10.0%

16.9%

2007

75

73

14

31

193

38.9%

37.8%

7.3%

16.1%

2008

69

68

12

32

181

38.1%

37.6%

6.6%

17.7%

2009

69

63

13

26

171

40.4%

36.8%

7.6%

15.2%

2010

107

90

25

38

260

41.2%

34.6%

9.6%

14.6%

2011

272

249

70

93

684

39.8%

36.4%

10.2%

13.6%

2012

359

331

64

91

845

42.5%

39.2%

7.6%

10.8%

Year of Exit from NHSC

HPSA &
same county

HPSA &
other county

Non-HPSA & same county

Non-HPSA & other county

Total Matched in P360

Total

415

391

103

201

1,110

37.4%

35.2%

9.3%

18.1%

 

Other Providers

2000

3

11

5

6

25

12.0%

44.0%

20.0%

24.0%

2001

4

6

3

2

15

26.7%

40.0%

20.0%

13.3%

2002

5

9

1

6

21

23.8%

42.9%

4.8%

28.6%

2003

14

14

5

6

39

35.9%

35.9%

12.8%

15.4%

2004

29

25

6

21

81

35.8%

30.9%

7.4%

25.9%

2005

33

31

4

18

86

38.4%

36.0%

4.7%

20.9%

2006

29

37

9

26

101

28.7%

36.6%

8.9%

25.7%

2007

26

31

7

25

89

29.2%

34.8%

7.9%

28.1%

2008

30

31

7

13

81

37.0%

38.3%

8.6%

16.0%

2009

42

32

11

14

99

42.4%

32.3%

11.1%

14.1%

2010

32

41

13

11

97

33.0%

42.3%

13.4%

11.3%

2011

79

51

13

32

175

45.1%

29.1%

7.4%

18.3%

2012

89

72

19

21

201

44.3%

35.8%

9.5%

10.4%

Table B.7: Retention Rates of NHSC Participants by Provider Type as of December 2013 — Mental Health (Second Analytic Dataset)

Year of Exit from NHSC

HPSA and same county

HPSA and other county

Non-HPSA and same county

Non-HPSA and other county

Total Matched in P360

Total

71

75

25

46

217

32.7%

34.6%

11.5%

21.2%

Physicians

2000

2

3

1

1

7

28.6%

42.9%

14.3%

14.3%

2001

4

1

2

2

9

44.4%

11.1%

22.2%

22.2%

2002

2

6

2

0

10

20.0%

60.0%

20.0%

0.0%

2003

4

2

3

4

13

30.8%

15.4%

23.1%

30.8%

2004

7

7

0

3

17

41.2%

41.2%

0.0%

17.6%

2005

1

6

2

3

12

8.3%

50.0%

16.7%

25.0%

2006

5

5

0

1

11

45.5%

45.5%

0.0%

9.1%

2007

4

5

2

6

17

23.5%

29.4%

11.8%

35.3%

2008

4

3

1

3

11

36.4%

27.3%

9.1%

27.3%

2009

4

6

3

4

17

23.5%

35.3%

17.6%

23.5%

2010

3

7

0

3

13

23.1%

53.8%

0.0%

23.1%

2011

13

13

5

6

37

35.1%

35.1%

13.5%

16.2%

2012

18

11

4

10

43

41.9%

25.6%

9.3%

23.3%

Year of Exit from NHSC

HPSA and same county

HPSA and other county

Non-HPSA and same county

Non-HPSA and other county

Total Matched in P360

Total

63

65

2

18

148

42.6%

43.9%

1.4%

12.2%

NP/PA's

2000

2001

2002

0

1

0

0

1

0.0%

100.0%

0.0%

0.0%

2003

2

2

0

1

5

40.0%

40.0%

0.0%

20.0%

2004

1

2

0

1

4

25.0%

50.0%

0.0%

25.0%

2005

3

2

0

0

5

60.0%

40.0%

0.0%

0.0%

2006

3

2

0

1

6

50.0%

33.3%

0.0%

16.7%

2007

2

3

0

1

6

33.3%

50.0%

0.0%

16.7%

2008

0

7

1

3

11

0.0%

63.6%

9.1%

27.3%

2009

3

7

0

0

10

30.0%

70.0%

0.0%

0.0%

2010

6

3

0

2

11

54.5%

27.3%

0.0%

18.2%

2011

22

13

0

1

36

61.1%

36.1%

0.0%

2.8%

2012

21

23

1

8

53

39.6%

43.4%

1.9%

15.1%

Year of Exit from NHSC

HPSA and same county

HPSA and other county

Non-HPSA and same county

Non-HPSA and other county

Total Matched in P360

Total

1,239

909

255

355

2,758

44.9%

33.0%

9.2%

12.9%

 

Other Providers

2000

14

12

2

3

31

45.2%

38.7%

6.5%

9.7%

2001

15

12

0

6

33

45.5%

36.4%

0.0%

18.2%

2002

17

13

1

9

40

42.5%

32.5%

2.5%

22.5%

2003

35

26

2

16

79

44.3%

32.9%

2.5%

20.3%

2004

60

51

14

24

149

40.3%

34.2%

9.4%

16.1%

2005

65

50

16

22

153

42.5%

32.7%

10.5%

14.4%

2006

99

60

11

26

196

50.5%

30.6%

5.6%

13.3%

2007

77

71

14

28

190

40.5%

37.4%

7.4%

14.7%

2008

68

66

16

19

169

40.2%

39.1%

9.5%

11.2%

2009

74

49

21

21

165

44.8%

29.7%

12.7%

12.7%

2010

102

79

26

22

229

44.5%

34.5%

11.4%

9.6%

2011

254

169

66

69

558

45.5%

30.3%

11.8%

12.4%

2012

359

251

66

90

766

46.9%

32.8%

8.6%

11.7%

Table B.8: Regression Models with Separate Estimates for Nhsc Sp and Lrp

 

1 Year

2 Years

3 Years

4 Years

5 Years

6 Years

> 6 Years

NOTE: The coefficients in the above table are obtained by estimating the same regression models as the models presented in Table VII.1-VII.4, but including only physicians. *, ** and ***: significant at 10%, 5% and 1%.

Primary Care Same HPSA

NHSC LRP

-2.017***

-1.327**

-1.122***

-1.087***

-0.887***

-0.895***

-0.810***

(0.114)

(0.093)

(0.096)

(0.105)

(0.112)

(0.145)

(0.123)

NHSC SP

-2.586***

-1.826***

-1.531***

-1.160***

-1.114***

-1.553***

-0.364

(0.144)

(0.149)

(0.189)

(0.180)

(0.272)

(0.338)

(0.383)

Observations

161,625

159,176

149,771

140,318

127,850

122,190

128,176

Primary Care – Any HPSA

NHSC LRP

-1.276***

-0.827***

-0.621***

-0.553***

-0.423***

-0.299**

-0.227*

(0.127)

(0.096)

(0.104)

(0.108)

(0.117)

(0.145)

(0.128)

NHSC SP

-1.756***

-1.027***

-0.967***

-0.646***

-0.365***

-0.830***

0.125***

(0.199)

(0.172)

(0.185)

(0.203)

(0.267)

(0.281)

(0.476)

Observations

161,625

159,176

149,771

140,318

127,850

122,190

128,176

Table B.9: Multinomial Logit Estimates — Primary Care

Variable

1 Year

3 Years

5 Years

Other Cty HPSA

Same Cty Non-HPSA

Other Cty Non-HPSA

Other Cty HPSA

Same Cty Non-HPSA

Other Cty Non-HPSA

Other Cty HPSA

Same Cty Non-HPSA

Other Cty Non-HPSA

Observations

195,189

178,925

149,059

NHSC Participant

2.402

2.347

1.718

1.408

1.342

1.049

1.164

0.992

0.871

(0.088)

(0.146)

(0.113)

(0.086)

(0.126)

(0.103)

(0.095)

(0.150)

(0.116)

Female

-0.063

0.196

-0.056

-0.009

0.164

0.022

0.015

0.156

0.069

(0.030)

(0.051)

(0.032)

(0.021)

(0.030)

(0.027)

(0.022)

(0.033)

(0.028)

Age 36 to 45

-0.278

-0.035

-0.344

-0.297

-0.095

-0.362

-0.303

-0.003

-0.390

(0.032)

(0.060)

(0.039)

(0.030)

(0.042)

(0.031)

(0.043)

(0.055)

(0.044)

Age 46 to 55

-0.540

-0.200

-0.808

-0.708

-0.246

-0.861

-0.776

-0.179

-0.996

(0.040)

(0.076)

(0.054)

(0.036)

(0.054)

(0.046)

(0.048)

(0.061)

(0.055)

Age 56 to 65

-0.695

-0.387

-1.055

-0.860

-0.397

-1.227

-0.969

-0.352

-1.302

(0.052)

(0.104)

(0.066)

(0.047)

(0.066)

(0.055)

(0.054)

(0.065)

(0.062)

Age Over 65

-0.990

-0.441

-1.497

-1.214

-0.581

-1.537

-1.350

-0.454

-1.644

(0.087)

(0.116)

(0.133)

(0.077)

(0.088)

(0.099)

(0.075)

(0.085)

(0.093)

Medical Doctor

0.474

-0.252

0.100

0.221

0.485

0.133

0.214

0.161

-0.023

(0.243)

(0.339)

(0.343)

(0.169)

(0.288)

(0.224)

(0.165)

(0.201)

(0.233)

Nurse Practitioner

0.226

-0.203

-0.084

-0.051

0.466

-0.074

-0.011

0.259

-0.211

(0.249)

(0.347)

(0.355)

(0.176)

(0.283)

(0.226)

(0.170)

(0.250)

(0.240)

Physician Assistant

0.430

-0.170

0.161

0.237

0.519

0.045

0.258

0.157

-0.190

(0.250)

(0.365)

(0.357)

(0.174)

(0.285)

(0.233)

(0.179)

(0.229)

(0.252)

Start Year 2005

-0.367

-0.109

-0.462

-0.903

-0.927

-0.944

-0.822

-0.710

-0.847

(0.046)

(0.098)

(0.063)

(0.049)

(0.083)

(0.051)

(0.042)

(0.071)

(0.046)

Start Year 2006

-0.101

-0.076

-0.264

-0.464

-0.781

-0.463

-0.385

-0.517

-0.365

(0.053)

(0.121)

(0.070)

(0.054)

(0.088)

(0.055)

(0.047)

(0.071)

(0.050)

Start Year 2007

0.118

0.320

-0.048

-0.415

-0.835

-0.483

(0.054)

(0.098)

(0.063)

(0.051)

(0.099)

(0.058)

Start Year 2008

0.088

0.000

-0.025

-0.272

-0.834

-0.286

(0.054)

(0.116)

(0.065)

(0.053)

(0.111)

(0.059)

Middle Atlantic

0.349

-0.129

0.208

-0.058

-0.341

-0.086

-0.030

-0.160

-0.153

(0.116)

(0.380)

(0.415)

(0.116)

(0.434)

(0.434)

(0.127)

(0.469)

(0.447)

East North Central

0.653

0.657

0.432

0.205

0.044

0.051

0.170

-0.021

0.024

(0.139)

(0.286)

(0.276)

(0.141)

(0.344)

(0.321)

(0.151)

(0.343)

(0.320)

West North Central

0.660

-0.316

0.149

0.202

-0.699

-0.203

0.222

-0.531

-0.004

(0.109)

(0.342)

(0.309)

(0.117)

(0.439)

(0.338)

(0.134)

(0.449)

(0.351)

South Atlantic

0.760

-0.324

0.230

0.420

-0.821

-0.076

0.362

-0.748

-0.069

(0.111)

(0.282)

(0.280)

(0.118)

(0.332)

(0.315)

(0.125)

(0.323)

(0.316)

East South Central

0.983

0.605

0.623

0.502

-0.129

0.255

0.423

-0.245

0.151

(0.137)

(0.492)

(0.325)

(0.143)

(0.486)

(0.356)

(0.148)

(0.471)

(0.359)

West South Central

0.971

0.391

0.723

0.455

-0.197

0.177

0.448

-0.221

0.101

(0.155)

(0.428)

(0.394)

(0.154)

(0.565)

(0.442)

(0.166)

(0.553)

(0.450)

Mountain

0.725

-0.112

-0.853

0.417

-0.475

-1.227

0.437

-0.387

-1.189

(0.128)

(0.438)

(0.298)

(0.166)

(0.461)

(0.323)

(0.184)

(0.458)

(0.330)

Pacific

0.603

0.065

-0.064

0.264

-0.377

-0.251

0.288

-0.374

-0.265

(0.123)

(0.303)

(0.294)

(0.133)

(0.384)

(0.349)

(0.144)

(0.370)

(0.350)

Log Family Income

-0.893

0.609

0.417

-0.959

0.937

0.450

-0.966

0.959

0.640

(0.105)

(0.374)

(0.397)

(0.119)

(0.383)

(0.409)

(0.139)

(0.374)

(0.410)

Poverty Rate

-0.019

-0.090

-0.069

-0.019

-0.072

-0.063

-0.021

-0.072

-0.052

(0.004)

(0.017)

(0.015)

(0.004)

(0.020)

(0.014)

(0.005)

(0.019)

(0.014)

Percent White

0.496

-0.566

0.670

0.908

-0.291

0.873

0.966

-0.261

0.839

(0.256)

(0.664)

(0.746)

(0.301)

(0.596)

(0.684)

(0.352)

(0.565)

(0.657)

Percent Black

-0.437

-1.890

-0.285

-0.199

-1.594

-0.660

-0.096

-1.336

-0.783

(0.274)

(0.862)

(0.859)

(0.271)

(0.822)

(0.792)

(0.316)

(0.746)

(0.773)

Pct HS Grads Over 25 Yrs

2.620

3.224

6.499

2.543

4.201

6.341

2.389

3.802

6.650

(0.561)

(1.888)

(1.779)

(0.587)

(1.895)

(1.744)

(0.615)

(1.797)

(1.766)

Pct Population Over 65

1.599

-8.019

-10.336

0.665

-9.437

-10.684

0.844

-9.051

-10.370

(0.781)

(4.194)

(3.159)

(1.159)

(4.405)

(3.076)

(1.443)

(4.279)

(2.994)

Intercept

4.817

-8.954

-8.367

7.690

-11.525

-6.847

8.235

-11.262

-8.625

(1.238)

(4.336)

(4.618)

(1.468)

(4.188)

(4.650)

(1.698)

(4.127)

(4.673)

Table B.10: Multinomial Logit Estimates — Mental Health

Variable

1 Year

3 Years

5 Years

Other Cty HPSA

Same Cty Non-HPSA

Other Cty Non-HPSA

Other Cty HPSA

Same Cty Non-HPSA

Other Cty Non-HPSA

Other Cty HPSA

Same Cty Non-HPSA

Other Cty Non-HPSA

Observations

19,046

16,841

13,123

NHSC Participant

3.110

0.012

2.545

1.789

-0.108

1.349

1.609

-0.316

1.106

(0.209)

(0.226)

(0.260)

(0.203)

(0.232)

(0.243)

(0.239)

(0.293)

(0.275)

Female

0.042

-0.034

0.038

0.015

-0.038

0.076

0.044

-0.055

0.185

(0.089)

(0.046)

(0.103)

(0.070)

(0.051)

(0.073)

(0.069)

(0.059)

(0.073)

Age 36 to 45

-0.080

-0.085

-0.317

-0.237

-0.105

-0.230

-0.032

-0.236

-0.609

(0.132)

(0.090)

(0.162)

(0.146)

(0.128)

(0.144)

(0.271)

(0.238)

(0.215)

Age 46 to 55

-0.206

0.026

-0.348

-0.542

-0.079

-0.609

-0.378

-0.226

-1.040

(0.141)

(0.091)

(0.168)

(0.158)

(0.134)

(0.148)

(0.270)

(0.241)

(0.217)

Age 56 to 65

-0.402

-0.061

-0.622

-0.782

-0.090

-0.836

-0.540

-0.219

-1.381

(0.144)

(0.090)

(0.180)

(0.161)

(0.133)

(0.152)

(0.272)

(0.232)

(0.219)

Age Over 65

-0.127

-0.047

-0.868

-0.764

-0.218

-0.901

-0.671

-0.261

-1.552

(0.180)

(0.121)

(0.256)

(0.183)

(0.142)

(0.179)

(0.271)

(0.237)

(0.243)

Medical Doctor

1.305

-0.159

0.911

1.142

-0.077

1.014

1.085

-0.064

0.960

(0.139)

(0.085)

(0.126)

(0.110)

(0.080)

(0.094)

(0.092)

(0.079)

(0.092)

Nurse Practitioner

0.893

-0.020

0.447

0.862

-0.189

0.519

1.100

-0.089

0.788

(0.222)

(0.127)

(0.244)

(0.171)

(0.123)

(0.165)

(0.145)

(0.145)

(0.166)

Start Year 2005

-0.784

-0.055

-0.487

-1.280

-0.314

-1.331

-1.093

-0.146

-0.878

(0.118)

(0.076)

(0.152)

(0.132)

(0.120)

(0.121)

(0.111)

(0.098)

(0.102)

Start Year 2006

-0.300

-0.053

-0.129

-0.832

-0.293

-0.709

-0.770

-0.023

-0.436

(0.196)

(0.108)

(0.211)

(0.168)

(0.171)

(0.163)

(0.164)

(0.168)

(0.152)

Start Year 2007

-0.212

-0.140

-0.135

-0.701

-0.393

-0.871

(0.153)

(0.086)

(0.171)

(0.147)

(0.132)

(0.157)

Start Year 2008

-0.187

-0.090

-0.253

-0.444

-0.249

-0.612

(0.153)

(0.095)

(0.213)

(0.133)

(0.116)

(0.146)

Middle Atlantic

0.046

-0.549

0.007

-0.215

-0.551

-0.476

-0.099

-0.496

-0.361

(0.309)

(0.537)

(0.498)

(0.253)

(0.422)

(0.482)

(0.207)

(0.402)

(0.517)

East North Central

0.466

0.629

0.568

-0.025

0.427

-0.091

0.168

0.396

0.090

(0.288)

(0.434)

(0.426)

(0.248)

(0.350)

(0.415)

(0.224)

(0.341)

(0.431)

West North Central

0.272

0.361

0.061

0.061

0.237

-0.281

0.489

0.105

-0.184

(0.274)

(0.472)

(0.374)

(0.242)

(0.385)

(0.354)

(0.230)

(0.375)

(0.368)

South Atlantic

0.691

0.652

0.860

0.408

0.503

0.430

0.651

0.376

0.328

(0.288)

(0.475)

(0.402)

(0.236)

(0.391)

(0.356)

(0.208)

(0.379)

(0.371)

East South Central

0.805

1.735

1.025

0.753

1.500

0.640

0.643

1.435

0.789

(0.313)

(0.567)

(0.454)

(0.298)

(0.520)

(0.413)

(0.268)

(0.529)

(0.426)

West South Central

0.317

0.954

1.108

0.087

0.621

0.371

0.057

0.641

0.534

(0.352)

(0.531)

(0.454)

(0.287)

(0.455)

(0.454)

(0.260)

(0.436)

(0.493)

Mountain

0.077

0.443

-1.017

0.042

0.203

-1.650

0.176

0.066

-1.322

(0.281)

(0.480)

(0.505)

(0.259)

(0.406)

(0.454)

(0.251)

(0.409)

(0.466)

Pacific

0.065

0.331

-0.110

-0.053

0.173

-0.687

0.165

0.105

-0.448

(0.333)

(0.544)

(0.408)

(0.255)

(0.441)

(0.416)

(0.217)

(0.425)

(0.401)

Log Family Income

-1.446

0.122

0.153

-1.510

0.120

-0.183

-1.474

0.187

-0.049

(0.261)

(0.374)

(0.512)

(0.233)

(0.331)

(0.464)

(0.271)

(0.308)

(0.535)

Poverty Rate

-0.048

-0.057

-0.055

-0.048

-0.059

-0.087

-0.049

-0.055

-0.083

(0.010)

(0.020)

(0.022)

(0.010)

(0.018)

(0.019)

(0.010)

(0.017)

(0.020)

Percent White

0.934

0.000

1.494

1.018

-0.106

0.374

1.404

-0.157

0.278

(0.540)

(0.851)

(1.009)

(0.510)

(0.840)

(0.748)

(0.560)

(0.797)

(0.716)

Percent Black

-0.249

-2.713

-1.609

-0.071

-2.728

-2.317

0.176

-3.051

-2.480

(0.621)

(1.213)

(1.172)

(0.586)

(1.110)

(0.896)

(0.620)

(1.075)

(0.881)

Pct HS Grads Over 25 Yrs

2.594

4.270

7.381

2.426

4.237

4.736

1.999

3.852

5.399

(1.398)

(2.470)

(2.225)

(1.363)

(2.172)

(2.074)

(1.254)

(2.211)

(2.164)

Pct Population Over 65

2.670

-2.922

-10.948

2.749

-4.081

-7.304

2.659

-4.862

-6.747

(2.091)

(3.719)

(4.129)

(1.987)

(3.531)

(3.570)

(1.855)

(3.532)

(3.650)

Intercept

10.926

-3.091

-7.088

13.817

-2.230

1.226

13.348

-2.592

-0.163

(3.149)

(4.656)

(6.071)

(2.927)

(4.183)

(5.506)

(3.270)

(3.944)

(6.380)

Appendix C: Detailed Discussion of the Economic Model of Location Decisions

This appendix provides a more detailed discussion of the economic model of location decisions that was sketched out in Chapter IV.

General Model of Location Decisions

An economic model of geographic location choices can be adapted from dynamic programming models of occupational choice (Keane and Wolpin, 1994) and military retention decisions (Asch and Warner, 2001; Asch et al., 2008).  The adaptation simply requires substituting “geographic location” for “occupation” or “sector of the economy”.  In the dynamic programming approach to the location decision in a given time period, an individual will calculate the value (utility) of each possible location and choose the location offering the highest value (utility).  The value of each location has three components.  The first component is the value that the individual places on the non-pecuniary factors associated with living in the location (climate, environment, local amenities, etc.).  This non-pecuniary value is assumed to be time-invariant and is denoted by the symbol θ.  The second component accounts for the pecuniary value of the location and has two parts: (1) the individual’s current period wage in the location (wt) and (2) the discounted value of expected future utility including wages if the individual chooses the location in period.27  The third component consists of a completely random period-specific location shock that is unrelated to the individual’s preference for the location.  Denoted , this shock accounts for the net impact of unobservable factors that might induce an individual to choose a location he or she dislikes or leave a location he or she likes.  Any number of factors might have period-specific (i.e., temporary) effects, including birth of a child, an illness, and death of a parent who was living elsewhere.

More formally, the value (utility) of location j in period t  is given by

(1)

where (1)  is the non-pecuniary preference for location j, (2)  represents the wage available at location j in period t, p is the personal discount rate on future utility, (3)  is the expected value of utility as of period t of the optimum location choice in period t + 1 and (4)  represents the random shock associated with the choice of location j in period t.  Furthermore,  is written as:

(2)

 is based on the individual choosing the period t+1 location that provides the maximum value in that period.  Intuitively, this calculation depends on the size of the random shock to the value of each location.  The larger the random shocks are, the more likely it is that the individual will find another location in period t+1 that has a value larger than the current one and the more weight will be placed on the values of other locations.  It has been shown (Ben-Akiva and Lerman, 1985)) that when the random shock  follows an extreme value distributed with standard deviation of , the expected value of the choice of location j in period t + 1,  equals

(3)

Equation (3) indicates that expected future utility varies positively with the standard deviation of random shocks, and it further shows that expected future utility depends on future utilities at all locations and not just the current one.28

The individual will choose location j, or will remain in location j, if   for all  .  Given the extreme value distribution for the random shocks in each period, the probability of choosing location j in period t is a logistic function of expected values (Ben-Akiva and Lerman, 1985):

(4)

If the individual is already in location j, equation (4) expresses the probability of remaining in the location.  If located elsewhere, equation (4) expresses the probability of moving to the location. 



27 The calculation of this expected value is discussed below.  Because the individual has the option of relocating at each period in the future, the expected present value of future wages does not equal the discounted value of future wages in the current location. 

28 In equation (3) the parameter , known as Euler’s constant, is approximately equal to 0.5776.

Individual Decisions with Two Locations

To simplify the general location choice model for the purpose of analyzing the NHSC, consider the case of only two locations.  With only two locations, an individual chooses location 1 if  .  This implies

             (5)

A further assumption significantly simplifies the condition for choice of location, namely that  is independent of location in period t.  This assumption is reasonable if job skills are perfectly transferable and the experience gained in one location is just as valuable in other locations as in the current location.  To a first approximation, this assumption is plausible in the case of health care providers.29  With this assumption, equation (4) can be rearranged as:

             (6)

The individual thus chooses location (1) if the location 1 preference differential  plus the location 1 wage differential  exceeds the (location 2) random shock differential, .  Equation (6) implies that individuals may choose location 1 in the face of a negative wage differential if their preference differential is high enough; conversely, individuals with a negative net preference for location 1 may choose it if they receive a wage premium for working there.

In the case of two locations, and with our simplifying assumptions, the probability of choosing location 1 reduces to

          (7)

Equation (7) makes it clear that probability of choosing location 1 increases with the net preference for location 1, , and the wage differential between location 1 and location 2, .  Notice that, for any given wage differential, there is a preference differential that makes individuals indifferent between location 1 and location 2 (in an expected value sense).  According to equation (7), an individual whose preference differential is equal to  will have a 50% chance of choosing either location.


29 If the future wage path is dependent on the period t location choice, the choice differential in equation (6) would need to include a term for the difference in expected future earnings due to period t choice.  For simplicity, we assume that the  on both sides of equation (4) is the same.

Cohort Rates with Two Locations

Suppose that a cohort of individuals enters the labor market after schooling and we wish to know what fractions of this cohort will make different choices over time.  Again assume there are two locations and two time periods.  The individuals in this cohort can choose location 1 in both periods, location 2 in both periods, location 1 in the first period and location 2 in the second period, and location 2 in the first period and location 2 in the second period.  Define six expected fractions as follows:

  • 1= fraction choosing location 1 in period 1 =
  • 2 = fraction choosing location 2 in period 1 =
  • 3 = fraction choosing location 1 in period 1 and in period 2 =
  • 4 = fraction choosing location 2 in period 1 and in period 2 =
  • 5 = fraction choosing location 1 in period 1 and location 2 in period 2 =
  • 6 = fraction choosing location 2 in period 1 and location 1 in period 2 =

 Conceptually, these expected fractions are constructed as weighted averages of the individual probabilities of choosing a given location in each time period, as given by equation (4).  In the case of two locations, the weights are based on the distribution of in the cohort.  To obtain actual expected fractions, it is necessary to make an assumption about the distribution of  in the population.  In simulations presented below, we assume that location preferences are independently normally distributed with means  and , respectively, and common standard deviation .30  With these assumptions, the distribution of is normal with mean  -  and standard deviation equal to

The retention rate in a location is the fraction of individuals who chose to locate there previously who choose to remain there.  Based on the above discussion, the period 2 retention rate in location 1 is given by   and the period 2 retention rate in location 2 is given by .  It is important to recognize that retention rates are conditional on prior choices and will depend on the composition of the cohort making the prior choices.


30 With these assumptions, the distribution of is normal with mean  -  and standard deviation

Two-Location Model with NHSC Program

We now introduce the NHSC program into the two-location model.  Most of the participants in the NHSC program are in the Loan Repayment Program (LRP), so we now focus on this program.  We model individuals’ decisions to apply for the program, NHSC selection of applicants into the program, location choices in period 1, the period in which participants are obligated to serve in a HPSA, and then location choices in period 2, the post-obligation period.

Let location 1 represent a HPSA and location 2 represent a non-HPSA.  An individual without outstanding student debt and who does not qualify for LRP has value functions and probability of choosing location 1 as shown in equation (1) above.  But an individual who has outstanding student debt has the following value function for location 1, where L is loan repayment for which the individual qualifies in the presence of the program:

       (8)

Assume that if an individual has outstanding debt, the individual always prefers to be in the program and receive LRP over not participating and working in location 1.  That is to say, .  This assumption simply requires that L exceed the wage differential between qualifying jobs and non-qualifying jobs.  But if a HPSA is a full location HPSA, this assumption is (almost) surely met because: (1) in a competitive market all employers will be paying the same wage; and (2) because in a full location HPSA all primary care positions qualify for LRP.

Based on this assumption, the condition for an individual with outstanding student loans to want to work in a HPSA and participate in NHSC is given by:

        (9)

The probability of applying for NHSC, conditional on having student debt, is thus given by:

                (10)

This probability is an increasing function of the wage differential  , the LRP amount L, and the individual’s preference differential .

Now consider acceptance into the program, which is conditional on application.  Let there exist an index function , where X represents observable factors that influence acceptance into the program (e.g., HPSA score and academic record) and the random error  measures the net influence of unobservable factors associated with application.  (These are factors unobservable to researchers but are observable by NHSC administrators who are evaluating applications.)  Conceptually, NHSC rank-orders applicants to the program, establishes some minimum value of A for acceptance into the program (Amin) and accepts those candidates for whom A > Amin.

Our concern is the potential relationship between net preferences for the HPSA and acceptance into the program.  A simple model is to allow  to be related to the (standardized) net preference for the program through the following linear equation:

                        (11)

where  is the standardized net preference for the HPSA and  is a standard normal random error that captures the influence of unobservables other than location preferences that affect acceptance into the program and is, by definition, uncorrelated with preferences.  The parameter  governs the correlation between  and ; this correlation, denoted  , increases with the parameter .31

The probability of acceptance into the program is given by:

     (12)

When , acceptance into the program depends on preferences, and the probabilities of application and acceptance are therefore not independent of one another.

Consider a cohort of individuals making their choices in time period 1.  There are four distinct groups in this cohort.  The first group consists of those who apply for NHSC and are accepted.  The second group consists of those who apply but are rejected.  The first two groups, of course, have outstanding student loans.  The third group consists of those who also have outstanding student loans but do not apply.  The last group consists of those who have no outstanding loans and therefore are not eligible to apply.

We wish to know how the retention in the HPSA of NHSC participants upon completion of their program obligations will compare with the retention of these other groups.  The retention rate of a given group is simply a weighted average of the retention probabilities of the individuals in the group, and the weights are based on the distribution of in the group.  In general, the distribution of  at the end of period t will be non-symmetric, and its mean and variance will both depend on how individuals in the population sorted themselves into the different groups at the start of the period.  To make this clear, remember that  and  were assumed to be normally (and thus symmetrically) distributed in the population, which implies that  is normally distributed.  But individuals with higher values of  are attracted to location 1, and individuals with higher values are attracted to apply for NHSC if they have outstanding loans.  Different groups will have different mean values of  after the period t location choices occur, and the distribution of  will vary depending upon factors such as wages in the two locations, the LRP amount, and the NHSC’s minimum standard for acceptance into the NHSC program.  Although it is difficult to derive exact analytical answers to the question of how HPSA retention will vary by group, we can make some generally (if not universally) valid statements based on an analysis of how the mean value of  varies from group to group at the end of period t.

We can now conceptually compare the retention of NHSC participants with the retention of (1) individuals who have student loans but do not apply, (2) NHSC rejects who work in the HPSA, and (3) individuals who do not have student loans but work in the HPSA.

By the assumption that , anyone with an outstanding student loan who is working in a HPSA must be an NHSC applicant.  Thus, anyone who has an outstanding student loan but did not apply must be working in the non-HPSA.  This group has no retention in the HPSA because none of them were located in the HPSA to begin with.

Now compare NHSC participants with rejected applicants.  Rejected applicants will still choose to locate in the HPSA if

      (13)

But because of the above assumption that, for individuals with outstanding student loans, the value of participation is always higher than the value of non-participation (i.e., - ), it follows that, for given values of the other variables in equation (13), the preference differential that renders an NHSC reject willing to locate in the HPSA must be larger than the differential that will make an NHSC participant locate in the HPSA.  It is therefore unambiguous that NHSC rejects who nevertheless locate in the HPSA will have higher average preferences for the location than the participants in the program.  Other things the same, they will have higher retention.  Their higher retention is, of course, due to the fact that NHSC rejects with low preferences select themselves out of the location by choosing the non-HPSA location after program rejection.

Now compare the participants with the individuals who never applied to the program but who work in the HPSA.  By assumption, this group is comprised completely of individuals without outstanding student loans.  Absent any systematic relationship between wage offers, location preferences, and outstanding student debt, it must also be true that non-participants who locate in a HPSA will tend to have higher average preferences for the location than NHSC participants and consequently higher retention.  We say tend to have because the retention differential between NHSC participants and non-participants without student loans depends on the strength of the relationship between NHSC applicants’ net preferences for the HPSA and acceptance into the program (i.e., the correlation coefficient  ).  The retention differential will narrow the larger is this correlation.


31 We may show that under the assumption that the variables on the right-hand side of equation (11) have standard deviations of 1, .  This correlation converges to 1 as  increases in value.

Model Refinements

In the course of developing and simulating our theoretical model, we made the following simplifying assumptions:

  1. Perfectly transferable human capital
  2. Perfectly competitive labor markets (no wage variation within markets)
  3. No moving costs

We now discuss the implication of relaxing these assumptions. The assumption that the human capital stock is perfectly transferable from one location to another implies that the amount of medical knowledge and experience a provider accumulates in a HPSA (in the NHSC program or not) is fully utilizable in another location once the provider moves out of the HPSA.   This further implies that the return (in the form of wages) to the current job is independent of total previous job experience.  However, it is likely that while serving in a HPSA the provider accumulates certain skills and expertise (such as working with underserved populations, a better understanding of the needs of these populations etc.) that cannot be put to use in, say, an urban setting with affluent patients.  If there is no perfect overlap between the skills and expertise the provider has after serving in a HPSA and the skills and expertise needed in a non-HPSA environment, then, all else constant, the mobility of providers out of HPSA is diminished, because the life-cycle income of providers with HPSA experience is lower than that of providers without HPSA experience.  Nonetheless, in the context of the NHSC LRP program, the period of time spent by providers in HPSAs is in general short and therefore, the specific human capital accumulated in HPSAs should not have an important impact on the participants’ remaining career, and consequently on their probability to move out of HPSAs after completion of initial service.

We also assumed that the wage offers providers get in HPSAs are the same; by the same token, we assumed that the wages providers get in non-HPSAs are the same as well.  In other words, the only wage variation in the model comes from the wage differential between HPSA and non-HPSA wages.  In our simulations, we further simplified this and assumed the wages were the same in both HPSAs and non-HPSAs.  A larger variation in the wages offered in HPSAs and a larger variation in the wages offered in non-HPSAs would imply a higher mobility of all providers, as they are looking for their best combination between wages and preferences.  This variation in wages across locations may be the result of different labor market structures.  These location-specific market forces may generate different wage rates in two different locations for the same set of skills and experience.  For instance, urban areas have a set of amenities that are preferred by large numbers of providers; all else constant, in order to enjoy these amenities, providers will accept lower wages in urban areas than in rural areas.

Finally, moving costs reduce the value of another location relative to the current location and thus reduce the frequency of moves from one location to another.  Consideration of moving costs should not affect the conclusions we have drawn about the conditions under which the NHSC will increase person-years in underserved areas.

References

Asch, B., Hosek, J., Mattock, M. and Panis C., 2008.  Assessing Compensation Reform: Research in Support of the 10th Quadrennial Review of Military Compensation, Santa Monica CA: RAND Corporation.

Artiga S., Arguello R, and Duckett P., 2013. Health Coverage and Care for American Indians and Alaska Natives. Kaiser Family Foundation. Available from: http://kff.org/disparities-policy/issue-brief/health-coverage-and-care-for-american-indians-and-alaska-natives

Association of American Medical Colleges, 2014. Funding Updates. Available from: https://www.aamc.org/advocacy/washhigh/highlights2014/366504/011714congressapprovesfy2014omnibusfundingbill.html

Ben-Akiva, M. and Lerman, S., 1985. Discrete Choice Analysis: Theory and Application to Travel Demand.  Cambridge, MA: MIT Press.

Beran, R.L., Lawson, G.E., 1998. Medical student financial assistance, 1996–1997. Journal of the American Medical Association, 280, pp.819–820.

Bland, C.J., Meurer, L.N. and Maldonado, G., 1995. Determinants of primary care specialty choice: a non-statistical meta-analysis of the literature. Academic Medicine, 70(7), pp. 620-641.

Brodsky K.L., 2005. Best practices in specialty provider recruitment and retention: challenges and solutions. The Commonwealth Fund. Available at: http://www.commonwealthfund.org/~/media/Files/Publications/Fund%20Report/2005/Aug/Best%20Practices%20in%20Specialty%20Provider%20Recruitment%20and%20Retention%20%20Challenges%20and%20Solutions/852_Brodsky_best_practices_specialty_provider_recr%20pdf.pdf

Brooks, R. G., Walsh, M., Mardon, R.E., Lewis, M. and Clawson, A., 2002. The role of nature and nurture in the recruitment and retention of primary care physicians in rural areas: a review of the literature. Academic Medicine, 77(8), pp. 793.

Council on Graduate Medical Education, 2004. State and managed care support for graduate medical education: innovations and implications for federal policy. U.S. Department of Health and Human Services: Health Resources and Services Administration. Available from: http://www.hrsa.gov/advisorycommittees/bhpradvisory/cogme/Publications/managedcarerpt.pdf

Department of Health and Human Services. (2014). Fiscal Year 2015 – Budget in Brief. Strengthening Health and Opportunity for all Americans. Available from: http://www.hhs.gov/budget/fy2015/fy-2015-budget-in-brief.pdf.

Duttera M.J. and Blumenthal D.S, 2000. Improving recruitment and retention of medical scholarship recipients in rural Georgia. J Health Care Poor Underserved. 11:135-143.

Duane, M., Dovey, S. M., Klein, L. S. and Green, L. A., 2004. Follow-up on family practice residents’ perspectives on length and content of training.  The Journal of the American Board of Family Medicine, 17, pp. 377-383.

Edgar E.P., 2009. Physician retention in the army medical department. U.S. Army War College.

Fordyce M.A., Doescher M.P., Chen F.M., and Hart L.G., 2012. Osteopathic Physicians and International Medical Graduates in the Rural Primary Care Physician Workfroce. Family Medicine. 4(6).

Gray B.M. and Grefer J.E., 2012. Career earnings and retention of U.S. military physicians. Defense and Peace Economics. 23(1).

Health Professions Resource Center for Health Statistics at the Texas Department of State Health Services, 2006. Recruitment and retention of health care providers in underserved communities in Texas. Available from: http://www.dshs.state.tx.us/chs/hprc/hb916rep.pdf

Health Resources and Services Administration, 2011(a). Executive summary for the 2011 HRSA congressional justification. Washington, DC: Department of Health and Human Services. Available at: http://www.hrsa.gov/about/pdf/executivesummary2011.pdf

Health Resources and Services Administration, 2011(b). Negotiated Rulemaking Committee on the Designation of Medically Underserved Populations and Health Professional Shortage Areas: Final Report to the Secretary. Available at: http://www.hrsa.gov/advisorycommittees/shortage/nrmcfinalreport.pdf

Health Resources and Services Administration, 2013(a). HPSA glossary. Washington, DC: Department of Health and Human Services. Available at: http://bhpr.hrsa.gov/shortage/hpsas/hpsaglossary.html

Health Resources and Services Administration, 2013(b). National Health Services Corps application process. Available at: http://nhsc.hrsa.gov/loanrepayment/applicationprocess/index.html#checkeligibility

Health Resources and Services Administration, 2013(c). Primary Medical Care HPSA Designation Overview. Available at:  http://bhpr.hrsa.gov/shortage/hpsas/designationcriteria/primarycarehpsaoverview.html

Health Resources and Services Administration, 2013(d). Guidelines for Primary Medical Care/Dental HPSA Designation. Available at: http://bhpr.hrsa.gov/shortage/hpsas/designationcriteria/medicaldentalhpsaguidelines.html

Health Resources and Services Administration, 2013(e). Guidelines for Mental Health HPSA Designation. Available at http://bhpr.hrsa.gov/shortage/hpsas/designationcriteria/mentalhealthhpsaguidelines.html

Health Resources and Services Administration, 2013(f). Shortage Designation: Health Professional Shortage Areas & Medically Underserved Areas/Populations. Available at: http://www.hrsa.gov/shortage/

Health Resources and Services Administration, 2013(g). Medically Underserved Areas/Populations, Guidelines for MUA and MUP Designation. Available at: http://www.hrsa.gov/shortage/mua/index.html

Health Resources and Services Administration, 2014. Justification for Estimates for Appropriation Committees. Available at http://www.hrsa.gov/about/budget/budgetjustification2014.pdf

Henderson M., Hunt, D. and Williams, J., 1996. General internists influence students to choose primary care careers: the power of role modeling. American Journal of Medicine, 101(6), pp. 648-653.

Holmes G.M., 2004. Does the National Health Service Corps improve physician supply in underserved locations? Eastern Economic Journal, 30(4), pp. 563-581.

Holmes G. M., 2005. Increasing physician supply in medically underserved areas. Labour Economics, 12, pp. 697-725.

Hooker, R.S., 2009.  Are physician assistants in America’s best interest?  Journal of the American Academy of Physician Assistants, 22(9), pp. 12.

Hosek J. and Martorell F., 2009. How have deployments during the war on terrorism affected reenlistment? Santa Monica CA: RAND Corporation.

Hotz, V. J. and Miller, R. A., 1988. An empirical analysis of life cycle fertility and female labor supplyEconometrica, 56(1), pp. 91-118.

Illinois Primary Health Care Association, 2013. Health professional shortage area (HPSA) guidelines. Available at: http://www.iphca.org/Services/DataResearch/Designations/HPSAGuidelines.aspx

Indyk D., Deen D., Fornari A., Santos M.T., Lu W., Rucker L., 2011. The influence of longitudinal mentoring on medical student selection of primary care residencies. BMC Medical Education, 11(27). Available from: http://www.biomedcentral.com/1472-6920/11/27

Keane M. P., and Wolpin, K. I., 1994. “The Solution and Estimation of Discrete Choice Dynamic Programming Models by Simulation and Interpolation: Monte Carlo Evidence”, Review of Economics and Statistics, 76(4), pp.648-72.

Keating E.G., Brauner M.K., Galway L.A., Mele J.D., Burks J.J. and Saloner B., 2009. Air Force physician and dentist multiyear special pay: current status and potential reforms. Santa Monica, CA: RAND Corporation Project Air Force.

King H. and Speckart C., 2006. Ten evidence-based practices for successful physician retention. Kaiser Permanente: the Permanente Journal. Available at: http://xnet.kp.org/permanentejournal/sum02/retention.html

Konrad T.R., Leysieffer K., Stevens C., 2000. “Evaluation of the Effectiveness of the National Health Service Corps,” Department of the Health and Human Services. Available at: http://mathematica-mpr.com/publications/pdfs/health/national_health_service_corps_fnlrpt.pdf

Leichter H.M., 1992.  Health Policy Reform in America. Armonk, NY: ME Sharpe Inc.

Lynch, D.C., Newton, D.A., Grayson, M.S. and Whitley, T.W., 1998. Influence of medical school on medical students' opinions about primary care practice. Academic Medicine, 73(4), pp. 433-435.

Magrid D., Calonge N., Lyons E., Bodily M. and Price D, 2001. The effect of continuity of care when physicians terminate employment with CPMG: impact on member disenrollment, utilization and quality. Presentation to the Care Experience Council.  Denver (CO): Colorado Permanente Medical Group, Kaiser Foundation Health Plan.

Massachusetts Medical Society, 2012. 2012 MMS Physician Workforce Study. Available at: http://www.massmed.org/workforce

Mountain States Group, 2012. Organizational Retention Survey Results. National Health Service Corps Clinician Retention Project. Available at: http://www.mtnstatesgroup.org/wp-content/uploads/2012/08/NHSC-SITE-SURVEY-RESULTS.pdf

Mundell B.F., 2010. Retention of military physicians: the differential effects of practice opportunities across the three services. Santa Monica CA: RAND Corporation. Available at: http://www.rand.org/content/dam/rand/pubs/rgs_dissertations/2010/RAND_RGSD275.pdf

National Conference of State Legislatures (NCSL), 2011. Distribution of Primary Care Physicians. Available at: http://www.ncsl.org/research/health/health-status-of-the-uninsured.aspx

National Health Service Corps, Undated.  Report to the Congress for Years 1990-1994. Rockville, Md: National Health Service Corps.

National Health Service Corps, 1995. Opportunities to stretch scarce dollars and improve provider placement (GAO/HEHS-96-28). Washington, DC: US General Accounting Office.

National Health Service Corps, 2000. A National Health Service Corps for the 21st Century. Available at: http://nhsc.hrsa.gov/downloads/servicecorps.pdf

National Health Service Corps, 2012. NHSC Clinician retention: a story of dedication and commitment. Available at: http://nhsc.hrsa.gov/currentmembers/membersites/retainproviders/retentionbrief.pdf

National Health Service Corps, 2013. National Health Service Corps Site Reference Guide. Rockville, Maryland: US Department of Health and Human Services. September 2013. Available at: https://nhsc.hrsa.gov/downloads/sitereference.pdf

National Health Service Corps, 2014. National Health Service Corps Loan Repayment Program. Available at: https://nhsc.hrsa.gov/downloads/lrpataglance.pdf

National Health Service Corps, 2014. National Health Service Corps Scholarship Program School Year 2014-2015 Application and Program Guidance. March 2014. Available at: < https://nhsc.hrsa.gov/downloads/spapplicationguide.pdf

Nicholson, S., 2002. Physician specialty choice under uncertainty. Journal of Labor Economics 20(4): 816-847.

Pathman, D.E., Konrad, T.R. and Ricketts,T.C., 1992. The comparative retention of National Health Service Corps and other rural physicians: results of a nine year follow-up study. Journal of the American Medical Association, 268, pp. 1552-1558.

Pathman, D.E., Taylor D.H., Konrad T.R., King T.S., Harris T., Henderson T.M., Bernstein J.D., Tucker T., Crook K.D., Spaulding C. and Koch G.G., 2000. State scholarship, loan forgiveness, and related programs: the unheralded safety net. JAMA, 284(16):2084-92.

Pathman D.E., Konrad T.R., King T.S., Taylor D.H. and Koch G.G., 2004(a). Outcomes of states' scholarship, loan repayment, and related programs for physicians. Med Care. 2004 Jun; 42(6):560-8.

Pathman D.E., Konrad T.R., Dann R. and Koch, G., 2004(b). Retention of primary care physicians in rural health professional shortage areas. Am J Public Health, 94(10): 1723-1729.

Pathman, D.E., Fryer, G.E., Phillips, R.L., Smucny, J., Miyoshi, T., and Green, L.A., 2006. National Health Service Corps staffing and the growth of the local rural non-NHSC primary care physician workforce. The Journal of Rural Health, 22, pp.285-293.

Pathman D.E. and Konrad T.R., 2012. Growth and changes in the National Health Service Corps (NHSC) workforce with the American Recovery and Reinvestment Act. J Am Board Fam Med. 25(5):723-33.

Pathman, D.E., Konrad, T.R. and Schwartz, R., 2012(a). Evaluating retention in BCRS programs. Cecil G. Sheps Center for Health Services Research at the University of North Carolina at Chapel Hill.

Pathman D.E., Fannell J., Konrad T.R., Pierson S., Tobin M. and Jonsson M., 2012(b). Findings of the First Year Retention Survey of the Multi-State/NHSC Retention Collaborative. Multi-State/NHSC Retention Collaborative. Available at:  http://www.oshpd.ca.gov/hwdd/pdfs/REAReports.pdf

Phillips, R.L., et al., Robert Graham Center, 2009.  Specialty and geographic distribution of the physician workforce: what influences medical student and resident choices? Available at: http://www.grahamcenter.org/online/etc/medialib/graham/documents/publications/mongraphs-books/2009/rgcmo-specialty-geographic.Par.0001.File.tmp/Specialty-geography-compressed.pdf

Rabinowitz H.K., 1988. Evaluation of a selective medical school admissions policy to increase the number of family physicians in rural and underserved areas. N Engl J Med; 319:480-6.

Rabinowitz H.K., Diamond J.J., Markham F.W. and Paynter N.P., 2001.  Critical Factors for designing programs to increase the supply and retention of rural primary care physicians. JAMA; 286:1041-8.

Rabinowitz H.K., Diamond J.J., Markham F.W., Rabinowitz C., 2005. Long-term retention of graduates from a program to increase the supply of rural family physicians. Acad Med; 80:728-32.

Rabinowitz H.K., Diamond J.J., Markham F.W. and Wortman J.R.  Medical school programs to increase the rural physician supply: a systematic review and projected impact of widespread replication. Acad Med. 83(3):235-43.

Rabinowitz, H.K., Diamond, J.J., Markham, F.W. and Santana, A. J., 2011. Increasing the supply of women physicians in rural Areas: outcomes of a medical school rural program. The Journal of the American Board of Family Medicine, 24(6), pp. 740-744.

Rabinowitz H.K., Diamond J.J., Markham F.W. and Santana A.J, 2013. Retention of rural family physicians after 20-25 years: outcomes of a comprehensive medical school rural program. J Am Board Fam Med. 26(1):24-7.

Ricketts T.C., Holmes G.M., Randolph R.M., Lee R., Taylor D.H. and Ostermann J., 2007. Designating places and populations as medically underserved: a proposal for a new approach. Journal of Health Care for the Poor and Underserved, 18(2007):578.

Rittenhouse D.R., Fryer G.E., Phillips R.L., Miyoshi T., Nielsen C., Goodman D.C. and Grumbach K., 2008. Impact of Title VII training programs on community health center staffing and National Health Service Corps participation. Annals of Family Medicine, 6(5), pp.397-405.

Rochester General Health System, 2013. Community School Loan Forgiveness Program. Available from: http://www.rochestergeneral.org/healthcare-professionals/physician-services/loan-forgiveness-program/

Rosenblatt R.A., Saunders G., Shreffler J., Pirani M.J., Larson E.H. and Hart L.G., 1996. Beyond retention: National Health Service Corps participation and subsequent practice locations of a cohort of rural family physicians. The Journal of the American Board of Family Medicine, 9, pp.23-30.

Rosenthal, M.P., Diamond, J.J., Rabinowitz, H.K., Bauer, L.C., Jones, R.L., Kearl, G.W., Kelly, R.B., Sheets, K.J., Jaffe, A., Jonas, A.P, et al., 1994. Influence of income, hours worked, and loan repayment on medical students' decision to pursue a primary care career. Journal of the American Medical Association, 271(12), pp. 914-917.

Ruddy, G., Phillips, R.L., Klein, L.S., McCann, J.L., Dodoo, M.S. and Green, L.A., 2005. Osteopathic physicians and the family medicine workforce. American Family Physician, 72(4), pp. 583.

Singer, J.D., Davidson, S.M., Graham, S. and Davidson, H.S., 1998. Physician retention in community and migrant health centers. Who stays and for how long? Medical Care, 36:1198-1213.

Salinsky, E. , 2010. Health care shortage designations: HPSA, MUA, and TBD. National Health Policy Forum. Available at  http://www.nhpf.org/library/details.cfm/2789

South Dakota Planning and Development Districts for the South Dakota Department of Health, Office of Rural Health, 2012. South Dakota Department of Health Office of Rural Health Recruitment and Retention Survey. Available from: http://doh.sd.gov/providers/assets/Recruitment-Retention-Survey-2012.pdf

U.S. General Accounting Office (GAO), 1995. Health care shortage areas: designations not a useful tool for directing resources to the underserved. (GAO/HEHS-95-200). Available from: http://www.gao.gov/archive/1995/he95200.pdf

U.S. General Accounting Office (GAO), 1995. National Health Service Corps: Opportunities to Stretch Scarce Dollars and Improve Provider Placement. (GAO/HEHS-96-28). Available from: http://www.gao.gov/assets/230/221979.pdf

U.S. General Accounting Office (GAO), Briefing for Congressional Committees, 2009. Military personnel: status of accession, retention, and end strength for military medical officers, and preliminary observations regarding accession and retention challenges. (GAO-09-469R). Available from: http://www.gao.gov/assets/100/96066.pdf

Verby, J.E., Newell, J.P., Andresen, S.A. and Swentko, W.M., 1991. Changing the medical school curriculum to improve patient access to primary care. The Journal of the American Medical Association; 266, pp.110-113.

Weissert C.S., Knott J.H. and Stieber B.E., 1994. Education and the health professions: explaining policy choices among the states. J Health Polit Policy Law. 19:361-392.

Warner, J. T. and B. Asch. 2001. A theory of compensation and personnel policy in hierarchical organizations with application to the United States military”. Journal of Labor Economics, 19 (3), 523-62.