June 1, 2001
The session coordinator was Mairéad Reidy of Chapin Hall who opened by explaining that the development of school readiness indicators has been a very important component of this project and that many attendees have done much work and made significant advances in this area. She indicated that there is very strong support for developing school readiness indicators that not only tell about the readiness of the children themselves but also get at the status of the family, the community, and the early childhood factors and supports that influence the child's readiness.
Reidy introduced the guest speaker, John Love, a Senior Fellow at the Mathematica Policy Research Institute, who has worked for the last 30 years conducting research program evaluations, policy studies with the early care, and education family programs. He is currently co-directing the National Evaluation of the Early Head Start Program for the Administration on Children, Youth, and Families. What follows is a transcript of Love's remarks. The slides from his Powerpoint presentation follow the text. After Love finished speaking, five other people spoke. They were Elizabeth Burke Bryant and Catherine Walsh of Rhode Island, Rebecca Hudgins of Georgia, Steve Heasley of West Virginia, and Debra McLaughlin of Massachusetts.
Readiness Indicators in Early Childhood Initiatives: The Ideal, the Practical, the Essential
About eleven years ago or so when the governors established this first national goal, we were filled with idealism about what this would mean for our work and for children and for society. But how do we go from those ideals to what is really practical in the work that we are doing? Over the years and seeing all the work that has been accomplished and what certain people consider to be practical, I think that we need to think about another step and that is go back and think about what is really essential. If we compare what is ideal and what we had been able to do so far, what are the elements of both that we should try to really do?
I have three themes in my remarks this morning and I want to go back to the goal as a starting point. And then look at some recent early childhood data collections, early child initiatives, the theme of this session, and particularly focusing on some program evaluations and what they might have to say about our work on indicators of readiness. And then conclude it with a discussion of what we might consider some of the essential elements of our readiness work.
What ever happened to Goal One?
We all can recite it by heart. "By the Year 2000, all children in America will start school ready to learn." Fourteen words carefully crafted by the fifty governors, probably the most succinct the governors have ever been in their lives. And it has inspired an enormous amount of work, and reflection and controversy. Just consider the amount of work that is represented in the three days of this meeting--work that is going on all over the country. But there is still a lot of controversy. And part of the controversy is, what does learning mean? Is it schoolwork? Is it reading? Different people have different focuses on what learning means. Part of the controversy, of course, is on what readiness means, controversy about what does it mean to start school--are we talking about kindergarten? Or is it first grade, or, in some cases, preschool? Are we really concerned about all children in America? Are there concerns for certain groups of children? Fortunately, there is no debate about the first phrase--the year 2000 has come and gone. And we don't know the answer because we are still grappling with all the issues around this phrase. I would like to consider what some early childhood initiatives can contribute to the work that we are all doing.
What Is The Relevance Of These Early Childhood Initiatives?
What I am thinking about in this framework of early childhood initiatives are large-scale data collections, large-scale surveys like the Early Childhood Longitudinal Study (ECLS) kindergarten cohort, which now has data from kindergarten and first grade. I am thinking about the ECLS birth cohort, which is just beginning data collection this year which will start and follow kids from birth into school. I am thinking about the longitudinal study of children with special needs that the Department of Education is doing. I am thinking about the Head Start Family and Child Experiences Survey (FACES) project, which has a nationally representative sample of Head Start programs and has descriptive data about the programs and about the children's development. I am thinking about the Head Start Impact Study that is just getting underway that we will look at--impact of the regular Head Start program for the first time on a nationally representative sample. I am thinking about the Early Head Start study that I have been associated with for the last 5 or 6 years.
These projects show us research that has been considered practical in terms of program evaluations and in terms of the dimensions of learning and development. They show the conceptualizing and operationalizing of readiness. They focus on the relevance of the schools because, for the most part, they are either school-based studies, like ECLS-K (kindergarten cohort), or they are studies that are concerned about children who are going into school, something like Head Start or Early Head Start. I think they provide some information about how and to what extent they have been successful in providing information about all children who are in these programs. And, in general, they provide a lot of experience on dealing with these issues that is useful for us.
The Role of Program Evaluation
I think the program evaluations are important because they have a special process. Often they go into greater depth in defining readiness, they often are seeing greater and greater periods of change. They certainly spend a lot of time trying to conceptualize and measure what they consider to be important about children's learning and development. Because they are evaluations, they exemplify information about process and the outcomes and ways of linking the two. And also because in evaluations they use experimental designs, they provide some causal information that will allow us to understand what might be really contributing to the outcomes of the children, the readiness indicators.
Typical Definitions of Readiness
Readiness for what? What are we getting the children ready for? We think about learning, school success, ready for reading, but I think the evaluations provide a different perspective because you can think about readiness as an outcome of the experience that children have from birth to school age. I would particularly emphasize the importance from birth to school age period, and including the prenatal period. All those experiences are important to prepare the child to be successful in school.
Lessons from Recent Evaluations
I am going to illustrate some of those points from the Early Head Start Evaluation. Here is a picture of an Early Head Start Center, but programs also exist in home-based approaches and some programs have a mixed approach, which includes center-based and home-based which meets different types of family needs. There is a summary report that is now available. It came out in January 2001. The full technical report has been sent to Congress, and it is now on the web. It is really interesting to think about different ways of doing research, but the Early Head Start Evaluation was set up with the national evaluation where you typically see a national evaluation doing data collection all over the country. At the same time they funded 15 local universities to do research at the local level with the programs that were participating in the national evaluation. So also in this report there you will see little snippets of the work that some of the local researchers have been doing. There is also some information from the programs directly so we can show that this work is being done by a whole consortium of people.
Martha Moorehouse: I think if you just read the summary report and not the full report you will miss some important findings. There are things that are in that technical report that are not in the interim report at all. Some of the things that are covered in the longer report are more on father's experiences with these programs and a lot more detail than we traditionally see in an early childhood program about what it means to work with the fathers. This study was implemented just as welfare reform took hold in 1996. And the study addresses what it means to try to serve families with very young children with these new affectations around welfare reform.
John: Because the study was an experimental design with a randomized control group, not only can you look at what difference the program made, and make those causal inferences about whether it was really the program that made the difference, but you can just look at the control group data and say what is sort of typical for infants and toddlers in low income families in this country. You can look at the demographics and judge for yourself whether these are typical low-income families, and you see such things as they get a lot of health services, and not much of anything else. Early Head Start doesn't have much of an impact on the few health outcomes that we looked at, because the program group wasn't getting much more than the control group was getting. But in other areas we see a lot of impact, but I will get to that.
We called this report Building a Future because we think these findings are doing just that or at least promise a different future. We think the main message is that we get a broad range of impacts, we refer to them as modest, some people would call them small even, but there are a lot of them and they cover all the dimension of children's development and learning. And they cover parenting and they cover the home environment and in aggregate it gives you a sense that these kids' lives could be quite different.
Programs Make a Difference
In addition to that overall message, there are some other nuances that seem particularly important when thinking about what is it about children's experiences that makes a difference, that helps establish whether or not they are ready to succeed in school. All of the program approaches that I talked about do have positive effects, they do so in different ways. So the center-based programs you'll see have their major impact on children's cognitive development. The home-based and the mixed approach programs don't have any impact on cognitive development as we measured it, but do on social-emotional development and on the parenting and home environment, whereas the center-based programs don't affect those things as much. And so the kinds of strategies that programs adopt can make a difference on the kinds of effects they have on the children and their development and their opportunities, and what future you might be building.
Standards and Quality Matter
The Head Start Performance Standards for 25 years now have been developed and refined and so forth to reflect the wisdom of practice and research on what should make a good, comprehensive, and high quality program. This is a study of the process or implementation of programs, and when the Early Head Start programs implemented those standards more completely and earlier in their development, then they had larger impacts on the children and families. So it does make a difference to apply those standards. In the technical report, you can go through tables that illustrate this point. So it does make a difference if you have standards and if you hold people to it, if you do monitoring, and if you find ways of assuring that those standards are being met.
Flexibility To Meet Family Needs Is Important
The third point is one of flexibility which comes from the fact that these mixed approach programs that have both home-based and center-based services in different mixes, have generally a stronger pattern of impact than the other two. And we don't really know why yet but our hypothesis is that those were the needs of the programs and that is why they grew up. They found it necessary to add other components to meet the needs of the families.
What is Ideal? What is Practical? What is Essential?
It seems there are ten aspects of our work--definitions, dimensions, community supports, indicators, assessments, measures, strategies, interpretations, involving stakeholders, and ages of children. I think the ideal age now is 0-5, and what is practical is preschool through five years old and essential is birth through age 5.
The program approaches (discussed above) show the importance of these kinds of supports . . . quality programs like this for infants and toddlers can make a difference. [Slides 12 & 13 show] some of the dimensions that I thought might take our discussion a little from the ideal to what is really important and I suspect it will differ across the states. I don't know that it is necessary to have a national goal that declares what is essential but it would be interesting to aggregate all that you do over the next year and see what becomes of some of these considerations.
Thank you. Reports can be found at: www.mathematica-mpr.com, click on the Early Childhood page, then the Early Head Start hyperlink, and there you will find a list of reports.
Elizabeth Burke Bryant and Catherine Walsh
Reidy then introduced Elizabeth Burke Bryant and Catherine Walsh from Rhode Island Kids Count. What follows is a transcription of Bryant's and Walsh's remarks.
Elizabeth Burke Bryant
It is a real pleasure to be here 2 and 1/2 years into this exciting work that we've done as part of the Child Indicators Work representing the Rhode Island Children's Cabinet. About 18 months ago, we held a meeting in Rhode Island where it really blossomed into a national conversation and work session on how do we define, identify, develop and talk about a set of early childhood school readiness indicators that can really resonate in a way that affects public policy. The important work that Ann Segal and Martha Moorehouse spearheaded with this particular grant initiative, what really came home to me that day is that when you have people like John Love at your disposal, having John's work really set the stage 18 months ago, we have been privy through this network of ours to a lot of the readiness work that is going on not just in Rhode Island, but in Vermont, in states like Maine where we are hearing great things happening with their Ready to Learn Agenda. Massachusetts played a key role in that meeting in Providence 18 months ago. And what we have to do now is to look at what's developed since then. What can we all share nationally, and what are some of the next exciting goals to set in moving this work forward?
I always start with the real foundation of our work which you heard a lot about yesterday but the fact that we had a common language that was set forth by our Children's Cabinet directors, which are the five mega-departments that deal with children's issues--the Department of Health, Department of Elementary and Secondary Education, Department of Mental Health, Department of Human Services, and Department of Children, Youth and Families. When we reflect on what has happened over the past two years, what is very exciting is that we worked with the Children's Cabinet, they worked with these goals, everything they've done interdepartmentally in the last 2 and 1/2 years has revolved around these goals and the indicators that have gone with them. Thanks to this Child Indicators project, we have the special technical assistance required to go deeply into school readiness indicators. And just to tell you the end of the story, thanks to a lot of the work we've done here, a lot of the sharing among the states represented here, we are going to go back and prepare for an exciting presentation on June 13 where all of our work that we've done with this project, with our state department agency partners, will be presented in a special 2 1/2 hour meeting with the Children's Cabinet, a meeting just dedicated to how are we doing in terms of ready to learn.
One of the things that we have tried to do as a state partnership of Children's Cabinet agencies is to really ask ourselves, "Why do all this work if it is going to stay on a shelf or in somebody's in box?" We have paid a lot of attention to the strategic dissemination of data and information. Our two ground rules are: 1) to select data that will push a public policy response forward and, 2) to choose your moment. Put the data in the hands of the policy makers when they most need it.
We also work with our state department partners to make sure that everything gets out there with a public engagement strategy and a data dissemination strategy that is really thought through in advance. All of our state department directors and state liaisons we obviously work with as partners all the time. We create panels on early childhood development so there is an academic/public/private laying of hands on some of the work that we are doing together to give it credibility. We have community leadership councils so we have parents and other people at the table to give us their first hand view of what it is like to be a parent of a young child in Rhode Island. We always have policy breakfasts or other events where we invite the media and we have the full range of people that you need, to give something credibility in a community. People always say of our events that it is everyone from our U.S. Senators to parents to people who are working as educators in our schools.
Our General Assembly likes to have hot off the press information, so if you are releasing anything you will want to work with your state departments on a strategy to get it to your General Assembly leaders. We recommend you cut a deal with the people who decide what goes on the desks of the legislators and what doesn't, that is one of those relationships that you want to cultivate. Our legislators don't want a lot of materials all year long, but when we release our annual fact book in April, it is on the desk of every legislator at the very next session.
Targeted mailings go out immediately. We put everything up on our website. We basically work with our state departments at the highest level of the policy and data staff. Levels that really do have strict requirements about the press but by "learning by doing" we figured out as the outside group working very closely with our partners in state department, they have been briefed ahead of time, they know what the messages are and that your directors are ready to comment. We have also cultivated a group of reporters that we call the children's beat reporters. In all cities you will find reporters that are passionate about this and would welcome the courtesy of a heads up phone call saying that you are going to release a report on childhood indicators next week, and can we send you a copy? Those kinds of phone calls bring real relationships that you can cultivate over time.
Why do we care so much about school readiness indicators? It has really been a privilege to work with this group because there is so much exciting work going on with the states. As we reflect on our work, it helps to ground our work to look at what is at stake for young children. And reflecting on a hallway conversation with Ann Segal, she said child indicators mean a lot to her because they are nothing less than the ability to say what do young children in this country need and how will you know if you got there. I think the work that we are doing with our state governments will really define that menu of what do we say as a country that a child absolutely needs and deserves and how do we hold ourselves and our elected officials accountable for knowing that they got what they needed.
Cathy Walsh will go through our buckets of indicators and where we are.
Catherine Walsh, Rhode Island
I am going to pass out the indicators list (see below) and I would like to walk you through it. The categories and things and some of the legends are around data collection. As we have been developing the indicators that are on that list, I think the most critical thing to know is that it says "near final" on the top for a reason. It is really very much of a working list that always gets changed and always gets adapted based on input from various groups. We have talked to kindergarten teachers and early childhood educators about it. The list is a constantly evolving thing. It really helps us stay focused on the child, that what we are talking about is what we want for all of our kids. We want kids to be engaged, curious, confident, and ready to learn. But it's not just about the child; we also want to know about community supports for families.
Rhode Island School Readiness Indicators (Near Final)
Please note, data sources indicators in italics are still pending. Data sources are available for indicators in bold.
Infants born healthy
Percentage of women with late or no prenatal care
Percentage of infants born low birthweight
Percentage of infants born very low birthweight
Infant mortality rate
Young children have healthy growth and development
Percentage of children with developmental screening at age 3
Child injury hospitalization rate for children 0-5
Child asthma hospitalization rate
Young children enrolled in public insurance have healthy growth and development
Percentage of children under 6 on Rite Care screened for lead poisoning
Percentage of children on Rite Care with dental exam by age 5
Percentage of children under 6 on Rite Care with regular, timely well-child visits
Percentage of children under 6 on Rite Care who have up-to-date immunizations
Percentage of children under 6 on Rite Care who have accessed mental health services
Children enter kindergarten healthy
Percentage of children with up-to-date immunizations (at K entry)
Children with a history of lead poisoning (at K entry)
Percentage of children without health insurance (at K entry)
Percentage of children with untreated dental problems (at K entry)
Percentage of children with untreated hearing or vision problems (at K entry)
Percentage of children with undetected disability/developmental problem that requires special education services (at K entry)
Percentage of children with IEPs, pre-k, K, 1-3, 4-6, 7-12
Children live in safe and stable families
Child abuse and neglect rate for children under age 5
Births to teens ages 15 to 17
Number of children under 6 in foster care
Percentage of children under 6 in foster care who are placed in a permanent home
Percentage of children under 6 in DCYF care who had multiple placements
Parents with mental health problems
Parents with substance abuse problems
Children born into families with unstable living situations
Family environments support early learning
Percentage of families with preschool children that read to their preschool child every day
Percentage of families with preschool children that regularly take their children on outings
Percentage of families with preschool children that regulate television viewing/computer use
Percentage of children (pre-K and K-3) with children's books at home and/or read to child
Percentage of children whose parents take them to the library sometimes.
Children have access to early care and education programs
Number of early care and education slots per 100 children ages birth to 3 in need of care
Number of early care and education slots per 100 children ages 3 to 5 in need of care
Percentage of income-eligible families using child care subsidies (i.e., child care subsidy "take-up rate")
Percentage of children enrolled in an early care and education program the year prior to school entrym
Children at high developmental or social risk receive early intervention
Percentage of low-income children in comprehensive child care program/Head Start
Percentage of eligible children enrolled in comprehensive birth to 3 program (i.e., Early Head Start, Early Start
Percentage of Family Independence Program (FIP) enrolled children participating in early care and education program
Percentage of eligible children enrolled in Early Intervention
Percentage of "at-risk" children enrolled in early care and education programs prior to school entry
Early care and education programs are of high quality
Percentage of child care center slots in accredited programs
Percentage of family child care slots in accredited programs
Percentage of child care center staff with early childhood education degree
Percentage of family child care staff with early childhood training
Percentage of early care and education slots in programs without health and safety violations
Percentage of early care and education programs of high quality versus poor quality
Percentage of child care center slots in programs with low staff turnover
Schools are ready for all children
Percentage of children in accredited kindergarten programs
Percentage of children in full-day kindergarten programs
Average daily attendance of kindergarten children
Average class size K-3 classrooms
Percentage of kindergarten (or K-3) teachers with a degree in early childhood education
Kindergarten children have the language and literacy skills needed to succeed in school
Percentage of children in K-3 who are at or above grade level in reading/language arts
Percentage of children who use their primary language appropriately to community needs and wants
Percentage of children who have age-and culturally-appropriate vocabulary
Percentage of children with age-appropriate familiarity and skills with books and print
Percentage of children with age-appropriate letter recognition
Percentage of children with age-appropriate literacy in primary language
Kindergarten children have the knowledge and cognitive skills to succeed in school
Percentage of children with difficulty learning academic subjects
Percentage of children with poor concentration or limited attention
Percentage of children with difficulty following directions
Percentage of children with working independently and being self-directed
Percentage of children K-1 or above grade level in mathematics
Percentage of children with age-appropriate preliteracy skills (numbers, letters, writing, language)
Percentage of children with age-appropriate numerical skills
Percentage of children with age-appropriate reasoning and problem-solving skills
Kindergarten children have the social/emotional competencies to succeed in school
Percentage of children with difficulty working with other students
Percentage of children who are disruptive in class
Percentage of children who constantly seek attention
Percentage of children who are overly aggressive to peers
Percentage of children who are anxious or worried
Percentage of children who are unhappy, depressed, sad
We have things on our list like program indicators because we think that if we really are talking about government and government funding and programs, we have to be talking about the programs that we are investing in. So you will see on our list we have things like Early Head Start and Head Start, early intervention, and special education. Those are there deliberately because we think we need better systems. How are those resources being allocated across communities? And the other piece that guides our work is are we talking about all kids or some kids? We really have to pay attention to the inequities -- that there are differences by communities and access in terms of basic life experience that happens and we have to pay attention to that in terms of support.
Again, we need to emphasize informing policy and it's about information. What this does for me, and as you think of all the data in your state that is potential and all the research data and all the work of Head Start and the work of every study that's been done on welfare reform, what we always come back to is--what are you using it for? And how are you going to use it? And I always point to describing, because very often people try to leap to measuring progress and to improving programs and to monitoring impact, but if you haven't done the work of describing what's up, how many Head Start kids are enrolled in each community in your state, how many are potentially eligible, you can't really do the monitoring impact work very well. And you wouldn't know what to do with it in terms of policy context because you wouldn't know which communities are under-resourced for specific programs and which communities are doing OK. So by describing basic information about what is happening with kids and families in your communities, the work in these other areas of measuring progress, improving programs, and monitoring impact, is going to be much easier and much more powerful in terms of your policy response. Don't minimize it.
Again, as you do your school readiness indicator work, the list will never look exactly like ours because it is the result of a process that involves lots of community input, what are your values, where are your priorities. This list is really meant to be a guide as to some of the categories. And how we decide what makes a good indicator involves much information, it needs to reflect an important child outcome. It has to be something people care about. Something people in your community care about, as opposed to the larger world. The other thing that helped guide the conversation as we walk through some of the indicators, is we are data piranhas. We look for data anywhere we can find it. And we are always looking for more. That mining of existing data is an incredible piece of the work and you need to be aggressive but kind as you pursue the data from various departments. We use a lot of administrative data from state departments. We are talking more about adding to existing surveys. We use lots of different strategies to get what we might want. And then the last thing is if you talk to your neighbor over the fence about an indicator, will they have a clue about what you are talking about.
Again, the two things that needs to guide your work are (1) what do you care about enough to measure and track, and (2) what do policy makers want to know?
The framework for indicator information: We really have framed our school readiness discussion in Rhode Island as a birth-to-fourth grade issue and we intentionally went to fourth grade, even though a lot of the early conversations were about birth to five years because we really thought we had to engage the education system. Many business leaders and many political leaders really care about K-12. The early years are not on the radar screen. So if you start at kindergarten you are really losing that opportunity to engage early childhood educators in the public school systems and to engage that whole school reform standards movement. So we have to do more at an earlier age if we are going to get there. So we have defined it as birth to 4th grade. We sort of double check our indicators list against some other buckets, the readiness of children, child health in all its various ramifications (mental health, hospitalizations, asthma), family factors, access to quality programs, community and neighborhood factors, etc.
Keeping the big frame is really critical to do our work. You need to say where do we start in our state? What might be possible in our state? Keep the big list running, even if it is a placeholder that says family structure, at least it is there and you can fill it in over time. It also stops people from saying you are being narrow minded, or narrowly focused. If you just start with what you can get, you're not expressing the whole picture. And when you go out in public to talk about it, the people who are in the child welfare system don't see themselves. Or the people who are in the schools, don't see themselves. So having a fully developed list even if you don't have all the data is useful.
Let me walk you through a little bit of this. In our state, we always highlight the core cities, the five communities in our state with the highest poverty rates. And we always keep that on the page because when you look at the state numbers for indicators, it can be a wash. And it can look pretty good. But then when you look at the poorest communities in your state, you see really huge differences. And so we do the core cities work to keep that on the page, and it shows major differences. Full-day kindergarten was one of our ready school indicators. We started tracking that about two years ago. And we have seen significant increases around full-day kindergarten access. And then we also mine our hospital discharge data. There is a wealth of data in your hospitals. That is a good entry point to find out what is happening with kids.
If you look at this school readiness indicators list, I am going to focus on a couple of pieces that I think are the most important or the most interesting. The top ones I think everyone has lots of access to. The third category down, enrolled in public insurance-this is a new category that we have just added, because we really think that while health insurance coverage in our states is very high (we have about 92 percent of kids in our state that are covered by health insurance), access and continuity of care is an issue. So we started to look more specifically at that subpopulation group. So within your indicators, trying to balance what you need for the whole population and what you might need to look at more deeply for subpopulations is another question you have to answer for yourselves.
Children enter kindergarten healthy. This is an effort to step back from the whole population and say, when a child walks into the kindergarten door, what is going on. And we haven't been that successful here. You'll see we have some information about lead poisoning, some around immunizations, but we think there is more work that needs to be done. This is an area that we are going to pursue over the next couple of years to really get the school's engaged in seeing that as part of their job.
Walsh concluded by citing some areas in which further work needs to be done and stressing the important connection between school readiness and early childhood initiatives. She introduced Rebecca Hudgins from Georgia and Steve Heasley from West Virginia.
Rebecca Hudgins from Georgia
Hudgins provided a quick summary of Georgia's early childhood initiative pilot project. This three-year, five-site pilot began in October 2000, in north Georgia. It looks at the combination of services around parent education, universal contact, intensive home visiting, adult education and developmental childcare. The way components are put together is different across the sites but each site is using the same set of core measurements or indicators. These measures are divided into population level or community level indicators and system changes indicators. Georgia is keen to bring together new partners, for example, bringing economic development people together with the social services people. We don't talk the same language. But we need to be talking the same language. She concluded by noting that one of the project's end measures, is the state-wide kindergarten assessment program that will allow county- and school-level assessments of what happens in kindergarten.
Steve Heasley, West Virginia
Heasley said that West Virginia has just begun a five-site pilot initiative focused on improving the quality of early childhood education programs, increasing accessibility, and creating better linkages. One part of this initiative addresses how well early childhood programs at the local level can work together to increase quality and accessibility. The West Virginia legislature is requiring an extensive evaluation of the initiative that is "looking both at some quantitative and qualitative measures of what happens to the kids at each sites." Graduate students are gathering data at each of these sites and will conduct case studies.
Debra McLaughlin from Massachusetts
McLaughlin noted that Massachusetts had difficulty developing a common agenda for birth-to-five issues and now that that has been accomplished is building on that effort. She credited Massachusetts Kids Count for the development of issue-oriented indicators and thought that Massachusetts had benefited from good technical assistance from HHS.