Transcript of Systems Building Challenges for Afterschool Initiatives:
Developing Common Measures of Quality Afterschool Investments Webinar
February 19, 2009

Participants

Lori Connors–Tadros, Afterschool Investments Project (AIP)
Roshanda Shoulders, Child Care Bureau
Shawn Stelow, Afterschool Investments Project (AIP)
Nicole Yohalem, Forum for Youth Investment
Emily Morgan, Collaborative for Building After–School Systems (CBASS)
Hillary Salmons, Providence After School Alliance (PASA)
Lucy Friedman, The After–School Corporation (TASC)

Presentation

Operator

Greetings ladies and gentlemen, welcome to today’s Afterschool Investments Webinar entitled Systems Building Challenges for Afterschool Initiatives, Developing Common Measures of Quality. At this time, all participants are in a listen–only mode. Later, we will conduct a question and answer session. I would now like to turn the call over to Dr. Lori Connors–Tadros of the Afterschool Investments Project. Dr. Connors–Tadros, you may begin.

Lori Connors–Tadros – Afterschool Investments Project

Welcome everyone. We are happy to have you on our call. One of the goals of the Afterschool Investments Project is to identify other major programs and sectors that are potential partners for CCDF administrators and supporting out–of–school time and we are very pleased to have representatives of CBASS, which I think you will be really thrilled to learn more about. They provide an excellent model of strategies for coordinating and partnership. The webinar series is one way that we try to bring information to the field and the State Child Care Administrators. We encourage you to go to our website. We recently updated our State and National Profiles and I think you will find them to be quite useful. The call today is focused on system building challenges and in reading and preparing for this Webinar, it was so clear to me that CBASS was born out of a desire to really learn and grow and support the field with a common voice. I think that this is very pertinent and aligned with the goals of CCDF administrators who are looking to support the childcare needs of children from birth through age 12 and beyond and so we believe that this webinar will be of use to State Administrators as they consider models of effective investments in mechanisms to strengthen local and State collaborations. It’s certainly a wonderful example of the power of partnership––How you really use system building efforts to reach all children and really bring services to scale and to sustain them.  I think most importantly, we are going to speak today about a really realistic and a politically powerful but research–based accountability framework that identifies salient outcomes, not only for youth and in all the ways we want to support them but also for programs and at the systems level. So, I am really quite excited to be participating with you in the webinar and I will now turn this over to Roshanda Shoulders at the Child Care Bureau who is our Federal Project Officer.

Roshanda Shoulders – Child Care Bureau

Thank you so much Dr. Connors–Tadros, you introduced me and everything so well so I am going to make it brief. Again, good afternoon and maybe good morning for some. I am Roshanda Shoulders, the Federal Project Officer for the Afterschool Investments Project. And again, the Afterschool Investment Project provides technical assistance to the Child Care and Development Fund, some know it as CCDF, administrators to support programs and development and administration on issues related to afterschool initiatives and CCDF is a multibillion dollar Federal and State partnership administered by my agency, the Child Care Bureau, which falls underneath the department of Health and Human Services. We promote daily economic self–sufficiency and help children succeed in school and lies through affordable high quality, early care and after–school program. I am excited in regards to today’s Webinar and so now, I am going to turn it over to Ms. Shawn Stelow, so she can continue on, but again I thank everyone who is on the phone.

Shawn Stelow – Afterschool Investments Project

Thank you very much Roshanda. I am Shawn Stelow. I am the Afterschool Investments Project Technical Assistance Lead for Regions 1, 2 and 9. It is a pleasure to be here with you today and I am glad you have all joined this Webinar. We really have a very interesting and timely topic to discuss and a terrific group of presenters. I know that you will enjoy all of their presentations and we will definitely find the discussion of high value. Joining us today are Nicole Yohalem from the Forum for Youth Investment, Emily Morgan with the Collaborative for Building Afterschool Systems, otherwise known as CBASS. Hillary Salmons with the Providence After School Alliance and Lucy Friedman from The After–School Corporation.

Just a few housekeeping issues today. You will be in a listen–only mode during the call. However, and you may have seen it scroll across the marquee just a bit ago, you can submit a question at any time by clicking your "Questions" button. We’ll pull and check the questions throughout the presentation and ask some of them during the presentation and hold others until the end. So, please feel free to submit your questions at any time during the conversation or there will be some time again at the end reserved for questions and answers.

Lori did such a good job introducing CBASS and what they have been doing and I know Emily will speak more about their work in a bit and I am going to move into a fuller discussion of Nicole Yohalem and her role in this project. She is a Program Director at the Forum for Youth Investment working to bridge research policy and practice in the out–of–school time field. Nicole plays a key leadership role in the Next Generation Youth Work Coalition working to advance the youth work profession. She is the author of numerous reports and briefs and leads the Forum’s efforts in publishing its Out–of–School Time Policy Commentary series and the Forum Focus. Before Nicole starts to help frame the discussion today, we have a short poll for you and that should be showing on your screen now. We would like to know who is participating on today’s call. We have got a few choices listed. Please click the button in front of the description that best describes your role and then click the vote button and in a bit we will show the results.

One last minute, second to respond and we’ll collect our responses. As you can see, the majority of those joining us today are State or local policy makers and representing intermediary organizations with service providers, directors, managers, a number of researchers. We are really pleased to see some frontline staff that have joined the call as well. So our presenters will keep that mix in mind as they move into their presentation. So Nicole, we’ll let you take it away.

Nicole Yohalem – Forum for Youth Investment

Alright, thank you so much. Oh, there’s my presentation, terrific. So it’s great to be here today. Sorry we can’t see all of your faces. We’re going to try to keep the format engaging because we know that you’re busy people and we are really glad you decided to join us. I am honored to kick this off. The folks that you are going to hear from in a few minutes are, you know the Forum, we do a lot of writing about this and thinking about this and helping, the other speakers are the folks who are out this really hard work every day so I am happy to set it up and you’re in for a real treat. So I am going to, one more thing about the forum, so you know by way of context, we have been working on this issue, for many have been interested in this issue of sort of improving the quality of youth programs for a long time and in the last year we really solidified that interest and commitment by forming a joint venture with the High/Scope Educational Research Foundation to start something called the David P. Weikart Center for Youth Program Quality. So we are actually now even more firmly doing focus on this work around quality improvement and I will mention a little bit, the work of the center will come up a couple of times so I just wanted to give you that in context.

So let’s go to the first slide. I really want to just sort of make the case with you this morning for why it is so important to focus on quality. I know you believe that or you wouldn’t be on this call but hopefully some of these talking points will help you as you locally at the state level and locally have to make this case in what we know is a very difficult economic climate. So the kind of outline of my presentation is my three reasons for focusing on quality. First that it matters and I will talk about evidence for that. Second is that it is measurable. We can really measure it and third is that it is malleable which really means that we can improve it but I wanted something that started with an "M". So those are my three topics and I will introduce a fourth at the very end and we’ll end on that.

So next slide on, just in a really general sense the reason that quality matters so much is that, you know, we are all interested I think in moving this yellow or orange gear, the youth outcome gear, improving youth outcomes is a goal we are all working toward, right? So you really get their scale, we believe that the focus needs to be on that middle gear which is community programs and support. So improving the quality of those community supports is really the intermediate step toward improving youth outcomes, so we are really focused on helping leaders like you all and leaders at lots of levels improve that middle in order to basically see some real improvements in that big youth gear. So, that’s the big picture explanation of why quality matters.

If you go to the next slide, I think the good news is that there really is a basic agreement now that young people need structured activities in their out of school hours and you know, ten years ago that wasn’t the case. So we really have some advocacy perspective. Things have moved over the last decade. There is also now very good evidence that youth programs, youth after school and youth development programs can produce positive changes in outcomes that I think everybody cares about, academic achievement, and social–emotional development. So we have come a long way in that sense. The bad news though, is that you know, if we are really honest with ourselves, the truth is that many programs don’t. Programs can but many don’t. But again, the main point here I think is that for a long time, our advocacy efforts have focused on making sure that we have enough activities for young people and while we still haven’t met that demand fully, we also know now that increasing availability of program slots just isn’t enough.

If you go to the next slide, so the important question here is why do only some programs make a difference and not others? And the short answer to that is – quality matters, which is at the top of the slide but here is the longer answer. When you control for participation, young people who participate in quality programs do better than those who participate in low quality programs and here are two studies that help to illustrate that in more specific ways. First of all, in 2007, Joe Durlak and Roger Weissberg did a meta–analysis, a very rigorous meta–analysis of 73 program evaluations. Many of you probably know this report so I didn’t put a lot of detail here. The programs, they ended up separating, overall let me just say, the 73 programs when they did their meta–analysis showed positive effects on many important youth outcomes. When they looked a little bit closer and they separated out those programs into two clusters. They put programs that had the certain four features that they used the acronym SAFE for. SAFE stood really briefly for Sequenced, Active, Focused and Explicit and there is more detail on that, you can get the report and read it if you wish–– but basically, characteristics of high quality programs. Those programs showed positive effects on almost every outcome they were looking. That’s school performance, social behavior, attitudes and beliefs towards learning and school. And the programs that didn’t have those four features showed no effect on any outcomes. So it turned out that what was really driving the overall positive findings of this meta–analysis was a real subset of high quality programs. And then the other study that helps us illustrate this point is forthcoming from the Center for Youth Program Quality which I mentioned earlier and that’s called the YPQI, Youth Program Quality Intervention Study. That’s a randomized controlled trial which should in about a month, there should be published findings on that but I’m going to give you a few little previews here today and basically they show the same story. Young people who participate in higher quality activities report higher levels of interest challenge, belonging and learning and that has been replicated across lots of samples that the Center has worked with. So we have some pretty strong evidence that these quality features really do matter. Those are a couple of studies that might help you make that case.

If you want to go to the next slide, this is just really sort of a summary point. So, the real take–away, the short way to say what I have just said for a few slides is that programs can improve outcomes by focusing on the quality of services. So, I think sometimes this gets framed as sort of an either/or should we measure outcomes, or should we measure quality or why should we focus all this energy on assessing quality when it is really outcomes we need to be focused on. These are not mutually exclusive things. So we have to focus on quality because it is critical to achieving the outcomes that these programs are focused on, so just to drive that home I put it on a separate slide.

So let’s go to the next slide which is the first of the ‘Quality is Measurable’, the second part of my talk here. Lots of good news here. There is really increased interest among researchers over the last couple of years in better understanding these out of school time settings as an interesting developmental context, so that’s terrific. I think the practice, this is one of those fields where practice has been out ahead of research for a long time and the research community is discovering that these development programs are a cool thing to study and that’s good, it’s going to help us and this first bullet really actually reflects a more general trend in the social sciences. It’s really a move away from this historical emphasis on understanding individual development which psychology really had us focus for a century toward a focus on the actual prevailing environment, programs, families, classrooms and neighborhoods and their effect on these outcomes so it’s a bigger shift that we are seeing in the social class. Very exciting that it is happening within our neck of the woods and so as a result of that shift or that interest growing, there is lots of momentum in the second bullet in terms of developing and really refining these point of service measures and I’ll talk about that in a second, that could help capture data about the specific practices that drive these outcomes. So the point of service is really basically where young people and adults are interacting in programs and doing stuff and the point of service quality that really predicts outcomes. So there are other things about programs, management, facilities, curriculums that are important but it is what unfolds between young people and adults at the point of service that really drives outcomes and what is good about that is that those are things that we have a lot of control over as staff, especially those of you who are calling workers on the phone. So these measures that are now starting to get at the point of service and not just some of the stuff that historically standards have spoken to around authorities and management, this is an exciting trend and as a result of all that, we’ve got some now, really we do have reliable valid tools to measure these actual practices that differentiate between effective and ineffective settings. So that’s all good.

If we go to the next slide, just a quick summary here; these are ten different instruments that are the kinds of things that I’m talking about here. There’s been significant movement in the field within the last year so we decided to update and republish a report, it’s sort of Guide to Quality Assessment Tools. We had done that originally in 2007. There is so much sort of movement going on and work on these tools that we decided to update it and republish it, so those are the ten tools are included. We have added a new instrument, the Coral, which is something that Public–Private Ventures developed. There is new data about the technical properties of many of these tools and there’s new products and services that are being developed, user guides, training, web–based applications, so just a lot of activity over the last year and that’s a report that you can get for free on our website if you want to learn more about any of these specific tools.

But if you go to the next slide, it’s a little bit of what we learned when we looked across all these tools. One big observation was just that there is a lot of similarity across definitions of quality. I think you need to hit return again Shawn, so all of those ten tools that were listed on the last slide, at some level they are measuring these six big constructs and they don’t use this language, they all have their own different language and specific items but relationships, the environment, both the physical environment and the culture of the program, youth engagement and engagement with staff, social norms or climate, opportunities to actually both build and learn and then routine or structure. These are things that all of these tools are measuring. The practitioners and researchers alike are seeing eye to eye on what matters for quality; so that’s terrific. Also by the way, a lot of those instruments also measure other stuff but these are the things that everybody is measuring.

The next slide, just a quick point to say that there are actually more differences, when you start to look at how quality is measured by these various tools, there’s more differences there than in terms of how to define, the loss of consensus around definition, some of the differences across those tools, one is data collection methods. So all of them include some level of observation which we think is really critical to actually observe practice. But some of them include other types of data like interview data or document review, who the target user is, whether it is really targeted toward leadership or frontline staff is another difference. The types of measures themselves, this gets a little bit technical but some of them have low inference versus high inference measures. That’s really about what level of judgment is required to make a rating. Ideally you want a low inference as possible kind of measure and then some are more diagnostic which basically tells you whether or not you’ve got a problem while others are both diagnostic and prescriptive. The tool actually tells you whether you’ve got a problem and embedded in the measure itself is instructions for how to make it better which is really terrific. The rating scale is different you know, the level of detail that’s provided to anchor the different measures is different and then the technical properties vary. So most of those ten instruments that folks have looked at, inter–rater reliability which is basically whether or not different observers who watch the same thing score the instruments similarly, so that’s great and then maybe six of the ten I think have started to look at this question of validity. You know, have made some effort that the instruments actually accurately measuring what it is supposed to measure so lots of big strides on the measurement front in the past year.

If we go to the next slide I am in my last section here, ‘Quality is Malleable’, so the big question here can these aspects of quality be improved and the answer is yes. It’s a good thing too because measurement for the sake of measurement is not something any of us are interested in, in engaging I think, we’re all about change or we wouldn’t be on this call. So these are things, relationships, social norms, these are things that this report that can do better on over time. So that’s exciting and I should say that measuring and improving the staff at the program, at the individual program level is a lot easier to sort of wrap your head around and we all know that sort of that’s doable. Actually improving the staff across a system is much trickier and that’s what CBASS City and the folks you are going to hear from are doing which is what makes it very exciting.

So again let me just give you another preview from the YPQI experimental trial that is yet to be published but probably will be in about a month or so. The information from that one more little tidbit from there the big message from that study is that you can improve staff performance and this again was in an experimental study and so basically the impact when folks had this intervention of assessment using a tool of the YPQI way and making an action plan, getting some training, getting some coaching, there were actually significant positive and statistically significantly changes in staff performance for those folks who were in the experimental group as compared to the control group. But that’s exciting, you’ll learn more about that soon, again you know we don’t have a lot of strong research even in teaching that professional development can make a difference. I think it’s going to be a really important study so the field for that, for that reason.

This slide gives you what we’re learning about the components that are effective in quality improvement systems. Most of these systems that you’re going to hear, you’re going to hear Lucy and Hillary talk a little bit some of this and other systems are building these things in. They are built around a better quality of standards that address, that include what should happen at the point of service, so they get it from the organizational and management safety structure, they also include point of service. They include ongoing assessment of how well the actual services compare to those standards. They include some kind of planning process for how to improve and they include training and technical assistance that really does fit those plans and that training and technical assistance typically in the more promising quality improvement systems include some sort of on–site coaching, it’s not just sending folks off to training workshops but some on–the–job active coaching.

And the next slide, those are the components, some of the lessons learned and these are a little bit related to that last slide. First the data particularly observational data about their own practice, this is a very powerful motivator for staff. The common language that’s embedded in some of these tools really does help pave the way for change when people start talking about the practice in common terms. The third bullet, it’s very important to have standards but to have with that tangible support and capacity building. So it is not okay and not fair to suddenly unleash a set of high quality standards on a community of programs and not put some support in place to help folks achieve those standards. The fourth bullet, managers appear to be critical intervention targets so while it is the practice of front line workers that makes the most powerful difference to kids and programs, management, midlevel management, supervisors, the folks who tend to have full time jobs tend to stay on the job a little bit longer, really changing their perception of their role and giving them the tools to be an onsite coach and to basically do observation and feedback from front line staff as part of their regular job is the really critical strategy here I think in some of these quality improvement systems. And then finally bringing together data about you know, quality is not the only thing we need information about, bringing together data about participation, about quality and about outcomes is the ideal. There are some places that are starting to do this. You can hear a little bit about what Providence is doing around bringing together some of these kinds of data and that is really the cutting edge of work. And I think I have one or two last slides.

Shawn Stelow – Afterschool Investments Project

Nicole?

Nicole Yohalem – Forum for Youth Investment

Yeah.

Shawn Stelow – Afterschool Investments Project

We actually have a question from Mary Staak that sort of relates to that last point and it reads, some studies suggest that children’s time is too structured, limiting their creativity and failing to recognize the value of open–ended free time, what are your thoughts on this issue? She has a follow–up of, is there a measure for child health would account for a childhood and young adult obesity?

Nicole Yohalem – Forum for Youth Investment

The second one I don’t think I have any immediate thoughts on but I can probably get back to you with specifics. But you know I’d be curious about what my colleagues think about this over, there’s two things, I mean there is the sort of programming idea and I do think there is some reality to, I mean there are some young people who are over programmed.  I think they tend to be middle and upper middle class kids whose parents are buying them a lot of lessons and activities, so I think it’s actually not an issue for the vast majority of young people in this country that we’re most concerned about. And then there’s this question of free time and you know what’s interesting is that it looks like in the, you know in the Durlak and Weisberg study I talked about, and in the other work the YPQI study, structure does appear to be a very important aspect of quality creating structured environments with some sort of sequenced learning experience. But I think that doesn’t, you know that structure doesn’t mean you’re being told what to do every minute so I think there’s some nuance and there’s a lot of interpretation and I think if you take a closer look at some of these tools and I can send you some specifics to look at maybe to get a better sense of how folks are as a detailed [INAUDIBLE] defining structure and I think you won’t find it to be completely prescriptive because I think you’re right, I think young people do need more flexibility especially during the out of school hours. But do either of my, any of my colleagues want to say anything before I wrap up here about that question, it’s a great question.

Shawn Stelow – Afterschool Investments Project

Lucy or Hillary, anything to say?

Lucy Friedman – The After–School Corporation

I absolutely agree and I think, I just was visiting a site here last week where there had been some concerns because they had game time, but when we went and visited it was very structured, all the kids were playing one of three games, they were engaged and actually in this case it was a middle school program. The parents had asked to make sure that the kids had some looser time. So and I think, and one of the site coordinators, the site coordinator mentioned that a lot of the kids that are English–language learners participate and that it’s a great opportunity for them to develop their English skills. So I think you can have it very structured and it feels you know not like the kid is being pressured every minute but you know it’s still a stable learning environment and an engaging learning environment.

Hillary Salmons – Providence After School Alliance

My sense is we were in a middle school in Providence that the standards and particularly the High/Scope tool have a youth development focus, I think we use, and if you look, when you get to looking at the standards, when these are engaged and they’re making choices and there’s variety and within those choices there’s room for them to chill and reflect and do some quiet work. I think that youth participating in defining what’s good for them in terms of engagement, this is where we’re setting some standards and having assessment tools and get practitioners to look at, how are you engaging youth and if they are choosing to do some things a little bit more independently or quietly or reflectively it’s a good thing and that’s why I think the quality assessment tool helps practitioners to think through those questions.

Shawn Stelow – Afterschool Investments Project

Thanks Hillary.

Nicole Yohalem – Forum for Youth Investment

This is Nicole, I’ll wrap that up and then finish up my talk. I think, I totally agree and I think, that’s what’s so powerful about these tools that we have not had a very explicit, we haven’t had very explicit guidance for youth development practitioners about what is good youth development practice, it’s been this sort of, you know a lot of principles, I think it’s a principle–driven kind of practice but these tools really articulate what good practice looks like and we needed that so.

All right so to review, those are my first, my three "M’s", quality matters, it’s measurable, we know it’s malleable but in this economic climate I think a question we need to ask ourselves is this quality marketable and I think the answer is yes and if you go to the last slide, what I’ve put here are three messages that I think can help you, they’re helping us make the case with local decision makers that we’re working with and I think can help you all sort of make the case not as investing in these services but investing in the quality of these services.

So first is the economic argument. You’ve got to connect to the economic argument for any kind of policy case you’re trying to build right now. So we’re at a critical moment in our nation’s history, we’ve got to increase the return on our investments and ensure our investments are aligned, efficient and effective. Okay. The second bullet is this plan started out, [INAUDIBLE] these programs have a potential to impact very important outcomes that we all care about whether you’re an employer or a school person or a parent, the reality is that many don’t, so we’ve got to focus on quality. The third bullet is, that’s a message to funders, investing in quality improvement is the way to protect your investments and leverage your investments in direct service. The additional cost of quality assessment and quality improvement is actually quite small relative to the cost of service delivery, relative to the money you’re putting into the delivering services, this is really a small additional cost and it’s essential in helping programs achieve the outcomes they’re aspiring to. So those are my messages I leave you with, I hope they’re helpful and I think I’m done.

Shawn Stelow – Afterschool Investments Project

Great, Nicole. Thank you so much for that terrific presentation and particularly the takeaways at the end about marketing quality because certainly this is a time where every dollar counts and folks are definitely concerned about money going forward. At this point I’m going to pause for one more question that we just got in from Carmen Gatti, apologies if I’m mispronouncing any names and her question is, is there a need for teachers to be engaged in action research to build a case for how to best meet the needs of the youth and what those needs are? And I’ll throw that out to the whole group.

Nicole Yohalem – Forum for Youth Investment

This is Nicole. I think answer is yes, I’m not totally clear whether the person is asking about teachers per se versus sort of folks working in Afterschool settings but I think, I think yes engaging the staff, the folks who are working with young people in thinking about how to best meet the needs of young people is exactly what needs to happen, it’s exactly what can happen when we introduce these kinds of tools and processes that help people be more reflective and you know in the same way we want better data to improve the practice and we know policy makers want better data and information to make decisions on, so do teachers and frontline staff they need information and data to better meet the needs of youth and they can do that with young people, there are places that starting to engage young people themselves in quality assessment so, so I think I’m saying a resounding yes but [INAUDIBLE].

Hillary Salmons – Providence After School Alliance

I think it’s one of the processes which Afterschool communities, when they have a tool from standards that they’re all talking about, one of the ways that they can, they can learn, they can work together to improve themselves is in fact through action research or learning communities where they’re collegially reflecting on how to make those improvements and contributing to each other ways to make that happen. So I think, I feel like sometimes when we talk about learning communities in our sort of technical, you know our way of growing everybody’s work it’s not that different from some for the action research that’s going on in the education community.

Shawn Stelow – Afterschool Investments Project

Great point, Hillary. At this point we’re going transition into some, learning more about the CBASS initiative, the collaboration for building up your school system initiative. I’d like to introduce Emily Morgan, she is the National Policy Coordinator for CBASS and will give us a bit of background and overview on what the CBASS, what CBASS does and more in particular about this initiative and looking at quality measures. So Emily, the floor is yours.

Emily Morgan – CBASS

Great, thanks Shawn and good afternoon. Like Shawn said I’m going to give a brief overview of CBASS and our work around quality measures and then turn it over to Lucy and Hillary who are going to give us some perspectives from this field. So the collaborative for building afterschool systems was formed by six afterschool intermediaries, Boston’s Afterschool Strategy, Baltimore’s Afterschool Strategy, Boston Afterschool and Beyond, Chicago’s Afterschool Matters, DC’s Children and Youth Investment Trust Corporation, Providence After School Alliance and The After–School Corporation in New York. With the goals of changing public policy to promote the development of afterschool systems across the country and actually we recently added two more partners to our group, the Bay Area Partnership in California and Prime Time Palm Beach County.

So we can go to the next slide. And the CBASS partners came together for a couple of key reasons. First they believed that to achieve better outcomes for kids communities needed to create integrated systems of high quality afterschool programming. They also believed that by forming a collaborative they represent a critical mass in the market and thus be better positioned to influence public policy. And third they believed that intermediaries have a critical role to play in building systems by maintaining a flexibility to innovate and the ability to leverage public funds and target resources to communities most in need. And in fact non–profit intermediaries are uniquely positioned to build these types of systems because of their autonomy and their independence from government and ability to withstand political shifts, their ability to leverage public and private dollars, advocate for additional resources and the fact that they are more nimble and flexible to government agencies. So, overall CBASS partners wanted to inform policymakers that afterschool is critical to improving schools and essential to creating better outcomes for disadvantaged kids. And we can go to the next slide here.

CBASS is engaged in three main activities. The partners work to distill lessons learned from past successful systems building effort through presentations and content sharing opportunities. They also work to build the fields through targeted technical systems to emerging intermediaries and they work to shape policies to solve what we consider as the second generation of sheer challenges to bring afterschool systems to the scale while also maintaining program quality. And if you’d like more information about CBASS or you know or past work and current challenges of the field, a link to the CBASS report is on the webinar page of the Afterschool Investments website.

Now we can go to the next slide. So today we’re going to focus on the larger policy issue of identifying and implementing quality measures for afterschool programs. And the CBASS partners identify this as a really important policy initiative for a couple of reasons. First they realize that afterschool programs are being assessed by narrow sighted academic outcomes such as standardized test scores and school grades, the types of outcomes that really don’t measure what afterschool does for youth. And as we all know it’s more than just academics that we need to look at. Secondly they recognize it’s a field that’s highly variable in terms of the evaluation tools being used and the capacity of providers to measure program quality and third they understood that in order to influence policy afterschool really needed to be able to demonstrate positive outcomes for youth. So the partners came together to identify a common set of practical measures that demonstrate quality programming and can be measured across the system.

We can go to the next slide. And as they began looking at different measures of quality they realized that whatever measures they finally selected really needed to first be able to hold programs accountable for quality and inform their continuous improvement. They also needed to measure productivity and successive programs and they needed to assess the value to youth of high quality programming at scale. And the CBASS work on developing common measures of program quality really builds on the work of our partner cities and so to give you some more context about that, Hillary Salmons and Lucy Friedman are going to describe the kinds of work that are happening in Providence and New York and then Lucy is going to describe how CBASS took this and other cities experiences to form a cross city measurement framework and how CBASS is intending to use that framework.

Shawn Stelow – Afterschool Investments Project

We will pass it over to Hillary in just a moment to give Hillary and Lucy and idea of who, and Emily, who is with us from the CBASS jurisdictions, could you please take a moment to answer the poll. So if you are currently in one of the CBASS jurisdictions please select appropriately or none of the above and we’ll give you a minute to vote.

Hillary Salmons – Providence After School Alliance

So if nobody from Providence clicks this button, then I can make stuff up.

Lori Connors–Tadros – Afterschool Investments Project

Well, I’m actually from Rhode Island, this is Lori so vote for me, originally I’m from Providence actually.

Hillary Salmons – Providence After School Alliance

Okay thanks Lori, I just thought if nobody was from Providence I could just make things up and there wouldn’t be anybody to...

Lori Connors–Tadros – Afterschool Investments Project

No, no. Rhode Island has to be represented.

Hillary Salmons – Providence After School Alliance

You’re going to keep me honest.

Lori Connors–Tadros – Afterschool Investments Project

That’s right.

Shawn Stelow – Afterschool Investments Project

I think it looks like there’s a least, there’s a good 30% of folks that are listening in today and participating today that are from a CBASS jurisdiction so that’s great, we’ve definitely got some of, some committed folks that have joined to hear more about your initiative. And just again, by way of introduction, Hillary Salmons is the Executive Director of the Providence After School Alliance which was founded in 2004 to expand and improve afterschool opportunities in Providence by expanding support for out–of–school time programs. Go ahead, Hillary.

Hillary Salmons – Providence After School Alliance

Thank you so much Shawn, I see we’ve got one Providence listener; she may be my colleague in the other room, so I’m pretty safe. The hardest thing about giving these presentations is I use my hands and arms a lot, flail them around, you can’t see me so I’ll try and make this exciting. Providence, I’m just going to give you a little background for all of you, is a city of 180,000. We run obviously between Boston and New York, an old manufacturing community and we have many famous colleges, Brown, RISD and one has an image of a sort of idyllic New England community but we actually have the third highest child poverty rate (nationally) for a city of our size, the second largest Hispanic population in the country and right now we’ve got the highest unemployment rate, I think second to Michigan and one of the nation’s largest foreclosure problems. So we have struggling schools and a pretty dire economic situation.

So with that backdrop it was a little better a few years ago when Mayor Cicilline called all of us together when he was campaigning for office and made his number one priority the development, healthy development, of the youth of our community and we all got together crafted a vision for how we create a full day of learning from the minute when youth wake up until they go to bed. So, in that planning process I am going to take you through a little bit of, focus part of that work in front of you as a bit of a road map for what we did around quality, but it is important as a backdrop for you to all understand that when we looked at where we needed to focus in terms of gaps and needs we decided that the elementary, younger age system was in pretty decent shape, there was a very big hole with middle school youth, there was very little programming and we needed to really concentrate there for the developmental reasons, youth needing programming and at the time we started, of the 6,000 middle school youth, I would say all but 500 were leaving the doors at 2:30 with very little to do, many of them reporting being home alone. So, as we thought about building the capacity, better connecting and building a city system that engaged recs (recreational) departments, schools, police department and all the non–profits from the Boys and Girls, The Y’s, we really reached out to our community and as Nicole said it is really important we focus on youth outcomes.

Our youth and families in a massive market street survey we did reported that one of the number one barriers to participating in programs was poor quality in the community and we had a very articulate guy named Ben Onye say it all and say it best, he wouldn’t walk across the street for a bad program but he would walk across the city for a high quality program. So we knew that if we were going to really look at building a middle school strategy to engage youth, we really needed to be sure that we focused first and foremost on ensuring that we all knew as a community what quality means and that we could actually deliver it and measure it and grow it. So you see in the first slide the road map that I am going to take you through as we talk about what steps we took as a community to do that. Two important things for you to know as we started our work, we decided that we didn’t have a ton of money in our community nor were we convinced we could leverage massive...we don’t have a big corporate community and a small philanthropic community and tight city budget. We felt that whatever we did in our quality effort had to be, as again Nicole said, something that the field was growing themselves, that they would want to own, that they would define and that would inform their own practice. We didn’t have the funding to be an authoritative funding body that would require everybody to meet a set of standards and take measures and improve prep practice; it had to be really owned by the community. So that really informed most our work.

So, I am going to take you through the steps on this road map of what we did so that you can get a sense of what your communities can do. If you go to the next slide, we will start with what we kicked off with, we thought, well, we needed to have a sense of standards, what does it look like, what is good practice and we all decided since we were going to be focusing on middle school and potentially moving into the high school on the next step, we needed to be sure that we were thinking about older youth and youth development standards. So we did what you all can do and you can go use our standards that are on our webpage and click them. We looked at all the best practices out there and the CBASS network that has since formed when we started... they were our best practice exemplars and we looked at what was already out there and sort of did a matrix and said okay, here are the set of standards that exist in some of the core areas like safe environment and relationship building and program and activities and we matrixed out the best of the sort of top 10 cities and said okay what’s missing, what is unique to Providence that we want to be sure to include in our standard and we grew what many of the best practitioners had already done before us. The Forum for Youth Investment was an incredibly invaluable resource as were TASC and all of our of current CBASS partners.

So don’t reinvent the wheel, look at best practice and if you have to then modify for your community, that’s easily done and we did that with a group of about 25 stakeholders that were a varied public to non–profit smaller service organizations and we developed a menu of standards that focused on these core areas. We made sure that in that process that we checked in with our state organizations to be sure that we were aligning with the early childhood care and the SACERs work. We wanted to make sure that it spoke to the 21st Century needs. So we really made an effort to set a statewide agenda which has been wonderfully beneficial because we ended up joining arms with our Rhode Island Afterschool Plus Alliance, our statewide alliance that Mott funds, and the 21st Century office to be sure that everything we articulated in our City of Providence would cross walk and work for the whole State.  Since starting this whole work we have now, we have Rhode Island State Afterschool standards and no longer just Providence standards. So that’s a process I think a community can go through realistically in six months if you take the best authority out there.

If you go to the next slide, we knew that standards, unless you could taste it and feel it and you could actually refine it, it wasn’t going to do you any good to have standards out there. The education community has spent years developing standards and then trying to figure out how to measure and implement them. So we knew that we had to really ensure that we had good measures, that were affordable in real time, that informed all of our practice across the whole system and that we could grow, you know learn from what those tools were telling us in real time and that we could start to change practice and map out a professional development and technical systems plan to support that. So the two tools that we have used that have been extremely helpful, we mapped our standards very nicely to the YPQA that High/Scope developed and we looked at all of the menu that Nicole had recommended and we felt that that assessment tool really spoke to the way we were practicing. It was going to be grassroots community owned and they were very clearly, you know when you read the rubric, you go oh gosh I can see how I can get from a second stage to a third stage and they really spoke to our standards. The great thing about this particular tool is the High/Scope team was very open to doing some additional add–ons and modifications which the director of quality management Elizabeth Devaney who is the genius behind all of this work, she really should be doing the presentation, so when you go on our website you have to connect with her, not me, she’s the guru here, she’s guided all the practice in this community. But they were very, very thoughtful about ensuring that we had some additional addendums to that tool that worked for us, particularly around administrative issues, improvements, apparent engagement issues and sort of office procedures.

But we felt that YPQA on the whole was really going to focus on on–the–ground provider improving the practice in the program. We have a network of 60 different providers; it’s just everything you can think of, everything from a science–based engineering program that has been developed by URI (University of Rhode Island) to a boxing program in a community center to a drama artist. So that was the other thing that was really important, that we look at a set of standards that were going to be useful for a real cross–walk of practitioners and be practical for a recreation center director, police officers doing a PAL program as well as a more sophisticated theater arts program, so that was also a criterion when we looked at the assessment. And then again they could be useful for the 21st Century site that were already funded in the State and we’ve been very fortunate that our 21st Century office has now required that the High/Scope will be used as an assessment, a self–reflective assessment. That was the other condition I think that we felt was really important when we looked at all our at both our tools, is that we use tools that were going to be assessment orientated and that also were validated and could be used as an evaluation tool. But what we’re using them as is a self–assessment reflective improvement process on a whole except for those that are required under 21st Century because we feel that the afterschool community is going to motivate a community in wanting to improve its practice and has got to own and look at ways to improve their practice based on these useful tools.

If we go to the next slide, I’m going to tell you a little bit about the other tool that is very useful for us and that is student participation. One of the first signs, particularly with middle school youth is if they, after the third or fourth day start to drop off, word starts to spread around the school community that that program may not be the hottest program to be taking. So what we’ve got to do is number one, make sure we don’t lose youth and reengage them and that we’re also working really quickly with our provider network to examine why the girls just disappeared from that tennis program when they haven’t from the other. So we are using a tracking tool that is useful across our whole network, first and foremost to manage where all our youth are, we have a campus model where we send youth off site about 20% of the time, so knowing what the participation rate is on the on–school properties as well as in some of the rec centers and the dojos and the environmental Save the Bay kind of programs. We want to know one, are they attending and are they coming regularly, are we getting bang for our buck, are we sure we are watching immediate trends. So we wanted to have it as a management tool to inform daily practice and we actually use the tracking tool to assign kids to buses and know whether a snack is, we’re ordering too many apples or not, so it’s a real time management tool, as well as enabling everybody in our network to look at their particular program or across the whole city which programs are thriving and which ones are retaining kids, some are recycling kids so we get a sense of pattern so that we can strategize effectively with all our partners about making necessary changes or expanding and replicating the program.

Shawn Stelow – Afterschool Investments Project

Hillary?

Hillary Salmons – Providence After School Alliance

Yeah?

Shawn Stelow – Afterschool Investments Project

I’m sorry Hillary while you are talking about some of the more management level issues with the tracking? We have a question from Meg Delyeau asking if the programs in Providence are required to be licensed. How does the licensing work in Providence?

Hillary Salmons – Providence After School Alliance

Licensing is not an issue for us in terms of our middle school and high school. Licensing has been the purview of the elementary school years and actually some of, the 21st Century programs, the afterschool programs and in–school programs do not have to be licensed. But that was an important concern of ours because we were hoping that we could actually, at the time we started our governor actually had enough money in the budget to fund childcare subsidized dollars to youths up to age 16 but the budget cut has been so horrific they’ve cut back now in terms of the age back to 12. We were hoping that if we could be license eligible in all our standard work and assessment work then maybe we could actually attract some of those subsidy dollars to our middle school network. Unfortunately, our economy has tanked and we can no longer do that, but we had that bearing much in mind and Elizabeth Devaney worked very intensely with the childcare network and our state agency to be sure that every move we made aligned and they were willing to recognize the High/Scope tool as a licensing tool for middle school aged youths.

Shawn Stelow – Afterschool Investments Project

Thank you, Hillary. That’s a great point.

Hillary Salmons – Providence After School Alliance

So again, aligning is always really key because one of the things that we didn’t want to do is create a set of measures that weren’t going to be useful for all youth programming and in fact we actually worked with a philanthropic community to make sure that they knew about every stage of our work around standards and assessment development and as a result the United Way for instance funds its afterschool programming as this charitable trust that recommends the use of our tool and our standards because the afterschool community is fragile and it can’t be reporting on different systems and its accountability, it gets more uniform, it’s far more productive and therefore they’ll use those tools to improve their practice for different funders. So in terms of student participation it’s extremely important that we stay on top of the trends of our middle school youths so that we can actually make modifications were necessary.

So if we move on to the next slide, I’ll try to share with you some of the lessons learned as we apply the use of these tools. For us the keeping of data is extremely important and one of the best ways of doing that is having someone like Elizabeth, a sort of intermediary sort of working and coaching and providing technical assistance to those are using tools. But the other thing that is really important, if you don’t use your data daily or weekly or regularly then there is no compelling reason for any of the providers or practitioners to load data well. If they know that the information is going to one provide them with essential emergency information in terms of the tracking tool for getting a hold of a kid when there’s an emergency but also relate to, tracking transit relates to potential continuing funding. If they are all interconnected and compelling in terms of daily use you’re loading data will be a lot better. But we make sure we check it and we are correcting it and we are asking questions about our data all the time. So real time application and, it also helped us, when we need to do data analysis and evaluation with the school department, we can do that comparatively because ours is clean and consistent. We have student ID numbers and if we don’t, our system is different from the school department but it enables us to also have a credible relationship with the school department because they are really impressed at how aware we are about attendance trends and participation trends and knowing about the quality of our program and youth engagement and that we can credibly say we’re not duplicating, we know kids are in your SES program these two days, we know the programs, other two days are in a sports program, oh my gosh kids are getting a four day menu. We keep track of all the programs whether they are ones that we fund, we do that with all the programs in the school. We have created a unified schedule across the whole middle school system which enables us to get a sense of all the tracking trends for youth.

So again, as I said Elizabeth Devaney and the management oversight is absolutely critical but you know, every single one of us prints out reports all the time to look at particularly on the tracking tool, what’s happening, what are the trends, what do we need to know, what questions do we need to all ask at all levels, at the city–wide management level as well as at the provider level.

So the final slide if you take it to the next slide is what have we learned in terms of good quality assessment? The High/Scope model has been really highly received by all of our practitioners. For us, that was really important because we didn’t want to be a big brother on assessment, we really wanted to be building trust and we are providing funding and technical assistance so people really needed to feel as if this wasn’t a policing effort, this was really about oh we see some vulnerability here or needs here, how do we help you and it has helped us having a quality advisor. It is a bit expensive to do that. It’s sort of an intensive, like having a coach but I think it has been very helpful for our providers to have someone walking through a work plan and identifying areas of improvement almost within weeks after taking the assessment and it’s enabled the boxing instructor to make some changes in a program immediately and see some improvement and the youth more engaged. So, when they see that improvement get real response, they are more likely to use it. Where it is a bit expensive, we have been finding it useful to have the quality advisor in its initial application stage for sure. So buy–in is important and we also find that the initial trends we are seeing on the High/Scope tool which has a great computer data base is sort of showing us and identifying where there are lower rankings and therefore need for support. That is informing our capacity building efforts and the workshops that Elizabeth arranges and it also informs the technical assistants that we zero in on with our partners as well as informing what we need to fund more of because it’s good and its working. So we have become real advocates for our providers because we are using their information and knowing what improvements they are making and for us the most important thing is that they are all really working very hard to improve their practice.

So that’s sort of where we are in that process and I think it might be wise for me to turn it over to New York City, it operates a different scale but with the same sort of depth of practice and thinking.

Shawn Stelow – Afterschool Investments Project

Hillary, we have one more question for you, that was sent in from Krista Galloway and her question is sort of a chicken–or–egg question, did you use the tracking data to convince the school department to be the partner or were they really on board from the start. I know you had a lot of strong support initially from the Mayor’s office. What was the support from the school department initially?

Hillary Salmons – Providence After School Alliance

I think the school department initially sat at the table and was waiting to see if we could actually produce a system–wide strategy. They were so used sort of short trends in funding and different kind of lead organizations coming in and 21st in some schools and not in others. Where I think they have started to engage more is because we have consistent about our practice and we are using good data, I mean at the district because we are holding ourselves to a very high sort of level of accountability and they became more and more impressed and intrigued with our consistency of practice because of our use of data. And now I have to say that we, it’s taken us about three years and I think there is a three year waiting period with almost all school interfaces; if it’s going to go away in three years, then they will not change their behavior but if it’s going to last beyond three years no matter how much the superintendent or the district heads love it and the Mayor love it I think that teachers and principals start to lean in more, because if they can rely on you and think about ways to grow practice and coordinate around helping you and you’re not going to be just a fly by night favor pet project of a mayor then I think they start to appreciated the effort we’re making around high quality programs.

Shawn Stelow – Afterschool Investments Project

That’s a great point, thanks Hillary. At this point let me reintroduce to you all Lucy Friedman who is the president of The After–School Corporation in New York City. For 10 years TASC has worked to fund after school programs in New York City, expand access of the city’s children to high quality programs and has served as a national leader in improving the quality and quantity of out of school time opportunities. Lucy you can take over from here.

Lucy Friedman – The After–School Corporation

Okay, thank you Shawn and Hillary gave a wonderful lead in, in terms of where we started as well. One is we are the only people from New York on this call so I can also say whatever I want but I am also going to be presumptuous and not tell people about New York City, but I guess to just remind people that we seem to be ground zero of the economic meltdown and how that is going to impact our programs we’re all waiting to see but I think some of us are feeling somewhat hopeful because I don’t think it’s a question anymore of do you eliminate afterschool? I think we’ve become too much a part of the system but we are and we will continue to feel cutbacks.

But Hillary’s point I think was important which is when we started, we hardly discovered afterschool but we really started with this notion of building a system and by building a system you would start changing expectations, the expectations of school leaders, the principals, the teachers, the parents of kids and that was really important to have the kind of consistency and data that Hillary spoke about. And one of the things that I think somewhat distinguishes us from some of our other CBASS partners is that we very much started with the goal of changing pubic policy. George Soros, when he made the investment in New York City, it was for New York City but the notion always was that we should be developing a system that could be a catalyst, could fuel changes throughout the country about afterschool and obviously the huge growth of the 21st century over the same 10 years we have been in existence has made that much more possible that we would even have anticipated 10 years ago. And because we knew we had a policy orientation, not only a direct service orientation, we, from the very beginning started collecting data and we’re trying to think from a systems perspective actually like Providence we offer youthservices.net, which is a very useful tool. We also from day one said we have to start visiting the sites. We need to see what’s happening. We again, I appreciate the way the program has developed at PASA and DC Trust and all of our partners. We were very impatient; we insisted that our sites looked the same.

You can go to the next slide now. Part of that was we wanted to start really seeing what could be achieved and we started with a cost model and so because we had a cost model it was very important that we collect attendance and enrollment data so we knew that we ultimately could say to government if you invest this much money you’ll get this much result. So our first two efforts were site visits, we wanted to see what was happening and why that was very important is we could see if one site was really researching quality in many of the ways that Nicole had talked about before and another one wasn’t we could say to the other one you have enough money, the same money as a different site in a different borough, you can do it, and that was very important. So we started with just site visits and participant tracking as our first sort of sense of data collection but over time we came to the same set of issues that Nicole’s discussion reflected and then Hillary’s did, which is that we wanted to start setting standards and from research that we had done with Policy Studies Associates which was doing an independent evaluation of the program, we started thinking about what should be happening in that setting, really at the point of service and we worked with Policy Studies Associates to create an observation tool which really again goes to many of the same sets of issues, probably with relationships being the most core of the issues that is really important to look at in terms of quality. So, we developed with Policy Studies Associates an observation tool and started applying that.

That then in turn led us to say we need to help our sites understand what they should be trying to achieve, we had a set of core elements, one adult for every 10 kids, a mix of staffing, a mix of programming, but that doesn’t go to the real quality issue. So we started developing what we called the Quality Self Assessment tool and at the same time, the New York State After School Network started being created and we realized that it would be a much more powerful tool if it wasn’t just a TASC tool but it was a tool that really had buy–ins from the whole community and I think what also was really important for us about this tool is that we had always thought that afterschool should be more than just about academics and by having a tool that covered a lot of domains, community relationships, individual relationships, the climate and the environment of the program, it really was a way of communicating that these programs were not just about academics. It also was a way of being able to say to the community and again to principals and policy leaders, this is what a good program looks like and I think Nicole is definitely right that there is a lot of consensus about what a good program is like which made this instrument much easier to develop and you will be able to, they’ll do a runner for you about how to access it, if any of you out there want to access it and we have just developed a guide on how to use the self assessment tool so it should be easily used by lots of different groups.

In New York not only does the State Department of Education adopt it but in New York City when they made the decision to build a system of after school very much modeled on the initial TASC system they also are using basically the same domain. So we have been able to create within New York a pretty common set of language and a common set of standards, again of course it doesn’t mean that every program is meeting those standards but it does mean that there is a building to a community of practice that makes sense. As we develop the quality, the New York State Afterschool Network we started realizing that’s what led us to think more about the power of working with other systems.

You can go the next slide. That really led us to think about how to work with systems in other cities and lead to the development of the Cooperative for Building After–School Systems or CBASS. When we got together, the original six cities and then adding two more we all realized that we would have so much more power in terms of policy both within our own jurisdictions but in other jurisdictions and nationally if we had some common measures, if we could say that we’re all counting the same thing and we all agree that the same thing makes sense in terms of measurement.

You can go to the next slide. One of the things that we realized early on was that there wasn’t a lot of comparability, we were talking about the same things but we were measuring them slightly differently. So we engaged Liz Reisner of Policy Studies Associates to work with us to identify a small number of practical measures and I guess what we thought about was that we wanted measures that were what we called low–burden, that didn’t cost a lot of collect and going back to the term that Nicole used also, where there wasn’t a lot of room for interpretation so if we collected something in New York, we could be pretty sure that in Providence or in Washington or in Baltimore they would be collecting it and counting it the same way. We also then wanted to think about three levels, the individual youth level, the program level, the setting level as Nicole talked about and the systems level. Then last summer with a grant from the WT Grant Foundation and in partnership with the Forum for Youth Investment we convened a group of afterschool systems and researchers to see if we could sort of hone down and come to agreement about a certain number of accountability measures.

Can you go to the next slide?  You can see that we had two kinds of sets of measures, one which we could collect pretty easily without a great deal of cost that many of us were already collecting. Can we go to the next slide? These were on the youth level, the program level, and the systems level. Are we having trouble going to the next slide?

Hillary Salmons – Providence After School Alliance

Probably because it’s pretty dense, it’ll take a little while to load it up.

Lucy Friedman – The After–School Corporation

Okay. So the youth level was sort of something I’m sure that most you out there in the fields are familiar with and think is important which is the daily program attendance and you know as much as some people say program attendance is an input and not an output I think that we all felt was, and this is particularly true for middle schools and high school programs, if kids aren’t coming, the program can’t have any impact. So we need to figure out a way to make sure that we’re all counting attendance. On one hand that seems so obvious and clearly in terms of sort of convincing educators that you’re serious is seems sort of bottom line and of course schools have always kept attendance but in the afterschool fields or the youth development fields there wasn’t the focus on attendance before the development of the afterschool field.

Hillary Salmons – Providence After School Alliance

And since a lot of kids shop with their feet in an urban community, if it’s no good they’ll go home, they won’t hang out there. Where in suburban communities mothers tend to be much more aware of where they are and where they are not and elementary schools kids by law have to be taken care of by someone.  So I think urban communities to that point, Lucy, are much more vulnerable, don’t you think?

Lucy Friedman – The After–School Corporation

Absolutely.

Hillary Salmons – Providence After School Alliance

They can walk.

Lucy Friedman – The After–School Corporation

Right, and we see in New York starting probably in 4th grade that kids start walking, start voting with their feet if they have that permission. Although legally Hillary is absolutely right it takes to fifth grade. On the program level the two things that we determine that we all were interested in and could keep track of were the youth–to–staff ratio and our policy in New York was 1:10 which is the licensing requirements in New York for kids under 10 and 1:15 for kids under 12 but that was a really important measure of quality. Going back to the kinds of issues that Nicole was talking about that if you want to have those kinds of relationships and you want the point of service to really make sure that kids are getting an experience that’s going to help them develop socially and emotionally you need that relatively high or actually low ratio of staff to young people.

On the systems level there was a series of measures that we felt was important, one was number of program slots, all the intermediaries are concerned with scale and concerned with how that these are not as we say boutique programs, these are programs that fill slots of kids so that it’s really important to have, keeping counting slots and it’s very important that we keep increasing slots. I think given the economic situation probably all of us, certainly New York if we can hold out own that’s what we can expect now but we will want to continue to grow. Again, just reflecting back on the conversation we’ve already had adopting quality standards is important, having a tracking system as Hillary described is important, having policies in place about these standards is important and then being able to provide technical assistance and training to the program so that you’re just not creating a standard and not helping the programs achieve that standard. Part of an intermediary’s function is to make sure that you have continued financial support. It’s really important in terms of changing the minds of schools and principals –– if you have a program that is here today and gone tomorrow and that happens more than once or twice, they’ll say, you know, this is not serious, this is not something that we can count on or respect and that is really important to engage the leaders in the community who are running the youth organizations, that they believe and they trust these numbers.

Shawn Stelow – Afterschool Investments Project

Lucy, this is Shawn, we have rejoined. I don’t know if you all know this but we disappeared momentarily because we have a network problem here at the office but have been able to rejoin you so sorry if you have not been able to advance slides.

Lucy Friedman – The After–School Corporation

We haven’t been. Okay.

Lucy Friedman – The After–School Corporation

Alright, so can we go onto the next slide.

Shawn Stelow – Afterschool Investments Project

We’re still rebooting for the next slide but, while we are waiting to resume, I had a question and if you’ve all covered this, just tell me you’ve covered this. As both of you spoke, both Lucy and Hillary, you spoke about the really great connection that you were able to form with all of the systems working with afterschool for school–aged children, those may be childcare as well as the education system and both of you have been able to get your investment tools to be used and accepted or approved by your State Department of Education which can be a real challenge. What I know, if you already spoke of some additional pieces that Rhode Island, rather than Providence initially and then Rhode Island added to the YPC list, I wondered if possibly those were perhaps the request of the State Department of Education there and if there were any other things or any kind of back and forth Lucy, maybe that you all had in New York that, I don’t want to call them in sections but perhaps considerations that you discussed with your State Department of Educations who ensured that you were dealing with the academic part?

Lucy Friedman – The After–School Corporation

Well in New York when we developed the NYSAN tool, people from the State Department of Education were part of the, we had a quality assurance group and they were part of that group so it was more organic than us developing something and then saying no, you have add this or that or the other. You know, my recollection is that they did not push the academics. Yes, it was certainly part of the programming activities and I think everybody agreed that it should be part of it but that it wasn’t the whole banana.

Hillary Salmons – Providence After School Alliance

Yeah and for Rhode Island the 21st Century Office, Jackie Ascrizzi has been in fact an early champion of thinking in a much more broad way around 21st Century in terms of social and emotional physical health and a broad strategy that wasn’t solely tied to academics and they were involved in the development of everything that we did from the get–go. In fact, in many ways, we were at a city level really trying to grow the 21st Century Office’s work and the State of Rhode Island Afterschool Alliance so we mapped everything together from the get go and wanted to be sure that; and that we were very fortunate in that Jackie Ascrizzi and her office always had a youth development focus in their thinking about 21st Century so there was no rub. In fact, they tried to use an assessment tool that was a little bit too education centric and cumbersome and they switched to the High/Scope YPQA because it had more of a practitioner focus and they were really about assessment and building practice so we have had a very open–minded Department of Education with a very progressive Commissioner McWalters who is really thinking in new ways about graduation requirements in a broader way.

Lucy Friedman – The After–School Corporation

And you know, to New York City’s credit when New York City, when Mayor Bloomberg built the system of out–of–school time, they were also very clear from day one that academics was a piece of it but not the whole piece. So I think one needs, I mean to respond to your question Shawn, it is really important to have leadership that gets that out of school plan programs are going to be more than just academic and sometimes they have to certainly be the leaders in that so that school principals get it.

Hillary Salmons – Providence After School Alliance

Well the philosophy is important to be sure you sort that out I think at the very beginning.

Lucy Friedman – The After–School Corporation

Mm hmm. So we can go onto the next slide.

Lori Connors–Tadros – Afterschool Investments Project

This is Lori, we’re just logging in and we’re going to get to that but I wanted to ask one question that expands out a little bit while we are waiting for one more second. Hillary had mentioned the role of State Child Care Administrator and I think the challenge is actually usually to find common ground among the various policy makers at both the State and local level and while we are waiting one more minute, does anybody have, you know, would you have a short reflection on how you have helped folks come to consensus around, you know given State perspectives and State needs from both Child Care CCDF as well as at the local level?

Lucy Friedman, The After–School Corporation

Yeah Shawn, when you talk about great relationships, I mean the reality in New York is that we don’t have a perfect system, an aligned system that school–age child care is administered through the social services agency and 21st Century and Extended Day is administered through the school system and the social services agency, outside of New York City, they haven’t as yet adopted the QSA tool. We are hoping that they will. In New York City, again with the Mayor’s leadership, they come together. So you know, it works but I think, you talk about great relationships, and I don’t exactly know Hillary’s history, but I know for us, it’s working well together but I want people around the country to know that I don’t think it ever comes easy.

Hillary Salmons, Providence After School Alliance

You have got to work for it. You have to have a seat at the table and have a credible argument and you have to have research best practices and I think where we have been stronger at sitting at the table with the child care community is because we have been using a validated tool, so that was one of the reasons in our selection process.  Elizabeth and our team have made sure they sit at the table with an early childhood team and also our Rhode Island Kids Count has been instrumental in brokering a streamline strategy so I don’t know if State Kids Count offices can be useful advocates but I think that they have been an advocate as has the United Way that has invested in the Early Childhood Bright Star system, making sure that there is alignment. So having partners like the United Way and Kids Count and advocates but also making sure that you’re always at the table to make the case that you’re about... really seriously research best practices is important.

Lucy Friedman – The After–School Corporation

Alright, why don’t I because I know we’re coming to the end, why don’t I just go quickly to the next, can we go to the next slide? Okay. So we all agreed on those first ten slides, ten measures and that they could be collected relatively easily. So we also knew that we also have this interest in sort of deeper measures of what is happening at the youth level and at the program levels and that these are harder things to measure easily and so what we determined is that we should, if we had our way, every intermediary would have the resources which we don’t have yet, for a subset, probably a random subset of programs collection additional information, one being daily school attendance and then the program level information which again goes very much back to what the High/Scope tool collects. Youth relationship with adults, opportunities for choice and again, the older kids get, the more you have to offer them choice or they don’t come. The active hands–on learning which comes obviously from the Durlak–Weissberg research and I think from general, what we know about youth development and we also know from research that the educational level of the director and the staff make a difference in terms of quality and so that would be something else to measure. But to get these on every program in every city is going to be a more expensive proposition and so High/Scope does some of it, our policy studies observation tool which measures many of the same things as High/Scope does some of it but we don’t have this yet in a systemic way and that’s certainly one of our next steps.

So we can go to the last slide. So our goal is to start collecting the common measures across our eight jurisdictions and continuing you know the conversation on common measures and have other jurisdictions and strengthen their ability to collect these measures but also to hear from them about they are doing it because you know, our eight cities certainly don’t have all the answers and we want to learn from others and continue to build the capacity of the Afterschool system so that they can continue to improve so we can all achieve the quality of every program that Nicole has convinced us is very important to do in order to have the outcomes we want.

Shawn Stelow – Afterschool Investments Project

Great, thank you so much to Hillary and Lucy as well as to Nicole and Emily for their presentations today. I think you have seen across the marquee to scrolling to continue to submit questions. We have one from Mary and I will read it to you and Lucy and Hillary, I think this is probably more directed towards the two of you.

Lucy Friedman – The After–School Corporation

Okay.

Shawn Stelow – Afterschool Investments Project

Growing body of research points to the importance of children and adults having fun out of doors. This issue does speak to the other domains of learning other than strict academics. Have any CBASS programs included outdoor learning as a frontal component of these out of school programs.

Lucy Friedman – The After–School Corporation

Because again we don’t run the programs directly we work through community organizations, we encourage them all to do that. We encourage trips and gardening has become a very popular activity as well as sports outside and Hillary does....

Hillary Salmons – Providence After School Alliance

Yeah, in the spring, we fund three different semesters, Fall, Winter and Spring and in the Spring we lose almost half our youth because they don’t want to be in the building any more so we are making, we make a concerted effort to do outdoor programming. If you all visit the www.edutopia.org/ site and the New Day for Learning, you can take a full tour of everything we do and you will see you know we’ve got a Save the Bay, we’ve got sailing, we’ve got tennis, we have a lot of environmental education outdoors and take lots of field trips, middle school youth talked a lot about wanting to do programming offsite and out of their neighborhoods so you’ll take a tour to the pounds or we do a Pets is Best program so a lot of our programming is about trying to get urban youth out of doors and healthy.

Lori Connors–Tadros – Afterschool Investments Project

Hi, this is Lori. I have one question, I wanted you to speak a little bit more about is really the systems building. I know we have Providence and New York here but can you give us a fuller story of how CBASS works together, how you...you’re speaking today about the work around measuring quality but could you give us a little bit of a larger picture of what the challenges and the promise, if you will, of your system building efforts are, I think that would be helpful for the state folks to hear as well as other communities that are looking to partner in order to collectively build systems.

Lucy Friedman – The After–School Corporation

Hillary, do you want to answer that or me?

Hillary Salmons – Providence After School Alliance

Well I’m conscious of the time shortage, but if you do visit that Edutopia site it is literally the micro–macro lessons, eight of them, on how to build a system and the reason why quality is essential is if we are going to brave funding in a systemic way target to use, then achieve, if you’re going to make the case for more funding, if you’re going to make the case for knitting your assets together in a more coherent way, it isn’t about quality, there’s going to be nothing compelling for the political community as well as the funding community as well as most importantly the youth of the parent community. So if you’re going to orchestrate what you are doing and maximize your resources, these are the core of everything you have to do.

Lucy Friedman – The After–School Corporation

And CBASS, you know, we stand ready we have other jurisdictions create systems to strengthen their systems and you know, every city has its own particular set of issues and politics and obviously some cities that Mayors are in control of education, others not and that’s a big variable. So we stand ready to help if any of the corps want specific help in analysis of what their issues are. We sure don’t have all the answers but we think working in a community practice, we get some better answers.

Hillary Salmons – Providence After School Alliance

And I think the next horizon for all of us in the afterschool role is to think about making a better integrated connection with schools and then sort of the new horizon is expanded learning and thinking about what a full day of healthy academic, social and emotional physical health are all about. And if we’re going to do that, again, if we’re organized in a systemic high quality way, I think we’re going to be better able to broker a partnership and an exchange with the education community. So that’s why I think systems, when it is one program trying to get into a school, it’s almost too much for a principal.

Shawn Stelow – Afterschool Investments Project

Thank you all so much, I think this has been a great conversation building from the research base which was presented by Nicole to the more practical base and how this work really hits the ground at both the systems level as well as how that, as well as feeding down to a program level and you know, Hillary as you said, at the heart of it, we are really about providing great programs for the communities and the families who participate in them. You should see on your screen now an evaluation that you can click through and submit, right now. Also, we will post another copy of the evaluation at the Afterschool Investments website. So if you’re short of time now and are unable to complete the evaluation now, you can visit the Afterschool Investments website at your leisure. We will post copies of the slide presentations on the Afterschool Investments website. It will take a couple of days for this to be up. You can also send an email to Soumya Bhat (sbhat@financeproject.org) here at our office who can send you a copy of those slide presentations if you would like them earlier and even sooner.

Again, thank you all very much for participating in today’s webinar and we hope that you will join us in future webinars. Thanks a lot.

Unidentified.

Thank you everybody, this was terrific.

Lori Connors–Tadros – Afterschool Investments Project

Feel free to contact us for anything you might need. Have a great afternoon.

Operator

Ladies and gentlemen, this concludes today’s teleconference. You may disconnect your lines at this time. Thank you for your participation.

Download FREE Adobe Acrobat® Reader™ to view PDF files located on this site.