Thanks for your questions Michael.  The video shows two girls talking about their projects with artifacts or final versions in their hands.  We do not have a library of student projects.  Although I agree that showing youth projects would be interesting, I also think that we view our work as not simply reflected in projects or products.  The relationships our youth build with UW STEM mentors are a key feature of the program and are a critical part of the success.  Mentors offer a counterpoint to some of the deficit messages youth receive from society at large - both implicitly and explicitly.  In STUDIO we believe, in the words of one mentor, that our job is to help youth recognize "the STEM in them" and to value and build on the assets in their community.  This is what ultimately impacts their interest, motivation, and identification with STEM.

The youth enroll for one academic year in the STUDIO program.  We enroll approximately 45 middle and high school students each year.  Our evaluation design involves a comparison group of students who are involved in non-STEM related programming.  We are using pre and post surveys, focus groups, interviews, and qualitative documentation of programming including images of projects, running records of interactions between staff, mentors, and youth.  Here is an abbreviated summary of what we found out through our work during Year 2 of the program (2015-2016).  

1. What are the strengths and weaknesses of the STUDIO program as implemented?

As in Year 1, Year 2 STUDIO participants gave the program high marks. On the Youth Post-survey (n = 38), 97% percent of youth gave STUDIO a grade of A or B. Ninety-seven percent of youth indicated they had fun in STUDIO; 92% of youth said when they are in the program, they feel like they belong; and 89% of youth said they looked forward to attending STUDIO.

Year 2 participants said they appreciated the program for many reasons, including the fun, hands-on activities, and that staff and mentors were caring and supportive and made learning about STEM fun. Youth said they liked STUDIO activities, with 97% saying the activities are interesting and 92% saying that they do things in STUDIO they don’t get to do anywhere else.

1. Does participation in STUDIO programming increase knowledge about STEM higher education and STEM careers?

A number of STUDIO participants increased their knowledge of STEM careers and career pathways. The percentage of youth who said they know some or lot about the types of things that people with careers in STEM do increased from 59% to 79% after STUDIO (results from retrospective pre- and post-test questions). The percentage of youth who said they know some or lot about the types of classes they need to take to have a career in STEM increased from 70% to 78% after STUDIO (results from retrospective pre- and post-test questions).

Year 2 participants generally reported greater gains in their knowledge of STEM careers and educational pathways after participating in STUDIO as well as more knowledge at year-end than either non-participating comparison youth or Year 1 STUDIO participants. For example, Year 2 STUDIO participants’ assessment of their knowledge about how to find information about STEM careers increased from a mean of 2.5 on the pre-survey (on a four-point scale), to 3.2 on the post-survey (vs. 2.7 for Year 2 comparison youth and 2.8 for Year 1 STUDIO participants).

1. Does participation in STUDIO programming produce measureable impacts on youth’s interests' and motivations in STEM?

Year 2 participants generally reported more interest in STEM careers at year-end than either comparison youth or Year 1 STUDIO participants. For example, Year 2 STUDIO participants’ mean score regarding their interest in working in a career that allows them to use engineering-related skills or knowledge was a mean of 4.7 on the post-survey (on a six-point scale) vs. 3.3 for Year 2 comparison youth and 4.0 for Year 1 STUDIO participants.

After participating in the program, Year 2 STUDIO youth were somewhat more likely than comparison youth to say they were interested in entering a career that would allow them to use either math-related skills or science-related skills. While STUDIO youth also reported greater interest in careers using technology and engineering skills after participating in the program, comparison youth reported even greater increases in their interest in technology and engineering at year-end, suggesting that other factors besides program participation may have affected youth’s interests in these areas of STEM.

1. Does participation in STUDIO programming impact STEM-related identities?

Survey findings suggest that participation in the program increased students’ confidence in STEM: 95% of participants agreed that STUDIO helped them see they are good at technology, 94% said STUDIO helped see they are good engineering, 85% said STUDIO helped them see they are good at math, and 82% agreed that STUDIO helped them see they are good at science.

The STUDIO project team deliberately made more explicit connections between STUDIO programming and STEM language in Year 2 than in Year 1. Year 2 programing included more activities that drew connections between what the students are doing in STUDIO and associated STEM education and careers. Results from the focus groups suggest that Year 2 participants connected STEM with what they were learning and doing in the program. For example, Year 2 focus group participants described STUDIO as a STEM program. They said STUDIO deepened their understanding and appreciation of engineering, in particular, and helped them make connections between STEM and their everyday lives.

Hi Laura,

Thanks!  See my response to Michael above.  It will give you a sense of how we are approaching the metrics now.  During Year 3 (2016-2017) we've been able to start using our qualitative data more to track the development of relationships in the program and to begin to outline the equitable facilitation practices that mentors are learning through their participation.  We see this as an important aspect of understanding STUDIO.  

Hi Kylie, 

Your question about what projects/materials work best or what would be the most fun for youth is something that we constantly reflect on. Our starting point for each project idea is asking youth what they're interested in and matching that to the mentors' expertise. Many youth have expressed an interest in pursuing video game design or a career in computer science and so mentors who had experience with programming designed a project around learning how to code a guessing game. In my opinion, projects that work well either allow youth to explore career interests, allow them to work with their hands or dive deeper into their personal hobbies. Last quarter, youth expressed interest in music and so STUDIO's STEM coordinator, Chris, created this amazing project for youth where they could learn sound, how speakers work and how to record their own music. I heard one youth say, "one thing that was pretty cool about Music was that we got to talk about real stuff." This youth also expressed how they were disappointed that their music wasn't up on SoundCloud yet. Giving a space for self-expression, art, and creativity and weaving in STEM elements seemed to really resonate with youth.

Thanks Kylie for the great question and thanks Elisa (STUDIO lead mentor for 2 years and current documentation team member) for such a helpful response.  We definitely are inspired by the work that you do to bring together STEM and the arts Kylie!

Hi Ann,
I started with STUDIO in Year 1 as one of the Lead Mentors and am now working with the documentation team to collect qualitative data. (Lead Mentors support the facilitation of the seminar that undergraduate mentors take on campus and other logistical work of the STUDIO program.)

 

From my observations of other mentors and my personal experiences as a mentor, I think that mentor-youth relationships evolved from being very surface level (youth seeing mentors as college students that know a lot about math and science and potential advice givers) to more of a friendship (trusting and respecting one another, looking forward to being in this space together, sharing stories about family).

 

A majority of youth attend the same schools and/or have a friend in the program which makes it easier to start connecting with other youth. At the beginning the year, icebreakers and more "get to know each other" type of activities are weaved into program to intentionally foster interaction. Activities throughout the year continue to do so by emphasizing collaboration between youth, for youth to teach and problem solve with one another, and the mentors are also there to encourage that type of interaction. Youth participate in STUDIO for 1 year and mentors are encouraged to make this same commitment to the program in order to sustain these relationships.

 

Watching shows that they like to watch, becoming familiar with their community and sharing yummy snacks (this last one always seems to work!) are some strategies that I try to use. One of STUDIO's graduate research assistants is working on collecting perspectives and strategies from current and past mentors to add to an interactive mentor handbook that that will support future mentors in navigating relationships with youth.

Hi Ann,

 

My name is Kari Nasu and I am currently a Lead Mentor for STUDIO (following Elisa's graduation). I began in Year 2 and have continued since. As for relationships among and with the youth, I'll have to echo much of what Elisa has articulated above!

 

I also believe that the mentor-youth relationships evolve from surface-level to personal. Upon first beginning a quarter in STUDIO, the interactions often consist of introducing one another and conversing about the project/content at hand. As the mentors and youth get more familiar with one another, both groups gain knowledge about the personal lives.

 

One strategy I find useful is bringing up a previous topic of conversation with the youth. For example, many times they will share what they do outside of STUDIO (sports, school projects, family trips, etc.). When they share such things, I enjoy bringing up the topic during a following week by asking how it's going or what has changed. This not only furthers the relationship, but can also provide opportunity to better understand how their interests came to be. This contributes to making the mentor-youth relationship more personal, as it makes it apparent that the mentor isn't just someone who helps them in projects - they're also a friend that cares about getting to know them. I believe that one of the reasons the relationships can undergo such transition, is because the youth and mentors consistently see one another over the course of an entire year (this is one of the points Elisa mentioned). Also, the projects are geared towards the youths' interests, which often prompts them to be engaged and interact with the mentors from the start.

 

In addition to a friend-like relationship between the mentors and youth, I believe the youth also see the mentors as helpers and encouragers for their projects. Many times when the youth aren't engaged or say they're "out of ideas," the mentors try to reengage them. Also, sometimes the youth are on a roll and need assistance accomplishing a certain task (I especially have woodshop in mind for this situation; often, the youth have a vision and need help stabilizing, cutting, or nailing the wood). Overall, one of our goals is for the youth to have a sense of ownership of their project and for the mentors to be available for goal-setting, motivation, and expertise when needed.

 

A strategy I use when in the situation of disengagement, is revisiting the youths' initial goals and reflecting on what they have at the moment. Sometimes, reconciling initial and new ideas is needed, and other times, brainstorming to spark further ideas is needed. It's very situational, but often, I find that talking through their thought processes is a good start. From there, it can lead to silly brainstorming or thoughtful decision-making.

 

As for youth-to-youth relationships, as Elisa said, many of them know one another from school or from living in proximity to one another. Also, in project groups, the youth are often paired with each other or there are group interactions (whether that be icebreakers, sharing of each person's project, sharing tools, etc.). Working in a space of collaboration among the youth I believe allows everything from lighthearted conversations, jokes, and productive teamwork.

 

All of this is exciting to me! In addition to seeing developing relationships and group dynamics throughout the quarters, I also enjoy seeing how the youth view STEM as they tinker and experiment. We like to call this how they "find the STEM in them."� Seeing how their goals shift or get more specific is very rewarding!

Kari, 

 Very glad to see this critical element of project development shared here!  Relationships are the core of the success as it is hard to venture into an idea without one....or even practice in building that element of a project without a mentor.

 Wonderful that you ensure that youth are listening to them and  that their interests are valued as they learn skills that may be foreign to them.  A collaboration that "allows everything from lighthearted conversations, jokes and productive teamwork sounds like an amazing atmosphere to nurture.  Congrats!

Hello, Anna. Thank you for your comment and question!

One of the major needs that we noticed in year 2 was to provide support for the mentors in designing 6 week projects. In year 2, some of the mentors lead various projects related to coding, rocket science, and sports science. During year2, reserach team was very limited to provide support  in designing activities and culminating projects that were more alinged with the youth's interest and mentors' expertise. This is not to say that projects in year 2 did not incorporate these components. The mentors just were not provided with appropriate support in curriculum design from the reserach team.

To connect the youth's and mentors' rich cultural and academic knowledg that they bring to the program, Leslie and the graduate students lead curriculum design workshops in year3. There were three workshops in total over the winter and spring quarters, and had following themes:1. Scope and sequence of curriculum, 2. Theory and practice of culturally respsonvie teaching, 3. Model lesson on how to incorporate pedagogical strategies to connect students' knowledge and the mentors' knowledge.

As we reserach mentors' role and learning in Studio, we noticed that it is crucial to connect mentors' expertise and the youth's interest. Mentors who designed and lead the curriculum reported that they built strong relationship with the youth because their expertise in a particular STEM field became more relevant and usefu in the activites that they designed and led.

Hi Betsy,

Thanks for your post and suggestion.   I'd love to see your materials and have you share how you are looking at CTE competencies as bridges.   Can you share an example?  Could we connect off-line to learn more?

Thanks so much!

Sure, happy to connect. I added my email into our video’s comment area.  You can head to the NH Dept of Ed website and drill into individual NH programs at this link: CTE Programs and competency lists

What I refer to as “bridging” however is the key factor.  Your project accomplishes the piece that frequently stalls the DOL union’s apprentice programs as you nurture the participant rather than focus on materials that they are required to know.   Don’t lose that element by the checklist type structure shown in the form of a paper crosswalking!  Glad to describe the process that we used in the MSP project when you have time.  

Betsy

Thank you Betsy!  This is really helpful.  I will be in touch.

 

Hi Lauren,

 

Thank you for your question! I'll attest to what Jiyoung has articulated below.

 

Throughout my experience in STUDIO (currently finishing my 5^th quarter), I've gained a lot of direction in my studies and future career goals, and I'm confident the other mentors are with me as well. In STUDIO, I've found much enjoyment in working in an interdisciplinary and diverse environment, and this has led me to strive for a future career with such characteristics. Being involved in such work has definitely led to my interest in working with diverse people groups in technology, which I'm not sure I would've discovered if I hadn't joined STUDIO.

 

As Jiyoung stated below, she's been conducting interviews with seasoned mentors about their experience in STUDIO. Also, all mentors write weekly reflections where they have the opportunity to share their thoughts. These two ways allow for the influence on mentors to be measured.

Hi Lauren,

This is a terrific question! Yes, we are beginning to try and capture this as Kari highlights here.  I think we underestimated how much this project would impact the mentors themselves when it was in the design phase.  We have had several mentors express an interest in teaching (one is now teaching computer science at a local high school) and youth development (one now works at Neighborhood House as an Americorps member) and Elisa (who posted above) has joined us on our documentation team this year.  So I think that we do see this as a potential path into an STEM-related educational career for certain mentors.  

Hi Anna,

This is a great suggestion.  We try to do this when possible (and when we know about relevant materials).  I'd love to know if there is a repository to find all these potential out-of-school time curricular resources in one place.  Do you know?  The other issues we work to address is how to make the curricular experiences align with STEM & youth development practices.  We use the Youth Program Quality Assessment tool in program and that helps us think about programming quality as well.  

Hi Selma,

We were just talking about this in our documentation team meetings over the past few weeks - so this topic is really on our minds!  We have not yet conducted a systematic analysis but we are noticing some patterns (youth choosing activities that could be viewed as stereotypically attractive to boys or girls) and we've been wondering about that.  Both from a research standpoint of understanding these patterns and from a practice standpoint.  I know some of the mentors are asking if and how we should critically engage the youth in thinking about these patterns.  We realize we need to do this systematic analysis first in order to have those discussions so it's something we're working to do as part of analyzing our documentation this summer.