1. Daisy Rutstein
  2. https://www.sri.com/about/people/daisy-wise-rutstein
  3. Senior Education Researcher
  4. Computer Science in Secondary Schools (CS3): Studying Context, Enactment & Impact
  5. http://pact.sri.com/
  6. SRI International
  1. Marie Bienkowski
  2. https://www.sri.com/about/people/marie-bienkowski
  3. Deputy Director
  4. Computer Science in Secondary Schools (CS3): Studying Context, Enactment & Impact
  5. http://pact.sri.com/
  6. SRI International
  1. Michelle Huynh
  2. Research Program Coordinator
  3. Computer Science in Secondary Schools (CS3): Studying Context, Enactment & Impact
  4. http://pact.sri.com/
  5. SRI International
  1. Eric Snow
  2. https://www.sri.com/about/people/eric-snow
  3. Senior Education Researcher
  4. Computer Science in Secondary Schools (CS3): Studying Context, Enactment & Impact
  5. http://pact.sri.com/
  6. SRI International
Public Discussion

Continue the discussion of this presentation on the Multiplex. Go to Multiplex

  • Icon for: Eric Snow

    Eric Snow

    Co-Presenter
    Senior Education Researcher
    May 15, 2017 | 10:47 a.m.

    The K12 Computer Science Framework was recently released (https://k12cs.org/), but contains limited support for teachers to assess the target practices. How can/should CS teachers, particularly at the secondary level, align their assessment practices with the Framework?

  • Icon for: Daniel Heck

    Daniel Heck

    Researcher
    May 15, 2017 | 04:39 p.m.

    Thank you for sharing your work on assessing computational thinking. It is addressing an important gap in a rapidly developing area of K-12 education. I will definitely look into more information on your website and am eager to see the assessment tasks.

    I'd be interested in knowing if any of your assessments have been used outside the ECS curriculum, or the extent to which you think that could be used as is, or in modified form, to assess computational thinking more generally.

  • Icon for: Daisy Rutstein

    Daisy Rutstein

    Lead Presenter
    Senior Education Researcher
    May 15, 2017 | 06:18 p.m.

    Thanks for the great question. We haven't currently used our assessment items outside of the ECS curriculum, although we have used a similar process to develop CT practice items for other projects. In our design process we did try to not align too closely with ECS in order to make our assessment items usable outside of ECS. If other curriculum are teaching similar concepts then I do believe they would find our questions useable. For example, we have an item that's on comparing two algorithms.  The algorithms we used are general and not code specific - so other curriculum that includes the concept of comparing algorithms could use that question.

     
    2
    Discussion is closed. Upvoting is no longer available

    Michelle Huynh
    Eric Snow
  • Icon for: Neil Plotnick

    Neil Plotnick

    Facilitator
    Teacher
    May 15, 2017 | 05:40 p.m.

    Assessment is easily one of the more difficult tasks that a teacher will face. I viewed the Unit-2 ECS assessment and was pleased to note that they require the students to write how they got their answers. Having students justify their thinking with writing is so much more powerful than just filling in an scantron multiple choice test. Do you have a community of practice in which teachers create and share assessments that they write?

     
    1
    Discussion is closed. Upvoting is no longer available

    Michelle Huynh
  • Icon for: Daisy Rutstein

    Daisy Rutstein

    Lead Presenter
    Senior Education Researcher
    May 15, 2017 | 06:26 p.m.

    Another great question - part of our current project is mapping out the development of an item bank. This would be a place that teachers and researchers could go to have access to CS assessment items. We are still working out details, such as how teachers or others might be able to create assessment tasks. So stay tuned!  This summer our focus will be on the development of items beyond our ECS items that we can add to the item bank.  For updates on this work you can go to pact.sri.com - once we have this work further along we will post something there.

     
    2
    Discussion is closed. Upvoting is no longer available

    Michelle Huynh
    Eric Snow
  • Omar Ruvalcaba

    Researcher
    May 16, 2017 | 02:46 a.m.

    I love that you're building this tool for teachers. Considering that you're doing this work with students of color who have been disenfranchised, I'd love to hear more about how you're considering race/culture and issues related to SES. I love the work you're doing but I would love to hear more about how you're involving community members to inform the research and how you're addressing needs specific to students of color. You have my vote! Good luck!

     
    1
    Discussion is closed. Upvoting is no longer available

    Michelle Huynh
  • Icon for: Daisy Rutstein

    Daisy Rutstein

    Lead Presenter
    Senior Education Researcher
    May 16, 2017 | 03:43 p.m.

    Thanks for voting for us!  When developing the assessments we have tried to make sure that the scenarios we picked are situations that would be familiar to a wide range of students. We have talked with some students and teachers about the situations and we are hoping to do more of that.

  • Icon for: Nicole Reitz-Larsen

    Nicole Reitz-Larsen

    Facilitator
    Educator
    May 16, 2017 | 10:56 a.m.

    Thank you for sharing your thoughts on computational thinking and being aware that there are students who have not had access to computer science in their schools.  Helping students be producers and be able to use computational thinking practices to solve a variety of problems is what will help them in school, outside of school and their career.

    It is nice that there are both paper and online tests available for students to take.  Have you noticed any differences in scores when students who haven't had a lot of access to computers take the online assessments vs paper? 

    Another question I have is what are the types of features built into the online testing that assess the students answers?  If they are particular vocabulary words, or other items - are teachers scaffolding discussions or activity wrap ups to make sure they are using those vocabularies and concepts so that students are able to naturally use such in their explanations?

     
    1
    Discussion is closed. Upvoting is no longer available

    Michelle Huynh
  • Icon for: Daisy Rutstein

    Daisy Rutstein

    Lead Presenter
    Senior Education Researcher
    May 16, 2017 | 05:02 p.m.

    We have done some initial analysis of the differences between the paper and pencil versions and the on-line versions.  We found the differences to be item specific and often there was an interaction effect with the teacher - so the differences were not the same across all of the classrooms.  Some questions in which the students performed better on the paper/pencil version had issues such as students had to scroll in the on-line environment. We are still working on addressing this so that there won't be an advantage of the paper versions.

  • Icon for: Daisy Rutstein

    Daisy Rutstein

    Lead Presenter
    Senior Education Researcher
    May 16, 2017 | 05:08 p.m.

    We do not currently have any systems in place that automatically provide feedback to the students or teachers.  However, when designing the items we did try to stay away from vocabulary terms.  If we did need to use a term that we didn't think was familiar to students we tried to support it.  In our scoring we concentrated on looking for evidence that students understood the concept -more than if they new particular terms.

  • Icon for: Nicole Reitz-Larsen

    Nicole Reitz-Larsen

    Facilitator
    Educator
    May 17, 2017 | 04:52 p.m.

    Thank you for answering questions around the two types of testing.  I appreciate the work you are doing.

  • Icon for: Lien Diaz

    Lien Diaz

    Facilitator
    Sr. Director
    May 18, 2017 | 01:08 p.m.

    I've been following the work of this project for quite some time now and it's been wonderful to see such great progress with helping to define computational thinking more clearly through these assessments, both in paper-pencil and online formats. One thing that caught my attention was Mr. Redhill's testimonial on how the knowledge that he's gained has increased his self-confidence as a teacher and has observed growth in his students by leaps and bounds. For me, this is golden --- that teacher efficacy can be increased via these assessments. Perhaps, it's in learning more about the content that the assessments are targeting and pedagogy that best supports student achievement that are factored in this type of teacher development. Does your project have more data/information on the impact of teacher knowledge gained on student achievement? If so, how is teacher knowledge of the assessments connected to how students perform on the assessments? 

  • Icon for: Neil Plotnick

    Neil Plotnick

    Facilitator
    Teacher
    May 18, 2017 | 01:22 p.m.

    Agreed! The best assessments should have two purposes. One, to allow teachers to gauge student mastery of a topic. Second, to inform teachers of the quality of their instruction.

     
    1
    Discussion is closed. Upvoting is no longer available

    Michelle Huynh
  • Icon for: Daisy Rutstein

    Daisy Rutstein

    Lead Presenter
    Senior Education Researcher
    May 19, 2017 | 02:10 p.m.

    Thanks for your comments! In our initial study we hadn't really looked at gains for the teacher knowledge. We are currently in the beginning phase of a follow-up study though that is focused more on teacher knowledge of assessment practices in general and the ECS assessments.  In this study we are going to be working with teachers to see what their needs are in terms of professional development related to assessments.  We are going to provide support for teachers through the use of PD and then follow up with teachers to see how this has helped them.  Hopefully this will give us more information on what information about the assessments can help teachers with their assessment and with their teaching.  So hopefully in the next few years we'll be able to provide more information on your question.

  • Icon for: Caitlin Martin

    Caitlin Martin

    Researcher
    May 22, 2017 | 12:15 p.m.

    This is such valuable work. Thanks for sharing. I realize that this was designed around the ESC curriculum, but it sounds like it is broad enough that it could potentially be adapted for use in other cases where CT assessments were needed. One question is that while ESC is primarily a high school course, do you think this PACT assessment could be used with younger grades (middle school, or even earlier) as this type of course is increasingly being implemented with younger students?  

     
    1
    Discussion is closed. Upvoting is no longer available

    Michelle Huynh
  • Further posting is closed as the showcase has ended.