Using Feedback to Improve

For our recent playtest with ReBoot: A Computational Thinking Quest, I had the opportunity to observe a second-year teacher, Emma, as she played through our prototype. Emma teaches elementary students and is right in the thick of designing lessons, solving classroom challenges, and growing in confidence with instructional strategies, making her a perfect fit for testing our game. Since ReBoot is designed to help pre-service and early-career teachers learn and practice computational thinking in a supportive, narrative-based environment, her perspective was exactly what we needed.

We conducted the playtest synchronously, as I work with Emma. She played the game while I observed her clicks, pacing, and decision-making, then followed up with a structured reflection. The entire session lasted about 30 minutes, including playtime and debrief. Our focus during this playtest session was on instructional clarity, core mechanic usability, narrative engagement, and emotional experience — all through the lens of a new teacher. I wanted to know: Could she follow the game’s instructions? Did the story help her stay engaged? Were the CT strategies relevant to her actual classroom experience? We captured the data using a structured interview. 

**

Playtester: Emma – 2nd year teacher

As a player, what was your purpose in this game, and how clear was that purpose from the beginning? Did it feel like a role you might have to take on as a teacher? (Game Instructions & Core Dynamic & Narrative)

I understood fairly quickly that the purpose of the game was to help fix things at Spark School by completing challenges that would repair the learning environment. I felt like a problem solver. It definitely felt like a role I would take on as a teacher, especially when BYTE mentioned things being out of balance or broken – it reminded me of when lessons don’t go the way we plan, and we have to troubleshoot in real time. I liked that I was framed as a helper and not just a passive player. 

Were there any moments when you felt confused about what to do next? If so, when did that happen, and how did you figure it out? (Instructions & Mechanics & Feedback/Usability)

There were a couple of moments where I wasn’t totally sure what to do next. Though, I think I figured out if I explore around the room, there are either visual cues to tell me something is happening. BYTE was helpful overall, though.

How easy or hard was it to understand and use the main actions in the game (like solving puzzles, making choices, interacting with tasks)? (Mechanics & Usability)

The main actions were pretty easy to pick up – solve the puzzles and choose where to go next. I didn’t feel overwhelmed or lost much with what to do. One thing I might suggest is that the skill-selection screen could have had a little more animation or guidance, maybe for the hallway, until the player gets the hang of the game. But overall, everything felt pretty intuitive once I got started.

If you could play more levels of this game, what skills, strategies, or knowledge do you think you would be building – and would those feel useful to you as a newer teacher? (Learning Objectives & Assessment)

I think if I played more levels, I’d get better at identifying and applying problem-solving strategies that mirror computational thinking – like breaking down problems or recognizing patterns. These are super relevant to teaching, especially when planning lessons or managing a classroom. It was cool to see those skills embedded in a game setting because it didn’t feel forced – it felt natural and useful.

How clear was it when you were succeeding, winning, or completing parts of the game? Was anything about progressing or reaching goals confusing? (Win/Lose Conditions & Feedback)

I generally knew when I had succeeded – like after solving a challenge, the you did it badge popped out. Wondering about the classroom, though. Maybe some small celebration or positive reinforcement would’ve made me feel more confident.

What part of the game did you enjoy the most, and what part (if any) was frustrating or less fun, especially thinking about it from a teacher’s perspective? (Usability & Enjoyment)

I enjoyed the storyline and the idea that I was helping fix something meaningful. BYTE felt like a supportive guide, and the clean design helped me focus. If I had to pick a frustrating moment, it was just the occasional uncertainty of “what now?” after a task – but it wasn’t enough to stop me from wanting to keep playing. From a teacher’s perspective, I appreciated how it modeled flexible thinking and calm problem-solving.

Did the skills or thinking strategies used in the game feel realistic for what a new teacher actually needs to be successful? (Connecting to a teaching lens) 

Yes, I actually thought they were fairly realistic. The whole idea of breaking down a problem and noticing patterns felt exactly like what I have to do every day as a teacher. The game didn’t feel too theoretical, which I appreciated – it showed practical ways of thinking I wish I had been more aware of in my first year.

Did you ever feel overloaded with information, steps, or choices? If so, when? (Checking cognitive load) 

Not really overloaded – it felt manageable. But there were a couple of points, like after completing a puzzle, where I was waiting for what to do next and wasn’t sure if I missed something. More being uncertain than overwhelmed.

**

Emma was the only playtester for this session, which allowed me to really dive deep into her responses. If I were to repeat this playtest, I’d love to gather a small group of 2–3 early-career teachers to compare insights and see where experiences converge or diverge. It would be interesting to hear how they confer with one another as they play. Technically, the session ran smoothly, and it was fun to have a colleague play our game. Emma provided thoughtful feedback and even unprompted ideas for improvement. As a designer, it was validating to see her grasp the core loop of the game and connect the thinking strategies to her real teaching experience. It was also helpful to dialogue about her role as the player. Through the play testing, I discovered some small holes we could easily fill early in the game to better drive the purpose of the game. I asked Emma if in our limited prototype that she was learning strategies to embed in her classroom. She indicated it was partially clear. When I explained the rest of the game (not prototyped), she suggested potentially adding something to the intro video or BYTE’s responses in the hallway to help teachers know that it was not only to help them be a better teacher but also to embed in lessons so students learn the skills as well. Big picture, players become designers of CT, but hearing how it wasn’t clear at the beginning opened up possibilities to strengthen the narrative early on. 

Key Takeaways: 

• Mechanics were mostly intuitive, though some moments after tasks ended felt a bit ambiguous. 
• Feedback and progress cues could be stronger – players wanted small wins or celebratory moments after solving problems, especially in classroom.
• CT strategies were perceived as realistic and useful for classroom application. However, early narrative framing for teachers as designers is needed. 
• BYTE is solid as a mentor figure.

What I learned: 

Playtesting with the intended audience is invaluable! Emma’s small moments of hesitation helped me notice where the game might need visual nudges or clearer feedback. When she didn't immediately connect that it was more than just practicing the skills as a teacher, I knew we had some work to do on that narrative. I also learned that new teachers crave validation. They want to know when they’re doing well, and ReBoot could easily offer more explicit support. If I were to improve the playtest process next time, I would probably record (with permission) the play test. It would be helpful to see facial expressions, document hesitations, and not just trust my scribbled notes for the interview. It might also be helpful to create a moment tracker to jot down timestamps where key moments of confusion or delight occurred while playing. 

I also learned the power of prototyping all phases of play. Only when I pushed in questioning did I uncover the glitch in the narrative. And not that it is a huge glitch, but even the smallest shift in the intro video or BYTE's responses early on could strengthen the game. 

How We’ll Use This Data

Emma’s feedback may directly influence the following: 

1. Improving transitions between challenges, specifically in the classrooms. 
2. Enhancing success feedback with micro-celebrations – especially in the classrooms. 
3. Making skill selection screens maybe more animated or guided 
4. Early framing in the narrative that player are not just restoring a broken system, but also preparing to become a designer of learning.

Overall, this playtest confirmed that ReBoot is moving in the right direction. The story, purpose, and educational value are coming through. With a few usability refinements, feedback enhancements, and some better framing of early narrative, the game can better support new teachers as they build confidence in applying computational thinking strategies in real classrooms.