Rob’s Playtest Feedback process

My Playtest Feedback Process

by Rob Heinsoo

I’m just about to start going through playtest feedback for 13th Age in Glorantha. I thought readers of this blog might be interested in how I process playtest feedback for 13th Age books.
Sometimes I read playtest feedback right away. But usually I wait and read as much of it as possible in a single big batch. Glorantha’s first playtest is going to take the big batch approach.

In either case, I take the good ideas I like out of it, or notes that seem to be identifying major problems, and write them down in my own words in single sentence summaries, sometimes noted as to whose feedback they came from. I keep these notebook pages of possible playtest changes going through the entire process. (I write small so I can fit a lot on a two page spread!)

When I’m ready to implement the changes, I start by reading the whole list of possible changes. After crossing off notes that have proven incorrect, I start in and work through the notebook pages list, crossing notes off as I deal with them or decide they aren’t actually problems. How do I decide when comments aren’t problems? A few ways, but mostly through uncovering that the rest of the feedback supports a feature a couple people found problematic, or discovering that the original comments were in fact inaccurate, or by creating new design elements that sidestep the issue, or by weighing the evidence and judging that what bothered the tester is a feature instead of a bug!

Sometimes I’ll get playtest advice that’s so good, accurate, and important that I want to make changes immediately. That happens most often during playtest feedback on classes, when something sparks that can fix a lingering problem or create a wonderful new dynamic.

In most cases, it’s better to wait a few days or weeks longer and make changes in one thoughtful extended pass, because even small changes can require multiple revisions scattered throughout the document. Revising the same sections multiple times because of repeated changes is not only maddening, it also seems to increase the risk of me screwing up a change that should have rippled out to multiple pages of the book.

I suspect that other designers handle playtest feedback differently. But I admit that I’m not sure. I haven’t asked many other designers how they handle the playtest revision process with RPGs.

Here’s a picture of what a typical page of playtest process looks like in my notebooks. These were notes from last year on Robin’s The Strangling Sea.

Yes, I’m still writing in notebooks. When I’m rolling with design work I’m usually just typing into a computer, but when I’m noodling ideas or writing notes about things I want to think about before acting on, I use a pen.

And while I’m taking photos, here’s the pile of all the notebooks I’ve used for 13th Age design. They’re all from my friend Sara’s company, MakeMyNotebook.com, I love the weight of the paper and their spiral-bound durability as well as the fun covers. I’ve used one full book already for 13th Age in Glorantha (blue robot) and it looks like I’ll use up at least another half (black fish).

(This was previously posted on Rob’s personal blog, robheinsoo.blogspot.com)