A Walk in the Dark A look in to the mind of an RPG designer

A Plea to D&D 5th Edition Playtesters

I've been a professional computer programmer for close to 30 years, and in that time I've had my share of interaction with testers Quality Assurance. My worst experience with "testers" was when I was doing video game development, both with my Quake II add-on "PainRift" and with my Half-Life total conversion "The Opera".

Gamers are a fickle bunch... They could be pleasant and very appreciative of the wonders you create, but they can also be brutally harsh. And I can tell you that it's virtually impossible to appease all of them. Look at some of the best games in history, games like Half-Life 2 or Portal 2 that have overwhelmingly positive reviews: if you dig deep enough, you will no doubt find a handful of people that hated it with a passion. And when they hate it, oh boy they are vocal. Saying "Portal 2 sucks" isn't enough... They manage to find the time to write a veritable master's thesis full of criticism.

The thing is that criticism of that nature isn't "testing"; it's opinion. If I create a video game and someone responds "this sucks", what exactly am I to do about that? How do I make it not "suck"?

I mention all of the above because of the upcoming playtest for Dungeons and Dragons 5th Edition... or D&D Next... or D&D Whatever It's Called™. I admire WotC's ambition to try and give the power over the game's design to the fans, but this will only work if the fans give information that's actually useful.

Just to give you an idea, here are some examples of feedback I received when I released the Beta version of "The Opera":

• "Guns are broken": Mind you, the game was released with TWENTY-TWO different guns, ranging from pea shooters (Czech Duo, Walther PPK) to handcannons (Thompson Center Contender, Desert Eagle 50AE) to long guns (Winchester 1887, Remington 12-gauge). There were upwards of 30,000 lines of code written to make all these guns work... How am I to guess which line(s) of code is "broken"?
• "Diving sucks": One of the aspects of TO was the ability to move in special ways, such as making John Woo-like dives and rolls with guns akimbo. Granted, mod developers don't have 100% control over player movement (or at least we didn't back then) since most of that code is internal to the game engine, so it wasn't an exact science. It was unavoidable that there be problems with collisions and the player was bound to run in to problems such as getting stuck in world geometry (like under a table or in a window); each one of these situations had to be dealt with individually on a case by case basis, so telling us that diving "sucks" really doesn't help us narrow down and correct the problem.
• "Why didn't you include the [insert new weapon here]?": This is the worst type of feedback. No matter how big you make the game, how expansive your arsenal of weapons is, it just isn't enough because one person out there wants to see their favorite gun. And it gets worse: the theme of "The Opera" was to mimic Asian action flicks in the John Woo style of gunplay, such as the movies Hard Boiled, The Killer, etc. Although we tried to include most of the weapons seen there (the Beretta 92FS was the signature weapon, the aforementioned Thompson Contender was from Hard Boiled, etc.), people still suggested weapons that made absolutely no sense. I recall several people suggesting, quite adamantly, that we include Vulcan miniguns and rocket-propelled grenades just because they were "cool". "You know what? You should have a CHAINSAW just like Ash! That would be so awesome!" Maybe I missed something here... When did Army of Darkness join the Hong Kong blood opera genre?
• "Why didn't you include the [insert weapon here] instead of the [insert weapon here]?" Like the above, people just aren't satisfied. We included the Beretta 92FS because it was the single most common weapon in the whole HKBO genre (heck, in at least two different movies I can think of Chow Yun Fat was carrying SIX of them on his body). But Beretta makes like 50 different weapon models, each one radically different. Nobody cared that the weapon we chose fit the theme of the Hong Kong blood opera... they prefered model XXX of the Beretta because the gun is "better" in the field or more effective in combat due to increased accuracy, larger clip, less recoil, etc. I even remember one person, who took great pride in mentioning that he was a former Marine, going in to intricate technical detail over the specifications of virtually every weapon we had and commenting on its combat effectiveness and why it would be "impractical" to use a given gun in a hostile situation. I appreciate the feedback, I really do, but we're not going to turn our game in to Call of Duty: Asian Ops.

The week after we released, our forums lit up with people grilling us over what we created. We know for a fact that the majority of people liked it, but the people that liked it were nowhere near as vocal as the few that despised it with such seething hatred that they were compelled to write volumes about how bad it was. As designers, we were forced to read those long posts that assault the very thing we've spent the last two years developing... and it hurt. It demoralized a lot of us, leaving us to wonder "why are we doing this?" The hundreds of loyal fans that loved our product were drowned out by a handful of people that sought to destroy us, and it was quite devastating.

Now, once D&D WIC™ is released that level of hatred will be unavoidable, and I'm sure that the WotC forums will be virtually unreadable (I know... they're hard to read now). But none of that helps the product.

So if you want to actually help the product, provide feedback that can be actually useful and beneficial to the designers. For example:

• Be as detailed as possible about your play experience, from beginning to end: Note that I'm not talking about mechanics or about the nitty gritty here... Tell them what it felt like to play. What was fun, what wasn't fun, what worked and what didn't.
• Unless told to be, do not try to be an editor: Of all the types of criticism that you can receive, the so-called "grammar nazis" are the worst. They will hate your game with a burning passion just because you used or didn't use an Oxford comma. But don't worry: part of the design process in any publication is to hire a team of editors to go through the document with a fine-toothed comb. That editing cycle may not have happened yet, or may not have been completed yet, by the time you get the testing materials. So unless WotC explicitly tells you to look for such things, do not waste time with punctuation, spelling, sentence structure, etc... There are people far more qualified than you that will do that before the product hits the shelves.
• Be as general as possible about "game math": Do not suggest to them that an attack should create 1d8 damage instead of 1d6; that's more detail than they probably want to know at this point. Telling them that the attack causes too little or too much damage would be enough, and trust them to be able to do the math far better than you could. This playtest is not a math problem; it's a playability test to ensure that the core mechanic works. 
• Try everything, even if you don't want to: Even in 4E, there are a lot of things that exist but haven't been errata-ed simply because people don't use them. In a recent Fourthcore Deathmatch game, we discovered a problem with a Wilden racial power, which I'm imagining went unnoticed so far because, honestly, who plays as a Wilden anyway? If you want to help create the best overall product you can, be ready to try things that you may not want to. Don't like mages? Play one anyway. Only like being elves? Be a half-orc. By trying things that you don't usually do, not only do you get to experience the things you've missed out on but you can provide valuable insight as if you were a newcomer trying the game for the first time.
• Try the unexpected: In my first game, PainRift, one of the most vicious bugs we discovered was that the game would crash violently if two rockets collided in mid air. No amount of prepared test procedures would take such an event in to consideration, and it was up to a few players mucking about and actively trying to "break" the game to figure out the reason for it. So when you're playing, don't try to do everything "by the book"; try to do things that aren't documented, that aren't mentioned everywhere. Actively look for exploits and loopholes in the game mechanic so that they can be addressed now and not become a serious problem later. An effective tester is someone who not only tries everything that's documented, but tries to do the crazy stuff that nobody's thought of yet.
• Have fun: Proper testing can be a grueling experience, but a necessary one. Real game testers spend hours on end playing the same game over and over again, doing repetitive tasks trying to reproduce the same nuance in game. Thankfully, D&D allows for far more flexibility, and the nature of the game itself makes it so you don't have to reproduce a "bug" in the same manner as you would in game. So if the game starts to become "not fun", that's a problem worth reporting; make note of it, explain what chain of events made the game no longer fun, then do something radically different so that the game becomes fun again. Once you do that, also document what you changed in order to make the game fun. Knowing that information will allow the designers to make the game fun more consistantly and avoid the pitfalls that lead to a un-fun game.

I hope the above helps you give WotC the kind of information they need to make the next iteration of Dungeons and Dragons the best that it can be.

-=O=-

EDITED: Someone actually reminded me that the Winchester in my own game was a Winchester 1887 (which was used in Terminator 2), not a Winchester 1911. I confused that weapon with the Colt 1911s (which were used in Face/Off). I think the fact that someone went through the effort to point this out more than proves my point.

EDITED #2: And it's a Czech Duo... Not a Colt Duo... Screw it, I give up. 😛