The Launch is the first impression most people will have with the game. If people know you pushed back the release date because you wanted to fix something that beta testers found not fun I’m sure they would be more understanding. I know it takes time and money. Investors need to understand that what you are fixing will make a better Launch and will make it so people will stick around longer in the game.

I’ve yet to see a beta that had a comprehensive, consistent and fully used plan to coordinate the information needs of the developers and the recognition and response needs of the testers. Those two sets of needs, after all, are mutually entangled.

While some individual developers can and do maintain excellent communication channels with testers, the betas I have participated in have usually suffered from an us/them dynamic that has closed channels of communication and discounted community understanding, intelligence and viable feedback (from the company side) and has evoked frustration, anger, bad behavior and negativity from the community side. The result? The company does not get the information it needs when it needs it. The community of testers becomes confused as to what is needed from them, at best, or clearly antagonistic and destructive, at worst.

In the ten years since EQ1 was beta’d, there has been ample opportunity to study the results or lack of results from both beta testing and playtesting (which too often seem to be conjoined in such a way as to make the feedback and bugs coming in less useable) and to use those results to better plan and design beta tests. However, honestly, I don’t see any real step forward in game developer understanding of how to structure a beta test for the specific results needed by the game company and the responses needed to keep a community of testers involved and progressing, and reasonably positive.

I’ve also not experienced a beta where there was a specific person at a high enough responsibility level to manage the beta from start to finish. Many betas in the past decade have suffered either from territorial politics within the company that kept information from getting to where it needed to go or from inconsistent communication from the game company to the testers and back. A person who manages the beta needs to be a neutral presence in the company (not someone beholden directly to designers or artists or engineers or marketers). However, a beta manager should also never be a doorkeeper. There would be too many ego and abuse of power issues involved. A beta manager position should also not be under the auspices of customer service or community management. An effective beta test should have a coordinator who knows the game inside and out and is able to understand the issues raised and correctly direct information to and from developers and testers.

It might be fun to have a beta design competition for a game that is not real. The designer would have to not only plan for the information needed for the artists, engineers, designers, marketing and community development needs, but also would need to come up with a target group of beta testers with specific enough characteristics to try to integrate.

I’d really like to be part of beta and/or playtest where I felt confident that the developers were asking the right questions, were clearly letting me know what goals needed to be met at a particular stage of the beta, and were recognizing fairly and respectfully the contributions of the testers.

Just a comment on the “fun” factor. When EQ1 went into beta 1, there was almost no content. You could drown in Qeynos. You could kill some rats. But it was fun. Why was it fun? First, because the world itself, experienced from a very small available playing area, evoked a psychological response from players that the world was “real” and inviting. That is, players knew from the beginning that their imagination and intellect were going to be securely engaged in play. Second, because enough communication channels were in place that the free dynamic of the community added to the fixed dynamic of the coded game created an explosion of immersion and involvement by players even though the full experience of Norrath was months away.

There have been a fair number of games recently that large enough numbers of people have not found fun, however, that perhaps adding in an evaluation of early immersion and community communication in beta could assist in assessment of whether or not a game is fun even in its most primitive form. Is it possible to tell very early on whether the world can successfully imprint itself on players so that they will, in turn, make it come alive and, in the end, have fun? It’d be worth finding out.

Finding out what works, where you can improve, and if your product is worth investing the time (both financial ROI and potential for virility) to make it gold.

Excellent post. You really should make some more often.

To expand on issues within the Engineer role a bit, I’ve often found that to hit milestones we’ve got to get things up and running NOW (or at least look like they’re running). Of course all of that undocumented and poorly laid out code ends up being refactored and cleaned up “someday”. So we end up heading into production with code that was never meant to be anything more than a prototype or tech test. That usually bites us in the ass…

The lesson I think, get something up and running right now with a small team both from an engineering and a design standpoint. Doesn’t have to look good, doesn’t have to be easy to use but it should embody the core principles of your game. Then budget both from a time and resources standpoint assuming that you’ve have to completely start over. You’re not actually starting over mind you, you’ve learned some valuable lessons. But ditch the code and revisit all your code and design decisions to see what worked and what didn’t in a wholesale manner. Don’t just keep something crappy because 4 other systems depend on it. That applies to engineering and design, now is the time to change things.

And that’s the sound of the beancounters dropping dead from heart attacks. The thing is it doesn’t really cost you all that much more in the long run since you’ve got less gristing and wasted work taking place at the end of development where your burn rate is 10+ times what it was. And you’ll often end up with a superior product.

The original purpose from the software field was to have the application battle-tested by real users under something approaching the actual situations it would be used. For a game, that meant letting real gamers in to play the game and see how the game fares.

In most cases, however, if there’s a problem it’s too late to actually do anything about it. A launch date has been set and boxes are shipped to the store, so even something like server instability at higher populations wasn’t something that could stop the machinery once it was set in motion. Therefore, the last Beta period became more about marketing than actual testing, because any bugs found were probably going to be in the final version anyway.

As far as betas ruining people from playing the game, there is some effect that I’ve seen first hand. First, if someone really goes in and tests a game throughly, there is the possibility of burning out on the game before it launches. Testing is necessary, but it’s not really all that fun. So, being asked to repeatedly test something over and over again can burn out even the most enthusiastic supporter; I saw this happen when we relaunched Meridian 59. The other thing that can happen is that people decide they want to play the game, so they don’t play the beta anymore because they don’t want to “waste time”. This can mean you don’t get good testing, or good word-of mouth, or possibly even enough people to make the world feel truly populated.

Anyway, there’s my take on things. Nice to see you updating again, finally.

Glad you did it and I’m happy to see an article from you!