Wednesday, February 3, 2010

UPDATE: Star Trek Online Adding More Servers, But Open Beta Bugs Still Persist

Development continues on one of the most anticipated MMOs

Developing and testing a video game is hard work, with some of the biggest games employing dozens or even hundreds of testers to report programming bugs during a beta testing program. The task is made even harder by Massive Multiplayer Online (MMO) games, which generally function in an open world with millions of variables on dozens of servers. Sometimes problems don't crop up until stress testing is conducted through software algorithms or real players going through the game.

For this reason, most MMOs today employ a beta program that is open to the public. Star Trek Online continued that trend with developer Cryptic Studios and its own beta program. Things started off with a Closed Beta on October 22 last year that was limited to several thousand players in order to find and fix the most common critical bugs. STO transitioned from Closed Beta to Open Beta on January 12, and recently ended the Open Beta program on January 26.

Most video games today ship with multiple bugs, as a consequence of the strict deadlines in the industry. Ship dates for games are set months in advance, and developers and publishers can face hefty penalties for missing those dates. As a result, many games ship with bugs that are fixed with patches or expansion packs. This may save a company millions, but also means that customer satisfaction could take a setback amongst influential early adopters.

MMOs, on the other hand, typically go through a process of continual improvement, not only on the server side, but on the user's client side as well. A subscription model allows for permanent staff to work on game play optimization for usage models that are unanticipated. Certain levels or scenarios may grow more popular over time, or exploits developed that must be closed.

DailyTech tried out Star Trek Online during the Open Beta period, and noticed a marked improvement in gameplay during the two week period. Load times and the number of critical bugs decreased during this time, even with the number of players growing constantly. Publisher Atari gave away over 100,000 Open Beta keys to its partners, such as NVIDIA and gaming websites. Over half a million bug reports were generated by the end of Open Beta through the in-game bug reporting system.

However, there are multiple bugs that still exist. Not all of these are critical, but do affect game play. One bug that kept popping up was during the transition from a ground environment to the space environment. The player's avatar is supposed to change from a humanoid to a space ship, but sometimes it doesn’t quite happen (see picture).

There are also some problems with Mission Rewards, in which a Bridge Officer or in-game item is supposed to be added to a player's inventory but isn't. In many cases it is just a matter of players not looking in the right place, but this is an issue that we have personally witnessed. In some cases, Bridge Officers with the wrong ability set are added instead.

Perhaps the most critical bug left is with the in-game cursor, which was completely invisible when I first started playing the game. I had to switch to a windowed mode and select a software cursor in the troubleshooting settings. This is a problem that has still not been fixed in the company's Head Start program, which allows customers who have pre-ordered the game to begin playing ahead of Tuesday's official launch.

Pre-orders are an important way for a company to gauge interest in a game. If a game is doing well before launch, a company may decide to purchase additional servers in order to meet anticipated demand. Most game developers bear a large upfront cost for game development over several years, and often don’t want to invest too much in hardware in case a game isn't as successful as hoped.

Unfortunately, the interest in Star Trek Online seems to have caught Cryptic off-guard, despite the fan base of several hundred million people. The company announced that it was ordering new servers during Open Beta, but dropped connections are still quite common. There are often problems logging in, retrieving characters, and loading maps. STO servers have been down for "emergency maintenance" at least a dozen times during the last week as more gamers have started playing.

The good news is that it means the company is working on it, and there have been several client patches to address numerous issues. We often forget that even the mighty World of Warcraft had numerous bugs at launch, and it took months to fix the most glaring bugs. Ultimately, long-term subscribers will wait it out, but others who are watching may not have the patience.

Update Feb 3:

We have received the following from Cryptic Studios:"We just wanted to update you on our in-game server capacity status. Throughout Head Start we noticed the overwhelming response to the game and we are working on changes to further increase the number of people who can play Star Trek Online at any given time. Our current plan is to have these changes in place sometime this coming weekend. We will, of course, keep you updated on progress as it happens".