Mega Server Tech and Player Communities

Belghast has sparked a discussion in light of the fiasco that is the Archeage (AA) launch. Everyone who’s tried to play it the past week have met the day-long queues. I think it’s a travesty. It seems clear they didn’t build a server system suited to the task.

AA is using a traditional MMO set-up (multiple, disconnected servers). It has features that makes server modifications extremely difficult and inflexible. AA has fewer servers than it needs for launch and because of the rigid server infrastructure, they can’t just add new ones. They’ve literally developed the game on a server platform that cannot possibly meet the game’s needs. Talk about painting oneself into a corner, XL and Trion have done just that. So what exactly is the problem? There’s two key issues making the game lame and unplayable.

Labor Points and Land

Labor points are the lynchpin in the game. They are the pivot that everything hinges on. This system CANNOT be altered without sending tsunami waves rampaging through every aspect of gameplay. LP are a kind of currency that players earn and which we spend on things like quests, crafting, and opening loot. Yes, you read that correctly: in order to open the loot on the enemies you’ve killed you have to pay a fee. You’ve also got to pay to report bots, abuse and other violations to game rules. LPs are crucial to the gameplay experience. You can’t play the game without them.

To earn LP all you have to do is be online. For Free-to-play (F2P) customers, online presence is the primary way to earn LP. You get them automatically every 5 minutes that you’re logged in. This means that optimizing your character requires that you never logout. Obviously this encourages AFK’ing and botting. Bots and AFK’ers are currently dictating the server schedule. They decide if the servers will be restarted. They decide if Trion must add new servers. Trion makes decisions based almost solely on their behavior right now. This gives new meaning to the phrase “the patients are running the asylum”.

There are other ways to earn LP, like buying things from the item shop with gold and/or real money. LP is also capped for non-paying customers at 2000. The goal of players is to make our time spent efficient. This means we prefer to earn it than to spend any kind of money on it, even in-game currency.

Due to the incentive to never logout and the dedication of AFK’ers to advancing their character, server queues are massive. It’s a problem that’s gorging on itself. As players try to stay online to earn LP, they’re even more likely to AFK for fear of not getting back online due to queues. It’s a problem that gets bigger the longer it persists. This is magnified during weekend periods, but those queues are still there as of this Wednesday morning. I suspect queues will be massive this coming weekend despite the updates Trion have been making.

The current implementation of LP is killing the game. Because of the way LP works, it’s causing self-perpetuating queues. But because of the way land and housing works, it’s preventing Trion and XL from alleviating those queues.

Players can own land in the game. In a game where land-ownership in the main world is available, the game economy depends greatly on stable populations. This means Trion can’t just add new servers to address the queues, because if they launch too many, those servers won’t have a strong population. This also means they can’t merge servers without instancing land, thus destroying the economy along with a dozen gameplay incentives (land value, land conquest, castle sieges, etc).

That’s 2 critical game features that are preventing players from playing the game in a normal way. So how should they have handled this? And how will this impact the game and population in the future?

Megaservers

First, I’ll acknowledge that megaservers aren’t a silver bullet. There’s no silver bullet. Optimal server tech that any given MMO requires will depend on the features of that game. For sandbox games, we have models all around us, showing us their strengths and weaknesses. The core and most important thing to keep in mind for sandbox games is that players are the primary content. This is true for MMOs in general, but the life and death of a game hinges on this principle for sandboxes. Players create the content. And when your primary content is something as dynamic as a human player, megaservers will always be a better solution because they allow the game to dynamically expand and contract based on player activity.

The fewer pre-made landmarks in the world, the more of a sandbox the game is and the more it will require a server solution that’s flexible. EVE Online is a great example of an MMO that strikes a great balance.

New Eden has dozens of pre-made stations (cities) and regions (continents) with NPCs who have their own lives, institutions and property. The player can be friends or enemies with them, but they can’t remove them from the world. Those are permanent features. But then New Eden offers the players a vast sandbox to modify the topography of the game. Player institutions hold immense power and importance. Locations become strategically valuable. Alliances with certain NPCs reinforce power. In this sandbox the devs give players an infrastructure, a skeleton, that we then modify to create the kind of game we have today. EVE also uses megaserver technology. Every player is on the same shard. This means the game scales extremely well. When the playerbase gets smaller, nothing changes and when it grows, nothing changes. The game accommodates small populations and large ones equally well, generally speaking (small hardware adjustments can be handled in hours, if not minutes most of the time). EVE has it’s own technical issues and I’m not sure how well their set-up scales with a game with a much more sizable population (say WoW’s population). But one thing’s for sure: they have the right set-up to make a playable sandbox for their game.

AA scales extremely poorly. Any drop in player population brings the prospect of server mergers, which would kill the game. Any expansion of the population causes server instability. For AA, their sandbox is far too small but it can’t be made larger without significant downtime, delays, and big changes to the servers. Sandboxes require a dynamic and vast game space, an environment that creates the illusion of an infinite frontier along with the quantity of land to make it real. Archeage is shaping up to be a gimmick. There’s no infinite frontier (the land is extremely finite, there are definite good locations and bad locations which can’t be made better/worse through player actions unlike EVE Online). They’ve advertised something and delivered only a shadow of what was promised.

The traditional server set-up comes with traditional problems.

  • The only way to respond to sudden over-population is new servers.
  • The only way to respond to low-population is server mergers.
  • The only way to merge servers in a game where players own the land is by instancing that land.
  • If AA instances the land, it will break the game.

AA is in a really bad spot.

Trion’s current solution is attrition: wait it out. Let the players who get fed up with queues quit and soon enough the playerbase will shrink to a size they can manage. Wait until the population hits whatever magic number they’ve written down as “the playerbase we expect to have in 3 months”. That’s one way to handle it, sure. But in the long term they have to make some really hard decisions which will definitely change the game.

  • Remove LP earnings while online and implement a different method to earn them.
  • Develop new server technology so that it scales well with population increases/decreases.
  • Develop smart queueing systems
  • Create more specialized servers

As long as players stand to lose progress by logging out, AFK’ing and botting will continue. The two are never going away anyway, but the game shouldn’t make it profitable.

There’s many examples of server technology which scales better than what they have. Star Wars Galaxies did it 10 years ago. The Secret World has a system, Elder Scrolls Online, and of course EVE. This isn’t new stuff. There are solutions to their server problem and only god knows why they chose the current set-up.

An MMO needs a smart queuing system. It needs to be able to handle disconnects in a smart way, and also needs to tell the player exactly what’s going on so that we can plan our play time accordingly. For example, logging in should give you a report on the status of any given server. Perhaps Kyrios is over-crowded. A smart queue would make recommendations, tell the player when peak hours are for any given server. Inform the player of when maintenance and restarts are before they queue. Make it impossible to lose your position in a queue.

Specialized servers would probably have a big impact on server stability. Create Patron servers if necessary, but there’s other kinds of servers too. Create “flood” servers that allow players a grace period to freely transfer to a different server when there’s room. Flood servers can safely be shutdown because no one can buy land or houses on them. It just gives players a way to get into the game, start earning LP and leveling while they wait for the server of their choice to expand.

There are solutions to Trions problem. The only question is whether they’ll take advantage of them or stick with their guns. Right now, they have no intention doing anything except wait for players to quit. That’s bad for business and really bad for a game that requires a strong population.

9 thoughts on “Mega Server Tech and Player Communities

  1. No opinion on the size of the F2P crowd impacting this or is the incentive currently there enough that Trion is just going to wait? Looking at Rift (SWTOR and Wildstar too), this may be smarter to live with queues until 8 weeks out, then re-assess. It’s certainly an interesting approach.

    • If that’s what’s going on, and it appears that it is, seems to cross ethical lines. They took money, made promises, and didn’t deliver. The head-start. Many players didn’t get their head start due to queues. I’m sure there were requests for refund and maybe those players are getting their refund. But they’re not exactly solving the problem, just shoving players out of the door in the hopes that their 6 servers per region plan will pan out exactly as they need it to. I think this situation is changing whatever numbers they believe they’ll have in 3 months.

      LP must be reworked. I’m pretty confident they’re going to do it eventually too. I don’t think AA’s future is too bright right now, because they’re doing so much self-inflicted damage on a launch.The reputation they’re getting is going to stick enough to ruin their 3-month charts.

      • Interesting I say. Interesting.

        If you would, can you explain how you see the difference between the F2P and P2P crows in AA? P2P, as I see it currently, get the following: LP while offline (helps queues), Queue priority (doesn’t help queues) and Land Plot (arguably the largest queue impact for start.)

        I’m confused as to the ethical aspect. The people who pay money get to play, the ones who don’t, wait in line. Are the queues for paying members not faster than most MMOs at launch?

        • It’s hard to explain because my own opinions have some conflict.

          On the one hand, I don’t think there’s anything wrong with charging players fees or subscriptions and offering them bonuses. On the other hand, when those bonuses give an advantage, we’re in pay-2-win territory. The economy isn’t some tiny part of the gameplay here. It’s the heart of it. And LP is the heart of that. AA is already P2W.

          Ethically? Trion took money for a service they didn’t deliver (headstart). And they knew the consequences. Ever hear the phrase “easier to ask forgiveness than permission”? That’s exactly what they just did.

  2. I think you’re being overly harsh on the current estimated problem level, but I guess that we’ll see how it goes in the next few months. I have a feeling that the Warlords release will have more of an impact than anything Trion does or does not do in the near future.

    • I don’t think I’m being too harsh, calling the situation for what it is. They’re in a really bad spot with seemingly no solution except the wait. I’m sure they plan to expand existing servers in some kind of way and maybe in a week or two they’ll have things under control. But I don’t think it’s harsh to say their game systems are causing them the most serious problems. They’re blaming players for playing the game as it’s designed to be played.

      LPs have to change. I think they’re making sure Warlords is even more successful, because I believe if they’d had a stronger launch with a much better execution of LP, they wouldn’t do nearly as bad as they’re about to.

      • I don’t think saying that it’s shaping up to be a gimmick, or that it is in a very bad spot right now, is merely calling it for what it is. You’re right that they have to work overtime to smooth things over with the playerbase, but how bad it hits them and their reputation is going to depend a lot on how many paying players they alienate. And I am not sure that it is possible to predict what impact a bad launch will have on a F2P game (i.e. even if it is as bad as you assert, I think you might be overestimating how much that will damage the game’s future). It’s quite possible that the majority of players who leave will not be paying players. I see a distinct possibility of a die-off of F2P players for a couple of months, while Trion sorts things out, then a gradual return of those same players. Reputation is a poor indicator of a F2P population, I think.

        • Well, I see your point about that particular harsh statement (gimmick), but that’s kinda how it’s played out. They made promises and knew they couldn’t keep them. Maybe “stunt” is a better word, but I used gimmick because this open world they’re building isn’t really open. The server structure makes it as static as WoW. There will be a dozen unique economies, some doing well, some not so well and then the whole promise of “own land in the open world!” starts to feel something like Garrisons in Warlords. Feels gimmicky.

          Of course, games can recover and you may be right. I’ll be glad to be wrong in a few months. But the ONLY way F2P games survive is by that great blob of masses who do not pay to play (http://xpup.me/1DFtpTX). If the majority of their players are paying players, we have a lot more to be afraid of than even this article let’s on. So let’s hope they don’t alienate those free players.

Join the fray, speak your mind, and follow on Twitter @trredskies!

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s