Why no dedicated servers?

[xX-Kuro-Xx]

I just got the warframe bug. This game is awesome.. but the client hosts? In 2018? Its horrible. Why? We gave that stuff in the late 90s.

Every (successful) game in 2018 uses dedicated servers, because client hosting makes a laggy, clunky (migration dialogs, no reconnects), and cheat filled experience, and because flexible computing clouds like amazon ec2 and google cloud make it pretty affordable.

And its not just big names like diablo3  csgo, league of legends, or overwatch.  Indie titles that became hits also use dedicated servers, like Path of Exile, PUBG (which not long ago was unknown), and Fortnite.

Why this wacky seti@home dedicated server thing? Instead of just using amazon ec2 flexible clouds?

 

 

[TalonBlue]

$$$

[taiiat]

P2P, centralized Servers, or mixed methods all have pros and cons. one is not better than another.

furthermore the Server tasks for a PvE game featuring so much AI isn't something you can run on a single core in a box. so it's expensive. and if you don't need to pay the cost, then it's a lot easier to be profitable....

• P2P Hosting does not make for poor sessions, it makes for inconsistent ones. meaning some will be oustanding, some will be quite bad, some will be ok. in contrast to centralized Servers that all sessions are ok (not bad, not great) so long as all that are playing in the session are close to the location of that particular Server.
• neither P2P nor centralized Servers nor mixed methods are cheat filled not cheat empty/proof. all are equally susceptible and have to deploy the same security measures to protect against manipulation of the game.

lest us forget that even some of your examples do poorly in your ignorant statement of superiority. Path of Exile has simpler AI calculations(and significantly slower response time needs), and both Battlegrounds and Fortnite BR have been fighting with having DEPLORABLE session performance due to the high Playercounts. with Call of Duty Blackout in the same boat. big money games that are perfect as you suggest, except they are not because they have an offensively bad online experience because the sessions are so laggy.

Edited November 20, 2018 by taiiat

[LarryYourWaiter]

P2P is a godsend here in Australia.  Most games with dedicated servers have them over in america and some games are simply unplayable - elder scrolls online for example, is simply unplayable here.

[Fluxinated]

Cuz Beta obviously  ( ° ͜ʖ °)

[(XBOX)GearsMatrix301]

Because they literally can’t afford it.

[Loviam]

59 minutes ago, Fluxinated said:

Cuz Beta obviously  ( ° ͜ʖ °)

 

41 minutes ago, (XB1)GearsMatrix301 said:

Because they literally can’t afford it.

It's also a strictly worse system for global mmos. If we had the option to always or never host settings that would be best 

[IIDMOII]

I think the original corridor content works pretty well for p2p. Lots of coop games use p2p. This isn't an mmo nor a competitive shooter so it's understandable.

However, I think they really missed an opportunity with their newer open world areas and making use of dedicated servers there. 

Plains and Vallis would be a lot more interesting with other groups of tenno and random events. Cool boss fights etc. A more seamless transition between town and the zone.

 

That being said though, the rep gates pretty much guarantee you run out of a reason to be there after a couple bounties so ..... wouldn't make sense to pay for dedis when they gate you so hard anyways.

Imo

[Prime-Ares]

20 hours ago, (XB1)GearsMatrix301 said:

Because they literally can’t afford it.

I find that hard to believe based on marks made from JUST 2015 - 2016 Alone:

 

• Revenue for 2016 fiscal year was up 122.7% to $147 million CAD.
• Gross Profit up 174% to $95.8 million CAD.
• Registered Losers Users up 40.8% to 28.2 million from 2015.
• Avg monthly active user up 30.3% to 2.17 million.
• Concurrent user up 27.9% to 61,869.

 

Sources:

http://leyoutech.com.hk/

http://www.files.services/files/359/2017/0428/20170428164100_63579193_en.pdf

 

Errors have been made

Edited November 20, 2018 by Prime-Ares
Stopped being an idiot - researched further - found DE would never be able to support this model

[Loviam]

22 minutes ago, Prime-Ares said:

I find that hard to believe based on marks made from JUST 2015 - 2016 Alone:

 

• Revenue for 2016 fiscal year was up 122.7% to $147 million CAD.
• Gross Profit up 174% to $95.8 million CAD.
• Registered Losers Users up 40.8% to 28.2 million from 2015.
• Avg monthly active user up 30.3% to 2.17 million.
• Concurrent user up 27.9% to 61,869.

 

Sources:

http://leyoutech.com.hk/

http://www.files.services/files/359/2017/0428/20170428164100_63579193_en.pdf

So you know how much it would cost to have enough servers to host 100k+ players a day? To purchase enough servers to handle that and maintain them?

[peterc3]

22 minutes ago, Prime-Ares said:

I find that hard to believe based on marks made from JUST 2015 - 2016 Alone:

If this is hard to believe, how much would spinning up servers be globally?

[(XBOX)GearsMatrix301]

23 minutes ago, Prime-Ares said:

I find that hard to believe based on marks made from JUST 2015 - 2016 Alone:

 

• Revenue for 2016 fiscal year was up 122.7% to $147 million CAD.
• Gross Profit up 174% to $95.8 million CAD.
• Registered Losers Users up 40.8% to 28.2 million from 2015.
• Avg monthly active user up 30.3% to 2.17 million.
• Concurrent user up 27.9% to 61,869.

 

Sources:

http://leyoutech.com.hk/

http://www.files.services/files/359/2017/0428/20170428164100_63579193_en.pdf

Ok, now do the calculations on the cost of purchasing and installing a server room, as well as required maintenance.

[Loviam]

Not only the costs. But the people cost of hiring staff to maintain them 

[Beartornado]

Warframe is barely a traditional online game and is only seen as such because having a connection is necessary to even play. It has more in common with something like Borderlands than large open-world MMOs that need dedicated servers to host hundreds of concurrent users. And honestly it doesn't need them, the fact that you can play solo and dictate what connections you are matched with is more than enough to get the best experience you can more often than most AAA companies can deliver on dedicated garbage.

[kotex073]

I really don't get it, i play several hours each day and often in public groups, the times i did get a wacky host migration past year i can count on one hand,

For the times i get matched up with someone which gives me a bad ping the same.

[(XBOX)zThulsaDoomz]

5 hours ago, LarryYourWaiter said:

P2P is a godsend here in Australia.  Most games with dedicated servers have them over in america and some games are simply unplayable - elder scrolls online for example, is simply unplayable here.

Pretty unplayable on this side of the pond as well buddy

[xX-Kuro-Xx]

First, I want to point out the reason I started this thread. I'm not a teenage gamer. I'm a 44y/o very successful retired computer engineer entrepreneur and tech investor. I think Warframe has a chance to be one of the great $1B+ games, and that's really exciting. However, I don't think it's going to get there without getting rid of the P2P inconsistencies. (and a few more things, but I'll save that for another discussion)

While it may seem like posting in the forums about this is a bit Don Quixote, I like getting ideas out there into the public community and seeing what happens. 

TL;DR -

• do whatever you want in solo
• dedicated servers are rented on demand in clouds, not purchased, and are not as expensive as you think
• there are no huge revenue p2p multiplayer games in 2018
• dedicated servers create a consistent experience, dramatically increasing quality and revenue
• dedicated servers have more opportunities for anti-cheat measures, by hiding information from the client in ways you can't do when a peer is a host
• P2P quality issues include clunky NAT issues, host migration, host leave/crash, (usually) no client reconnects, lag lag lag, and cheats cheats cheats
• multiplayer games that make $30M-$1B a year all use dedicated servers, and did even when they were small (Everquest, WoW, Eve, Diablo3, Path of Exile, Starcraft2, League of Legends, PUBG, Fortnite, SMITE, etc. etc. etc. etc) .. dedicated servers are a key part of why these games are great and become very financially successful
• it's not all-or-nothing, the most important areas to deploy dedicated servers are open-world and PVP, both of which are less expensive than 4-player small instances because they handle 2x or 4x or 10x as many human players, but the instance CPU usage does not go up nearly that much.

 

Quote

P2P, centralized Servers, or mixed methods all have pros and cons. one is not better than another.

P2P Hosting does not make for poor sessions, it makes for inconsistent ones. meaning some will be outsanding, some will be quite bad, some will be ok. in contrast to centralized Servers that all sessions are ok (not bad, not great) so long as all that are playing in the session are close to the location of that particular Server.

Let's start by just admitting that when you're soloing content, it's almost always going to be better to run it all locally on your own machine.. so if the cheating surface area that exposes is acceptable, then this will always be a better experience for solo play. 

As for multiplayer, it's not the network architecture of P2P that's fundamentally bad. It's the inconsistent quality and trustworthiness of the peer hosts. In my view, the quality ranking looks something like:

your-client self-host >>  best dedicated server-host > best peer-host >> average dedicated server-host >>>>> average peer-host >>>>>>>>>>>>>>>>>>>>> worst peer-host

So I'll agree that things are the best when your-client self-hosts (i.e. when happen to be the host). However, with 4 people in a multiplayer game, three other people are not the host. So one person has slightly better than server-based experience, and three people have an experience somewhere between notably worse than dedicated servers (average peer-host), and complete garbage (worst peer host)

If you're playing from low-player-count remote regions, you might hate dedicated servers... but the problem with remote regions is not having servers close enough to you, it's having enough players close enough to you that want to play the same content at the same time. P2P can only make that "better" for the remote player by making three other people suffer the lag of a remote region and possibly a poor peer-host. It can't make other players near you want to play the same content at the same time.

Quote

furthermore the Server tasks for a PvE game featuring so much AI isn't something you can run on a single core in a box. so it's expensive. and if you don't need to pay the cost, then it's a lot easier to be profitable....

IMO, the typical outcomes we see in the game industry are more like, spend some of your revenue on dedicated server hosting costs, and make 5-100x as much because your game is consistently higher quality, with consistently less cheating. Cheating and poor quality experiences kill game popularity. Game R&D and server hosting costs grow much slower than revenue, so the more popularity and money you can get out of a high quality game, the most the dedicated server costs disappear into the noise.

I think we can agree that (a) there is cost to dedicated servers, (b) some games are more expensive to host than others, (c) hosting requires heavily optimizing the server side calculations, and (d) sometimes that involves tradeoffs in the design and implementation choices. However, games with dedicated servers also generally are much more successful, because games that have a consistently high quality experience generally are the ones with consistently higher player bases and consistently higher revenue. 
Another way to look at this, is that IMO, people look at successful games and think "of course they can afford dedicated servers, they're huge", but in fact, the correlation is often the opposite. They are huge in part because they choose to use dedicated servers (usually from the start).

The main exception to this I'm aware of is games with P2P distributed deterministic state simulation (mostly RTS games), where every client simulates the entire game in lockstep based on a fully deterministic simulation. However, even Starcraft 2 added a dedicated server to the lockstep peer-group for anti-cheat and other server side features. Also, while this architecture has some great benefits for very large numbers of AI units, it's generally only been done with 2d fixed point integer math, because subtle floating point rounding differences that occur across CPU/GPUs immediately cause desync. 

Quote

(from Loviam): So you know how much it would cost to have enough servers to host 100k+ players a day? To purchase enough servers to handle that and maintain them

One estimate says that in 2014 (4 years ago), they were on path to make $50M in annual revenue and $16M in annual profit. Extrapolate 4 years, and that's probably $150M and at least $60M. That said, they are also majority owned by a Chinese agricultural company. (weird)

Hardly anyone purchases servers anymore. With flexible computing clouds like Amazon EC2 and Google Cloud, not only do you pay by the hour for compute resources, but you can spin them up and down as needed based on demand (or lack of demand). There are even some "gaming centric" cloud providers trying to offer CPUs and networks with less jitter for game clouds. The only companies hosting their own servers are juggernauts like Blizzard and Riot. 

An instance for a game like Warframe would probably cost between $0.04 and $0.40 an hour, depending on how well optimized the code is and volume discounts on the hosting purchase. 

Let's first look at the warframe@home volunteer PvP servers. I don't know how many simultaneous PvP instances there are, but lets estimate. Steam-charts says there are ~65k average players right now. There are some not seen by steam, so let's call that ~80k. Currently PvP is not very popular, so maybe there are 200 simultaneous PvP instances (with 4v4 that's 1600 players in PvP). That's somewhere between $200/day ($70k/yr) and $2000/day ($700k/yr). Which is tiny compared to their revenue and a modest cut into profit.. so why are they leaning on the community to volunteer their CPU resources? 

The situation is very similar for open-world instances. There is no reason to be doing P2P on open-world instances, compromising both the player experience, and the type of game experiences you can create. Doing dedicated server open-world instances is more economically efficient than small instances, as you can efficiently push the player counts on an instance much higher, so you don't need as many instances to handle the players. (it's typical to have 50-500 players on dedicated servers, depending on visibility distance related issues. 

Dedicated servers for 4 player PVE instances would certainly cost a bit more, and could only be justified if they believe it will dramatically increase revenues (I think it would). If solo missions are still client side, and 25% of currently online players are running a multiplayer mission at all times, and the average multiplayer count is near 3, then today there would be around 7k simultaneous PVE 4 player instances. That would be between $2.4M and $24M a year. The latter figure is a good cut into their current profits, but I think it's an important part of getting them to $500M+ revenue a year. 

There are no more hugely successful P2P games anymore, and $500M+ online games have consistently low latency. If they want a shot at this, they need to figure out a path to consistenly low latency. This has to involve dedicated servers. It might also involve adjustments to the level AI and/or networking design.

Quote

• neither P2P nor centralized Servers nor mixed methods are cheat filled not cheat empty/proof. all are equally susceptible and have to deploy the same security measures to protect against manipulation of the game.

They are *not* all equally susceptible, because dedicated servers have more options for hiding information from the client entirely, and more options for validating what game activities are valid in a trustable context. 

For example, Xray hacks are only possible when the client knows the position of other players. Room/Portal renderers can easily send you only information for clients near you. World of Tanks/Warships goes one step further and makes this work in outdoor environments. The server does line-of-sight occlusion/visibility checks between each player and several distinct points on enemy vehicles -- and only sends clients information if those enemies are unoccluded. They have 15x15 battles, so this is pretty viable for PvP, though not viable for PvE mobs which are typically hidden from the client using simpler distance and room checks. There is *no* way to do this information hiding for a peer-host, because they are the host!

That said, I do agree that there ways to address cheating scenarios other than xray in a P2P game.... just that most P2P games don't actually use them.

Quote

lest us forget that even some of your examples do poorly in your ignorant statement of superiority. Path of Exile has simpler AI calculations(and significantly slower response time needs), and both Battlegrounds and Fortnite BR have been fighting with having DEPLORABLE session performance due to the high Playercounts. with Call of Duty Blackout in the same boat. big money games that are perfect as you suggest, except they are not because they have an offensively bad online experience because the sessions are so laggy.

I very much disagree. I have played PUBG since launch, and have played 2136 games as of today, which I estimate to be about 640 hours of play, and I think it's amazing and much better lag and quality than Warframe multiplayer. There have been some periods of acute lag early in the game development, especially when all 100 players are alive and the server tick-rate is lower, but overall it was playable, and today it's pretty darn smooth and much better than warframe P2P. PUBG also has had issues with cheating (aimbot, xray) because they don't hide most information from clients, and with a very long standing movement interpolation bug they just fixed that for a long time made lag-switching and laggy clients have an unreasonably unfair advantage. They recently fixed the latter and now things feel even better and super smooth.

Likewise for the other dedicated server games I play or have played - League of Legends, Starcraft 2, WoW, EVE, etc. etc. They are all butter smooth compared to Warframe. 

I admit it doesn't "matter as much" when there is lag in sloppy low investment PVE grinder, but I would never put any serious effort into this as a multiplayer game (pve endgame or PvP) with the current laggy online play.

Remember that I started this discussion saying I think the gameplay feels great and I love it. They got my $25 for a B+ quality single player game. That's about what it's worth in it's current state. I really wish it was an awesome multiplayer game so I could go deep, play it for a decade, and spend hundreds or thousands, like I have with LoL, EVE, WoW, etc.
 

Edited November 21, 2018 by xX-Kuro-Xx

[Heiven]

Here's DE main website https://www.digitalextremes.com/contact

Go there and fill it in and ask them directly or use the support of warframe page to ask them.Here in the forums we can all Only speculate as to why DE does the things there way,also making lecture's in the subject of servers and games is pointless at this point!

Edited November 20, 2018 by Heiven

[Andrew7811]

I get that warframe does P2P connectivity to save on server money. But if you are going to implement a P2P communication system for a game, at least do it properly. Earlier today I literally had someone decide they are done harvesting kuva 45 minutes into a kuva survival mission and leave as the host. But then host migration fails, and the game sends me back to my landing craft. This results in me losing roughly 15K kuva I accumulated throughout the mission. I would like to have the option to just go to extraction by myself if this sort of situation pops up.

[schilds]

Warframe has client-server architecture, not peer to peer. For example, peer to peer architecture would have  desynchronisation issues rather than host migrations. This is really an argument about player hosting vs dedicated hosting. I suppose you could call player hosting "peer" hosting, but let's not confuse it with peer to peer :-P.

[xX-Kuro-Xx]

21 hours ago, schilds said:

Warframe has client-server architecture, not peer to peer. For example, peer to peer architecture would have  desynchronisation issues rather than host migrations. This is really an argument about player hosting vs dedicated hosting. I suppose you could call player hosting "peer" hosting, but let's not confuse it with peer to peer :-P.

There are different uses of client-server and p2p depending on context.. Much like a chef calls a tomato a vegetable and a botanist calls it a fruit... A gamer says a game is "client server" when there is a dedicated centralized server, and we call it "peer 2 peer" when clients talk to other clients about game state and there is no centralized server.

Regarding the technical networking topology, things get even more confusing, because warframe certainly has centralized servers to manage lobby state and the like. Then, when in a game, one of the "peer clients" is elected the "game state server". So the only thing we can call "client server" is the game state topology.

-----

Suffice to say, having end-user clients host games P2P style leads to horrible average case gaming experiences. In addition to the variability of CPU performance, clients have asymmetric connections, with much less upload bandwidth than download bandwidth. They also have other users sharing the network, and unknown router issues. 

Some of you may have better experiences because if you have UPnP setup you may more often be picked as the host, and if you are the host you are basically playing on your own local server like a single player game. 

I have a (temporary) configuration issue with my xfinity modem that is preventing UPnP from working. I have not had time to fix it, and that may be contributing to me being a client more often (or always?). There may also be issues with STUN not working as well without UPnP. 

For me, at some point in nearly every multiplayer match, players and ai-mobs start blinking around at a crazy pace, presumably catching up from some kind of buffering or lag. I also sometimes shoot and shoot at a mob and it's just invulnerable until it completely disappears. (sometimes I notice it has a green glowing effect. maybe this indicates the mob is leash resetting or that my client's update information is too old?)

I've now turned off public matches by default. The single player game is much more consistenly good.. though i wish multiplayer worked better.

----

And before someone chimes in with "ohh, you have to fix your UPnP"... For *years* i've been playing (and spending money) on games like WoW, and starcraft2, and PUBG, and League of Legends with zero problems or any thought about UPnP configuration. MOST gamers don't have working UPnP, as most US providers (including comcast) ship routers that have it turned off by default.

The successful part of the multiplayer online PC gaming industry gave up on all this P2P gaming stuff somewhere around 2005. It just turns out that if you want to be a successful game, you have to isolate the issues one client may have from others, using dedicated servers. This provides consistent <150ms latency gaming for 95% of people, instead of P2P which offers <30ms for 15% of people and 300ms+ with loss and jitter for 85% of people. 

---

Warframe has mega-addicting gameplay, and the single player experience is incredible... but this "peer-hosted multiplayer" thing makes most multiplayer games horriffic.. (unless maybe you are one of the few to have UPnP configured, and win the roulette spin to be the host and you have a good game with the other 3 in your game have a potentially horrible laggy experience)

 

Edited November 22, 2018 by xX-Kuro-Xx

[(PSN)drpunk-yo]

16 hours ago, xX-Kuro-Xx said:

 

• there are no huge revenue p2p multiplayer games in 2018

 

GTA Online?

[xX-Kuro-Xx]

12 hours ago, (PS4)drpunk-yo said:

GTA Online?

1) GTA 5 is an 85M+ player single player game, only a small percentage play online, and a tiny percentage of those sales are on PC (most are on console)

2) it's not clear how many people play GTA 5 online, because there are no published player count numbers

3) players have responded to the laggy p2p experience by making a GTA 5 mod for player-run GTA-5 dedicated servers.

GTA 5 is more like the old-age style of Quake or Unreal where there was a really successful single player game that tacked on some muliplayer features so enthusiasts could play multiplayer for a while, often on player-run servers. This isn't an MMO where people play 300-10,000 hours on the MMO.

An MMO (especially a free to play MMO) makes money by having a huge and growing active playerbase that plays for hundreds or thousands of hours, not selling 85M people a game that they play for 100 hours and then get ready to buy the next installment. 

Maybe this is what Warframe is shooting for, a 2000-era single player game with multiplayer features tacked on. In this case, it would probably be better to remove public auto-match and all this horrible peer-hosted gameplay, and just let people play with friends or run their own dedicated servers for PvP and other specific purposes. 

However, if they want a shot at the huge money PVE MMO gravy train (and a great PVE MMO gaming experience), they'll need to move up a couple decades and recognize that a big part of the MMO/PvP gravy train that's been going on since 2005 is all using dedicated server. It's not like this is a guarantee, but it's a big part of every MMO success im aware of. 

 

Edited November 22, 2018 by xX-Kuro-Xx

[(PSN)Cargan2016]

On 2018-11-19 at 6:39 PM, xX-Kuro-Xx said:

I just got the warframe bug. This game is awesome.. but the client hosts? In 2018? Its horrible. Why? We gave that stuff in the late 90s.

Every (successful) game in 2018 uses dedicated servers, because client hosting makes a laggy, clunky (migration dialogs, no reconnects), and cheat filled experience, and because flexible computing clouds like amazon ec2 and google cloud make it pretty affordable.

And its not just big names like diablo3  csgo, league of legends, or overwatch.  Indie titles that became hits also use dedicated servers, like Path of Exile, PUBG (which not long ago was unknown), and Fortnite.

Why this wacky seti@home dedicated server thing? Instead of just using amazon ec2 flexible clouds?

 

 

 

[(PSN)Cargan2016]

On 2018-11-19 at 7:22 PM, (XB1)GearsMatrix301 said:

Because they literally can’t afford it.

they couldnt afford it when it came out originally they where very close to going out of business Warframe was their last shot at staying open.  Now how ever it will be a challenge to migrate it over to dedicated servers and keep content rolling out at halfway reasonable rate at same time.