Strict Nat: Improvements And Testing

[[DE]Glen]

Today we enabled some experimental code that works to solve problems caused by Strict NAT and other routing problems; it still needs work but our preliminary tests have us very excited.

 

About 6% of our players have routers that enforce Strict NAT; these people get a warning message when they log in that looks like this:

 

 

 

 

This is a networking problem that impacts all kinds of games including games on consoles; the unusual thing about Warframe is that we actually detect this problem and warn you about it so you can fix it. Unfortunately non-technical people often get the misimpression that there is something wrong with Warframe that we need to fix.

 

Some people may not even be able to fix their NAT but the nice thing about detecting this problem is that we can intelligently avoid most of the issues that it can cause.

 

You see, Strict NAT is like a recessive gene that causes a horrible disease -- you can be a carrier but not show any symptoms; it only causes problems when you mate (or ‘matchmake’) with someone else who is also a carrier.

 

Because Warframe knows who has Strict NAT all we have to do is avoid matching people together that it knows will have connection problems. This works fairly well until:

 

 

 

 

This only happens to around 0.36% of invitations (6% of 6%) but for a game the size of Warframe that tiny percentage works out to be a lot of very angry people who just want to play together.

 

Sometimes people are behind multiple layers of NAT which is becoming increasingly common as the world runs out of IPv4 addresses (ironically the switch to IPv6 can make matters worse because the mixture of v4 and v6 adds another layer of NAT at the ISP where you cannot use UPnP, NAT-PMP, or forward ports manually).

 

In cases like these our network analysis is tricked into thinking everything is rainbows and unicorns but when you try to join your friend you get:

 

 

 

 

When most people see this message they just assume that we have a bug in our netcode that needs to be fixed; unfortunately in these situations we are at the mercy of The Internet; see RFC5128 -- like Strict NAT, there are problems with the network at large that we just cannot fix.

 

The only solution is to use proxy servers: if the network does not allow a P2P connection our new code now steps in and acts as an intermediary. This increases latency (sometimes referred to as “lag”) but in a lot of cases suffering some extra latency is much preferred to not being able to play with your friend at all.

 

Last weekend, with the help of some international volunteers, we tested a few worst-case scenarios; even with the worst lag possible the system still worked. In fact many of the games I played on the weekend were through the proxy and even though this was adding at least 80ms of latency to my games I could barely notice it!

 

Today we did some more live tests and after fixing a few bugs we opened up this feature to the public and waited; much to our relief we saw the server start taking on more and more proxy duties until it reached a steady load that was nowhere near the capacity of the server. All systems were definitely nominal!

 

It is worth noting that this should not only solve a lot of Strict NAT problems but also a number of other problems that cause players to fail to join -- it’s a fail-safe system we can use as a last resort of everything goes pear-shaped.

 

Things have run so well that we’re going to leave the experimental code running for the foreseeable future; we still have work to do, though: the next step will be to set up a load balancing system so that we can set up proxies in various places around the globe to minimize the overhead of using them (the speed of light is a harsh mistress).

 

We are also going to leave the network analysis warning on login; although we can cope with more problems like this it’s generally in your best interest to fix the problem yourself because this will give you the best online experience possible.

 

A lot of you will be pleased to simply see we are working on Strict NAT even if it doesn’t affect you, and for those who just want the TL;DR: We are rolling out solutions for Strict NAT using proxy servers to ensure those affected can matchmake and play Warframe with others!

[Jahadaya]

So many networking threads today from you guys :D

 

Thank yuh Jesus! 

[DeathKoala]

Glen, you are a saint.

[TwelfthAngel]

Edited January 26, 2015 by Conestt

[MechaKnight]

This sounds great. It's been a really good week for Warframe.

[7grims]

 

 

 

 

I hope its the last time I see this, and hopping this is the ultimate fix :D

 

But, are all the matchmaking/squad/lobby UI bugs also related to this?

Or is that yet another buggy thing?

[LisRestall]

OMG, I love you guys *cries* I'm one of those suffering from strict NAT, I'm solo-ing T4 voids like 90% of the time because I can't play with my friends most of the time. Anyhow, +1 I always thought a proxy/game server would finally fix strict NAT after researching and reviewing about it.

[BRad_Skirata]

glad to hear this!

 

but why black text on dark background -.-

not everyone uses the darker Theme, ik, but the pain for those of us who do!

shouldn't have to highlight to read is all i'm sayin

[[DE]Saske]

tl;dr ? ;D

[[DE]Glen]

glad to hear this!

 

but why black text on dark background -.-

not everyone uses the darker Theme, ik, but the pain for those of us who do!

shouldn't have to highlight to read is all i'm sayin

 

I pasted from google docs into the forum -- I have no idea how to even change the color.

 

[StallordD]

I'm really glad you guys are clearing up all these misconceptions about how networking works. I hope this reduces the amount of uninformed replies bashing you guys for issues that aren't entirely in your control, or at the very least, gives those people a bit of peace of mind.

[[DE]Glen]

tl;dr ? ;D

 

#Enjoyban

[DJ_Redwire]

All of my 10^10/10.

 

Edited January 27, 2015 by DJ_Redwire

[kubbi]

Thanks a lot for the hard work! I've been especially hoping for matchmaking fixes. Glad to hear its been getting attention.

[BRad_Skirata]

I pasted from google docs into the forum -- I have no idea how to even change the color.

 

Yeah google docs get a lot of people, even me once xP

you can change the color before you c/p on google docs or use bbcode "[ color=??? ] TextToBeColored [ /color ]" on the forums after you c/p

[Dualstar]

So today might have been the last day might ever have seen Failed to Join.

 

Thats as great a start to the year as any i guess.

 

Now all i need is a bucket of content to shove my head into and im a happy tenno for a few more weeks.

[delakroix]

This is great! NAT, though useful, may be restrictive like the case we have ingame. Not all people understand NAT and how it works(been a system admin my whole career life and took me a year to get it). More so, working with clunky ISP provided routers can be a real ordeal when trying to do port forwarding.

 

Thank you DE!

Edited January 27, 2015 by delakroix

[ndantony]

More cool, good to know, information. 

 

Thanks, dude, the standing giant-head horse with the fro!! :P

[taiiat]

I pasted from google docs into the forum -- I have no idea how to even change the color.

 

 

Yeah google docs get a lot of people, even me once xP

you can change the color before you c/p on google docs or use bbcode "[ color=??? ] TextToBeColored [ /color ]" on the forums after you c/p

or, preferably, you can copy without Formatting. 

Browsers of all brands have started deciding that people want that all of a sudden. suddenly deciding that if someone copies text from a page that they wanted the text color and background color.

 

when.... all they wanted was the damn words.

 

 

i flip back and forth between a Utility Plugin to copy without Formatting, or pasting into a plain text editor and copying back out of it to drop the formatting. (mostly because the Plugin doesn't work on some webpages, of which i'm not sure why - most issues are with WYSIWYG Editors, not that i'm surprised, since they're pure cancer).

IIRC there's also a shortcut been added to various Browsers to Copy without Formatting, but it's not the default copy shortcut for some reason....

you'd think that when adding a feature that most won't want, IT would get the new shortcut, rather than stealing the conventional one.

Edited January 27, 2015 by taiiat

[(PSN)iBoiz]

Will this feature also work on consoles or is it just for pc?

[SonicSonedit]

[DE]Glen, I don't want to be that guy, but we both know that there are a lot of games which work around strict nat somehow. For example, all of the game which use steam matchmaking servers, allow you to be host even if you have a strict nat. A lot of late console games do this too. That's exactly why people see problem in Warframe and not their routers and they are partially right. Ofcourse it's mostly a router or (as was in my case) ISP problem (my ISP was strict-NAT only disregard routers or anything else, now I pay bonus fee for "white" ip), but still lots and lots of games work around strict NAT.

You can easily check this by yourself: launch any steam game (CS, L4D2, Portal) and you will find you can be host with strict NAT without any problems.

 

Sincerely, your veteran player.

[Reiduxed]

This is the best thing I've heard from you guys for quite a while. I know it won't be the elixir to everything but it gives me some hope and something to look forward to. Thank you.

[Fiiiinnn]

Still having Strict Nat issues :( 

 

Thank you guys so much for working to fix it.

[Ritchel]

im using a vpn for this to be fixed although it creates a different problem in other region, like unable to join random people in pubs.

i hope this fix it, me and my ph friends (since the strct nat is always with our country's main isp and their router) lost interest in the game because of my unfixeable issue.

finally you decided to look at this,

Edited January 27, 2015 by Ritchel

[Seqan]

I pasted from google docs into the forum -- I have no idea how to even change the color.

 

I see a lot of complex solutions for this. Here's mine.

 

Copy/paste into notepad first, then copy/paste into the forums. That clears all formatting. You can also in some browsers, right click and "copy plain text" or something like that.

 

As for the Strict NAT, nice! If I may make a suggestion though... Post the TL;DR first rather than at the end. I for one love your explanations, but they are rather detailed. =P