It's time to fix host migration bugs once and for all instead of individual fixes

[Scylax]

For years there have been issues with host migrations where something isn't preserved correctly between hosts. This has always been patched on a case-by-case basis, but it's clear after years of doing this that no matter how much effort goes into fixing and catching migration issues in QA, that the host migration bugs aren't going anywhere. It's time to solve them once and for all by fixing the underlying cause. As an external user, I have little insight into what that is, but I imagine DE engineers are fully capable of addressing this, whether it's moving migration handling from state tracking within the implementation, per-feature to some global state that arbitrates the implementation like React+Redux, or something else. 

Patching host migration issues on a case-by-case basis like they are now is not sustainable. It would be nice to explore different approaches.

Edited December 14, 2019 by Scylax

[Berzerkules]

Just had host migration in eso. After host migration I can't attack. Can't use skills. Can't go into operator. Die and lose focus. I nuke 2 waves so people can leach and I get screwed. 

Host migration in arbitration defense. Host leaves and defense target starts with half life and is in the middle of enemies. Instant fail. Lose all rewards. 

Host migration on railjack missions. Host leaves and I'm started alone with boarding parties on board. I fail lose everything. 

Host leaves and everyone else loses. This system does not work.

Neither does posting about it on the forums.

[Zilotz]

I proposed before a fix for host migration that will solve 99% of such problems - to make a constant sync of all in-mission data between all players so migration would be nearly instant and will save all mission states(only small problem would be 1-2 sec of desync on host dropping).

But they did not respond about it at all, maybe they didn't notice.

 

[LjMjollnir]

Zilotz sadly that would increase the overhead ALOT.. and some people just shouldnt be host... you know you can tell when someone is hosting from a potato or a really bad internet right away... but Warframe still counts that as being playable..

They need to up those standards to make sure the best suited Internet/PC is running as host... and these things can be done without a "pause/host migration" screen the big AAA titles do it and have been for many years 😉 tho also alot of those are server based but the players move between zones(and servers) almost seamlessly in quite alot of them

Ive always had hopes that the background sim should be running on each machine in the game with the dedicated host being the one to do the checks on each player... that way if the host does leave everyone else already has the sim running and it picks the next best host..  i kind of feel this is how the game already runs but they are not sending everything.. since yes we always miss out on something during host migration.. all the small data counts and it just isnt being packaged and sent

[sam686]

Host migration death, fails to pause while host migrating.
https://forums.warframe.com/topic/1037365-host-migration-death-fails-to-pause-while-host-migrating/

Host migration index bug. Plenty of topics up to a year ago. No fix.
https://forums.warframe.com/topic/1151172-game-freezing-when-host-is-migrating-in-an-index-match/

Sometimes the game lost connection, but no host migration attempts?
https://forums.warframe.com/topic/1144389-video-deck-12-public-matchmaking-have-big-problems-someones-slow-load-fails-join-mid-game-out-of-map-and-locked-door/

The new content, Empyrean, have plenty of host migration bugs.
https://forums.warframe.com/topic/1153045-raijack-host-migration-issues/
https://forums.warframe.com/topic/1152927-host-migration-results/
https://forums.warframe.com/topic/1152400-railjack-enemy-interiors-colliding-after-host-migration/
https://forums.warframe.com/topic/1151964-empyrean-hotfix-2702/page/12/?tab=comments#comment-11247816

Will we ever see a fix for all these problems?

[Zilotz]

On 2019-12-15 at 11:23 PM, LjMjollnir said:

Zilotz sadly that would increase the overhead ALOT.. and some people just shouldnt be host... you know you can tell when someone is hosting from a potato or a really bad internet right away... but Warframe still counts that as being playable..

They need to up those standards to make sure the best suited Internet/PC is running as host... and these things can be done without a "pause/host migration" screen the big AAA titles do it and have been for many years 😉 tho also alot of those are server based but the players move between zones(and servers) almost seamlessly in quite alot of them

Ive always had hopes that the background sim should be running on each machine in the game with the dedicated host being the one to do the checks on each player... that way if the host does leave everyone else already has the sim running and it picks the next best host..  i kind of feel this is how the game already runs but they are not sending everything.. since yes we always miss out on something during host migration.. all the small data counts and it just isnt being packaged and sent

For overhead i partially agree, on network level it would increase several times the traffic.

For PC resources - it would be like 1-2% of what warframe eats by itself so will not even be noticeable.

If we implement additional checks for packet delivery and being sure (including relaying packets between clients if host has worse connection to one of clients than sending data thru other client(s)) everyone has everything - yep, it will eat up bit more BUT in that case dynamic host migration can happen painlessly even every cycle which is many time s a second.

Yes, dynamic routing and integrity checks is a hard task to implement but once its done it will make much better the gameplay.

So i decided to propose scheme for network interaction -
 

Spoiler

 

Phase 1 - all to host, at this phase all clients send their data to host updating mission state.

Phase 2 - all to all, at this phase everyone sends to all others mission data and receives all available mission data + ping calculation from timestamps.

Phase 3 - host to all, if host fails phase 3 in 2*ping time then new host is selected by best ping, if failure is persistent as no client is able to maintain good connection to the team then we fallback to phase 2 only mode with slightly more delay after trying phase 4.

Phase 4 - dynamic routing proposed based on ping(from timestamps calculated on phase 2)

Now a bit of explanation for phase 4 with some examples:

Case A:

Clients ABCD are in the game, not using numbers as number 1 means host.

Client A: Before mission was starting as host

Clients BCD: Started as non-hosts.

Initializing mission, during load screen data travels from A to BCD to setup initial state of mission.

Mission started:

Phase 1 - Setting on first cycle data from Focus school buffs, Auras and passives(it's first cycle so players didn't even move yet, some milliseconds passed from start).

Phase 2 - Everyone synchronized data, timestamps now calculated, we know that A-B has ping of 35, A-C has ping of 50, A-D 38 and B-C 110, B-D 25, C-D 30.

Let's say on Phase 3 there was some packet loss from A to B so we moved to phase 4 to overcome it if possible.

So B did not receive good response from A and sends to CD "Guys, is A down?"

C and D reply "No, it's up and running!"

B knows already that C is far(110) and D is near(25) so sends "How's is ping to A?" to them both.

D replies "It's 38 for me!"

C replies "It's 50 for me!"

B sums up the result route to A - for C it will be 110+50=160 but for D it will be 25+38=63

Then B sends to D "Could you please relay packets from A for me?"

D answers "Sure, will tell to A about it!"

D sends to A "Hey, B cannot connect to you directly so i will relay packets, send packets for B to me with mark 'Packet for B'!"

A answers "Okay, here is mission setup B was missing:'Packet for B: initial mission data'"

So they did negotiate and everyone is okay, good enough ping for everyone and no loss of packets.

Still some other trouble can happen.

Case B:

We have the same setup from end of case A just later during mission Client A had a game crash from Railjack bugs.

BCD send to each other "Is A okay?"

Then they try to reach it and reply "Nope, looks dead to me!"

BCD "We all agree that A is dead, is Phase 3 snapshot recent enough? 1 cycle back? Okay, falling back to state 43.5milliseconds ago."

Selecting host: B-C 110, B-D 25, C-D 30, D is good for both C and B so becomes host.

Game continues just lagging for around 45-100ms instead of long host migration message blocking everyone, loss of data and bugs/failure from migration. 

 

 

Edited December 17, 2019 by Zilotz
Proposing scheme or network interaction