Suggested fix to Host Translation.

[MrAires]

ABSTRACT

The current problem is that much of the data concerning a mission instance is saved on the Host, and the exit process prioritizes the host's ability to exit, over the ability of the other players to play the game at all. There is substantial unreliability in the current process and a high likelihood of critical failures would will lead to player loss.

IDEA

A possible fix would be to use a scratch data struct to encode mission state data.

METHOD

The exact syntax of this struct would come down to optimization studies in engine, however my proposed solution is to use a "key and map" structure using UTF-8 char values.

The idea behind the key would be to create connections between abstract data and specific ids. This essential or "key" data would only be appended to during a mission, which would increase data integrity whereas the physically volatile locations of items in each level would be "mapped" out at regular intervals in a separate data structure.

 

EXAMPLE

The Key:

data type1-HEXID1,

data type2-HEXID2,...

::The described data types would be any sort of player placed objects, (such as Mirage's traps,) or any permanent player specific state (active abilities or DUVIRI decrees.) Any HEXIDs would be either unique in that instance or predefined elsewhere.

The Map:

HEXID1,

CHAR-CHAR-CHAR-CHAR-CHAR//1

...

CHAR-CHAR-CHAR-CHAR-CHAR//N

HEXID2,

....

HEXIDN

::The map structure will be set using UTF-8 Char encoding to allow the placement of high bit depth data in a low latency format (as it will have less physical ones and zeros than a corresponding float) and does not require any terminating values/spaces as those are inherent parts of the encoding process.

The five char values would be uint-16 data values which correspond to: (map cell index)(mission map cell index)(mapcell_X)(mapcell_Y)(mapcell_Z)

The encoding process would take a map cell's X Y and Z float size, divide by the item location from some predefined datum then convert to the uint-16 form. The resolution of this conversion can be arbitrarily large, such that you may implement this as 0-3-7, 0-256-511, or 0-32000-64000 corresponding to 0-.4-.9. Numerical normalization will increase extensibility and allow for map specific unwrapping processes to be done on the new host. So if at some later time DE decides a series of open world maps should be stitched together, that can be done fairly seamlessly using the same structure.  

Additional data could be encoded by simply having key CHAR or HEXID values for players or what not. This would definitely come down to the engine.

DISCUSSION

Why uint-16 as UTF-8 char instead of just using floats!?? 

The specific conversions to the char type will allow for an easier handoff process to the new host, and increase speed by reducing the net number of bits required to be sent to do said handoff, with a tradeoff of using marginally more CPU cycles on the host side.

How is this faster if the Host unexpectedly drops?

This process does not guarantee speed nor would it be done upon the host leaving, but rather this data would be sent at regular intervals (20hz, 1hz or slower) to the clients so that when the host leaves, each possible host will already have a copy and can implement the most recent map. Ideally you would have some number of map slots, preferably 3 or more, with some set arbitrary cache size such that if a host leaves during a write process the map being written to may be trashed, an older map may be checked for validity and if that fails the third can be loaded.  

Something to remember is that the serialization of maps allows for this write process to occur piecemeal throughout the game loop and could be delegated as a lower priority action. 

If it isn't faster, then why bother?

The aim of this process is not to accelerate the host migration process itself, but to make it more reliable and more user-friendly. The current process is currently barely functional, and often leads to complete breakdowns of content: Archon mission breakdowns, Duviri circuit failures with loss of progress, and substantially more.

Players won't want to spend longer than they have to waiting for host migration

Players would rather have marginally longer wait times on loading screens than have to redo possibly hours of frustrating or difficult content due to some seemingly minor flag breaking.

[Exxion]

Your primary assumption is incorrect. The issue with host migrations isn't that the host doesn't correctly send map locations, etc. to the clients during gameplay. This data is already successfully replicated, as evidenced by the fact that clients can see it in the mission. The issue is that not everything is reinitialized properly when the new host takes over and the session restarts. Changing the data structure in which the mission state is stored will not fix anything, because the issue isn't that the data structure is wrong, but that its contents are.

2 hours ago, MrAires said:

::The map structure will be set using UTF-8 Char encoding to allow the placement of high bit depth data in a low latency format (as it will have less physical ones and zeros than a corresponding float) and does not require any terminating values/spaces as those are inherent parts of the encoding process.

What do you even mean by this? Floats don't have terminating values either, and they have <17 million redundant values out of over four billion possible values (assuming subnormals are enabled). You can't get significantly more efficient than this.

[Nekomian]

I'm of the same mindset of Exxion, I don't think changing the structure of data transports into some embedded key value pair structure will fix the fundamental issues that host migrations face, which is reinitialization issues upon switching hosts (or even just switching hosts correctly, as it should default to a solo session if connection errors arise due to weird P2P routing). Clients already have data on their end of what individual abilities and information their game state is in, but for some reason drops it (or does not attempt to renegotiate it with other clients upon switching hosts).

To give a very brief arbitrary example: if host A, client B, and client C are all using abilities and client A is host and leaves, both client B and C now get to decide who will be host; they both know their game states (as well as the mission state) and need only send that info to whoever will be the host and to instantiate it on their end when the mission resumes / everything syncs (i.e. "i'm client B using roar and it's at 35.3s left, sync that info with the host if necessary, and set it to that on my side when the mission resumes"). That info seems to be getting lost (or disregarded) in the transfer, so things occur like timers resetting, mission progress getting stuck, and reverting to defaults where info is omitted. While changing the data structure and format in which this occurs might help in other ways, it feels like an overengineered solution when they just need to handle these functionality issues and could simply use their existing structure.

[MrAires]

On 2023-06-08 at 12:55 PM, Exxion said:

Your primary assumption is incorrect. The issue with host migrations isn't that the host doesn't correctly send map locations, etc. to the clients during gameplay. This data is already successfully replicated, as evidenced by the fact that clients can see it in the mission. The issue is that not everything is reinitialized properly when the new host takes over and the session restarts. Changing the data structure in which the mission state is stored will not fix anything, because the issue isn't that the data structure is wrong, but that its contents are.

Just because something is rendered, doesn't mean it is present. Those can just be SFX nodes with no context connected to them. The only nodes that are present fully on client are the ones that survive the transfer; that seems fairly obvious.

On 2023-06-08 at 12:55 PM, Exxion said:

What do you even mean by this? Floats don't have terminating values either, and they have <17 million redundant values out of over four billion possible values (assuming subnormals are enabled). You can't get significantly more efficient than this.

Edited yesterday at 01:45 PM by Exxion

So. when speaking on peer to peer communication, sending 32bits (or 16 depending on implementation of float; because iso is a joke or something) for the number 1, vs, 1 singular bit, is a massive difference in efficiency. My masters was in brute force hardware based data transfer and when you are operating in an environment such as netcode you can't just throw everything. Each one and zero matters and you can't afford the fat from floats for something you are sending over the socket at 30hz. ya think 5 floats per point, 40-80 bits each, maybe up to 1000 points, so that's 40k-80k per cycle, 120-240kbit/s on just the map data on the low end. I'm sure you would need more than that for an actual implementation. 

On 2023-06-08 at 2:58 PM, Naroxas44 said:

I'm of the same mindset of Exxion, I don't think changing the structure of data transports into some embedded key value pair structure will fix the fundamental issues that host migrations face, which is reinitialization issues upon switching hosts (or even just switching hosts correctly, as it should default to a solo session if connection errors arise due to weird P2P routing). Clients already have data on their end of what individual abilities and information their game state is in, but for some reason drops it (or does not attempt to renegotiate it with other clients upon switching hosts).

To give a very brief arbitrary example: if host A, client B, and client C are all using abilities and client A is host and leaves, both client B and C now get to decide who will be host; they both know their game states (as well as the mission state) and need only send that info to whoever will be the host and to instantiate it on their end when the mission resumes / everything syncs (i.e. "i'm client B using roar and it's at 35.3s left, sync that info with the host if necessary, and set it to that on my side when the mission resumes"). That info seems to be getting lost (or disregarded) in the transfer, so things occur like timers resetting, mission progress getting stuck, and reverting to defaults where info is omitted. While changing the data structure and format in which this occurs might help in other ways, it feels like an overengineered solution when they just need to handle these functionality issues and could simply use their existing structure.

See above, the data is obviously only visually on the clients, and not at all actually present.

 

There is a vast difference between a host instance and a client instance. It makes no sense to be actually running every player's abilities on every client, that is a lot of redundant execution. 

The reason you wouldn't transmit EVERYTHING, is because there is a physical latency related to using a network connection, and this latency does not have a consistent rate. Sure your ISP may guarantee two gbit, but that isn't the whole story here. There is a long chain of different processes between the host, whatever overwatch DE has, then to you and back through to the host again;  those processes are occuring at anywhere between 20-60hz, so you can't have ALL state data included and expect someone playing on a switch in S#&amp;&#036;ty wifi to be able to play with any stability.

 

Further, because only the visual data is present on the client, there wouldn't be any specific character bonuses attached; all of that would be host side; unless they modified their rendering pipeline to embed that data, which would be a nightmare. 

 

The host migration topic can't really be addressed without advanced knowledge of the future state from the host physically ready in the clients PC and currently there is no such data. Else this wouldn't be an issue 

 

[Nekomian]

12 minutes ago, MrAires said:

It makes no sense to be actually running every player's abilities on every client, that is a lot of redundant execution. 

Clients can/do execute their own lua scripts per ability (and arcanes / other calculations), which is shown in logs during batch loading. Some of it may be visual, but it makes no sense to offload all of the damage or ability calculation to the host (like LOS checks), or they'd be computing for essentially 4 clients when the load can be spread out evenly between all 4, with final calculated numbers being sent to host to apply the effect and relay it (which is why sometimes there's a "delay" hitting an enemy as a client if the latency is quite large). To be clear I'm not stating each client is calculating other clients' abilities or damage, because that is redundant, but each client should be calculating their own to avoid a situation where the host has a way greater burden (or to the point where if a migration can't occur it crashes the session due to loss of information).

Host migrations that occur abruptly (due to host crashing or their internet cutting out) do currently happen and are recoverable, so it's definitely not all visual. I'm also not suggesting all info constantly be sent at a quick poll rate, but that the client can track (and re-sync) certain things with the host independent of them, and there should be a way to make that info recoverable on migration. The example I gave is a demonstration of that, the client should keep track of their own abilities and timers and apply them on migration. Even if for whatever reason that info was only visual, the UI still has that info (i.e. this ability is "active" and this is the time remaining) and should be able to re-transmit and validate that to a host without any other context. This can occur regardless of the actual structure of the transported data, it's more of a functionality issue.

Keep in mind all data also does not have to be sent constantly, and clients can still sync to host occasionally to avoid getting lost. As an example, say we get to wave 5 and host wants to extract - all clients synced the current wave number with host when it transitioned, and now all understand that after migration it should move to 6. This only requires one-time transfers of specific data to sync to clients. There's many things that can utilize this that don't need constant updates, and many likely are but are just not being utilized or reinitialized properly. This doesn't need a complete data structure overhaul to function as intended, it just needs to apply the intended behaviors properly on migrating.

[Exxion]

4 hours ago, MrAires said:

Just because something is rendered, doesn't mean it is present. Those can just be SFX nodes with no context connected to them. The only nodes that are present fully on client are the ones that survive the transfer; that seems fairly obvious.

Hm, yes, it sure is purely visual when I step into range of a Reservoir and immediately get its effects.

EDIT: Actually, I think Reservoirs specifically don't apply effects immediately regardless of ping, but the following still stands.

What exactly is there besides the visual data? The visual data of a lingering ability would include, at minimum, what ability it is, its range value, its position, and its owner (for colors). The only other information is strength and duration. It would make very little sense not to send these, too, as they would improve clientside gameplay.

The problem is not that the clients do not have this information. It's that they don't retrieve and reuse all the information they have when taking over as host.

4 hours ago, MrAires said:

So. when speaking on peer to peer communication, sending 32bits (or 16 depending on implementation of float; because iso is a joke or something) for the number 1, vs, 1 singular bit, is a massive difference in efficiency. My masters was in brute force hardware based data transfer and when you are operating in an environment such as netcode you can't just throw everything. Each one and zero matters and you can't afford the fat from floats for something you are sending over the socket at 30hz. ya think 5 floats per point, 40-80 bits each, maybe up to 1000 points, so that's 40k-80k per cycle, 120-240kbit/s on just the map data on the low end. I'm sure you would need more than that for an actual implementation. 

OK, hold on. I may have misunderstood what your original post was saying. It sounded like you were saying to convert the floats to uint16 (a valid way to save some data, though I would argue it's not enough precision), then convert the uint16s to UTF-8 (which would make absolutely no sense whatsoever). If you're just saying the keys should be UTF-8, then yeah, that makes sense. You would never use floats as keys anyway, though.

EDIT: On further reflection, 16 bits is more than enough precision given the hierarchy. Also, I see your logic for UTF-8, but maintain that it's not helpful except maybe on one of the five values, unless you insist on using the same structure for both normal missions and open worlds (which is worse for both).

Also, UTF-8 is never one single bit. It is a variable-length encoding of between 1 and 4 bytes per code point, and potentially multiple code points per character (though you would never use multiple code point characters in this context).