Riven Slot Hardcap (forcefully dissolving my Sortie reward)

[Jukantos]

3 hours ago, Tsukinoki said:

Two things your missing here (which if you are a programmer is surprising and I wouldn't want to rely on your programs if they deal with big data like Warframe has to):

-First is transferring the Riven information to the client from the servers.  That is where a lot of the limitation is focused is trying to keep the bandwidth requirements when you log in reasonable so that people don't sit at a spinner for a few minutes at a time.

-Second is retrieving the riven information from the database to begin with!  Think about it: In order to find a players Riven information they have to query their database.  They have millions of records that they have to go through to find all of a users rivens.  Even with heavily optimized queries that takes some time.  If they didn't have a limit on how many records per player that process of fetching the data from the database would take longer and longer and longer and longer as the number of Rivens in the systems increases.  So regardless of how they compress the data to send it to the clients they also have to deal with fetching the data originally from the database.

Trust me, dealing with fetching data from a table of tens-to-hundreds of millions of rows takes a while.  Even in well normalized databases.

Its a two part problem that DE solved by capping how many records are possible so that there is less maximum data to send to the client, and there is less data to search through on the database end.

You're forgetting one simple possibility: Using a Checksum. Since a Database Select has to be done anyway to confirm wether or not your e-mail and passwort are correct, it shouldnt be that hard to calculate a checksum over the entire datamass of the rivens and simply compare if anything has changed, while storing the last state of the rivens of anyone who had logged in on the local machine. Only if the checksum is not equal, an access onto the database to retrieve the riven information would even become necessary. Since most players do not play from more than one computer/console, for the vast majority of logins you wouldnt even need to retrieve the full riven information, just a checksum that could a) be locally calculated on the players machine in fractions of a second and b) would only take a fractional amount of both database access time and bandwidth to compare with.

For all that is holy i really really really do hope they do not simply have a database of rivens with player accounts being linked in a dataset of a single riven, but on the contrary, that there's a riven table for every user so you have a lot of smaller files / database tables to access (and all you need is to retrieve the player account which you have to do anyway if you get an email adress and a password thrown at you)

Honestly, we see a 1-5 second delay upon login anyway, most of the time covered up by the login reward animation. How the transfer of a checksum in addition to the confirmation of your account getting logged in would take so long is a mystery to me. Sure you might see a ~2 second delay if you've changed your riven collection on another machine and it has to download (oh no!) a megabyte or two in the background (keyword, background, people tend to take 10+++ seconds before they jump into a mission / mods / arsenal after logging in, so the time is there)

I think the limit was created to make sure nothing goes down in the testing phase of this system (hell imagine how desastrous it would have been if the game had gone crash-prone as a result of rivens on the launch of the war within) and has truely outlived its justification time window. I doubt DE used horrific programming for the rivens, so optimizing the structure and getting rid of this limit should in no way be a mammoth task.

Ánd again, ask yourself, how many players ACTUALLY have a desire to own 100, 200, 300 rivens? How many people would buy those slots? Would they accept tiny background loading times that they wouldnt notice most of the time as a tradeoff? For me personally, that's a gigantic yes.

[OffaLimb]

All that talk of text files and XML is completely not correct.  There is no way to efficiently deal with the constant changing of data for those file formats to be used.  So we can, with almost 100% surety, that they are use a SQL type database to store the rivens.  With that in mind, here's how I think they are doing it.

There are basically 2 tables.  One for the actual rivens themselves and one for the riven attributes.  The riven table contains the information of PK of just some number, user ID, cycled times, challenge ID, challenge status, and a bunch of numbers that link over to the attributes PKs.  The reason each riven has to have it's own entry is because of the cycled times.  Even if 2 people have the exact same riven, odds are they didn't use the same number of cycles to get there.  So it would be easiest to just have each riven have it's own entry.

Now, for the attribute table it's just going to be a big table that consists of a PK ID number, what it does, and reference to image IDs the basic Mods database.  Probably includes at least one null entry too since not all rivens have the same number of modifiers. Finally there is a 3rd table that deals with challenges, and I'm guessing that it's basically just a PK ID and challenge.

The way all this works (as I see it) is that when you "roll" a riven, it generates some random numbers that it puts in the attribute parts of the riven table, and increments the counter.  Mainly because of this counter each riven has to be unique in the database.  As they have also stated when talking about rivens in the past - the idea is not to have a riven for everything, but that you are supposed to pick-and-choose which rivens you really want. 

So the reason for the limit is to actually 2-fold.  One to prevent the database from getting out of control, and the other as a game mechanic to force you to not have all the rivens.  The reason they charge for more than the standard slot count is basically a fee for you increased the size of the database by your own choice.

[booty_hunter]

20 hours ago, Jukantos said:

Okay, im a programmer myself, so i'd take a wild guess and claim these rivens literally can not make up more than 1 Megabyte of information. No matter how badly you encode them, you can literally write down their numbers (name as a string, stats with an ID, stat amount and number of rerolls) in a text file and youll probably have a hard time reaching 50 kilobytes with the information connected to 1 account.

Server hardware space is relatively cheap. DEFINITELY cheaper than what we spend in platinum to unlock these slots. The only possible thing i can see as a limit is the transfer of information to the client on login, but then again, come on it's likely not even half a megabyte. How hard can it be to quadruple that, mind you ONLY for the players who will pay for it with platinum! ~95%+++ of accounts (especially the dead ones as well as newer players) wont even trigger this whatsoever.

So honestly, what technical reason for this limit could exist? I'd love to hear an explanation for that. It makes no sense that something as tiny as the riven data should have any impact on DEs systems, unless (LOL) we actually have a library of all the players world wide's riven inventories in our game client in case we host for them (that however would be so technically ludicrous i'm surprised i even came up with it as a possible scenario)

theres 26 million of us losers

Edited May 24, 2017 by booty_hunter

[Jukantos]

22 minutes ago, OffaLimb said:

All that talk of text files and XML is completely not correct.  There is no way to efficiently deal with the constant changing of data for those file formats to be used.  So we can, with almost 100% surety, that they are use a SQL type database to store the rivens.  With that in mind, here's how I think they are doing it.

There are basically 2 tables.  One for the actual rivens themselves and one for the riven attributes.  The riven table contains the information of PK of just some number, user ID, cycled times, challenge ID, challenge status, and a bunch of numbers that link over to the attributes PKs.  The reason each riven has to have it's own entry is because of the cycled times.  Even if 2 people have the exact same riven, odds are they didn't use the same number of cycles to get there.  So it would be easiest to just have each riven have it's own entry.

Yes it's obvious they're using some form of database which is almost certainly an SQL Database, but you're missing my point. A Riven only consists of a tiny amount of information, so small in fact you could put it all in a relatively short string and have the client pick it apart to turn it into an actual riven mod. All you need is Name, up to 4 stats (ID? How many stats are there, like 25?), 4 numbers for the respective stats and a final number to indicate the amount of rerolls and you have everything you need to build it in the game

My point is, the entire riven information for an entire account could be compressed to such a string to be shorter than 1 MB and then if you wanted to REALLY be efficient with your bandwidth you could additionally generate a checksum across the string and save the last riven configurations on the client side and only check if the checksum still checks out (so if no changes have happened since the last login on the client and only IF there has been change you need to download the full riven information.

So technically you dont need a dedicated database for the rivens whatsoever. You simply take the database for accounts you already have and expand it by two columns. Upon login you do a SELECT command that includes both the stuff that is already transmited to complete the login, as well as the checksum. If the client's checksum is identical, this is all the exchange that has to happen. If it is not, the client downloads the second, larger string and uses it to bring the riven collection up to date.

For 99% of logins (when you didnt change your riven collection while logged in on another device) this would mean an additional bandwidth consumption ONLY as large as the checksum. I'm pretty certain they are already using something like this to make sure rivens havent changed when logging in, if not and they do a full checkup this could actually be MORE EFFICIENT for bandwidth on most logins. The other few scenarios (rivens changed on another device, first login on new client) you have to download the collection in full (which is currently already happening every single time) and at its worst case that's going to be more bandwidth for the file transfer (but it is not an additional select on the database, as the login server can simply hold the long string available during the login process until the full login is confirmed). If pulling it out of the database is done together with the rest of your login authentification, you have a larger database in volume, but NOT more search effort (as you are literally just adding columns to your one table of players)

Think about it, why would you ever make a seperate table of rivens? of up to 60 datasets per player? You can easily encode everything that defines all the rivens one player owns into a single not very long string.

Edited May 24, 2017 by Jukantos

[Jukantos]

1 minute ago, booty_hunter said:

theres 24 million of us losers

Ok but http://steamcharts.com/app/230410

lists warframes average daily players on steam as 30k, with an all time peak ever at 70k, so my estimate of 300k concurrent players that i calculated with is most likely a way too high estimate (which was the intended way to calculate it)

[OffaLimb]

Well regardless of the technical considerations - I would like to point you over to

Particularly the last sentence of the first section that says " The current maximum number of 15 Riven Mods is a reflection of this, as limitations encourages creativity and choice."

So we can debate the technical details about the best and most efficient way to implement rivens and how it shouldn't slow things down and could be easy to manage.  However it doesn't change that DE wants you to have a limited number of rivens so you can't just riven everything.

[(PSN)vinster]

Seeing as how new weapons are periodically released, I feel that an increase in the max limit should be considered a few times a year.  I still would like to keep my old rivens and get rivens for the newly released weapons (eg. Cycron, Zenith, and what may come in future).

When melee rivens are a thing, I'm sure DE will up the limit.  Perhaps they will announce a nice surprise about this at TennoCon2017, but I do hope sooner.  I bet if they look up the stats, they can see that the riven trade has contributed to their bottom line and not enabling players to buy and keep more rivens just doesn't make sense.

[booty_hunter]

15 hours ago, Jukantos said:

Ok but http://steamcharts.com/app/230410

lists warframes average daily players on steam as 30k, with an all time peak ever at 70k, so my estimate of 300k concurrent players that i calculated with is most likely a way too high estimate (which was the intended way to calculate it)

The data of those who stopped playing are still being kept. we also dont know how big their servers are.

[Ditto132]

It is most likely not server limitations but gameplay limitations, DE just doesn't want you to have that much Rivens.

If it was a server limitations of any kinds, why DE still allow so many players to keep multiple copies of the same mod with different ranks?
And duplicate maxed mod for when your companion needed it?

If so, wouldn't the idea of allowing the player to adjust their mod's rank to the level that they wanted, be accepted by DE, and to reduce the burden on the server?

[Skaleek]

On May 24, 2017 at 0:24 PM, Jukantos said:

Yes it's obvious they're using some form of database which is almost certainly an SQL Database, but you're missing my point. A Riven only consists of a tiny amount of information, so small in fact you could put it all in a relatively short string and have the client pick it apart to turn it into an actual riven mod. All you need is Name, up to 4 stats (ID? How many stats are there, like 25?), 4 numbers for the respective stats and a final number to indicate the amount of rerolls and you have everything you need to build it in the game

My point is, the entire riven information for an entire account could be compressed to such a string to be shorter than 1 MB and then if you wanted to REALLY be efficient with your bandwidth you could additionally generate a checksum across the string and save the last riven configurations on the client side and only check if the checksum still checks out (so if no changes have happened since the last login on the client and only IF there has been change you need to download the full riven information.

So technically you dont need a dedicated database for the rivens whatsoever. You simply take the database for accounts you already have and expand it by two columns. Upon login you do a SELECT command that includes both the stuff that is already transmited to complete the login, as well as the checksum. If the client's checksum is identical, this is all the exchange that has to happen. If it is not, the client downloads the second, larger string and uses it to bring the riven collection up to date.

For 99% of logins (when you didnt change your riven collection while logged in on another device) this would mean an additional bandwidth consumption ONLY as large as the checksum. I'm pretty certain they are already using something like this to make sure rivens havent changed when logging in, if not and they do a full checkup this could actually be MORE EFFICIENT for bandwidth on most logins. The other few scenarios (rivens changed on another device, first login on new client) you have to download the collection in full (which is currently already happening every single time) and at its worst case that's going to be more bandwidth for the file transfer (but it is not an additional select on the database, as the login server can simply hold the long string available during the login process until the full login is confirmed). If pulling it out of the database is done together with the rest of your login authentification, you have a larger database in volume, but NOT more search effort (as you are literally just adding columns to your one table of players)

Think about it, why would you ever make a seperate table of rivens? of up to 60 datasets per player? You can easily encode everything that defines all the rivens one player owns into a single not very long string.

Been saying this for a long time. I think you can store a single riven as less than 20 KB (generous) of data.