Jump to content
[DE]Rebecca

Drop Rates, Datamines, and Digital Extremes (DDD).

Recommended Posts

Seriously Rebecca...  a website might not cut it. Destiny did it's lore via Smartphone and it was aweful. Long term the values need to be implemented into the Codex... with a tutorial what the Codex is in the first place.

[EDIT] Curious... my original post contained a suggestion to hire VoiD_Glitch and let him handle the problem and programming. Why would a mod want to remove that?

Edited by Zeranov
Post was altered by DE...
  • Upvote 3

Share this post


Link to post
Share on other sites
9 hours ago, (PS4)VoiD_Glitch said:

My only goal was to upload any information that I could retrieve in a readable and useful format. It was an effort to assist the community. That was my highest priority. All Digital Extremes had to do was ask me to take something down (it seems like they specifically wanted Languages.bin down), prior to a legal letter, and it could have been as simple as that. I do not appreciate being the involved in a legal matter that should not even involve me in the first place.

You didn't answer my question, and you do know it.

  • Upvote 2

Share this post


Link to post
Share on other sites

Forenotice: Any opinion or suggestion that follows, Lotus, should be considered with the Summary and Afterword in mind.
Advisory: :sentient:  terminology at play.

Void Glitched Datamining Incident:

Spoiler

 

Natah.

I have a simple suggestion regarding the datamining incident and the Void Glitch:

Allow him to continue digging, but here's the twist: He sends it to you ( [DE]Rebecca ) so DE can erase, if there is any, spoilery content from the datamined files.
This 'DE-approved' datamined file pack can then be sent back to Void Glitch, or uploaded to, say, the Warframe wiki. 

This would, however, require the law enforcement entities to loosen their grip on him, and would need both sides (DE and him) to talk to eachother more directly, without a Warframe between them preferably (i.e: Skype). 

This solution would effectively be a DE-sided certification process on datamined content, and theoretically would satisfy both parties.
Should this resolution to the situation be forgone for a more complicated, excessively expensive, solution, then that Grofit won't go to making new content if that is the case.

With this cooperation in mind, Natah, DE will be able to redact, if not outright remove any content-spoiling content from the build that is ready to be released.

 

Suggested solution (most efficient, supposedly least expensive, improves connection to Tenno):

Spoiler

Find a means to directly commune to Void Glitch (i.e: Skype, Discord) to set up a DE authentication system (which may require some Corpus Crewmen to operate) where Void Glitch's current and potential future datamined content is directly transferred to DE via [DE]Rebecca's Discord (which has such a feature, or Skype, somehow). When the content has been received, the authentication system can then proceed by ripping out any content that alludes to unreleased content that correlates to spoilers in such a way that all that should remain are what already exists, and what is already known (i.e: Harrow gameplay).
Short edition:
I suggest, [DE]Rebecca /  Natah, to use Skype or Discord to talk to Void Glitch directly without surrogates/Warframes/Spectres/Crewmen/Corpus_Proxies. After arranging a means to transfer datamined content, Void Glitch would continue operations but will instead transfer datamined data to DE via the method arranged, where DE itself will allocate resources to redact and nullify content-spoiling content. The output would be sent back to Void Glitch for him to upload, and it wouldn't spoil anything.

Afterword: This means Natah and Void Glitch would talk, voice to voice, if not face to face, directly. This way, the situation would be solved with far less cost than would be anticipated with stereotypical measures.


Surplus: The reason I, again, am addressing [DE]Rebecca as Natah instead of Lotus is because of last-year-or-so's situation where all Tenno were called Losers. This is because of the fact that Lotus was Natah before things changed, and it sufficient.

Summary: A solution to the incident with maximum benefit, if nobody else has said this before.

Share this post


Link to post
Share on other sites
10 minutes ago, NightmareT12 said:

You didn't answer my question, and you do know it.

It's one thing having to deal with bully tactics from a foreign owned corporation.

Quite another when fellow Tenno accidentally (I'm sure) start doing DE's lawyers' jobs for them.

 

  • Upvote 3

Share this post


Link to post
Share on other sites
7 minutes ago, yingji said:

It's one thing having to deal with bully tactics from a foreign owned corporation.

Quite another when fellow Tenno accidentally (I'm sure) start doing DE's lawyers' jobs for them.

 

Hate on me if you wish, but that's how it goes. If he had thought things thorugh and had limited to upload only the drop chances, this wouldn't have happened.

Instead he had a Discord, a GitHUb repository uploading other stuff like the Text Language files (what good does that do for us, what benefit?), and he is literally the only one that has tools to reverse engineer the game -- and since we don't have access to the whole thing, we can't just assume or deduce.

In case you can't understand what I'm saying: He didn't do this nor influenced anyone (of that I am sure), but by sharing the information only he (for he keeps his tools only for himself and has never shared them) and DE can get, it goes down to 2 possible scenarios:

  • He posted information that, indirectly, allowed people to do the Umbra stuff -> He needs to be more careful in the future for any other thing he does.
  • He didn't, which means there's someone else using his nose on the engine -> which is a bigger threat for the game itself (for devs, for us, for everyone).

Only by clarification of that all of us (we, DE and VoiD) can be 100% sure of what the heck the issue is here and the most correct course of action to take.

It's not about being a lawyer or picking sides, white knighting or whatever the hell people want to make out of this. It's by giving some thought and making sure this doesn't happen again by knowing exactly what and how it happened.

Edited by NightmareT12
  • Upvote 2

Share this post


Link to post
Share on other sites

Please also make sure that stats like flight speed and punchthrough (also maybe range caps in weapons like the Ignis, Glaxion and Phage) are added to the weapon arsenal UI.

Share this post


Link to post
Share on other sites
24 minutes ago, NightmareT12 said:

Hate on me if you wish, but that's how it goes. If he had thought things thorugh and had limited to upload only the drop chances, this wouldn't have happened.

Instead he had a Discord, a GitHUb repository uploading other stuff like the Text Language files (what good does that do for us, what benefit?), and he is literally the only one that has tools to reverse engineer the game -- and since we don't have access to the whole thing, we can't just assume or deduce.

In case you can't understand what I'm saying: He didn't do this nor influenced anyone (of that I am sure), but by sharing the information only he (for he keeps his tools only for himself and has never shared them) and DE can get, it goes down to 2 possible scenarios:

  • He posted information that, indirectly, allowed people to do the Umbra stuff -> He needs to be more careful in the future for any other thing he does.
  • He didn't, which means there's someone else using his nose on the engine -> which is a bigger threat for the game itself (for devs, for us, for everyone).

Only by clarification of that all of us (we, DE and VoiD) can be 100% sure of what the heck the issue is here and the most correct course of action to take.

It's not about being a lawyer or picking sides, white knighting or whatever the hell people want to make out of this. It's by giving some thought and making sure this doesn't happen again by knowing exactly what and how it happened.

'Literally the only one that has tools'? This is why I advise Void_glitch to stop posting ... 

Do you personally know what software is used to facilitate data mining? do you have any of it on your computer? do you have any idea how widespread those tools are and the knowledge to go with them? or to what extent and in what locations it is a normal, legal activity to look inside files stored on your own computer?

Do you know how many other people check inside the Warframe files on their computers?

Do you know what hacks were done to reveal Umbra? do you know what data was required to achieve the hacks, or how it was obtained?

When you state like it's some kind of known fact that Void_Glitch is 'literally the only one that has tools' then it's not about hate, it's about calling out false statements.

At least try to act responsibly a little bit, thanks.

Edited by yingji
  • Upvote 3

Share this post


Link to post
Share on other sites
23 minutes ago, yingji said:

'Literally the only one that has tools'? This is why I advise Void_glitch to stop posting ... 

Do you personally know what software is used to facilitate data mining? do you have any of it on your computer? do you have any idea how widespread those tools are and the knowledge to go with them? or to what extent and in what locations it is a normal, legal activity to look inside files stored on your own computer?

Do you know how many other people check inside the Warframe files on their computers?

Do you know what hacks were done to reveal Umbra? do you know what data was required to achieve the hacks, or how it was obtained?

When you state like it's some kind of known fact that Void_Glitch is 'literally the only one that has tools' then it's not about hate, it's about calling out false statements.

At least try to act responsibly a little bit, thanks.

He himself has said many times that the tools he uses to see a lot of the stuff have been done by himself and kept to himself.

I know there's cahce extractors and sound convertors -- I have them. But there is more to it, that alone can't show all the stuff VoiD has access to. I can assure you.

EDIT: I know I wasn't inventing things:

 

Below VoiD confirms this.

Edited by NightmareT12
  • Upvote 4

Share this post


Link to post
Share on other sites
58 minutes ago, Koldraxon-732 said:

<stuff>

Easier way.

Don't encrypt gameplay data. Take action against anyone breaking the encryption on the rest of the files (and change the encryyption used)

 

But...no, cat's out the bag now. Lawyers letters are irreversible.

Share this post


Link to post
Share on other sites

If Primed Streamline can really be traded then that points to a flaw in the game's architecture: Inventory is stored in a database on DE's servers. If I can use my game client to put items which aren't released yet into the server-side inventory database then something is seriously wrong. 

E.g even if I have the source code to my home banking software I couldn't just use it to double the amount of money in my bank account.

Due to the peer-to-peer architecture I assume there's some trust involved that my game client is not lying about what dropped during my mission. But the server still knows what node I was on and can at least check if what I say dropped is plausible. Only then modify my inventory. A Primed Streamline wouldn't last through the next synchronization of my inventory. If you find one then please ban my account.

And AFAIU trading doesn't have to involve trust at all: This can happen directly on the server only. Reduce amount in one database record and increase it in another. And use a database transaction for it to prevent partial updates if something goes wrong. Just like other software is doing it for multiple decades now.

PS: I nevertheless love this game and like what you're doing with it. Keep up the great work :clem:

Edited by zoknoz
fixed the clem emote
  • Upvote 4

Share this post


Link to post
Share on other sites
21 hours ago, (PS4)VoiD_Glitch said:

I was not involved in the situation in which unobtainable contents were hacked into the public build of the game.

While that may be true, how did you produce original ingame (presumably captura) images of Oberon Prime Access, which you subsequently posted here

21 hours ago, (PS4)VoiD_Glitch said:

Datamining does not equate to hacking

Between the two, what would you call preemptively obtaining and showing off a model of an unreleased primed warframe?

Edited by DVNC
  • Upvote 2

Share this post


Link to post
Share on other sites
18 hours ago, [DE]Rebecca said:

Soon we'll have a webpage that is automated to provide drop information for those who seek it - this screenshot is just proof of concept that we've got something in the works (using real data with effective rarity % in this particular vault!)."
 

My question is why you decided to take this very heavy handed approach to, what you have known to be a problem for a long time without having the aforementioned site in place first ?

Instead we are now left with the word "soon" which coming from DE carries next to no weight (for me at least), when i consider the amount of times "soon" has been tossed out as a response on Dev streams only to see the subject forgotten or delayed to such an extent, that calling it "soon" is just a slap in the face to the community.

 

 

  • Upvote 7

Share this post


Link to post
Share on other sites
20 hours ago, Chipputer said:

 In all honesty, the datamining information was only useful to people who pretended it gave them more information than it did. At one point it was useful for finding alt-fire stats. Those are in the game now. Others seem to think it's useful to know that a rare item is exceedingly rare, after all. Whatever. I'm not here to judge what you spend your time obsessing about if you're one of those people. I'd rather just play the game and get it when it comes. Publishing those rates, in an official capacity, isn't going to suddenly make you have an easier time getting those items.

Sorry I'm calling bullshlt on you here, remember the whole fiasco with Chesa kubrows not being able to be hatched from farmed eggs?

Or that other time they accidently gave a prime part a ridiculously low drop rate?

There is absolutely no way ppl would have figured that out quickly without dataminers

How many months would ppl have gone farming fruitlessly for something that they actually had no chance of getting because DE messed up?

Those 2 situations highlight why data mining is important.

Had we all took your "I'd rather just play the game and get it when it comes" attitude we'd all still be wondering why tf that prime part isn't dropping and why tf is that kubrow not hatching.

But yeah sure lets act high and mighty, and pretend we're superior to people who just dont want to get screwed over

Edited by Dragazer
  • Upvote 8

Share this post


Link to post
Share on other sites

144A4C0C4989200726DB88

Wondering about decorations (or distractions) this time, because we certainly have our fair share of disconnection and disorder at the moment.

  • Upvote 2

Share this post


Link to post
Share on other sites
2 minutes ago, maybe_not said:

144A4C0C4989200726DB88

Wondering about decorations (or distractions) this time, because we certainly have our fair share of disconnection and disorder at the moment.

Don't forget the good measure of dilusion too....  Lot's of folks that work for non-profits and bring knives to gunfights walk among us. 

Share this post


Link to post
Share on other sites
19 hours ago, --B--Kyle- said:

You know what would be awesome? Just an idea ... On the screen you choose the relic drop, could have the ducats value of each piece, this could help who is farming prime items just for Baro.

This. Or at least color code the text on relic drops to correspond to their rarity.

Not knowing how rare things are, with like 10 seconds to make a choice, is one of the worst aspects of the game. 

 

Pretty sure rainman is the only one who can remember the value of all components. 

  • Upvote 1

Share this post


Link to post
Share on other sites
2 hours ago, Zeranov said:

Seriously Rebecca...  a website might not cut it. Destiny did it's lore via Smartphone and it was aweful. Long term the values need to be implemented into the Codex... with a tutorial what the Codex is in the first place.

Yes. The information should be in the game to begin with. Codex makes the most sense to me. 

  • Upvote 3

Share this post


Link to post
Share on other sites
18 hours ago, Tesseract7777 said:

A couple things here though: 

1) From what I have understood, they never blamed void_glitch directly for the hacks, or said they did them. Rather, they said that info he posted online made it easy for hackers to reverse engineer things and hack the game, getting themselves things like Excalibur Umbra and Primed Streamline. 

VoiD_Glitch got unfairly screwed over, there really isn't any other way to put it.

If you are good enough to be able to hack Excal Umbra and Primed Streamline into the game, then data mining the information yourself to do so would be a piece of cake.

 DE as usual has not addressed the underlying problem (hackers) and has instead decided to take the easy way out (Which doesn't work anyways) at someone else's expense.

This solves nothing, people are just going to datamine as usual, but now they won't be able to publish it online.

Those with the the skill and intent to hack things into the game, aren't going to be stopped by this, they'll just get the info they need themselves 

Edited by Dragazer
  • Like 1
  • Upvote 13

Share this post


Link to post
Share on other sites

I respect your need to protect content that you want to remain a secret, especially in terms of upcoming content spoiler.

Your censorship of game technical data is unacceptable, especially since the datamine is currently the only way we have of holding your team accountable to what you are supposedly promising; the situations around the Chesa Kubrow and Archwing weapon acquisition come to mind. If you are concerned that the datamine makes your design practices look bad, then maybe you should focus less on the datamine and more on your design practices.

I also have absolutely no faith that you will not be pursuing 0.000001% drop chances now that you have proven that you are legally able to strong-arm anyone who obtains evidence that you are doing so from speaking out against it. The claims that your team "play like the rest of us" are also laughable, given the absurd amount of poor design choices that players still have to deal with that at least one of you would have surely picked on by now.

Poor form, I am sorely disappointed by your decision and your lack of any intention to actually co-operate with Void to reach a compromise.

  • Upvote 10

Share this post


Link to post
Share on other sites

So this is not good at all.

Let me be clear here, DE, you've lost a lot of respect from me.

  • It is utter nonsense that anything that void_glitch did contributed to "hacking" of player account databases. The data he has up was braindead simple stuff, some lua, reward decks and string tables. If you have a flaw in your client and/or server that allowed someone to inject object ID's into their inventory, that your servers then trusted? You should not be pointing fingers elsewhere.
  • This will not stop people trying to hack your client, end of list, it will provide you with no extra security.
  • Drop chances are not the only things that datamining gave us. It's nice you're planning on picking up, and spending resource on, some or what players were getting from the datamine but let's be honest, even your supposedly "systemic" population of codex data keeps leaving things out because you do fail to plumb in new routes to data and/or break old routes... repeatedly. Why should we believe that this will be any different. You have minimal motivation to dedicate more time to it's maintenance.
  • This was a fan who was doing your work for you and doing it better than you have shown yourselves to be interested in doing (I don't begrudge your "content before rigour" style, but if you don't do it right then someone else will do it for you)
  • Finally why do you keep letting people check secrets into your live DB branch? I know you're probably using perforce and it sucks but seriously, just update your tooling. In you really can't merge later then have pre-commit scripts that blank out DB entry fields that are sensitive or come up with a better way of representing your DB patches. Other people have solved this problem.

Obviously, you're a company, you can do what you like with your "IP" but remember, your income stream is different from many other companies it is much less a function of the stuff you make, it's more a function of our desire to give you cash, which changes drastically when you make player-hostile decisions.

And whether I agree with each issue or not this is not the first wedge issue that has notable number of the player-base seriously unhappy with you.

And Rebecca, You do a wonderful job and it was lovely to meet you in Birmingham but this post is really not reassuring in the slightest. It's really vague and/or conflating issues to the point of wilful inaccuracy in some places, which may be what you were told to do, but it doesn't look good at all, at least to me.

Edited by SilentMobius
  • Like 1
  • Upvote 18

Share this post


Link to post
Share on other sites

I understand Rebecca's hands are tied now that Legal is involved... but PMing Void Glitch while publicly going 'Ya we can find a compromise! It'll be awesome. We can put this behind us and everything'll be smooth sailing' only for the actual message to be 'talk to legal.' What did you think would happen? I realize you're in a personal no-win given publicly going 'lawyer up' is seen as stonewalling, but putting on smiles and happy talk publicly and then privately point to legal is DISHONEST stonewalling. 

All of this is happening at the worst time for DE given Tennocon.

The overriding thought I have is 'how do we make things right on all ends with the least amount of bloodied noses?' I don't know. Smarter people than me are involved with this and they say DE technically doens't have a case, but given the financial imbalance they could just bury Void in legal if they so chose.

 

I hope they do not, as that would further cement an 'us v them' attitude.

  • Upvote 3

Share this post


Link to post
Share on other sites
20 hours ago, [DE]Rebecca said:

Free-to-play is done differently by each developer - and we're a developer that has basically signed ourselves up to all aspects of making free-to-play our own. When we do free to play, we build systems that immediately present players with a choice: time or Platinum. Defeating an RNG god is a privilege many of us have shared. Each year (five and counting) that we add to Warframe's development adds to not only game complexity, but to how Digital Extremes maintains this balance and proposition.

I'll start this post off with a picture (that we'll come back to at the end): Back in November 2016 we colour coded our Sortie Drops to better indicate rarity as seen here:

smI9iRtfbACqGZvNZ629ER6w70evvzEP7srnz8AG


 

Here we have Rarity colour conventions to set expectations about what you'd be likely to receive when playing a given Sortie. This was an attempt to display the key value that these datamines provided: a value proposition of chance and time investment, and that's not something we want to take away. This is something we've been actively iterating on, but now we have an urgency to do it faster.

But a visual guide isn't as good as numbers; datamining provided those. That's something we lived with and at some times appreciated until the revealed contents were being hacked, sold, and added to the public build.

What happened yesterday (that was discussed on Reddit)  isn't about hiding drop rates for us. We're working on sharing these on our website in a procedural, automatic way (similar to recent Chinese laws, but universally)... but it's not about that. This is about reverse engineering and vectors being opened with the intent to scam trade, forge items, create 3rd party servers and generally leak unfinished content and game internals that can be just as misleading as undisclosed information. Seeing a trade for Primed Streamline take place for thousands of Platinum meant people were about to get caught in the crossfire of tolerated data-mining and malicious use of datamined information. We sought legal counsel when hacking took place and evidence piled up, never before then. We acted to stop the willful and selfish compromising of the game’s code for personal gain. How Digital Extremes decides to share its information is not up to the discretion of lone individuals.

Soon we'll have a webpage that is automated to provide drop information for those who seek it - this screenshot is just proof of concept that we've got something in the works (using real data with effective rarity % in this particular vault!).

53USo3gnQupcjeFKZ4IUDppgsmQHWWBVY8-bjbqB

We asked you who does it best so we could figure out a quick way (since the urgency is on) to reflect a currently appreciated method of developers giving %'s. Where could we seek wisdom of other developers’ actions to demystify the RNG process and publish drops? The trick is that this behaviour is not strongly or exhaustively exhibited elsewhere in the industry (unless the law demands it in certain regions). There were a few (only one from a game I play), but clearly we have an opportunity here! We want to set a new standard of transparency with our community and give our players further insight into the processes in which they are assigned rewards.

What's next will be the damage stats you can't see (ex. Exalted Weapons) which we may rely on adding to the Wikia (however temporarily) all in the name of empowering our players who value these stats to build as detailed as they want.

Walking the road of free-to-play is something we've been doing alongside our community since day one of Warframe. Each hiccup has led to us being better developers and this is another one of those moments. There is work involved in getting this information up, but stay tuned for the soft launch which we'll announce when it's complete on our end!

I still don't get why warframe players care so much about drop % numbers? Still have to play to get what you want. Or buy it. So is this the point of all this nonsense.? Bc a random mod has .002% drop rate, instead of getting dirty it's now about the easier route. Buying it? Don'tmatter to me either way, DE is amazing for what they do & hope it continues.

Share this post


Link to post
Share on other sites
9 hours ago, ..atom.. said:

1. more openness about drops and mechanics is always good and should have been in the game already. DE should really be thankful of the community maintaining the wiki and even dataminers for sharing details like drop rates, exalted weapons mechanics, melee stats like range, etc

2. How long is it going to take before lawyers realize that the simple act of trying to repress something they don't like online is likely to make it so that something that most people would never, ever see (like a photo of a urinal in some random beach resort) is now seen by many more people? Let's call it the Streisand Effect.

— Mike Masnick

if someone thinks editing my comment and deleting my opinion on the matter will do anything good for the company image i can assure them it will do the opposite and undermine confidence even more. 

this was taken out by moderators apparently:

"x. i think going the lawyer route against Void_Glitch for something he apparently didn't do, is a bad way to handle things instead of 1st trying to talk to the guy. This kind of stuff will undermine confidence and sales way more than any datamining can."

this is not an attack on DE ... the thing is players calling them out on bs is an act of love ultimately, we do this because we love the devs, the community and the game. it's all we can do to try to keep them on the straight path because like DE, we the playerbase want to keep this collaboration going. i understand that spirits are high but please put the guns down, shooting without reason will only make it worse and i know nobody really wants that. 

  • Upvote 11

Share this post


Link to post
Share on other sites
26 minutes ago, MarrikBroom said:

I understand Rebecca's hands are tied now that Legal is involved... but PMing Void Glitch while publicly going 'Ya we can find a compromise! It'll be awesome. We can put this behind us and everything'll be smooth sailing' only for the actual message to be 'talk to legal.' What did you think would happen? I realize you're in a personal no-win given publicly going 'lawyer up' is seen as stonewalling, but putting on smiles and happy talk publicly and then privately point to legal is DISHONEST stonewalling. 

All of this is happening at the worst time for DE given Tennocon.

The overriding thought I have is 'how do we make things right on all ends with the least amount of bloodied noses?' I don't know. Smarter people than me are involved with this and they say DE technically doens't have a case, but given the financial imbalance they could just bury Void in legal if they so chose.

 

I hope they do not, as that would further cement an 'us v them' attitude.

How is "I have sent you a PM via reddit." interpreted as that? I mean, I was also hoping it would be something else but seriously lol

  • Upvote 2

Share this post


Link to post
Share on other sites

So when will the primed streamline be released to the user?

  • Upvote 1

Share this post


Link to post
Share on other sites
Guest
This topic is now closed to further replies.

×
×
  • Create New...