The Great Ensmallening

[[DE]Glen]

TLDR: We’re going to free up 6.6GB of space on PC next week but we need a 6.5GB update to do it. Most platforms will see similar improvements.

My memory is hazy about those early days back in 2012 when we launched our Closed Beta but I distinctly remember the entire download being small enough to fit on a CD-ROM. In the 8 years since launch we’ve seen the install image grow over 100 times in size and when you consider all the amazing content that was added over the years it’s easy to see why: that’s three open worlds, over a dozen tile-sets, 28 quests, almost 500 weapons, and over a thousand mods, to say nothing of several remasters that updated older content.

There’s so much content in Warframe now that some people can’t fit it on their smaller SSDs and have to run it off of a slow HDD instead. There’s so many things in the game that even for people with large NVME SSDs the game can’t always load everything as fast as we’d like. Luckily we have found some new technology that is going to help us -- we’re going to make the game smaller and load faster – but there’s a catch: you need to download remastered content.

We mentioned in the summary we’re going to free up 6.6GB of space in but in fact the real number will be much larger than that: we’re probably going to free up at least 15GB by the time we’re done. To reduce the pain for people updating we’re going to break these remaster updates into three parts spread out over the rest of 2020. We’re also taking advantage of these “mini-remasters” to make quality and performance optimizations to formats we have haven’t touched since we launched (back when DirectX 9 was our only supported renderer!).

One of the things we’re doing to reduce our footprint involves changing how our texture data is compressed. Although it isn’t a perfect analogy it’s similar to the trade-off between image-quality and file-size you get with different image formats: PNG is lossless but can mean very large files, JPEG files can be very small but may contain visible artifacts of the compression. The technology we’re using is called Oodle Texture and we’re extremely pleased with the savings we’re getting: our files are roughly half the size on disk and the visual differences are negligible.

In this first update we’re going to change our Lightmaps. We spent a lot of time carefully analyzing the quality of the results and we doubt anyone will even notice anywhere but the harshest of tests. That said, here are some of the most challenging cases we found (we’ve disabled the diffuse textures to further expose the subtle differences between them).

Before

After

Before

After

The nice thing about this solution is that we can easily tweak where needed: the game can default to the compact and fast format and we can increase resolution to preserve the quality of special assets (like that shelf in your quarters).

In the second update we will apply this same technology to the rest of the textures in the game and we expect we’ll see similar if not more savings. Again, we spent time looking carefully at the quality tradeoff and again we’re extremely pleased. Here’s a particularly challenging case: a normal-map:

Before (19 MB)

After (8 MB)

In this case the compression reduced the footprint from approximately 19MB to 8MB and the differences are extremely difficult to discern.

As someone who recently had to attach an external USB drive to my vintage test laptop from 2013 I’m personally very excited to see these changes roll out. I know the download will be inconvenient but getting 15GB of storage back and doubling my effective texture load-bandwidth is going to be awesome!

 

[Raylo555]

neat

[D1videdByZer0]

Could also lead to FPS increases across the board, hopefully.

[-FrutyX-]

CoD MW should take an example from this ... Great work !

[DrLego]

Good stuff. Definitely something worth the effort to do.

[RaahOne]

Really excited for this, my old lappy will be happy.

[tarfeef101]

Next up: an option to get textures at a higher resolution and say "screw disk space"?? 

 

Cool stuff though, I go through days of effort to shave MBs off of docker images, so saving this much will be a great benefit for players and your CDN, I'm sure.

[Letter13]

4 minutes ago, D1videdByZer0 said:

Could also lead to FPS increases across the board, hopefully.

Probably not, the textures are being compressed but are remaining the same resolution, meaning the graphics API is still having to process the same amount of pixels when it loads textures into VRAM or manipulates the texture.

[(XBOX)AMONGTHEWEAK]

The compressening? 

[PookieNumnums]

sounding kinda thicc. can we get a show hoof glyph?

[Proscriptor]

faster loading sounds awesome

[Shinyah]

Sometimes you need to make a choice to lower the size of a game:
- The hard way : Optimize the game
- Or the easy way : Remove content (like planets for example) 😉😉

Great job DE !

[.LeiA._]

Tis a great news! Now I have more space for games and porn!

 

[Zahnny]

I'm not tech savvy enough to understand all of this, I just hope such major changes won't cause a massive influx of bugs.

[PookieNumnums]

1 hour ago, [DE]Glen said:

In this first update we’re going to change our Lightmaps

is this going to improve how effective and responsive Mirage's Eclipse is to the light/shadow in game? It is so wonky at the moment that some shadows are triggering damage buffs, and some really really super duper bright areas are not registering any light at all. Hydron is an easy example.  Right up on the defense pod. One second youre in the light... one step over youre in the dark... but no visible change in shadows on the floor, on your character, or anywhere...

[PookieNumnums]

1 hour ago, .LeiA._ said:

Tis a great news! Now I have more space for games and porn!

 

people still save pr0ns in 2020? i thought thats what the internet was for..

[(PSN)Yuki5e7en]

Will that minimize my ps4's loud fan noise? 

[PookieNumnums]

1 hour ago, (PS4)Yuki5e7en said:

Will that minimize my ps4's loud fan noise? 

no. 

[Brick]

Sweet. Improvements like these are always great!

[[DE]Glen]

20 minutes ago, PookieNumnums said:

is this going to improve how effective and responsive Mirage's Eclipse is to the light/shadow in game? It is so wonky at the moment that some shadows are triggering damage buffs, and some really really super duper bright areas are not registering any light at all. Hydron is an easy example.  Right up on the defense pod. One second youre in the light... one step over youre in the dark... but no visible change in shadows on the floor, on your character, or anywhere...

No because Eclipse doesn't sample from the lightmaps.

[-Haxboi-]

Juicy! This looks so good!

[Draakon0]

Could you go into detail why release it in 3 different parts instead of 1 big one? Like what is making it "more painful" for us as players if it comes in 1 big part.

[MaraWuti]

7 minutes ago, [DE]Glen said:

No because Eclipse doesn't sample from the lightmaps.

...then what does she use?

[NightmareT12]

40 minutes ago, tarfeef101 said:

Next up: an option to get textures at a higher resolution and say "screw disk space"?? 

I second this, please. I love the game artstyle but sometimes the differences across Warframes themselves are a bit too much: You can get super clean textures like with current Mesa Prime, and sometimes the textures will look squarey (ie: the tintmask on Excalibur Umbra or Chroma Prime), which hits a bit visually...

Also, I need to ask (mainly to understand), I see in the normal maps a difference in the "black spots" where they are less accentuated, why is that?

Thanks for the comprehensive and detailed explanation as well, Glen 🙂

[[DE]Glen]

6 minutes ago, Draakon0 said:

Could you go into detail why release it in 3 different parts instead of 1 big one? Like what is making it "more painful" for us as players if it comes in 1 big part.

Some people have to work with bandwidth quotas so by spacing it out over multiple months we can try to help people keep under their limit.