Jump to content
pittax

Optimising download cache is murdering your SSDs

Recommended Posts

I've noticed that cache optimisation has been acting crazy last couple of months (maybe before too and I just have not noticed). The size of the installation directory appears to double in size, and constant disk writing implies that it essentially copies everything, doing god knows what with all the processing power it uses. The first time I noticed this was when the installation size of warframe more than doubled, I ran out of disk space and the whole process hung up at 85ish percent.

At very least it should check that it has the available disk space and when it is running out of space. However in the days of SSDs and limited writes the whole behaviour in itself is not acceptable imho.

It is currently faster and less damaging to my hardware if I just delete the warframe and redownload it, instead of trying to optimise the cache (especially if I do it every other update or less frequently), which is just ridiculous.

  • Haha 3
  • Applause 1
  • Upvote 1

Share this post


Link to post
Share on other sites

Isn't optimizing the cache essentially just defragging the install folder? You don't defrag an SSD, cause it has access to all sectors at once, unlike a spinning disk. So, if you have it installed on an SSD and it tells you to "Optimize the cache", just ignore it.

Share this post


Link to post
Share on other sites
Just now, Gabbynaru said:

Isn't optimizing the cache essentially just defragging the install folder?

No it is not.

Optimizing the cache does a lot of things behind the scenes, such as better compression of some objects, cleaning out of unneeded items, rebuilding an actual "cache" of specific assets and files that helps the game to load faster.

If a cache gets out of date it can cause a variety of problems, including actually causing bugs that only occur until you re-cache things.  As an example I had a friend that refused to recache things and he got graphical weirdness all the time until he finally decided to optimize his cache...which also happened to reclaim nearly 2 gigs of space.

  • Like 4

Share this post


Link to post
Share on other sites
1 minute ago, Tsukinoki said:

No it is not.

Optimizing the cache does a lot of things behind the scenes, such as better compression of some objects, cleaning out of unneeded items, rebuilding an actual "cache" of specific assets and files that helps the game to load faster.

If a cache gets out of date it can cause a variety of problems, including actually causing bugs that only occur until you re-cache things.  As an example I had a friend that refused to recache things and he got graphical weirdness all the time until he finally decided to optimize his cache...which also happened to reclaim nearly 2 gigs of space.

Huh... Thank you for letting me know! Learning something new every day, I guess.

By the way, wouldn't tweeting Steve get better results than just posting it on the forums? Cause this sounds like a serious issue they should look into ASAP.

Share this post


Link to post
Share on other sites

i plan to upgrade on ssd my potato slow down alot wile optimizing cache not to say it takes ~25min

Share this post


Link to post
Share on other sites

Optimising is indeed extremly taxing on older SSDs.

Newer ones should survive the process a lot better.

Edited by Walkampf
  • Haha 1
  • Upvote 1

Share this post


Link to post
Share on other sites
7 minutes ago, Walkampf said:

Optimising is indeed extremly taxing on older SSDs.

Newer ones should survive the process a lot better.

I've NEVER had this issue, what so ever, so I'm not sure whats going on.

  • Upvote 1

Share this post


Link to post
Share on other sites
1 hour ago, pittaxx said:

However in the days of SSDs and limited writes

People are still spreading this misconception?

Unless they're running some off-brand piece of crap bought from Aliexpress, chances are that an average user will experience disk failure for some other reason WAY before they hit the write limit.

  • Woah 1
  • Upvote 1

Share this post


Link to post
Share on other sites

During the normal update process, the Warframe cache gets more and more mixed up between old and new files so that the files aren't in a logical order anymore.  Also small amounts of disk space gets wasted, where an old file and its updated version aren't the same size.  The cache optimization process sorts and organizes everything, so that  1) chunks of old unused files are removed so that wasted disk space is given back and 2) files are sequenced in a way that's faster to load.  If you are concerned about the longevity of your solid state disk, you might choose to only run the Optimize process once in a while instead of every time it asks. It won't hurt anything aside from making the game take up more space than necessary and load a little slower for you.

  • Like 1
  • Applause 3
  • Upvote 1

Share this post


Link to post
Share on other sites
vor 1 Stunde schrieb Tsukinoki:

No it is not.

Optimizing the cache does a lot of things behind the scenes, such as better compression of some objects, cleaning out of unneeded items, rebuilding an actual "cache" of specific assets and files that helps the game to load faster.

If a cache gets out of date it can cause a variety of problems, including actually causing bugs that only occur until you re-cache things.  As an example I had a friend that refused to recache things and he got graphical weirdness all the time until he finally decided to optimize his cache...which also happened to reclaim nearly 2 gigs of space.


I was also sometimes unable to load into the plains right after the game told me to optimize the cache. But I'd rather re-download the game than optimize it.

Share this post


Link to post
Share on other sites

perhaps you have a point on checking if there is space first before going ahead with trying to Optimize, if the Launcher isn't already doing that.
i don't run so low on Disk space as to find out whether that is actually the case or not however.

outside of that there is at best no difference between Optimizing and redownloading the entire game. and it's going to usually be worse because you have to write all 35GByte to your SSD instead of spending most of the time reading and then moving what data needs to be moved.

 

2 hours ago, Gabbynaru said:

Isn't optimizing the cache essentially just defragging the install folder?

no.
specifically, Hard Disks get Defragmented, but most of what Optimizing does is ensuring that the local data is..... in an optimal location to be read efficiently from the file Index, as well as determining if there's data that should be discarded (not needed anymore).

1 hour ago, Pr1A said:

Unless they're running some off-brand piece of crap bought from Aliexpress, chances are that an average user will experience disk failure for some other reason WAY before they hit the write limit.

while it is true that any relatively recent SSD will expect to have an endurance floor of ~700TByte written before performance starts heavily degrading, and 1-2PetaByte written before the NAND effectively fails... avoiding unnecessary writes isn't a bad practice.

  • Like 3

Share this post


Link to post
Share on other sites
8 hours ago, taiiat said:

well as determining if there's data that should be discarded (not needed anymore).

This is the only reason I run it when a large update happens.  Otherwise I would ignore it on my SSD.  

Share this post


Link to post
Share on other sites
1 hour ago, DatDarkOne said:

This is the only reason I run it when a large update happens.  Otherwise I would ignore it on my SSD.  

i don't run it often either - usually just when i know what an Update has changed will make a large impact on the game (like recently where the entire internal file structure was redone, that was definitely worth doing it for). hehe.

  • Satisfied 2

Share this post


Link to post
Share on other sites

On my NVME SSD, Samsung Evo 970 500GB, Ryzen 2400g. Windows 10 with anti-virus off and disabled.
Copy entire Warframe folder: 1m 16s, 36.1 GB, with about 12% CPU usage.
Warframe Optimize: 2m 8s, but only optimized 32.3 GB looked at date modified, CPU usage frequently jumps between 20% to 50%.

It might be anti-virus using up CPU usage. But, it is likely that Warframe itself use encrypted data that use lots of CPU and slow down optimize, not making full use of a turbo fast NVME SSD. Such encrypted data may also be why the Warframe loading screen is not as fast as we want it to be.

There is a reason to copy or re-write all data, it is SSD data decay slow down data reads to a possible data loss especially for TLC (triple level cells). There is Samsung Evo 840 firmware update that just automatically rewrite all old data to keep it at full read speed. https://www.google.com/search?q=samsung+840+evo+slow+read+update
The more expensive SLC (single level cell) and MLC (multi level cell) will withstand a whole lot more writes, like on some of Samsung Pro series.

  • Upvote 1

Share this post


Link to post
Share on other sites
3 hours ago, sam686 said:

The more expensive SLC (single level cell) and MLC (multi level cell) will withstand a whole lot more writes, like on some of Samsung Pro series.

Can you even buy SLC flash drives any more? I really do want one - longevity and reliability are pretty important to me. Not just in active use, but at rest - flash memory decays to nothing in a few years if not refreshed whereas hard drive platters will stay magnetised for tens of years. It's why I still treasure my WD Velociraptor drive - the fastest of the old spinning rust.

Share this post


Link to post
Share on other sites

Protip: Reading, writing, and shuffling files on a good old platter hard drive shortened their life too.

Using an electrical or mechanical device causes wear of sorts. Be it wear on bearings and such or electromigration in memory circuits, simply by using a device, you are shortening its life just by using it. If you cry and cower under you desk about 'wearing the drive out', then you'd better just stop using it altogether. If you have an older less robust drive, then maybe you should replace it with a new one. But unless you enjoy waiting longer on load times and having old junk clutter up your drive for no reason, let it clean up the cache. Most of you will probably upgrade your PC before you have to worry about drive failure unless you have terrible cheap junk in it anyway.

Share this post


Link to post
Share on other sites

Modern MLC drives have a ton of writes on them. TLC I would probably be more concerned, esp if it's the OS drive but if it's for games MLC will last plenty long on average. Usually to the point the space it provides is no longer viable. This is esp true for M.2 drives since most MB have 2 slots and after that you're using PCI-e slots.

Most users are more likely to buy a new SSD out of necessity of space rather than cell failure. Obviously you want to avoid strain through writes and keep the drive low on space much as possible but there's not much need to worry about it.

 

14 hours ago, DoomFruit said:

Can you even buy SLC flash drives any more? I really do want one - longevity and reliability are pretty important to me. Not just in active use, but at rest - flash memory decays to nothing in a few years if not refreshed whereas hard drive platters will stay magnetised for tens of years. It's why I still treasure my WD Velociraptor drive - the fastest of the old spinning rust.

Yea but for the price you could just buy two MLC drives for the same GB per dollar and still end up with more life span by manually doing a mirror copy every once in a while leaving the drive otherwise completely unused. Outside the cell wear you lower the probability of other device failures by having two with the same data.

Just not worth the money unless it's for commercial use.

Share this post


Link to post
Share on other sites
3 hours ago, Xzorn said:

Yea but for the price you could just buy two MLC drives for the same GB per dollar and still end up with more life span by manually doing a mirror copy every once in a while leaving the drive otherwise completely unused. Outside the cell wear you lower the probability of other device failures by having two with the same data.

Just not worth the money unless it's for commercial use.

That's... actually quite a fair point. Hmm.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×
×
  • Create New...