Not insignificant flaw with cache optimization

[doomagent13]

Just as a forewarning, I do not know the details of how cache optimization works.  This is all based on fixing a problem I had, as well as observing the file system while optimization ran.

 

22.7.0 added an automatic trigger for cache optimization if enough space was wasted.  It had been a while since I manually ran optimization, so it triggered.  However, it got stuck at 37%.  Repeatedly, for extended periods of time, and despite running Verify.  Between Resource Monitor and going to the folder with the cache files, I figured out the issue.  The way it performs the optimization involves more or less duplicating the cache file being optimized.  It wouldn't be THAT big of an issue, except the largest cache file is OVER 14 GB. (To be specific, it is "F.TextureDx9.cache")  At the time, I only had about 10 GB free hard drive space, so the optimization couldn't be finished. (It's a 250 GB partition on a 4-year old MacBook Pro.)  I ended up uninstalling Source FilmMaker to free up enough space.

 

There are two problems that jump out at me.  First, the launcher doesn't give any indication if it fails due to a lack of disk space.  It just sits there, part way through the process.  Not even the logfiles gave any indication of what the problem was.  Second, it has to duplicate such a large file.  I don't know if optimization could be changed to no longer rewrite the cache next to the original, but I would think the biggest cache file could be broken into 2 or 3 separate files.  Just for comparison, the 5 largest cache files right after optimization weigh in at ~14.8 GB, ~2.7 GB, ~1.7 GB, ~1.1 GB, and ~350 MB.  Given the current number of cache files, I am inclined to think 1 or 2 more would not have any significant impact on load times, while it could potentially make updates and optimization faster and/or more efficient.

[taiiat]

this sounds like a fallback method, if the system does not have enough Memory to process the files in Memory.

also btw having prolific amounts of Memory is very good for Optimization speed.

[TheDarkness506]

So t h a t's why it needs 14 GB of disk space to free up 640 MB! Seems reasonable.