Looking for Leaks

[Cray]

Meditating in a relay right now, game minimized. Will check in this afternoon.

[Cherac]

14 hours ago, Cherac said:

tl;dr: 980, i7, W10 64, 375.70, before and after ~6ms

  Reveal hidden contents

Apollodorus 60 minutes

Frametime:
 Before: 5.9-6.0 ms
 In mission: ~7.0-11.0 ms
 After: 6.1-6.2 ms
 
Loadout
 Mesa
 Dera Vandal (Corrosive/Heat)
 AkMagnus (Toxin)
 Venka Prime (Corrosive - or Viral, can't remember)
 Carrier Prime (no weapon)

Mission Details
 Used:
    All weapons
    All warframe powers
    Ground slam/channeling (for particles)
    Focus power
    Energy restores
 Looted:
    Vacuumable pickups
    Chests/lockers/resource nodes
    Convergence
 Stats:
    1774 Total kills (169 head, 184 melee)
    37 Upgrades
    132 Abilities
    1872 Pickups
    2 Alt+Tab while paused (esc menu)

Related PC Specs
 24" IPS 1920x1200 @ 60
 GTX 980 4GB @ 1190? (stock speed)
 i7 4770 @ 3.4
 8GB DDR3 @ 1866
 375.70 drivers
 3.1.2.31 GeForce Experience
 Windows 10 64bit on boot SSD
 Warframe on storage SSD
 Steam on storage HDD

Non-default Global 3D Settings
 Anisotropic Filtering 16x
 DSR 2.00x
 Smoothing 20%
 MFAA enabled

Ingame Graphical Settings
 Display Mode - Fullscreen
 Resolution - 2715x1697 (DSR 2.00x)
 Refresh Rate - 59.95 Hz
 Aspect Ratio - Auto
 Vsync - Off (normally off)
 Max Framerate - Unlimited (normally 120)
 PhysX - Enabled
 Tessellation - Enabled
 Local Reflec. - Enabled
 Blur Reflec. - Enabled
 Volum. Light. - Enabled
 HDR - Enabled
 Adapt. Expo. - Enabled
 Glare - Enabled
 Ambi. Occl. - Enabled
 High Shader Q. - Enabled
 Geometry - High
 Particles - High
 Shadows - High
 Textures - High
 Anisotropic - Off
 Trilinear - On
 AA - SMAA(Medium)
 DoF - Disabled
 Motion Blur - Disabled
 Bloom - Enabled
 Intensity - ~1/5
 Color Correc. - Disabled
 Dyn. Lighting - Enabled
 Char. Shadows - Enabled
 Weapon Trails - Disabled
 Elemental FX - Disabled

Launcher Settings
 All enabled
 Verified/Optimized after 19.1.0 or something

Running programs
 Steam, overlay + FPS counter
 Discord, no overlay
 Shadowplay, overlay, not recording
 Firefox, 5 tabs, one being a music stream (HTML5)
 Chrome, 2 tabs
 Logitech Gaming Software, mouse, Warframe key bindings
 Corsair keyboard software, Shadowplay key bindings
 DisplayFusion, 300+ wallpapers on random shuffle
 
Other
 I don't know if this is related or not, but everytime I load into the orbiter my framerate drops to about 3 FPS for a few seconds.
 It happens even after a fresh launch/login. I also get a few small hitches when moving from the front of the orbiter to the
 arsenal station for the first time after loading in. I do have quite a few decorations in the transference room though.
 
 I also tried to record something in Warframe this week and the Shadowplay overlay was bugging out a bit. It was very unresponsive at times.
 This one might be related.

 

Woke up to a windows update this morning. Ran the update (between two versions of 1607), ran 40 min Apollodorus with the usual settings seen above and still nothing.

For what its worth, I also remember having somewhat similar performance issues a while ago. I think it was some time before the Index Preview so it's been a while, but I specifically remember it started occurring after either a Windows update or an Nvidia driver update. Something was leaking memory and filling up my RAM forcing me to reboot at least once a day to clear it. This was causing performance degradation in video games and I think Warframe suffered the most. I narrowed the leak source to my browsers and while I did change a few things at once, I'm pretty sure disabling hardware acceleration fixed most of it. Things I remember increasing the leaks (filling up the RAM faster) were tab hoarding/quickly opening new pages or watching streams (Twitch, Youtube videos, Crunchyroll.) I used to have a second monitor at the time and often had some Twitch stream open on it while playing.

I compared the replicated results here again and 7/8 had a browser open while the last one had Rainmeter which I think can use hardware acceleration. It was around 3 or 4 am when I did the comparison so I might have misread or otherwise mistaken something.

Anyways, going with this I ran a new test before I went to bed:

• Enabled Firefox hardware acceleration
• A tab with a Twitch stream open
• A tab with a Youtube music stream open
• Task manager, RAM usage ~60% out of 8GB after having the PC in active use for the whole day
• Warframe, initial frametime ~6ms
• Apollodorus
• ~7 min, small hitch when I walked into a large room in the tileset
• ~9 min, huge fps drop when I unleashed the focus power, recovered after a second or so
• Paused the game to check the RAM usage and it was hitting 97-99% without anything really hogging up that much space, top hog Firefox was showing <1GB
• Continued to play, two more hitches, one was after blowing up a red barrel, another after killing multiple enemies at once
• More framerate/frametime fluctuation than usual, noticed the game got smoother once which means the framerate degraded at some point but was able to recover
• Extracted at 20min
• Frametime after mission again ~6ms so no reproduction

Edited December 10, 2016 by Cherac
Missed some details

[Romiith]

6 hours ago, [DE]Glen said:

I wonder if we can attack this another way: if you are one of the lucky ones that can reproduce it with PhysX effects enabled:

1. run Windowed or Borderless so you can alt-tab without smashing the driver
2. kick up Task Manager and in the Processes tab show the Commit Size and Handles columns
3. mark down the two values before, go do a mission, mark down what they are after

If it's a raw handle leak then a handle dump might provide some evidence.

The other reproduction I've heard of is to idle in a relay over night; if this works and you wake up to insane frame-times I might be able to get a clue from a procdump snapshot if you're pro with the command prompt. A dump is probably only going to catch it in the act if whatever's bloating is the overwhelming majority of the frame time -- eg: it's normally 2ms staring at the floor but after a long nap in the relay and back to the liset it's 60ms then you've got a good chance of it winning.

I went back to physx ON nvidia dvr ON and tried again. Saw the same performance degradation.  Before: Commit Size ~2.4M, Handles ~930. After: Commit Size ~2.6M, Handles ~930. Commit size has been rising steadily since I returned to the Liset and is still rising as I type this.

Edit: Maybe 5 minutes later, just standing around in my Liset I'm down now to between 30 and 40 FPS which is about 10% of what I expect. Commit size is up to 2.8M.

 

Edited December 10, 2016 by Romiith

[[DE]Glen]

2 hours ago, DeltaPhantom said:

Meditating in a relay right now, game minimized. Will check in this afternoon.

I *think* minimized might actually skip out all of the heavy lifting and won't exercise the driver so it might not be a fair test, sorry.

[[DE]Glen]

38 minutes ago, Romiith said:

I went back to physx ON nvidia dvr ON and tried again. Saw the same performance degradation.  Before: Commit Size ~2.4M, Handles ~930. After: Commit Size ~2.6M, Handles ~930. Commit size has been rising steadily since I returned to the Liset and is still rising as I type this.

Edit: Maybe 5 minutes later, just standing around in my Liset I'm down now to between 30 and 40 FPS which is about 10% of what I expect. Commit size is up to 2.8M.

Hmm ok so not a handle leak, that's too bad. Have you tried just booting fresh and idling in liset? I wonder if there isn't even a need to run a real mission.

[RextintorZ]

I just finished a 30 minute run on Apollodorus with all settings turned on, V-Sync off, PhysX on and frames limited to 120fps (monitor set to 120hz). When I first looked down at that spot, my frame time was at 8.3ms.

I joined the mission, brought Telos Boltace with my Nekros and kept with the spin-to-win strategy. Soon, I was able to use my Naramon Focus power and then I could get invisible practically the entire time.

I didn't notice any freezes or hitches during the mission, but after 15 minutes the Fame Time was oscillating between 8.3 and 8.4ms. After the 30 minute mark, I returned to my Liset, looked down at that same spot and the Frame Time was still at 8.3ms.

 

GPU: NVIDIA GTX 1080 G1 Gaming (Gigabyte)
Monitor: BenQ XL2720Z (at 120hz)
CPU: Intel i7 6700K
RAM: 16(8x2)GB DDR4 2133Mhz
OS: Windows 7 64 bits SP1
Driver Version: 372.90
Frame Time (Before): 8.3ms
Frame Time (After): 8.3ms
Programs running: Windows Media Player, Steam, Logitech Gaming Software, Gigabyte Xtreme Gaming Engine, NVIDIA Control Panel (no GeForce Experience installed), Killer Network Manager and SoundBlaster X-Fi3.

Edited December 10, 2016 by RextintorZ

[Romiith]

56 minutes ago, [DE]Glen said:

Hmm ok so not a handle leak, that's too bad. Have you tried just booting fresh and idling in liset? I wonder if there isn't even a need to run a real mission.

You are correct, no mission is required. Just standing in the Liset is enough. After 25 minutes my frame time has increased from 3ms to 20ms

[Nep_Blanc]

I tested this issue as reported by @[DE]Glen I have so far not seen any proof or any issues but neither the less I am going to test this intensively.

If you know me I play this game in long sessions that mostly take more then 6 hours if I stream, so far I've not seen degrading performance over time.
Neither the less I am going to try to repro this and see if it brings any results

1. Gpu model: 2x MSI GTX 970 Factory OC and one of them runs the game and the other runs the PhysX processing.
2. OS: Windows 10 x64 Pro with Cortana and some more BS features uninstalled, No DVR and or Xbox features possible. (Update pending)
3. Nvidia Driver version: 376.19 Latest as of speaking
4. Frame time on fresh restart of WF: ~ 4.7 ms with ~225 FPS
5. Frame time After: ~4.8 ms with ~225 FPS notable not a great issue here.
6. Programs running: Chrome(20 tabs), Firefox(7 tabs), Thunderbird, Discord, Deepbot, Notepad++, Steam(Overlay on), teamviewer, Rainmeter, Mcafee virus protection, Snipping tool, teamspeak and more apps that are services of Windows.

I used speed nova with jat kittag, spammulor and akstiletto prime. Went 40 mins in did not notice any issues at all and do not use any features of Nvidia recording software, In my eyes it is broken and should not be used.

 

Chao, The Roaring Lion

Edited December 10, 2016 by TheRoaringLion

[[DE]Glen]

13 minutes ago, Romiith said:

You are correct, no mission is required. Just standing in the Liset is enough. After 25 minutes my frame time has increased from 3ms to 20ms

AMAZING! Ok ok -- this is great -- how's your command-prompt fu? Have you got Discord? If you're willing to let it bloat for a good while so it's running really badly we can grab some dumps that may catch it in the act. If you wanna swing by the Casual Carousel I can walk you through it -- but tragically I'm not at my desk right now and won't be for several hours. If you're a ninja just get ProcDump, put it in your path (system32 will do), then at the command prompt you can just fire off "procdump warframe" to make DMP images of what the game is doing that I can load up and peek at.

[Romiith]

8 minutes ago, [DE]Glen said:

AMAZING! Ok ok -- this is great -- how's your command-prompt fu? Have you got Discord? If you're willing to let it bloat for a good while so it's running really badly we can grab some dumps that may catch it in the act. If you wanna swing by the Casual Carousel I can walk you through it -- but tragically I'm not at my desk right now and won't be for several hours. If you're a ninja just get ProcDump, put it in your path (system32 will do), then at the command prompt you can just fire off "procdump warframe" to make DMP images of what the game is doing that I can load up and peek at.

I was sysadmin now developer, so I'm fairly competent. I'll get to it.

[H8RAID]

I just ran 50 minutes on Apollodorus per Glen's instruction running full screen at 1080p with no performance issue(s) to report.  In mission, frame time was consistently 8 to 15 ms from start to finish.  Edit: I do not have any of the GeForce bloatware running. 

1. GPU Model: GTX 1070
2. Operating System: Windows 10 Home 64-bit
3. Nvidia Driver Version: 375.70
4. Frame Time Before: ~3.5 ms
5. Frame Time After: ~4 ms
6. Programs Running: Steam, Firefox, SB-Z control panel, Corsair peripheral utility, Logitech peripheral utility, Avast

Good luck on your search!

Edited December 10, 2016 by H8RAID

[LSG501]

Just to help out and to make sure it isn't a pascal thing.  I didn't see any issues on my setup other than framerate drops when extra effects where active like naramon shadow step or the aura's around ancients/eximus which is to be expected albeit maybe more than I expected (up to 100 fps lost from normal of 200-250fps)...35 minutes Apollodorus.

1. GPU Model: EVGA GTX 1060 6GB SC
2. Operating System: Windows 10 Pro 64bit - version 1607 build 14393.351
3. Nvidia Driver Version : 375.95
4. Frame Time Before: 3-4ms
5. Frame Time After: 3-4ms
6. Programs Running: in background - aida 64, bvckup2, afterburner v4.3.0.9267, spacenavigator, intellipoint, usual drivers from intel etc and plex media server (not streaming anything).  Windows firewall and AV etc.
7. Rest of system: i7 4790k no overclock, 16GB ddr3 1866 ram, 250GB Samsung EVO 850 SSD (game installed here). 

What I used: Wukong, Rakta Cernos, Spira Prime, Galatine Prime and Carrier Prime without a weapon.

Graphics settings in game:

Spoiler

Graphics Settings in Nvidia control panel: No custom profile for warframe.

Spoiler

 

Edited December 10, 2016 by LSG501

[-NR-Maggi]

CPU: Intel i7-6700K

RAM: 32 GB

GPU: ASUS Nvidia Geforce 1080 STRIX-8G

OS: Windows 10 Professional

Fullscreen mode

Programms running in the background: Chrome, Steam, Teamspeak, Warframe

 

I just set up this new computer and have since then this problem. On the old PC with a AMD GPU and lower specs overall it worked perfect.

Right now I have to scale down particle effect and turn of NVidia Physx to have a viable performance. But even with them turned down my FPS drops lower and lower the longer the mission goes. I noticed that bigger maps (like excavation maps) let my FPS drop even faster. 

 

[Lerzu]

1. GPU: GTX 1050 Ti
2. OS: Windows 10 64-Bit
3. Nvidia Driver Version (376.09)
4. FT Before: ~4ms
5. FT After about 20 minutes of sitting: ~21ms and still rising
6. Programs Running: Steam, GOG Galaxy, GeForce Experience, Firefox, Waterfox, MSI Afterburner

After countless unsuccessful tries, I decided to install GFE and use the same settings as shown in those screenshots and the bug finally happened. With Instant Replay on, my performance keeps getting worse even while sitting at the starting point, looking at the floor. After walking around the Orbiter and being amazed by how much my framerate stutters, I noticed that scenes where the GPU video engine had to work harder to encode the video, the lower my GPU power dropped. Then turning to look at something that's easy to encode made the power rise again, but it never went back to full power. Even after turning off the background recording it still keeps getting worse, though maybe not as fast. Starting the game with Instant Replay already off keeps everything working perfectly so the overlay from GFE shouldn't cause any issues.

The weird part is that this has happened to me before without GFE. I often have a video playing on the second screen and that uses the GPU video engine so maybe hardware accelerated videos are at fault? I'll look at some 4k and 8k videos to see what happens.

Welp, nothing happened with those videos.

Edited December 11, 2016 by Lerzu
Info on the videos

[Cray]

7 hours ago, [DE]Glen said:

I *think* minimized might actually skip out all of the heavy lifting and won't exercise the driver so it might not be a fair test, sorry.

Damn, and I left it going since 7AM...

I can try without minimizing later, but I'm getting the feeling I'm not prone to the issue.

[LSG501]

48 minutes ago, Lerzu said:

After countless unsuccessful tries, I decided to install GFE and use the same settings as shown in those screenshots and the bug finally happened. With Instant Replay on, my performance keeps getting worse even while sitting at the starting point, looking at the floor. After walking around the Orbiter and being amazed by how much my framerate stutters, I noticed that scenes where the GPU video engine had to work harder to encode the video, the lower my GPU power dropped. Then turning to look at something that's easy to encode made the power rise again, but it never went back to full power. Even after turning off the background recording it still keeps getting worse, though maybe not as fast. Starting the game with Instant Replay already off keeps everything working perfectly so the overlay from GFE shouldn't cause any issues.

The weird part is that this has happened to me before without GFE. I often have a video playing on the second screen and that uses the GPU video engine so maybe hardware accelerated videos are at fault? I'll look at some 4k and 8k videos to see what happens.

I've seen reports/forum (not warframe) posts about issues with geforce experience (I've never installed it) being buggy since a couple of updates ago causing issues on other games etc so it's possible warframe could be affected in the same way.. 

Edited December 10, 2016 by LSG501

[taiiat]

https://pcpartpicker.com/user/taiiat/saved/8fLQzy
w7 x64 SP1
Video Driver 376.09
running in a Window

background applications...
Teamspeak, Discord, a fork of Chromium i use, Steam (all sorts of Overlays and hooks to Warframe are already disabled), ShareX, Dropbox, AtNotes, Rainmeter, FRAPS, X-Mouse Button Control, Corsair Gaming Software, Afterburner, GfExperience (Shadowplay on), NOD32, SB-Z Control Panel.

 

i have most of the visible stuff in Shadowplay disabled already, as i don't like being bothered for no reason.

 

starting at ~2.45ms Frame Time right after restarting Warframe and logging in

went and did some stuff in Web Browser to kill time.
from statements earlier, just stood in my ship for ~45 minutes. ~2.45ms Frame Time at the end.

 

- - - - - 

 

note: i use Shadowplay for all of my recording, and i leave it on, background recording at all times. in all games. and have for a long time. i can't say i've noticed anything encountering issues because of it.

[GIGARAM]

Hi sorry but ive not been able to recreate the bug im running 

GPU Model EVGA 1080 GTX F.T.W.
Operating System (Windows 10 PRO 64-Bit 1607.  OS BUILD 14393.479)
Nvidia Driver Version (376.19 Release Date 12/05/2016)
Frame Time Before ~2ms
Frame Time After ~2ms
Programs Running (Steam, GeForce Experience, OneDrive, Synaptics Pointing Device(for the deathstalker ultimate) Razer Synapse, Origin, Garmin Express, Realtek Audio Manager, Battlenet, Uplay.)
 

[Azilian]

In my experience yes and no.

1. GPU Model: Gigabyte 980Ti G1
2. Operating System: Windows 10 64-Bit
3. Nvidia Driver Version: 376.19
4. Frame Time Before: 2.8ms
5. Frame Time After: 3ms
6. Programs Running: Steam, GeForce Experience, Chrome, Google Drive and Avast.

I came across severe sustained FPS drop while running LUA Defence as Chrome with 2 others, started around wave 10 dropping my fps from around 130ish to 90 then dropping further to 70 and again to 30fps in the space of about 5 minutes. This did not sort itself out when returning to the Liset until I quit out the game and restarted the client. 

In regards to the mercury test I have posted my results above (30 min) :)

Have noticed quite a difference in performance lately. Someone could be onto something when saying it could be the Geforce Experience overlay.

[Helch0rn]

On 10.12.2016 at 4:26 PM, [DE]Glen said:

AMAZING! Ok ok -- this is great -- how's your command-prompt fu? Have you got Discord? If you're willing to let it bloat for a good while so it's running really badly we can grab some dumps that may catch it in the act. If you wanna swing by the Casual Carousel I can walk you through it -- but tragically I'm not at my desk right now and won't be for several hours. If you're a ninja just get ProcDump, put it in your path (system32 will do), then at the command prompt you can just fire off "procdump warframe" to make DMP images of what the game is doing that I can load up and peek at.

since I am a noob,

can you provide the command line for the proc dump?

[-Krux-]

Well I can say this much: I have a "high-end card" and don't experience this problem. So maybe it isn't related to card.

I'm using a GTX Titan X Hyrdo, 12 GB GDDR5

[ZizWing]

GTX 1080
Windows 10 64 Bit
i5 3570k
16GB DDR3
Geforce Experience version: 3.1.2.31
Driver version: 376.91

Did 2400 cryotic run on hieracon and fps eventually stayed around 30 with drops to low 20's.

I then restarted the game, turned off nvidia share and did a run with the exact same group on hieracon to 3200 cryotic and stayed at a constant 60 fps (yes I forgot to turn vsync off, sorry.)

But considering turning off share negated the issue (so far) it gives hope we can fix it or at least notify nvidia.

Share sure has been causing a lot of issues lately, first the division and now warframe too? I hope they fix it soon. FYI this seemed to have started with Geforce experience 3.0 and up.

Hope this info helped a bit. I didn't pay much attention to the frame times but in the liset they ended up at around 22-24 from the usual 16-17 (with vsync on).

Also Physx was ON in both cases.

 

Edited December 11, 2016 by zY_Spartan_Yz
spelling

[taiiat]

5 hours ago, Helch0rn said:

can you provide the command line for the proc dump?

to use Procdump, you will need 'cd' (Change Directory) to the location where you have Procdump located while in Cmd.

alternatively if you put it in System32, that's where Cmd defaults so you won't have to change your active directory.
then you just need to call the application and what to target, which is as simple as 'Procdump Warframe'.

[hangman401]

1. GPU: EVGA GTX 1050 ti SC

2. OS: Windows 10 64 bit

3. Driver: 376.19

4. Frame time before: 14.5ms

5. Frame time after: ~30ms

6: Background programs: steam, discord, razer synapse, geforce experience, evaga torq x10 gaming mouse program, msi afterburner, sentey  symph gaming headset program.

 

I'm not sure if my data is pertinent since I have a 1050, but I figured it'd still apply since I can normally run the game above 100fps with everything on, but then got this bug today which cut my fps to about 40 as well as the frame time increase. Also, it wasn't infested missions that triggered it, it was a relic mission on Earth against Grineer.

 

Edit: Also, I have the nvidia overlay and a discord overlay on WF, which I didn't enable the Nvidia overlay till last night and wasn't experiencing this issue until today, interestingly.

Edit 2: Happened again. Different tileset and mission. Still Grineer, but this time it was on the ship tile. It's making my fps and frame time increase a ton. Tried disabling Nvidia overlay and it still remained the same.

Edited December 12, 2016 by hangman401

[p3z1]

1. GPU Model - NVIDIA GeForce GT 730M
2. Operating System - Windows 7 Ultimate, 64-bit
3. Nvidia Driver Version - 364.72
4. Frame Time Before - ~17ms
5. Frame Time After - ~17ms
6. Programs Running - Steam + Overlay w/ 1 Tab open in in-game Browser, Hamachi

Guess I didn't replicate it. Maybe I haven't upgraded my GFX Driver for a while now?