So, Borderless mode this is still a problem, but by changing to Fullscreen the issue was solved for me. Others should try it out and see if that works for them as well.
Now if the problems with the game crashing if you tab out during a loading screen while using Fullscreen mode were also fixed, everything would be perfect.