Not smooth experience when dropping by 1 fps

[Vnexa]

My monitors refresh rate is 100hz, i have set the cap to 120fps. When fps drops by just one, i looks like it dipped below 100fps mark even tho counter clearly doesnt show it. Uncapped I can push average 400fps with my settings. Keep in mind i have Amd FreeSync on and my monitor supports FreeSync and has it enabled. I tried enabling Vsync, but it doesnt lock the fps but instead it behaves as it is uncapped (im running it on DX11 setting, i dont know if that metters). Any fixes or suggestions are welcome.