I had a strange problem when it comes to Minecraft. I used to record my gameplay with an 128 resource pack and experienced no lags at all. Then I didn't play for some time, and when I tried to record another gameplay video (this time with a 64x64 pack) it felt like walking through jelly.
I tried all kinds of changes in the graphics menu, uninstalled an older Java version, but then it ocurred to me that the only really new thing I was using was ... Optifine.
Yes, I know, it's supposed to improve your fps, but for some reason I could no longer record smoothly with Bandicam, regardless which codec I was using (I tried Cuda, Lagarith and DivX). The videos looked mostly normal, but the gameplay was horrible.
So maybe you could try recording without Optifine and see if it helps.