Yeah, I'm pretty sure this is top-of-the-list annoyance for everyone no matter how much they might complain about the QTE being invasive or the game being too difficult in the beginning and/or too easy in the middle, or certain narrative elements not making sense, or even the fact that you have to place yourself in just the right spot in order to be able to pick up something or use something... and preferably unarmed. (It might be my imagination, but it seems loot is a little afraid of swords.)
...It's especially annoying when you're caught in the middle of a, say, 10-scene battle that doesn't allow you to save, and suddenly your keyboard stops working, or your mouse stops working and you get killed and have to start all over again. ...Or, say, Zoltan gets caught in the middle of 2 scoia'tael and just keeps running in place, you run back and forth to try to get him to respond, you bump into him, into one of the rude elves... finally the only thing that gets him loose is dodge-tumbling into him... odd.
But it actually seems to be much more common than random occurrences of Geralt being unresponsive to the mouse or the keyboard... there're a lot of times when he's free of anyone attacking him and he simply will not cast a sign -- for just two or three precious seconds -- or throw a bomb . . . and times when he's in combat and for some reason will not dodge. It happens so often it's ridiculous. It's not at all just occasional, but the thing is... when it does happen it's usually fleeting . . . for some odd reason: at times the game just doesn't respond right away.
So... no, Jingafex, you're not the only one.