Personally, I rebind my signs to 1-5.
Sure, I needed to look for other keys within reach for selecting my swords, but they're usually a once-off thing (unless you're fighting operator with swords), so it's not as critical to have close by.
That setup works perfectly so long as ... you don't get hit.
Sometimes the instant cast keys glitch out due to interrupted animations, and I have to use Q to make the sign come out.