I've enabled a bunch of no-prefix shortcuts in tmux (like Alt-c for new window, Alt-n for next), which has made my life so much better. The problem was, my shortcuts didn't always work. Maybe every 5-10 presses, something would just swallow the keypress, and I'd have to do it again. Sometimes it would send input to the terminal, which was even worse.
I worked this out for myself, but I wanted to post it as a question because I couldn't find anything when I was searching. Lots of people talk about the "menu access key" problem, but this is actually unrelated.