Here's an even weirder part. Sometimes when it misses a downshift and gets a false neutral, I upshift into the lower gear! Yep, I'm trying to go back to the gear I just left and it shifts the other way.
What has happened is that you have actually passed the gear you wanted and dropped into the false neutral between the lower two gears. If you were shifting down to third from fourth, you actually overshot third into the gap between second and third, so when you shifted you went back to third.
Looking at the "Service Manual," there are grooves that are for a stop plunger that would keep the cam plate from rotating past the gear detents, but I can't see any plunger in there. Can't say I'm going to tear my engine down to check it out, but the design without it looks a bit flaky.
It looks like the little hook thingy that shifts the gears is supposed to hold it in gear, too. If that spring gets a bit weak, it could cause these shifting difficulties.
It looks like you need to ease up on how hard you are shifting, but I, too, catch one in the middle occasionally. It's gotten better as the miles go on, but maybe it's just we who are getting broken in!
Bare