(1) What can I do to increase the chances that I'll be able to get the nuts and bolts together? When I have everything apart, how can I best clean the filthy, worn bolts without [further?] damaging the threads? Do I use WD-40? PJ1 contact cleaner?
Just about any oil based solvent and an old toothbrush should clean off the accumulated junk.
Back in the old days, we just tossed them into a pan of gasoline but what did we know?
Hard to start and hard to screw on are two different things.
Hard to start may be because the lead in chamfer on the male threads is damaged or it doesn't exist. If this tapered area doesn't exist, careful work with a small flat file can recreate it so the threads will engage the female threads better.
Hard to screw on after they are started nuts may be self locking.
These nuts can have a small dimple in their outside that forms a bump on the inside. The intent is the distorted threads on the inside will slightly interfere with the male threads. On some, the deformation is large enough to be visible when looking thru the nut.
If the nut is self locking it's a good idea to oil the external and internal threads before screwing the nut to the bolt. This can keep the nut from tearing up the male threads.
(2) When I say the bolts are non-standard, what I mean is that they have very, very thin heads. They need to be very, very thin in order not to catch on the axle fork while the sprocket is rotating. The heads have hex-shaped edges (in a standard size), but because they are so thin you can't really force them very hard without bending or stripping them. What is the proper way to screw something like this into a nut?
Use a box end wrench or a socket. You may have to grind or file it to remove the chamfer that leads into the hex so it can maintain as much engagement as possible.
(3) Is there a name for bolts like those with very, very thin heads? Where does a person go when he needs an unusual fastener?
Special. As was mentioned, the original maker is the best place to go for a replacement.
If that is not possible a standard hex head bolt can always have its head thickness reduced by filing or grinding off the unwanted material. The catch here is that often, special bolts not only have a reduced head thickness but they also have a reduced hex size to clear things that are close to them.