My general attitude towards bike maintenance, born of profound ignorance, is to try the simplest fix first and don't get into other things before that fix is implemented and tested, since multiple simultaneous fiddlings will only make a problem murkier.
I'm pretty sure the original poster's battery was shot. A new battery should hopefully sort out the original problem. Replacing the rectifier, installing a magneto, or adapting a Mr. Fusion to the Flux Capacitor assembly may not be at all necessary.
My 2005 can't be a far cry from the original poster's 2004 electrically-speaking. Accordingly, I've taken the liberty of doing a few voltage measurements at the battery terminals just as a ready reference to what might be considered "normal", at least with mine. To its credit, my 2005 recently did a 16 hour run in the continually pissing rain without so much as a hiccup. So, I'd say its electrics are pretty reliable as Enfields go. Aside from having had its starter sprag gearing and the little dogbone connector that overrode the headlamp switch both removed, and an electronic ignition installed by a previous owner, as far as I know it's pretty much stock, with about 14,000 miles on it, ten of which are mine.
It's presently just about 50 degrees Fahrenheit, having gone down last night to just below freezing at 31. The bike has not been on a run in three days and is not presently hooked to a battery tender or charger.
Measurement at the battery terminals cold before starting: 12.6 volts.
Started first kick! (choke on, whisper of throttle)
1 minute after starting, low idle: 15.4 volts
5 minutes after starting, low idle: 15.2 volts
10 minutes after starting, low idle: 15.1 volts
With headlamp on after 10 minutes, low idle: 13.4
volts
After about 10 minutes of running at low idle, engine was stopped, allowed to sit another 5 and measured one last time: 12.8 volts.
I hope this data will help guide the original poster's ride back to full health.