The brake pedal is about the lowest thing hanging on the bike.
The centerstand is also down pretty low.
They will drag.
However, what sometimes happens with the brake pedal is that one of both of the securing bolts for the crossover rod comes loose, and the pedal then flops down and digs into the road. This actually can flip you off the bike, as you found out.
It's very important to check that those bolts stay secure.
I have heard of some people putting a little piece of spring wire under the pedal bar somewhere, to act as an "emergency catch" for the pedal if it drops.
I personally have never experienced that, but I've read about it from several people who did have it happen, and now you may be another one.
The right-side shifter conversion will do away with all that crossover linkage shit, but it costs a few hundred bucks.
It gives much better shifting, better ground clearance, looks better, and allows the use of taller gearing in the sprockets. And does away with the brake pedal linkage going under the bike.
Then the brake pedal is on the left side, and is firmly connected to the rear brake by the adjustable brake rod. They don't fall off.
These bikes were originally designed to have shifting on the right, and rear brake pedal on the left, and when US regs forced the changeover, they had to make linkages to switch sides with the pedals. It is better to return it to the old style if you can deal with the expense and getting used to shifting on the right and braking on the left.