I've asked my insurers (horses mouth, the underwriters), and they are quite happy for me to remove it, and have noted thus on my policy.
I've also checked with VOSA, and they have no interest in whether it's fitted or not. Just have to remove the warning light too.