If you use a silicone RTV, be sure it says it is compatible with aluminum.
The more common RTV's like the ones used for bathroom fixtures, tile, etc. give off acetic acid while they are curing.
If this is trapped in a aluminum hole, the acid will attack the aluminum.
RTV's of this type give off a vinegar smell.
Some of the high temperature RTV's sold at many auto supply stores are aluminum safe and should stand a better chance of surviving in that location.