Unless the o-rings are damaged or already leaking it is best to leave them alone.
Although Viton (FKM) is exceptionally good with motor oil, it isn't a great deal better than the Nitrile usually used for the seals on a oil filter.
As an example, I have changed the oil over 15 times on my UCE powered motorcycle over a 4 1/2 year period and the original packings are doing fine. (Well, on the last oil change I did change the o-rings but that was only because I have so many that came with the oil filter kits I bought from our host.)