A good way to prevent swimmer's ear is to put homemade drops in your ears after swimming. Make a home brew of half and half alcohol and vinegar. (Yes, rubbing alcohol and generic cooking white vinegar.) You can also buy drops, and the little over-the-counter bottle is convenient. You can buy the bottle, pour out the contents, and replace it with your home remedy. You may want to set the drops outside when you go swimming so they will warm up a bit before you put them in your ears. They won't be cold and irritating.
This mix will actually work both for prevention and for treatment, if you forgot to use it ahead of time. Nine times out of ten, this will be all you will need. The vinegar kills the fungus, and the alcohol dries up any moisture that remains.
If you are an ear-wax-builder-upper, you may need to flush your ears periodically (such as once per month) using an infant bulb syringe and lukewarm tap water, to remove packed in ear wax prior to putting in the home brew drops.