Remembering what I can of engineering school, the oxides, hydroxides, and fine metal particles not playing nice. This is why washing often helps. The fine metal/oxide particles are extremely fine (like diatomaceous earth fine) and need a good cleaning to let go. Ever polish aluminum? It turns the polish black. If you sand aluminum with 3,000 grit paper the residue will be black. Aluminum oxides form on the surface almost instantly when exposed to air. That's the "white" powdery stuff. It'll turn hands black also. The apparent black color is related to the very fine particle size of the oxide "powder" which doesn't reflect light so well. A lot of aluminum ladders do this. This is common on weathered aluminum and even on anodized aluminum if there is sealer failure.
But that does nothing to help the situation (engineer joke).
Here's what I would do:
Wash it well, scrub it, use dish soap, acetone, etc. to get that thing clean.
See if that helps. This may be an ongoing thing.
Go to a marine/boat store (not the Home Depot/big box stuff) and pick up some marine aluminum cleaner (like they use for cleaning pontoon hulls.) Brush it on and follow directions...you shouldn't need to "polish" it to a shine.
You can try a marine aluminum sealer from the same store. There are some that work very well, but must be reapplied every so often.
See if that works.
You can try using heavy duty shrink wrap tubing if you can find it large enough. But that may not keep water from getting trapped between the handle and the wrap.
That should keep your hands clean, not sure how long it will last, though.
Have it anodized and sealed or even finished in Cerakote or something similar. Many larger gun stores are equipped to do this as well as other coatings that are normally used on barrels. Make sure the process does not heat the aluminum past an annealing point.
This will work, but it can be a bit $$.