I finally found bone conduction headphones that stay on while swimming (and they sound great)

The H2O Audio Tri Pro headset keeps the tunes going in the pool, and have a snug fit to stay in place while you swim.