I haven't taken mine out for a swim yet, but for small dogs, I'd use a life jacket just in case--I believe that small dogs are less buoyant than larger dogs.
Here are a couple of articles:
Pet Life Jackets -- Just Another Accessory or a Necessity? Knowing Dogs l Melanie Schlaginhaufen Dog Training : Life Jackets for Dogs?