Bait can be purchased on many different islands. When you are on an island such as Outset island you will see a boat sailing around near the coast. This boat is a store. If you swim out you can buy items from it.
If you go out to the shop you can purchase bait in sets of 3. In order to be able to buy any bait, you will need to own a bait bag.
The bait bag can be bought from the same boat vendor, but not on every island as the items he sells change on each island you visit. If you visit him on Outset island you will be able to purchase the bait bag.