There is ....you can pretty much pull up anywhere thats not private property.....there are public places like "Front Street Park"... I would google parks or swimming holes on Saratoga lake.
Would def book in advance but lots of options single/double kayaks, paddleboards, hammockcraft. Not sure about the swimming, we thought about it but there was a lot of seaweed and stuff all the way up to the top of the water in lots of spots. Probably if you found the right area you would be fine.
Not sure about the swimming. But I wod book about a week in advance if you can. They do have a ton of kayak's and paddle boards so I've never had a problem making a reservation.
Thanks! Your answer is awaiting moderation.