There is a place that rents kayaks on Harbor Road in East Setauket. Not sure of exact location or if they still rent them in the off-season. You might want to Google where to rent a canoe, rowboat & kayak. Good luck!
I'm not sure if they're open after Labor Day, but you could try the boat launch ramp in Kings Park.
Call the park. They were very helpful in finding anything I needed.
Thanks! Your answer is awaiting moderation.