Assuming I show up with kids, what would be the best time to avoid the "clothing optional" crowd?

I'd say that if you were to go during normal times, aka business hours on most days you should be fine.

Everyone's pretty well dressed as it's popular place everyone's pretty well dressed

I have never heard of optional clothing there. But if you want to be on the safe side, any sunny hours are good.