Do you have to call to make a reservation?

No. But always wise to call ahead to learn times, rules, requirements and sites, availability and all that Our forrest, and state have to offer.

Unfortunately, it's first come, first served, for most, but I would call.

Yes I would call. Some have reservation sites and have first come sites

The only campgrounds that are open now due to covid-19 or the bigger ones and it's on a first come first serve basis as far as I know as for camping in the woods anywhere you can do that