Can you have a family retreat there

It is classified as a national park. It is open to the public. However be aware that all the historic homes were sold to private buyers so people live there all around the park. Parking can be difficult especially on nice weekends. If you're looking to stay over night that would have to happen on the local economy. But we've spent time with extended family here looking at the museum and monuments. It will depend on what you want it of your retreat. If you're wanting covered pavilions and space for a large gathering, you may be better off with Newport news park or something like that. If you're wanting a nice way to spend the day with a small group with some nice spots for pictures this will work. I hope this helps.

It would be good for a day trip kind of thing.

I do think there is a camp ground there but would check online first