There is a swimming hole nearby. It's about a mile back towards Marblemount off hwy. 20. An abandoned gravel pit that is fed by a creek and empties into the river. The folks at the General Store in Newhalem may give you directions. The Parks people are probably not aware of it.
No fishing swimming at campground. Skagit River is there but normally running too fast for swimming. Diablo and Ross Lake are a few miles away though.
Thanks! Your answer is awaiting moderation.