Can I switch out a cable box that isn't working at a Cox store?

As long as your cable box is from Cox, yes you can

With very few exceptions, we definitely swap out your Cox equipment here. Feel free to swing by and we'll take care of it.

Yes you can

I would assume so if it's their equipment take it in

Yes. I would call first just to make sure or at least give them heads up someone is coming with a faulty box and they can have a good one on hand.