We did. About 6 of us with 2 children visited. The person with the key card has to let arrivals into the pool area but it was fine.
I think you will be okay as long as it's not excessive
Most hotels will not let visitors swim at the hotel, even if they have relatives staying there.
Visit yes, swim is a different question that you would have to ask the hotel manager for their policy.
Thanks! Your answer is awaiting moderation.