No you don't.
Two good reasons to leave them on... 1 plastic water bottle caps are bad news for our environment and like plastic straws they have been found in the noses of sea turtles and I have heard rumors that the government is going to punish the consumers of bottled water with some type of sin tax. And the second reason would be that they weigh the plastic bottles so the caps on will get you more money on your refund.
Thanks! Your answer is awaiting moderation.