Do I need to have a membership to use Costco's pharmacy?

Yes

No you do not have to have a membership to use the pharmacy. Tell them at the door that you are picking up or dropping off a perscription and they should let you in. I know several people who use the pharmacy and they do not have a membership

You need a membership to enter the building, the pharmacy is in the building...keep your membership up-to-date!