Do you need to make a reservation to eat lunch at the tea room?

Not usually, but sometimes they may have party's or baby showers. Also busier around holidays.

You do not have to but you can just to make sure you are seated. They take those with reservations first. If you have a large party, it would be wise.

For a large party, yes. They also have a private space that may be reserved for larger gatherings. For a lunch with few friends during the week it's pretty easy to get in.