Bryant Grill has brunch, lunch, dinner and a cafe. Not sure about tea but it is a lovely place.
This particular cafe doesn't have an indoor seating. There are few other options around the park: Angelina Paris NYC (corner of 6th avenue and 40th street), L'Adresse American Bistro (6th avenue near 41st street), Le Pain Quotidien (40th street, between 5th and 6th avenues), and Lady M Cake Boutique (40th street, between 5th and 6th avenues). Also, inside the park there is Bryant Park Grill restaurant with indoor seating.
Its not indoor at all, but it has roof, got good cocktails and food, but not sure about tea, if not you can go to a starbucks
Thanks! Your answer is awaiting moderation.