Found the answer to my question, right from the owners mouth. It was called Colonial Chef. Now known as Sport of Kings.
Before Sport of Kings, I believe it was The Chessmen. Pretty sure that's where the rook over the door came from. It happens that chess is called the sport of Kings.
As far as i can remember its always been sport of kings
The person running it now has for over 30 some years and it was already a sort of restaurant with employees before that.
Thanks! Your answer is awaiting moderation.