How much time should you give yourself to enjoy it?

I think about 1 hour is enough, but it depends on your level of interest. For a typical person, 1 hour in a rose garden will be plenty.

1.5 hours is my vote

To totally explore the gardens and the many varieties of roses, I suggest that you allow yourself at least 3-4 hours. I go there often and I find the peace and tranquility very relaxing. This time of year there's not much to see but soon the blooms will return.....

Agreed. We spent probably approximately 30 minutes there, walked up to the Japanese garden and came back after. There's a beautiful view of the city from the Rose Garden.

I agree with that 1 hour will likely be sufficient. However, there are several attractions in the vicinity. If you have time, you might want to schedule more time to explore the area.