I'm flying first class, and am a delta skymiles member/amex cardholder. Can I use the lounge during my 2 hour layover in Atlanta?

Yes. As long as you are flying Delta that day and are currently a member/Amex cardholder.

International first class, being a delta platinum medallion and flying any class international, holding a delta reserve amex, or holding the amex Platinim card.

I know if you are traveling 1st or business class international yes. I completed an business class international flight 2.21.19 and used the lounge in Atlanta, Paris and Riyadh on these flights.

I'd you are flying first class international, then yes. If your amex is a business one, then yes.