Yes. Black. Anything with no calories is generally ok to consume outside of your eating window. And you should always drink lots of water.

On my I.F. days, I’ll have black coffee as my pre-workout, and I’ll start my eating window right after my workout (for that post-workout protein).