If a buyer wishes to contact you outside of Fiverr, just let them know it’s against the terms of service. If they cannot accept that, they’re not worth the time and risk. It’s that simple.
As for getting flagged, there is supposed to be a warning that pops up if you’re typing something that might get you flagged.
https://community.fiverr.com/forums/topic/213655-many-buyers-wish-to-connect-with-me-outside-of-fiverr-what-should-i-do/?do=findComment&comment=1253232