I can't see any reason not to use it, or to give clients the ability to disable it!
Remembering that clients rely upon you as an email admin to get emails to them, it's our job as an email admin to work tirelessly to make sure they get those emails securely too. I think it would be a disservice if we simply changed the security to give get an email a little quicker for the first time.
We have it also set to 60 seconds retry. Obviously some email servers will not retry immediately and that is where the delay may come in. However, as an email admin I am more than happy to explain to clients the vast benefits to them, to the email server speed and to the larger community for the use of Greylisting. No clients have the ability to change any security/spam/virus settings within our email servers.