I'm having the same problem. Some service providers seem to be blocking our server now due to excessive retries.
Here is one that has over 5,000 attempts. Others are at over 20,000. It seems like in many of these cases, one of the recipients has a permanent error. In the email below, one of the recipients does not exist on the email server, yet our server continues to retry it.
[2020.09.23] 14:10:37.826  Attempt to ip, '22.214.171.124' success: 'True'
[2020.09.23] 14:10:37.826  Delivery for firstname.lastname@example.org to email@example.com has bounced. Reason: Remote host said: 550 No Such User Here"
[2020.09.23] 14:10:37.826  Removed from RemoteDeliveryQueue (1 queued or processing)
[2020.09.23] 14:10:39.060  Added to RemoteDeliveryQueue (1 queued; 5/20 processing)
retry: 5604;09/21/2020 22:25:04
creationdate: 09/21/2020 14:06:17
Received: from mcgonigels.com (ec2-3-18-57-153.us-east-2.compute.amazonaws.com [126.96.36.199]) by hyperion.tecture.com with SMTP
Mon, 21 Sep 2020 14:06:17 +0000
Date: Mon, 21 Sep 2020 14:06:16 +0000
To: firstname.lastname@example.org, email@example.com
From: "[Rusty]" <firstname.lastname@example.org>
Subject: McGonigels Contact Form Message [snip]
X-Mailer: WPMailSMTP/Mailer/smtp 1.8.1
Content-Type: text/plain; charset=UTF-8