2
ARC protocol
Question asked by Sabatino - 1/17/2024 at 2:45 AM
Unanswered
Hi everyone.
I'm terribly confused.
A client of mine uses office365 for his domain
Today he asked me for a change on the DNS and when I opened the DNS I was confused.
Let me explain.
The customer domain domain.tld
mx domain-tld.mail.protection.outlook.com.
v=spf1 include:spf.protection.outlook.com -all

and so far no doubt.

Then I had to create a dkim key for a newsletter service and I realized that he didn't have the dkim for microsoft
The classic selectors which are then cnames to xxx.xxx.onmicrosoft.com to understand each other. He only has dkim keys on DNS associated with other newsletter services

I said to myself, ok I have to warn him, he has the missing dkim key.
But then I went into my webmail I realized that
dkim pass
spf pass
dmarc pass

How is it possible? I looked at the message header and found references to ARC. But I understood that ARC still needs DKIM keys or am I wrong?
Maybe I said to myself there is the envelope sender, but he isn't there.
The sender is correctly info@domain.tld

Help me understand.
Thank you
Sabatino Traini
      Chief Information Officer
Genial s.r.l. 
Martinsicuro - Italy

10 Replies

Reply to Thread
0
Douglas Foster Replied
DMARC validates the From address.   
  • For a message with FROM: USER@DOMAIN.TLD, you need a DKIM private key to apply a signature for d=DOMAIN.TLD
  • For a message with FROM: USER@DOMAIN_TLD.ONMICROSOFT.COM, you need a DKIM private key to apply a signature for d=DOMAIN_TLD.ONMICROSOFT.COM.   
But do Outlook 365 even have access to the DNS so that can configure a private key within _dmarc.DOMAIN_TLD.ONMICROSOFT.COM ?   I suspect not.  Even if it is possible, the subdomain "domain_tld.onmicrosoft.com" is intended as a transition tool for people onboarding with Microsoft, and it should not be used for actual mail.    Most email coming from user@domain.onmicrosoft.com is spam.

You do not need a private key for Microsoft.com, which is good because you will not get one.

Of course, for SPF to pass for newsletters coming from your server, he needs to add your servers to his SPF record.

You do not need to worry about ARC for this newsletter.   ARC is a tool for forwarders, particularly mailing lists.   The forwarder adds an ARC Set indicating the authentication results that he observed when he received the messages.   If the final recipient trusts the forwarder to be truthful, he can use the ARC data to evaluate the message as if it had been received directly instead of being forwarded.   Office 365 adds an initial ARC Set to every outbound message, but they have not received the message from an untrusted source over unauthenticated SMTP, their results are not credible.   I have seen both irrelevant failure or false pass.   (The false pass was DKIM PASS on a message with no DKIM signatures at all.)

If you allow off-site forwarding,  you need ARC because of the new requirements from Google and Yahoo.   This has been raised on a different forum post and SmarterTools has promised that it will be ready soon.

For the full scoop on ARC, the RFC is at this link:

This post from Barracuda.com summarizes the new requirements from both Google and Yahoo, and provides links to the official statements from both companies:


1
Millennium Systems Replied
I believe you are missing the point Sabatino is making. He sees no O365 DKIM key in the DNS for the domain, yet the message he got passes the DKIM check.
0
Sabatino Replied
That's right Millennium Systems

But Douglas Foster also said that false passes occur on dkim

We should understand why.

They can't put the original message and its headers here, but I will open a ticket with SM. Let's see what they tell me
Sabatino Traini Chief Information Officer Genial s.r.l. Martinsicuro - Italy
0
Douglas Foster Replied
Right, I did not fully process the question, but I did accidentally provide the right answer.

Every message coming from Office 365 has an ARC set signed by Microsoft.com.   An ARC set is three headers, one of which is "ARC-Authentication-Results".   This header asserts the authentication results that were supposedly tested and detected by the message signer, in this case Microsoft.    There is a long list of tests that can be included, not just SPF/DKIM/DMARC.

The problem here is that Microsoft lies about the results.   The problem is not DKIM, the problem is that Microsoft makes up results for tests that they did not perform (asserting a DKIM result in expectation that a signature will be applied downstream) or which are not applicable (calculating an SPF result based on the handoff between a client's server and Offic365 acting as an outbound gateway).

ARC results only work if the sender can be trusted to provide correct information.   ARC sets from Microsoft are not trustworthy.    DKIM is not the problem.

DMARC Replay
The only DKIM problem discussed in the IETF Standards process is called "DKIM Replay".    
  • A bad guy gets a free account on Gmail or Outlook.com.  Stealing credentials and using a compromised account is optimal.
  • Then he sends a malicious message to another account under his control, obtaining a signature from Google or Microsoft in the process.
  • Finally, he uses that second account to retransmit the malicious message to a long list of potential victims.   The message still passes DKIM, so it looks to the recipient like a simple forward, when it is actually a malicious forward.

Partial defenses against malicious forwards:
  
For senders, put a create time and an expire time on every signature.   This does not prevent malicious forwarding, because a lot of traffic can be generated before the signature expires, and the bad guy may be able to generate another one easily enough.

For evaluators, look for multiple messages with the same signature.  Start blocking messages after a volume limit is reached.   This works pretty well for huge services like Google, probably less effective for a hosting service that will see fewer simultaneous attacks.

And obviously, for account owners, don't let your credentials get compromised.

Google has added an extra field to their DKIM signatures to indicate the original destination domain.   A savvy recipient can look at that information to determine if the message was forwarded or not, and use that clue to help detect a reply attack.   Google has submitted a draft to IETF which explains the concept and the tags used, but no official action has been taken yet.  I have not seen the tag adopted by any other sending organization.

1
Sabatino Replied
If I was confused before, I'm even more confused now.

From what you say I understand that this adds a problem to dkim which becomes unreliable.
Sabatino Traini Chief Information Officer Genial s.r.l. Martinsicuro - Italy
0
Douglas Foster Replied
It is not a DKIM bug.   DKIM PASS says that the message was handled by domain X and it has not been altered since the signature was applied.   Reply attacks work because neither of those rules are broken during the forward.  

DKIM Replay allows a sender to shift his dirty work from the initiating Gmail or Outlook account, where he will likely be throttled, to a server under his control, where he will not be throttled.   So if  you have existing negative reputation about the replay server identity, you will still block the message.  If not, you will have to depend on content filtering. 

All forms of forwarding are a problem, because the forwarding service hides some or all of the originator's identity.  If the forwarder allows unwanted messages through his service, you are poorly positioned to filter based on originator identity;

Sender verification (SPF/DKIM/DMARC) can only ensure that the sender identities are true, so that reputations can be applied based on those identities.   If you have no sender reputation information, then you have to depend on content filtering.   For example, we regularly receive and block malicious message from from properly-identified Gmail accounts.

Sometimes you need to exempt a trusted sender from some or all spam checks (a.k.a. whitlelisting).  The worst possible scenario is to allow whitelisting of a spoofed identifier:  A successful spoof switches the message from untrusted to trusted, and the trusted status exempts the message from spam checks.   Therefore, whitelisting should be used with caution (because whitelisted sources can get compromised) and should require some form of sender verification, so that spoofing risk can be ruled out. 

This means that whitelisting requires multi-part allow rules:
1) If identifier = <trusted sender ID> and
2) That sender is verified using <authentication technique>

Often we need proxy verification that cannot be provided by SPF and DMARC.   For example, you may get a password reset message from your favorite website.   The message is sent through Sendgrid and the message does not have a DKIM signature for the website domain.  To provide whitelisting for this situation:
1) Mail From domain = Sendgrid.net
2) Mail From domain is verified by SPF
3) From domain is <trusted vendor name>
0
Sabatino Replied
I studied over the weekend.
First of all, the title of this post is wrong as I don't think it has anything to do with arc.

My initial doubt was...
domain: domain.tld on office 365

No dkim key set on domain.tld dns

server that sends domain-tld.onmicrosoft.com

The email arriving on SM reveals DKIM pass


And I couldn't understand why.

Then thinking better I think I understood.

First of all, MS servers are authorized in the spf record of domain.tld

The dkim are calculated on keys present on the domain domain-tld.onmicrosoft.com

The dkim pass simply says that the key is correct, calculated according to the selector indicated on domain-tld.onmicrosoft.com

Which combined with the SPF record gives me a good degree of safety.

I got lost thinking that the dkim key had to be valid on a selector that necessarily had to be found on the sender's DNS, i.e. on the domain.tld DNS

If my reasoning is right SM could add a new option.

Let me explain.
If a domain does not have dkim enabled, it could sign with a general dkim from the SM server and therefore based on the selector on the dns
_domainkey.myserversmtp.tld


The advantage is not having to create the dkim key on each individual domain.
Sabatino Traini Chief Information Officer Genial s.r.l. Martinsicuro - Italy
0
Douglas Foster Replied
Inbound Trafic
As you inferred, the SmarterMail DKIM check only evaluates whether the signatures that are present produce PASS or not.   The documentation is vague, but I think the SmarterMail DKIM test produces PASS if all signatures pass or if no signatures exist.  The result is FAIL if at least one signature fails.   When the DKIM test is enabled, it must produce PASS for a message to be given Trusted Sender status.

DKIM PASS is generally considered useful as part of a DMARC test, where the signature has to both validate and be in the same organization as the From address.   I consider the standalone DKIM test rather useless.

Remember that mailing lists will generally add content to subject or body which invalidates prior signatures.  It is one of the bitter complaints against DMARC.   If your users participate in mailing lists, this behavior makes our life more difficult.

Outbound Traffic
For Outbound traffic, a DMARC policy with alignment rule "relaxed" will allow any DKIM signature or SMTP Mail From address to authenticate any From address in the organization, where "same organization" is defined by the list from "publicsuffix.org" (as corrected and amended by your local policy.)  This could allow you to achieve the result you want of having one signature for many domains.   However, SmarterMail always applies a signature that exactly matches the sending domain, which is better practice anyway.

You could use a non-SM outbound gateway to apply a signature which represents your server farm, as Google and Microsoft do, but it communicates nothing useful.  The same result can be achieved by performing a forward DNS lookup on the HELO and Reverse DNS host names.

Given the recent moves by Google and Yahoo, every sending domain should publish a DMARC policy, and should ensure that outbound messages are signed so that they can be authenticated using both SPF and DKIM.


0
Sabatino Replied
but also by setting the dkim to strict
adkim=s
the check is simply done on the domain indicated in d= of the dkim signature

so what MS does continues to be valid, i.e. sign with
domain-tld.onmicrosoft.com

It is not true that it is the same as HELO and reverse dns

spf verifies that the ip comes from an authorized ip and dkim verifies that the message has not been modified during transport
Sabatino Traini Chief Information Officer Genial s.r.l. Martinsicuro - Italy
0
Sabatino Replied
but also by setting the dkim to strict
adkim=s
the check is simply done on the domain indicated in d= of the dkim signature

so what MS does continues to be valid, i.e. sign with
domain-tld.onmicrosoft.com

It is not true that it is the same as HELO and reverse dns

spf verifies that the ip comes from an authorized ip and dkim verifies that the message has not been modified during transport
Sabatino Traini Chief Information Officer Genial s.r.l. Martinsicuro - Italy

Reply to Thread