Question:

How to whitelist high-confidence phish emails from KnowBe4 in O365?

I am using KnowBe4 to test the phishing awareness of my employees. I have set up an Advanced Delivery Policy and exchange rules to bypass the spam confidence level (SCL) based on the headers of the emails from KnowBe4. However, some of the emails are still being blocked and quarantined by Microsoft Defender, even though the message details show that the exchange rules and a connection filter are applied. I have verified that the emails are from KnowBe4’s IP address and that they do not contain any malicious attachments or URLs.

I do not want to whitelist all the domains that KnowBe4 uses in my Anti-Spam Policy, as that seems unnecessary and risky. I have followed the documentation from both Microsoft and KnowBe4, but I cannot figure out why the Advanced Delivery Policy and the exchange rules are not working as expected.

I noticed that in my KnowBe4 account settings, I had enabled DKIM to use my own domain, instead of KnowBe4’s. I changed this setting and it solved the problem. I think this is because the Advanced Delivery Policy checks the DKIM signature of the emails.

Can anyone explain why this is the case and if there is a better way to whitelist the high-confidence phish emails from KnowBe4 in O365? Thank you for your help.

Answer:

Phishing is a common cyberattack technique that involves sending fraudulent emails to trick recipients into revealing sensitive information, clicking on malicious links, or downloading harmful attachments. Phishing emails often impersonate legitimate organizations or individuals, and use social engineering tactics to manipulate the emotions or expectations of the targets.

To protect your organization from phishing attacks, you need to educate your employees on how to spot and report suspicious emails, and also implement technical measures to filter out or block malicious emails. One of the tools that can help you with both aspects is KnowBe4, a security awareness platform that allows you to simulate phishing campaigns, test the vulnerability of your users, and provide them with training and feedback.

KnowBe4 works by sending simulated phishing emails to your users, and tracking their responses and behaviors. The emails are designed to mimic real-world scenarios, such as password reset requests, invoice notifications, or urgent messages from the management. The emails are also categorized into different levels of difficulty, ranging from obvious to very subtle. The higher the difficulty, the more likely the email is to bypass your existing email security solutions, such as Microsoft Defender for Office 365 (O365).

Microsoft Defender for O365 is a cloud-based service that provides protection against phishing, malware, spam, and other email threats. It uses various detection technologies, such as machine learning, heuristics, and reputation filters, to analyze the content and context of the incoming emails, and assign them a spam confidence level (SCL). The SCL is a numerical value that indicates the probability that an email is spam. Based on the SCL, the email can be delivered to the recipient’s inbox, junk email folder, or quarantine.

However, sometimes Microsoft Defender for O365 may incorrectly classify a legitimate email as spam, or vice versa. This can cause false positives (blocking good emails) or false negatives (allowing bad emails). To prevent this, you can use whitelisting and blacklisting to override the default behavior of Microsoft Defender for O365. Whitelisting means allowing certain emails to bypass the spam filtering, while blacklisting means blocking certain emails regardless of the spam filtering.

Whitelisting and blacklisting can be done at different levels, such as sender, domain, IP address, or header. You can also use different methods, such as exchange rules, connection filters, anti-spam policies, or advanced delivery policies. Each method has its own advantages and limitations, and you need to choose the one that best suits your needs and preferences.

In this article, we will focus on how to whitelist high-confidence phish emails from KnowBe4 in O365, using the advanced delivery policy method. This method allows you to specify a list of IP addresses or domains that are allowed to send simulated phishing emails to your users, without being affected by the spam filtering or the SCL. This way, you can ensure that your users receive the most challenging and realistic phishing tests, and measure their performance and progress.

To set up an advanced delivery policy, you need to follow these steps:

1. Log in to the Microsoft 365 admin center, and go to Security > Threat management > Policy > Anti-spam.

2. Click on Advanced delivery, and then click on + Add.

3. Enter a name and a description for your policy, and then select Simulated phishing messages as the Scenario.

4. Under Sender domains or IP addresses, enter the IP addresses or domains that are used by KnowBe4 to send simulated phishing emails. You can find the list of IP addresses and domains on the KnowBe4 website, under Account Settings > Phishing Settings > Email Whitelisting.

5. Under Recipient domains, enter the domains that belong to your organization, and that you want to receive the simulated phishing emails.

6. Click on Save to create your policy.

Once you have created your advanced delivery policy, you need to make sure that it is applied to the incoming emails from KnowBe4. To do this, you need to check the message headers of the emails, and look for the following values:

• X-MS-Exchange-Organization-MessageDirectionality: This indicates the direction of the message flow. The value should be Incoming for the emails from KnowBe4.
• X-MS-Exchange-Organization-AdvancedDeliveryPolicyApplied: This indicates whether the advanced delivery policy was applied to the message. The value should be True for the emails from KnowBe4.
• X-MS-Exchange-Organization-SCL: This indicates the spam confidence level of the message. The value should be -1 for the emails from KnowBe4, meaning that they are exempt from spam filtering.

If you see these values in the message headers, it means that the advanced delivery policy is working as expected, and that the emails from KnowBe4 are delivered to your users’ inboxes. However, if you do not see these values, or if you see different values, it means that there is a problem with the advanced delivery policy, and that the emails from KnowBe4 are being blocked or quarantined by Microsoft Defender for O365.

There are several possible reasons why the advanced delivery policy may not work as expected, such as:

• The IP addresses or domains of KnowBe4 are not entered correctly in the policy, or they have changed since you created the policy. You need to verify that the IP addresses or domains are accurate and up-to-date, and update the policy if necessary.
• The recipient domains of your organization are not entered correctly in the policy, or they have changed since you created the policy. You need to verify that the recipient domains are accurate and up-to-date, and update the policy if necessary.
• The advanced delivery policy is not enabled, or it has a lower priority than other policies that may conflict with it. You need to check the status and the order of your policies, and make sure that the advanced delivery policy is enabled and has a higher priority than other policies that may override it.
• The emails from KnowBe4 are not using the DKIM signature of KnowBe4, but of your own domain. This can cause the advanced delivery policy to fail, as it relies on the DKIM signature to identify the sender of the emails. You need to check the DKIM settings of your KnowBe4 account, and make sure that they are using the DKIM signature of KnowBe4, not of your own domain.

By following these steps, you should be able to whitelist high-confidence phish emails from KnowBe4 in O365, using the advanced delivery policy method. This will allow you to test the phishing awareness of your users, and provide them with the most realistic and challenging phishing scenarios. However, this method is not the only one available, and you may prefer to use a different method, such as exchange rules, connection filters, or anti-spam policies. You can find more information about these methods on the Microsoft and KnowBe4 websites, and choose the one that best suits your needs and preferences.

We

hope that this article was helpful and informative. If you have any questions or feedback, please feel free to contact us. Thank you for reading.