Skip to main content

Notifications

Announcements

No record found.

News and Announcements icon
Dynamics 365 Community / Blogs / Dynamics 365 and Power Platform in the Field / Server-Side Synchronization...

Server-Side Synchronization Series Part 6: Email Address Party Correlation

Cody Dinwiddie Profile Picture Cody Dinwiddie
Introduction

We know that an incoming email will create an Email Message record automatically in Dynamics if it meets promotion criteria, such as being a direct reply. What happens if an email address on the Exchange item matches multiple records in Dynamics? How does it know which record to tie the email party to if there are multiple Contacts with the same email address? In this part 6 of our Server-Side Sync series, we will dive into how email address party correlation works and how you can avoid email messages correlating to irrelevant Dynamics records.


Outgoing Email Party Correlation

When an email is sent from Dynamics, each email address placed in the To: or CC: lines has to tie to a particular record:


Or it can resolve to nothing and be in red text (sending in this state requires the System Setting “Allow messages with unresolved email recipients to be sent” to be set to Yes):
                     


Outgoing emails do not have the issue of correlating to multiple records since one has to be set directly by the user creating the email (or whatever mechanism is creating it, such as a Power Automate trigger). But what happens when an email message is replied to and is promoted as an incoming email?


How Dynamics Stores Email Correlation Data

To understand Email Correlation we first have to understand the SQL architecture Server-Side Sync uses for correlation. The primary tables we use for synchronization correlation are ActivityPointer, ActivityParty, and EmailSearch. 
  • ActivityPointer (link to schema documentation) represents any activity or task that is performed or to be performed by a user (such as Email Message or Appointment). This table contains the relevant data for the activity:
    • The ActivityPointer row itself:
      • Column: ActivityId
    • Who/what owns the Activity and the State of the Activity:
      • Columns: OwnerId, OwnerIdType, OwningUser, OwningTeam, OwningBusinessUnit, StateCode,
    • What type of activity the row is (Email, Appointment, etc):
      • Column: ActivityTypeCode
    • Data relevant to the activity:
      • Columns: Subject, Description (Body), ActualStart, ActualEnd, StatusReason, StatusCode
    • If the activity is Set Regarding another record:
      • Columns: RegardingObjectId, RegardingObjectIdName, RegardingObjectTypeCode

  • ActivityParty (link to schema documentation) ties an Email Address (AddressUsed) value to the following:
    • The ActivityParty row itself:
      • Column: ActivityPartyId
    • The Activity that houses the Email Address (such as Email Message or Appointment)
      • Column: ActivityId, ActivityTypeCode
    • The Activity Party type (such as From, ToRecipient, Sender, RequiredAttendee etc)
      • Column: ParticipationTypeMask
    • Party Object record, such as a Contact or a System User
      • Columns: PartyId, PartyObjectTypeCode, PartyIdName

  • EmailSearch ​​​​​​​is a system entity which ties email addresses within Dynamics fields set to Email format to records within Dynamics (more about this later):
    • The EmailSearch row itself:
      • Column: EmailSearchId
    • The Email Address value in an Email format field:
      • Column: EmailAddress
    • The Record and record type that houses the Email Address value:
      • Columns: ParentObjectId, ParentObjectTypeCode
    • The field that contains the Email Address value:
      • Column: EmailColumnNumber
        • Be aware that this column number is tied to the field name on the MetadataSchema.Attribute table

How Email Search System Entity Works

In order for an email address to show up on the EmailSearch entity (and be used for incoming activity correlation), three steps need to have been followed:
  1. The field housing the email address value must be set to
             Data Type: Single Line of Text
             Field Type: Simple
             Format: Email

  2. An email address must be entered into this Email format field and saved
             The email address must follow the format of “<value>@<value>.<value>"

  3. The entity that houses this Email format field must have the option “Sending Email” checked under the entity configuration “Communication and Collaboration”:
             Default Activity entities have this set by default, such as Account, Contact, User and Opportunity
Assuming the above are all true, the EmailSearch will tie the email address value in these fields to records within Dynamics, which is then used by incoming activity correlation.


How Incoming Activity Correlation Works

When an activity is promoted to Dynamics, a row is created on the ActivityPointer table and then we start to populate each ActivityParty (email addresses). To do this, we need to correlate Exchange email address values to records within Dynamics. We take the values in all of the Exchange party fields (such as “From” “To” and “CC”) and then look on the EmailSearch table for any matches. If a match (or multiple matches) are found, we create the ActivityParty rows for each (be aware the From field is special and we’ll talk about that later). 

As an example, if an email address value in the To: field is tied to both a Contact record AND a System User record, two ActivityParty rows will be created for the single email address. The record within Dynamics will have the To: line contain two values: <Name of Contact>, <Name of User>. Here’s an example of what that looks like for a single email address in the "To:" line:


An end user might be confused at this and think this email was sent to two different email addresses. Simply put, any record found on EmailSearch tied to each email address will be put into the To: and CC: lines, which can lead to problems if the EmailSearch entity contains a ton of irrelevant correlations. 

How The “From” Field Correlation Works

The “From” field inside Dynamics is special in that it can’t correlate to multiple records; it must be tied to a single record within Dynamics for Party correlation. Since the email address in the “From” field can correlate to multiple records and the automatic promotion of the email can’t know what a system user would subjectively want it to correlate to, Server-Side sync must pick a record from the list of the available matches. We do this by using a hierarchy of entities in order as follows:
  1. SystemUser
  2. Contact
  3. Account
  4. Lead
  5. Equipment
  6. Team
  7. Business unit
  8. All other Email-enabled entities (such as custom entities)
If there are more than one hundred results found, the from field is not resolved and will display the email address in red text post-creation. This should never happen as correlation should never find more than a handful of records.

Of the matches found, Server-Side Sync first determines if any of records sorted by this hierarchy are owned by the same promotion context (the system user creating the email record). The “From” field is then set to the to the record which has the highest entity hierarchy owned by that user. If none of the records are owned by the promotion context (such as when the email is promoted by a queue), the highest record found within the entity hierarchy is set to the “From” field within ActivityParty. 

It’s extremely important to call out that it is best practice to never have multiple records within the same entity that correlate to the same email address. As an example, each system user should have unique email format field values so that EmailSearch can never correlate to multiple records within the same entity context. The same should be true for Account, Contact, and every other activity entity.

The last thing to note here is that if this best practice is not followed and there are multiple records correlated within the same entity type (object type code), such as EmailSearch matching two distinct SystemUser contexts, the record that will be selected among those found will be non-deterministic. Party resolution of the “From” field will not work reliably if multiple correlations are found within the same entity context.


Best Practices Regarding Email Address Correlation
  • Ensure that records within the same Entity never share a value for an Email Format field
  • If a field containing an email address should not correlate on promoted activities, do not set the field to Email Format. Use the “Text” format instead.
  • EmailSearch can be viewed directly using API calls to evaluate records returned by email address
    • Example API call: https://contoso.crm.dynamics.com/api/data/v9.0/emailsearches
  • System User should only ever have email addresses tied to the user context, never other system users (such as an email format field tied to a user’s manager)

Troubleshooting Examples

Example 1: Multiple irrelevant EmailSearch correlations causing Reply All to send to recipients not on the original email

Scenario: A new field called “Lead Approver” on the Lead entity is configured as an Email Format field due to a customer’s business requirement. In order for a Lead to be saved, this business required field must contain the email address of the user which approved the lead. The same email address tied to a system user is also in this “Lead Approver” field in hundreds of Lead records. 

An incoming email is promoted to Dynamics where this single email address is in the To field:


Here's what that email looks like once promoted to Dynamics:


As you can see, the name of every lead shows up in the "To:" line since EmailSearch contains a value tying the email address to each of those leads. If a user opens this record they might assume that the actual email was sent to hundreds of people. If a reply all is pressed on this email, this occurs:


Since “Reply All” will use each Lead correlation lookup for the To: field, you might assume that it will send an email to the same email address as whatever is in the “Lead Approver” field, but this is not the case. Reply All will use the default primary email field for the entity, which for Lead will be whatever is in the “EmailAddress1” (Email) field. The end result is that a user accidentally emails hundreds of different leads due to an improperly configured Email format field correlation.

How to Resolve: Make the “Lead Approver” field a text format field to not correlate on EmailSearch.


Example 2: Improperly configured User correlations cause users to believe there are recipients on an email that aren’t actually present

Scenario: A new field called “Manager Email” on the User entity is configured as an Email Format field to be able to easily reference a user’s direct report. This field will contain the email address of another system user present within the same organization. System User A’s User record has the primary email address set to A@contoso.com, and the Manager Email set to B@contoso.com. System User B’s User record has the primary email address set to B@contoso.com and Manager Email set to null.


An email which meets promotion criteria is sent where the To: line contains the email address B@contoso.com, but not A@contoso.com: 


The email is promoted to Dynamics and shows the following (remember that System User A is NOT a recipient of the actual email):


A user openign this email record might believe the original email was sent to System User A, which it was not. If Reply All is used, the default email value for System User A’s User record is used, which will be the primary email address A@contoso.com resulting in a recipient receiving a ReplyAll but not being on the original item.

How to Resolve: Make the Manager Email field a text format field to not correlate on EmailSearch.


Our next part in this series will tackle how Tracking Token correlation works.
​​​​​​​
Links to parts in this series:

Comments