-
Notifications
You must be signed in to change notification settings - Fork 395
New Identity Resolution Graphs #521
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 1 commit
Commits
Show all changes
5 commits
Select commit
Hold shift + click to select a range
4e8353e
New Identity Resolution Graphs
9f6adc9
Update src/personas/identity-resolution/identity-resolution-settings.md
josephineliu 43697b5
Update identity-resolution-settings.md
josephineliu 5e07f87
Update personas-space-set-up.md
josephineliu 5abdcbf
Update personas-space-set-up.md
josephineliu File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file was deleted.
Oops, something went wrong.
157 changes: 157 additions & 0 deletions
157
src/personas/identity-resolution/identity-resolution-settings.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,157 @@ | ||
| --- | ||
| title: Identity Resolution Settings | ||
| --- | ||
| # Setting Up Identity Graph Rules | ||
| Before connecting a source to a Personas space, we recommend first reviewing our default Identity settings and configuring custom rules as needed. Updates to configurations will only be applied to all new data flowing through the space after the changes have been saved. Thus, if this is your first time setting up your Identity Graph, we recommend getting started with a *Dev* space [here](/docs/src/personas/identity-resolution/personas-space-set-up.md). | ||
|
|
||
| > note "" | ||
| > **NOTE:** The Identity Resolution table can only be edited by workspace owners and users with the Identity Admin role. | ||
josephineliu marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| ## ExternalIDs | ||
| Segment creates and merges user profiles based on externalIDs we use as identifiers. You can view these externalIDs in the Identities tab of a User Profile in the User Explorer: | ||
|
|
||
|  | ||
|
|
||
| By default, Segment promotes the following traits and IDs in track and identify calls to be used as externalIDs: | ||
|
|
||
| | External ID Type | Message Location in Track or Identify Call | | ||
| | ------------------ | ------------------------------------------------------------------------------------------------------------- | | ||
| | user_id | userId | | ||
| | email | traits.email or context.traits.email | | ||
| | android.id | context.device.id when context.device.type = 'android' | | ||
| | android.idfa | context.device.advertisingId when context.device.type = 'android' AND context.device.adTrackingEnabled = true | | ||
| | android.push_token | context.device.token when context.device.type = 'android' | | ||
| | anonymous_id | anonymousId | | ||
| | cross_domain_id | cross_domain_id | | ||
| | ga_client_id | context.integrations['Google Analytics'].clientId when explicitly captured by users | | ||
| | group_id | groupId | | ||
| | ios.id | context.device.id when context.device.type = 'ios' | | ||
| | ios.idfa | context.device.advertisingId when context.device.type = 'ios' AND context.device.adTrackingEnabled = true | | ||
| | ios.push_token | context.device.token when context.device.type = 'ios' | | ||
|
|
||
| You'll notice that these identifiers have the *Provided by Segment* label next to it under *Identifier Type*. | ||
|
|
||
| To create your own custom externalID, click on *Add Identifier*. | ||
|
|
||
| These custom identifiers must be sent in the custom `externalIds` in the `context` object of any call to our API. The four fields below (id, type, collection, encoding) are all required: | ||
|
|
||
| | Key | Value | | ||
| | ---------- | ---------------------------------------------------------------------------- | | ||
| | id | value of the externalID | | ||
| | type | name of externalID type (`app_id`, `ecommerce_id`, `shopify_id`, etc) | | ||
| | collection | `users` if a user-level identifier or `accounts` if a group-level identifier | | ||
| | encoding | `none` | | ||
|
|
||
| The following example payload adds a custom `phone` externalID type: | ||
|
|
||
| ``` js | ||
| analytics.track('Subscription Upgraded', { | ||
| plan: 'Pro', | ||
| mrr: 99.99 | ||
| }, { | ||
| externalIds: [ | ||
| { | ||
| id: '123-456-7890', | ||
| type: 'phone', | ||
| collection: 'users', | ||
| encoding: 'none' | ||
| } | ||
| ] | ||
| }) | ||
| ``` | ||
| We recommend adding custom externalIDs to the Identity Resolution table *before* events containing this identifier flow through the space. Once an event with a new type of externalID flows into the space, the externalID will automatically be added to the table if it wasn't manually added. However, when the externalID is automatically added, it will default to our preset priority and limit, as explained below. | ||
|
|
||
| ## Flat Matching Logic | ||
| When a new event flows into Personas, we look for any profiles that match any of the identifiers on the event. | ||
|
|
||
| There are three cases that can occur: | ||
|
|
||
| **Case One: Create New Profile** | ||
| When there are no pre-existing profiles that have matching identifiers to the event, we create a new user profile. | ||
|
|
||
| **Case Two: Add Event to Existing Profile** | ||
| When there is only one profile that matches all identifiers in an event, we attempt to map the traits, identifiers and events on the call to that existing profile. If there is an excess of any identifier on the final profile, we defer to our merge protection rules outlined below. | ||
|
|
||
| **Case Three: Merge Existing Profiles** | ||
| When there are multiple profiles that match the identifiers in an event, we attempt to merge profiles and first check our merge protection rules as outlined below. | ||
|
|
||
| ## Merge Protection Rules | ||
| Identity Admins should review and configure the merge protection rules in the Identity Resolution Settings page to protect the identity graph from inaccurate merges. | ||
|
|
||
| One common example of a use-case that can cause inaccurate merges is the Shared iPad setup. For example, many companies now have iPads available in-store for customers to register for an account or submit order information. If different users submit information on the same device, there will now be multiple events sent with the same deviceID. Without merge protection rules in place, we might see all these different users merged into the same user profile based on this common identifier. | ||
|
|
||
| Our three merge protection rules allow Identity Admins to block incorrect values from causing incorrect merges, to set the maximum number of values allowed per externalID, and to customize the priority of these externalIDs. | ||
|
|
||
| ### Blocked Values | ||
| We recommend proactively blocking certain values from ever been used as identifiers. While these values will remain in the payload on the event itself, it will not be promoted to the externalID object Segment uses to determine user profiles. | ||
|
|
||
| This is extremely important when developers have a hard-coded value for fields like user_id during QA or development that then erroneously makes it production. This can cause hundreds of profiles to merge incorrectly and can have costly consequences when these spaces are already feeding data into a production email marketing tool or push notification tool downstream. | ||
|
|
||
| In the past, we've seen certain default values across many different customers cause mass amounts of profiles to merge incorrectly. Segment suggests that for every externalID, customers opt into automatically blocking the following suggested values: | ||
|
|
||
| | Value | Type | | ||
| | ----------------------------- | --------------- | | ||
| | Zeroes and Dashes (`^[0\-]*$) | Pattern (REGEX) | | ||
| | -1 | Exact Match | | ||
| | null | Exact Match | | ||
| | anonymous | Exact Match | | ||
|
|
||
| Before sending data through, we also recommend adding any default hard-coded values that your team uses during the development process. In the UI today, the Identity Admin can add blocks against only exact matches to the inputted value. | ||
|
|
||
| However, we currently have Limited Availability for a feature that allows customers to create custom REGEX blockers as well. This can be useful for cases where an externalID was incorrectly implemented. To enable this, please reach out to your customer success manager. | ||
|
|
||
| ### Limit | ||
|
|
||
| Identity Admins can specify the total number of values allowed per externalID type on a profile. This will vary depending on how companies define a user today. In most cases, companies rely on `user_id` to distinguish user profiles and Segment recommends the following default configurations: | ||
|
|
||
| | ExternalID | Limit | | ||
| | --------------------- | ----- | | ||
| | user_id | 1 | | ||
| | all other identifiers | 5 | | ||
|
|
||
| There are specific cases that will deviate from this default. For example, a case where a user can have more than one user_id but only one email, like when a user is defined by both their shopify_id and an internal UUID. In this case, an example setup may be: | ||
|
|
||
| | ExternalID | Limit | | ||
| | --------------------- | ----- | | ||
| | email | 1 | | ||
| | user_id | 2 | | ||
| | all other identifiers | 5 | | ||
|
|
||
| We are currently offering Limited Availability of a new type of limit called "Time-Based Limits" that will allow for any profile to have a specified limit of values within a certain time range. This is particularly useful for externalIDs such as anonymousID and ga_client_id which are constantly renewed and will be increasingly accrued over a lifetime. We know that customers can easily accrue over 100 anonymous_ids over a few years of app usage. However, rather than setting the absolute limit of anonymous_ids to 100, we can now create a sliding range that intelligently gates how many anonymous_ids any user should reasonably accrue within a specific time period. | ||
josephineliu marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| ### Priority | ||
|
|
||
| The priority of an identifier is taken into consideration once we exceed the limit of any identifier on the final profile. | ||
|
|
||
| Let's take as an example a Personas space with the following Identity Resolution configurations: | ||
|
|
||
| | ExternalID | Limit | Priority | | ||
| | ------------ | ----- | -------- | | ||
| | user_id | 1 | 1 | | ||
| | email | 5 | 2 | | ||
| | anonymous_id | 5 | 3 | | ||
|
|
||
| A profile already exists with user_id `abc123` and email `[email protected]`. A new event comes in with new user_id `abc456` but the same email `[email protected]`. If we mapped this event to this profile, the resulting profile would then contain two user_ids and one email. Given that user_id has a limit of 1, we've now exceeded the limit of an identifier so check the priority of these identifiers. Because email and user_id are the only two identifiers on the event and email is ranked lower than user_id, we demote email as an identifier on the incoming event and try again. | ||
|
|
||
| At this point, the event searches for any profiles that match just the identifier user_id `abc456`. Now there are no existing profiles with this identifier, so a new profile is created with user_id `abc456`. | ||
|
|
||
| By default, we explicitly order user_id and email as rank `1` and `2`, respectively. All other identifiers are in alphabetical order beginning from rank `3`. This means that if the only identifiers ever sent in on events flowing into personas are user_id, email, anonymous_id and ga_client_id, the rank would be as follows: | ||
|
|
||
| | ExternalID | Priority | | ||
| | ------------ | -------- | | ||
| | user_id | 1 | | ||
| | email | 2 | | ||
| | anonymous_id | 3 | | ||
| | ga_client_id | 4 | | ||
|
|
||
| If a new android.id identifier appeared without first giving it explicit order, the order would automatically reshuffle to: | ||
|
|
||
| | ExternalID | Priority | | ||
| | ------------ | -------- | | ||
| | user_id | 1 | | ||
| | email | 2 | | ||
| | android.id | 3 | | ||
| | anonymous_id | 4 | | ||
| | ga_client_id | 5 | | ||
|
|
||
| Thus, if you require an explicit order for all identifiers, configure this in the Identity Resolution settings page before sending in events. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If we're changing the name of this file or deleting it, we need to set up a redirect. 🤔
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
how do i set up a redirect?