Embrace

Embrace’s Alert feature is a powerful feature that allows your team to be the first to know about issues and to be proactive about potential problems. It allows teams to create alerts based on metrics, first-seen issues, performance, and network-related errors. In addition, you can configure alert thresholds to fit your needs, enabling you to differentiate warnings from errors. 

Table of Contents:

• How to Send Alerts

  • To Slack

  • To Email

  • To Webhooks

• Alerts Summary Page

  • Alert Tabs 

• Examples of Types of Alerts

  • Filters

  • Creating a Crash "Metric" Type Alert

  • Creating a Performance "Metric" Type Alert

  • Creating a Session Outcome "Metric" Type Alert

  • Creating a General Health "Metric" Type Alert

  • Creating a "First Seen Issues" Type Alert

  • Creating a "Proactive Network Errors" Alert

How to Send Alerts: 

Alert notifications can be sent by Slack, email, and Webhooks.

To Slack: 

To configure Slack: Admins must first integrate Slack with Embrace. From the dash, select an app, then proceed to Settings - Notifications. Select the appropriate channels to send alerts to within your organization. 

Following, Admins will be able to configure what alerts to send to Slack by selecting “Send to Slack” under “Notify your Team” when editing or adding a new alert. Admins can also configure the notification level for individual alerts (ex: @channel, @here). 

*Slack channels can be removed as needed. A popup will appear if you try to delete a Slack channel that is associated with a preexisting alert. In this case, an Admin can click the link to go to the corresponding Edit Alert page and update the channel to an active one before completing the deletion. Similarly, the “When an issue is reopened” message will appear if the Slack channel you want to delete is the assigned one for reopened issues. It will link to the Settings page of any apps in the organization where this is the case. You can click through and update the channels where needed before continuing with the deletion.

To Email:

To send alerts to a specific team member or all teams, select “Email Team Members” under “Notify your Team” when editing or adding a new alert. 

Via Webhooks:

Webhooks enables easy integrations with services like PagerDuty, Microsoft Teams, and more. 

To send alerts via Webhooks: Admins, must first add the endpoint URL and test the integration.  From the dash, select an app, then proceed to Settings – Notifications – Webhook Integration. Follow the link to the instructions on the page for help. 

Following, Admins will be able to send alerts to webhooks by selecting “Send to Webhooks” under “Notify your Team” when editing or adding a new alert.

Alert Summary Page:

On the alerts page, you will see the “Add Alert” button in addition to three tabs:

• Triggered Alerts - A list of alerts that are currently triggered based on the configuration of the alert.
• Manage Alerts - A list of all created alerts; from here, we can edit, duplicate, mute, or delete alerts.
• Alert History - History view of alerts that were triggered. 

Examples of Types of Alerts:

To create a new alert, go to the alerts page and select “Add Alert.” Once selected, you will see the alert configuration window. 

There are various metrics you can alert on such as performance, session outcomes, and general health metrics. To see the different options, select the dropdown next to “Alert Type”. Alert configurations will vary based on the Alert Type selected.

⚠️  Note: You will need to provide the alert name to save it. Do not refresh the page until you have saved the alert. 

Filters:
Embrace has 10 available filters, giving you more control to reduce noise and focus on the exact conditions that matter to your team. 

Here are some examples of how others have utilized filters: 

• OS version or build; Add these filters to discover potential fraud coming from uncommon device and version characteristics. For example, if you notice a spike in usage on a very old OS, this could point you towards spam or bot activity.
• Log property key and value; Add log property filters to better investigate errors that can happen from distinct sources. For example, if your Apple Pay purchases frequently fail compared to other vendors, you can isolate that purchase failure type for better visibility.
• Session property key and value; Add session property filters to focus your attention on underperforming experiments and promotions. Lower engagement metrics from newly acquired users could signal that your most recent UA campaign was targeting the wrong audience.

Creating a Crash “Metric” Type Alert:

In this example, we create a metric alert based on “Crash User Percentage” with a time window of an hour. We set the following values to define our trigger thresholds:

• Minimum user count: Set to 25, will not trigger if the user count is less than 25.
  • This is the minimum number of total users within the query window we should consider, affected or not (the denominator), before we report. For example, if you set a 10% threshold with a minimum user count of 10, it is reasonable to receive an alert for 2 affected users for a population of 10 users for that hour. 
  • The minimum user count is meant to prevent noise from alerts at a low volume. We suggest picking a larger minimum user count, 100 or 1000 (depending on your DAU), to reflect what you expect to be the average number of unique users there are during regular business hours.

• Error Threshold: Set to above 1%. If needed, you have the option to select the following conditions:
• Error Recovery Threshold: Set to below or equal to 0.75%, will not trigger unless the error threshold drops to or below 0.75%. By default, the only condition available for this field would be “Below or equal to.”
• Warning Threshold: Set to above 0.50%. By default, the only condition available for this field would be “Above.”
• Warning Recovery Threshold: Not using, but it works the same way as the “Error Recovery Threshold.” By default, the only condition available for this field would be “Below or equal to.”

*Utilize the Filter, to focus on a specific app version or a particular crash message. You can also use “Breakout By” to apply the alert rules for every crash message to avoid having to create multiple alerts.

Creating a Performance “Metric” Type Alert:

Moments are Embrace’s way of monitoring the timing and outcome of key user moments within your mobile apps. They are best used to track critical user flows that are generally short in nature. The goal is to understand where these user flows are running slow, ultimately leading to user frustration and app abandonment. To implement custom moments, check out the Embrace integration documents. 

Here are just a few examples of the many ways you can be alerted to underperforming key moments:  

• When startup abandonment exceeds 0.5% of sessions
• When the purchase completion rate drops below 97%
• When sending a chat message stalls out more than 2% of the time
• When median startup time exceeds 3000ms

*Utilize the Filter, to focus on a specific moment name, property key, or value.

Creating a Session Outcome “Metric” Type Alert:

These additional metrics allow your team to be notified of spikes in a wider range of failure types. For example, e-commerce and social media apps can surface when they are exceeding resource constraints with excessive images and videos. In addition, mobile games can quickly investigate new ANRs that might be hurting their Google Play Store ranking.

Here are some examples of different ways you can alert on session outcomes:

• ANR exit percentage
• User terminated session percentage
• Crashed session percentage
• OOM session percentage

* Utilize the Filter, to focus on a specific app version, device, and OS metrics, or session property keys and values. 

Creating a General Health “Metric” Type Alert:

These metrics can point your team towards potential problem areas. For example, a spike in low memory warnings could indicate excessive memory usage, insufficient caching, or the presence of memory leaks. A decrease in session duration times could mean that users are struggling to use new features or abandoning slow experiences. Metric alerts are evaluated in the previous real-time frame (i.e., 10:02 to 11:02). egardless of the time window selected, each alert is evaluated every minute.

Here are various examples of different ways you can alert on session outcomes:

• Low memory session percentage
• Session average duration
• App health

*The App Health metric is the percentage of sessions that do not contain a crash or error log. It’s a great way to monitor the overall stability of your application. You can track this metric across features and releases for quick visibility into regressions or improvements.

* Utilize the Filter, to focus on a specific app version, device, and OS metrics, or session property keys and values. 

Creating a “First Seen Issues” Alert:

The configuration is minimal for these types of alerts since we are just evaluating the first time a crash is seen within our database. There is no filtering or threshold configuration available. 

In this example, we create a “First Seen Issue” alert based on “First Seen Crash.”

⚠️ Note: For “First Seen Issues” alert types, the alert name will default to the alert type value selected. Do not refresh the page until you have saved the alert.

Creating a “Proactive Network Errors” Alert:

In this example, we create a “Proactive Network Error” alert based on Network 500s Errors. Proactive Alerts are based on the most recent complete hour of data, checked every 5 minutes. For example, a Proactive Alert checked at 2:05 PM would evaluate using data from 1:00 - 2:00 PM.

For any domains/endpoints that you do not want to be included in this alert, add them under “Manage exclusions”. 

We set the following values to define our trigger thresholds:

• 5XX Error Percentage Threshold: Set to 2, will not trigger unless the threshold percentage is equal to or greater than 2%.
• Minimum 5xx Error Count: Set to 10, will not trigger if the 5XX network count is less than 10. 
  • This is the minimum number of total users within the query window we should consider, affected or not (the denominator), before we report. For example, if you set a 10% threshold with a minimum user count of 10, it is reasonable to receive an alert for 2 affected users for a population of 10 users for that hour. 
  • The minimum user count is meant to prevent noise from alerts at a low volume. We suggest picking a larger minimum user count, 100 or 1000 (depending on your DAU), to reflect what you expect to be the average number of unique users there are during regular business hours.

⚠️ Note: For “Proactive Network Errors” alert types, the alert name will default to the alert type value selected. Do not refresh the page until you have saved the alert.

* To drill down on a specific domain/pain, utilize the Network Requests alert type and corresponding filters. For example, domain & path, status code range = 500-599. 

If you ever need assistance with alerts, please do not hesitate to reach out to us! We are here to help.