The WebEngage Knowledge Base Developer Hub

Welcome to the WebEngage Knowledge Base developer hub. You'll find comprehensive guides and documentation to help you start working with WebEngage Knowledge Base as quickly as possible, as well as support if you get stuck. Let's jump right in!

Get Started    

Analyzing Push Campaigns

A step-by-step guide on how you can analyze the performance of a push campaign in your dashboard

Once you launch a Push campaign through the channel or a journey, its impact on user engagement and revenue can be analyzed in detail through Campaign Overview.

How to Access

Let's quickly show you how you can access campaign analysis through the various sections of your dashboard:

Access Campaign Overview through List of Campaigns

A list of all the campaigns created through the channel can be found under List of Campaigns or the central hub of Push. As shown below, you can click on the (hyperlinked) Campaign’s Name to analyze a campaign in further detail.

Accessing a Push campaign's overview (click to enlarge)

Access Campaign Overview Through Journey Overview

A list of all the campaigns attached to a journey can be found under Campaigns in the journey's Overview. As shown below, you can click on the (hyperlinked) Campaign Name to analyze the Push campaign in further detail.

Accessing a Push campaign's overview through its journey (click to enlarge)

Access Campaign Overview through Segment Overview

A list of all the campaigns sent to a segment of users can be found under Campaigns in the segment's overview section. As shown below, you can click on the (hyperlinked) Campaign Name to analyze the Push campaign in further detail.

Accessing a Push campaign's overview through its target segment's overview (click to enlarge)

Access Campaign Overview through a User Profile

When analyzing a user's profile, you can choose to analyze the campaigns sent to them in further detail by clicking on the (hyperlinked) Campaign Name listed under Most Recent Campaigns.

Before we dive into campaign analysis, let's get you acquainted with the top panel.

Understanding Top Panel

Top Panel indicating campaign details (click to enlarge)

As highlighted above, the top panel provides a quick snapshot of the campaign with key details like:

Click to enlarge

  • (Campaign) Type: As selected at Step 2: When, while creating the campaign.

  • (Campaign) Type: As selected when you initiate camaign creation in your dashboard. Push campaigns can be of the following types:

    • One-time

    • Triggered

    • Recurring

    • Transactional

    • Journey (can be created only through the Journey interface)

    • Relay (can be created only through the Relay interface)

    • As shown below, all journey campaigns can be traced back to the journey they're attached to by clicking on the (hyperlinked) campaign type, Journey. Doing so will direct you to the journey's Live View.

Click to enlarge

  • (Campaign) Status: Indicates the current status of the campaign, as per the settings specified at Step 2: When, while creating it.

    • If you save a campaign without launching it, then it will be saved as a draft. The same is reflected by the status, Draft.

Now, let's get you acquainted with its features:

Modify Campaign

As shown below, using the icons placed next to the Campaign's Name, you can choose to Edit, Delete, Duplicate, Pause/Resume and Tag the campaign, as per your marketing needs.

Click to enlarge

  • Given the short lifespan of one-time campaigns, Pause has been disabled for such campaigns.
  • Further, all modification icons are hidden for journey campaigns, since they can be modified only through the journey they have been created through.
  • Once you Pause a Running campaign, you can choose to Resume it's delivery to the target audience by clicking the Resume icon (that replaced the Pause icon).

🚧

Related Read: Modifying Push Campaigns

A step-by-step guide on how you can modify Push campaigns.

Download a Report

🚧

Detailed Read

Reports can be downloaded by only those Admins that have access to Account Management. If you are unable to do so, then please get in touch with the account owner.

You can easliy download a copy of campaign stats for a specific time frame for offline analysis, uploading it to another platform and so on. Here's how you can go about it:

Step 1: Click the Download icon

As shown above, in doing so, you will be prompted to specify the kind of data you want to analyse.

Step 2: Specify the time frame for which the stats should be shown (Data to Include)

As shown above, you can specify the time frame as:

  • Campaign's Start Date till Present Date
  • Custom Dates

🚧

Detailed Read

Step 3: Specify Level of Data Granularity

You can create upto 5 versions of a campaign and test each one to determine the copy that resonates with your audience. Thus, while downloading a report, you can choose to analyse consolidated campaign stats OR a variation-wise breakdown.

As shown above:

  • Select Campaign Level Granularity to view the average performance of all campaign variations.

  • Select Variation Level Granularity to view the individual performance of each variation and identify the top performing version.

Step 4: Click Download!

Schedule a Report

In our quest to make campaign reporting a seamless experience, we built Scheduled Reports - customizable and automated reports delivered straight to your inbox!

Click to enlarge

As highlighted above, using the Reporting icon, you can set up a periodic report to have all the stats shown under Overview, emailed to multiple the team members.

Here's how you can go about it:

Click to enlarge

Step 1: Click on the Reporting icon

  • As shown above, in doing so you will be prompted with a pop-up allowing you to customize the report's settings.
  • The report's name will be the same as the campaign's name. This will be included in:
    • The emailed report's subject line.
    • under the Scheduled Reports section of your dashboard.

Step 2: Define the Frequency of report delivery

You can choose to have a report delivered on a Daily, Weekly or Monthly basis, as per your needs.

  • As shown above, we have selected Weekly as the Frequency for our report.

Step 3: Specify the time of delivery (When)

As shown above, you can choose to have the report delivered at a specific time of the day. The same can be defined in an AM/PM format against the field, When.

Step 4: Specify the time frame for which the stats should be shown (Data to Include)

Too many data points always lead to chaotic insights. This is why we have made it possible for you to specify the time frame (number of days) for which you'd like to receive campaign stats.

Click to enlarge

As shown above, you can select a time frame between Last 1 Day to Last 30 Days against Data to Include.

🚧

Detailed Read

Step 5: Select Level of Data Granularity

At WebEngage, you can create upto 5 versions of a campaign and test each one with a small test segment to determine the copy that resonates with your audience. Thus, while scheduling a report, you can choose to analyse consolidated campaign stats OR a variation-wise breakdown.

As shown below:

  • Select Campaign Level Granularity to view the average performance of all campaign variations.

  • Select Variation Level Granularity to view the individual performance of each variation and identify the top performing version.

Step 6: Add team members as Subscribers

As shown above, you can choose from a list of all the team members who have access to your WebEngage dashboard. Doing so will ensure that all the added users receive the campaign performance report, as per its settings.

  • However, if a team member is not a part of your WebEngage account, then you will not be able to add them as a Subscriber.

Step 7: Click the Schedule button

  • A notification will pop-up on the bottom right corner of the page, confirming that the report has been scheduled. (Understanding Report Stats)

  • You can modify or delete the report anytime you like through the Scheduled Reports section, nested under Settings in your account.

🚧

Please Note

  • Reports can be configured by only those Admins that have access to Account Management. If you are unable to configure, then please get in touch with the account owner.

  • Additional reports for campaigns sent through Push and campaigns sent through all the channels combined can be scheduled through Scheduled Reports, nested under Settings in your dashboard.

See Campaign Details

As demonstrated below, clicking Show Details reveals a thorough recap of all the rules and settings specified while creating the campaign. This section can be minimized by clicking Hide Details.

Click to enlarge

Here's a list of all the details shown here:

  • Campaign ID: A unique campaign ID is generated by us each time you create a campaign.

  • Target Devices: As selected at Step 1: Audience, while creating the campaign, this can be any of the following options:

    • Both Android and iOS
    • Only Android
    • Only iOS
  • Target Android Apps: If you have added multiple app credentials while configuring Push for all the versions of your Android app, then you will be able to target all those apps with a Push campaign. Hence, this field indicates all the apps targeted by the campaign.

  • Target iOS Apps: If you have added multiple app credentials while configuring Push for all the versions of your iOS app, then you will be able to target all those apps with a Push campaign. Hence, this field indicates all the apps targeted by the campaign.

  • Start Date / Send Date: Indicates the date on which the campaign was launched or will be launched (if the campaign is still scheduled), as specified at Step 2: When, while creating it.

  • End Date: Indicates the date on which the campaign will end or ceased getting delivered.

  • Time zone: Applicable only to one-time campaigns, it indicates whether the campaign was sent in the user's timezone, or in your project's timezone. The same can be specified at Step 2: When while creating the campaign.

  • Tags: Tags is a handy feature that helps you categorize your campaigns as per their purpose, target audience, frequency or any other parameter that makes them easier to search for.

    • If a campaign is tagged: All the tags added to the campaign will be shown here. You can click on a hyperlinked tag to add more tags to the campaign or remove existing ones.
    • If no tags have been added: As shown below, you can click on Add Tags to create new tags for the campaign and add existing ones to it.

clikc to enlarge

  • Audit Logs: Click View to access a history of all the interactions your account admins have had with the campaign. Each action, including campaign creation, message edits and changes made to its settings is recorded here. (Detailed Read)

  • Conversion Event: Indicates the action users are expected to perform on your app after receiving the campaign, as specified at Step 4: Conversion Tracking, while creating it.

    • If you have not set up Conversion Tracking for the campaign or the journey it's attached to, then the same will be indicated by the status Disabled, here.
  • Conversion Deadline: Indicates the duration till which the campaigns' Conversion Event will be tracked after it has been delivered. The same can be specified at Step 4: Conversion Tracking while creating the campaign.

    • If you have not set up Conversion Tracking for the campaign or the journey it's attached to, then the same will be indicated by the status Disabled, here.
  • Control Group: Indicates whether or not a Control Group was set up to measure the campaign's effectiveness against organic user behavior at Step 4: Conversion Tracking, while creating it. The same is indicated by:

    • Enabled
    • Disabled
  • Frequency Capping: Indicates whether or not the campaign was sent with Frequency Capping enabled at Step 2: When while creating it. The same is indicated by:

    • Follow
    • Ignore
  • DND: Indicates whether or not the campaign was sent with DND Hours enabled at Step 2: When while creating it. The same is indicated by:

    • Follow
    • Ignore
  • Queueing: Indicates the duration for which the campaign will be queued for delayed delivery if we are unable to deliver it immediately to a user. It can be specified at Step 2: When while creating the campaign.

  • Push Notifications can get queued due to the following reasons:

    • The daily, weekly, monthly frequency cap on the number of messages a user can receive has been extinguished.
    • DND Hours are currently applicable to a few users as per their time zone.
    • The user's device is currently out of service network area.
  • If you choose to disable Queueing for the campaign while creating it, then the same will be indicated by the status - Disabled.

  • Throttling: Indicates the message rate limit specified for the campaign at Step 2: When. It enables you to control the number of messages sent to users per minute. (Detailed Read) It's status is indicated by:

    • Follow
    • Ignore
  • Send Winner: Indicates whether or not automated Variation testing was configured for the campaign while creating it, by setting up Send Winning Variation Automatically at Step 4: Conversion Tracking. The same is indicated by:

    • Enabled
    • Disabled
    • If Send Winner is enabled, then the following details will help you out with a quick recap of the rules of testing:
    • Size: Indicates the size of test audience with which all the Variations of the campaign will be tested with to determine a winning variant.
    • Time to Test: Indicates the duration for which all the Variations of the campaign will be tested for.
    • Win Criteria: Indicates the parameter (Clicks, Conversions) based on which a winning Variation will be determined.
  • Created By: Indicates the name of the user who created the campaign.

  • Created On: Indicates the date on which the user started creating the campaign.

  • Last Edited By: Indicates the name of the user who has made the most recent edit to the campaign.

  • Last Edited On: Indicates the date on which the most recent edit was made to the campaign.

Define the Scope of Analysis

You can customize the stats shown under Overview and Analyze by selecting a time frame and data view as per your analytical needs.

Here's how you can go about it:

Step 1: Select a Time Frame

By default, all stats under Overview and Analyze are shown from the campaign's Start Date till present day (if the campaign is running) or till upto 45 Days from the campaign's End Date (if the campaign has ended).

Click to enlarge

For example, in the above visual, stats for the campaign, Eyeliner: Friday Tips + Purchase are shown from it's Start Date, April 19, 2019, till the present day (at the time of documenting) April 30, 2019.

Using the date range filter, you can analyze the campaign's performance for a custom time frame, just like we choose to analyze the stats from April 19 to April 21.

📘

The following time periods can be selected here:

  • Today

  • Yesterday

  • Last 30 Days

  • Last 60 Days

  • Last 90 Days

  • Custom Dates (as selected by you)

Step 2: Select a Data View

As discussed under Campaign and Channel Performance Indicators, you can choose to analyze campaign events in two ways:

Funnel View: Helps you analyze the performance of the campaign, based on the campaign events (deliveries, clicks, failures) tracked for messages sent within the selected time frame.

Events View: Helps you analyze the performance of the campaign, based on the campaign events (deliveries, clicks, failures) tracked within the selected time frame, even if the messages were sent before the selected period.

By default, a Funnel View is shown for the stats shown across Overview and Analyze. You can select an Events View using the overflow menu highlighted below:

Click to enlarge

🚧

Please Note: Scope of analysis cannot be defined for 'List of Users'

List of Users includes campaign engagement details for all the users that have been a part of the campaign's target audience throughout its lifetime. For the same reason, it cannot be analyzed for a particular time frame or data view.

Now, let's show you how you can analyze a Push Campaign to gain valuable insights into its impact on user engagement, conversions, and revenue.

How to Analyze

🚧

Must Read

Please refer to Campaign and Channel Performance Indicators for a complete list of all the user-channel interactions tracked for Push campaigns.

Click to enlarge

As shown above, using the navigation bar, you can toggle between the following sections to slice-and-dice data and analyze multiple aspects of the campaign:

Now, let's dig into each section, starting with Overview.

Overview

The default view of this section helps you analyze the overall performance of the campaign from it's Start Date till present, split by Variations, against key performance indicators like Sent, Message Deliveries, Clicks, Conversions, Revenue, Failures and so on.

You can alter this by selecting a time frame and data view as per your analytical needs.

1. Performance Overview

This subsection presents a summary of the campaign against parameters like the number of messages Sent, Failed, Delivered, Queued and indicates its overall performance against key metrics like Unique Clicks, Unique Conversions, and Total Revenue.

Click to enlarge

Let's go over each card:

Sent

Indicates the total number of messages that have been sent to the campaign's target audience within the selected time frame, including all the messages that failed or got queued for delayed delivery.

  • Failures: Indicates the total number of messages that failed to get delivered within the selected time frame. As shown below, you can further analyze message failures by hovering over the field.

Click to enlarge

  • For example, in the visual above, we can see that maximum messages failed due to Uninstalls (534).

  • This implies that the users had already the uninstalled the app before the Push Notification was sent to them. The same was conveyed by FCM (if Android app) / APNS (if iOS app) as the reason for campaign failure. Hence, all these users will be marked unreachable through Push.

  • You can further analyze this behavior through the Uninstalls section of your dashboard.

🚧

Related Read: Why a Push Campaign can Fail

Complete list of all the reasons due to which a Push campaign can fail to get delivered.

Delivered

  • Delivery Rate: Calculated against the total number of messages Sent, it indicates the percentage of messages Delivered, within the selected time frame for the chosen data view.

  • The card also indicates the total number of messages Delivered within the selected time frame for the chosen data view.

  • Queued: Indicates the total number of messages that are currently queued for delayed delivery to a few users. As shown below, you can further analyze the reasons for queueing by hovering over the field.

    • Queued messages cannot be analyzed for a particular time frame or data view as it represents the current state of message delivery to the target audience.

Click to enlarge

  • For example, in the visual above, we can see that at present, maximum messages have been queued due to the account's Frequency Capping settings. This implies that any or all of the following scenarios have occurred:

    • The upper limit on the total number of messages a user can receive within a Day/Week/Month has been reached for a few users of the target audience.
    • The time gap (maintained between the delivery of subsequent messages) is currently being observed for a few users of the target audience.
  • If a delivery window doesn't open up for a user before the queueing duration ends, then the message will fail to get delivered to them. (Detailed read on how delivery time is determined for queued messages)

🚧

Related Read: Why a Push Campaign can get Queued

Complete list of all the reasons due to which a Push campaign can get queued for delayed delivery to a user.

Unique Clicks

  • Unique Click Rate: Calculated against the total number of messages Delivered, it indicates the share of users that clicked on the Push Notification within the selected time frame, for the chosen data view.

  • The card also indicates the number of users that clicked on the Push Notification , helping you gauge the campaign's true impact on user engagement.

  • The impact of Push Amplification on boosting the ROI for your Android app is indicated in green here. (How It works)

Unique Conversions

  • Unique Conversion Rate: Calculated against the total number of messages Delivered, it indicates the share of users that have performed the campaign's Conversion Event at least once within the selected time frame, as per the chosen data view.

  • The impact of Push Amplification on boosting the ROI for your Android app is indicated in green here. (How It works)

  • The card also indicates the number of users converted through the Push Notification, helping you draw a fair comparison between the number of users who clicked on the campaigns, users who converted and the total Revenue.

  • For example, a quick look at the campaign's Performance Overview below reveals an Average Revenue per User (ARPU) of $12.K (that's a great ROI for a one-time Push campaign!)

Click to enlarge

🚧

Please Note

Conversions will not be tracked for your campaign/journey if you have not specified a Conversion Event at Step 4: Conversion Tracking, while creating it.

Detailed Read on how Conversion Tracking works.

  • Compare Unique Conversions with Control Group: If you have enabled Control Group while creating the campaign, then as highlighted below, you will be able to draw a scientific comparison between the campaign's conversion rate and the organic conversion rate - helping you understand its true impact.

Click to enlarge

👍

Analyzing Push Campaign's Conversion Rate against Control Group

In the above visual, we've analyzed a One-time Push campaign nudging users to purchase plushies. Here's what its Performance Overview reveals:

  • Campaign's Unique Conversions: 1.15%

  • Unique CG Conversions: 1.13% (Control Group)

  • Conversion Uplift: 1.77%

These stats imply that the campaign played a minimal role in motivating users to purchase as most of them were organically inclined to buy the plushies.

However, had we not tested the campaign with a Control Group, the e-commerce app would have wrongfully attributed all the conversions to the campaign!

Thus, it's always advisable to test your campaigns (and journeys) with a Control Group before sending it to the entire target audience to ensure that it helps amplify the desired user behavior, and not hinder it. Here's how you can automate Variation testing with a Control Group.

Now, let's walk you through the metrics shown here:

  • Unique CG Conversions: Calculated against the total number of users included in the Control Group (CG), it indicates the share of users that performed the campaign's Conversion Event at least once within the selected time frame, even though they didn't receive the campaign.

  • Conversion Uplift: Indicates the percentage increase or decrease in the Unique Conversion rate achieved by the campaign, calculated against the Unique Conversion rate of the Control Group.

    • It's shown beside the campaign's Unique Conversion rate, in Green for positive uplift, in Red for negative uplift, and in Grey for no uplift.

👍

Fact Check

Conversion Uplift equals [(Unique Conversion Rate of Campaign/ Unique Conversion Rate of CG) x 100] - 100

Revenue

Indicates the amount of Revenue tracked for the Total Conversions that have occurred within the selected time frame, as per the chosen data view.

  • Revenue numbers are always shown in the currency selected by you while setting up Revenue Mapping

  • The card also indicates the impact of Push Amplification on boosting the ROI for your Android app. (How It works)

🚧

Please Note

Revenue will not be tracked for your campaign if:

  • You have not set up Revenue Mapping for your account

  • You have not specified a Conversion Event while creating the campaign (or its journey)

  • You have not mapped the specified Conversion Event as a Revenue Event in your account

Detailed Read on how Revenue tracking works and how Revenue is attributed to a campaign.

Impact of Push Amplification on Unique Clicks, Conversions & Revenue

Our Android SDK v3.12.0 makes it possible for you to amplify push notification deliveries, helping you achieve up to 30% higher Clicks, Conversions and Revenue!

Click to enlarge

As shown above, you can analyse the real-time impact of WebEngage's Push Amplification Engine on your campaign's through the stats indicated in green. All you need to do is upgrade/integrate your Android apps with the latest SDK version and we'll take care of the rest :)

👍

Understanding How Amplification Stats are Calculated

With reference to the Unique Clicks indicated in the above visual:

  • Unique Click Rate = 0.45%

  • Amplification Rate = 18%

    • Unique Android users that clicked on the notification = 9,677

    • 18% of 9,677 users = 1,693 - indicates users that clicked on notification after receiving it through the amplification engine

    • This means that the amplification rate is a subset of the total unique click rate. It's calculated specifically for Android users that are using the app version that contains the WebEngage Android SDK v3.12.0.

Amplification stats are calculated in a similar manner for Unique Conversions and Revenue. You can hover over the corresponding tooltips to access these details for your campaign.

2. Variation Comparison

Click to enlarge

The most insightful section of Campaign Overview, Variation Comparison helps you analyze the impact of each message Variation against key metrics like their Layout (each Variation can have a different template), Dismisses, Clicks, Conversions, Revenue and so on, for the chosen time frame and data view.

However, depending on the settings specified while creating the campaign, the stats shown here may vary. Let's get you acquainted with the impact of each setting:

🚧

Skip to: How to customize and analyze this section as per your strategic needs.

2.1. When Variation Distribution is Set Manually

If you have manually specified the audience size for all the Variations at Step 4: Conversion Tracking, then using this section, you will be able to identify a winning Variation by analyzing the performance of each. The following aspects of the test will be highlighted here:

  • Percentage of the audience targeted by each Variation (calculated against the total target audience)
  • The performance of each Variation against all the performance indicators tracked for Push campaigns

Let's demonstrate a short use-case to show you how it works:

👍

Analyzing Manual Variation Distribution Test Results

Let's take the example of an OTA (online travel agency). In the visual below, we have analyzed the performance of a one-time Push campaign offering a special deal to users located in New York City on flights to Vietnam.

  • The campaign's target audience was defined as the segment, NYC_USA. (includes all users who are currently located there)

  • The Conversion Event of the campaign was set as the Custom Event, Flight-Ticket-Booked.

They created two versions of the campaign's message and tested both with the entire target audience (12,472 users). Thus, Variation A and B were equally distributed amongst the users.

End Goal of the Test: Identify a message copy that resonates with the audience and replicate it for all the Push Notifications sent to the segment NYC_USA, in the future.

Here's what their analysis looked like...

Click to enlarge

Key takeaways:

  • Variation B emerges as the winner as it resulted in maximum Conversions (0.12%), contributing $1700 in Revenue, while no Unique Conversions were recorded for Variation A.

  • Interestingly, Variation A achieved a higher Unique Click Rate (2.31%) than Variation B (0.85%).

    • This implies that even though a lesser number of users engaged with Variation B, most of them were motivated to purchase the flight tickets to Vietnam after receiving the Push Notification.
  • Thus, the message copy of Variation B proved to be the most effective for the target segment, NYC_USA.

🚧

Skip to: How to customize and analyze this section as per your strategic needs.

2.2. When Send Winning Variation is Enabled

If you have enabled Send Winning Variation Automatically at Step 4: Conversion Tracking while creating the campaign, then you will be able to analyze the test results through this section. As shown below, the following aspects of the test will be highlighted here:

  • The winning Variation (which was automatically sent as the only campaign to the entire target audience)
  • The performance of each Variation that was tested with the test audience

Click to enlarge

For example, in the above visual, we can see that Variation C was determined as the winner as it achieved the highest Unique Conversion Rate. This is because Conversions was selected as the Win Criteria of the test while setting it up. Similarly, you too can select a custom Win Criteria for testing the Variations, as per your business goals.

🚧

Skip to: How to customize and analyze this section as per your strategic needs.

2.3. When Control Group is Enabled

You can compare each Variation's conversion rate with the Control Group's organic conversion rate to determine it's true impact on driving conversions. This can be done by enabling Control Group at Step 4: Conversion Tracking while creating the campaign or the journey (the campaign is attached to).

As shown below, the extent to which a Variation influences users to perform the Conversion Event can be measured through its Conversion Uplift, indicated alongside Total Conversions and Unique Conversions, in Green, for positive uplift, in Red, for negative uplift and in Grey for no uplift.

Click to enlarge

👍

Analyzing Variations with Control Group through Conversion Uplift

In the above visual, we have compared the performance of 2 Variations of a Push Notification and a Control Group against Unique Conversions, Total Conversions, and Revenue.

  • Both Variations and the Control Group were equally divided amongst the campaign's test audience.

  • Since Control Group was enabled for the campaign, Unique Conversions was the sole Win Criteria used to identify a winner version of the message.

On comparing the Variations with the Control Group we learn that:

  • User who did not receive the campaign, as a part of the Control Group, converted at a rate of 0.17% (Total Conversions), contributing $1.2K in Revenue.

  • Variation A influenced the maximum number of users to convert, achieving a Conversion Uplift of 788% and contributing a whopping $18K in Revenue!

    • As a result, Variation A was automatically sent as the only campaign to the entire target audience. (How it works)
  • Variation B achieved a Unique Conversion rate of 0.05%, representing 80% lower conversions than the Control Group.

    • This implies that Variation B actually had a negative impact by deterring users from making a purchase.

🚧

Related Reads

  1. Why you should (almost) always use a Control Group to measure the effectiveness of your campaign and how it works.

  2. How you can test Variations with a smaller test audience and automatically send the winner to the entire target audience of the campaign!

Now, let's show you how you can slice-and-dice this section to gain relevant insights:

Step 1: Customize Table

Click to enlarge

By adding values to the query bar highlighted above, you can choose to analyze Variations split by the Device, OS or vice-versa, against all the performance indicators tracked for Push campaigns.

In doing so, you will be shown a secondary view of each Variation, split by the selected option, helping you analyze user-campaign interactions in further detail, as shown below.

Click to enlarge

📘

The following options can be selected under both the fields:

Variations: Helps you compare the performance of all the Variations for the selected time frame and data view.

Technology

  • Device Type (Mobile, Desktop, Tablet)

  • OS (iOS, Android)

Step 2: Select Performance Indicators

Click to enlarge

As shown above, using the filter icon placed on the top right, you can select/deselect metrics to customize the table as per your analytical needs. For example, in the above visual, we have analyzed each campaign Variation against Unique Clicks, Total Clicks, Unique Conversions, and Total Conversions.

It reveals that users that received Variation B, performed the campaign's Conversion Event multiple times (indicated by the fact that Unique Conversions equal 22, while Total Conversions equal 53).

📘

The following metrics can be selected here:

Overall (Sent): Helps you analyze the total number of messages that were sent to the target audience within the selected time frame, including all the messages that may have failed or may still be queued.

Overall (Delivered): Helps you analyze the total number of messages that were successfully delivered to the target audience within the selected time frame.

Totals (Total Dismisses, Total Clicks, Total Conversions, and Total Click-through Conversions)

Uniques: (Unique Dismisses, Unique Clicks, Unique Conversions, Unique Click-through Conversions)

(Detailed read on what each performance indicator denotes)

Step 3: Preview Variation's Message & Layout

Click to enlarge

As shown above, using the View icon placed next to each Variation's header, you can preview the Push Notification for both, iOS and Android users. This is a great way to understand how varied copies and layouts impact user-campaign interactions.

For example, in the above visual, on comparing the layouts we learn that Variation B (that recorded the highest conversions) was a Banner style notification, while Variation A was a simple Text notification.

  • This indicates that adding a visual to the notification could have played a major role in motivating users to convert.

Step 4: Select a Data Format

Click to enlarge

While the default view shows you percentage values (%) for all the campaign performance indicators, using the overflow menu highlighted above, you can choose to analyze these stats as numbers (by selecting #).

  • Percentage value of messages that are Queued or have Failed (Failures) is calculated against the total number of messages Sent.

  • Percentage value of all the performance indicators like Dismisses, Clicks, Conversions is calculated against the total number of messages Delivered.

3. Carousel / Ratings Review

An additional section is added to Push Campaign Overview if you have selected Carousel or Rating as a layout for a campaign Variation, helping you analyze the impact of each layout on user engagement.

3.1. Carousel Images

Click to enlarge

If you select Carousel as the layout at Step 3: Message, while creating a Variation, then, as shown above, you will be able to analyze the impact of each image of the carousel on user engagement against Impressions and Clicks. Let's quickly walk you through all the columns of this section:

  • Type: Indicates the type of creative (image, GIF) added to the carousel while creating the campaign.

  • Unique Impressions: Indicates the number of users that viewed a carousel image on their device (mobile, tablet), within the selected time frame.

  • Total Impressions: Indicates the total number of times users viewed a carousel image on their device (mobile, tablet), within the selected time frame.

  • Unique Clicks: Indicates the number of users that clicked on the carousel image within the selected time frame.

  • Total Clicks: Indicates the total number of times users that clicked on the carousel image within the selected time frame.

3.2. Ratings

Click to enlarge

If you select Rating as the layout at Step 3: Message, while creating the Push campaign, then you will be able to analyze the responses collected for all the users against a scale of 1 - 5 stars, as shown above.

4. Engagement Trends

Click to enlarge

This section helps you analyze day-wise user engagement trends for each Variation against the following performance indicators, as per the chosen time frame and data view:

  • Sent
  • Total Failures
  • Total Queued
  • Unique Dismisses
  • Unique Clicks
  • Unique Conversions

Let's analyze a use-case to get you acquainted with the workings of this section:

🚧

Skip to: How you can customize the stats shown here as per your analytical needs.

👍

Use-case: Analyzing Engagement Trends for a Triggered Campaign

For example, in the visual below, we have analyzed the engagement trends of a Triggered Push campaign for the Last 7 days (May 1 - 7) as a Funnel View.

Click to enlarge

It reveals that for the selected time frame:

  • An average of 17K Push Notifications were Sent to users each day.

  • An average of 5K Push Notifications were Dismissed by users each day.

  • An average of 700 Push Notifications Failed to get delivered each day.

  • Interestingly, After May 3, the number of Unique Conversions increasingly surpassed the number of Unique Clicks. This could indicate any of the following possibilities:

    • Theory 1: Users who received the Push campaign between May 4 - 7 had a high intent to convert and the campaign played a minimal role in influencing their behavior. (their conversion was attributed to the campaign because they performed its Conversion Event after receiving it - detailed read)

    • Theory 2: A few users who received the Push campaign between May 1 - 3, chose to perform the Conversion Event after a few days. Hence, even though they may have clicked on the notification on the day same, they transacted between May 4 - 7, causing the high difference between Unique Conversions and Unique Clicks. (detailed read on how the campaign's Conversion Deadline makes this possible)

  • Highest number of Unique Conversions occurred on May 7 (2,335).

👍

To gain a better perspective on these stats, let's:

1. Widen the time frame of analysis to April 14 - May 7 to understand the conversion trends

2. Check it's Conversion Deadline

Here's what we found:

1. The campaign's Conversion Deadline is 7 Days.

Click to enlarge

  • This implies that if a user performs the campaign's Conversion Event within 7 days of receiving the Push Notification, then it will be counted towards the campaign's Conversions (Unique & Total).
  • For example, if a user received the campaign on May 1 and converted on May 7, then it will be counted towards the campaign's Unique Conversions for May 7.

2. Further, the number of Unique Conversions surpass the number of Unique Clicks throughout the selected time frame.

Click to enlarge

  • This indicates that Theory 2 holds true! - A high number of users did not perform the campaign's Conversion Event immediately after receiving the Push Notification. They took a few days to decide before transacting.

We hope this has equipped you with a robust understanding of how you can leverage Engagement Trends to gain valuable insights into user behavior and the campaign's overall performance.

Now, let's show you how you can customize this section as per your analytical needs:

Step 1: Select a Variation

Click to enlarge

By default, engagement trends are shown only for Variation A (the first version of the campaign). Using the dropdown placed on the top left, you can choose to analyze trends for each Variation against - Sent, Failures, Queued, Unique Dismisses, Unique Clicks and Unique Conversions.

  • If you have enabled Control Group for the campaign, then you can choose to analyze its Unique Conversion trends by selecting 'Control Group' from the dropdown highlighted above.

Step 2: Select a Format of Visualization

Click to enlarge

While the default format of visualization is a Line Graph, as shown above, using the overflow menu placed on the top right, you can change this to a Bar Graph or a Table, as per your analytical needs.

Analyze

While Overview presents a comprehensive view of the campaign's performance, this section helps you slice-and-dice data in multiple ways to gain maximum insights into user-campaign interactions.

First Impression

The default view of this section shows you a Day-wise trend for all the messages that have been Sent from the campaign's Start Date till the present date or its End Date (whichever occurs first).

Click to enlarge

Using the query bar highlighted above, you can choose to analyze the campaign's performance against a specific performance indicator, split by 2 parameters for a custom time frame and data view.

Let's Analyze a Use-case to Show You How it Works

🚧

Skip to: How you can slice-and-dice data to gain valuable insights into user-campaign interactions.

👍

Use-case: Analyzing a Triggered Push Campaign Aimed at Curbing Flight Search Abandonment

Let's take the example of a flight booking app. Recently, they noticed an increase in flight search abandonment - users viewed the flight details but left without booking it.

Hence, they decided to engage these users with a highly-personalized Push Notification.

  • They created a triggered campaign targeted at all users who have not booked a flight even after 30 minutes of viewing flight details.

  • The end goal was to motivate these users to complete their transaction by offering them benefits like free meals and priority check-in.

  • They tested 2 Variations of the Push campaign to understand their users' motivations better.

    • Variation A offered a free in-flight meal.
    • Variation B offered priority check-in.
  • Variation B was determined the winning Variation after 1 week of testing and was automatically sent as the only campaign to all user who performed the trigger event hereafter. (Here's how it works)

Now, let's analyze Variation B of the campaign to understand its performance and users, better.

👍

Analysis 1 (in continuation of the use-case)

Identifying the top Countries contributing to Total Conversions and the Days on which maximum users converted, for the Last 2 weeks.

Click to analyze

As shown above, here's how they went about their analysis:

Step 1: Select a custom time frame of last 2 weeks using the date range filter. (April 24 - May 7)

Step 2: Select Variation B from the first dropdown placed on the top left (since it's the winning variation and is being sent to all users as the only campaign).

Step 3: Select Total Conversions under the field - Show.

Step 4: Retain the default option of Days under the field - Over.

Step 5: Select Country under the field - Split By.

It reveals that:

  • The Conversion Event of the campaign was performed 96,907 times by the users over the last 2 weeks!
  • Maximum converted users are located in Saudi Arabia.
  • Users performed the Conversion Event the maximum number of times on May 3 (4,803), Friday followed by April 30 (4,636), Tuesday and May 6 (4,438), Monday.

👍

Analysis 2 (in continuation of Analysis 1 of the use-case)

Now let's analyze the Type of Users that contributed to Total Conversions over the last 2 weeks, from the top Countries.

Click to enlarge

As shown above, here's how they went about their analysis:

Step 1: Select Country under the field - Over. (and remove it from Split By, as it becomes easier to visualize the data)

Step 2: Select User Type under the field - Split By and click enter!

It reveals that:

  • An (almost) equal number of Known (25, 760) and Unknown Users (23,875) from Saudi Arabia (top Country) performed the campaign's Conversion Event.

  • This suggests that even though most of the (Unknown) converted users did not have a high intent to purchase initially (as they had not signed into the app), they were highly motivated to book the tickets owing to Variation B's priority check-in offer.

👍

Analysis 3 (in continuation of the Analysis 2 of the use-case)

Now that we have a fair idea of the Type of Users and the top Country contributing to Total Conversions, let's find out the top Cities (of Saudi Arabia) where the converted users are located and their OS preference.

This will help us:

  • Identify their nearest airport, allowing us to contextually re-engage these users with offers and updates on flight prices.

  • Identify the most preferred OS (iOS/Android) so that we can customize the layout and message for each.

Click to enlarge

As shown above, here's how they went about their analysis:

Step 1: Select City under the field - Over (and remove the values selected for the previous analysis).

Step 2: Select OS under the field - Split By and click enter!

It reveals that:

  • Highest number of users performed the Conversion Event from Riyadh, followed by Jeddah.

    • This implies that the flight booking app could experiment with engaging users located in these cities with offers on flights taking off from the respective airports to drive repeat purchases.
  • However, the device preference of users located in both the cities reveals an (almost) equal split between Android and iOS.

    • This implies that the campaigns will need to be targeted at both, Android and iOS.

We hope this has equipped you with a broad understanding of how you can gain in-depth insights into your campaign, it's users and their preferences through Analyze. Now, let's walk you through all the steps of analysis:

Step 1: Select a Variation/Control Group

Click to enlarge

If you have created multiple Variations for a Push campaign, or have enabled Control Group for it, then using the dropdown highlighted above, you can choose to selectively analyze the performance of each.

However, if your campaign consists of just one Variation, then this dropdown gets hidden.

📘

Any one of the following options can be selected here:

  • Overall: Helps you analyze the combined performance of all the Variations against each metric for the selected time frame.

    • For example, if you choose to analyze Revenue, Overall, then we will sum up the Revenue tracked for each Variation to help you analyze their collective impact.
  • Variation A/B/C/D/E: Helps you analyze the performance of a Variation, against a selected performance indicator, as per the chosen time frame and data view.

  • Control Group: If you have enabled Control Group at Step 4: Conversion Tracking while creating the campaign or its journey, then selecting this option will help you its Unique Conversions/ Total Conversions.

🚧

Related Read

Please refer to Campaign Variations and Testing for a quick read on how you can test multiple Variations of a campaign with a smaller audience and automatically send the winner to the entire target audience!

Step 2: Select a Performance Indicator

Click to enlarge

You can choose to analyze the overall performance of the campaign or each Variation against a performance indicator or campaign event (like Dismisses, Clicks, Conversions, Revenue) by selecting an option under the field - Show. As highlighted above, the total value of the selected performance indicator is also indicated under the query bar.

📘

Any one of the following campaign events can be selected here:

(Detailed read on what each performance indicator denotes)

Step 3: Select the Dimension(s) for Analysis

Click to enlarge

As demonstrated for the use-case discussed above, by selecting an option under the fields - Over and Split By, you can choose to analyze the selected performance indicator (campaign event) against two dimensions (system attributes). This is a great way to dig into multiple aspects of the campaign like:

  • The top Locations and Devices used to perform the selected campaign event

  • The Types of Users who performed the campaign event and their OS preferences

  • The trend of occurrence of selected campaign event against Hours of the Day, Days or the Week or Months of the Year and so on.

📘

The following options can be selected under both the fields:

  • Time (Days, Weeks, Months)

  • Time Block (Hours of Day, Days of Week, Months of Year)

  • User Type (What it means)

  • Location (Country, City)

  • Technology (Device Type, OS Name, Device Manufacturer, Device Model, App Version, App ID, Platform)

List of Users

🚧

Must Read

Please ensure that you have a good understanding of the various types of Users and related concepts before proceeding. Doing so will help you understand the workings of this section, better.

This section has been specially designed to help you track down campaign engagement details for each user that targeted by the campaign, over its entire lifetime. Thus, analyzing List of Users comes in handy when digging into user-campaign engagement details for Triggered, Recurring or Journey, and Relay Push campaigns (as they are on-going cycles of communication).

First Impression

By default, List of Users shown you a Summary of user-campaign interactions for All Users who have been included in the campaign's target audience, from its Start Date till the present date or its End Date (whichever occurs first).

Click to enlarge

🚧

Please Note

All the stats and engagement details are gleaned over the entire lifetime of the campaign. For the same reason, List of Users cannot be analyzed for a particular time frame or data view.

Now, let's get you acquainted with its features:

Customize List

Here's how you can customize the List of Users as per your analytical needs:

Step 1: Select User Type

As discussed under Users, at WebEngage, users can be two types, Known (identified users) or Unknown (anonymous users).

Click to enlarge

As shown above, using the first dropdown placed on the top left, you can select a user type to analyze campaign engagement as per your needs.

📘

The following options can be selected here:

Step 2: Select a Parameter

Click to enlarge

As shown above, using the second dropdown placed on the top left, you can customize the List of Users by selecting a parameter. For example, in the above visual, we have customized the list of Known Users by Conversions.

📘

The following options can be selected here:

1. Summary: Shows you an overall summary of all the users included within the target audience, against the following columns:

  • Latest Message Status: Indicates the status of the message sent to the user with a date-time stamp as any one of the following:

    • Sent on (date-time)
    • Failed on (date-time)
    • Queued on (date-time)
    • Delivered on (date-time)
    • Dismissed on (date-time)
    • Clicked on (date-time)
    • Converted on (date-time)
    • Added to Control Group on (date-time): Shown for all users who were not sent the campaign as they were added to the Control Group, as specified by you while creating the campaign.
  • Sent Date: Indicates the date-time on which the message was sent to the user.

  • Delivered Date: Indicates the date-time on which the message was delivered to the user.

  • Total Clicks: Indicates the total number of times the user has clicked on the Push Notification, from the date of delivery, till present.

  • Latest Click Date: Indicates the date-time on which the latest click occurred.

2. Clicks: Helps you analyze all the users have clicked on the campaign over its entire lifetime, against the following columns:

  • Total Clicks

  • Latest Click Date

3. Conversions: Helps you analyze all the users that have performed the campaign's Conversion Event, over its entire lifetime, against the following columns:

  • Total Conversions: Indicates the total number of times the user performed the Conversion Event within the specified deadline.

  • Latest Conversion Date: Indicates the date-time on which the latest conversion occurred.

  • Latest Conversion Type: Indicates the campaign event (Clicks, Sent) to which the conversion was attributed to as Click-through Conversion or Send-through Conversion.

  • Revenue: Indicates the total amount of Revenue contributed by the user (calculated against Total Conversions).

4. Failures: Helps you analyze all the users for whom the campaign failed, against the following columns:

  • Failure Date: Indicates the date-time on which the message failed to get delivered to the user.

  • Failure Type: Indicates the reason due to which the Push Notification failed to get delivered. (complete list and what each reason indicates)

5. Queued: Helps you analyze all the users for whom the campaign got queued for delayed delivery over its entire lifetime, against the column:

(Detailed read on what each performance indicator denotes)

Download List

Click to enlarge

As shown above, using the Download icon placed on the top right of the table, you can choose to download a user-wise engagement report for all the users targeted by the campaign, over its entire lifetime. Here's how you can go about it:

Step 1: Click the Download icon.

  • In doing so, you will be prompted with a pop-up to re-confirm your decision.

Step 2: Click the Download button on the pop-up window to proceed.

  • An email containing a link to the downloadable file will be sent to your registered email address shortly.

Search for a User

Click to enlarge

As shown above, using the search box placed on the top right, you can dig into the engagement status of a particular user by searching for their User ID.

Access User Profiles

Click to enlarge

As shown above, you can access the profile of a user targeted by the campaign by clicking their (hyperlinked) User ID. Doing so will help you analyze the user's channel preferences, personal details device preferences, latest interactions with your campaigns, app, website, the technical aspects of each interaction much more!

🚧

Must Read

Please refer to Analyzing User Profiles for a robust understanding of how you can dig into the preferences and behavior of individual users.

We hope this has equipped you with a robust understanding of how you can mine valuable insights into a campaign to understand its true impact on user engagement and revenue. Please feel free to drop in a few lines at support(AT)webengage.com in case you have any related queries or feedback. We're always just an email away!

Updated 6 months ago

Analyzing Push Campaigns


A step-by-step guide on how you can analyze the performance of a push campaign in your dashboard

Suggested Edits are limited on API Reference Pages

You can only suggest edits to Markdown body content, but not to the API spec.