Analyzing SMS Campaigns

Once you launch an SMS campaign through the channel or a journey, its impact on user engagement and revenue can be analyzed in detail through Campaign Overview.

How to Access

Let's quickly show you how you can access campaign analysis through the various sections of your dashboard:

Access Campaign Overview through List of Campaigns

A list of all the campaigns created through the channel can be found under the List of Campaigns or the central hub of SMS. As shown below, you can click on the (hyperlinked) Campaign’s Name to analyze a campaign in further detail.

Access Campaign Overview Through Journey Overview

A list of all the campaigns attached to a journey can be found under Campaigns in the journey's Overview. As shown below, you can click on the (hyperlinked) Campaign Name to analyze the SMS campaign in further detail.

Before we dive into campaign analysis, let's get you acquainted with the top panel.

Understanding Top Panel

As highlighted above, the top panel provides a quick snapshot of the campaign with key details like:

• Campaign Name: As defined at Step 1: Audience, while creating the campaign.

• (Target) Audience: Indicates the segment targeted by the campaign, as defined at Step 1: Audience, while creating the campaign.

  • You can click the (hyperlinked) segment's name to analyze its users and rules of segmentation.

• (Campaign) Type: As selected when you initiate camaign creation in your dashboard. SMS campaigns can be of the following types:

  • One-time

  • Triggered

  • Recurring

  • Transactional

  • Journey (created only through the Journey interface)

  • Relay (can be created only through the Relay interface)

  • As shown below, all journey campaigns can be traced back to the journey they're attached to by clicking on the (hyperlinked) campaign type, Journey. Doing so will direct you to the journey's Live View.

• (Campaign) Status: Indicates the current status of the campaign, as per the settings specified at Step 2: When, while creating it.

  • If you save a campaign without launching it, then it will be saved as a draft. The same is reflected by the status, Draft.

Now, let's get you acquainted with its features:

Modify Campaign

As shown below, using the icons placed next to the Campaign's Name, you can choose to Edit, Delete, Duplicate, Pause/Resume and Tag the campaign, as per your marketing needs.

• Given the short lifespan of one-time campaigns, Pause has been disabled for such campaigns.
• Further, all modification icons are hidden for journey campaigns, since they can be modified only through the journey they have been created through.
• Once you Pause a Running campaign, you can choose to Resume it's delivery to the target audience by clicking the Resume icon, as highlighted above.

🚧

Related Read: Modifying SMS Campaigns

A step-by-step guide on how you can modify SMS campaigns.

Download a Report

🚧

Please Note

Reports can be downloaded only by Admins who have access to Account Management. Please get in touch with the account owner if you are unable to do so.

You can download a copy of campaign stats for offline analysis, upload it to another platform, and so on. Here's how you can go about it:

Step 1: Click the Download icon

As shown above, in doing so, you will be prompted to specify the kind of data you want to analyse.

Step 2: Specify the time frame for which the stats should be shown (Data to Include)

As shown above, you can specify the time frame as:

• Campaign's Start Date till Present Date
• Custom Dates

🚧

Detailed Read

• How Data to Include works

• Understanding Report Stats

Step 3: Select Data Breakdown

In this dropdown you can choose from 4 options i.e. Day-wise, Week-wise, Month-wise, and 30 days rolling window. Each of these have been explained below.

• Day-wise: When you choose the day-wise option the report that will be sent to you will have an aggregate of each day i.e. each row will consist of each campaign or variations day wise statistics.
  For example, if you want to download a report for the last 7 days, by choosing the day wise data breakdown option, you will receive a report with 7 rows each containing data for each of the days.
• Week-wise: When you choose this option the report will consist of the aggregate of each week, i.e. Every row will contain the week-wise statistics for each campaign or variation, with each week defined from Monday to Sunday.
  For instance, if you download a report for the past 1 month on the 30 Nov ’23 , and set the data breakdown option to week-wise, then you will receive a report that consists of 5 rows of aggregated data for each week starting from Monday to Sunday. Whereas, if a month starts during the middle of the week, for example the 1st of the month is on a Thursday, then the row for the first week will be from Thursday to Sunday.
• Month-wise: On choosing this option your statistics will be aggregated for each month i.e. each row will consist of month wise statistics. (calendar month will be considered)
  If you wish to download a report spanning from January 1st to November 30th and select the month-wise data breakdown option, you will receive 11 rows. Each row will contain aggregated data for each respective month, providing a comprehensive breakdown of information (based on calendar month).
• 30 day (Rolling window): On choosing this option, your statistics will be aggregated based on every 30 days irrespective of the calendar month, i.e. Every row will include the rolling window statistics for each campaign or variation for up to 30 days.
  For example, if you choose a date range from 15th June to 14th August, two rows will be generated i.e. 15th June to 14th July and 15th July to 14th August.

Step 4: Specify Level of Data Granularity

You can create up to 5 versions of a campaign and test each to determine the copy that resonates with your audience. Thus, while downloading a report, you can choose to analyze consolidated campaign stats OR a variation-wise breakdown.

As shown above:

• Select Campaign Level Granularity to view the average performance of all campaign variations.

• Select Variation Level Granularity to view the individual performance of each variation and identify the top-performing version.

Step 5: Click Download!

Schedule a Report

In our quest to make campaign reporting a seamless experience, we built Scheduled Reports - customizable and automated reports delivered straight to your inbox!

Using the Reporting icon, you can set up a periodic report to have all the stats shown under Overview, emailed to multiple team members.

Here's how you can go about it:

Step 1: Click on the Reporting icon

• As shown above, in doing so you will be prompted with a pop-up allowing you to customize the report's settings.
• The report's name will be the same as the campaign's name. This will be included in:
  • The emailed report's subject line
  • under the Scheduled Reports section of your dashboard

Step 2: Define the Frequency of report delivery

You can choose to have a report delivered on a Daily, Weekly or Monthly basis, as per your needs.

Step 3: Specify the time of delivery (When)

You can choose to have the report delivered at a specific time of the day. The same can be defined in an AM/PM format against the field, When.

Step 4: Specify the time frame for which the stats should be shown (Data to Include)

Too many data points always lead to chaotic insights. This is why we have made it possible for you to specify the time frame (number of days) for which you'd like to receive campaign stats.

As shown above, you can select a time frame between Last 1 Day to Last 30 Days _against _Data to Include.

🚧

Detailed Read

• How Data to Include works

• How a combination of Data to Include & Frequency determine the date of stats included in the report

• Understanding Report Stats

Step 5: Select option for Data Breakdown

In this dropdown you can choose from 4 options i.e. Day-wise, Week-wise, Month-wise, and 30 days rolling window. Each of these have been explained below.

• Day-wise: When you choose the day-wise option the report that will be sent to you will have an aggregate of each day i.e. each row will consist of each campaign or variations day wise statistics.
• Week-wise: When you choose this option the report will consist of the aggregate of each week, i.e. Every row will contain the week-wise statistics for each campaign or variation, with each week defined from Monday to Sunday.
• Month-wise: On choosing this option your statistics will be aggregated for each month i.e. each row will consist of month wise statistics. (calendar month will be considered).
• 30 day (Rolling window): On choosing this option, your statistics will be aggregated based on every 30 days irrespective of the calendar month, i.e. Every row will include the rolling window statistics for each campaign or variation for up to 30 days. For example, if you choose a date range from June 15th to August 14th, two rows will be generated i.e. June 15th to July 14th and July 15th to August 14th.

Click here to know more about the use cases of each.

Step 6: Select Level of Data Granularity

At WebEngage, you can create up to 5 versions of a campaign and test each one with a small test segment to determine the copy that resonates with your audience. Thus, while scheduling a report, you can choose to analyse consolidated campaign stats OR a variation-wise breakdown.

As shown below:

• Select Campaign Level Granularity to view the average performance of all campaign variations.

• Select Variation Level Granularity to view the individual performance of each variation and identify the top-performing version.

Step 7: Add team members as Subscribers

You can choose from a list of all the team members who have access to your WebEngage dashboard. Doing so will ensure that all the added users receive the campaign performance report, as per its settings.

However, if a team member is not a part of your WebEngage account, then you will not be able to add them as a Subscriber.

Step 8: Click the Schedule button

• A notification will pop up on the bottom right corner of the page, confirming that the report has been scheduled. (Understanding Report Stats)

• You can modify or delete the report anytime you like through the Scheduled Reports section, nested under Settings in your account.

🚧

Please Note

• Reports can be configured by only those Admins that have access to Account Management. Please get in touch with the account owner if you're unable to do so.

• You can schedule additional reports for campaigns sent through SMS and all the channels combined through Settings > Scheduled Reports in your dashboard.

See Campaign Details

As demonstrated below, clicking Show Details reveals a thorough recap of all the rules and settings specified while creating the campaign. This section can be minimized by clicking Hide Details.

Here's a list of all the details shown here:

• Campaign ID: A unique campaign ID is generated by us each time you create a campaign.

• SMS Type: As selected by you at Step 1: Audience while creating the campaign, SMS campaigns can be of the following types:

  • Promotional
  • Transactional

• Start Date / Send Date: Indicates the date-time on which the campaign was launched or will be launched (if the campaign is still scheduled), as specified at Step 2: When, while creating it.

• End Date: Indicates the date-time on which the campaign will end or ceased getting delivered.

  • Applicable only to triggered and recurring campaigns, An End Date can be specified at Step 2: When while creating it.

• Time zone: Applicable only to one-time campaigns, it indicates whether the campaign was sent in the user's timezone, or in your project's timezone. The same can be specified at Step 2: When while creating the campaign.

• Tags: Tags is a handy feature that helps you categorize your campaigns as per their purpose, target audience, frequency or any other parameter that makes them easier to search for.

  • If a campaign is tagged: All the tags added to the campaign will be shown here. You can click on a hyperlinked tag to add more tags to the campaign or remove existing ones.
  • If no tags have been added: As shown below, you can click on Add Tags to create new tags for the campaign and add existing ones to it.

• Audit Logs: Click View to access a history of all the interactions your account admins have had with the campaign. Each action, including campaign creation, message edits and changes made to its settings is recorded here. (Detailed Read)

• Conversion Event: Indicates the action users are expected to perform on your app after receiving the campaign, as specified at Step 4: Conversion Tracking, while creating it.

  • If you have not set up Conversion Tracking for the campaign or the journey it's attached to, then the same will be indicated by the status Disabled, here.

• Conversion Deadline: Indicates the duration till which the campaigns' Conversion Event will be tracked after it has been delivered. The same can be specified at Step 4: Conversion Tracking while creating the campaign.

  • If you have not set up Conversion Tracking for the campaign or the journey it's attached to, then the same will be indicated by the status Disabled, here.

• Control Group: Indicates whether or not a Control Group was set up to measure the campaign's effectiveness against organic user behavior at Step 4: Conversion Tracking, while creating it. The same is indicated by:

  • Enabled
  • Disabled

• Frequency Capping: Indicates whether or not the campaign was sent with Frequency Capping enabled at Step 2: When while creating it. The same is indicated by:

  • Follow
  • Ignore

• DND: Indicates whether or not the campaign was sent with DND Hours enabled at Step 2: When while creating it. The same is indicated by:

  • Follow
  • Ignore

• Queueing: Indicates the duration for which the campaign will be queued for delayed delivery if we are unable to deliver it immediately to a user. The same can be specified at Step 2: When while creating it.

• An SMS can get queued due to the following reasons:

  • The daily, weekly, monthly frequency cap on the number of messages a user can receive has been extinguished.
  • DND Hours are currently applicable to a few users as per their time zone.

• If you choose to disable Queueing for the campaign while creating it, then the same will be indicated by the status - Disabled.

• Throttling: Indicates the message rate limit specified for the campaign at Step 2: When. It enables you to control the number of messages sent to users per minute. (Detailed Read) It's status is indicated by:

  • Follow
  • Ignore

• Send Winner: Indicates whether or not automated Variation testing was configured for the campaign while creating it, by setting up Send Winning Variation Automatically at Step 4: Conversion Tracking. The same is indicated by:

  • Enabled
  • Disabled
  • If Send Winner is enabled, then the following details will help you out with a quick recap of the rules of testing:
    • Size: Indicates the size of test audience with which all the _Variations _of the campaign will be tested with to determine a winning variant.
    • Time to Test: Indicates the duration for which all the Variations of the campaign will be tested for.
    • Win Criteria: Indicates the parameter (Clicks, Conversions) based on which a winning Variation will be determined.

• Created By: Indicates the name of the user who created the campaign.

• Created On: Indicates the date-time on which the user started creating the campaign.

• Last Edited By: Indicates the name of the user who has made the most recent edit to the campaign.

• Last Edited On: Indicates the date-time on which the most recent edit was made to the campaign.

Now, let's show you how you can analyze an SMS Campaign to gain valuable insights into its impact on user engagement, conversions, and revenue.

How to Analyze

🚧

Must Read

Please refer to Campaign and Channel Performance Indicators for a complete list of all the user-channel interactions tracked for SMS campaigns.

As shown above, using the navigation bar, you can toggle between the following sections to slice-and-dice data and analyze multiple aspects of the campaign:

• Overview
• Analyze
• List of Users

Select the Time Frame of Analysis

🚧

Please Note: You Cannot Analyse 'List of Users' for a Specific Duration

List of Users includes campaign engagement details for all the users that have been a part of the campaign's target audience throughout its lifetime.

By default, all stats under Overview and Analyze are shown from the campaign's Start Date till present day (if the campaign is running) or till upto 45 Days from the campaign's End Date (if the campaign has ended).

For example, in the above visual, stats for the campaign, Mother's Day Offer are shown from it's Start Date, May 10, 2019, till the present day (at the time of documenting) May 13, 2019.

Using the date range filter, you can analyze the campaign's performance for a custom time frame

📘

The following time periods can be selected here:

• Today

• Yesterday

• Last 30 Days

• Last 60 Days

• Last 90 Days

• Custom Dates (as selected by you)

Now, let's dig into each section, starting with Overview.

Overview

Here you can analyze the campaign's performance from its Start Date till the present day (if the campaign is running) or till 45 days from the campaign's End Date (if the campaign has ended), against key performance indicators like Clicks, Conversions, Revenue, Deliveries, Failures for each Variation. You can alter this by selecting a time frame.

1. Performance Overview

This subsection presents a summary of the campaign against parameters like the total number of messages Sent, Failed, Delivered, Queued and indicates its overall performance against key metrics like Unique Clicks, Unique Conversions, and Revenue.

Let's go over each card:

Sent

Indicates the total number of messages that have been sent to the campaign's target audience within the selected time frame, including all the messages that failed or got queued for delayed delivery.

• Failures: Indicates unique users who failed to receive the message within the selected time frame. As shown below, you can hover over the field to analyze the reasons for failure.

For example, in the above visual we see that the maximum number of SMS messages failed to get delivered due to Hard Bounces. This suggests that either the phone numbers shared by users invalid/defunct or that a few users have been out of service network area for a prolonged duration.

🚧

Related Read: Why an SMS Campaign can Fail

Complete list of all the reasons due to which an SMS campaign can fail to get delivered.

Delivered

• Delivery Rate: Calculated against the total number of messages Sent, it indicates the percentage of messages Delivered, within the selected time frame.

• The card also indicates the total number of messages Delivered within the selected time frame.

• Queued: Indicates unique users for whom the messages are currently queued for delayed delivery. As shown below, you can hover over the field to analyze the reasons for queueing.

  • You cannot analyze queued messages for a particular time frame as it represents the current state of message delivery to the target audience.

• For example, in the visual above, we can see that the maximum number of messages (1,226) are currently queued for delayed delivery due to DND queue. This indicates that the DND hours specified by you are currently applicable to 1,225 users, as per their time zone. We will attempt to deliver the SMS campaign to these users as and when the DND hours pass for each.

• However, if a delivery window doesn't open up before the queueing duration expires, then the campaign will fail to get delivered to the respective users. (detailed read on how delivery time is determined for queued messages)

🚧

Related Read: Why an SMS Campaign can get Queued

Complete list of all the reasons due to which an SMS campaign can get queued for delayed delivery to a user.

Unique Clicks

• Unique Click Rate: Calculated against the total number of messages Delivered, it indicates the share of users that clicked on a link included in the SMS message within the selected time frame.

• The card also indicates the number of users that clicked on the link included in the SMS, helping you gauge the campaign's true impact on user engagement.

Unique Conversions

• Unique Conversion Rate: Calculated against the total number of messages Delivered, it indicates the share of users that performed the campaign's Conversion Event at least once within the selected time frame.

• The card also indicates the number of users converted through the SMS, helping you draw a fair comparison between the number of users who clicked on the campaigns, users who converted, and the total *Revenue.

👍

Comparing Number of Clicks, Conversions and Total Revenue for an SMS campaign

For example, in the visual above, we have analyzed the performance of an SMS journey campaign _targeted at users looking to purchase _Mother's Day gifts. It offers a customized deal on cosmetics to all users who enter the journey.

A quick look at the campaign's Performance Overview reveals:

• An Average Revenue Per User (ARPU) of Rs. 1,500 or $22

• A Click-through Conversion Rate of 44.8% - compared to the average CTR of 19% for e-commerce SMS campaigns, this is a massive uplift of 135.7%!

🚧

Please Note

Conversions will not be tracked for your campaign/journey if you have not specified a Conversion Event at Step 4: Conversion Tracking, while creating it.

Detailed Read on how Conversion Tracking works.

• Compare Unique Conversions with Control Group: If you have enabled Control Group while creating the campaign, then as highlighted below, you will be able to draw a scientific comparison between the campaign's conversion rate and the organic conversion rate - helping you understand its true impact.

👍

Analyzing SMS Campaign's Conversion Rate against Control Group

In the above visual, we've analyzed a journey SMS campaign Here's what its Performance Overview reveals:

• Unique Conversion Rate of Campaign: 1.50%

• Unique Conversion Rate of Control Group (CG): 1.38%

• Conversion uplift: 8.69%

Clearly, the campaign had a positive impact as it motivated a higher number of premium users to renew their subscription before it expires!

It's always advisable to test your campaigns (and journeys) with a Control Group before sending it to the entire target audience to ensure that it helps amplify the desired user behavior, and not hinder it. Here's how you can automate Variation testing with a Control Group.

Now, let's walk you through the metrics shown here:

• Unique CG Conversions: Calculated against the total number of users included in the Control Group CG, it indicates the share of users that performed the campaign's Conversion Event at least once within the selected time frame, even though they didn't receive the campaign.

• Conversion Uplift: Indicates the percentage increase or decrease in the Unique Conversion rate achieved by the campaign, calculated against the Unique Conversion rate of the Control Group.

  • It's shown beside the campaign's Unique Conversion rate, in Green for positive uplift, in Red for negative uplift, and in Grey for no uplift.

👍

Fact Check

Conversion Uplift equals [(Unique Conversion Rate of Campaign/ Unique Conversion Rate of CG) x 100] - 100

Revenue

Indicates the amount of Revenue tracked for the Total Conversions that have occurred within the selected time frame.

• Revenue numbers are always shown in the currency selected by you while setting up Revenue Mapping

🚧

Please Note

Revenue will not be tracked for your campaign if:

• You have not set up Revenue Mapping for your account

• You have not specified a Conversion Event while creating the campaign (or its journey)

• You have not mapped the specified Conversion Event as a Revenue Event in your account

Detailed Read on how Revenue tracking works and how Revenue is attributed to a campaign.

2. Variation Comparison

The most insightful section of Campaign Overview, Variation Comparison helps you analyze the impact of each message Variation against key metrics like Clicks, Conversions, Revenue* and so on, for the chosen time frame.

However, depending on the settings specified while creating the campaign, the stats shown here may vary. Let's get you acquainted with the impact of each setting:

🚧

Skip to: How to customize and analyze this section as per your strategic needs.

2.1. When Variation Distribution is Set Manually

If you have manually specified the audience size for all the Variations at Step 4: Conversion Tracking, then using this section, you will be able to identify a winning Variation by analyzing the performance of each. The following aspects of the test will be highlighted here:

• Percentage of the audience targeted by each Variation (calculated against the total target audience)
• The performance of each Variation against all the performance indicators tracked for SMS campaigns

Let's demonstrate a short use-case to show you how it works:

👍

Analyzing Manual Variation Distribution Test Results

Let's take the example of an e-commerce app that exclusively sells formal clothing and accessories for women. They recently noticed a drop in their checkout rates and were worried about the downward trends.

• On analyzing the behavior, marketers of the app learned that a high number of users who started the checkout process left mid-way.

• They weren't sure of the exact reasons contributing to checkout abandonment and formed the following hypotheses:

  • Hypothesis 1: The users simply needed more time to make the purchase decision and thus, left mid-way.

  • Hypothesis 2: The users weren't comfortable with sharing credit card details to process the online payment (given the fact that the e-commerce app was launched just 3 months ago).

  • Hypothesis 3: The shipping cost was a bit higher than competitor e-commerce platforms and thus, users decided to hold off on the purchase.

• They decided to conduct a multivariates test and created 3 Variations of an SMS campaign to test all the hypotheses.

  • Variation A simply nudged users to complete the purchase before the products run out of stock. (to test hypothesis 1)*

  • Variation B offered users an option to complete their transaction through Paypal. (to test hypothesis 2)

  • Variation C offered free shipping on all purchases made on the same day of receiving the SMS. (to test hypothesis 3)*

Here's what their analysis looked like...

Key takeaways:

• Variation B emerges as the winner as it achieved the highest Unique Conversion rate of 3.8%, contributing a whopping $5,780 in Revenue!

  • This implies that Hypothesis 2 holds true! - A high number of users were uncomfortable sharing their credit card details and preferred to make the payment through a trusted platform like Paypal.

  • This insight is further reinforced by the fact that Variation C achieved the highest Unique Click rate, 14%, but a lower Unique Conversion rate (3.6%) than Variation B - suggesting that even though some users were lured by the offer of free shipping, they were skeptical of making a payment directly on the e-commerce platform.

  • Thus, adding Paypal as a payment option to the platform was definitely a good idea!

🚧

Skip to: How to customize and analyze this section as per your strategic needs.

2.2. When Send Winning Variation is Enabled

If you have enabled Send Winning Variation Automatically at Step 4: Conversion Tracking while creating the campaign, then you will be able to analyze the test results through this section. As shown below, the following aspects of the test will be highlighted here:

• The winning Variation (which was automatically sent as the only campaign to the entire target audience)
• The performance of each Variation that was tested with the test audience

For example, in the above visual, we can see that Variation B was determined as the winner. This is because Conversions was selected as the Win Criteria of the test while setting it up, and Variation B (3.8%) achieved a higher Unique Conversion Rate _than _Variation A (3.4%).

Similarly, you too can select a custom Win Criteria for testing the Variations, as per your business goals.

🚧

Skip to: How to customize and analyze this section as per your strategic needs.

2.3. When Control Group is Enabled

You can compare each Variation's conversion rate with the Control Group's organic conversion rate to determine it's true impact on driving conversions. This can be done by enabling Control Group at Step 4: Conversion Tracking while creating the campaign or the journey (the campaign is attached to).

As shown below, the extent to which a Variation influences users to perform the Conversion Event can be measured through its Conversion Uplift, indicated alongside Total Conversions and Unique Conversions, in Green, for positive uplift, in Red, for negative uplift and in Grey for no uplift.

👍

Analyzing Variations with Control Group through Conversion Uplift

In the above visual, we have compared the performance of 2 Variations of a triggered SMS campaign with a Control Group.

• Both Variations _and the _Control Group were equally divided amongst the campaign's Test Audience.

• Since Control Group was enabled for the campaign, Unique Conversion rate was the sole Win Criteria used to identify a winning version of the message.

The analysis reveals that:

• Users who did not receive the campaign as a part of the Control Group converted at a rate of 8.5% - indicating the organic Unique Conversion rate.

• Variation A influenced the maximum number of users to perform the campaign's Conversion Event, achieving a Conversion Uplift of 38.82% and contributing $6,000 in Revenue!

  • As a result, Variation A was determined as the winner and was sent to the entire target audience of the campaign (indicated by the test results highlighted above).

• Variation B achieved a Unique Conversion rate of 8.1%, indicating a decrease of 4.7% in conversion, compared to the organic behavior of the Control Group.

  • This indicates that the message had a negative impact as it deterred users from performing their organic behavior.

🚧

Related Reads

1. Why you should (almost) always use a Control Group to measure the effectiveness of your campaign and how it works.

2. How you can test Variations with a smaller test audience and automatically send the winner to the entire target audience of the campaign!

Now, let's show you how you can slice-and-dice this section to gain relevant insights:

Step 1: Select Performance Indicators

As shown above, using the filter icon placed on the top right, you can select/deselect metrics to customize the table as per your analytical needs.

📘

The following metrics can be selected here:

Overall (Sent): Helps you analyze the total number of messages that were sent to the target audience within the selected time frame, including all the messages that may have failed or may still be queued.

Overall (Delivered): Helps you analyze the total number of messages that were successfully delivered to the target audience within the selected time frame.

Totals (Total Clicks, Total Conversions, and Total Click-through Conversions)

Uniques: (Unique Clicks, Unique Conversions, Unique Click-through Conversions)

• Failures: Helps you analyze all the reasons due to which an SMS campaign could fail to get delivered. Here's the complete list.

• Queued Helps you analyze all the reasons due to which an SMS campaign could get queued for delayed delivery. Here's the complete list.

• Revenue (Total Revenue, Click-through Revenue)

(Detailed read on what each performance indicator denotes)

Step 2: Preview Variation's Message

As shown above, using the View icon _placed next to each _Variation's header, you can preview the body copy of each, for Android and iOS devices to identify the message that led to higher engagement and conversions.

Step 3: Select a Data Format

While the default view shows you percentage values (%) for all the campaign performance indicators, using the overflow menu highlighted above, you can choose to analyze these stats as numbers (by selecting #).

• Percentage value of messages that are Queued or have Failed (Failures) is calculated against the total number of messages Sent.

• Percentage value of all the performance indicators like Clicks, Conversions is calculated against the total number of messages Delivered.

3. Engagement Trends

This section helps you analyze day-wise user engagement trends for each Variation against the following performance indicators, as per the chosen time frame:

• Sent
• Total Failures
• Total Queued
• Unique Clicks
• Unique Conversions

Let's analyze a use-case to get you acquainted with the workings of this section:

🚧

Skip to: How you can customize the stats shown here as per your analytical needs.

👍

Analyzing Engagement Trends for a Recurring Subscription Renewal SMS Campaign

For example, in the visual below, we have analyzed the engagement trends of a daily Recurring SMS campaign for 30 days (Sep 1 - Sep 30) as a Funnel View.

About the campaign:

• The campaign has been created by an ed-tech platform that offers monthly subscriptions for several preparatory mock exams along with tuition.

• End goal: Drive renewal rates. For the same reason, the campaign's Conversion Event is set as the custom eventSubscription-Purchased with a Conversion Deadline of 2 days.

• Hence, it's targeted at all users whose subscription expires within the next 24 hours. Here's what the rules of segmentation look like:

  • All users who have performed the event - Subscription Purchased where custom attribute, Subscription-Expiry is between Now and Next 24 hours.

On analyzing the campaign for the selected time frame we learn that:

1. All the messages Sent on Sep 17 failed to get delivered to the entire target audience, consisting of 508 users. Let's dig into the Failures report to see if we can find any answers:

• In the above visual, we can clearly see that 508 messages have failed due to the reason - SSP Quota Limit Reached. This indicates that on Sep 17, the account ran out of SSP credits, due to which their SMS Service Provider was unable to send the messages.
  • This can easily be fixed by increasing the credit limit of your _SSP account _or renewing your credits and sending the messages again to all users for whom the campaign failed.

2. As seen above, the Unique Click and Unique Conversion trends suggest that most of the users who clicked on the campaign did not perform the Conversion Event.

   • This is a worrisome insight as it indicates that most users were not keen on availing the service of the ed-tech platform again. Marketers of the platform will need to dig deeper into their retention cohorts to find their answers.

3. The highest number of messages were Sent on Sep 25 - 1,277.

4. The highest number of Unique Click_s also occurred on _Sep 25 - 91. However, an abysmally low number of Unique Conversions - 14, occurred on this day.

5. The highest number of Unique Conversions occurred on Sep 12 - 19.

We hope this has equipped you with a robust understanding of how you can leverage Engagement Trends to gain valuable insights into user behavior and the campaign's overall performance.

Now, let's show you how you can customize this section as per your analytical needs:

Step 1: Select a Variation

By default, engagement trends are shown only for Variation A (the first version of the campaign). Using the dropdown placed on the top left, you can choose to analyze trends for each Variation against - Sent, Failures, Queued, Unique Clicks and Unique Conversions.

• If you have enabled Control Group for the campaign, then you can choose to analyze its Unique Conversion trends by selecting 'Control Group' from the dropdown highlighted above.

Step 2: Select a Format of Visualization

While the default format of visualization is a Line Graph, as shown above, using the overflow menu placed on the top right, you can change this to a Bar Graph or a Table, as per your analytical needs.

Analyze

While Overview presents a comprehensive view of the campaign's performance, this section helps you slice-and-dice data in multiple ways to gain maximum insights into user-campaign interactions.

First Impression

The default view of this section shows you a Day-wise trend for all the messages that have been Sent from the campaign's Start Date till the present date or its End Date (whichever occurs first).

Using the query bar highlighted above, you can choose to analyze the campaign's performance against a specific performance indicator, split by 2 parameters for a custom time frame.

Let's Analyze a Use-case to Show You How it Works

🚧

Skip to: How you can slice-and-dice stats to gain valuable insights into user-campaign interactions.

👍

Use-case: Analyzing a Recurring SMS Campaign Aimed at Converting Newly Acquired Users

Let's take the example of an e-commerce platform that asks all it's users to signup before they start browsing through products on their website and app.

• In a bid to drive conversions and revenue, marketers of the platform decided to motivate all newly acquired users to make their first purchase by offering them a cashback of 15%!

• The end goal is to:

  • Convert newly acquired users

  • Motivate them to make a second purchase (so that they can redeem the cashback amount)

• Hence, they set up a daily recurring campaign for all users who have signed up over the last 7 days but have not made a purchase yet. Here's what the rules of segmentation look like:

  • All users who have performed the event: Signup-Complete in the last 7 days AND have not performed the event: Checkout-Complete

  • The campaign is sent daily to all the new users that get added to the segment (detailed read on how segments work)

(all signups on their app and website are tracked as the custom event, Signup-Complete, while all purchases made on both platforms are tracked as the custom event, Checkout-Complete)

• Conversion Event of the campaign is set as Checkout-Complete with a Conversion Deadline of 2 Days.

Now, let's analyze the campaign over the Last 30 Days to understand its impact on Conversions and Revenue.

👍

Analysis 1 (In continuation of the use-case discussed above)

Identifying Hours of the Day during which the highest number of (total) conversions occurred between April 1 - 30.

As shown above, here's how they went about their analysis:

Step 1: Select Total Conversions under the field - Show.

Step 2: Select Hours of the Day under the field - Over and click enter.

It reveals that:

• Users performed the campaign's Conversion Event, Checkout-Complete 58 times during the month.

• The highest number of conversions (14) occurred between 11 am - 12 pm, followed by 4 pm - 5 pm (7).

👍

Analysis 2 (in continuation of analysis 1 of the use-case)

Now, let's analyze the value of the conversions or the Total Revenue against Hours of the Day to understand the trend of high-value transactions.

_As shown above, select _Revenue under the field - _Show_ and click enter!

It reveals that:

• A total of Rs. 77,295 was generated through the SMS campaign throughout the month.

• Contrary to the Total Conversion trends, transactions of the highest value occurred between 3 pm - 4 pm, amounting to Rs. 17,325.

  • This indicates that even though a low number of conversions occurred between this time frame (throughout the month), most of these were high-value transactions.

👍

Analysis 3 (in continuation of analysis 2 of the use-case)

Now, let's analyze the campaign for the month of March to gain a better perspective on these stats and behavioral trends. We'll start with analyzing Total Conversions against Hours of the Day.

As shown above, here's how they went about their analysis:

_Step 1: Select the time frame as _March 1 - 31 using the date range filter.

Step 2: Select Total Conversions under the field - Show and click enter. (we will retain Hours of the Day under the field - Over from our previous analysis)

It reveals that:

• Users performed the campaign's Conversion Event, Checkout-Complete 38 times during the month.

• Keeping up with the trends of April 2019, the highest number of conversions (12) occurred between 11 am - 12 pm, throughout March 2019 too, followed by 3 pm - 4 pm (7).

👍

Key Takeaway

On checking the recurring campaign's delivery time, we learn that it has been set for delivery at 11:15 am, daily. This indicates that most users made a purchase as within an hour of receiving the message!

We hope this has equipped you with a broad understanding of how you can gain in-depth insights into your campaign, it's users and their preferences through Analyze. Now, let's walk you through all the steps of analysis:

Step 1: Select a Variation / Control Group

If you have created multiple Variations for an SMS campaign, or have enabled Control Group for it, then using the dropdown highlighted above, you can choose to selectively analyze the performance of each.

However, if your campaign consists of just one Variation, then this dropdown gets hidden.

📘

Any one of the following options can be selected here:

• Overall: Helps you analyze the combined performance of all the Variations against each metric for the selected time frame.

  • For example, if you choose to analyze Revenue, Overall, then we will sum up the Revenue tracked for each Variation to help you analyze their collective impact.

• Variation A/B/C/D/E: Helps you analyze the performance of a Variation, against a selected performance indicator, as per the chosen time frame.

• Control Group: If you have enabled Control Group at Step 4: Conversion Tracking while creating the campaign or its journey, then selecting this option will help you its Unique Conversions/ Total Conversions.

🚧

Related Read

Please refer to Campaign Variations and Testing for a quick read on how you can test multiple Variations of a campaign with a smaller audience and automatically send the winner to the entire target audience!

Step 2: Select a Performance Indicator

You can choose to analyze the overall performance of the campaign or each Variation against a performance indicator or campaign event (like Clicks, Conversions, Revenue) by selecting an option under the field - Show. As highlighted above, the total value of the selected performance indicator is also indicated under the query bar.

📘

Any one of the following campaign events can be selected here:

• Overall (Sent): Helps you analyze the total number of messages that were sent to the target audience within the selected time frame, including all the messages that may have failed or may still be queued.

• Totals (Total Clicks, Total Conversions, Total Click-through Conversions)

• Uniques: (Unique Clicks, Unique Conversions, Unique Click-through Conversions)

• Failures: Helps you analyze all the reasons due to which an SMS campaign could fail to get delivered. Here's the complete list.

• Queued Helps you analyze all the reasons due to which an SMS campaign could get queued for delayed delivered. Here's the complete list.

• Revenue (Total Revenue, Click-through Revenue)

(Detailed read on what each performance indicator denotes)

Step 3: Select the Dimension(s) for Analysis (Over & Split By)

As demonstrated for the use-case discussed above, by selecting an option under the fields - Over and Split By, you can choose to analyze the selected performance indicator (campaign event) against two dimensions (system attributes). This is a great way to dig into multiple aspects of the campaign like:

• The top Locations and Devices through which the highest number of users performed the selected campaign event.

• The type of users (Known or Unknown) engaged through the campaign and their Location.

• The trend of occurrence of selected campaign event against Hours of the Day, Days or the Week or Months of the Year and so on.

📘

The following options can be selected under both the fields:

• Time (Days, Weeks, Months)

• Time Block (Hours of Day, Days of Week, Months of Year)

• User Type (What it means)

• Location (Country, City)

• Technology (Device Type)

List of Users

🚧

Must Read

Please ensure that you have a good understanding of the various types of Users and related concepts before proceeding. Doing so will help you understand the workings of this section, better.

This section has been specially designed to help you track down campaign engagement details for each user that targeted by the campaign, over its entire lifetime. Thus, analyzing List of Users comes in handy when digging into user-campaign engagement details for Triggered, Recurring, Transactional, Journey or Relay SMS campaigns (as they are on-going cycles of communication).

First Impression

By default, List of Users shown you a Summary of user-campaign interactions for All Users who have been included in the campaign's target audience, from its Start Date till the present date or its End Date (whichever occurs first).

Now, let's get you acquainted with its features:

Customize List

Here's how you can customize the List of Users as per your analytical needs:

Select Time Frame

As discussed under Users, at WebEngage, users can be two types, Known (identified users) or Unknown (anonymous users).

As shown above, you can select the _Time Frame _from the drop-down and customize your result. From the date range picker, you can select the date range for which you want the data. You will be able to view/ download data for a max of 30 days at one time.

Select Parameter

As shown above, using the second dropdown placed on the top left, you can customize the List of Users by selecting a parameter.

Download List

As shown above, using the Download icon placed on the top right of the table, you can choose to download a user-wise engagement report for all the users targeted by the campaign, over the selected time frame. Here's how you can go about it:

Step 1: Click the Download icon.

• In doing so, you will be prompted with a pop-up to re-confirm your decision.

Step 2: Click the Download button on the pop-up window to proceed.
*You will receive an email on your registered email address containing a link to the downloadable file.

Search for a User

As shown above, using the search box placed on the top right, you can dig into the engagement status of a particular user by searching for their User ID, Email, and Phone Number.

🚧

Why is the User ID shown multiple times?

Sometimes multiple rows get populated with the same User ID when searching for a user. Each row contains a different message status with a varying date-time of Delivery, Read, and Clicks.

This is because the same user was sent the recurring SMS campaign in multiple runs, and in each run, they interacted differently with the message. Thus, if a triggered, recurring, transactional, journey, or relay campaign is sent to a user multiple times, then you will be able to analyze their interaction with each one.

Access User Profiles

As shown above, you can access the profile of a user targeted by the campaign by clicking their (hyperlinked) User ID. Doing so will help you analyze the user's channel preferences, personal details, device preferences, latest interactions with your campaigns, app, website, the technical aspects of each interaction much more!

🚧

Must Read

Please refer to Analyzing User Profiles for a robust understanding of how you can dig into the preferences and behavior of individual users.

We hope this has equipped you with a robust understanding of how you can mine valuable insights into a campaign to understand its true impact on user engagement and revenue. Please feel free to drop in a few lines at [email protected] if you have any queries. We're always just an email away!