This is the third (but not final) post in the series on Visitor Engagement.  One of the problems with the Visitor Scoring method that I previously described, is that, at the end of the day, you’re still somewhat limited to viewing scores at the “average” level, by segment.

That presents a number of challenges because the average is precisely that…and the underlying scores vary dramatically within each segment.

But there is a way to see what each and every visitor score is, or even within the different segments…and it’s called the Unique Visitor ID.  You can see at the visitor level, how many times they’ve returned, how many “things” they’ve done, such as searches, product views, revenue etc.

This is really the icing on the proverbial cake.

Visitor ID

There’s a seldom used implementation method in SiteCatalyst called Dynamic Variables which easily enables all of this capability.

All you need is a spare eVar and you’re in heaven.

Step 1

Simply populate an eVar with “D=s_vi” and the result is that SiteCatalyst will populate the eVar with the unique visitor cookie ID.  If you test it with WASP or something similar, you’ll only see that D=s_vi got passed, but the magic begins.

s.eVar56="D=s_vi";

Step 2

Go into you Admin console, and on the report suite where you want this collected, go to Conversion > Unique Visitor Variable section.

You’ll be presented with a dropdown of available eVars that can be selected to use as the Visitor ID…simply select the one that you’re using and hit save.

That’s it – you’re done!  All there is to it.

Rest assured, it does not impact anything to do with the way that SiteCatalyst measures unique visitors – its completely separate and safe to use.

And now it pops

So, the first thing you’ll see is a new report on your eVar. The report is a list of all Visitor Cookie ID’s captured during the timeframe selected.  As it’s a conversion variable, you can add in whatever success events you like:

Add in your Visitor Score and you can see what their points are.  As you can see from above, I’ve highlighted an interesting visitor (#7)…they were on their 2nd visit, they searched 12 times, viewed 16 courses and submitted an application.  Their score is currently 542.  I’m wondering if visitor #5 is a candidate for an app too…similar behaviour is forming.

Segment-based Visitor ID’s

Ok, now open up a report such as Campaigns (we use the Unified Traffic Sources VISTA rule so it may look a little different if you don’t use it).

Remember the challenge I mentioned at the top of this post…with average scores.

We see below that the average visitor score for Organic Search is 7.88.  But, that’s just an average.  We know that from our scoring methodology, you have to have a higher score than that if you’re interacting with content.

So, when we break down the Search Organic by Visitor ID’s, we see a very different story.

Look at that…the top 10 all have scores above 300!

So to get to an average of 7.88, we must have an awful lot of really low end scores that come in through organic search.  And actually, that makes sense…we have a lot of reference content across our site, such as Library and School content that is very popular and that’s likely where these users with low scores are going.  Not the ones we’re really interested in from this scoring standpoint though.

Extracting Visitor ID’s

The next fancy bit of functionality, once you’ve enabled the Unique Visitor ID, is the ability to extract those visitors that have met a certain criteria (success event).

If you now click on an item in, for example, the Last Campaign report (at the individual campaign tracking code level), instead of heading off to the summary page for that item (which by the way, is useful also), you’ll get a new menu:

If you select the success event you want the visitor ID’s for, you’ll end up in a data warehouse data extract screen, where the report can be scheduled for immediate delivery:

Here we’re extracting visitor ID’s for those that submitted an application having come to us from Google Australia.

The Icing

I think the real icing on this cake is the ability to look at all visitor scores at the individual visitor level, irrespective of the segment, so that you can get an idea of what their scores are, to help better gauge those low/medium/high engagement buckets.  It’s also really useful to see at the segment too.

You’ll need to use Excel to process the data (and one thing I should point out is that an eVar can only have a maximum of 500,000 unique values in a given month – so if you get more visitors than that, then this might not work for you).

Other ways to use this is to extract emails that have responded to a campaign – if you capture the memberID in the eVar you can then extract the data for later use.

The next thing we need to do is re-configure our Discover segment for the interaction index and amalgamate these metrics into the overall engagement scoring methodology.

This is a follow on post to my previous one about measuring that elusive engagement.  This post focuses on the aspect of applying a score to visitor interactions, as they interact with your content and applications.

Visitor scoring is fairly simple – especially in SiteCatalyst, and by leveraging the data in Discover through segmentation, (and ultimately in SiteCatalyst 15), it’ll give you even more insight into visitor engagement.

Visitor scoring measures and assigns a relative value to individual customers and prospects based on their actions and behaviors over time. You can determine intent and engagement – even before visitors convert.

Once you’ve identified your most valuable visitors, you can dissect their actions to determine the campaigns, keywords, referring sites and offline touch points that engage them – and invest more on these efforts.

Scoring

The basic premise of a visitor score is to give them “points” for certain activities.  What score you decide to give them is entirely up to you…the important thing is that they accrue points over time, and you can use those points to create segments of visitors that have exceeded certain thresholds.

In doing so, you’ll be able to compare different visitors, for example, those with a high score, to those with a low score, from different traffic sources, or across different campaigns.

The theory behind this is that visitors with higher scores are likely to be more engaged – they’ve accrued more points over time, by doing the things you want them to do.  It’s not just looking at conversions, although they will probably feature in your scoring methodology.

Imagine for a moment that you have a number of visitors:

Visitor 1 – comes to your site from Organic search, looks at 5 different products, watches 2 videos, signs up for your newsletter, but doesn’t buy anything.

Visitor 2 – comes to your site from Organic search, looks at 1 product and buys it.

Visitor 3 – comes to your site through an email campaign, and views 8 of your products.

Visitor 4 – comes to your site through Paid Search, looks at searches for a product, views it once and then looks at 3 FAQs.

Which visitor is more valuable to you?  Who is more engaged?  Visitor scoring will assist you to better understand the answer to that – and when you slot this Visitor Scoring methodology into the Interaction Index, and use Discover, you really do get a good proxy for engagement.

Ultimately most sites exist for a specific reason, whether it be to convert the visitor to a purchase, or allow them to self serve, or engage them with your content etc.   And we all want them to do that.  But only a small percentage will.  In fact, in general, it’s around 3-5% of visitors that actually “convert”.

First things first

What you need to do initially is agree on a set of interactions across your site, and then apply an arbitrary score to them, either between 1 and 10 or 1 and 100.

For example:

1. Home page viewed or Landing Page Viewed = 1 pt
2. View a general content page = 5 pts
3. View a Product = 10 pts
4. Search = 20 pts (I tend to think that if a visitor is going to take the time to search for something they can’t find, they’re more engaged, hence the higher score)
5. Watch a video = 30 pts
6. Use an interactive tool = 40 pts
7. Sign Up for a newsletter = 50 pts
8. Provide feedback/rate and review/comment = 75 pts
9. Buy a product or conduct a self service transaction = 100 pts

Don’t worry about the actual scores, but do leave room for additional interactions that you can add in if you need to (but remember to communicate it out if you do add in others, as scores will go up).

Now that you’ve scored various activities, you need to implement the code to measure those interactions.

Implementing the scoring

In SiteCatalyst that’s easy enough to do – you need an eVar and a success event.

Set up the success event as a Counter Event. 

Note that you can now have counter events where they take a value other than 1, which is what is needed for the  scoring.  However, I believe you’ll need version H23 of the s_code to support this. 

Create the eVar as a Counter eVar, not a text eVar.

Visitor scoring is done by simply passing the score value as a +number into the eVar and the success event is set with the =number (yes, you no longer need to pass the event through the s.product string).

So, for example, on your homepage, you simply add the following:

s.events="event1=1"; s.eVar1="+1";

On your search page, you’d include:

s.events="event1=20"; s.eVar1="+20";

(The above obviously assumes you’ve used event1 and eVar1 for Visitor Scoring.

Complete that process for each of the key interactions and you’re set.  We implemented ours directly into our s_code through a variety of s.pageName value matches, or product views, or other success events occurring.

To put this in context, we’ve implemented the following scoring:

1. Homepage view = 1pt
2. Searched for something = 5pts
3. Viewed a course = 20pts
4. Completed a form = 30pts
5. Used one of our interactive tools = 50pts
6. Opted in to something = 70pts
7. Submitted an application = 100pts

Making it legible

First thing you need to do when you implement this type of thing is make it all a bit more usable.

When you look at the raw data (your eVar against conversions), you’ll see something like:

Remember that the success event (Application Submitted) is associated back to the value seen in the eVar, when the success event was applied – so, in the example above, 37 apps were submitted when visitors had a score of 190; 22 had a score of 211, etc.  Notice also that as we’re showing both Apps and Leads, we also see that they don’t correspond; Leads has a very different scoring result (see below).

Overall, it doesn’t tell you much – you actually want to group the eVars together.  So, use SAINT to classify them into buckets. 

Important Tip: When you use Excel to classify your eVar, the key column will be the interaction score.  You need to keep that column showing 2 decimal places – when you export from SAINT, it has the decimal places on it, when you open in Excel, the decimal places disappear, and if you re-save without decimals and upload again, they keys will be different, and the reports won’t work.  So, put the key column to 2 decimal places and then classify it.

That brings about the problem of what your buckets should be…well, that’s up to you.  Only your data will tell you that.  But, you can experiment with different buckets to see what works best.  To be honest, it’s best to do this in Excel, rather than SAINT, as you’ll want instant gratification.  And you’ll obviously need data collected before you can classify…so it’s best to run it without classifications for a while.

We classified ours following a bit of analysis using Excel to figure out the best buckets for Applications Submitted and Optins.  As it turned out, after a lot of playing around, we chose a logarithmic scale, as it seemed to group everything the best:

What we see in Excel is that with these buckets, 80% of applications occur before a visitor has a score of 320.  We also note that most apps occur when visitors have a score of between 160-320…our sweet spot.

So, using SAINT, we classified our Engagement Value eVar into our buckets and uploaded back to SAINT. 

Tip: Due to the size of the file, we typically use an FTP upload now – it took less than 10 minutes to classify the data in the reports.

Now re-run the above SiteCatalyst report, this time using your buckets:

Remember above I said that leads obviously have a very different score?  You can see that in the above report, that Lead Completes tend to happen when the visitor has a score of between 40 and 160.

What we see from Excel with Leads is that 80% of leads occur before a visitor has a score of 80 – which means that those that become leads, do so pretty quickly; they haven’t visited much other content (otherwise their score would be higher) – which is great news for us!

Another way to classify the scores is to set “low, medium, and high” buckets for engaged visitors.  I’m still trying to figure out what we should use as those buckets, as we have a very large spread.  Standard Deviation will probably assist in that one eventually.

Calculated Metrics

At this point in time, you probably also want to create a few calculated metrics to get some averages out.

Ones that I’d recommend are:

1. Score per Visit = [Engagement Score] / [Visits]
2. Score per Search = [Engagement Score] / [Instances (Report-Specific)]
3. Score per Referrer = [Engagement Score] / [Instances (Report-Specific)]

Why is Instances in there twice?  Because a) Visits are not available across every report type and b) the naming convention just makes it a bit easier to understand (IMHO).

Did you know you can also trend calculated metrics?

Now that you’ve got all this wonderful new capability, what do you do with it…?

Segment it (of course)

There’s all sorts of different ways you can use this…all basically segmentation based.

You can look at traffic sources, campaigns, keywords, geographics, user type segments, etc.  You can trend and compare your segmented values over time.

In the example on the right, we see that while Australian visitors have an average score of 5.82, the highlighted items have much larger scores, indicating that they are engaging with more of the things we want them to do.  The example on the right was extracted using Discover, as in SiteCatalyst 14 the capability with geo-demographic reports is somewhat limited to country/visits.

 

In the next example, we’re looking at scores by course interest.  The course category is actually an eVar that is set in various places across the site – similar to a product category.  When a visitor browsers different bits of content, we’ll set their course category differently, which is then used for Test & Target purposes…but works well here too.

We see that while Undergrad had more course views during this short time frame, they scored actually slightly less than Postgrad visitors.  This would indicate that PG visitors tend to read more, which would also make sense, as it’s a bigger purchase decision for them.

And as a final example, scores by different campaign types (Organic, Paid, Campaigns, etc).

Hmmm…seems Paid Search is doing really well; not a huge amount of traffic during this period, but they are interacting a lot.  Break that down by keyword or Adword group and you’ll get even more insight.

Here we’re looking at Branded vs. Non Branded keywords (and their top keywords):

Seems that branded search terms generate a higher interaction, and in fact when they type in the full name of the university, they have the highest interaction.

Multi-scoring techniques

You’re not limited to having just one scoring methodology.  We’re actually in the process of implementing a second one for a completely different reason.  All you need is a spare eVar, another success event, and bit of time.  Rinse and repeat!

Comparative and Trending

Of course, you can also trend on the calculated metrics to ensure your overall interaction score is going in the right direction, and you can trend by different segments too.

Likewise, you can do comparative analysis using different dates to compare the interaction scores.

Extending the scoring

In SiteCatalyst 15, you can leverage the segments for visitors with scores higher than X – you can either use the value of the event, or you can use the SAINT classifications. 

Additionally, you can use these in DataWarehouse and Discover – or start from Discover and put the segments back into SiteCatalyst.

And, you can leverage the segments and scores in Test & Target to further optimise user journeys and conversions.

There’s just a whole heap of different ways to use this information.

Next post

In my next post, I’ll combine the above interaction Visitor scoring into the Discover segments for a full on Engagement Analysis, replacing the previously described interaction score with this one.

Now, there’s a hot topic.  Measuring engagement.  One of the most widely debated topics in web analytics.

What is engagement and how do we measure it?

Engagement, unfortunately, is not derived from a single measure.  It’s not time on site.  It’s not how many pages they viewed.  It’s not bounce rates and it’s not about conversions.

Engagement is about a lot of things.  What is an engaged visitor and how do you measure engagement?

“Visitor Engagement is an estimate of the depth of visitor interaction against a clearly defined set of goals.” Eric T. Peterson and Joseph Carrabis.

A while ago, I came across their paper through Web Analytics Demystified, entitled “Measuring the Immeasurable: Visitor Engagement”.  While I won’t go into it in any detail, I will suggest that you read it, as it’s the background of this post.

The premise of the paper is that visitor engagement is made up of 7 different metrics, and expressed through one formula:

where:

Engagement can be expressed as the average of the sum of indexes, across specific segments, according to a:

1. Click Depth Index – which captures the contribution of page and event views
2. Duration Index – capturing the contribution of time spent on site
3. Recency Index – which captures the visitor’s “visit velocity”—the rate at which visitors return to the web site over time
4. Loyalty Index- the level of long-term interaction the visitor has with the brand, site, or product(s)
5. Brand Index – the apparent awareness of the visitor of the brand, site, or product(s)
6. Feedback Index – qualitative information including propensity to solicit additional information or supply
7. Interaction Index – visitor interaction with content or functionality designed to increase level of Attention

According to Eric T. Peterson, “Visitor Engagement is a function of the number of clicks (Ci), the visit duration (Di), the rate at which the visitor returns to the site over time (Ri), their overall loyalty to the site (Li), their measured awareness of the brand (Bi), their willingness to directly contribute feedback (Fi) and the likelihood that they will engage in specific activities on the site designed to increase awareness and create a lasting impression (Ii).”

When applied at the visitor level, on a per-visitor basis, they combine to form a pretty good proxy for visitor engagement.

Having read the paper, I was intrigued, and decided to use Discover to implement this…to some pretty insightful results.

A summary of the indexes

Click Depth Index

The percentage of your overall audience that has a minimum threshold of an acceptable number of page views per session.  If you see that on average, visitors “convert” after viewing at least 5 pages, then your minimum threshold would be 5 pages per visit.

Duration Index

The percentage of your overall audience that has a minimum threshold of an acceptable amount of time on site per session.  If you see that on average, visitors “convert” after spending at least 10 minutes on your site, then your minimum threshold would be 10 minutes.

Recency Index

The percentage of your overall audience that returns and converts within an acceptable amount of time (generally days).  If you notice that most visitors convert between 1 and 10 days, then you’d be looking for visitors with a return frequency of <= 10 days.

Loyalty Index

The percentage of your overall audience that has a repeat visit frequency in excess of a minimum threshold.  For example, if you notice that many visitors convert after visiting your site more than three times, then your threshold would be a visit count of at least 3.

Brand Index

The percentage of your overall audience that comes to your site either directly, or through branded search terms.

Feedback Index

The percentage of your overall audience that completes feedback on your site, or participates in rating or reviewing content, or commenting on blogs.

Interaction Index

The percentage of your overall audience that interacts with specific content on your site, or engages in an activity on your site.  There are no thresholds for this index – they are simply counts of.

Note: while you can count pre-defined activities on your site, it is better to score visitor interaction.  I’ll be doing a post on visitor scoring shortly.

Using Discover

Discover was built for this!  It’s very easy to create segments within Discover and apply them across various views to gain insight.

Firstly what we did was to look at some of the thresholds to understand what our “Anonymous” segment of traffic does.  Our anonymous segment is made of non-student and non-staff traffic, which we already have segments for in both Discover and SiteCatalyst.

We figured out what our minimum page views per session should be, the average duration, frequency of visit etc, by looking at them from a conversion standpoint…i.e. how many pages does a converter see, on average, before converting.

Once we’d done that, our 7 segments were easy to define as follows:

1. Click Depth Index – Visitor container, Path length > 10
2. Duration Index – Visitor container, Seconds spent per visit > 1800 (30 minutes)
3. Recency Index – Visitor container, Return Frequency <= 7-14 days
4. Loyalty Index – Visitor container, Visit number >= 2
5. Brand Index – Visitor container, Organic Search Keyword contains “Murdoch” or Visit without referrer
6. Feedback Index – (we don’t use this one)
7. Interaction Index – Visitor container, any of the following events: Lead Complete, Application Complete, Form Complete, Tool Name

The 8th segment was All Visits.  In each case, we used the Visitors metric to view the number of visitors that were part of each index.

If we view this against referring sites, what we end up with is the number of visitors that match each segment rule:

Export to Excel

What we need to do now is export the data to Excel to do the averages and generate the final engagement value.

Simply select the first item “None”, click Ctrl+A for select all, then click Ctrl+C for copy.

Open Excel, and paste the raw data into a new sheet.

Then it’s simply a matter of calculating one columns percentage as a percentage of the All Visits – Visitors column.

Once you’ve done that for each column, you have the indexes for each segment.  Now you just average all of the indexes to get an engagement metric.

For example, we see in the above that Direct Traffic, “none” in the above report, has an overall engagement value of 23%.  But if we look at the other columns, we also see that they are at the median value on Click Depth, whereas traffic from deewr.gov.au is well above the median.

A couple of interesting things have also been highlighted in the above, for example, traffic from Google Singapore is actually far more engaged than traffic from Google Australia – now that’s interesting.

Of course, you should always look at engagement via larger segments, for example, by Campaign, by Site, by Time of Day, Day of Week etc.

While Tuesday comes out overall for a better engaged visitor, I’ve highlighted other interesting things, such as on Saturday and Sunday visitors click more, but more visitors spend time on Saturdays.  During the week is better for branded search term visitors, and Wednesdays seems to be better overall for key interactions.

Multiple Sites

If you have multiple sites, such as microsite etc, you might want to check engagement across them to see if they are dramatically different, so you can then begin to try to understand why.

In the above calculation, I’ve removed the Feedback and Interaction indexes from the calculation, as they would skew the results.  It’s interesting that while the main University site has an engagement index of 23.67, versus a median of 20.43%, sites like Mobile and Maps have a very high engagement value.

Segmentation

Once you’ve got the basic engagement working and you’re looking at things from the overall perspective, you can then easily begin to look at engagement by different segments.

In Discover, you can create segments on-the-fly and apply them across your other segments, by simply dragging the new segment onto the filtered workspace. For example, we segment the above by Anonymous visitors, after we’ve built the overall segments.  We can do the same for Converting Visitors, or Social Network visitors, or Campaign Visitors, or just Mobile visitors, or different content areas across the site etc.  Discover makes it very easy to do this.

Once the spreadsheet is set up, all you need to do is copy the data back into the sheet and you’ve re-run the engagement metric – in about 5 seconds.

Discover just rocks for this real-time, conscious stream of thought, type of analysis.

In summary

There’s lots of different ways to look at engagement, and hopefully, this will help you understand that there is no single metric, and engagement values change based on various lenses.  But, with the above combination of metrics from the very useful paper by Eric T. Peterson, I believe that we’re closer to understanding engagement, which will help us to modify our sites, or target content better, to try to achieve better levels of engagement by those who are below the medians.

As an aside, part two of this post will be about Visitor Scoring, which is a better Interaction Index than the one demonstrated above – and can be used directly in SiteCatalyst reports.  It involves a bit of custom code for your s_code, and a bit of forethought, but easy enough to do…but I’m saving that for a bit later this week.

Drop me a line or comment below with ways that you are measuring visitor engagement.

I thought I’d share a real humdinger with you.

It’s campaign time again.  Not just any old campaign, it’s our main recruitment campaign of the year.

What we normally do on this campaign is behaviorally target content to users based on their application stage.

Why?  Because we know from previous tests that behaviorally targeting content for re-engagement purposes not only lifts our application completion rate, but provides more relevance to the user when they visit our site – instead of just seeing a standard campaign message each time.

Given the value of an application, and the lift we see, it’s well worth the effort to go down this path.

But this time, the gates of hell opened and out rode the fifth horseman.

We typically use 4 different stages, with one experience (content) per stage.  The challenge we always faced was that for each stage, we always wanted to run A/B tests to see which converted better – but could never figure out how to do it when it’s coupled with a behaviorally targeted campaign.

A few months ago, I worked with one of the Omniture Test & Target genius rock stars, implementation engineer Randall, who worked through a nested mbox example with me, which could be used to nest A/B tests within a single Landing Page campaign.  Unfortunately at the time we didn’t get to implement anything (or even try it – but it looked great on paper).

Knowing that our primetime Mid Year recruitment campaign period was nearly on us, I dusted off the email and had a good ‘ole read, as I felt it had some opportunities for us in a variety of ways.

And so the madness begins…

Always up for a challenge, I strapped on my Test & Target boots, and suggested that we could perhaps, possibly, maybe, ‘ish, try, have a crack at, putting together a behaviorally targeted campaign, where at each stage of the process we not only change the offer (easy) but run an A/B test (difficult) to test how well different offers convert people to the next stage.

Be careful what you wish for…because not only did they like that idea, they compounded it by throwing in another campaign on top of that one.

Mid Year campaign is normally restricted to Undergrad recruitment.  At least, that’s been the (now) historical position.

But no, this year, we’re doing simultaneous Undergrad and Postgrad Mid Year campaigns.

Yikes!!!  Go big or go home!  In for a penny, in for a pound.

So, to summarize, we want a simultaneous 4 stage behaviorally targeted campaign with a series of A/B tests within them.  And we need to switch users between campaigns based on whether they meet certain criteria, or whether they’ve viewed specific types of content.

Hmmm…a real humdinger!

Test & Target to the rescue

It gets a bit tricky to do, so I’ll do my best to explain it.

Before I do though, here’s an illustration of what was actually created.  Each of the rectangles with the little red circles is a separate Test & Target campaign.  You’ll see there are 11 of them.  The final bottom row is content…you’ll see there are 12 of them.  And you might also notice a total of 10 nested mboxes!

In a bit more detail, we’ve ended up with:

• 1 primary campaign as a Landing Page campaign that switches between an UG or PG experience, with a default offer for those we know nothing about.  The UG or PG experience then calls on (via a nested mbox):
• 1 of the 2 recruitment campaigns (UG or PG) as Landing Page campaigns so we can target one of the four  stages of the application, via nested mboxes, which calls on:
• 1 of the 8 A/B..n campaigns to determine the offer displayed

There’s a total of:

• 10 nested mbox offers…
• 12 content offers (the things people actually end up seeing)…
• 5 expression targets…
• And 6 profile targets…

Holy moly, I can hear you say.  Why so complicated?  It’s complicated because of the need to switch people between the campaigns based on what content and stage they’re at.

So, how does it all work?

The top campaign:

As we want to target initially by user interest (they’ve previously viewed Undergrad or Postgrad content, or they’ve visited an Undergrad or Postgrad course, or they’ve visited the Undergrad microsite or Postgrad site, or they’ve come in from an Undergrad of Postgrad email campaign, or they’ve visited the Figure Out Your Course tool or the Postgrad tool), we need to be able to get them into the relevant Undergrad or Postgrad campaign.

So, we need to use a Landing Page Campaign, which is set to re-evaluate the rules (and hence the offers) each time it displays.  This allows a user to see different experiences each time, based on the rules above.  It’ll switch a user to the opposite campaign if they go, say, from an Undergrad course to a Postgrad course.

But, we don’t want to show them an offer just yet.  We’re only doing this to determine which campaign they need to get into – and change that decision on the fly.

Hello nested mbox

On our home page, we already have an mbox (marketing box) defined, with default content.  So we start off by creating a campaign to use that mbox.

The following shows the top level campaign, targeting the default mbox on the page – HP_lowerleft.

Instead of serving content back, we want to serve back a different campaign (Undergrad or Postgrad), so we actually need to serve back a nested mbox as our offer.

A nested mbox is just an mbox served into an existing mbox, which then makes another call to Test & Target to get more content.

So, if I meet the criteria for Undergrad, then the content offer that Test & Target serves back to me contains a new mbox.  The content offer looks like:

<div class="mboxDefault">
</div>
<script type="text/javascript">
mboxCreate('UndergradNested');
</script>

I repeat that type of content offer for the Postgrad campaign offer, but call the other mbox ‘PostgradNested’.

From a conversion standpoint, we set the Conversion as an SiteCatalyst:event…meaning that whatever the user does, the campaign thinks they’ve converted, and as they restart with the same experience, it re-evaluates them each time, to determine which offer to show them.

That’s campaign #1.

We’ll call it the Mid Year Parent Campaign – and it’s the one at the very top of the illustration.

Now that I have that, I need to create two sub campaigns; one for Undergrad, the other for Postgrad. These campaigns will serve something back into the UndergradNested mbox, or the PostgradNested mbox.

Main Sub Campaigns

So, as I also want to serve back behaviorally targeted content, based on their stage of application, I need to repeat the above type of scenario, but have the experiences based on the users stage.

Stage of application

Once again, a Landing Page campaign allows a user to see different experiences each time, based on the rules evaluated.

So, we created another Landing Page campaign, with 4 offers, each offer targeting the stage that the user has completed:

1. They’ve started an application but haven’t completed it yet
2. They’ve qualified for entry, but haven’t started an application yet
3. They’re a repeat visitor and haven’t qualified yet
4. They’re a new visitor and haven’t qualified yet

But, again, we don’t want to show them an offer just yet – we want to show an A/B test for offers 3 & 4, and different content for offers 1 & 2.  So, again, we serve back a nested mbox offer, that aligns with the users stage.  In the example below, I’ve shown the nested mbox content for Experience D:

<div class="mboxDefault">
</div>
<script type="text/javascript">
mboxCreate('1stTimeVisNested');
</script>

A/B..n Sub-Campaigns

Now that I have the Undergrad main behaviorally targeted campaign created, I need to create the 4 sub-campaigns which will serve back real content that the user will see.

So, I create a new A/B..N campaign, target the mbox 1stTimeVisNested created above, and create 2 different experiences for it.  That’s the A/B test for the 1st Time Visitors who haven’t qualified.

I repeat that for Repeat Visitors who haven’t qualified, but this time I target the campaign at the mbox RepeatVistNested.  This one also has two offers (the A/B test for this campaign).

Then I create two more A/B campaigns which will serve back content into the respective mbox defined during the other two stages of the application.

Complicated I know – hope you’re still with me.

Rinse and Repeat

Once all that’s been created, it’s a fairly simple matter to duplicate each campaign for a Postgrad version, modify the offer to create new distinct mboxes for each one, and set up the A/B campaigns, with their respective offers.

In doing so, we now have a total of 11 campaigns and 10 nested mboxes, with 12 content offers.

Now to the targeting of it all.

Targeting Expressions and Profiles

Interestingly, this turned out to be the hardest part.

The very first campaign needs to determine whether to ultimately display Undergrad content or Postgrad content.

So, as we have OR’s in our rules, we need to use an Expression Target (basically Javascript that sits in the T&T target rules and returns a true or false).  We have to use an expression because the rules at the campaign level are all AND-based rules…which don’t work for us.

So, we have two Expression Targets on the main top level campaign…one to determine Postgrad the other to determine Undergrad.

By way of explanation, the Undergrad target expression basically says:

1) Get the value of the last campaign that the user clicked on (that’s our campaign tags)
2) Return true if the users last:
a) coursecategory was Undergrad, or
b) tool used was Mid Year, or
c) LastMicrosite visited was Undergrad, or
d) campaign tag contained “undgrad”, or
e) the current URL query string contains a special one for us to test it with!

The same is basically true for the Postgrad target, just different values.

These two expression targets are then used in the campaign, highlighted below.  Despite the fact that it says “…is present”, what it actually means is “… is true”.  Never understood why that is, but there we go.

Then we need to set up our Application Stage targeting – basically telling us if the user has Qualified for Entry and hasn’t started and app, or if they’ve started one but haven’t completed it yet.  So we added more targets as follows:

You’ll notice in both that there are a combination of both user and profile based rules.  User rules are script based and are set up in the Profiles area of T&T and profile based rules are values passed into T&T from an mbox.

We use a lot of mboxes across the site and pass quite a few profile parameters in through the mboxes – which makes it easier for us to target later.  All of our key stages have been pre-tagged with mboxes and various parameters.

The only one to use a user-based profile is our lastcampaign profile.  Anytime a visitor comes to our site and the visit contains a campaign tracking code, the code is automatically passed into Test & Target and saved into their user profile, via a script, shown below.

You’ll also notice some of the profile.parameters that have been passed in – note that you can’t edit them.

In testing all of this, I just couldn’t figure out why I always ended up in the repeat visitor campaign for either UG or PG.

The reason was easy to overlook, and worth mentioning:

I’d actually structured the order of offers back to front in the second level campaigns…stupid me, I did know this, I clean forgot when I created the campaigns (Omniture: it would be great though to be able to re-order experiences – but you can’t you have to delete and start again).  The reason being, you have to put the highly targeted ones at the top and the broadest match at the bottom, as T&T evaluates in the order that the experiences are displayed.  And thanks to Veronica at Omniture for spotting a misplaced apostrophe…they’ll get you everytime!

Testing everything

Yes, it’s a challenge – this many campaigns and offers means lots of new sessions, clearing of cookies, and re-establishing the stage of the user journey, but after a few hours, confirmation was established that everything was as it should be.

The biggest problem was that as we were serving back nested mboxes, we couldn’t preview the content in-situ as you would normally do through On-Site Preview, as it looks for the mbox on the page…and they don’t “exist” yet because they’ve not been served back yet.

But we go there.

Performance?

With all of these nested mboxes flying around, I was concerned by performance.  But, I was really happily surprised…it’s virtually undetectable.  Using WASP we’re able to see the order that the mboxes are served in and they match up correctly.

In Summary

So, there we have it.  Two behaviorally targeted campaigns, with triple backflip A/B twists thrown in.  The horse is back in the stable, the gates of hell are securely closed again and Rapture didn’t happen.

Test & Target came to the rescue and allows us to do what we need to do from a campaign standpoint.

I can sleep now – the campaign is live, you can experience the different offers by viewing the homepage, then a Postgrad course, then an Undergrad course, then repeating your visit later, or starting an app and not completing it.  If you feel so inclined…