There’s no excuse nowadays not to be optimizing across multiple channels.  There’s heaps of tools available from no cost tools to paid tools and there’s plenty of consultants willing to help you get out there and lead you down a path of optimization.

Just remember though, 90% of optimization is done by people, only 10% by the tool.  It’s people that create the optimization strategy, the ideas to test, the segments, the content, the creative and of course, the insights that come from the results – both good and bad.

So how do you know if free is ok or whether you should invest in something a little more robust?  It depends how serious you are about optimization.  It also depends on what analytic tools you run as well.  It depends on what you actually want to achieve.

And there’s quite a few distinct differences between Test&Target and Google Website Optimizer, which should make the decision a clear one for you if you’re serious about the whole thing.

This is just a sample of the differences…as I was writing this, I came to the conclusion that there are in fact too many to list (such as mobile testing, in-flash testing, ad network testing), so I tried to focus on what I think are the key differences – the ones that in my opinion, will make or break your venture into optimization.

A/B Testing

Both offer it but with big differences.

Google offers it as a landing page redirect, which also means that you need to create two different versions of a full page and host both pages.  Of course, once the test is complete, you’ll need to do something to redirect the tested page to the correct URL.  There is virtually an unlimited amount of content variations you can create though.

Test&Target offers both Landing Page redirects (similar to Google’s A/B) and an in-page A/B test, again using virtually an unlimited number of content variations.  With a T&T A/B test you don’t need to create new versions of the page to test and don’t have the hassle at the end of the campaign.

MVT Testing

Both offer it.

With GWO, their MVT test is more similar to T&T’s A/B test in that you create multiple variations of the content and use multiple content areas defined on the page. 

T&T’s MVT test has multiple content regions that you vary the content in.  The biggest difference is the testing methodology…see method of testing below.

Landing Page (and content rotation)

GWO uses the A/B version (redirect) test to do this, locking the visitor into a specific piece of content.

T&T takes a different approach, of which there are two flavours.  One is a landing page redirect, where you can test complete pages.  The other is a Landing Page Campaign.  The big difference here is that with a landing page campaign, the visitor is not locked into a single offer displayed – they can experience different offers as their profile changes (see profiles below).  A visitor might see different content during the day than at night because of trends (shopping from work or home).  They might see different offers based on the affiliate they came in from, or from the type of search they conducted.  These are typically used for things like Landing Page Optimization using keyword reinforcement.   With GWO you need multiple pages created to achieve this.  T&T, 1 page, 1 campaign.

Method of testing

Here’s a big difference.

GWO uses full factorial.

T&T uses Taguchi.

Full factorial means that your test must include every permutation of your variables. For example, if you are testing 4 headlines, 4 hero shots and 4 call to action colors, your full factorial is 4x4x4, or 64 variations. That requires a lot of traffic to be able to complete the test with statistical validity in a reasonable time frame. GWO does not support the Taguchi Method, which allows you to include the most options with the least amount of testing and generates results far quicker, with less traffic, than either full factorial or an A/B test.

Segmentation Targeting

Here’s a massive one.  And to be honest, I don’t think you should be testing if you can’t segment your tests.  In my opinion, it doesn’t provide you much of an insight.

With GWO it’s all or nothing.  Everyone sees the campaign (depending on the % of traffic you’ve sent to the campaign).

With T&T you can target specific segments of traffic, like first time visitors, or visitors who’ve purchased, or visitors who come from an organic brand search term.  You can target on URL, you can target on page query string, on country, by browser or device, by social network or other referring domain, by campaign, by behaviour, such as visits of more than 10 minutes, or more than 5 pages or, in fact by a whole suite of custom rules that you can create.

Segmentation will be required when you want to gradually “roll out” new tests, like a site redesign to only new visitors at first. Existing visitors have probably learned your existing design, and the new one could actually reduce conversion because it requires new learning on behalf of the user.  Something to really consider.

Targeting Rules

Again a big difference in capability.

GWO has no targeting – period.

In T&T you can create “targeting expressions” which are JavaScript based rules within the engine to determine whether someone meets the targeting criteria or not.  If they do, either enter them into the campaign, or show them a particular variation.  Really good for testing just those who have viewed a product but not purchased it yet.

User Profile parameters

Another big difference.

GWO – sorry, doesn’t exist, there’s no targeting capability, so it doesn’t need them.

T&T on the other hand allows you to pass data into the platform through the tags and further enhance the visitor profile stored by the platform so you can target variations of content to the visitor.  For example, these can be used when you want to target content based on a visitors category affinity, or based on their total spend – maybe give them a really special offer.  (Ok this is really behavioural targeting).

Traffic Throttling

Both offer this, but to varying degrees.

With GWO you select the % of traffic you want to enter into the campaign. GWO will then faithfully serve test pages equally to the percentage of traffic you specify. If you have a strong performing page that you want to make even stronger, you actually risk losing money during the test period if your challenger variations are not strong performers. Sending only a portion of traffic to test variations, for example, can reduce this risk. For example, you may choose to send 70% to your control and 30% to your challenger, or 50% to your control and 25% to your two challengers.

With T&T you can select it at the campaign, the targeted content region, and the content variation.  Lots of flexibility.  And for good reason. 

Conversion tracking

Both offer this, but again, radically different in their capabilities.

With GWO you select the page that the user must visit, combined with their code, signifies a conversion has occurred.  A count of.  Yes, you can add the same code to multiple pages so that you can count a conversion when someone visits any of the pages, providing the code is the same.

With T&T you have multiple options.  It could be a click to the content; it could be a specific page with a tag on it, or one of a myriad of SiteCatalyst success events that you’re already tracking.  Many options enabling deeper insight into conversion tracking.  Couple this segment tracking and conversion success metrics and you have very powerful insights.

Conversion Success

This is critical in my opinion. 

With GWO you only get the conversion tracking above.

With T&T you get to see conversion success points along the way.  For example, you can use success metrics already tracked through SiteCatalyst to monitor such things as leads or sales etc; you can use the targeting expressions or the user profile parameters, or any of the other segmentation based options.  Very handy for figuring out if your campaign lifted conversions but your average order size tanked in the process.

Segment Success

The whole thing about testing is that we, as visitors, do things differently.  So looking at how different segments respond to content variations is fundamental.

With GWO, there is no option for this.
With T&T you can add virtually an unlimited number or combination of segments to monitor how they’re responding.  Again, not much good to have a winning variation if your repeat visitors suddenly decide to spend much less on average than your new visitors, as a result of the campaign.  Your campaign conversion might have lifted but your product managers will want to know why their bottom line has been decimated.

Multiple Live Campaigns

Not really. 

In GWO you can only have multiple live campaigns if they’re different content areas.  You can’t run different campaigns targeting the same content area at the same time.  That means you can’t support your marketing departments campaigns without multiple pages.

T&T allows you to do this.  Based on the campaign entry rules that you define, you can run multiple campaigns into the same content areas.  Why would you want to do this?  So many reasons.  Many companies run multiple simultaneous campaigns.  If you have a modularized design (most likely) then your homepage is likely to feature certain campaign-based areas.  You need to target those campaigns to different segments who meet certain criteria.  With T&T you can.

Nested content

I’m not aware that GWO can do this.

T&T can for sure.  You might be wondering why you’d want this.  Probably for the more advanced, but it goes back to the multiple live campaigns, and content re-engagement when combined with landing page optimization.  Gets quite tricky, but definitely possible.

Campaign Preview

With GWO, sorry that’s a no-go.

With T&T use Onsite to preview what your variations will look like on the page, before you go live.  Allows you to validate your test before you go live.  Pretty handy.

Tag based

Yes both are tag based.  Both require you to put tag onto the page. 

T&T uses a <div> tag coupled with a lightweight piece of JavaScript available to any future test, whereas GWO uses fairly heavy JS, customised for this particular test only.

Reporting

I’ve saved what I consider to be one of the biggest challenges for the end.

GWO gives you conversion reports.  That’s about it.

T&T gives you a whole suit of reports within the T&T interface.  Plus, if you’ve integrated with SiteCatalyst, then all of your metrics are available to you within SiteCatalyst. 

Summary

Herein lies the key.  If you’re optimizing for conversion, then you need to fully understand the impact of your testing.  If you don’t you’re flying blind.  What looks like a good result from a conversion standpoint, may in fact be the worst performing result you could have imagined, because you didn’t segment your results, or you didn’t apply all of your other analytic metrics to the results, such as AOV, or Revenue, or Leads etc.

Optimization unfortunately should not to be taken lightly.  There are many factors involved in understanding whether something was successful or not – you cannot just take a conversion lift at face value.  You might have impacted something else…or you might have seen a massive lift in something else, in which case, you’re a total hero when you report it up the chain.

If you haven’t bought into a paid tool and you’re just getting your feet wet with testing, there’s probably nothing wrong with getting to grips with some of the concepts using Google Website Optimizer.  But, in doing so, you also need to understand it’s limitations and the limitations it will inherently place on your testing and results.

However, if you’re really serious about using optimization as part of your overall digital strategy, and understand that in today’s day and age, digital optimization and behavioural targeting is a fact of life, then you should be investing in paid tools, which have the full capability to support your needs.

Of course, more important than the tools are the brains behind your optimization program. 90%.

I thought I’d share a real humdinger with you.

It’s campaign time again.  Not just any old campaign, it’s our main recruitment campaign of the year.

What we normally do on this campaign is behaviorally target content to users based on their application stage.

Why?  Because we know from previous tests that behaviorally targeting content for re-engagement purposes not only lifts our application completion rate, but provides more relevance to the user when they visit our site – instead of just seeing a standard campaign message each time.

Given the value of an application, and the lift we see, it’s well worth the effort to go down this path.

But this time, the gates of hell opened and out rode the fifth horseman.

We typically use 4 different stages, with one experience (content) per stage.  The challenge we always faced was that for each stage, we always wanted to run A/B tests to see which converted better – but could never figure out how to do it when it’s coupled with a behaviorally targeted campaign.

A few months ago, I worked with one of the Omniture Test & Target genius rock stars, implementation engineer Randall, who worked through a nested mbox example with me, which could be used to nest A/B tests within a single Landing Page campaign.  Unfortunately at the time we didn’t get to implement anything (or even try it – but it looked great on paper).

Knowing that our primetime Mid Year recruitment campaign period was nearly on us, I dusted off the email and had a good ‘ole read, as I felt it had some opportunities for us in a variety of ways.

And so the madness begins…

Always up for a challenge, I strapped on my Test & Target boots, and suggested that we could perhaps, possibly, maybe, ‘ish, try, have a crack at, putting together a behaviorally targeted campaign, where at each stage of the process we not only change the offer (easy) but run an A/B test (difficult) to test how well different offers convert people to the next stage.

Be careful what you wish for…because not only did they like that idea, they compounded it by throwing in another campaign on top of that one.

Mid Year campaign is normally restricted to Undergrad recruitment.  At least, that’s been the (now) historical position.

But no, this year, we’re doing simultaneous Undergrad and Postgrad Mid Year campaigns.

Yikes!!!  Go big or go home!  In for a penny, in for a pound.

So, to summarize, we want a simultaneous 4 stage behaviorally targeted campaign with a series of A/B tests within them.  And we need to switch users between campaigns based on whether they meet certain criteria, or whether they’ve viewed specific types of content.

Hmmm…a real humdinger!

Test & Target to the rescue

It gets a bit tricky to do, so I’ll do my best to explain it.

Before I do though, here’s an illustration of what was actually created.  Each of the rectangles with the little red circles is a separate Test & Target campaign.  You’ll see there are 11 of them.  The final bottom row is content…you’ll see there are 12 of them.  And you might also notice a total of 10 nested mboxes!

In a bit more detail, we’ve ended up with:

• 1 primary campaign as a Landing Page campaign that switches between an UG or PG experience, with a default offer for those we know nothing about.  The UG or PG experience then calls on (via a nested mbox):
• 1 of the 2 recruitment campaigns (UG or PG) as Landing Page campaigns so we can target one of the four  stages of the application, via nested mboxes, which calls on:
• 1 of the 8 A/B..n campaigns to determine the offer displayed

There’s a total of:

• 10 nested mbox offers…
• 12 content offers (the things people actually end up seeing)…
• 5 expression targets…
• And 6 profile targets…

Holy moly, I can hear you say.  Why so complicated?  It’s complicated because of the need to switch people between the campaigns based on what content and stage they’re at.

So, how does it all work?

The top campaign:

As we want to target initially by user interest (they’ve previously viewed Undergrad or Postgrad content, or they’ve visited an Undergrad or Postgrad course, or they’ve visited the Undergrad microsite or Postgrad site, or they’ve come in from an Undergrad of Postgrad email campaign, or they’ve visited the Figure Out Your Course tool or the Postgrad tool), we need to be able to get them into the relevant Undergrad or Postgrad campaign.

So, we need to use a Landing Page Campaign, which is set to re-evaluate the rules (and hence the offers) each time it displays.  This allows a user to see different experiences each time, based on the rules above.  It’ll switch a user to the opposite campaign if they go, say, from an Undergrad course to a Postgrad course.

But, we don’t want to show them an offer just yet.  We’re only doing this to determine which campaign they need to get into – and change that decision on the fly.

Hello nested mbox

On our home page, we already have an mbox (marketing box) defined, with default content.  So we start off by creating a campaign to use that mbox.

The following shows the top level campaign, targeting the default mbox on the page – HP_lowerleft.

Instead of serving content back, we want to serve back a different campaign (Undergrad or Postgrad), so we actually need to serve back a nested mbox as our offer.

A nested mbox is just an mbox served into an existing mbox, which then makes another call to Test & Target to get more content.

So, if I meet the criteria for Undergrad, then the content offer that Test & Target serves back to me contains a new mbox.  The content offer looks like:

<div class="mboxDefault">
</div>
<script type="text/javascript">
mboxCreate('UndergradNested');
</script>

I repeat that type of content offer for the Postgrad campaign offer, but call the other mbox ‘PostgradNested’.

From a conversion standpoint, we set the Conversion as an SiteCatalyst:event…meaning that whatever the user does, the campaign thinks they’ve converted, and as they restart with the same experience, it re-evaluates them each time, to determine which offer to show them.

That’s campaign #1.

We’ll call it the Mid Year Parent Campaign – and it’s the one at the very top of the illustration.

Now that I have that, I need to create two sub campaigns; one for Undergrad, the other for Postgrad. These campaigns will serve something back into the UndergradNested mbox, or the PostgradNested mbox.

Main Sub Campaigns

So, as I also want to serve back behaviorally targeted content, based on their stage of application, I need to repeat the above type of scenario, but have the experiences based on the users stage.

Stage of application

Once again, a Landing Page campaign allows a user to see different experiences each time, based on the rules evaluated.

So, we created another Landing Page campaign, with 4 offers, each offer targeting the stage that the user has completed:

1. They’ve started an application but haven’t completed it yet
2. They’ve qualified for entry, but haven’t started an application yet
3. They’re a repeat visitor and haven’t qualified yet
4. They’re a new visitor and haven’t qualified yet

But, again, we don’t want to show them an offer just yet – we want to show an A/B test for offers 3 & 4, and different content for offers 1 & 2.  So, again, we serve back a nested mbox offer, that aligns with the users stage.  In the example below, I’ve shown the nested mbox content for Experience D:

<div class="mboxDefault">
</div>
<script type="text/javascript">
mboxCreate('1stTimeVisNested');
</script>

A/B..n Sub-Campaigns

Now that I have the Undergrad main behaviorally targeted campaign created, I need to create the 4 sub-campaigns which will serve back real content that the user will see.

So, I create a new A/B..N campaign, target the mbox 1stTimeVisNested created above, and create 2 different experiences for it.  That’s the A/B test for the 1st Time Visitors who haven’t qualified.

I repeat that for Repeat Visitors who haven’t qualified, but this time I target the campaign at the mbox RepeatVistNested.  This one also has two offers (the A/B test for this campaign).

Then I create two more A/B campaigns which will serve back content into the respective mbox defined during the other two stages of the application.

Complicated I know – hope you’re still with me.

Rinse and Repeat

Once all that’s been created, it’s a fairly simple matter to duplicate each campaign for a Postgrad version, modify the offer to create new distinct mboxes for each one, and set up the A/B campaigns, with their respective offers.

In doing so, we now have a total of 11 campaigns and 10 nested mboxes, with 12 content offers.

Now to the targeting of it all.

Targeting Expressions and Profiles

Interestingly, this turned out to be the hardest part.

The very first campaign needs to determine whether to ultimately display Undergrad content or Postgrad content.

So, as we have OR’s in our rules, we need to use an Expression Target (basically Javascript that sits in the T&T target rules and returns a true or false).  We have to use an expression because the rules at the campaign level are all AND-based rules…which don’t work for us.

So, we have two Expression Targets on the main top level campaign…one to determine Postgrad the other to determine Undergrad.

By way of explanation, the Undergrad target expression basically says:

1) Get the value of the last campaign that the user clicked on (that’s our campaign tags)
2) Return true if the users last:
a) coursecategory was Undergrad, or
b) tool used was Mid Year, or
c) LastMicrosite visited was Undergrad, or
d) campaign tag contained “undgrad”, or
e) the current URL query string contains a special one for us to test it with!

The same is basically true for the Postgrad target, just different values.

These two expression targets are then used in the campaign, highlighted below.  Despite the fact that it says “…is present”, what it actually means is “… is true”.  Never understood why that is, but there we go.

Then we need to set up our Application Stage targeting – basically telling us if the user has Qualified for Entry and hasn’t started and app, or if they’ve started one but haven’t completed it yet.  So we added more targets as follows:

You’ll notice in both that there are a combination of both user and profile based rules.  User rules are script based and are set up in the Profiles area of T&T and profile based rules are values passed into T&T from an mbox.

We use a lot of mboxes across the site and pass quite a few profile parameters in through the mboxes – which makes it easier for us to target later.  All of our key stages have been pre-tagged with mboxes and various parameters.

The only one to use a user-based profile is our lastcampaign profile.  Anytime a visitor comes to our site and the visit contains a campaign tracking code, the code is automatically passed into Test & Target and saved into their user profile, via a script, shown below.

You’ll also notice some of the profile.parameters that have been passed in – note that you can’t edit them.

In testing all of this, I just couldn’t figure out why I always ended up in the repeat visitor campaign for either UG or PG.

The reason was easy to overlook, and worth mentioning:

I’d actually structured the order of offers back to front in the second level campaigns…stupid me, I did know this, I clean forgot when I created the campaigns (Omniture: it would be great though to be able to re-order experiences – but you can’t you have to delete and start again).  The reason being, you have to put the highly targeted ones at the top and the broadest match at the bottom, as T&T evaluates in the order that the experiences are displayed.  And thanks to Veronica at Omniture for spotting a misplaced apostrophe…they’ll get you everytime!

Testing everything

Yes, it’s a challenge – this many campaigns and offers means lots of new sessions, clearing of cookies, and re-establishing the stage of the user journey, but after a few hours, confirmation was established that everything was as it should be.

The biggest problem was that as we were serving back nested mboxes, we couldn’t preview the content in-situ as you would normally do through On-Site Preview, as it looks for the mbox on the page…and they don’t “exist” yet because they’ve not been served back yet.

But we go there.

Performance?

With all of these nested mboxes flying around, I was concerned by performance.  But, I was really happily surprised…it’s virtually undetectable.  Using WASP we’re able to see the order that the mboxes are served in and they match up correctly.

In Summary

So, there we have it.  Two behaviorally targeted campaigns, with triple backflip A/B twists thrown in.  The horse is back in the stable, the gates of hell are securely closed again and Rapture didn’t happen.

Test & Target came to the rescue and allows us to do what we need to do from a campaign standpoint.

I can sleep now – the campaign is live, you can experience the different offers by viewing the homepage, then a Postgrad course, then an Undergrad course, then repeating your visit later, or starting an app and not completing it.  If you feel so inclined…

In many cases homepages are either relatively static, or promotional driven.  The problem is that homepages are often still the starting point of a users journey on the site and not every user should see the same content.

In our case, at Murdoch University, despite the fact that we use SEO and SEM tactics to drive clicks to deeper content, we know from our metrics that many users either bookmark our homepage, or search for “Murdoch University” or a derivative thereof, which means that they click through to our homepage.

The role of the homepage is to channel users into areas of the site as quickly as possible.  With many different audience groups, numerous campaigns, and many stakeholders, real estate is highly sought after.  So it’s crucial that we are able to address content relevance – make the content on the homepage as relevant to what the user is looking for when they visit, because relevance yields greater conversion.

Enter Omniture Test and Target.

Using Test and Target, we’re able to easily modify the content displayed on the homepage (and many other areas of the site), by using their sophisticated behavioral targeting technology, thereby making our content far more relevant to users when they visit, with the goal of optimizing their experience, and ultimately leading to more sales.

For example, if a user has previously been to our site, started an application but not yet completed it, why point them to the content that talks about how to apply, when you can prompt them to complete their application.

Likewise, if they have expressed an interest in a certain category of information, get them back into that stream with as few clicks as possible.

As Test and Target stores a profile of an individuals browsing activity on our site, coupled with SiteCatalyst data, and various parameters that we set on specific events throughout our site, we can use that as a kind of category or activity affinity, and alter content display based on those parameters, on a user by user basis.

For example, our homepage has 4 key modules on it.  Currently the two left modules are T&T driven.  The left hand module is promotional based.  So we’re currently running a Postgraduate promotion.  If the user has been to a PostGraduate course previously, they will see the PostGraduate promotion.  If they have been to an Undergraduate course, they’ll see our Course Search module.

For the next module, it shows either a Future Students module (default), a Domestic Students module, or an International Students module.  If the user has been into the Domestic Students section, then in the future, when they see the homepage, they’ll see the Domestic module.  Likewise, if they are an International visitor, or they visit the International section, when they see the homepage, they’ll see the International module.  If we know nothing about them, i.e. they are within Australia but haven’t been to the site before, they’ll see the Future Students module.

Implementing Test and Target in this manner is very easy.  There are a number of key parts:

1. Put the code onto the page in the area where you want to display the content – this code is called an mbox.  It’s a combination of a <div> tag and a piece of javascript.
2. Put some default content inside the <div> tag.  This is displayed is nothing else can be displayed.  For us, the Course Search and the Future Students content is default.
3. Put mbox code onto other key pages and set specific parameters within those mboxes.  I call these “listeners” – they don’t actually display any content, they just pass critical information into the user profile when they are activated, such as on key pages, or key events like starting or completing an application (see below).
4. Put some alternative content into Test and Target (called offers).
5. Create targeted campaigns in Test and Target with rules about what content should be displayed (either default or the content created in step 4).
6. Activate the campaign and hey presto – behavioural targeting now works for you.

Ok, well, there might be a bit missing, like your content and targeting strategy, but you get the drift.

Category and Activity Affinity

Throughout our site, we pass a number of parameters through the “listener” mboxes, which are placed on strategic pages.  For example, in the above Domestic and International sections of the site, we have created a parameter called profile.sitesection and we pass in a value of either Domestic or International.  We can also pass in other values, such as “Research”, “Library”, “Student” or “Staff” on other pages, so that we can expand on the targeted content.  The strategy largely depends upon who

Our T&T campaign that we created in step 5 then looks for either of those values and depending on the value, displays the relevant content.  If it doesn’t find a value, it serves the default content – the Future Students content.

Another area that we utilize this is within our online application system.  We pass in a value when they start an application and another when they complete an application.  This allows us to look for those values, and depending upon a combination of them, we know the status of an application (not started, started not completed, started and completed) and can then display relevant content to either engage them to start an app, prompt them to complete an app, or other default content.

Within T&T we can also set conversion events.  These events are mostly used when you are testing different variations of promotions (I’ll post an entry about testing content in the near future) to see which one drives the best result.  For behavioral targeting we’ve found that we use these more for general reporting, rather than actual optimization.

Practical usage ideas

Content relevance is effective for every industry vertical.  For example:

• ISP’s could use it to target content to customers with accounts, versus potential new customers.
• Banks could use it to target product category affinity – those that might have expressed an interest in mortgages versus credit cards.
• Retailers could use it to target product category affinity too – if a user expresses an interest in TV, show them TV products on the homepage over other products.
• Tourism operators could target by region of interest or by activity, such as kayaking, or fishing.
• Media companies could use it to display news from a certain category, for example, finance news over fashion news.

One of the most successful strategies is re-engagement; get an abandoned user back into a process quickly.  You know they’ve abandoned; they’re back on your site; re-engage them and try to convert them.

Like everything though, it’s important that the content strategy is thought through first and the objectives and success measures are clearly defined, before trying to implement.