There’s no excuse nowadays not to be optimizing across multiple channels.  There’s heaps of tools available from no cost tools to paid tools and there’s plenty of consultants willing to help you get out there and lead you down a path of optimization.

Just remember though, 90% of optimization is done by people, only 10% by the tool.  It’s people that create the optimization strategy, the ideas to test, the segments, the content, the creative and of course, the insights that come from the results – both good and bad.

So how do you know if free is ok or whether you should invest in something a little more robust?  It depends how serious you are about optimization.  It also depends on what analytic tools you run as well.  It depends on what you actually want to achieve.

And there’s quite a few distinct differences between Test&Target and Google Website Optimizer, which should make the decision a clear one for you if you’re serious about the whole thing.

	GWO	T&T
A/B Testing	Yes*	Yes
MVT Testing	Yes	Yes
Landing Page	Yes	Yes
Content Rotation	No	Yes
Method of Testing	Full Factorial	Taguchi
Segmentation Targeting	No	Yes
Targeting Rules	No	Yes
User Profile parameters	No	Yes
Traffic Throttling	Yes	Yes
Conversion Tracking	Yes	Yes
Conversion Success	No	Yes
Segment Success	No	Yes
Multiple live campaigns	Partial	Yes
Nested content	No	Yes
Campaign preview	No	Yes
JavaScript Tag based	Yes	Yes
Reporting	Limited	OMG, yes

This is just a sample of the differences…as I was writing this, I came to the conclusion that there are in fact too many to list (such as mobile testing, in-flash testing, ad network testing), so I tried to focus on what I think are the key differences – the ones that in my opinion, will make or break your venture into optimization.

A/B Testing

Both offer it but with big differences.

Google offers it as a landing page redirect, which also means that you need to create two different versions of a full page and host both pages.  Of course, once the test is complete, you’ll need to do something to redirect the tested page to the correct URL.  There is virtually an unlimited amount of content variations you can create though.

Test&Target offers both Landing Page redirects (similar to Google’s A/B) and an in-page A/B test, again using virtually an unlimited number of content variations.  With a T&T A/B test you don’t need to create new versions of the page to test and don’t have the hassle at the end of the campaign.

MVT Testing

Both offer it.

With GWO, their MVT test is more similar to T&T’s A/B test in that you create multiple variations of the content and use multiple content areas defined on the page. 

T&T’s MVT test has multiple content regions that you vary the content in.  The biggest difference is the testing methodology…see method of testing below.

Landing Page (and content rotation)

GWO uses the A/B version (redirect) test to do this, locking the visitor into a specific piece of content.

T&T takes a different approach, of which there are two flavours.  One is a landing page redirect, where you can test complete pages.  The other is a Landing Page Campaign.  The big difference here is that with a landing page campaign, the visitor is not locked into a single offer displayed – they can experience different offers as their profile changes (see profiles below).  A visitor might see different content during the day than at night because of trends (shopping from work or home).  They might see different offers based on the affiliate they came in from, or from the type of search they conducted.  These are typically used for things like Landing Page Optimization using keyword reinforcement.   With GWO you need multiple pages created to achieve this.  T&T, 1 page, 1 campaign.

Method of testing

Here’s a big difference.

GWO uses full factorial.

T&T uses Taguchi.

Full factorial means that your test must include every permutation of your variables. For example, if you are testing 4 headlines, 4 hero shots and 4 call to action colors, your full factorial is 4x4x4, or 64 variations. That requires a lot of traffic to be able to complete the test with statistical validity in a reasonable time frame. GWO does not support the Taguchi Method, which allows you to include the most options with the least amount of testing and generates results far quicker, with less traffic, than either full factorial or an A/B test.

Segmentation Targeting

Here’s a massive one.  And to be honest, I don’t think you should be testing if you can’t segment your tests.  In my opinion, it doesn’t provide you much of an insight.

With GWO it’s all or nothing.  Everyone sees the campaign (depending on the % of traffic you’ve sent to the campaign).

With T&T you can target specific segments of traffic, like first time visitors, or visitors who’ve purchased, or visitors who come from an organic brand search term.  You can target on URL, you can target on page query string, on country, by browser or device, by social network or other referring domain, by campaign, by behaviour, such as visits of more than 10 minutes, or more than 5 pages or, in fact by a whole suite of custom rules that you can create.

Segmentation will be required when you want to gradually “roll out” new tests, like a site redesign to only new visitors at first. Existing visitors have probably learned your existing design, and the new one could actually reduce conversion because it requires new learning on behalf of the user.  Something to really consider.

Targeting Rules

Again a big difference in capability.

GWO has no targeting – period.

In T&T you can create “targeting expressions” which are JavaScript based rules within the engine to determine whether someone meets the targeting criteria or not.  If they do, either enter them into the campaign, or show them a particular variation.  Really good for testing just those who have viewed a product but not purchased it yet.

User Profile parameters

Another big difference.

GWO – sorry, doesn’t exist, there’s no targeting capability, so it doesn’t need them.

T&T on the other hand allows you to pass data into the platform through the tags and further enhance the visitor profile stored by the platform so you can target variations of content to the visitor.  For example, these can be used when you want to target content based on a visitors category affinity, or based on their total spend – maybe give them a really special offer.  (Ok this is really behavioural targeting).

Traffic Throttling

Both offer this, but to varying degrees.

With GWO you select the % of traffic you want to enter into the campaign. GWO will then faithfully serve test pages equally to the percentage of traffic you specify. If you have a strong performing page that you want to make even stronger, you actually risk losing money during the test period if your challenger variations are not strong performers. Sending only a portion of traffic to test variations, for example, can reduce this risk. For example, you may choose to send 70% to your control and 30% to your challenger, or 50% to your control and 25% to your two challengers.

With T&T you can select it at the campaign, the targeted content region, and the content variation.  Lots of flexibility.  And for good reason. 

Conversion tracking

Both offer this, but again, radically different in their capabilities.

With GWO you select the page that the user must visit, combined with their code, signifies a conversion has occurred.  A count of.  Yes, you can add the same code to multiple pages so that you can count a conversion when someone visits any of the pages, providing the code is the same.

With T&T you have multiple options.  It could be a click to the content; it could be a specific page with a tag on it, or one of a myriad of SiteCatalyst success events that you’re already tracking.  Many options enabling deeper insight into conversion tracking.  Couple this segment tracking and conversion success metrics and you have very powerful insights.

Conversion Success

This is critical in my opinion. 

With GWO you only get the conversion tracking above.

With T&T you get to see conversion success points along the way.  For example, you can use success metrics already tracked through SiteCatalyst to monitor such things as leads or sales etc; you can use the targeting expressions or the user profile parameters, or any of the other segmentation based options.  Very handy for figuring out if your campaign lifted conversions but your average order size tanked in the process.

Segment Success

The whole thing about testing is that we, as visitors, do things differently.  So looking at how different segments respond to content variations is fundamental.

With GWO, there is no option for this.
With T&T you can add virtually an unlimited number or combination of segments to monitor how they’re responding.  Again, not much good to have a winning variation if your repeat visitors suddenly decide to spend much less on average than your new visitors, as a result of the campaign.  Your campaign conversion might have lifted but your product managers will want to know why their bottom line has been decimated.

Multiple Live Campaigns

Not really. 

In GWO you can only have multiple live campaigns if they’re different content areas.  You can’t run different campaigns targeting the same content area at the same time.  That means you can’t support your marketing departments campaigns without multiple pages.

T&T allows you to do this.  Based on the campaign entry rules that you define, you can run multiple campaigns into the same content areas.  Why would you want to do this?  So many reasons.  Many companies run multiple simultaneous campaigns.  If you have a modularized design (most likely) then your homepage is likely to feature certain campaign-based areas.  You need to target those campaigns to different segments who meet certain criteria.  With T&T you can.

Nested content

I’m not aware that GWO can do this.

T&T can for sure.  You might be wondering why you’d want this.  Probably for the more advanced, but it goes back to the multiple live campaigns, and content re-engagement when combined with landing page optimization.  Gets quite tricky, but definitely possible.

Campaign Preview

With GWO, sorry that’s a no-go.

With T&T use Onsite to preview what your variations will look like on the page, before you go live.  Allows you to validate your test before you go live.  Pretty handy.

Tag based

Yes both are tag based.  Both require you to put tag onto the page. 

T&T uses a <div> tag coupled with a lightweight piece of JavaScript available to any future test, whereas GWO uses fairly heavy JS, customised for this particular test only.

Reporting

I’ve saved what I consider to be one of the biggest challenges for the end.

GWO gives you conversion reports.  That’s about it.

T&T gives you a whole suit of reports within the T&T interface.  Plus, if you’ve integrated with SiteCatalyst, then all of your metrics are available to you within SiteCatalyst. 

Summary

Herein lies the key.  If you’re optimizing for conversion, then you need to fully understand the impact of your testing.  If you don’t you’re flying blind.  What looks like a good result from a conversion standpoint, may in fact be the worst performing result you could have imagined, because you didn’t segment your results, or you didn’t apply all of your other analytic metrics to the results, such as AOV, or Revenue, or Leads etc.

Optimization unfortunately should not to be taken lightly.  There are many factors involved in understanding whether something was successful or not – you cannot just take a conversion lift at face value.  You might have impacted something else…or you might have seen a massive lift in something else, in which case, you’re a total hero when you report it up the chain.

If you haven’t bought into a paid tool and you’re just getting your feet wet with testing, there’s probably nothing wrong with getting to grips with some of the concepts using Google Website Optimizer.  But, in doing so, you also need to understand it’s limitations and the limitations it will inherently place on your testing and results.

However, if you’re really serious about using optimization as part of your overall digital strategy, and understand that in today’s day and age, digital optimization and behavioural targeting is a fact of life, then you should be investing in paid tools, which have the full capability to support your needs.

Of course, more important than the tools are the brains behind your optimization program. 90%.

As I find myself sitting in Auckland International Airport, waiting 12 hours for my next flight to LA (it’s 2am Perth time so I’m not quite with it), I figured what better time to write about success event pathing, as apparently my success event pathing for this trip wasn’t quite optimised for connecting flights at reasonable hours.

One of the things that you want to know about your site is the order in which certain events happen.  If you’re using success events for various things, like tracking self-service transactions, or leads, purchases etc, it’s important to know the most popular order in which visitors interact.

Now, admittedly, if you’re lucky, you could get some of this through page pathing…but as they’re pages, you end up with a bunch of clutter in between and unless those key events happen almost simultaneously, you’ll struggle to get the information you need.

You could use the Path Finder if you’re on SiteCatalyst v14, but it v15 it’s no longer available.  You could use the Fall Out report to generate some insights, but again, it’s not optimal.

If you want to know which events people interact with after a specific event, or prior to a specific event, then you need to implement Success Event Pathing.

Quite simply, you just set an s.prop each time specific success events are set.  You pass the name of the success event into the s.prop and ask Client Care to enable pathing on the prop.

What could be easier.

In the following example, s.prop45 is out “Success Event Pathing” traffic prop.
/* Success Event Pathing */
if (s.events){
if (s.events.match(/event1,/)){s.prop45="Internal Search";
}else if (s.events.match(/event2,/)){s.prop45="Lead Start";
}else if (s.events.match(/event3,/)){s.prop45="Lead Complete";
}else if (s.events.match(/event4,/)){s.prop45="App Start";
}else if (s.events.match(/event5/)){s.prop45="Course View";
}else if (s.events.match(/event6/)){s.prop45="Tool Used";
}else if (s.events.match(/event10/)){s.prop45="Form Start";
}else if (s.events.match(/event11/)){s.prop45="App Complete";
}else if (s.events.match(/event12/)){s.prop45="Form Complete";
}else if (s.events.match(/event16/)){s.prop45="Social Share";
}else{}
}
First we check that s.events exist and if it does, then we’re looking to match certain event numbers.  If they match, then we set prop45 to be a common name for the success event.

Notice the commas in the some of the matches?  That’s because we almost always have event13 added to our event list (as a page velocity event) so it’ll likely contain a comma, and we need to not mix up event1 with event 12 etc.

So we end up with a pretty standard traffic report that looks something like this:

This obviously shows that Internal Search is the most popular thing to do (not surprising given this is an un-segmented report).  Once I segment it, to show only those that are not staff or students, it’s a little more in line with what we’d expect…course views being the top activity for that audience type.

Ok, so that’s just the basic traffic report.

We can see from the above traffic report that they are interacting with areas across the site, from internal search, to filling in forms, to applications etc…but we don’t know the order this all occurred in.

Next success event flow

Let’s take a look at what those visitors do after they’ve looked at a course, by opening the Pathing Next Page Flow report.

We can see that following course views, in many cases, no other activity is recorded (for this visit).  You have to remember that pathing is unfortunately not cross-visit.

Some of them do interact with search, become a lead etc, during the same visit.

Previous success event flow

Now if you open the Pathing Previous Page Flow report for this traffic variable, you’ll be able to see what success events led to a specific event…such as an Application (or a purchase).

What we see again is that Apps typically start on a new visit (unsurprising) and during this period (when applications are actually fairly low anyway), course views lead to apps (that’s good).  Internal search and leads also generate course views, which lead to apps.  So, the flow is good, the volume is ok.  And another insight that makes us happy is that internal search accounts for very few applications…meaning people are not having to search for something prior to starting an application…and that’s’ great news.

There we have it – the order in which these success events occurred, giving you the insight into what happens before conversions.

And as success events are the most important things you measure on your site, it’s a logical step to create pathing on them to understand that order, so that you can optimise things further.

Now, where’s that coffee shop?