OMG – have you seen the new Discover?  If not, why not?  If so, you’re probably in heaven and I completely understand.

Discover v3 is the latest release of the ad-hoc analytics software from Adobe – and I have to say, Tim Lott and the team over there have done a fantastic job with this release, opening up capabilities that, once you start, you’ll wonder how you ever lived without them.

So, what’s all the fuss about?

Discover 3 takes the concept of visualising data to the next level.  Trends, funnels, paths, site analysis, pivoting tables – you name it, it’s got it, all updated – and it does it across multi-visit.  Yes, no longer are you limited to analysing within the same session – Discover 3 now brings cross-visit analysis to town.

Not only that, but the biggest and best feature is heterogeneous pathing.  Hetero what…?  Had to look that one up myself.  I think that means “a bunch of different things all connected in diverse ways”.  Anyway, for us that means we can now use any dimension, prop, evar, classification, and/or events together when looking at the flow of user journeys and conversions, to help us better understand the full story of what’s happening across our sites.

And it comes with a fancy, dark and broody new interface – all designed to make it easier on your eyes as you’ll be spending even more hours uncovering all of those nuggets and story-telling to the nth level.

There are way too many new features to talk about because the whole thing has changed on its head – and this is not meant to be a training piece on the product (there’s a whole heap of awesome new videos in SiteCatalyst Help to watch if you want lots more information).  But, I’ve got some key examples below that we’ve found to be very useful at answering a specific question (or two) that D2 couldn’t help us answer.

Figure out Fallout – across anything really

So, in the old Discover, your fallout reports would only ever look at pages.  You’d select the pages, drag them on to the canvas, and you’d get the visits that followed that path, with each fallout step between.

No more.

Now you can generate fallout reports on any Event or Dimension of data.  Yes.  Events AND Dimensions.  Oh my god.

Want to know how many people went from one section of your site (maybe you don’t specifically want to know the section pages), through a specific page, or type of product, then conducted a search, and then converted?

Now you can.  And you can see it at the Visit level (same session) or at the Visitor level (cross-session).

In the above example, I’m looking at the number of visitors that went across the Future Students section of the site (s.prop) then went through the Figure Out Your Life tool (another s.prop), then viewed any Health Sciences course (SAINT classification from a product string), then started an application and finally submitted the app (two success events).

These are checkpoints – and the visitors did a lot in between.  For example, if you click on the little green arrow between the two checkpoints, it shows what the “things” that they did before getting to your next designated checkpoint.  Noticed I called them “things”…that’s because you can look at anything.

For example, above, if I look at the Fall Through from the section mu:fs, and see what other sections visitors went to before the tool, then I click on the green arrow, which then shows:

And we can see that the next section some of them went to was Courses.  So, how about I want to see what Courses, or what Products they viewed…?  Just change the drop down and pick something else.

How cool is that?  I can now not only see that some people went into Courses, but looked at a Health Sciences course, before they returned to the Figure Out Your Life tool.

And I’m looking at things across multiple visits – so even if they came back and didn’t follow this path to complete it, they’d still appear in this fallout report.

Visualise user behaviour – across anything really

One of the really great features of Discover is the Site Analysis capability – I’ve been a fan of that feature for many years now.  And with Discover 3, it’s been taken to a whole new level as well.

Once again, D2 would only allow you to visualise site pages – the movement of traffic between those pages.  And, to be honest, it got a little difficult to do it with some of the later upgrades to D2.

D3 – scrap that.  Visualise any Event, Prop, eVar, Classification, at the Visit level or the multi-visit level.  And don’t forget to apply your segments too.

The example above helped us to answer one of those “that’s been bugging the crap out of me forever” type of question that someone will inevitably ask of you.  For us, it was around the level of interaction between a certain tool on our site, with respect to a particular segment of audience, and how they get to their final destination.  Discover 3 allowed us to very quickly analyse user interactions by simply dropping the respective props, eVars and success events onto the canvas, apply the segment, and see what those users are doing.

Fantastic result.  This visual type of analysis is so important to understand things at macro levels before deep diving into micro levels.  It visually tells a story that you can immediately connect with.

But wait, there’s more.

“If you call within the next 10 minutes, we’ll talk about not one, not two, but three new features.”

Ranking data

What a change.  Totally new way to do this and great improvements on both usability and time savings.

Remember in D2 where you had a bunch a metrics and bunch of segments.  For instance, you might be looking at Paid Search, Organic Search and Direct Traffic as your segments, and within each one, Visits, Revenue and Orders.

In D2, your metrics columns wouldn’t align – so you wouldn’t have all of your Orders together.

In D3, you can now change the way it displays. You build it all first in the new Table Builder, then click Replace Table, and off it goes and does everything.  So, the traditional report would be Segments of Traffic, showing Visits, Page Views etc, against our Dimension, in this case Section of the site.

But notice how the Visits, Page Views etc are discreet within their own segment…

No problem…in the table builder, drag the Metrics above the Segments, hit Replace Table and:

…and now your columns of data match up – at the segment level, not at the Metric level.  Much easier to work with.

Not to mention you can now pivot data entirely, getting your segments down the left and your dimensions across the top.  And it won’t let you build something if it’s wrong (notice the little green circles with ticks in them).

And you can drag on all of your different breakdowns too – drag them into the box and when you hit Replace Table – you guessed it.

Building segments has changed a little bit too.  You can now swap Visitor, Visit and Page level containers without having to re-create the bits within it – you just re-designate them as Visitor level or Visit level.  Much easier.

Work over the new year

Oh, yes, finally analysis that crosses over the year divide.  Used to be a major issue for many companies.  No more.  Done.  Fixed.  Ok, that doesn’t seem like a big deal – but it was.  Huge.

Summary

Those are just a few of the incredible list of new features that the product team put together, and that we’ve been anxiously waiting to blog about since working with them throughout the product beta.

This is a really well rounded product release, containing lots of features that we’re all crying out for, features that are simply just not possible to achieve within SiteCatalyst.

So, if you’re serious about demonstrating the value of web analytics back to your business, then you should not be avoiding this product.  It adds value as soon as you turn it on, making you even smarter than you already are.

I’ve really only scratched the surface here – there’s so much more capability than this post can give justice to.

If there was ever a time to seriously look at Discover v3, well, in the analytics world, it’s the new black.

We’re just working with a client that uses Google Analytics for their measurement platform and as part of our overall strategic plan, we’ve done some supporting analysis for them, reviewing the last year with respect to a specific segment or two.

They get a surprisingly large amount of traffic for a WA business, which is especially significant when it comes to using Google Analytics.  On average, they get about 25 million page views per month – so, over the course of the year, that’s somewhere around 300 million page views.

During our supporting analysis, Google announced a new feature – controlling the sampling rate.  According to their blog post:

Control your report calculation
One way we speed up the serving of data is through what we call “fast-access mode”, which applies to reports generated from large data sets. In the coming weeks, we will be peeling back the curtain on how “fast-access mode” works and letting you control the number of visits used to calculate reports.

Out with the old: fast-access mode
If a report requires calculation on more than 250,000 visits, we select a statistically random sample of 250,000 visits and estimate the report results based on that data. This makes reports faster to load, and our testing indicates that the data returned is highly accurate.

In with the new: control your report calculation
Now you will have the ability to control the number of visits used to calculate your reports, and we inform you of exactly how many visits are used in report calculation.

Out of interest on this new feature, I decided to play around with the settings and look at differences in the reports and conversions, using different sampling rates, and the ultimate reason for the improvement being speed – and I was quite surprised by the results.

Using their slider, I changed the sampling effect from the default, which is in the middle, moving it towards the right, or Higher Precision.  Each time, I switched over to the conversion reports to see what the results were.

On the default setting, the Paid Non Branded search conversion rate was 10.32%, but at the extreme right, it was 4.95%.  Interestingly, while Paid Search conversion deteriorated, Organic Non Branded improved.

Likewise, the other conversion rates all “tightened up” as I progressed from the default to the higher precision.

Notice the size of the sample set though – from its default it’s using 2.6% of visits; at higher precision it’s still only using 5.15% of visits (500,000 our of 9.7 million) – and it was this that caused me some concern.

When I pushed it all the way to left, Faster Processing, the results were useless.  It was based on <0.01% of traffic; there were no conversions for Organic Non Branded, or Paid (both Org and NB).  So, while super quick, super inaccurate.

What are the real rates?

If we’re seeing such a variance in conversion rates from 2.6% to 5.15% of overall traffic, then I wonder what the actual conversion rates are, without sampling applied?  Unfortunately, we have no way of knowing, as far as I can tell.

While sampling certainly helps Google with returning results quickly, I think they should allow the opportunity to see what “actual” is, so that at the very least you can determine your margin of error across any report.  There’s absolutely no point in having really quick reports, when the results are misleading.

In the above conversion report, Paid Non Branded converts at 10% using the default setting, whereas in fact it’s less than half of that rate.

I would certainly recommend that if you are using Google Analytics, you look at the impact being applied to your reports through this sampling effect, especially if you are optimising conversions.

One of the things that’s not “native” to SiteCatalyst reporting is Time Spent by Referring Domain, which can be quite a useful segment to use.  So here’s a quick tip to get it working.

The report below shows the end result; Time Spent by Traffic Source (a new custom insight prop).  The entry marked as “direct” are visitors that had no referrer string.

Time Spent is enabled by pathing a traffic prop (custom insight).  So to get Time Spent by Traffic Source you need to do a couple of little things.

Step 1 – Create a Prop

In the admin, create a new traffic prop and an eVar (if you want conversions by this too).  I named mine Traffic Source (p) and Traffic Source (c) – just to not confuse them with the native Referring-based reports.

Then ask Client Care to enable pathing on the traffic prop.

Step 2 – Modify your s_code

The following snippet of code should be added into your s_code, along with all of your other rules.  I’ve used s.prop7 and s.eVar16 for my new Traffic Sources reports.

/* Set persisting referrer */
if(document.referrer){
s_trafficsource = document.referrer.split(‘/’)[2]; // get the host
    if(s_trafficsource != window.location.host){
        s.prop7=s.getAndPersistValue(s_trafficsource,’s_referrer’,7);
        s.eVar16 = s.prop7;
        }else{
        // same referrer as site
        s_prop7=s.getPreviousValue(s.prop7,’gpv_ref’);
        s.prop7=s.getAndPersistValue(s.eVar16,’s_referrer’,7);
        s.eVar16=s.prop7
        }
    }else{
    // no referrer – so direct
    s_trafficsource = "direct";
    s.prop7=s.getAndPersistValue(s_trafficsource,’s_referrer’,7);
    s.eVar16=s.prop7
    }
s.prop7=s.getAndPersistValue(s.eVar16,’s_referrer’,7);

You’ll see from the above that it also uses the getAndPersistValue and getPreviousValue plugins.  If you don’t have them they are available for download through the Help section of SiteCatalyst.

Basically, if there’s a referring domain, it’s initially set and persisted through the getAndPersistValue plugin.  Then it’s passed from page to page via the combination of getPreviousValue and getAndPersistValue.

For those that don’t have a referrer, then it’s just set as "direct” initially and then persisted.

And that’s all there is to it.  The pathing reports that this enables actually don’t show too much so you might want to disable them (they do show multi-traffic sources during the same visit though).

I thought I’d write this one given the general lack of information available on the web around getting Flowplayer reporting into SiteCatalyst v15.

I recently worked with a client who uses Flowplayer and we needed to get them ready to upgrade to v15.  The challenge was, Flowplayer has no information about integration and what’s out there on the web seems to be for v14 code.  Additionally, with Flowplayer you can’t use the AppMeasurementExtension – you have to use the JavaScript methods.

So, with a little struggling and some help from Chris Meares (a fellow beta tester) over at Semphonic, we managed to get it all working properly.

I’ve added code files for you at the bottom of this post because the formatting is not the greatest.

Step 1 – Configure Video Measurement

Firstly, you need to assign your eVars, s.props and events for video measurement and configure them in the admin UI of SiteCatalyst.  Note that you’ll need to give one of the eVars full-subrelations.  Also you’ll need to enable pathing on the prop if you want to see video paths.

As per the previous post, you need 3 eVars, 7 events and an s.prop.

Once you saved your configuration the new video reports will become available in the reporting interface.

Step 2 – Media Module

Get the latest media module and put it into your s_code file.

I’ve made available the media module that I used as I could no longer find it anywhere in the admin.  Replace your existing media module (if you have one), with this one.

If you don’t have a media module, then this goes into your s_code, just after the section named

/****************************** MODULES *****************************/
/* Module: Media */

[paste the media module here]

/************* DO NOT ALTER ANYTHING BELOW THIS LINE ! **************/
var s_code=”,s_objectID;function s_gi(un,pg,ss){var …

Step 3 – Add media variables to s_code

Just before the Media Module section of your s_code and just after your tracking server value, you will need to add in the media variables:

s.loadModule(“Media”)
s.Media.autoTrack=false;
s.Media.trackMilestones=”25,50,75,100″;
s.Media.segmentByMilestones =true;
s.Media.playerName=”Flowplayer”;
s.Media.trackWhilePlaying=true;
s.Media.trackUsingContextData =true;
s.Media.trackVars=”events,eVar31,eVar51,eVar52,prop31″;
s.Media.trackEvents=”event9,event55,event56,event57,event58,event59,event60″;
s.Media.contextDataMapping = {
“a.media.name”:”eVar31,prop31″,
“a.media.segment”:”eVar51″,
“a.contentType”:”eVar52″,
“a.media.timePlayed”:”event55″,
“a.media.view”:”event9″,
“a.media.segmentView”:”event57″,
“a.media.milestones”:{
25:”event58″,
50:”event59″,
75:”event60″,
100:”event56″
}
}

You’ll need to modify the variables as per your configuration from Step 1.  Make sure you put them all in the right place and use all of the variables that you set up.

Step 4 – Modify your video player code

This step requires that you go back to your video player code and add in special handlers to “listen” so that they react when certain things happen.

With Flowplayer, you’ll have access to the clip: function.  This is where the handlers are put so that you can call the applicable Omniture video tracking functions:

<script type=”text/javascript”>
var omniMediaDetails = {“title”:”Test Video”}; // Name of Video
$f(“fzplayer”, { src: “flowplayer.commercial-3.2.5.swf”, wmode: “transparent”  }, {
clip: {
/* ——————————- */
// track start event for this clip
onStart: function(clip) {
s.Media.open(omniMediaDetails.title,clip.fullDuration,clip.url);
s.Media.play(omniMediaDetails.title,0);
},
/* ——————————- */
// track pause event for this clip. time (in seconds) is also tracked
onPause: function(clip) {
s.Media.stop(omniMediaDetails.title,parseInt(this.getTime()));
},
/* ——————————- */
// track resume event for this clip. time (in seconds) is also tracked
onResume: function(clip) {
s.Media.play(omniMediaDetails.title,parseInt(this.getTime()));
},
/* ——————————- */
// track stop event for this clip. time is also tracked
onStop: function(clip) {
s.Media.stop(omniMediaDetails.title,parseInt(this.getTime()));
},
/* ——————————- */
// track fast foward and rewind. time is also tracked
onSeek: function(clip) {
s.Media.play(omniMediaDetails.title,parseInt(this.getTime()));
},
/* ——————————- */
// track finish event for this clip
onFinish: function(clip) {
s.Media.stop(omniMediaDetails.title,parseInt(this.getTime()));
s.Media.close(omniMediaDetails.title);
}
},
/* ——————————- */
// show stop button so we can see stop events too
plugins: {
controls: {
stop: true
}
}
});

</script>

Step 5 – Verify

To verify that the correct variables are being sent at the correct time, use WASP or Charles or Fiddler.  The output should be something like what’s shown below.  You’ll get an event/events fired off at the start/25/50/75/100% milestones:

If you see something similar to what’s shown below as your pev3 variable, then you’re not using the correct Media Module:

No fixed length video?

If you are streaming videos with no end, (e.g. a running newscast or something like that), then you’ll need to change your tracking to s.media.trackSeconds instead of s.media.segmentByMilestones.

We added a variable into the player code (s_videolive) to set whether it was a “live” feed or if it was a known-length video, and then modified the in-page footer code as follows:

   if(s_videolive && s_videolive == “false”){
// Non Streaming Video
s.Media.trackMilestones=”25,50,75,100″;
s.Media.segmentByMilestones =true;
}else{
// Streaming Video
s.Media.trackSeconds = 30;
s.Media.segmentByMilestones =false;
}
var s_code=s.t();
if(s_code)document.write(s_code);

If it’s not a live feed then its tracked using start/25/50/75/100 milestones, otherwise, it’s tracked every 30 seconds.

Files

Adobe Omniture SiteCatalyst v15 Media Module

Media Variables

Flowplayer Sample Code

Help Files

Adobe has also put together some very good documentation on measuring video, especially around the use of the AppMeasurementExtension.

Example Video

Watch a video measured using SiteCatalyst, Flowplayer, and TagManager.

There’s no excuse nowadays not to be optimizing across multiple channels.  There’s heaps of tools available from no cost tools to paid tools and there’s plenty of consultants willing to help you get out there and lead you down a path of optimization.

Just remember though, 90% of optimization is done by people, only 10% by the tool.  It’s people that create the optimization strategy, the ideas to test, the segments, the content, the creative and of course, the insights that come from the results – both good and bad.

So how do you know if free is ok or whether you should invest in something a little more robust?  It depends how serious you are about optimization.  It also depends on what analytic tools you run as well.  It depends on what you actually want to achieve.

And there’s quite a few distinct differences between Test&Target and Google Website Optimizer, which should make the decision a clear one for you if you’re serious about the whole thing.

This is just a sample of the differences…as I was writing this, I came to the conclusion that there are in fact too many to list (such as mobile testing, in-flash testing, ad network testing), so I tried to focus on what I think are the key differences – the ones that in my opinion, will make or break your venture into optimization.

A/B Testing

Both offer it but with big differences.

Google offers it as a landing page redirect, which also means that you need to create two different versions of a full page and host both pages.  Of course, once the test is complete, you’ll need to do something to redirect the tested page to the correct URL.  There is virtually an unlimited amount of content variations you can create though.

Test&Target offers both Landing Page redirects (similar to Google’s A/B) and an in-page A/B test, again using virtually an unlimited number of content variations.  With a T&T A/B test you don’t need to create new versions of the page to test and don’t have the hassle at the end of the campaign.

MVT Testing

Both offer it.

With GWO, their MVT test is more similar to T&T’s A/B test in that you create multiple variations of the content and use multiple content areas defined on the page. 

T&T’s MVT test has multiple content regions that you vary the content in.  The biggest difference is the testing methodology…see method of testing below.

Landing Page (and content rotation)

GWO uses the A/B version (redirect) test to do this, locking the visitor into a specific piece of content.

T&T takes a different approach, of which there are two flavours.  One is a landing page redirect, where you can test complete pages.  The other is a Landing Page Campaign.  The big difference here is that with a landing page campaign, the visitor is not locked into a single offer displayed – they can experience different offers as their profile changes (see profiles below).  A visitor might see different content during the day than at night because of trends (shopping from work or home).  They might see different offers based on the affiliate they came in from, or from the type of search they conducted.  These are typically used for things like Landing Page Optimization using keyword reinforcement.   With GWO you need multiple pages created to achieve this.  T&T, 1 page, 1 campaign.

Method of testing

Here’s a big difference.

GWO uses full factorial.

T&T uses Taguchi.

Full factorial means that your test must include every permutation of your variables. For example, if you are testing 4 headlines, 4 hero shots and 4 call to action colors, your full factorial is 4x4x4, or 64 variations. That requires a lot of traffic to be able to complete the test with statistical validity in a reasonable time frame. GWO does not support the Taguchi Method, which allows you to include the most options with the least amount of testing and generates results far quicker, with less traffic, than either full factorial or an A/B test.

Segmentation Targeting

Here’s a massive one.  And to be honest, I don’t think you should be testing if you can’t segment your tests.  In my opinion, it doesn’t provide you much of an insight.

With GWO it’s all or nothing.  Everyone sees the campaign (depending on the % of traffic you’ve sent to the campaign).

With T&T you can target specific segments of traffic, like first time visitors, or visitors who’ve purchased, or visitors who come from an organic brand search term.  You can target on URL, you can target on page query string, on country, by browser or device, by social network or other referring domain, by campaign, by behaviour, such as visits of more than 10 minutes, or more than 5 pages or, in fact by a whole suite of custom rules that you can create.

Segmentation will be required when you want to gradually “roll out” new tests, like a site redesign to only new visitors at first. Existing visitors have probably learned your existing design, and the new one could actually reduce conversion because it requires new learning on behalf of the user.  Something to really consider.

Targeting Rules

Again a big difference in capability.

GWO has no targeting – period.

In T&T you can create “targeting expressions” which are JavaScript based rules within the engine to determine whether someone meets the targeting criteria or not.  If they do, either enter them into the campaign, or show them a particular variation.  Really good for testing just those who have viewed a product but not purchased it yet.

User Profile parameters

Another big difference.

GWO – sorry, doesn’t exist, there’s no targeting capability, so it doesn’t need them.

T&T on the other hand allows you to pass data into the platform through the tags and further enhance the visitor profile stored by the platform so you can target variations of content to the visitor.  For example, these can be used when you want to target content based on a visitors category affinity, or based on their total spend – maybe give them a really special offer.  (Ok this is really behavioural targeting).

Traffic Throttling

Both offer this, but to varying degrees.

With GWO you select the % of traffic you want to enter into the campaign. GWO will then faithfully serve test pages equally to the percentage of traffic you specify. If you have a strong performing page that you want to make even stronger, you actually risk losing money during the test period if your challenger variations are not strong performers. Sending only a portion of traffic to test variations, for example, can reduce this risk. For example, you may choose to send 70% to your control and 30% to your challenger, or 50% to your control and 25% to your two challengers.

With T&T you can select it at the campaign, the targeted content region, and the content variation.  Lots of flexibility.  And for good reason. 

Conversion tracking

Both offer this, but again, radically different in their capabilities.

With GWO you select the page that the user must visit, combined with their code, signifies a conversion has occurred.  A count of.  Yes, you can add the same code to multiple pages so that you can count a conversion when someone visits any of the pages, providing the code is the same.

With T&T you have multiple options.  It could be a click to the content; it could be a specific page with a tag on it, or one of a myriad of SiteCatalyst success events that you’re already tracking.  Many options enabling deeper insight into conversion tracking.  Couple this segment tracking and conversion success metrics and you have very powerful insights.

Conversion Success

This is critical in my opinion. 

With GWO you only get the conversion tracking above.

With T&T you get to see conversion success points along the way.  For example, you can use success metrics already tracked through SiteCatalyst to monitor such things as leads or sales etc; you can use the targeting expressions or the user profile parameters, or any of the other segmentation based options.  Very handy for figuring out if your campaign lifted conversions but your average order size tanked in the process.

Segment Success

The whole thing about testing is that we, as visitors, do things differently.  So looking at how different segments respond to content variations is fundamental.

With GWO, there is no option for this.
With T&T you can add virtually an unlimited number or combination of segments to monitor how they’re responding.  Again, not much good to have a winning variation if your repeat visitors suddenly decide to spend much less on average than your new visitors, as a result of the campaign.  Your campaign conversion might have lifted but your product managers will want to know why their bottom line has been decimated.

Multiple Live Campaigns

Not really. 

In GWO you can only have multiple live campaigns if they’re different content areas.  You can’t run different campaigns targeting the same content area at the same time.  That means you can’t support your marketing departments campaigns without multiple pages.

T&T allows you to do this.  Based on the campaign entry rules that you define, you can run multiple campaigns into the same content areas.  Why would you want to do this?  So many reasons.  Many companies run multiple simultaneous campaigns.  If you have a modularized design (most likely) then your homepage is likely to feature certain campaign-based areas.  You need to target those campaigns to different segments who meet certain criteria.  With T&T you can.

Nested content

I’m not aware that GWO can do this.

T&T can for sure.  You might be wondering why you’d want this.  Probably for the more advanced, but it goes back to the multiple live campaigns, and content re-engagement when combined with landing page optimization.  Gets quite tricky, but definitely possible.

Campaign Preview

With GWO, sorry that’s a no-go.

With T&T use Onsite to preview what your variations will look like on the page, before you go live.  Allows you to validate your test before you go live.  Pretty handy.

Tag based

Yes both are tag based.  Both require you to put tag onto the page. 

T&T uses a <div> tag coupled with a lightweight piece of JavaScript available to any future test, whereas GWO uses fairly heavy JS, customised for this particular test only.

Reporting

I’ve saved what I consider to be one of the biggest challenges for the end.

GWO gives you conversion reports.  That’s about it.

T&T gives you a whole suit of reports within the T&T interface.  Plus, if you’ve integrated with SiteCatalyst, then all of your metrics are available to you within SiteCatalyst. 

Summary

Herein lies the key.  If you’re optimizing for conversion, then you need to fully understand the impact of your testing.  If you don’t you’re flying blind.  What looks like a good result from a conversion standpoint, may in fact be the worst performing result you could have imagined, because you didn’t segment your results, or you didn’t apply all of your other analytic metrics to the results, such as AOV, or Revenue, or Leads etc.

Optimization unfortunately should not to be taken lightly.  There are many factors involved in understanding whether something was successful or not – you cannot just take a conversion lift at face value.  You might have impacted something else…or you might have seen a massive lift in something else, in which case, you’re a total hero when you report it up the chain.

If you haven’t bought into a paid tool and you’re just getting your feet wet with testing, there’s probably nothing wrong with getting to grips with some of the concepts using Google Website Optimizer.  But, in doing so, you also need to understand it’s limitations and the limitations it will inherently place on your testing and results.

However, if you’re really serious about using optimization as part of your overall digital strategy, and understand that in today’s day and age, digital optimization and behavioural targeting is a fact of life, then you should be investing in paid tools, which have the full capability to support your needs.

Of course, more important than the tools are the brains behind your optimization program. 90%.

As I find myself sitting in Auckland International Airport, waiting 12 hours for my next flight to LA (it’s 2am Perth time so I’m not quite with it), I figured what better time to write about success event pathing, as apparently my success event pathing for this trip wasn’t quite optimised for connecting flights at reasonable hours.

One of the things that you want to know about your site is the order in which certain events happen.  If you’re using success events for various things, like tracking self-service transactions, or leads, purchases etc, it’s important to know the most popular order in which visitors interact.

Now, admittedly, if you’re lucky, you could get some of this through page pathing…but as they’re pages, you end up with a bunch of clutter in between and unless those key events happen almost simultaneously, you’ll struggle to get the information you need.

You could use the Path Finder if you’re on SiteCatalyst v14, but it v15 it’s no longer available.  You could use the Fall Out report to generate some insights, but again, it’s not optimal.

If you want to know which events people interact with after a specific event, or prior to a specific event, then you need to implement Success Event Pathing.

Quite simply, you just set an s.prop each time specific success events are set.  You pass the name of the success event into the s.prop and ask Client Care to enable pathing on the prop.

What could be easier.

In the following example, s.prop45 is out “Success Event Pathing” traffic prop.
/* Success Event Pathing */
if (s.events){
if (s.events.match(/event1,/)){s.prop45="Internal Search";
}else if (s.events.match(/event2,/)){s.prop45="Lead Start";
}else if (s.events.match(/event3,/)){s.prop45="Lead Complete";
}else if (s.events.match(/event4,/)){s.prop45="App Start";
}else if (s.events.match(/event5/)){s.prop45="Course View";
}else if (s.events.match(/event6/)){s.prop45="Tool Used";
}else if (s.events.match(/event10/)){s.prop45="Form Start";
}else if (s.events.match(/event11/)){s.prop45="App Complete";
}else if (s.events.match(/event12/)){s.prop45="Form Complete";
}else if (s.events.match(/event16/)){s.prop45="Social Share";
}else{}
}
First we check that s.events exist and if it does, then we’re looking to match certain event numbers.  If they match, then we set prop45 to be a common name for the success event.

Notice the commas in the some of the matches?  That’s because we almost always have event13 added to our event list (as a page velocity event) so it’ll likely contain a comma, and we need to not mix up event1 with event 12 etc.

So we end up with a pretty standard traffic report that looks something like this:

This obviously shows that Internal Search is the most popular thing to do (not surprising given this is an un-segmented report).  Once I segment it, to show only those that are not staff or students, it’s a little more in line with what we’d expect…course views being the top activity for that audience type.

Ok, so that’s just the basic traffic report.

We can see from the above traffic report that they are interacting with areas across the site, from internal search, to filling in forms, to applications etc…but we don’t know the order this all occurred in.

Next success event flow

Let’s take a look at what those visitors do after they’ve looked at a course, by opening the Pathing Next Page Flow report.

We can see that following course views, in many cases, no other activity is recorded (for this visit).  You have to remember that pathing is unfortunately not cross-visit.

Some of them do interact with search, become a lead etc, during the same visit.

Previous success event flow

Now if you open the Pathing Previous Page Flow report for this traffic variable, you’ll be able to see what success events led to a specific event…such as an Application (or a purchase).

What we see again is that Apps typically start on a new visit (unsurprising) and during this period (when applications are actually fairly low anyway), course views lead to apps (that’s good).  Internal search and leads also generate course views, which lead to apps.  So, the flow is good, the volume is ok.  And another insight that makes us happy is that internal search accounts for very few applications…meaning people are not having to search for something prior to starting an application…and that’s’ great news.

There we have it – the order in which these success events occurred, giving you the insight into what happens before conversions.

And as success events are the most important things you measure on your site, it’s a logical step to create pathing on them to understand that order, so that you can optimise things further.

Now, where’s that coffee shop?

Implementing web analytics successfully throughout an organisation requires more than just a “fire-and-forget” approach to the platform.  There are many elements that go into making it worthwhile for your business, including:

• a measurement strategy
• resources
• domain expertise
• data integration
• data visualisation
• site performance analysis
• online marketing analysis
• social media and competitive analysis
• measurement adoption throughout the organisation
• measurement governance
• and ongoing optimisation

Last year I posted some thoughts around “how to create a good measurement strategy” and we’ve now taken it one step further by having a questionnaire on Digital Balance to help you visualise your measurement footprint in relation to what we think is achievable for most organisations, and in relation to other respondents.

See how you shape up against everyone else.

I’ve come across a few clients now that either aren’t using SAINT, are using it in a limited way, or are using it for campaigns only.  Maybe people are confused by what it does, or daunted by it, or just don’t know what it can be used it for.  It’s got uses that extend way beyond campaigns.

So, in this post, I’ll re-cap a bit about what SAINT actually is, and how it can be used, across a whole multitude of things.

And classifications can be used for millions of records.  We recently uploaded a very large list of customer ID’s and segments per customer, which enables some fascinating insight into customer behaviour.

SAINT – the acronym

To get this out the way, SAINT stands for SiteCatalyst Attribute Importing and Naming Tool.  It’s a way to classify a SiteCatalyst variable into more meaningful terms, and enabling you to group them together in certain ways.

What’s a Classification?

Basically, when you “classify” a SiteCatalyst variable, you are extending the information available on that variable through additional meta-data. 

Classifications are most frequently used on campaigns.  When you run a campaign you track it through a campaign code – a unique code that you set to identify that specific campaign element, such as “eml123”.  You add the tracking code (typically in the format www.youdomain.com/page.html?cid=eml123) to a link that’s driving traffic to your site.

Your s_code is most likely looking for any page query string to contain the parameter “cid”, and once it sees it, it’ll put the value “eml123” into the s.campaign variable.

It only needs to see it once…typically on the landing page.  As long as it saw it and recorded it then you can see success events further down track, tied back to the campaign code.

Looking at your campaign reports, you’ll see one called “Tracking Code”, and in there, you’ll see all of the unique values that have been passed through the s.campaign variable.

But by themselves, they’re pretty difficult to read.  “eml123” doesn’t mean much to anyone.

So what if you want to view them by type of campaign, or source of clickthrough, or media type etc.  Do you need to create a new conversion variable for each one?  No. 

This is where classifications come into play.

You can simply tell SiteCatalyst, through SAINT, that there is additional information that represents the unique campaign code, and using that information, you can view reports and conversions by the extended data, slicing and dicing to your hearts content.  Obviously you need to upload that data using the SAINT template, but that’s all pretty straightforward.

So, what else can it do?

Well there’s plenty of things that can be classified. 

We’ve used classifications across a broad spectrum of values, including:

• Products – the obvious one, classified into category, sub category, manufacturer, supplier, etc
• Internal promotions – the next most commonly used one, classified in the same way (generally) as external campaigns
• External Search Terms – classified against branded or non-branded terms
• Internal Search Terms – classified against type of term, such as product, information, support, sales etc
• Customers – classified against customer demographics, business segments, locations, products owned, mosaic segment etc.
• Behavioural Segments – classified against profile characteristics (such as described in Moving Beyond Business-Based Segmentation)
• Videos – classified against genre, length, player etc.
• and the list goes on…

If you’re using multiple eVars to capture similar information, or information that is essentially meta-data to do with another eVar, then you should be using SAINT to classify from a single eVar.

And it’s not just eVars that can be classified.  Traffic props can be classified too. 

Use Hierarchies

I’ve also come across plenty of clients that don’t have the hierarchies configured.  To configure hierarchies is very simple using the admin.

The benefit to hierarchies is that they allow you to view rolled-up metrics, and then allow you to drill-down into your chosen hierarchy.

Once you apply a hierarchy to your classifications, your menu structures change to support that hierarchy.

The most common use of hierarchies is within campaign structures, but they apply to all classifications.  Below I’ve shown the resulting menu structure for Murdoch’s equivalent of products – courses:

Once you open a report, for instance, Course Area (shown below), you initially see the rolled-up metrics.  Once you click on the + sign, you drill into that classification to report on the next level.

So, if you don’t see the classification drilldown in your menu’s…

…ask your admin to classify as it will surely help in your day to day reporting capability.

While Jerome is busy posting about the in’s and out’s of implementing Search&Promote, I thought I’d wade in with a post on why businesses should consider their search as a missed opportunity.

I’ve previously posted on what I think is a hidden gold mine of information called internal search.  It’s an area of the site that many companies, quite frankly, ignore.

Not too sure how to do anything with it, we assume it’s working just fine serving up results to random queries, so we’ll leave it alone and focus on our core purpose, driving people into our conversion funnel.

Or something along those lines.

If that’s you, go stand in front of a mirror and slap yourself a few times!  Wake up and smell the coffee…there’s much more to search than that!  And while you’re busy trying to optimise other areas of your site, you’re also blindly missing a massive opportunity for conversion optimisation.

The usual culprits

Almost all internal search engines (those used to search your own content) rely heavily on the traditional methods used for SEO – namely optimised pages of content.  They will dutifully rummage around your content on a predetermined frequency, building indexes that are searchable by the masses.

But there’s a whole land of missed opportunity if you rely on just organic searching.

Some of the more advanced search engine appliances will be able to up weight & down weight those results.  Some of them include facets (those subcategory links on the left hand side that are quite popular these days, especially amongst retail sites).  Most of them include some form of miss-spelt word or Did You Mean… capability.

Your opportunity to Search&Promote

You’re presented with a golden opportunity here.

Search&Promote can make your existing search engine look like it’s a digital yellow pages.  And how many people really want to use Yellow Pages nowadays?

What if you want to group certain types of results, such as content or products, and display them first.  What if you want to rank results, or in fact pre-filter results based on what the user has searched for, or interacted with, previously?

What if you wanted to display a different set of results for customers versus non-customers?

What if, by doing the above, you could marginally lift conversion rates?  Would that make a difference?

This opens up a whole new level for internal search.  You begin to have some control over the content that is actually displayed, rather that letting the search engine determine it.  And you can then begin to have your search contribute more to your conversions.

Make search work for you

If you’re already running SiteCatalyst and Test&Target, then adding Search&Promote really seems to be a no-brainer.

How much traffic do you get across your search pages?  Go and have a look.  How much of that traffic ultimately converts?  Take a look (you can use participation metrics to check that one out).

Now compare that to traffic that didn’t search internally.  How do they compare?

Given the volume of traffic across your search page, are there opportunities to engage them further?  If you could optimise your search page and influence conversions, what would be needed before you can show a return on investment.  I’ll bet it’s not much.

Optimisation Calculator

I’ve included as part of this post an Optimisation Calculator (excel spreadsheet) that can help you determine annual incremental revenue from optimisation efforts.

To use it, just put in your traffic, conversion and revenue amounts in the coloured cells, as well as a “what if we could it increase it by…” percentage rate.

In the example I’ve used, suppose you have 40,000 unique visitors to your internal search page.  Suppose they contribute to only 77 purchases, with $36,500 revenue generated from them.  That’s a search conversion rate of about 0.19%.  Not too much.  Some companies would think it’s not worth it.  They’re not there for that reason.

Think again.

If you could lift search conversions by an incrementally small amount, say 17% overall (taking the conversion rate from 0.19% to 0.23%, then you incrementally add nearly $100,000 to your annual revenue, given the example above.

If you can achieve the same conversion rate that your site gets overall (in the example, I’ve used 2.78%) you’d incrementally add a whopping $5.8million.

Did that make you sit up and look closer?

It’s a “suite” integration

If you have SiteCatalyst and Test&Target then you’re definitely one step ahead of your competition.  If you add in Search&Promote and spend the time to integrate all three products, so they leverage each other, you can, without a doubt influence internal search conversions.

With Search&Promote, you can do all of this and more – up weight/down weight results based on SiteCatalyst data such as top content; integrate external datasources such as product databases into the search results and have a different look and feel for those results, placing them at specific locations on the page.  You can target offers or promotions to users based on their search terms or previous browsing history by adding mboxes to the results page.  You can even change the entire search filter based on things like the last product type they viewed – why show them mens pants when they previously viewed female pants, and came back to do a search.

We must open the doors of opportunity.
But we must also equip our people to walk through those doors.
Lyndon B. Johnson

Well, the doors are wide open.  You’re suddenly in the land of opportunity.  Are you going to step through?

Remember, content relevance is just as, if not more, important on the search results page as it is across your other pages.  But many companies just forget about it assuming their search engine is doing a reasonable job.

Take a look at exit rates from search.

The other very much overlooked area for internal search is staff-based applications.  How many companies run intranets?  How many of you have multiple search boxes, searching different things across your company?  I’ve seen companies with between 3-10 different search types available to them, from internal product database, customer databases, content, FAQs, manual lists of links, and so on.  Search&Promote can integrate all of those into one easy to use search tool – saving hours of frustration for staff who can’t find the thing they wanted.

A small percentage increase in conversions, in a segment of traffic that virtually everybody forgets about (or discounts as not worth it) can make a whole heap of difference to your business.

Internal search…it’s an internal gold-mine waiting for you to excavate.  And the excavator is Search&Promote.

I challenge you to go take a look – spend an hour analysing your internal search traffic and play with a few “what if’s”.  Then decide if you can afford to ignore your internal search.

“I can’t find anything!”

This is the most common response we came across during the scoping and implementation of Search and Promote as the new internal search for Murdoch University.

Hardly surprising, given the issues with internal search that I covered in my previous post, but amazingly consistent!

In fact, one of the great truths we found during this project is that people truly don’t care where content is located, or whether it’s authenticated and/or accessible – they just wanted to type something in the search box, immediately find what they’re looking for, then carry on with their work.

We’ve now completed the implementation across our internal sites, and it’s working really well – so well that we’re now 2-3 weeks away from covering our external sites.

In my last post I promised to run through the implementation, however there’s a lot to talk about, so today I’ll cover SEO metatags (or the lack thereof), using multiple content sources, and how we integrated Search & Promote with SiteCatalyst to dynamically alter search result ranking.

Given the issues with internal search across campus and the wide range of staff and students that were more than happy to tell us just how bad it was, we decided to first implement Search & Promote across the internal sites where our primary audience are current staff and students.

Through the implementation of SiteCatalyst a few years back  across our network sites we have been able to segment our staff and student traffic, so we knew from the onset just how many searches each segment were doing, and how long on average they were taking.

Looking specifically at staff, approx 2,400 people collectively performed 234,131 searches in 2010, spending an average of 202 secs per search. Wow!

That equates to 13,137 hours, which, at an average of $40/hr, comes out to a $524,498 productivity cost. This figure alone should catch the attention of your key stakeholders and finance people.

Armed with that knowledge, we set the following key objective for the  Search & Promote trial across our internally facing sites;

• Reduce time staff spent searching by 10% by delivering a single set of filterable results, transparent of source, influenced by recent traffic.

Now that we had a clear objective, we could begin on the planning and implementation. We were greatly aided by a project team at Search & Promote – thanks John, Wally and Richard; you were all very helpful, and it was great working with each of you.

The first step was to set up the organic crawl of our internal sites, which largely consisted of listing the appropriate entry points;

Image: URL entry points in Search and Promote

And their corresponding URL masks (note the test feature that allows you to try your masks before saving them);

Image: URL masks in Search & Promote

Search & Promote works on a number of pages crawled – your licensing allows you to go to a certain number of pages, and after that the pages are not added to your index. There was a bit of tweaking to figure out what that level was, however there’s a cool feature in Search & Promote that allows the crawl to continue and count the number of pages that you’ve gone over by so you at least have an idea of where you are. From there you can either increase your licensed limit, or identify the larger than expected sites and par down the number of pages found by using the error logs and URL masks.

Compensating for the lack of SEO content

One of the issues I’d talked about previously was a lack of the bare minimum SEO metadata across many sites, most of which we had no direct control over. We tackled this by using the metatag injection feature in Search & Promote, which can be configured to dynamically inject metadata during a crawl, based on a URL pattern. This metadata is then included in the index as if the metadata was already embedded within each page, and can range from standard title/description metatags, to custom tags that can be use to create search filters (facets).

We soon found, however, that a significant portion of internal content required authentication to access, which meant that the crawler could not get in to that content. The Search & Promote crawler can be given credentials to access that content, however our concern was that content was authenticated for a reason, and to show even a title or extract from authenticated content on a public search may give away too much.

Given that the “we can’t find anything!” comment included authenticated content and applications, we needed an alternate option for this implementation to be successful.

At Murdoch we have a database called the A-Z index, which is maintained by our IT area, and over the past 5-6 years has grown to include an entry for most of our authenticated content and applications. This was a perfect source of information, now we needed to somehow incorporate this content into our search results.

Enter a feature in Search & Promote called ‘index connectors’.

Incorporating multiple sources of content

The index connector feature within Search & Promote allows you to define a third party xml feed, xml file, or comma/tab delineated file as an alternate source of content to be crawled.

The IT at Murdoch team were able to provide us an xml feed out of the A-Z index which allowed the Search & Promote crawler to include each entry/link within the feed in its scheduled crawls, together with custom mappings for each tag within the entries  to predefined custom metatags;

Image: Sample from our A-Z Flat XML feed

Image: Setting up the A-Z Flat XML feed as an index connector

Not only were we able to crawl the feed and include all the authenticated content as separate entries (‘restricted’ in the above screenshots), but we were able to alter the look and feel of the specific A-Z results within the wider search results, and account for a lack of  description within the feed.

The side-effect that we hadn’t counted on, but worked to our benefit, is that the A-Z index had entries for related non-Murdoch sites that were still of value to staff and students.

By having entries for the non-Murdoch sites in the A-Z as wayfinders, we didn’t need to crawl the actual sites themselves. This resulted in a significant reduction in the number of sites/pages we needed to organically crawl, while still providing our audience with a complete set of search results.

Using this same index connector functionality we were also able to incorporate the university’s campus directory listings via a new xml feed; whereas with the A-Z feed we only wanted to incorporate the results within the wider results set, we wanted results from the campus directory to always be the first results and be displayed in a table format, but more on the styling and positioning of these multiple content sources later.

Allowing for cyclical requests to ensure the most relevant results appear

In my previous post on Search & Promote, one of the key advantages the product had over its competitors was the ability to natively integrate with SiteCatalyst.

Via SiteCatalyst we already knew that our internal search terms follow highly cyclical patterns as our student (and staff) needs change over the semester. For example, the term ‘timetable’ is searched for throughout the semester, however the anticipated result changes as the semester progresses. At the beginning of semester, people are looking at for their semester timetable and towards the end their exam timetable.

In the past we’ve used custom coded mechanisms to help staff and students find what they’re looking for, however with Search & Promote we can take that to a whole new level!

Search & Promote allows you to define a data source within SiteCatalyst, in our case Global Production > Page Views, and then add ranking weight based on those values – the higher the weight, the higher the impact the SiteCatalyst data will have over your search results.

We defined s.prop41 under our Global Production suite in SiteCatalyst as SearchPromoteURL, and then used it to cross reference the Search & Promote crawled URLs with the associated Page Views data in SiteCatalyst;

Image: Using page view data from SiteCatalyst to influence ranking

Now, every day the last seven days worth of aggregated SiteCatalyst page view data is automatically downloaded and fed into the Search & Promote custom defined field SearchPromoteURL, which in turn is used in a ranking rule that increases the relevance of highly trafficked pages in the last seven days;

Image: Aggregated page view data in Search & Promote

A good example of this in action are our sample and past exam papers in our Library website, where there is a separate page per letter – with the SearchPromoteURL ranking rule disabled, the pages are literally ranked A through to Z, as the other active ranking rules see them as equally relevant. However when the SearchPromoteURL ranking rule is in place, the top ranked exam page is Exams B, followed by P and I.

In the admin data report for “exams” below you can see how the ranking, relevance and score metrics are all the same for the exam paper pages, and that the differentiating ranking  factor is delivered by the page views;

Image: Admin view of results for 'exams' and the different ranking scores that order them

The same ranking results can be seen on the front-end at http://search.murdoch.edu.au/?q=exams;

Image: Corresponding public search results for 'exams'

This is exactly what we set out to achieve, and it’s so far looking to have worked pretty well!

In part 2 of this post, I’ll cover how we combined all our sources of search results into a single set of user-centric, filterable search results, well as how we fared against our original objective of reducing time our staff spent search by 10%.