One of the things that stopped many companies from upgrading to SiteCatalyst v15 was the use of video.  Well that’s no longer the case, as v15 Video Measurement has now been fully released and is ready to go.

Video tagging changed from v14 to v15 and an updated way of measuring video was introduced, which required a few more things to be configured before it would actually work.  In my previous post Lights, Camera, Action…Video Measurement, I focused on how easy it was integrating the Brightcove platform for SiteCatalyst v14.

So, this is a functional post on how to integrate video measurement into SiteCatalyst 15, if you’re using the Brightcove platform.

Integrate and measure

There are still only really 10 steps involved:

1. From your Omniture Reporting Suite admin, download the AppMeasurementExtension.swf file.
2. Edit your report suite settings to enable video measurement (see below).
3. Create a config file.
4. Host the AppMeasurementExtension.swf file and the config file on your server.
5. Update your crossdomain.xml file to allow brightcove.com.
6. Upload your video to Brightcove.
7. Add a setting for your AppMeasurementExtension to be called.
8. Attach the video to a player and generate the website code.
9. Add the code to your website.
10. Play the video through your site and you’re done.

How easy could that be.  No flash programming.  No fuss.

New reports

The new set of video reports are fairly comprehensive.  You can also sub-relate by other eVars, and add in different success events so you can gauge how well they help in converting visitors.  Pathing on the video name also shows which videos they go through.

eVars, events and s.props

Video in v15 requires that you use a minimum of 4 events, 3 eVars, 1 optional s.prop (and a partridge in a pear tree).  While that sounds like a lot, they actually make much more sense than the old way of doing things.

So, your 3 eVars will be used for:

• Video Name
• Segment
• Content Type

Your 4 events will be used for:

• Video Time (in seconds) spent watching
• Video Views
• Video Completes (100% view)
• Video Segment Views

It’s also a good idea to use an 3 additional events as well, to track percentage viewed:

• 25% viewed
• 50% viewed
• 75% viewed

(Ok, so that’s actually 7 success events you need).

And your optional s.prop will be used for pathing between the different Videos.

Note: the Video Name eVar requires Full Sub Relations.  Now, on the face of it, that seems a bit weird given that all eVars are now fully-subrelatable. The reason for this is that if you’re running on v14 you can upgrade your video measurement now, while you’re waiting to upgrade to v15. Just note that with full-subs on the eVar, it won’t show up in the list.

Note: the Video Name sProp requires Pathing enabled before it becomes available in the dropdown too.

SiteCatalyst Video Management

In your report suite admin, after you’ve configured all of your eVars, events and s.props, select Edit to go to the new Video Management > Video Reporting section.

You’ll be presented with a list of requirements for Video.

When you select the checkbox, the dropdown next to it will appear listing all available eVars, events etc.  The first one, Video Name, will only appear if Full Subs are enabled on the eVar.  You can actually select new eVars and events from the list and it will create them and rename them in your admin console…but, it won’t do the optional events at the bottom.

Once you’ve populated the list of required variables, you can add in the optional variables (s.prop and your 3 percentage viewed events).  The optional events get added at the bottom where it says “Include additional Event”.

Hit save and you’re ready to create your config file.

It’s also a good idea to write them down on a bit of paper, so when you create you’re config file, you’ve got a handy reference.

Ours are as follows:

Standard eVars:

• Video Name = eVar50
• Segment = eVar36
• Content Type = eVar37

Standard events:

• Video Time Played (in seconds) spent watching = event31
• Video Views = event32
• Video Completes (100% view) = event33
• Video Segment Views = event34

Optional s.prop

• Video Name = prop23

Percentage viewed events:

• 25% viewed = event35
• 50% viewed = event36
• 75% viewed = event37

The main config file

If you’ve used Brightcove and SiteCatalyst before you’ll know that you create a config file with all of your SiteCatalyst variables declared in it, which is loaded into the AppMeasurementExtension file through the video player.  Easy enough to do.

Simply create a new XML file called config.xml and put it somewhere on your servers.

The file should contain the following:

<config>
<account>your-rsid</account>
<debugTracking>true</debugTracking>
<visitorNamespace>your-namespace</visitorNamespace>
<trackingServer>your-tracking-server</trackingServer>
<media>
<autoTrack>true</autoTrack>
<trackMilestones>25,50,75,100</trackMilestones>
<trackVars>events,eVar50,eVar36,eVar37</trackVars>
<trackEvents>
event31,event32,event33,event34,event35,event36,event37
</trackEvents>
<segmentByMilestones>true</segmentByMilestones>
<trackUsingContextData>true</trackUsingContextData>
<contextDataMapping>
<a.media.name>eVar50,prop23</a.media.name>
<a.media.segment>eVar36</a.media.segment>
<a.contentType>eVar37</a.contentType>
<a.media.timePlayed>event31</a.media.timePlayed>
<a.media.view>event32</a.media.view>
<a.media.segmentView>event34</a.media.segmentView>
<a.media.milestones>
<item name="25">event35</item>
<item name="50">event36</item>
<item name="75">event37</item>
<item name="100">event33</item>
</a.media.milestones>
</contextDataMapping>
</media>
</config>

As you’ll immediately notice, the config.xml file got a whole lot longer than the original.

The context data mapping section describes each variable and the name.  I found it easier to make a list of the names and values, write them down, then check them against my config when I’d finished, just to make sure everything was in the right spot.

Notice that event33 (our Video Complete event) is matched against the item.name=”100”.

AppMeasurementExtension

The next part is downloading the latest AppMeasurementExtension.swf file from your code repository inside  SiteCatalyst.  Once you’ve downloaded it, you need to upload it onto your server somewhere.

Brightcove

When you’ve uploaded your video to Brightcove, you edit/create a new player to include a custom plugin.

The plugin needs to know where the App Extension and Config file is.  This demo is located at:

http://www.elephantsandanalytics.com.au/omniture/
AppMeasurementExtension.swf
?s.configURL=www.elephantsandanalytics.com.au/config.xml

Note that this is all on one line.

And that’s pretty much it.  Again, how easy was that.

Now when you browse the video, and use something like Fiddler or WASP, you’ll see a number of new calls being made to the Adobe Omniture servers, at the beginning of the video play, during the plays and at the end.

Happy video’ing.

This topic was requested by one of my readers – thanks for the inspiration Dan. 

And it comes back to segmentation.  And the value derived from measuring your customers/members behaviors across your digital channels, and the impact they could be having on your conversion rates if you don’t segment.

Ok, so, we’ve all likely got some form of customer – either a lead, or a customer who has purchased, or a member, such as a policy-holder or a service-holder.  For the purpose of this post, I will call members, customers.

Why do we want to measure their activity?

Well there’s a number of reasons.

If we run a self-service site, we might want to know what self-service transactions are being done beyond the login (within the customer self-service portal, nowadays commonly called My{something or other}).

That’s pretty straightforward and you can achieve that with traditional metrics if you are reporting solely within that “site”.

But customers are not all the same.  There are business customers, there are residential customers, there different types of customers within each of those segments – business types, business size, residential locations, MOSAIC-based classifications of customers, demographics and so forth.  And guaranteed they will do stuff differently.  They purchase different products and services.  They’re interested in different content and so forth.

And many self-service sites are purely transactional in nature – change your address, update something or other, check usage of something or other etc.  Most of your content will be on the “outside” and these guys use that content too.  But they have a closer affinity to your brand and will use that content differently.

And the other big thing about them is, they’re customers.  They’ve converted.  Not only are they your golden opportunity to

a) cross-sell/up-sell and
b) observe their activity and figure out why they converted so you can better optimize

but (and possibly more importantly) unless you measure their activity, they are negatively impacting your conversion rates on your regular website and your average revenue per visit is understated.

Huh? Why?

Because they probably go through that site to get to their My{something or other} self-service site.

Consider this:

Let’s say you’re an insurance company.  Your acquisition team wants to know the sites conversion rate for product sales and the average revenue per visit.  You also have a self-service site and the way that most of them get to that site is through your regular homepage.  Suppose your overall traffic to your site is 350,000 visits per month and you get 10,000 product sales per month, with revenue of $500 per sale.  Suppose also that 200,000 visits per month go to your fabulous self-service site where they’re busy updating their info, checking on their policy and so forth.

If you DON’T strip out known customers, you’re under-reporting. 

Your average order value is unaffected – it’s still $500.  But your visit conversion rate is showing as 3%, and your average revenue per visit is $14.29.

If you strip out the known customers, your conversion rates just went through the roof.  It’s now 7% (10,000/150,000) and the average revenue per visit is $33.33 ($5,000,000/150,000).

Ok, a little extreme maybe, but you get the general idea.

So, measuring customers not only provides more accurate conversion rates, but also gives you better insights into their activity.

And SiteCatalyst allows you do all that – and more. 

Customer segmentation

Right, so you want to measure customer activity for a variety of reasons.

For basic customer segmentation, you’ll need an eVar and an s.prop and one or two success events.  The eVar will enable visibility of conversions back to them (revenue and the likes); the s.prop is used to segment your traffic-based reports by customer/non-customer type and the success events are for things like logins, failed logins etc.

If you’re measuring self-service activity, you’ll have another eVar for self-service transaction type and a counter success event.  These two are set every time the user completes a transaction.  You pass the type of transaction to the eVar and set the success event.  All pretty much no-brainer stuff.

If you really want to get value, you’ll also use an eVar to capture their memberID, userID or some other unique value that can be used for customer-type segmentation.

Basic segmentation – customer/non-customer

When a user does something that only a customer can do, or if they make a purchase, you set the eVar as “customer” or something similar.  You also set the success event, such as login, if that’s what they did. 

You will also need to set the s.prop to “customer” and you need to make sure it’s remembered and set again on every single page view.

The way to do this is through the getAndPersistValue plugin – a handy little plugin that will set a traffic prop automatically from a value in the cookie.

For example:

Let’s say that you have a My{something or other} self service site, and the user logs in.  Upon login, you would use the following in your s_code:

/* My Self Service Login */
/* Set User Type */
if((s.pageName.match(/loggedin/)){ // change for your page
/* serialised login and self service success success event*/
s.events="event1,event2";
s.eVar10="Customer"; // Customer Non Customer
s.eVar11="Login"; // Self service transaction name
}
/* Check existence of persisting prop10 - user type */
s.prop10=s.getAndPersistValue(s.eVar10,'s_prop10',365);
if(!s.prop10&&!s.eVar10) {
s.prop10="Non Customer";
s.eVar10=s.prop10;
}
/* Set persisting prop10 to value of the eVar10 */
s.prop10=s.getAndPersistValue(s.eVar10,'s_prop10',365);


Basically what happens is when the user logs in, the eVar is set to Customer.  The success events count the number of successful logins and also a general self-service transaction count.

Line 8 says set s.prop10 from the value of eVar10 and store it in a cookie for 365 days. 

Under non-logged in status, eVar10 will be empty, so line 9 is true (eVar10 has no value but s.prop10 may have a value).  If that’s the case, set it Non Customer and set the eVar to Non Customer.

Then line 14 re-evaluates s.prop10 from eVar10 and resets itself.  Odd I know, but it works.

You end up with a number of different states:

1) For non customers eVar10 and s.prop10 = Non Customer

2) For customers in the same session, eVar10 and s.prop10 will be Customer

3) For customers in a later session, eVar10 and s.prop10 will be Customer (because of the getAndPersistValue)

A word about the success events:

I’ve used two success events – event1 as a discrete count of logins and another as a general count of self service transactions.  You could combine, but I like the separate count and you can serialise the login event so you only count it once per session – giving an idea of true logins, rather than repeat logins per session.  The self-service transaction type (eVar11) would generate a report of all the different types of transactions by customer type, when viewed using the Self Service Transaction event (event2), or specifically logins (event1).

The Result

Now you have the capability to report on not only logins and self-service transactions types, but also your conversion rates are more representative of real rates by Non-Customers (and Customers – very important for retention purposes), and you can see their behaviors across pages when you use the s.prop. 

In v15, or with Discover, you can create and use these two segment values for more analysis to filter out or filter in based on Customer or Non Customer.

Of course, the next thing you’ll want to do is further enhance your Customer segments with demographics, or those additional business segments, using extracts from your customer database.  Take a look at 1 million rows and it still wants more, for an understanding of how that works.

So, segment, strip out your non-customers and re-run your conversion rates.  Now do the same for customers.  And now start looking at what customers did prior to conversion so you can influence those non-customers to do the same thing. 

My final tip…use Test&Target to provide more relevant content to Customers or Non Customers…because, now you can identify them.

Well, it’s August and true to their word, Adobe upgraded us to SiteCatalyst v15 on the 1st, and so I thought I’d share a few of the golden nuggets within v15.

I was thinking about how to order them…do I go by not bad to flamin’ eck, that’s awesome? Or start with the big bang and then let it continue to smoulder throughout? 

The problem is there are too many new and great features that you can’t really put them in any type of order.  They appeal to you on different levels, from functionality, to UI, to analysis, to reporting, to combination segmentation and sub reporting.

And as this post is kind of huge (sorry you might a coffee and a bagel on this one), here’s a little taste of what’s covered in it:

•  Segment, the all powerful segmentation
•  New segments
•  Site Overview Report
•  Segmented Overview Report
•  Side by Side segments (well sort of)
•  Key Metrics report
•  Normalization (one of my new best friends)
•  Visits, Visitors and PageViews
•  Full Sub Relations – multiple breakdowns on eVars
•  Traffic prop breakdowns
•  Login as another user
•  Calendar events specific to report suites
•  Significant changes

So let’s start with the big one that everyone knows about, or at least should.

Segments

Unless you’ve been living under a rock for the last 5 months, you’ll have heard about the segmentation capability within SiteCatalyst v15.

Out of the box, it comes with 7 pre-defined segments that are shared across SiteCatalyst, Discover, and Test&Target.

These segments were chosen because, according to Ben Gaines at Adobe, they are “valid (and important!) for all of our users—across vertical and market size. We’ve also seen these types of segments predict different behavior across a variety of actions: registration flows, purchases, and general site browsing.”

New Segments

You can, of course, create your own segments on the fly and apply them to any report as well.  If you’re a Discover user, you can create them in Discover and save them back into SiteCatalyst for later use.

I’ve not yet figured out how to create a segment available across all report suites, without creating it in Discover.  Perhaps Adobe can help on this one?

Segments can also be used in Test&Target too. There’s a one-click little target icon next to the segment box which opens up a new A/B..n campaign in Test&Target, although it doesn’t specifically target custom segments that you’ve created as that’s a little more complex – but the preconfigured ones are available for use immediately.

Site Overview report

This new report is actually a dashboard, but, it’s highly useful and can be modified to your needs.  Using the same features as dashboards, you can set this one to your landing page when you log in (no more pinwheel).

From those reportlets, you can get to the main report by clicking on the name of the reportlet.

You can also change the date and the whole thing will rerun against the new dates.

Segmented Overview

If you want to see a particular reportlet using a specific segment without running the whole dashboard again, just click on the report suite name within the reportlet and a little popdown appears, allowing you to select not only the report suite, but also a segment to use.

When you select a new segment, it will re-run the reportlet, not the dashboard, against the new segment.

Of course if you want to run the entire dashboard against the new segment, then just select the segment in the main segment dropdown and you’ll get instant gratification.

Side by Side Segments

Ok, so you still need Discover to do comparative segmentation, but, there is a sneaky little way to show two segments at the same time, using a dashboard report.  Generate your base report, for example, I’ve used calculated metrics on the new Key Metrics report (see below for more info).

Then apply a segment, and add it to a dashboard.  Apply another segment and add it to the same dashboard.

Then go to the dashboard layout editor (which is also new), and just put both reportlets into the new dashboard.

What you get is a visualisation of the two (or more) segments, plotted over time.

Key metrics report

One of my personal new favourites here is the Key Metrics report, which allows you to put multiple metrics on a time-based report.  This was always one of the big challenges before, but, they listened to us, and here it is.  And there’s a bunch of nifty things about this report that I just love!

Firstly, as shown above, you can add multiple metrics, or calculated metrics to the report – something you could never do before.  For instance, you can add Visitors, Visits, Page Views, Conversion Metrics, Conversion Rates and so on.

Now that you’ve got the key metrics trended side by side, every worthy analysis ninja will look to segment that information – so, just go ahead and apply those segments.

One of the great things about this report is you can do calculated metrics too…side by side (see the image for side by side segments above).

Normalization

Ah, what a great idea this was.  Normalize your data on the Key Metrics Report.  As you can see from the above screen shot, the page views data is overwhelming the trending report, making it difficult to view some of the trends for the other metrics.

But that’s ok now – just normalize it.  Same report as above, just normalized.  And the other metrics pop up.

Ok, so those metrics that I’ve used don’t really tell us anything.  How about this one then?

I’ve highlighted dates when certain metrics weren’t in line with the norm.

Now you have something to go and look at.  Why, on those dates, did those metrics “pop”.  What did you do?  Did they vary by segment?  And what can you do to nudge those that didn’t behave similarly?  You can learn from that.

Love this report.  One of my new best friends.

Visits Visitors & PageViews

Yes, these are now available across most conversion reports.  You used to be able to get Visits and Visitors by special request, but now, they’re enabled and you get Page Views as well.  And they’re particularly useful across things like campaign reports, referring domains, search engines, keywords etc.

Full Sub Relations – multiple breakdowns on eVars

Remember the days when you had to think carefully about setting up subrelations on eVars…should you use Full or Basic?  And remember the impact if you got it wrong, or realized later that you needed Full Sub Relations on a key eVar?

Well, those days are gone.

You now get full subrelations on all eVars (although I don’t quite understand why the admin asks you still for the type of subrelation you want when you create an eVar – Omniture…can you add any info here?)

In this example, I’ve used two eVars, both are set up in the admin as Basic Subrelations, but I’m able to now break one down by the other, and then I’ve added a filter to remove the keyword “murdoch” from the report.  As you can see, it’s a conversion report, with Unique Visitors, Visits and Page Views, as well as success events.

Once again, you can sub-segment this report against any of your existing (or on-the-fly) segments. Yay!!!

Traffic Prop breakdowns

Another great feature is the ability to now breakdown key traffic props, such as the referring domain report.  It used to have basically “instances” and allowed you to put in success events.

No more.

Now you can not only see Visits, Visitors and Page Views, as well as success events, but you can also now break it down by things like Average Time on Site, and, all eVars…wow!  Couldn’t do anything like that in v14.

And you can flip it too…start with a conversion report, such as our Figure Out Your Life segments, and you can break them down, by Referring Domains, or Time Spent per Visit etc.

Ok, long post I know, but, there’s so much going on that I’ll probably add more posts in the future on the new features.

Just to touch on some of the other new things as well…

Login as another user

Do you have a user that’s experiencing problems?  Well, now you can log in as that user.  Go to the admin and views the users, then click on “Login as this user”.

Report specific calendar events

One of the problems with SC14 calendar events was that they showed up across every other report suite…which made it very unhelpful to users that had somewhat restricted views, such as global sites and regional sites.  The UK office didn’t really care that you ran a specific promotion, but they saw it anyway.

Now you can apply a calendar event to a specific report suite only.  To be honest, the link for it is a little tough to spot, so I’ve put a whopping great big arrow to it:

Ok, now I’ve covered the things that I’m excited about, there are a few things that new users need to understand with this version.

This has all come about because of the way that data is stored.  It used to be pre-processed into the reports, which resulted in the limitation on segmentation and other capabilities such as breakdowns.  But Discover always worked off the original unprocessed data, which also led to some differences in the data, especially around unique visitor counts and classification deduping.

SiteCatalyst v15 now runs off the raw data – the same as Discover, so the datasets are the same, hence the reports are the same.

Key Differences

There are some key differences between v14 and v15 that you need to be aware of:

Visits for Non-Cookied Visitors

All visitors, regardless of them accepting a cookie are now included in Visit counts and pathing data.  But this increases your Visits metric, so your conversion rates will likely go down a bit.  In testing, the increase of visits was about 0.5% for first-party cookies, and 5-12% for third party cookies.  Another great reason why you should be on first party cookies (contact your account manager if your tracking server has 2o7.net in it).

Time Spent metric 

Both the time spent per visit and the average time spent on page metric now use all server calls to generate the metric, which is much improved on SC14.  What this means is that non-page view data is included in time spent reports, such as custom links etc.  And, they’re no longer bucketed.  It now works off an average for each and every individual page view.

De-duplicated Visits and Visitors in Classifications 

Classifications are now correctly de-duplicated, meaning that when you group things using classifications, they are now de-duplicated, whereas before, they would count each instance of a visit or visitor.

Ben Gaines wrote an excellent blog post about these and a couple of others which is definitely worth reading and getting to grips with.

Summary

What can you say?

Thank you, Omniture Business Unit within Adobe, for listening to us.  These changes make the platform even more useful than it already was and clearly makes it a powerhouse in the web analytics space.

With these, and many other changes made, we’re able to provide our organizations with even more insights that lead to more business optimization capabilities, a better ROI and hopefully more analysis ninjas.

Thanks guys!

So I promised that I would finally put fingertip to keyboard and talk a little bit more about using Visitor Scoring…to finish up the series that I started a while ago.

If you’ve read my previous posts, you’ll know that we implemented a series of metrics for engagement measurement, culminating in a per-visitor score.

I wanted to share with you some of the insights and benefits of doing all of this, particularly in Discover.

An Engaged Segment

If you remember from the previous post, we talked about using a total of seven different metrics to try to ascertain engagement, or, when used in combination, could identify an engaged user segment.

The final metric was a measure of interaction and there were two ways to implement; either by just counting the number of visitors that participated in specific actions on your site, or the second way; by creating a scoring methodology at the visitor level and leveraging that as the final metric.

A little tougher to do, as it involved some additional code across the site, but definitely implementable, and now, we’re able to use that information to gain even greater insights.

The reason you should segment is because not everyone on your site converts – in fact, probably only around 3-5% actually do what you want them to do.  So you use segmentation to analyse those that have converted and try to understand what made them different, what their different behaviours were, and if possible, try to use that as predictors of future behaviour by the other 95-97%, so that you can lift conversions.

Creating the engaged segment

Using Discover we created a Visitor container that contained the following rules, which we had previously determined worked best for our business.

The rules applied are from 6 of the original 7 indexes that defined engagement for us:

1. Click Depth Index – which captures the contribution of page and event views
2. Duration Index – capturing the contribution of time spent on site
3. Recency Index – which captures the visitor’s “visit velocity”—the rate at which visitors return to the web site over time
4. Loyalty Index- the level of long-term interaction the visitor has with the brand, site, or product(s)
5. Brand Index – the apparent awareness of the visitor of the brand, site, or product(s)
6. Interaction Index – visitor interaction with content or functionality designed to increase level of Attention

Notice the counts at the bottom of the segment creator – those seem to be a bit confusing as the number on the left (216 Visitors) represents the number of visitors that meet the criteria of the segment, but the number on the right (1,071,458 visitors) does not appear to be a count of visitors for the timeframe (for some reason I have yet to figure out).

Additionally, for this particular segmentation example, I wanted to ensure that our engaged visitors have actually become a lead (one of our primary goals).  So our segment includes that, as well as them having a score of greater than 200, which means they’re interacting with a fair amount of content or doing a fair amount of activity (such as applying to come to study with us).

Saving the segment, it’s then ready for use.

Using the segment

One of the first things I looked at was where our engaged users were coming from, their average scores, conversion rates and application rates (or purchase rates), compared to all visitors who are not staff or students.

And I fell off my chair.

While the number is quite small comparatively, due in part to the timeframe I’m using, the score, lead conversion rate (leads/visitors) and application rate (applications/visitors), are significantly higher.  This is to be expected due to the segment, but I wasn’t quite expecting the numbers to be that high.

Visual Site Analysis

I was keen to understand, visually, where these highly engaged users go on our site, so I used the Pathing Site Analysis report in Discover (one of my personal favourites).

Firstly I started with all visits to get a sense of what they do across common pages that we want them to interact with:

Fairly widespread usage of key pages.  The thickness of the arrow indicated volume of traffic from one page to another – the bulk of traffic goes to the Courses homepage, from the site home page.

Next, I applied the same engaged segment:

Ok, I must need velcro pants because I fell off my chair again.

It appears that our highly engaged users take a whole different path.  The colour indicates their propensity to become a lead.  The big red steps are basically the lead capture process through our main tool – Figure Out Your Course.

And they seem to browse around first within courses then become a lead – which is also good to know.  But once they become a lead, they tend to leave the site – which means we need to ensure that we’re effectively communicating with them through other channels, such as email at a later date.

Breaking down the traffic sources

In breaking down the Organic Search by time spent, we also see that a significant portion of highly engaged visitors spend more than 10 minutes on the site, and are more likely to convert to leads having done so, as compared to all visitors – another significant insight for us to use.

In summary

This is just one example of many that could be used on your site.  Once you’ve identified your engaged users, you can segment them further by demographics, or by customer type, or by content viewed, or by member/non-member and so forth.

You can view revenue by engaged/non engaged (bound to be vastly different), or average order value etc. 

If you use Test&Target, you’ll be in a great position to leverage the engaged user segment, targeting non-engaged users differently to increase their engagement levels.

All of this will help you gain better understanding into their behaviours, so that you can then further optimise your site to improve those conversions.

I’m keen to hear from others that have used Visitor Scoring, or Engagement Metrics across their site, coupled with Test & Target to lift conversions.  Let me know what your thoughts or successes are.

I’m incredibly excited.  I’ve been asked to attend and present at the massive Adobe Max 2011 conference, in LA, in October this year.  Wow!

Historically this has all been around Adobe products (Flash, Cold Fusion, Photoshop etc), but with their acquisition of Omniture a while ago, they now have sessions on the Omniture product suite…and I’m doing one of them.

So it’s off to LA I go in for the first week in October.

Can’t wait!

When it comes to searching across the web, we all know that Google is king, but does this still hold true across your own internal network?

Over the past 12 months we have wrestled with this question, particularly in an environment with multiple search mechanisms, manually maintained indexes, and masses of sites that were created when metadata was primarily used to categorise instead of search.

In a series of posts, I’m going to go through our experiences in improving search across our internal network – I’m not suggesting we have found the magic search bullet, or that we’re anywhere near finished tweaking and tinkering, but I do know we’re a hell of a lot closer than where we were at this time last year.

The problem

In our travels across campus, we kept hearing “I can’t find what I’m looking for!”  – not surprising, given that we had;

1. 500+ individual sites, ranging in age (earliest was 1997), metadata (none to Dublin Core to ‘something’) and ownership
2. Inconsistent use of key SEO elements, such as title, headings, tags and meta descriptions across the majority of our sites
3. Multiple sources of content and internal search mechanisms, each with their own set of search results
4. Manually maintained indexes, all categorised and sub-related, together with an in-house redirect mechanism
5. An internal audience of staff and students with heavily cyclical search requests – a search for ‘physics’ at the beginning of semester is more likely to be for text books, and at the end of semester past exam papers

Image: Multiple source-centric result sets;

Given Google’s dominance in search, we quickly went down the path of a Google Search Appliance, or ‘Mini’, which is a self contained rack mounted system that gives you God-like powers over the Google algorithm. We were bringing in a little bit of Google in to magically transform our disparate set of sites into a cohesive set of search results.

Once plugged in, the Mini worked really well – for pages that were properly formatted for organic search.

Pages that were missing or incorrectly using titles, headings and metadata didn’t fare so well, and we found the search results were not the most relevant, as the Mini couldn’t make much sense of most of the content it crawled. We also found that there was no clear way to incorporate the feeds from other systems, with the “how do I…” answers primarily provided by a community of Search Appliance users and resellers, and not Google themselves.

Given the wide ownership of the sites we were working with, updating each with appropriate SEO friendly content was unrealistic. What we needed was a way to;

1. compensate for the lack of SEO content,
2. incorporating multiple sources/ formats of content,
3. allow for cyclical requests to ensure the most relevant results appear, and
4. combine all the different sources of search results into a single set of user-centric search results.

Enter Adobe Search&Promote

If you’re a regular visitor to this blog, it will come as no surprise that Tim is a power user of Omniture products, steadily working his way around the product wheel. We became aware of the Search&Promote product (then called SiteSearch) which promised to solve our key internal search issues.

Search&Promote uses a search algorithm to organically crawl your sites, in addition to ranking rules based on a wide range of configurable data. Once you’ve defined your rules, you can adjust the overall balance between your ranking rules and natural search relevance.

Where there is a lack of metadata, Search&Promote can be configured to dynamically inject metadata on crawl, based on a URL pattern. Additional custom metadata can also be injected to create facets (filters) that allow users to drill further down into predefined categories.

If your multiple sources of content can be transformed into XML feeds, then that content can be crawled, categorised, and integrated with the organic results by Search&Promote.

Yes, there are other internal search products on the market that will do the above, however there is one thing that Search&Promote has over its competitors – the ability to tightly integrate with SiteCatalyst and Test & Target.

We’ve known for some time that internal search terms follow highly cyclical patterns as our student (and staff) needs change over the semester. We’ve helped them find what they’re looking for using of real-time SiteCatalyst data in search-as-you-type and tag cloud mechanisms, however with Search&Promote we now have the opportunity to take internal search to the next level.

In the report below (7 day moving average) you can see two popular search results across three semesters peaking at different times during the semester;

Notice how ‘bookshop’ peaks at the beginning of semester, then dies down, only to peak again at the beginning of the following semester. No surprises here, but it does coincide with a significant increase in page views across the Bookshop website.

Then look at the results for ‘timetable’ – there’s a peak at both the beginning and end of semester. The difference here is that people are actually looking for two different pieces of content – their semester timetable at the beginning, and their exam timetable at the end – using the same keyword. Again, the rise in search terms coincides with increased page views across each piece of content.

So, in theory, by looking at the last week’s worth of traffic across our group of sites, we should be able to determine what content students are looking for, then re-rank the search results accordingly. For example, the term ‘timetable’ at the beginning of semester will push results related to the semester timetable to the top, and at the end of the semester push results related to the exam timetable to the top.

Exciting stuff!

Next post – the implementation.

While this might be a quickie, it’s a biggy.  A big one in terms of the amount of data just uploaded through SAINT.  In fact, we’ve just uploaded around 1 million rows of data, with 6 columns per row.

And it didn’t even blink!  Gotta love that!

So why do we have a million rows of data?
Customer segmentation of course.

This was actually done for one of our other clients.

The rationale?

To segment conversions and transactions by customer type, segment, previous segment, needs group etc.  And SAINT enables that capability.

How?

Firstly create an eVar that the raw identifier will go into.  This might be an account number, a customer ID etc.  Then, using the admin, create the classifications on the eVar for the relative columns you need.  At this point I always create the classification hierarchy as well, just so I can envision how I want the data to be reported and drilled down though.

When you create the classifications, the SAINT file is also created and made available for download.

I opened the SAINT template in Excel and copied my customer segment data into it in blocks of 100,000 records.  There’s a number of reasons for this, not the least of which is to keep the file size down, but also to make it easier if an upload does fail – at least you can deal with 100,000 rows better than 1 million rows.

So I’ve now got 10 files, each file contains 100,000 rows and 6 columns of data per row.  Each file was about 5mb.

You can’t upload that much data through the browser, so you need to use the FTP Import capability.

In the SAINT admin, select Import File, click on the FTP Import and then Add New:

You’ll then get a popup that asks you to select a bunch of things to create an FTP account:

Select the data to be classified, move the report suite or suites to the box on the right, select the import options and add in your email address.

Check the box and hit save.

A new FTP account has just been created on the Omniture servers and you’ll get a confirmation screen showing the address, username and password.

Open it up in an FTP client and upload your SAINT files to the FTP server.

You’re not quite done yet though.

You also need to create a series of empty files, with a .fin extension, named exactly the same as your SAINT files.  These are “finish” files and are crucial to the upload.  They’re completely empty files – any text editor can create them.  Just make sure they are named exactly the same, case sensitive.

Upload those .fin files and you’re done.

Now, go have a coffee, have some lunch or dinner or whatever and come back later.

Progress

You can kind of check on progress by refreshing the FTP list of files.  Omniture removes the files from the FTP directory when it begins to process them, so you can kind of get an idea of where things are.

Time Frame

I uploaded the files around 4pm. 

At 10:30pm I did a data extract by FTP of all data to see where it was up to…it was done.  Shortly thereafter, I got an email saying it was done, without any failures.

Easy as pie.  No muss no fuss.

While we’re using customer segments, it could just have easily been customer demographics, technographics or any other form of data.  The point is, 1 million rows and it didn’t even blink.

There’s a few things to watch out for though when importing that much data.

There is a limit on the amount of unique values (500,000) that will be reported against in a given month.  We’re ok – we won’t see that limit.

Recommendations are that file sizes be kept under 30mb for the initial load, and then subsequent refreshes less than 5mb.  So we’re still ok.

And the import time will vary depending on many things, including how busy their import routines are.  You get in the queue and everyone loves a queue.

But that was it.  1 million rows of customer data now available for segmentation nirvana in SiteCatalyst – and DataWarehouse, and Discover, and Test and Target.  We’re off to the races!

And while the first run of this was a manual run, future updates can easily be automated now that the FTP site is created.  Just remember your .tab and .fin files must be named the same.

This is the third (but not final) post in the series on Visitor Engagement.  One of the problems with the Visitor Scoring method that I previously described, is that, at the end of the day, you’re still somewhat limited to viewing scores at the “average” level, by segment.

That presents a number of challenges because the average is precisely that…and the underlying scores vary dramatically within each segment.

But there is a way to see what each and every visitor score is, or even within the different segments…and it’s called the Unique Visitor ID.  You can see at the visitor level, how many times they’ve returned, how many “things” they’ve done, such as searches, product views, revenue etc.

This is really the icing on the proverbial cake.

Visitor ID

There’s a seldom used implementation method in SiteCatalyst called Dynamic Variables which easily enables all of this capability.

All you need is a spare eVar and you’re in heaven.

Step 1

Simply populate an eVar with “D=s_vi” and the result is that SiteCatalyst will populate the eVar with the unique visitor cookie ID.  If you test it with WASP or something similar, you’ll only see that D=s_vi got passed, but the magic begins.

s.eVar56="D=s_vi";

Step 2

Go into you Admin console, and on the report suite where you want this collected, go to Conversion > Unique Visitor Variable section.

You’ll be presented with a dropdown of available eVars that can be selected to use as the Visitor ID…simply select the one that you’re using and hit save.

That’s it – you’re done!  All there is to it.

Rest assured, it does not impact anything to do with the way that SiteCatalyst measures unique visitors – its completely separate and safe to use.

And now it pops

So, the first thing you’ll see is a new report on your eVar. The report is a list of all Visitor Cookie ID’s captured during the timeframe selected.  As it’s a conversion variable, you can add in whatever success events you like:

Add in your Visitor Score and you can see what their points are.  As you can see from above, I’ve highlighted an interesting visitor (#7)…they were on their 2nd visit, they searched 12 times, viewed 16 courses and submitted an application.  Their score is currently 542.  I’m wondering if visitor #5 is a candidate for an app too…similar behaviour is forming.

Segment-based Visitor ID’s

Ok, now open up a report such as Campaigns (we use the Unified Traffic Sources VISTA rule so it may look a little different if you don’t use it).

Remember the challenge I mentioned at the top of this post…with average scores.

We see below that the average visitor score for Organic Search is 7.88.  But, that’s just an average.  We know that from our scoring methodology, you have to have a higher score than that if you’re interacting with content.

So, when we break down the Search Organic by Visitor ID’s, we see a very different story.

Look at that…the top 10 all have scores above 300!

So to get to an average of 7.88, we must have an awful lot of really low end scores that come in through organic search.  And actually, that makes sense…we have a lot of reference content across our site, such as Library and School content that is very popular and that’s likely where these users with low scores are going.  Not the ones we’re really interested in from this scoring standpoint though.

Extracting Visitor ID’s

The next fancy bit of functionality, once you’ve enabled the Unique Visitor ID, is the ability to extract those visitors that have met a certain criteria (success event).

If you now click on an item in, for example, the Last Campaign report (at the individual campaign tracking code level), instead of heading off to the summary page for that item (which by the way, is useful also), you’ll get a new menu:

If you select the success event you want the visitor ID’s for, you’ll end up in a data warehouse data extract screen, where the report can be scheduled for immediate delivery:

Here we’re extracting visitor ID’s for those that submitted an application having come to us from Google Australia.

The Icing

I think the real icing on this cake is the ability to look at all visitor scores at the individual visitor level, irrespective of the segment, so that you can get an idea of what their scores are, to help better gauge those low/medium/high engagement buckets.  It’s also really useful to see at the segment too.

You’ll need to use Excel to process the data (and one thing I should point out is that an eVar can only have a maximum of 500,000 unique values in a given month – so if you get more visitors than that, then this might not work for you).

Other ways to use this is to extract emails that have responded to a campaign – if you capture the memberID in the eVar you can then extract the data for later use.

The next thing we need to do is re-configure our Discover segment for the interaction index and amalgamate these metrics into the overall engagement scoring methodology.

This is a follow on post to my previous one about measuring that elusive engagement.  This post focuses on the aspect of applying a score to visitor interactions, as they interact with your content and applications.

Visitor scoring is fairly simple – especially in SiteCatalyst, and by leveraging the data in Discover through segmentation, (and ultimately in SiteCatalyst 15), it’ll give you even more insight into visitor engagement.

Visitor scoring measures and assigns a relative value to individual customers and prospects based on their actions and behaviors over time. You can determine intent and engagement – even before visitors convert.

Once you’ve identified your most valuable visitors, you can dissect their actions to determine the campaigns, keywords, referring sites and offline touch points that engage them – and invest more on these efforts.

Scoring

The basic premise of a visitor score is to give them “points” for certain activities.  What score you decide to give them is entirely up to you…the important thing is that they accrue points over time, and you can use those points to create segments of visitors that have exceeded certain thresholds.

In doing so, you’ll be able to compare different visitors, for example, those with a high score, to those with a low score, from different traffic sources, or across different campaigns.

The theory behind this is that visitors with higher scores are likely to be more engaged – they’ve accrued more points over time, by doing the things you want them to do.  It’s not just looking at conversions, although they will probably feature in your scoring methodology.

Imagine for a moment that you have a number of visitors:

Visitor 1 – comes to your site from Organic search, looks at 5 different products, watches 2 videos, signs up for your newsletter, but doesn’t buy anything.

Visitor 2 – comes to your site from Organic search, looks at 1 product and buys it.

Visitor 3 – comes to your site through an email campaign, and views 8 of your products.

Visitor 4 – comes to your site through Paid Search, looks at searches for a product, views it once and then looks at 3 FAQs.

Which visitor is more valuable to you?  Who is more engaged?  Visitor scoring will assist you to better understand the answer to that – and when you slot this Visitor Scoring methodology into the Interaction Index, and use Discover, you really do get a good proxy for engagement.

Ultimately most sites exist for a specific reason, whether it be to convert the visitor to a purchase, or allow them to self serve, or engage them with your content etc.   And we all want them to do that.  But only a small percentage will.  In fact, in general, it’s around 3-5% of visitors that actually “convert”.

First things first

What you need to do initially is agree on a set of interactions across your site, and then apply an arbitrary score to them, either between 1 and 10 or 1 and 100.

For example:

1. Home page viewed or Landing Page Viewed = 1 pt
2. View a general content page = 5 pts
3. View a Product = 10 pts
4. Search = 20 pts (I tend to think that if a visitor is going to take the time to search for something they can’t find, they’re more engaged, hence the higher score)
5. Watch a video = 30 pts
6. Use an interactive tool = 40 pts
7. Sign Up for a newsletter = 50 pts
8. Provide feedback/rate and review/comment = 75 pts
9. Buy a product or conduct a self service transaction = 100 pts

Don’t worry about the actual scores, but do leave room for additional interactions that you can add in if you need to (but remember to communicate it out if you do add in others, as scores will go up).

Now that you’ve scored various activities, you need to implement the code to measure those interactions.

Implementing the scoring

In SiteCatalyst that’s easy enough to do – you need an eVar and a success event.

Set up the success event as a Counter Event. 

Note that you can now have counter events where they take a value other than 1, which is what is needed for the  scoring.  However, I believe you’ll need version H23 of the s_code to support this. 

Create the eVar as a Counter eVar, not a text eVar.

Visitor scoring is done by simply passing the score value as a +number into the eVar and the success event is set with the =number (yes, you no longer need to pass the event through the s.product string).

So, for example, on your homepage, you simply add the following:

s.events="event1=1"; s.eVar1="+1";

On your search page, you’d include:

s.events="event1=20"; s.eVar1="+20";

(The above obviously assumes you’ve used event1 and eVar1 for Visitor Scoring.

Complete that process for each of the key interactions and you’re set.  We implemented ours directly into our s_code through a variety of s.pageName value matches, or product views, or other success events occurring.

To put this in context, we’ve implemented the following scoring:

1. Homepage view = 1pt
2. Searched for something = 5pts
3. Viewed a course = 20pts
4. Completed a form = 30pts
5. Used one of our interactive tools = 50pts
6. Opted in to something = 70pts
7. Submitted an application = 100pts

Making it legible

First thing you need to do when you implement this type of thing is make it all a bit more usable.

When you look at the raw data (your eVar against conversions), you’ll see something like:

Remember that the success event (Application Submitted) is associated back to the value seen in the eVar, when the success event was applied – so, in the example above, 37 apps were submitted when visitors had a score of 190; 22 had a score of 211, etc.  Notice also that as we’re showing both Apps and Leads, we also see that they don’t correspond; Leads has a very different scoring result (see below).

Overall, it doesn’t tell you much – you actually want to group the eVars together.  So, use SAINT to classify them into buckets. 

Important Tip: When you use Excel to classify your eVar, the key column will be the interaction score.  You need to keep that column showing 2 decimal places – when you export from SAINT, it has the decimal places on it, when you open in Excel, the decimal places disappear, and if you re-save without decimals and upload again, they keys will be different, and the reports won’t work.  So, put the key column to 2 decimal places and then classify it.

That brings about the problem of what your buckets should be…well, that’s up to you.  Only your data will tell you that.  But, you can experiment with different buckets to see what works best.  To be honest, it’s best to do this in Excel, rather than SAINT, as you’ll want instant gratification.  And you’ll obviously need data collected before you can classify…so it’s best to run it without classifications for a while.

We classified ours following a bit of analysis using Excel to figure out the best buckets for Applications Submitted and Optins.  As it turned out, after a lot of playing around, we chose a logarithmic scale, as it seemed to group everything the best:

What we see in Excel is that with these buckets, 80% of applications occur before a visitor has a score of 320.  We also note that most apps occur when visitors have a score of between 160-320…our sweet spot.

So, using SAINT, we classified our Engagement Value eVar into our buckets and uploaded back to SAINT. 

Tip: Due to the size of the file, we typically use an FTP upload now – it took less than 10 minutes to classify the data in the reports.

Now re-run the above SiteCatalyst report, this time using your buckets:

Remember above I said that leads obviously have a very different score?  You can see that in the above report, that Lead Completes tend to happen when the visitor has a score of between 40 and 160.

What we see from Excel with Leads is that 80% of leads occur before a visitor has a score of 80 – which means that those that become leads, do so pretty quickly; they haven’t visited much other content (otherwise their score would be higher) – which is great news for us!

Another way to classify the scores is to set “low, medium, and high” buckets for engaged visitors.  I’m still trying to figure out what we should use as those buckets, as we have a very large spread.  Standard Deviation will probably assist in that one eventually.

Calculated Metrics

At this point in time, you probably also want to create a few calculated metrics to get some averages out.

Ones that I’d recommend are:

1. Score per Visit = [Engagement Score] / [Visits]
2. Score per Search = [Engagement Score] / [Instances (Report-Specific)]
3. Score per Referrer = [Engagement Score] / [Instances (Report-Specific)]

Why is Instances in there twice?  Because a) Visits are not available across every report type and b) the naming convention just makes it a bit easier to understand (IMHO).

Did you know you can also trend calculated metrics?

Now that you’ve got all this wonderful new capability, what do you do with it…?

Segment it (of course)

There’s all sorts of different ways you can use this…all basically segmentation based.

You can look at traffic sources, campaigns, keywords, geographics, user type segments, etc.  You can trend and compare your segmented values over time.

In the example on the right, we see that while Australian visitors have an average score of 5.82, the highlighted items have much larger scores, indicating that they are engaging with more of the things we want them to do.  The example on the right was extracted using Discover, as in SiteCatalyst 14 the capability with geo-demographic reports is somewhat limited to country/visits.

In the next example, we’re looking at scores by course interest.  The course category is actually an eVar that is set in various places across the site – similar to a product category.  When a visitor browsers different bits of content, we’ll set their course category differently, which is then used for Test & Target purposes…but works well here too.

We see that while Undergrad had more course views during this short time frame, they scored actually slightly less than Postgrad visitors.  This would indicate that PG visitors tend to read more, which would also make sense, as it’s a bigger purchase decision for them.

And as a final example, scores by different campaign types (Organic, Paid, Campaigns, etc).

Hmmm…seems Paid Search is doing really well; not a huge amount of traffic during this period, but they are interacting a lot.  Break that down by keyword or Adword group and you’ll get even more insight.

Here we’re looking at Branded vs. Non Branded keywords (and their top keywords):

Seems that branded search terms generate a higher interaction, and in fact when they type in the full name of the university, they have the highest interaction.

Multi-scoring techniques

You’re not limited to having just one scoring methodology.  We’re actually in the process of implementing a second one for a completely different reason.  All you need is a spare eVar, another success event, and bit of time.  Rinse and repeat!

Comparative and Trending

Of course, you can also trend on the calculated metrics to ensure your overall interaction score is going in the right direction, and you can trend by different segments too.

Likewise, you can do comparative analysis using different dates to compare the interaction scores.

Extending the scoring

In SiteCatalyst 15, you can leverage the segments for visitors with scores higher than X – you can either use the value of the event, or you can use the SAINT classifications. 

Additionally, you can use these in DataWarehouse and Discover – or start from Discover and put the segments back into SiteCatalyst.

And, you can leverage the segments and scores in Test & Target to further optimise user journeys and conversions.

There’s just a whole heap of different ways to use this information.

Next post

In my next post, I’ll combine the above interaction Visitor scoring into the Discover segments for a full on Engagement Analysis, replacing the previously described interaction score with this one.

Now, there’s a hot topic.  Measuring engagement.  One of the most widely debated topics in web analytics.

What is engagement and how do we measure it?

Engagement, unfortunately, is not derived from a single measure.  It’s not time on site.  It’s not how many pages they viewed.  It’s not bounce rates and it’s not about conversions.

Engagement is about a lot of things.  What is an engaged visitor and how do you measure engagement?

“Visitor Engagement is an estimate of the depth of visitor interaction against a clearly defined set of goals.” Eric T. Peterson and Joseph Carrabis.

A while ago, I came across their paper through Web Analytics Demystified, entitled “Measuring the Immeasurable: Visitor Engagement”.  While I won’t go into it in any detail, I will suggest that you read it, as it’s the background of this post.

The premise of the paper is that visitor engagement is made up of 7 different metrics, and expressed through one formula:

where:

Engagement can be expressed as the average of the sum of indexes, across specific segments, according to a:

1. Click Depth Index – which captures the contribution of page and event views
2. Duration Index – capturing the contribution of time spent on site
3. Recency Index – which captures the visitor’s “visit velocity”—the rate at which visitors return to the web site over time
4. Loyalty Index- the level of long-term interaction the visitor has with the brand, site, or product(s)
5. Brand Index – the apparent awareness of the visitor of the brand, site, or product(s)
6. Feedback Index – qualitative information including propensity to solicit additional information or supply
7. Interaction Index – visitor interaction with content or functionality designed to increase level of Attention

According to Eric T. Peterson, “Visitor Engagement is a function of the number of clicks (Ci), the visit duration (Di), the rate at which the visitor returns to the site over time (Ri), their overall loyalty to the site (Li), their measured awareness of the brand (Bi), their willingness to directly contribute feedback (Fi) and the likelihood that they will engage in specific activities on the site designed to increase awareness and create a lasting impression (Ii).”

When applied at the visitor level, on a per-visitor basis, they combine to form a pretty good proxy for visitor engagement.

Having read the paper, I was intrigued, and decided to use Discover to implement this…to some pretty insightful results.

A summary of the indexes

Click Depth Index

The percentage of your overall audience that has a minimum threshold of an acceptable number of page views per session.  If you see that on average, visitors “convert” after viewing at least 5 pages, then your minimum threshold would be 5 pages per visit.

Duration Index

The percentage of your overall audience that has a minimum threshold of an acceptable amount of time on site per session.  If you see that on average, visitors “convert” after spending at least 10 minutes on your site, then your minimum threshold would be 10 minutes.

Recency Index

The percentage of your overall audience that returns and converts within an acceptable amount of time (generally days).  If you notice that most visitors convert between 1 and 10 days, then you’d be looking for visitors with a return frequency of <= 10 days.

Loyalty Index

The percentage of your overall audience that has a repeat visit frequency in excess of a minimum threshold.  For example, if you notice that many visitors convert after visiting your site more than three times, then your threshold would be a visit count of at least 3.

Brand Index

The percentage of your overall audience that comes to your site either directly, or through branded search terms.

Feedback Index

The percentage of your overall audience that completes feedback on your site, or participates in rating or reviewing content, or commenting on blogs.

Interaction Index

The percentage of your overall audience that interacts with specific content on your site, or engages in an activity on your site.  There are no thresholds for this index – they are simply counts of.

Note: while you can count pre-defined activities on your site, it is better to score visitor interaction.  I’ll be doing a post on visitor scoring shortly.

Using Discover

Discover was built for this!  It’s very easy to create segments within Discover and apply them across various views to gain insight.

Firstly what we did was to look at some of the thresholds to understand what our “Anonymous” segment of traffic does.  Our anonymous segment is made of non-student and non-staff traffic, which we already have segments for in both Discover and SiteCatalyst.

We figured out what our minimum page views per session should be, the average duration, frequency of visit etc, by looking at them from a conversion standpoint…i.e. how many pages does a converter see, on average, before converting.

Once we’d done that, our 7 segments were easy to define as follows:

1. Click Depth Index – Visitor container, Path length > 10
2. Duration Index – Visitor container, Seconds spent per visit > 1800 (30 minutes)
3. Recency Index – Visitor container, Return Frequency <= 7-14 days
4. Loyalty Index – Visitor container, Visit number >= 2
5. Brand Index – Visitor container, Organic Search Keyword contains “Murdoch” or Visit without referrer
6. Feedback Index – (we don’t use this one)
7. Interaction Index – Visitor container, any of the following events: Lead Complete, Application Complete, Form Complete, Tool Name

The 8th segment was All Visits.  In each case, we used the Visitors metric to view the number of visitors that were part of each index.

If we view this against referring sites, what we end up with is the number of visitors that match each segment rule:

Export to Excel

What we need to do now is export the data to Excel to do the averages and generate the final engagement value.

Simply select the first item “None”, click Ctrl+A for select all, then click Ctrl+C for copy.

Open Excel, and paste the raw data into a new sheet.

Then it’s simply a matter of calculating one columns percentage as a percentage of the All Visits – Visitors column.

Once you’ve done that for each column, you have the indexes for each segment.  Now you just average all of the indexes to get an engagement metric.

For example, we see in the above that Direct Traffic, “none” in the above report, has an overall engagement value of 23%.  But if we look at the other columns, we also see that they are at the median value on Click Depth, whereas traffic from deewr.gov.au is well above the median.

A couple of interesting things have also been highlighted in the above, for example, traffic from Google Singapore is actually far more engaged than traffic from Google Australia – now that’s interesting.

Of course, you should always look at engagement via larger segments, for example, by Campaign, by Site, by Time of Day, Day of Week etc.

While Tuesday comes out overall for a better engaged visitor, I’ve highlighted other interesting things, such as on Saturday and Sunday visitors click more, but more visitors spend time on Saturdays.  During the week is better for branded search term visitors, and Wednesdays seems to be better overall for key interactions.

Multiple Sites

If you have multiple sites, such as microsite etc, you might want to check engagement across them to see if they are dramatically different, so you can then begin to try to understand why.

In the above calculation, I’ve removed the Feedback and Interaction indexes from the calculation, as they would skew the results.  It’s interesting that while the main University site has an engagement index of 23.67, versus a median of 20.43%, sites like Mobile and Maps have a very high engagement value.

Segmentation

Once you’ve got the basic engagement working and you’re looking at things from the overall perspective, you can then easily begin to look at engagement by different segments.

In Discover, you can create segments on-the-fly and apply them across your other segments, by simply dragging the new segment onto the filtered workspace. For example, we segment the above by Anonymous visitors, after we’ve built the overall segments.  We can do the same for Converting Visitors, or Social Network visitors, or Campaign Visitors, or just Mobile visitors, or different content areas across the site etc.  Discover makes it very easy to do this.

Once the spreadsheet is set up, all you need to do is copy the data back into the sheet and you’ve re-run the engagement metric – in about 5 seconds.

Discover just rocks for this real-time, conscious stream of thought, type of analysis.

In summary

There’s lots of different ways to look at engagement, and hopefully, this will help you understand that there is no single metric, and engagement values change based on various lenses.  But, with the above combination of metrics from the very useful paper by Eric T. Peterson, I believe that we’re closer to understanding engagement, which will help us to modify our sites, or target content better, to try to achieve better levels of engagement by those who are below the medians.

As an aside, part two of this post will be about Visitor Scoring, which is a better Interaction Index than the one demonstrated above – and can be used directly in SiteCatalyst reports.  It involves a bit of custom code for your s_code, and a bit of forethought, but easy enough to do…but I’m saving that for a bit later this week.

Drop me a line or comment below with ways that you are measuring visitor engagement.