While this might be a quickie, it’s a biggy.  A big one in terms of the amount of data just uploaded through SAINT.  In fact, we’ve just uploaded around 1 million rows of data, with 6 columns per row.

And it didn’t even blink!  Gotta love that!

So why do we have a million rows of data?
Customer segmentation of course.

This was actually done for one of our other clients.

The rationale?

To segment conversions and transactions by customer type, segment, previous segment, needs group etc.  And SAINT enables that capability.

How?

Firstly create an eVar that the raw identifier will go into.  This might be an account number, a customer ID etc.  Then, using the admin, create the classifications on the eVar for the relative columns you need.  At this point I always create the classification hierarchy as well, just so I can envision how I want the data to be reported and drilled down though.

When you create the classifications, the SAINT file is also created and made available for download.

I opened the SAINT template in Excel and copied my customer segment data into it in blocks of 100,000 records.  There’s a number of reasons for this, not the least of which is to keep the file size down, but also to make it easier if an upload does fail – at least you can deal with 100,000 rows better than 1 million rows.

So I’ve now got 10 files, each file contains 100,000 rows and 6 columns of data per row.  Each file was about 5mb.

You can’t upload that much data through the browser, so you need to use the FTP Import capability.

In the SAINT admin, select Import File, click on the FTP Import and then Add New:

You’ll then get a popup that asks you to select a bunch of things to create an FTP account:

Select the data to be classified, move the report suite or suites to the box on the right, select the import options and add in your email address.

Check the box and hit save.

A new FTP account has just been created on the Omniture servers and you’ll get a confirmation screen showing the address, username and password.

Open it up in an FTP client and upload your SAINT files to the FTP server.

You’re not quite done yet though.

You also need to create a series of empty files, with a .fin extension, named exactly the same as your SAINT files.  These are “finish” files and are crucial to the upload.  They’re completely empty files – any text editor can create them.  Just make sure they are named exactly the same, case sensitive.

Upload those .fin files and you’re done.

Now, go have a coffee, have some lunch or dinner or whatever and come back later.

Progress

You can kind of check on progress by refreshing the FTP list of files.  Omniture removes the files from the FTP directory when it begins to process them, so you can kind of get an idea of where things are.

Time Frame

I uploaded the files around 4pm. 

At 10:30pm I did a data extract by FTP of all data to see where it was up to…it was done.  Shortly thereafter, I got an email saying it was done, without any failures.

Easy as pie.  No muss no fuss.

While we’re using customer segments, it could just have easily been customer demographics, technographics or any other form of data.  The point is, 1 million rows and it didn’t even blink.

There’s a few things to watch out for though when importing that much data.

There is a limit on the amount of unique values (500,000) that will be reported against in a given month.  We’re ok – we won’t see that limit.

Recommendations are that file sizes be kept under 30mb for the initial load, and then subsequent refreshes less than 5mb.  So we’re still ok.

And the import time will vary depending on many things, including how busy their import routines are.  You get in the queue and everyone loves a queue.

But that was it.  1 million rows of customer data now available for segmentation nirvana in SiteCatalyst – and DataWarehouse, and Discover, and Test and Target.  We’re off to the races!

And while the first run of this was a manual run, future updates can easily be automated now that the FTP site is created.  Just remember your .tab and .fin files must be named the same.

This is the third (but not final) post in the series on Visitor Engagement.  One of the problems with the Visitor Scoring method that I previously described, is that, at the end of the day, you’re still somewhat limited to viewing scores at the “average” level, by segment.

That presents a number of challenges because the average is precisely that…and the underlying scores vary dramatically within each segment.

But there is a way to see what each and every visitor score is, or even within the different segments…and it’s called the Unique Visitor ID.  You can see at the visitor level, how many times they’ve returned, how many “things” they’ve done, such as searches, product views, revenue etc.

This is really the icing on the proverbial cake.

Visitor ID

There’s a seldom used implementation method in SiteCatalyst called Dynamic Variables which easily enables all of this capability.

All you need is a spare eVar and you’re in heaven.

Step 1

Simply populate an eVar with “D=s_vi” and the result is that SiteCatalyst will populate the eVar with the unique visitor cookie ID.  If you test it with WASP or something similar, you’ll only see that D=s_vi got passed, but the magic begins.

s.eVar56="D=s_vi";

Step 2

Go into you Admin console, and on the report suite where you want this collected, go to Conversion > Unique Visitor Variable section.

You’ll be presented with a dropdown of available eVars that can be selected to use as the Visitor ID…simply select the one that you’re using and hit save.

That’s it – you’re done!  All there is to it.

Rest assured, it does not impact anything to do with the way that SiteCatalyst measures unique visitors – its completely separate and safe to use.

And now it pops

So, the first thing you’ll see is a new report on your eVar. The report is a list of all Visitor Cookie ID’s captured during the timeframe selected.  As it’s a conversion variable, you can add in whatever success events you like:

Add in your Visitor Score and you can see what their points are.  As you can see from above, I’ve highlighted an interesting visitor (#7)…they were on their 2nd visit, they searched 12 times, viewed 16 courses and submitted an application.  Their score is currently 542.  I’m wondering if visitor #5 is a candidate for an app too…similar behaviour is forming.

Segment-based Visitor ID’s

Ok, now open up a report such as Campaigns (we use the Unified Traffic Sources VISTA rule so it may look a little different if you don’t use it).

Remember the challenge I mentioned at the top of this post…with average scores.

We see below that the average visitor score for Organic Search is 7.88.  But, that’s just an average.  We know that from our scoring methodology, you have to have a higher score than that if you’re interacting with content.

So, when we break down the Search Organic by Visitor ID’s, we see a very different story.

Look at that…the top 10 all have scores above 300!

So to get to an average of 7.88, we must have an awful lot of really low end scores that come in through organic search.  And actually, that makes sense…we have a lot of reference content across our site, such as Library and School content that is very popular and that’s likely where these users with low scores are going.  Not the ones we’re really interested in from this scoring standpoint though.

Extracting Visitor ID’s

The next fancy bit of functionality, once you’ve enabled the Unique Visitor ID, is the ability to extract those visitors that have met a certain criteria (success event).

If you now click on an item in, for example, the Last Campaign report (at the individual campaign tracking code level), instead of heading off to the summary page for that item (which by the way, is useful also), you’ll get a new menu:

If you select the success event you want the visitor ID’s for, you’ll end up in a data warehouse data extract screen, where the report can be scheduled for immediate delivery:

Here we’re extracting visitor ID’s for those that submitted an application having come to us from Google Australia.

The Icing

I think the real icing on this cake is the ability to look at all visitor scores at the individual visitor level, irrespective of the segment, so that you can get an idea of what their scores are, to help better gauge those low/medium/high engagement buckets.  It’s also really useful to see at the segment too.

You’ll need to use Excel to process the data (and one thing I should point out is that an eVar can only have a maximum of 500,000 unique values in a given month – so if you get more visitors than that, then this might not work for you).

Other ways to use this is to extract emails that have responded to a campaign – if you capture the memberID in the eVar you can then extract the data for later use.

The next thing we need to do is re-configure our Discover segment for the interaction index and amalgamate these metrics into the overall engagement scoring methodology.

This is a follow on post to my previous one about measuring that elusive engagement.  This post focuses on the aspect of applying a score to visitor interactions, as they interact with your content and applications.

Visitor scoring is fairly simple – especially in SiteCatalyst, and by leveraging the data in Discover through segmentation, (and ultimately in SiteCatalyst 15), it’ll give you even more insight into visitor engagement.

Visitor scoring measures and assigns a relative value to individual customers and prospects based on their actions and behaviors over time. You can determine intent and engagement – even before visitors convert.

Once you’ve identified your most valuable visitors, you can dissect their actions to determine the campaigns, keywords, referring sites and offline touch points that engage them – and invest more on these efforts.

Scoring

The basic premise of a visitor score is to give them “points” for certain activities.  What score you decide to give them is entirely up to you…the important thing is that they accrue points over time, and you can use those points to create segments of visitors that have exceeded certain thresholds.

In doing so, you’ll be able to compare different visitors, for example, those with a high score, to those with a low score, from different traffic sources, or across different campaigns.

The theory behind this is that visitors with higher scores are likely to be more engaged – they’ve accrued more points over time, by doing the things you want them to do.  It’s not just looking at conversions, although they will probably feature in your scoring methodology.

Imagine for a moment that you have a number of visitors:

Visitor 1 – comes to your site from Organic search, looks at 5 different products, watches 2 videos, signs up for your newsletter, but doesn’t buy anything.

Visitor 2 – comes to your site from Organic search, looks at 1 product and buys it.

Visitor 3 – comes to your site through an email campaign, and views 8 of your products.

Visitor 4 – comes to your site through Paid Search, looks at searches for a product, views it once and then looks at 3 FAQs.

Which visitor is more valuable to you?  Who is more engaged?  Visitor scoring will assist you to better understand the answer to that – and when you slot this Visitor Scoring methodology into the Interaction Index, and use Discover, you really do get a good proxy for engagement.

Ultimately most sites exist for a specific reason, whether it be to convert the visitor to a purchase, or allow them to self serve, or engage them with your content etc.   And we all want them to do that.  But only a small percentage will.  In fact, in general, it’s around 3-5% of visitors that actually “convert”.

First things first

What you need to do initially is agree on a set of interactions across your site, and then apply an arbitrary score to them, either between 1 and 10 or 1 and 100.

For example:

1. Home page viewed or Landing Page Viewed = 1 pt
2. View a general content page = 5 pts
3. View a Product = 10 pts
4. Search = 20 pts (I tend to think that if a visitor is going to take the time to search for something they can’t find, they’re more engaged, hence the higher score)
5. Watch a video = 30 pts
6. Use an interactive tool = 40 pts
7. Sign Up for a newsletter = 50 pts
8. Provide feedback/rate and review/comment = 75 pts
9. Buy a product or conduct a self service transaction = 100 pts

Don’t worry about the actual scores, but do leave room for additional interactions that you can add in if you need to (but remember to communicate it out if you do add in others, as scores will go up).

Now that you’ve scored various activities, you need to implement the code to measure those interactions.

Implementing the scoring

In SiteCatalyst that’s easy enough to do – you need an eVar and a success event.

Set up the success event as a Counter Event. 

Note that you can now have counter events where they take a value other than 1, which is what is needed for the  scoring.  However, I believe you’ll need version H23 of the s_code to support this. 

Create the eVar as a Counter eVar, not a text eVar.

Visitor scoring is done by simply passing the score value as a +number into the eVar and the success event is set with the =number (yes, you no longer need to pass the event through the s.product string).

So, for example, on your homepage, you simply add the following:

s.events="event1=1"; s.eVar1="+1";

On your search page, you’d include:

s.events="event1=20"; s.eVar1="+20";

(The above obviously assumes you’ve used event1 and eVar1 for Visitor Scoring.

Complete that process for each of the key interactions and you’re set.  We implemented ours directly into our s_code through a variety of s.pageName value matches, or product views, or other success events occurring.

To put this in context, we’ve implemented the following scoring:

1. Homepage view = 1pt
2. Searched for something = 5pts
3. Viewed a course = 20pts
4. Completed a form = 30pts
5. Used one of our interactive tools = 50pts
6. Opted in to something = 70pts
7. Submitted an application = 100pts

Making it legible

First thing you need to do when you implement this type of thing is make it all a bit more usable.

When you look at the raw data (your eVar against conversions), you’ll see something like:

Remember that the success event (Application Submitted) is associated back to the value seen in the eVar, when the success event was applied – so, in the example above, 37 apps were submitted when visitors had a score of 190; 22 had a score of 211, etc.  Notice also that as we’re showing both Apps and Leads, we also see that they don’t correspond; Leads has a very different scoring result (see below).

Overall, it doesn’t tell you much – you actually want to group the eVars together.  So, use SAINT to classify them into buckets. 

Important Tip: When you use Excel to classify your eVar, the key column will be the interaction score.  You need to keep that column showing 2 decimal places – when you export from SAINT, it has the decimal places on it, when you open in Excel, the decimal places disappear, and if you re-save without decimals and upload again, they keys will be different, and the reports won’t work.  So, put the key column to 2 decimal places and then classify it.

That brings about the problem of what your buckets should be…well, that’s up to you.  Only your data will tell you that.  But, you can experiment with different buckets to see what works best.  To be honest, it’s best to do this in Excel, rather than SAINT, as you’ll want instant gratification.  And you’ll obviously need data collected before you can classify…so it’s best to run it without classifications for a while.

We classified ours following a bit of analysis using Excel to figure out the best buckets for Applications Submitted and Optins.  As it turned out, after a lot of playing around, we chose a logarithmic scale, as it seemed to group everything the best:

What we see in Excel is that with these buckets, 80% of applications occur before a visitor has a score of 320.  We also note that most apps occur when visitors have a score of between 160-320…our sweet spot.

So, using SAINT, we classified our Engagement Value eVar into our buckets and uploaded back to SAINT. 

Tip: Due to the size of the file, we typically use an FTP upload now – it took less than 10 minutes to classify the data in the reports.

Now re-run the above SiteCatalyst report, this time using your buckets:

Remember above I said that leads obviously have a very different score?  You can see that in the above report, that Lead Completes tend to happen when the visitor has a score of between 40 and 160.

What we see from Excel with Leads is that 80% of leads occur before a visitor has a score of 80 – which means that those that become leads, do so pretty quickly; they haven’t visited much other content (otherwise their score would be higher) – which is great news for us!

Another way to classify the scores is to set “low, medium, and high” buckets for engaged visitors.  I’m still trying to figure out what we should use as those buckets, as we have a very large spread.  Standard Deviation will probably assist in that one eventually.

Calculated Metrics

At this point in time, you probably also want to create a few calculated metrics to get some averages out.

Ones that I’d recommend are:

1. Score per Visit = [Engagement Score] / [Visits]
2. Score per Search = [Engagement Score] / [Instances (Report-Specific)]
3. Score per Referrer = [Engagement Score] / [Instances (Report-Specific)]

Why is Instances in there twice?  Because a) Visits are not available across every report type and b) the naming convention just makes it a bit easier to understand (IMHO).

Did you know you can also trend calculated metrics?

Now that you’ve got all this wonderful new capability, what do you do with it…?

Segment it (of course)

There’s all sorts of different ways you can use this…all basically segmentation based.

You can look at traffic sources, campaigns, keywords, geographics, user type segments, etc.  You can trend and compare your segmented values over time.

In the example on the right, we see that while Australian visitors have an average score of 5.82, the highlighted items have much larger scores, indicating that they are engaging with more of the things we want them to do.  The example on the right was extracted using Discover, as in SiteCatalyst 14 the capability with geo-demographic reports is somewhat limited to country/visits.

In the next example, we’re looking at scores by course interest.  The course category is actually an eVar that is set in various places across the site – similar to a product category.  When a visitor browsers different bits of content, we’ll set their course category differently, which is then used for Test & Target purposes…but works well here too.

We see that while Undergrad had more course views during this short time frame, they scored actually slightly less than Postgrad visitors.  This would indicate that PG visitors tend to read more, which would also make sense, as it’s a bigger purchase decision for them.

And as a final example, scores by different campaign types (Organic, Paid, Campaigns, etc).

Hmmm…seems Paid Search is doing really well; not a huge amount of traffic during this period, but they are interacting a lot.  Break that down by keyword or Adword group and you’ll get even more insight.

Here we’re looking at Branded vs. Non Branded keywords (and their top keywords):

Seems that branded search terms generate a higher interaction, and in fact when they type in the full name of the university, they have the highest interaction.

Multi-scoring techniques

You’re not limited to having just one scoring methodology.  We’re actually in the process of implementing a second one for a completely different reason.  All you need is a spare eVar, another success event, and bit of time.  Rinse and repeat!

Comparative and Trending

Of course, you can also trend on the calculated metrics to ensure your overall interaction score is going in the right direction, and you can trend by different segments too.

Likewise, you can do comparative analysis using different dates to compare the interaction scores.

Extending the scoring

In SiteCatalyst 15, you can leverage the segments for visitors with scores higher than X – you can either use the value of the event, or you can use the SAINT classifications. 

Additionally, you can use these in DataWarehouse and Discover – or start from Discover and put the segments back into SiteCatalyst.

And, you can leverage the segments and scores in Test & Target to further optimise user journeys and conversions.

There’s just a whole heap of different ways to use this information.

Next post

In my next post, I’ll combine the above interaction Visitor scoring into the Discover segments for a full on Engagement Analysis, replacing the previously described interaction score with this one.

One of the nice things about Omniture is the ability to export information out to other systems.  We use this feature to generate tag clouds on our site, based on the most popular courses viewed over the last 30 days, segmented for different audiences.

In order to do this, there are a few things that need to be done first.

Firstly, we report course views as products, passing a shortened name of the course from our content management system and database, to the s.products variable, such as:

s.products = “;Marketing-and-the-Media”;
s.events = “prodView,event5″;

We have set up event5 as a success event, signifying a course view.

As we have multiple pages associated to a course, we make sure that we only pass the s.products and s.events values once per course view, irrespective of the page within the course a user is looking at.  This is done by using some custom code within our s_code file.

In SiteCatalyst, we then use SAINT classifications to generate Course-based reports, associated to schools, faculties, type of course (undergrad or postgrad) etc.  This allows us to get in-depth information on our course activity, along with conversions etc.

Audience segmentation

A common reporting segmentation for us is to compare Australian traffic to International traffic, so we have created two segments, using the segment builder.

The Australian segment includes any visit where the GeoCountry was Australia.  The International segment includes any visit where the GeoCountry was not Australia.

Exporting the data

We then use DataWarehouse to create two reports, based on the last 30 days of activity.  Each report uses the segment defined above, with the Course name and the number of Product views (as we use the product variable to set course views).

These two reports are scheduled on a daily basis to export the data to our FTP servers as a CSV file.

Once we have the files, we import the data into a database along with the date of the file, so we can use that later.

Now we have the last 30 days of activity, by each course, by traffic source as a dataset that we can use on our site.  It’s then a fairly straightforward process to match the course name with the URL of the actual course, so it can be used as the link on the tag cloud.

The end result

Each day we query the database and using standard tag cloud calculations, we are then able to re-produce the data back out onto our site.  We currently feed the data back out as an XML file which is read by our Course Browser flash tool – showing both a Domestic and an International view of the most popular courses.

We’re also working on something similar for internal search terms, which will be used to populate a “search as you type” functionality on our search forms, but it will be segmented by audience type – Staff, Student or Anonymous (being general traffic).  That one is a little tougher, because we have to associate the most common destination clicked on, with the searched-for term.  But more on that in a later posting, once we have it working.

So, using a combination of Omniture SiteCatalyst, DataWarehouse and segmentation, we’re able to easily offer our users with quick navigation methods to various pieces of content, thereby enhancing their user journey.

Search is a veritable gold mine that is frequently ignored.

I’m not talking about Search Engines and Keywords, I’m talking about your internal search. Providing you track internal keyword searches, you can gain a wealth of understanding.

Internal search is generally used as a quick wayfinding method, highlighting areas of content that are well used, but are not readily available.  And more often than not, it’s seasonal as well.

The following report shows the top 5 keywords used by students on our University website, smoothed using a 21 day moving average.

Two things immediately jump out.  Firstly, notice how searches for Bookshop increased dramatically in late February, then declined, then Library picked up substantially in March, followed by Exam Timetable in late April (spiking suddenly in late May).

Very seasonal activity, centered around the Student’s life cycle at the University.  In February, having started a new semester, they all needed books.  Then as semester progressed, they all needed access to the Library, and lastly, as exams approached, they all wanted to know their Exam Timetables.

Great – but how can we use this?

Well, firstly this points to the fact that there is no clear navigation on our site to these destinations.  There’s actually a reason for that – our public site is not designed for active current students as they should be using the Student Portal.  That aside, we can assist though in making these destinations easier to get to.

One thing we can do with Omniture is to export these top keywords to our database on a daily basis and then represent them on the site, through things like tag clouds and quick search results, making their lives easier in the process.

By using real data and automating the process, the seasonality of searches will also come through to the site as well.

The other interesting thing about the above chart is their search for timetable, as opposed to exam timetable.  When they start a new semester, they are interested in their class timetables and frequently search for them.  As time progresses and the memory kicks in, that activity slows down.  Again, this can be used in a number of ways – not only online, but through mobile apps and offline support media as well.

In order to track search keywords, simply put the keyword into an s.prop.  You might also want to put the number of results into another s.prop so that you can cross-reference the results on a keyword by keyword basis.  This would highlight any search terms  that return no results.

Segment your keywords

As always, understanding your audience is critical online.  If you segment your audience types, you’ll also be able to see the different searches conducted by different audiences.  We do this and the results are (obviously) very different.  But, without the ability to do it and show it, you’ll be taking an educated guess that they are different.

To do this, put the audience type into another s.prop and use the getAndPersist() plugin to set a cookie.  That way, all activity can actually be segmented by audience type.

Now you can provide some really smart assistance

When you start to build your tag clouds or your quick search lists, you can base the results on the value in the cookie, thereby customizing the information to the user and making it more relevant to them.  Just a few little tricks like this and you’ve helped to increase the usability of your site, which will go along way in driving customer loyalty (and hopefully revenue).