So I promised that I would finally put fingertip to keyboard and talk a little bit more about using Visitor Scoring…to finish up the series that I started a while ago.

If you’ve read my previous posts, you’ll know that we implemented a series of metrics for engagement measurement, culminating in a per-visitor score.

I wanted to share with you some of the insights and benefits of doing all of this, particularly in Discover.

An Engaged Segment

If you remember from the previous post, we talked about using a total of seven different metrics to try to ascertain engagement, or, when used in combination, could identify an engaged user segment.

The final metric was a measure of interaction and there were two ways to implement; either by just counting the number of visitors that participated in specific actions on your site, or the second way; by creating a scoring methodology at the visitor level and leveraging that as the final metric.

A little tougher to do, as it involved some additional code across the site, but definitely implementable, and now, we’re able to use that information to gain even greater insights.

The reason you should segment is because not everyone on your site converts – in fact, probably only around 3-5% actually do what you want them to do.  So you use segmentation to analyse those that have converted and try to understand what made them different, what their different behaviours were, and if possible, try to use that as predictors of future behaviour by the other 95-97%, so that you can lift conversions.

Creating the engaged segment

Using Discover we created a Visitor container that contained the following rules, which we had previously determined worked best for our business.

The rules applied are from 6 of the original 7 indexes that defined engagement for us:

1. Click Depth Index – which captures the contribution of page and event views
2. Duration Index – capturing the contribution of time spent on site
3. Recency Index – which captures the visitor’s “visit velocity”—the rate at which visitors return to the web site over time
4. Loyalty Index- the level of long-term interaction the visitor has with the brand, site, or product(s)
5. Brand Index – the apparent awareness of the visitor of the brand, site, or product(s)
6. Interaction Index – visitor interaction with content or functionality designed to increase level of Attention

Notice the counts at the bottom of the segment creator – those seem to be a bit confusing as the number on the left (216 Visitors) represents the number of visitors that meet the criteria of the segment, but the number on the right (1,071,458 visitors) does not appear to be a count of visitors for the timeframe (for some reason I have yet to figure out).

Additionally, for this particular segmentation example, I wanted to ensure that our engaged visitors have actually become a lead (one of our primary goals).  So our segment includes that, as well as them having a score of greater than 200, which means they’re interacting with a fair amount of content or doing a fair amount of activity (such as applying to come to study with us).

Saving the segment, it’s then ready for use.

Using the segment

One of the first things I looked at was where our engaged users were coming from, their average scores, conversion rates and application rates (or purchase rates), compared to all visitors who are not staff or students.

And I fell off my chair.

While the number is quite small comparatively, due in part to the timeframe I’m using, the score, lead conversion rate (leads/visitors) and application rate (applications/visitors), are significantly higher.  This is to be expected due to the segment, but I wasn’t quite expecting the numbers to be that high.

Visual Site Analysis

I was keen to understand, visually, where these highly engaged users go on our site, so I used the Pathing Site Analysis report in Discover (one of my personal favourites).

Firstly I started with all visits to get a sense of what they do across common pages that we want them to interact with:

Fairly widespread usage of key pages.  The thickness of the arrow indicated volume of traffic from one page to another – the bulk of traffic goes to the Courses homepage, from the site home page.

Next, I applied the same engaged segment:

Ok, I must need velcro pants because I fell off my chair again.

It appears that our highly engaged users take a whole different path.  The colour indicates their propensity to become a lead.  The big red steps are basically the lead capture process through our main tool – Figure Out Your Course.

And they seem to browse around first within courses then become a lead – which is also good to know.  But once they become a lead, they tend to leave the site – which means we need to ensure that we’re effectively communicating with them through other channels, such as email at a later date.

Breaking down the traffic sources

In breaking down the Organic Search by time spent, we also see that a significant portion of highly engaged visitors spend more than 10 minutes on the site, and are more likely to convert to leads having done so, as compared to all visitors – another significant insight for us to use.

In summary

This is just one example of many that could be used on your site.  Once you’ve identified your engaged users, you can segment them further by demographics, or by customer type, or by content viewed, or by member/non-member and so forth.

You can view revenue by engaged/non engaged (bound to be vastly different), or average order value etc. 

If you use Test&Target, you’ll be in a great position to leverage the engaged user segment, targeting non-engaged users differently to increase their engagement levels.

All of this will help you gain better understanding into their behaviours, so that you can then further optimise your site to improve those conversions.

I’m keen to hear from others that have used Visitor Scoring, or Engagement Metrics across their site, coupled with Test & Target to lift conversions.  Let me know what your thoughts or successes are.

I’m incredibly excited.  I’ve been asked to attend and present at the massive Adobe Max 2011 conference, in LA, in October this year.  Wow!

Historically this has all been around Adobe products (Flash, Cold Fusion, Photoshop etc), but with their acquisition of Omniture a while ago, they now have sessions on the Omniture product suite…and I’m doing one of them.

So it’s off to LA I go in for the first week in October.

Can’t wait!

When it comes to searching across the web, we all know that Google is king, but does this still hold true across your own internal network?

Over the past 12 months we have wrestled with this question, particularly in an environment with multiple search mechanisms, manually maintained indexes, and masses of sites that were created when metadata was primarily used to categorise instead of search.

In a series of posts, I’m going to go through our experiences in improving search across our internal network – I’m not suggesting we have found the magic search bullet, or that we’re anywhere near finished tweaking and tinkering, but I do know we’re a hell of a lot closer than where we were at this time last year.

The problem

In our travels across campus, we kept hearing “I can’t find what I’m looking for!”  – not surprising, given that we had;

1. 500+ individual sites, ranging in age (earliest was 1997), metadata (none to Dublin Core to ‘something’) and ownership
2. Inconsistent use of key SEO elements, such as title, headings, tags and meta descriptions across the majority of our sites
3. Multiple sources of content and internal search mechanisms, each with their own set of search results
4. Manually maintained indexes, all categorised and sub-related, together with an in-house redirect mechanism
5. An internal audience of staff and students with heavily cyclical search requests – a search for ‘physics’ at the beginning of semester is more likely to be for text books, and at the end of semester past exam papers

Image: Multiple source-centric result sets;

Given Google’s dominance in search, we quickly went down the path of a Google Search Appliance, or ‘Mini’, which is a self contained rack mounted system that gives you God-like powers over the Google algorithm. We were bringing in a little bit of Google in to magically transform our disparate set of sites into a cohesive set of search results.

Once plugged in, the Mini worked really well – for pages that were properly formatted for organic search.

Pages that were missing or incorrectly using titles, headings and metadata didn’t fare so well, and we found the search results were not the most relevant, as the Mini couldn’t make much sense of most of the content it crawled. We also found that there was no clear way to incorporate the feeds from other systems, with the “how do I…” answers primarily provided by a community of Search Appliance users and resellers, and not Google themselves.

Given the wide ownership of the sites we were working with, updating each with appropriate SEO friendly content was unrealistic. What we needed was a way to;

1. compensate for the lack of SEO content,
2. incorporating multiple sources/ formats of content,
3. allow for cyclical requests to ensure the most relevant results appear, and
4. combine all the different sources of search results into a single set of user-centric search results.

Enter Adobe Search&Promote

If you’re a regular visitor to this blog, it will come as no surprise that Tim is a power user of Omniture products, steadily working his way around the product wheel. We became aware of the Search&Promote product (then called SiteSearch) which promised to solve our key internal search issues.

Search&Promote uses a search algorithm to organically crawl your sites, in addition to ranking rules based on a wide range of configurable data. Once you’ve defined your rules, you can adjust the overall balance between your ranking rules and natural search relevance.

Where there is a lack of metadata, Search&Promote can be configured to dynamically inject metadata on crawl, based on a URL pattern. Additional custom metadata can also be injected to create facets (filters) that allow users to drill further down into predefined categories.

If your multiple sources of content can be transformed into XML feeds, then that content can be crawled, categorised, and integrated with the organic results by Search&Promote.

Yes, there are other internal search products on the market that will do the above, however there is one thing that Search&Promote has over its competitors – the ability to tightly integrate with SiteCatalyst and Test & Target.

We’ve known for some time that internal search terms follow highly cyclical patterns as our student (and staff) needs change over the semester. We’ve helped them find what they’re looking for using of real-time SiteCatalyst data in search-as-you-type and tag cloud mechanisms, however with Search&Promote we now have the opportunity to take internal search to the next level.

In the report below (7 day moving average) you can see two popular search results across three semesters peaking at different times during the semester;

Notice how ‘bookshop’ peaks at the beginning of semester, then dies down, only to peak again at the beginning of the following semester. No surprises here, but it does coincide with a significant increase in page views across the Bookshop website.

Then look at the results for ‘timetable’ – there’s a peak at both the beginning and end of semester. The difference here is that people are actually looking for two different pieces of content – their semester timetable at the beginning, and their exam timetable at the end – using the same keyword. Again, the rise in search terms coincides with increased page views across each piece of content.

So, in theory, by looking at the last week’s worth of traffic across our group of sites, we should be able to determine what content students are looking for, then re-rank the search results accordingly. For example, the term ‘timetable’ at the beginning of semester will push results related to the semester timetable to the top, and at the end of the semester push results related to the exam timetable to the top.

Exciting stuff!

Next post – the implementation.

While this might be a quickie, it’s a biggy.  A big one in terms of the amount of data just uploaded through SAINT.  In fact, we’ve just uploaded around 1 million rows of data, with 6 columns per row.

And it didn’t even blink!  Gotta love that!

So why do we have a million rows of data?
Customer segmentation of course.

This was actually done for one of our other clients.

The rationale?

To segment conversions and transactions by customer type, segment, previous segment, needs group etc.  And SAINT enables that capability.

How?

Firstly create an eVar that the raw identifier will go into.  This might be an account number, a customer ID etc.  Then, using the admin, create the classifications on the eVar for the relative columns you need.  At this point I always create the classification hierarchy as well, just so I can envision how I want the data to be reported and drilled down though.

When you create the classifications, the SAINT file is also created and made available for download.

I opened the SAINT template in Excel and copied my customer segment data into it in blocks of 100,000 records.  There’s a number of reasons for this, not the least of which is to keep the file size down, but also to make it easier if an upload does fail – at least you can deal with 100,000 rows better than 1 million rows.

So I’ve now got 10 files, each file contains 100,000 rows and 6 columns of data per row.  Each file was about 5mb.

You can’t upload that much data through the browser, so you need to use the FTP Import capability.

In the SAINT admin, select Import File, click on the FTP Import and then Add New:

You’ll then get a popup that asks you to select a bunch of things to create an FTP account:

Select the data to be classified, move the report suite or suites to the box on the right, select the import options and add in your email address.

Check the box and hit save.

A new FTP account has just been created on the Omniture servers and you’ll get a confirmation screen showing the address, username and password.

Open it up in an FTP client and upload your SAINT files to the FTP server.

You’re not quite done yet though.

You also need to create a series of empty files, with a .fin extension, named exactly the same as your SAINT files.  These are “finish” files and are crucial to the upload.  They’re completely empty files – any text editor can create them.  Just make sure they are named exactly the same, case sensitive.

Upload those .fin files and you’re done.

Now, go have a coffee, have some lunch or dinner or whatever and come back later.

Progress

You can kind of check on progress by refreshing the FTP list of files.  Omniture removes the files from the FTP directory when it begins to process them, so you can kind of get an idea of where things are.

Time Frame

I uploaded the files around 4pm. 

At 10:30pm I did a data extract by FTP of all data to see where it was up to…it was done.  Shortly thereafter, I got an email saying it was done, without any failures.

Easy as pie.  No muss no fuss.

While we’re using customer segments, it could just have easily been customer demographics, technographics or any other form of data.  The point is, 1 million rows and it didn’t even blink.

There’s a few things to watch out for though when importing that much data.

There is a limit on the amount of unique values (500,000) that will be reported against in a given month.  We’re ok – we won’t see that limit.

Recommendations are that file sizes be kept under 30mb for the initial load, and then subsequent refreshes less than 5mb.  So we’re still ok.

And the import time will vary depending on many things, including how busy their import routines are.  You get in the queue and everyone loves a queue.

But that was it.  1 million rows of customer data now available for segmentation nirvana in SiteCatalyst – and DataWarehouse, and Discover, and Test and Target.  We’re off to the races!

And while the first run of this was a manual run, future updates can easily be automated now that the FTP site is created.  Just remember your .tab and .fin files must be named the same.