A week on Summon (revisited)

After yesterday’s blog post, I thought I’d have a go at narrowing down my definition of a “separate search”.
If a user enters some search terms, and then uses 2 facets to refine the search before clicking on a result, I was classing that as 3 separate searches — what niggled me overnight was that that approach might inflate the facet use statistics …after all, 30.6% of all searches used at least one facet felt a little high given that I’m forever hearing staff moan that students never use the facets, no?!
For today’s blog post, I’ve removed all searches that didn’t lead to a result click. (There’s a small caveat that my jQuery code currently doesn’t capture a result click for links to the OPAC where the user clicked on the availability message (highlighted in red below) — this is because my jQuery code that captures the result clicks runs once the page has loaded, but before the AJAX’d availability information has been retrieved. When I get some time, I’ll see if I can find a way around that.)

So, let’s see how much of a difference that makes to yesterday’s stats

  • 29.4% of searches used at least one facet to refine the results
  • 10.4% of searches were refined using the content type facet (e.g. newspaper articles, book reviews, books/ebooks, journal articles, etc)
  • 7.8% of searches were refined to just items with full text available online
  • 9.4% of searches were refined by publication date
  • 5.6% of searches were refined to just articles from scholarly publications (including peer-review)
  • 3.7% of searches were refined using the language facets
  • 2.5% of searches were refined using the subject term facets
  • 2.1% of searches used a Boolean operator, with nearly all of them being AND

So, that overall figure for the % of searches which used at least one facet hasn’t dropped by much from yesterday’s figure of 30.6%.
Anyone who follows me on Twitter will know that I like to cheekily mock the importance of Boolean and the data from the last 7 days reveals a few things:

  1. no-one who used a Boolean NOT in their search clicked on a result
  2. only 0.07% of searches (that’s just 7 searches in every 10,000!) used a Boolean OR, which is arguably the most useful operator to use
  3. unless you’re using a search that includes one of the other Boolean operators, the use of AND is pretty much redundant as it’s the default Boolean operator in a search (i.e. the search “dogs AND cats” is the same as “dogs cats”)… so why are we telling students to use it in Summon?

After poking a bit of fun at someone for entering a 356 word search query yesterday, I can reveal that the longest search in the last 7 days that resulted in a result click was 98 words (it was a paragraph copied and pasted from a journal article).
I guess the big question here is why the disconnect between the “students don’t use facets” mantra and the actual usage data?
Finally, I thought I’d figure out how many results are clicked on after a search…
summonclickspersearch

6 thoughts on “A week on Summon (revisited)”

  1. Great usage stats. Did you share your jquery code for tracking click usage? Would love to try it, managed to use the stuff you posted at http://www.daveyp.com/blog/archives/1739 to customize a lot of things. Just playing around now but trying to be careful not to over complicate the Summon UI too much.

  2. Hey Aaron!
    Matthew Reidsma has posted his code on gitbub and mine is very similar to that.
    I’m capturing the keywords and facets by using the following jQuery…

        var debugInfo = $('.summon-debug').html( );
        $.ajax({
    	url:"http://library3.hud.ac.uk/myreading/perl/user/summonLink.pl",
    	type:'POST',
    	dataType: "jsonp",
    	crossDomain: true,
    	data:"q="+encodeURIComponent(debugInfo)
        });
    

    The debugInfo is very JSON-ish, so not too difficult to parse, e.g. this is an example where the user searched for “research design” and selected: full text only, “management” as a subject term, scholarly only journal articles, and English language only…

    qparams: {
      "s.ho": "t",
      "s.role": "authenticated",
      "s.l": "en",
      "s.q": ""research design"",
      "s.pn": "1",
      "s.fvf": [
        "IsFullText,true,",
        "SubjectTerms,management,",
        "IsScholarly,true,",
        "Language,English,",
        "ContentType,Journal Article,"
      ]
    }
    
  3. Hi Dave!
    Thanks a lot for sharing, I’ll look at it. Between you and Matthew sharing, I am playing with so many things. 🙂

  4. This has really whet my appetite to get some more meaningful usage stats from Summon, but there are still a few issues I am struggling with.
    I have got Matthew’s code working, but being greedy I want more! I’m intrigued to know how you managed to figure out the number of clicks after a search. How did you manage to record new searches (as opposed to new pages or refinements being added)? Did you use jQuery to bind to the search button? Do you record the serverSession id to match up the following clicks to the original search? Or do you just match on the search query?
    There are so many bits of data I would love to add in: search interface used (we have a couple of custom interfaces here), number of results returned etc. so we can see which interface design returns the best results.
    I think I may have bitten off more than I can chew here! Very inspiring post though Dave!

Comments are closed.