"Self-plagiarism is style"

"whatcha talking 'bout?" version 4!

16th December 2006

"whatcha talking 'bout?" version 4!

posted in misc |

Ok, this is pretty much the final definitive version of analysing the latest hot topics from library/librarian blogs…

hotstuff
http://161.112.232.18/hotstuff.php

The page is updated approximately every 15 minutes and uses the following methodology…

1) posts older than 48 hours are analysed and the frequency by which every unique word appears is calculated

2) posts from the last 48 hours are analysed in the same way and the word frequency is compared to the older posts

3) when a word has become noticeably more frequently used in the last 48 hours, it'll appear in the word cloud — the bigger the increase in frequency, the larger the size in the cloud

4) if a word appears in multiple new blog posts then the shading is darker

5) if a word only appears lots of times, but only in a small number of new posts, then the shading will be lighter

So, in the last 24 hours, the usage of "2007" has increased substantially.  In new posts the word has a frequency of 28%, but only 7% in older posts.

Several bloggers have picked up on the sale of ProQuest — if other bloggers talk about it today, then it will grow in size and be shaded darker.

The usage of the word "disallow" has also increased, but it only appears in a single blog post (by the Baby Boomer Librarian) and is therefore shaded lightly.

Unlike the previous versions, this one doesn't require a stop word list — words like "library" and "the" tend to have a high frequency of usage in both old and new posts, so the relative difference in usage is usually too small for the words to appear in the cloud.

The other cool thing is that this version uses the "network effect" — the more posts it has to work with, the better the cloud becomes as delivering the latest hot topics.  For example, Stephen Abram's RSS feed is currently delivering posts from the last 3 days and he usually ends them with "Stephen", so he's currently making a strong appearance in the cloud.  However, over time, the number of older archived posts with the word will increase which means he'll no longer (relatively speaking) be a hot topic in the cloud …although not in real life, of course!

This entry was posted on Saturday, December 16th, 2006 at 11:52 am and is filed under misc. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.

There are currently 14 responses to “"whatcha talking 'bout?" version 4!”

Why not let us know what you think by adding your own comment! Your opinion is as valid as anyone elses, so come on... let us know what you think.

  1. 1 On December 16th, 2006, Dave Pattern said:

    I've spent a couple of hours sifting though dozens of blogrolls and the code is now picking up feeds from around 450 library related blogs.

    Given the increase in blogs, I've changed the page so that it updates every 30 minutes.

    posted on December 16th, 2006 at 5:08 pm

  2. 2 On December 18th, 2006, Dave Pattern said:

    I spotted this morning that the aggregator was choking on some of the atom feeds, so I've fixed that. It does mean that there's been a huge influx of new posts and it'll take a few days for some of the more common words (like "it's" and "new") to drop in frequency.

    posted on December 18th, 2006 at 6:44 pm

  3. 3 On December 18th, 2006, Dave Pattern said:

    Whilst we wait for the archive of older posts to mature, I've tweaked the cloud to show words that rarely appear in the archived posts in green — these are potentially new buzzwords and new topics of conversation.

    Also, given the large number of feeds I'm pulling in, I've dropped to only updating the cloud every hour. This is partly because the physical server the code is running on is already overdue for retirement, but I'm in the middle of prep'ing a replacement.

    posted on December 18th, 2006 at 8:36 pm

  4. 4 On December 18th, 2006, Steve Campion said:

    What a great tool. It cuts right to the main trends and topics and picks up on the terminology, too. Thank you!

    posted on December 18th, 2006 at 10:10 pm

  5. 5 On December 18th, 2006, Dave Pattern said:

    Steve — many thanks! If I've coded it right, then it should get better at picking out the trends and topics over time.

    posted on December 18th, 2006 at 10:26 pm

  6. 6 On December 19th, 2006, Dave Pattern said:

    For reference, here's a list of the blogs which I'm currently fetching the feeds from…

    http://161.112.232.18/hotlinks.php

    posted on December 19th, 2006 at 8:29 pm

  7. 7 On February 7th, 2007, anders pearson said:

    You might want to think about applying a logarithmic scale to the cloud: http://thraxil.org/users/anders/posts/2005/12/13/scaling-tag-clouds/

    posted on February 7th, 2007 at 6:39 pm

  8. 8 On April 26th, 2007, Christine Peterson said:

    Great idea! I love the way it provides a quick overview of what people are discussing.

    One thing — you are listing the old address for my blog. Library Technology in Texas is now at http://libtechtexas.wordpress.com/

    posted on April 26th, 2007 at 4:37 am

  9. 9 On April 26th, 2007, Dave Pattern said:

    Hi Christine! Many thanks for that — I've updated the blog details, so it should pick up your new feed today :-)

    posted on April 26th, 2007 at 6:19 am

  10. 10 On July 28th, 2007, walt crawford said:

    A little late to notice it, but I just noticed: Walt at Random isn't in your master list. I may not be a librarian, but I'll assert that W.a.R. is part of "libraryland." Your call, of course.

    posted on July 28th, 2007 at 2:19 am

  11. 11 On August 29th, 2007, Librarians Matter » Blog Archive » Thali tags said:

    [...] some less exciting aspect of administering lint months ago. "Hey Corey, Dave Pattern has done this with a whole bunch of international blogs, wouldn't it be cool if we had something like this [...]

    posted on August 29th, 2007 at 1:15 am

  12. 12 On September 14th, 2007, NAG 2007 Conference workshop » "Self-plagiarism is style" said:

    [...] HotStuff - tag cloud of popular topics (more info) [...]

    posted on September 14th, 2007 at 2:45 pm

  13. 13 On July 27th, 2008, Hot Stuff is cooling down » "Self-plagiarism is style" said:

    [...] to an upcoming server upgrade, the Hot Stuff service will be winding down at the end of next week. If you're using the widget on your blog [...]

    posted on July 27th, 2008 at 12:18 pm

  14. 14 On August 7th, 2008, what do bibliobloggers talk about? » "Self-plagiarism is style" said:

    [...] one final "hurrah" from the Hot Stuff service, I thought it would be fun to put all of the data into Wordle. Every day, for the last 2 [...]

    posted on August 7th, 2008 at 12:02 pm

Leave a Reply