Today, I’m guest-posting over at one of our publisher’s sites, Elementary Spirits, discussing this very topic.
If you’re unsure about how to interpret some of our stats, I’m breaking it down for you over there. Check it out and if you have questions, be sure to leave a comment on Barb’s blog!
Every year, since 2004, Technorati has published the State of the Blogosphere report. The State of the Blogosphere report is considered the best up to date source of information about size, motivations, and practices of long tail publishers.
This year, Lijit helped Technorati by supplying some information from within the blogs that make up the Lijit Network. Lijit performed the analysis on the raw data and only supplied the aggregate insights documented in this post.
Analysis was centered on four distinct areas of interest including Search Engine Referrals, Blogroll Promotion, the Impact of Twitter, Advertising and Analytics.
Methodology
Data for this post was collected from two primary sources both directly collected by Lijit.
The first source of data was the ~11K active Lijit publishers that have the Lijit Search Widget installed on their publications. Lijit builds a unique search corpus for each publisher. This search corpus includes the publisher’s publication, his user-generated content, and the network of the publishers who influence the publisher (i.e., his Blogroll). This network of influencers results in a crawl footprint of over 2.5M publications that we actively index in order to maintain the search functionality on the 11K publisher sites. The second source of data used in this post comes from information gathered on those 2.5M sites in the extended network.
Data was reduced to something we refer to as the ‘typical publisher’. For some measurements, some publishers were omitted from the sample when in our opinion the specific publisher or publishers represented a singularity in the data that masked the typical publisher substantially. In addition, for some of the analysis points, we removed publications with less than 100 page views a day. Where lower page view publishers were removed we point it out. As page views drop into lower numbers some of the data begins to skew and it begins to get difficult to distinguish active and inactive publishers.
Search Engine Referrals
A typical site within the Lijit publisher network receives 27% of its page views from clicks on results in horizontal search engine result pages. As expected, the highest single source of referrals to the typical publisher site is Google at 23.5%. Yahoo and Bing were next, accounting for about 3.2% of referrals. Twitter and Facebook were nearly identical and total about 1.6% of traffic.
| 23.52% | |
| Yahoo | 2.15% |
| Bing | 1.07% |
| 0.83% | |
| 0.80% | |
| MSN | 0.02% |
| Direct to Site | 21.50% |
| Site Self-References + Other Sites | 50.02% |
Lijit categorizes publications into 23 topical/vertical subject areas. The Tech vertical saw the highest percent of page views from search engine referrals at 41%. The remaining topical areas were fairly consistent with regards to percent referrals.
The percent of page views that come from search engine referrals is fairly constant with the audience size of the publication. The exception to this are publications of less than 100 page views a day that receive a slightly larger percent of page views from search engine referrals at around 30%.
It’s unclear why smaller publications get a larger percent of page views from search engine referrals, but may be linked to the ever growing length of horizontal search engine queries. According to a Hitwise January 2009 Search report, over 50% of queries are now 3 terms or more on the major horizontal search engines. This suggests that as the length of the average query string gets longer, more referrals get passed to smaller publications due to the specificity of the queries. This is a positive trend for smaller publishers.
Blogroll Promotion
Based on the 2.5M publications crawled by Lijit, the number of blogs in the average blogroll is 47, a surprisingly high number. Although not always a prominent feature on a publisher’s site, cross promotion of bloggers by other bloggers is clearly a significant factor in publication readership growth.
The typical publication within the Lijit network of 2.5M sites appears in 6.4 other Blogrolls. In other words, the typical blog is pointed to by 6.4 other blogs. The difference between a blog appearing in 6.4 other Blogrolls and pointing to an average of 47 other blogs is largely due to blogs pointing outside of the Lijit crawl footprint. The Blogosphere is a very large place.
The Impact of Twitter
Publications with greater than 100 page views a day received on average 0.83% of their page views from Twitter referrals. This percent tracked very closely to Facebook referrals at 0.80%. Publications below 100 page views a day saw a higher percent of page views from Twitter referrals than Facebook referrals.
Besides horizontal search engines, Twitter is the largest driver of referrals to the typical publication.
Lijit Search aggregates user-generated content that a publisher generates, into search results that display on the publisher’s site. Aggregating this content around a publisher’s site creates a stronger brand association for the reader with that publisher and site.
The most common user-generated content source included within a Lijit Search profile is Twitter. About 50% of Lijit publishers include Twitter in their Lijit Search results. This is a change from prior years. In 2007, 26.6% of publishers included Twitter as a content source in their Lijit Search results. In 2008, 42% of Lijit accounts included Twitter as a content source within their Lijit Search results. In 2009, 50% of publishers included Twitter as a content source within their Lijit Search results.
Twitter was by far the fastest growing content source to be included by Lijit publishers. Clearly, publishers embrace the micro-blog format. Going forward, Lijit intends to track the percent of publishers that use Twitter for blog post promotion as we suspect this number is quite high.
Advertising and Analytics
As Lijit crawls the extended network of publications, we track the widgets and tags we find on those publications. For the first time, Quantcast overtook Google Analytics as the most frequent analytics tag found on publications. This is likely due to Quantcast tags being included in some publishing platform templates.
Comparing 2008 to 2009, there has been a 68% increase in the number of sites with Ad tags installed. This indicates to us that monetizing sites is high on the priority list of most publishers.
Last year, when we ran the analysis, Google Ad tags made up 67% of the Ad tags found. This year that percentage has dropped to 47%, indicating publishers are experimenting with other Ad networks. This is probably not an indication of publishers leaving Google but rather publishers trying other Ad networks and using Google at the end of the Ad rotation.
More Data to Come…
With Lijit’s install footprint of 11K active installed base and a crawl footprint of 2.5M publications, Lijit is becoming the defacto source of information from within publications. Starting in 2010 Lijit will publish a more comprehensive study of what’s happening inside the Blogosphere.
I just checked my TripIt account and so far I have been traveling for 25% of 2009. That means, having just started June, that I have been gone from Boulder for more than a solid month. (Well, not a solid month, more like a total of a month worth of days, but I bet you get it.)
Which also means I get a lot of time to read. Lately, I have been reading a book called The Know-It-All by AJ Jacobs. In the book, which is non-fiction, AJ reads the entire encyclopedia to become the smartest person in his family.
In many ways, I view Google, Yahoo!, and MSN (I mean BING!) similarly. They are out there attempting to catalog the worlds information so that when asked a question, they can provide you with THE one right answer.
Often, they do. But as time as passed and the information store has grown, context has been lost. When you do the search for “seal” what is returned? The animal? The singer?
Google hedges its bets by displaying images of both, but guesses the singer in the results…
There was a recent article in DM News that search queries of three words or less were down 3%, 5% and 1% respectively, but that searches of four to eight words had grown from 3% to 20%.
In short, searchers have learned that to get relevant results, you have to use more than three words.
My friend Kevin Lee of Did-It was quoted as saying:
“Longer queries are a sign of the searchers becoming more educated and savvy and essentially being trained by the fact that results for shorter queries tend to return less relevant results than longer searches.” (emphasis mine)
Frankly, while not surprising, that boggles my mind. Searchers have learned that the major search engines are inherently inefficient and ineffective at applying context to a search query. How is this progress?
That major search engines are relying on the searcher to provide the context via additional keywords in the query. How does that create a better search experience?
Which, of course, brings us to Lijit.
Across our network approximately 75% of the searches are three words or less. Why is that?
Because we allow the publisher to provide the context. Their readers, by going to a trusted source first, already believe that the results that are returned will be contextual to the query. Therefore, they don’t need to use multiple word queries to return relevant results.
How do we know that this is the case?
1) On average, we get a higher click through rate on organic results than people using Google Custom Search.
2) On average, we get a higher click through rate on ads than the industry standard for text ads.
So what is better for publishers?
The “one right answer” approach that the major search engines take, or the “every publisher has a right answer” approach that we take?
I suppose that is up to the searcher.
But, I can guarantee it will take less keywords in the query to find the answer on a Lijit publisher.
Our (my) intention was to produce these reports and blog posts on a semi-frequent basis. It turns out that working in a startup means that your priorities shift quickly. This report sadly became a victim of these shifts, and hasn’t had the attention it deserves. So I am making a concerted effort to publish these more frequently than once every 9 months.
I took some time this go around to clean up our widget classification quality and relevance. As business models change, and web products morph, it can be difficult to accurately classify every service with a widget. If you have have any feedback regarding the way I classify a widget, let me know in the comments. I would also be interested to hear if there are specific widget verticals that we aren’t reporting on that you would like to see.
Below are the top 50 widget providers by domain, ordered by the percentage of blogs which contain at least one widget from the provider. Google is the obvious leader here. The top 20 hasn’t changed much in the last 9 months, which shows Google’s strength in the ad and analytics space.
Woohoo - Check out Lijit !!!! To be fair, we are number one in the graph now due to the reclassification of the snap.com widget. We previously considered them a search widget, but their primary tool does website previews and doesn’t relate to the search vertical in my opinion.
Our survey is primarily focused on widgets that are permanent fixtures on a blog, not those that are embedded in posts. Our crawler makes a note of widgets found in front-page posts, and these are primarily video widgets.Here is a graph of relative popularity of video widgets in blog posts. You’ll notice there are a lot more video providers in the list. It’s good to see some of the smaller guys grabbing some of the market share.
Our definition of “widget” is
any regularly-occurring functionality on a blog powered by an external service, voluntarily installed by the blog owner, and powered by Flash or Javascript.
Our crawl is “centered” on blogs with our Lijit widget. Our crawler then expands outwards by following blogrolls. This will give a bias to the overall results.
UPDATE:
Number of blogs examined: 184,431
Blogs with widget of any sort: 146,636
Total number of widget installations found:1,222,155
Survey period: 9/26/2007 - 11/06/2008
It has been a really long time since we last updated the widget statistics. Due to popular demand, they are back. Our hope is to be a bit more consistent with our widget stats posts. You may notice some changes since the last post we did on this topic, some subtle, some drastic.
Overall, we see the same general patterns we have seen in the past. The main differences are some of the widget adoption gains for certain services. I have attempted to highlight these below. One other theme appears to be the overall widget adoption numbers. Based on our analysis, these numbers have doubled in certain circumstances. This really highlights the growth in widget popularity amongst our sampled blogs.
Number of blogs examined: 47,378
Blogs with widget of any sort: 37,281
Total number of widget installations found: 1,200,928
Survey period: 9/26/2007 - 2/12/2008
Below are the top 50 widget providers, ordered by the percentage of blogs which contain at least one widget from the provider. Once again we see the classic power-curve (aka long tail) shape, with Google the clear leader. There are some movers though. Services such as del.icio.us, twitter, and the ever popular LIJIT , have increased in popularity. There were also significant gains from MyBlogLog and ning .
*Note that “tinyurl.com” is actually a cloaked reference to PayPerPost.com.
The items to pay attention to here are “social network” and “search” . Both of these categories have edged above “trackbacks” . The largest change evident here is the increase in the popularity of search. Search popularity has gone from 9% to 21% since the last time we reported.
Below are the popularity of widgets within the top verticals. Note that each pie graph represents the percentage widget distribution among all widget objects from the vertical. Contrast this with the numbers below each chart which show the percentage widget distribution among all blogs which contain a widget from the vertical. This distinction is important because many blogs will use use multiple widgets of the same type. The degree of overlap can be estimated by how far the percentage totals go over 100%. For example, in Analytics you can see that the percent total comes to 151%, indicating that 51% of Analytics widgets appear with a competitor’s.
Our survey is focused on widgets that are permanent fixtures on a blog, not those that are embedded in posts. However, our crawler makes a note of widgets found in front-page posts, and these are primarily video widgets.Here is a graph of the relative popularity of video widgets in blog posts.
Our definition of “widget” is any regularly-occurring functionality on a blog powered by an external service, voluntarily installed by the blog owner, and powered by Flash or Javascript.
Our crawl is “centered” on blogs with our Lijit widget. (Or as we call it, the Lijit Wijit). Our crawler then expands outwards by following blogrolls. This may give a bias to the overall results.
It has been a while since the last widget statistics. But we have good news! Derek and Daniel have now automated the process and we will be running results on a weekly basis. Over time, this will give us a clear picture of the ebb and flow of widget popularity. Let us know if there are other statistics you would like included in these weekly runs.
Number of blogs examined: 48,189
Blogs with widget of any sort: 31,953
Total number of widget installations found: 247,121
Survey period: 9/26/2007 - 10/16/2007
Below are the top 50 widget providers, ordered by the percentage of blogs which contain at least one widget from the provider. Once again we see the classic power-curve (aka long tail) shape, with Google the clear leader.
*Note that “tinyurl.com” is actually a cloaked reference to PayPerPost.com.
Below are the popularity of widgets within the top verticals. Note that each pie graph represents the percentage widget distribution among all widget objects from the vertical. Contrast this with the numbers below each chart which show the percentage widget distribution among all blogs which contain a widget from the vertical. This distinction is important because many blogs will use use multiple widgets of the same type. This is especially clear for advertising widgets where 92% of all blogs with ads use at least one AdWords widget, yet the AdWords widget accounts for only 83% of all advertising widgets found. The degree of overlap can be estimated by how far the percentage totals go over 100%. For example, in Analytics you can see that the percent total comes to 147%, indicating that 47% of Analytics widget appear with a competitor’s.
| Domain | % of Blogs | |
| 1 | google-analytics.com | 62.27% |
| 2 | sitemeter.com | 27.69% |
| 3 | statcounter.com | 19.70% |
| 4 | feedburner.com | 16.04% |
| 5 | quantserve.com | 9.30% |
| 6 | hittail.com | 2.06% |
| 7 | getclicky.com | 1.56% |
| 8 | reinvigorate.net | 1.44% |
| 9 | blogflux.com | 1.44% |
| 10 | performancing.com | 1.37% |
| 11 | bravenet.com | 1.32% |
| 12 | 103bees.com | 1.14% |
| 13 | histats.com | 1.02% |
| 14 | measuremap.com | 0.68% |
| 147.03% |
| Domain | % of Blogs | |
| 1 | googlesyndication.com | 91.50% |
| 2 | blogads.com | 9.25% |
| 3 | adbrite.com | 3.90% |
| 4 | doubleclick.net | 2.23% |
| 5 | fmpub.net | 1.81% |
| 6 | glam.com | 0.74% |
| 109.43 |
| Domain | % of Blogs | |
| 1 | snap.com | 0.51 |
| 2 | lijit.com | 0.28 |
| 3 | google.com/coop | 0.08 |
| 4 | google.com | 0.07 |
| 5 | sphere.com | 0.05 |
| 6 | eurekster.com | 0.04 |
| 7 | blogbar.org | 0.04 |
| 107.98% |
| Domain | % of Blogs | |
| 1 | technorati.com | 0.91 |
| 2 | haloscan.com | 0.13 |
| 3 | wholinked.com | 0.03 |
| 106.75% |
Our survey is focused on widgets that are permanent fixtures on a blog, not those that are embedded in posts. However, our crawler makes a note of widgets found in front-page posts, and these are primarily video widgets. Here is a graph of relative popularity of video widgets in blog posts.
Our definition of “widget” is any regularly-occurring functionality on a blog powered by an external service, voluntarily installed by the blog owner, and powered by Flash or Javascript.
Our crawl is “centered” on blogs with our Lijit widget. (Or as we call it, the Lijit Wijit). Our crawler then expands outwards by following blogrolls. This may give a bias to the overall results.
In April we published the first-ever widget statistics. Comscore recently followed suit and released their own set of widget metrix. There are large differences in what we are attempting to measure. They are looking at unique visitors, here at Lijit we are looking at publisher adoption. They are looking at social networks, we are looking at blogs. They consider only Flash widgets, we also look at JavaScript.
These statistics are based on a crawl of 29,139 blogs during the week of June 5th. Our definition of “widget” is any regularly-occurring functionality on a blog powered by an external service, voluntarily installed by the blog owner, and powered by Flash or Javascript.
As before, this crawl was “centered” on blogs with our Lijit widget. Or as we call it, the Wijit. Our crawler then expanded outwards by following blogrolls. This may give a bias to the overall results. However comparing these results to our smaller sample set in April, this effect seems to be minimal.
Our first graph shows the top 50 widgets, ordered by the percentage of blogs which contain at least one widget from the provider. Once again we see the classic power-curve (aka long tail) shape, with Google the clear leader.
The overall percentages are lower than in our last report. My best guess is that this is due to the larger scope of this crawl. As we venture futher from Lijit users, we encounter more un-widgeted blogs.
PayPerPost.com does not officially appear in this list. However, some detective work showed that they wrap all of their widget placements behind a Tinyurl address. Tinyurl.com doesn’t provide a widget of their own, but is merely a service redirect to other URLs. It is not clear why PayPerPost would pursue this strategy other than to obscure the face that a particular blog is a member of the PayPerPost service. PayPerPost tracking widgets appear on 1% of all blogs.
Many widgets provide similar functionality. The following graph shows the popularity of each type of widget.
As we saw last time, analytics widgets are the most popular with advertising in second place. Bloggers clearly want to know about their readers, even more than they want to monetize those readers! The “Social Network” type is dominated by MyBlogLog. Similarly, Twitter is the only entrant in the “Microblogging” category.
Within each vertical we can examine the breakdown between the competing widget providers. Note that each pie graph represents the percentage widget distribution among all widget objects from the vertical. Contrast this with the numbers below each chart which show the percentage widget distribution among all blogs which contain a widget from the vertical. This distinction is important because many blogs will use use multiple widgets of the same type. This is especially clear for advertising widgets where 90% of all blogs with ads use at least one AdWords widget, yet the AdWords widget accounts for only 75% of all advertising widgets found. The degree of overlap can be estimated by how far the percentage totals go over 100%. For example, we can see that 43% of analytics widgets do not appear alone.
| 1 | google-analytics.com | 49.48% |
| 2 | sitemeter.com | 39.03% |
| 3 | statcounter.com | 24.52% |
| 4 | feedburner.com | 12.78% |
| 5 | quantserve.com | 5.18% |
| 6 | performancing.com | 1.95% |
| 7 | hittail.com | 1.92% |
| 8 | bravenet.com | 1.67% |
| 9 | blogflux.com | 1.64% |
| 10 | reinvigorate.net | 1.24% |
| 11 | getclicky.com | 1.12% |
| 12 | 103bees.com | 1.03% |
| 13 | measuremap.com | 0.92% |
| 14 | histats.com | 0.82% |
| 143.30% |
| 1 | googlesyndication.com | 89.22% |
| 2 | blogads.com | 15.07% |
| 3 | adbrite.com | 4.34% |
| 4 | fmpub.net | 1.99% |
| 5 | doubleclick.net | 1.78% |
| 6 | chitika.net | 1.54% |
| 7 | glam.com | 0.99% |
| 8 | b5media.com | 0.19% |
| 115.12% |
| 1 | snap.com | 60.06% |
| 2 | lijit.com | 11.53% |
| 3 | google.com/coop | 10.26% |
| 4 | google.com | 8.39% |
| 5 | eurekster.com | 6.12% |
| 6 | sphere.com | 6.07% |
| 7 | blogbar.org | 5.31% |
| 107.74% |
| 1 | technorati.com | 86.00% |
| 2 | haloscan.com | 22.50% |
| 3 | wholinked.com | 2.33% |
| 110.83% |
Video widgets do not strictly fall into our definition because they are not permanent fixtures on a blog like, for example, a blogroll widget. However, we did capture statistics on the relative popularity of video widgets appearing on the front page of blogs we examined. YouTube is the clear winner here. This is interesting considering that YouTube did not show up at all in Comscore’s Widget Metrix.
| 1 | youtube.com | 92.99% |
| 2 | video.google.com | 8.41% |
| 3 | dailymotion.com | 5.24% |
| 4 | blip.tv | 2.22% |
| 108.86% |
As you can see here, Lijit uses only the latest in technology when crawling blogs and compiling widget statistics.
Stay tuned and subscribe for the next round of widget stats coming soon.
Note: The Weekly Lijit stats email falsely reported that Feedburner was the most popular widget. This was true in our first pass but after subsequent filtering Google emerged as the top widget. Sorry this stat did not get changed in the email. -Stan
Newsweek declared 2007 to be “The Year of the Widget” For all this widget talk there are precious few numbers. What widgets are popular? What is the distribution within widget verticals?
Our Lijit spiders look at a lot of blogs as they assemble all the content for our user’s searches. In our last release we added some code to take a look at what widgets are running around out there in the wild blogosphere, and now we’ve got some numbers. Due to some technical problems we had to throw out a lot of data but we’ve got ten thousand blogs in this crawl. That’s a still a small sample so these numbers should be taken with a heaping serving of salt. However, even through all that sodium we can see some clear trends.
Our first graph shows the top 50 widgets, ordered by the percentage of blogs which contain at least one widget from the provider. We see the classic power-curve (aka long tail) shape, with Google the clear leader.
Next, check out the widget popularity by the type of widget, the “vertical”. We see that bloggers clearly want to know about their readers, even more than they want to monetize those readers. They are also very interested in knowing what other people are linking to them. “Ecosystem” refers to widgets that identify a blog as being part of a group, for cross promotional purposes and such.
Within each vertical we can examine the breakdown between the competing widget providers. Note that each pie graph represents the percentage widget distribution among all widgets from the vertical. Contrast this with the numbers below each chart which show the percentage widget distribution among all blogs which contain a widget from the vertical. This distinction is important because many blogs will use use multiple widgets of the same type. This is especially clear for advertising widgets where 90% of all blogs with ads use AdWords, yet the AdWords widget accounts for only 75% of all advertising widgets. The degree of overlap can be estimated by how far the percentage totals go over 100%. For example, we can see that 47% of Analytics widgets do not appear alone.
| google-analytics.com | 3264 | 56.18% |
| sitemeter.com | 1822 | 31.36% |
| statcounter.com | 1317 | 22.67% |
| feedburner.com | 1003 | 17.26% |
| quantserve.com | 402 | 6.92% |
| hittail.com | 136 | 2.34% |
| performancing.com | 136 | 2.34% |
| 103bees.com | 107 | 1.84% |
| blogflux.com | 82 | 1.41% |
| measuremap.com | 81 | 1.39% |
| bravenet.com | 78 | 1.34% |
| getclicky.com | 68 | 1.17% |
| reinvigorate.net | 49 | 0.84% |
| histats.com | 44 | 0.76% |
| 8589 | 147.83% |
| googlesyndication.com | 2431 | 90.14% |
| blogads.com | 428 | 15.87% |
| adbrite.com | 142 | 5.27% |
| glam.com | 88 | 3.26% |
| fmpub.net | 61 | 2.26% |
| b5media.com | 54 | 2.00% |
| doubleclick.net | 47 | 1.74% |
| 3251 | 120.54% |
| technorati.com | 1690 | 92.00% |
| haloscan.com | 249 | 13.55% |
| wholinked.com | 44 | 2.40% |
| 1983 | 107.95% |
| snap.com | 528 | 48.26% |
| google.com | 423 | 38.67% |
| lijit.com | 119 | 10.88% |
| sphere.com | 67 | 6.12% |
| eurekster.com | 54 | 4.94% |
| 1191 | 108.87% |
*The numbers for Lijit skew high as our search used Lijit users’ blogs as seeds to begin the crawl (Before we had to throw out most of the blog data as invalid.).
These statistics are based on a crawl of 8552 blogs done over April 11-15, 2007. This crawl was “centered” on blogs with the Lijit widget (or as we call it, Wijit), and expanded outwards by following the blogrolls. Due to some bugs in how we stored this data, we had to throw out a large number of blogs. These stats only include numbers which we are sure to be correct. This is our first time, after all! For this sample we take a wide definition of widgets, including non-visible widgets such as as Google Analytics. Also note that we do not include image-only widgets, this includes the Feedburner subscriber count badge and the LinkedIn badge. Also not included are “widgets” that are automatically added by a blogging platform, like those from Blogger or Typepad. Because our crawl expanded outwards by blogrolls, and because of the general disconnect between the traditional blogosphere and social networks, widgets from sites like MySpace are not included in these stats.
In the coming weeks we will crawl again and again and continue to iterate these stats. Watch this blog to for further updates.
Privacy Policy | Terms of Use | Publisher Agreement | Legal Terms
Copyright © 2010 Lijit Networks Inc. All rights reserved.