Search-Powered Web Apps

This was our third year having a booth at Blog World and it just keeps getting better for us. Instead of going on and on about what a great conference it was and how many times our company got mentioned in sessions or during keynotes, we thought pictures might do a better job of telling our story…

Our booth, along with the swag we were handing out, attracted quite the crowd. The  Purell was a big hit…either due to the fact that folks were shaking a lot of hands or that they agreed that Las Vegas was a dirty city.

But, one of the main reasons we go to Blog World is to be immersed in a world we love…that of blog publishers. Seeing their eyes light up when we explain and demo what we do is something that just can’t be described.

Not to mention all the opportunities we have for spreading our message to a larger audience…

However, probably our most favorite thing to do while at a show like Blog World is to thank our current users. Folks like the masterminds behind I Can Has Cheezburger, Neatorama, and WebUrbanist. And slapping a few stickers on cute mommybloggers doesn’t hurt the cause either. (Hi Extraordinary Mommy!)

A conference like Blog World takes a lot of energy. Not only is it a big show, but the fact that it takes place in Vegas makes it all the more tiring. To make sure we were up to the task, we made sure we had all of the essentials covered…

Thanks to everyone we met and to all those who told us how much they loved what we’re doing. Wear your Lijit shirts with pride and we hope to see you next year at Blog World 2010!

[Photo credits: Perry, Grace, Greg and http2007]

Every year, since 2004, Technorati has published the State of the Blogosphere report. The State of the Blogosphere report is considered the best up to date source of information about size, motivations, and practices of long tail publishers.

This year, Lijit helped Technorati by supplying some information from within the blogs that make up the Lijit Network. Lijit performed the analysis on the raw data and only supplied the aggregate insights documented in this post.

Analysis was centered on four distinct areas of interest including Search Engine Referrals, Blogroll Promotion, the Impact of Twitter, Advertising and Analytics.

Methodology

Data for this post was collected from two primary sources both directly collected by Lijit.

The first source of data was the ~11K active Lijit publishers that have the Lijit Search Widget installed on their publications. Lijit builds a unique search corpus for each publisher. This search corpus includes the publisher’s publication, his user-generated content, and the network of the publishers who influence the publisher (i.e., his Blogroll). This network of influencers results in a crawl footprint of over 2.5M publications that we actively index in order to maintain the search functionality on the 11K publisher sites. The second source of data used in this post comes from information gathered on those 2.5M sites in the extended network.

Data was reduced to something we refer to as the ‘typical publisher’. For some measurements, some publishers were omitted from the sample when in our opinion the specific publisher or publishers represented a singularity in the data that masked the typical publisher substantially. In addition, for some of the analysis points, we removed publications with less than 100 page views a day. Where lower page view publishers were removed we point it out. As page views drop into lower numbers some of the data begins to skew and it begins to get difficult to distinguish active and inactive publishers.

Search Engine Referrals

A typical site within the Lijit publisher network receives 27% of its page views from clicks on results in horizontal search engine result pages. As expected, the highest single source of referrals to the typical publisher site is Google at 23.5%. Yahoo and Bing were next, accounting for about 3.2% of referrals. Twitter and Facebook were nearly identical and total about 1.6% of traffic.

Lijit categorizes publications into 23 topical/vertical subject areas. The Tech vertical saw the highest percent of page views from search engine referrals at 41%. The remaining topical areas were fairly consistent with regards to percent referrals.

The percent of page views that come from search engine referrals is fairly constant with the audience size of the publication. The exception to this are publications of less than 100 page views a day that receive a slightly larger percent of page views from search engine referrals at around 30%.

It’s unclear why smaller publications get a larger percent of page views from search engine referrals, but may be linked to the ever growing length of horizontal search engine queries. According to a Hitwise January 2009 Search report, over 50% of queries are now 3 terms or more on the major horizontal search engines. This suggests that as the length of the average query string gets longer, more referrals get passed to smaller publications due to the specificity of the queries. This is a positive trend for smaller publishers.

Blogroll Promotion

Based on the 2.5M publications crawled by Lijit, the number of blogs in the average blogroll is 47, a surprisingly high number. Although not always a prominent feature on a publisher’s site, cross promotion of bloggers by other bloggers is clearly a significant factor in publication readership growth.

The typical publication within the Lijit network of 2.5M sites appears in 6.4 other Blogrolls. In other words, the typical blog is pointed to by 6.4 other blogs. The difference between a blog appearing in 6.4 other Blogrolls and pointing to an average of 47 other blogs is largely due to blogs pointing outside of the Lijit crawl footprint. The Blogosphere is a very large place.

The Impact of Twitter

Publications with greater than 100 page views a day received on average 0.83% of their page views from Twitter referrals. This percent tracked very closely to Facebook referrals at 0.80%. Publications below 100 page views a day saw a higher percent of page views from Twitter referrals than Facebook referrals.

Besides horizontal search engines, Twitter is the largest driver of referrals to the typical publication.

Lijit Search aggregates user-generated content that a publisher generates, into search results that display on the publisher’s site. Aggregating this content around a publisher’s site creates a stronger brand association for the reader with that publisher and site.

The most common user-generated content source included within a Lijit Search profile is Twitter. About 50% of Lijit publishers include Twitter in their Lijit Search results. This is a change from prior years. In 2007, 26.6% of publishers included Twitter as a content source in their Lijit Search results. In 2008, 42% of Lijit accounts included Twitter as a content source within their Lijit Search results. In 2009, 50% of publishers included Twitter as a content source within their Lijit Search results.

Twitter was by far the fastest growing content source to be included by Lijit publishers. Clearly, publishers embrace the micro-blog format. Going forward, Lijit intends to track the percent of publishers that use Twitter for blog post promotion as we suspect this number is quite high.

Advertising and Analytics

As Lijit crawls the extended network of publications, we track the widgets and tags we find on those publications. For the first time, Quantcast overtook Google Analytics as the most frequent analytics tag found on publications. This is likely due to Quantcast tags being included in some publishing platform templates.

Comparing 2008 to 2009, there has been a 68% increase in the number of sites with Ad tags installed. This indicates to us that monetizing sites is high on the priority list of most publishers.

Last year, when we ran the analysis, Google Ad tags made up 67% of the Ad tags found. This year that percentage has dropped to 47%, indicating publishers are experimenting with other Ad networks. This is probably not an indication of publishers leaving Google but rather publishers trying other Ad networks and using Google at the end of the Ad rotation.

More Data to Come…

With Lijit’s install footprint of 11K active installed base and a crawl footprint of 2.5M publications, Lijit is becoming the defacto source of information from within publications. Starting in 2010 Lijit will publish a more comprehensive study of what’s happening inside the Blogosphere.

Our fearless leader Todd Vernon, CEO of Lijit, was interviewed by Robert Scoble during a visit he made to our plush world wide headquarters here in Boulder Colorado. Have a view here or check it out on Building43.com!

We are super excited to announce a new update to our Wordpress Plugin. The same plugin that brought sweet search functionality and stats information to your wordpress blog is now even better.  You might not notice much, as the core of this update revolves around backend functionality. So what does that mean to you???

• Our search form override functionality is now compatible with just about any wordpress theme out there.  (if you find one that doesn’t work, let us know, and we’ll try to add it)
• The plugin enables new widget functionality which results in faster widget loading for your readers.

If you are using Wordpress 2.7 or higher, you should see auto upgrade notices for version 1.1 showing up in your admin console within the next 24 hours. You can just auto upgrade from your plugins screen. Or, you can always manually install/upgrade the plugin from the Lijit Wordpress Plugin page. For handy install tips and instructions, you can always reference our Lijit Help pages for guidance.

As always, if you have any suggestions or feedback, don’t hesitate to drop us a line at support@lijit.com.

Here at Lijit, we love finding out about new blog tools that make it easier to discover great content. (Hey, it’s a big part of what we do!) We first met the folks from outbrain at SXSW in 2008, when both of our companies were helping with the b5 Blog Network Camp. Since then, we’ve both grown up a bit. It’s not unusual now to hear both Lijit and outbrain discussed as must-have tools for any blogger.

In the interest of research, I wanted to give outbrain a try on my work blog, I quit for Lijit. It really is a simple one-click install and only took me a couple of minutes to get it up on my blog. I love that I can now get instant feedback and, underneath my posts, I’m recommending other quality content for my visitors to read. For a free service, outbrain rocks. I’m looking forward to checking out the reports that outbrain provides and to find out even more about the people coming to my blog. Who doesn’t love stats?

If you’re interested in an easy blog enhancement, be sure to give outbrain a try. Their widget inherits the look and feel of your blog, making for a seamless integration that can only help your readers. We really like what outbrain is doing and think you will too!

You can read more about outbrain on their blog and follow them on Twitter.

We’re honored to help spread the word about one of our favorite tech sites, Crenk. If it’s tech news you’re looking for, you’ll find it covered on this blog. Whether it’s product reviews, mobile apps, or something software-related, this blog has all the resources to guarantee that you’ll make an informed decision.

Steven, the founder of Crenk, was kind enough to write a post about using Lijit on his site, so it only makes sense that we return the favor. Plus, how can you not love a tech site that has a pink widget? I sent Steven some questions about his blogging experiences and I’m excited that he was game to answer them.

• How long have you been blogging and what made you start?

I have been blogging for a bit of a year now, since October 2007. I started blogging because I had a keen interest in startups and wanted to voice my opinions.

• What has been the greatest thing you’ve gotten from blogging?

Blogging has allowed me to actually talk about a topic I love and definitely helped me in learning how to express my thoughts correctly.

• Of all the posts you’ve written, what has been your favorite and why?

This is my favorite post because it actually has meaningful discussion in the comments.

• What are the top 3 tech tools that you love using and why?

Spotify (great for creating playlists and listening to in the office), Tweetdeck (it simplifies Twitter) and Filezilla (the best FTP program I have ever used).

• How has Lijit helped you as a blog publisher?

Lijit has simplified the search process for my blog. It allows me to easily add a great search tool and not have to worry about quality of search tool, plus allows me to focus on content.

In addition to the wonderful content he provides on his blog, Steven (using the Crenk name) is a poweruser on Twitter. You can follow him here and learn more about his blog here. Many thanks to Steven for helping us out with this interview and for using Lijit on his blog. We appreciate both!

This is another in our series of guest posts written by Lijit employees. In addition to the business development she does for Lijit, Grace likes making lists, laughing, snowboarding, and the lost art of writing letters.

November 15th will mark the two months that I have been Lijit. Now, part of being Lijit isn’t just about having the right ‘tude or rocking MC Hammer balloon pants (okay, bad joke but you get it all the time when you work for a company called Lijit). It’s much more. Let me give you the newbie’s look into what being Lijit is really all about:

–Intelligence: Our team is quick, witty, and good at what they do. We get it. We get our product, our users, and each other. I bet our combined IQ sky rockets our past our office window and over Mount Sanitas.

–Progressive: Speaking of our product, Lijit’s trust-based search is pushing boundaries and not only giving an empowering tool for online publishers, but for their readers. I am Lijit today because I was so impressed with the search and installed it first on my own blog before even realizing they were located in Boulder, the town I had just moved to. Doing business development, I work with hundreds of people in a week from all walks of life and through many mediums. Nine out of ten times, when I explain what we do at Lijit and that our service is free, I can literally hear their jaw drop (yes, over the phone, even on Twitter). Excellence=Free=Lijit=Unparalleled. How Lijit is that, to be able to work for something you believe in while it also is helping people?

–Laughter & Fun: I’m not just saying this to sound cliche, let me explain. There isn’t a day that goes by in our brightly lit, lime green hued office, where laughter isn’t heard. Whether it’s collective over Buck’s Russian accent, an inside joke within the Estrogen Lounge between the Product Evangelist/Biz Development ladies (me, Tara, and Jacqueline), a company outing at the racetrack watching our CEO spin out in his car and come in last, or it’s Friday late afternoon and a few people decide they need to unwind with Rock Band and Billy Joel begins to blare throughout the office. We stay serious and focused but never forget the importance of keeping the mood light, fun, and humorous. [On a side note, I truly think it's a hiring prerequisite that each person has an excellent sense of humor at Lijit.]

–Equality: My previous employment ranged from a non-profit to hierarchical public relations agencies and sometimes a pompous air seemed to creep in from the higher level down to the lowest. That’s not what being Lijit is about. Here everyone makes themselves available for questions, concerns, and each individual’s well-being. We make time for each other and it’s as though we really exercise the notion that each piece is relevant to the greater whole.

So you see, being Lijit requires a balance of laughter, highly-attuned intelligent minds, a progressive and unparalleled product, and a permeating mentality of equality. Sounds like a pretty good medley, eh? I think so.

Last week, the Defrag conference came to Denver. In its second year, Defrag is unlike the standard conferences Tara and I have attended on behalf of Lijit over the past year or so.

Defrag’s tag line “Accelerating the aha moment,” is rather appropriate. I liken it to a dinner where the participants are all highly educated and someone like Professor William Duggan, Columbia Business School and author of Strategic Intuition speaks on where the origins of innovation are in the brain.

Or, there is a rousing discussion around the aggregation and dissemination of content around the web with T.A. McCann, Gist; Tom Keller, IntenseDebate; Daniel Ha, Disqus; Avinoam Rubenstain, my6sense.

Imagine listening to Charlene Li discuss how to harness the implicit value of the social graph?

But, more importantly, because of the small size, interaction increases. I sat down with Howard Lindzon and was able to talk about Lijit and StockTwits at length, which just is so difficult to do at larger conferences.

It was great to have a conference of the quality of Defrag locally, and I am excited to see what Eric brings to Denver next year!

The Lijit team spent last weekend at the Blog World Expo in Vegas. In addition to speaking with bloggers, answering questions, and thanking our current users, we had the opportunity to debut our new trade show booth. While this may not sound exciting, it represented a big deal because it’s the first real booth-like thing that Lijit has ever had. We put a lot of heart into our trade show presence last year, but we put a lot of money into it this year and it made a difference.

Armed with the popular Lijit swag of small laptop stickers and uber comfortable t-shirts, we decided that we needed something more. We wanted to provide even more value and functionality to the bloggers attending the conference, so we decided to help people clean up their search. And their hands. (Don’t forget what a dirty city Vegas is…)

Besides spreading the joy of hand-sanitizing, we also helped to spread the word of our search. The beauty of Blog World is that everyone there is passionate about what they’re doing, so from wine bloggers to mommy bloggers, we had the chance to talk to people about what they love doing and to show them how Lijit can make them even better at it.

In addition to hearing feedback from blog publishers already using Lijit (big thank-you!), we also had the chance to convert some new publishers to our search goodness. Welcome…

Quickrelease.tv
Volleyball Voices
Vino Las Vegas
MoneyNing
Frugal Upstate
John Hawkins Unrated
Don Nozzle
The Wiz of Odds
Money Making Scoop

While we didn’t have much time to walk around the trade show floor, there were a few companies that caught our eye, including MindTouch, Zemanta, Woopra, JS-Kit, Photrade, ContentRobot, and BlogCatalog. This represents just a small sampling of the many Blog World exhibitors present and we only wish there had been more time to find out about all the companies in attendance.

We loved having the chance to co-sponsor some giveaways with b5media, because they are always such a fun group to be around. In addition to the poker chips they were handing out for the game in their booth, we got to see some of our favorite b5 bloggers, like Mary Jo…

Lijit was mentioned in a few other Blog World recaps. John Andrews, Senior Marketing Manager for Walmart, was impressed with us (as you can see in this video clip) and we sold Ben Koo on the up-and-coming Boulder tech scene. David called us some of his favorite naggers, which he meant as a compliment (I believe) and Carlton liked our Purell swag so much, he blogged about it. Additionally, Lijit captured Greta’s interest and managed to make an impression on Karen again this year.

The entire Lijit team worked hard while at Blog World and must thank all those who stopped by our booth to say hello! We appreciate your interest and support in making this year’s Blog World an even bigger success than last year. See you in Vegas next year!

[photo credit: carltonreid]

This post is by our very own Derek Greentree, who visited the Yahoo! campus for a field trip. In addition to using his business cards for the first time while on this trip, he was also very excited about being the recipient of developer swag. And he was nice enough to share it with the rest of us in the office.

Two weeks ago, I flew out to sunny California to attend Yahoo! Open Hack Day, an interesting developer event periodically held by Yahoo!. Attendees get access during the event to up-and-coming APIs (and other technology) in development at Yahoo!, and are given a challenge to create a mashup or other interesting hack and demo it at the end of the event. You can find information about the hacks that were demoed at the hack day blog.

If you’ve never been to the Yahoo! Sunnyvale campus, you probably won’t know quite what to think, as it’s filled with purple carpet, exclamation points on the walls, and emoticons everywhere. It’s interesting to see a very large corporation try to promote a youthful and exuberant appearance at a corporate headquarters with security guards, a cafeteria, fountains, volleyball courts, outside dining, and multiple buildings. I arrived Friday morning and was also surprised at how organized the event was; after entering, I was always greeted with friendly faces willing and able to help me find what I needed.

The theme of the weekend was APIs and openness. First, I’m happy to see that Yahoo! is getting behind OAuth, an open standard for API authentication. I wish everyone supported this simple mechanism (*cough* Facebook *cough*), as the various APIs offered by services out there desperately need to settle on a single, well-understood mechanism for allowing users to grant access to private data.

The second most exciting thing demoed (for me) that weekend was YQL, which will be a single URL that takes a SQL-like query and returns data from many Yahoo! services–like Flickr, mybloglog, Yahoo! Messenger, Yahoo! Mail, and others. Currently, each of these services has a different API and a different authentication mechanism, which means that supporting them is a pain. Having one mechanism for authenticating to the Yahoo! API (OAuth) and for querying any data within it will greatly simplify the code base we use here to interface with external services.

Next was Yahoo! BOSS, a search API. Many services (like Google and even Yahoo!, using yet another API) expose search services to external users, but BOSS is special. The problems with all the major existing APIs out there is draconian usage restrictions. Many of them, for example, don’t allow you to:

• Reorder search results
• Change the display of search results
• Use the API more than a certain (low) number of times per day
• Inject advertising of your own into search results

This is a mistake: allowing services to use your search data in unique and interesting ways is a win-win for the service and the search provider. With BOSS, on the other hand:

“BOSS (Build your Own Search Service) is different–it’s a truly open API with as few rules and limitations as possible. With BOSS, developers and startups now have the technology and infrastructure to build next generation search solutions that can compete head-to-head with the principals in the search industry. BOSS will grow and evolve with a focus on providing additional functionality, tools, and data for developers.”

I attended an excellent presentation by Vik Singh, a member of the BOSS team, on some example usage. He’s also developed a very cool Python library called the BOSS Mashup Framework that lets you whip up interesting mashups using Yahoo! Search with simple, elegant code. If you’re a developer and into this stuff, you definitely should check things out.

Perhaps most importantly, the weekend made me think about APIs that Lijit wants to offer (or could offer) to the outside world. We have access to a lot of interesting and unique data here, and I’ve seen more than a few startups that would benefit by having access to the work we’ve done. Look for more on this in the future, but I think providing access to Lijit’s data via a set of APIs could produce some very interesting mashups, and help Lijit grow in cool and unique ways.

[Photos found on Flickr and used via CC license: freshelectrons, Jinho.Jung, and bluesmoon]