Archive for the ‘industry’ Category

Transparency is More Than A Word


One of my favorite bloggers (who still *ahem* hasnt installed Lijit on his blog) is Loic Le Meur the founder of Seesmic. As Loic built Seesmic, a short form video startup, he recorded a video blog about the struggles and triumps of building a startup. Every day.

That’s transparency.

At Lijit, we embrace transparency. Like so many other companies, we understand that the increased access that users have to a company’s founders and employees (for example, you can follow the following members of Lijit on Twitter:

  • Todd, CEO
  • Tara, Community Catalyst
  • Micah, VP, Business Development
  • Leslie, Senior Director of Product and Operations
  • Daniel, Integration Engineer
  • Mike, Designer
  • and many others…including Lijit itself (yes the company itself tweets!)

But that is just one form of transparency. That is the transparency that speaks to what we are doing in building the product.

As publisher advocates, its important that we explain to publishers exactly what installing Lijit provides them in terms of functionality, and, more importantly, how we get that info, and what we do with the data.

We, of course, have a privacy policy and terms of use that outline specifically what we do with data.

As a Lijit publisher, you know that we provide a wonderful suite of stats around searches and searcher behavior. To provide those stats, we collect the following pieces of data around the search itself, the publisher, and the behavior associated with the query:


  • Wijit views
  • Wijit tag cloud clicks (side note: if you dont use the search cloud, you are missing out on 3-5x the total number of searches you could be getting. Im just saying…)
  • Wijit searches
  • Searches on a users profile page
  • Search paging
  • Search result clicks
  • Site/result clicked
  • Re-Search (side note: Yeah, me again. If you arent using this feature, you are almost suggesting to your readers to click the back button once they get to your site from a search engine. You like giving traffic back to the search engines do you? I didnt think so.)
  • Re-Search result clicks
  • Site visitor behavior across installed publishers
  • Search(terms, clicks) behavior across installed publishers


  • Account Demographic Info
  • “Blacklist terms” – publisher selected “negative” terms
  • Wijit data (is it installed? Its style, etc.)
  • GEO data (collected through a provider)
  • Trust and content relationships (content sources, blogroll, mybloglog, tags, etc.)

All of these data points are shown in our stats package, which a publisher can make public or keep private. For example, Brad Feld opens his stats to the public.

At Lijit, transparency is not a word we just throw around.

For us, our singlar belief in providing a service that helps publishers be better publishers means that there are no secrets. We gain nothing if we dont view our relationship to our publishers as a partnership. So, ask us, you might be surprised at the answer.

(As an example of this openness, I have started to leave my email address: micah [at] lijit [dot] com and my cell phone number (720) 231-7120 on FriendFeed and other places. Have a question? Call me. Drop me a line. I will always be open to helping and telling you how I will dominate the sushi eating contest.)

The Importance of Virtualization


I’m proud to share another in a series of guest posts written by Lijit employees. This week we present an installment from Mike, who seemed overly excited about writing and sharing this post.

Hi, I’m Mike Merideth, the Director of IT here at Lijit, and I’m going to talk a little bit about the nuts and bolts of how we do what we do. Over the past year I’ve had the opportunity to design and implement the production network and server infrastructure on which Lijit runs. It’s been a great year of challenges and breakthroughs, but if there’s one key architectural concept that has gotten Lijit to where it is today, it is virtualization. We use Xen for our virtualization technology, which has the advantage of being free Software (both in the “free beer” sense and the “free speech” sense). CentOS 5.1 (a Linux distribution which is based on the market leader RedHat) includes this functionality out of the box, and has performed very well for us.

So why does Lijit use virtualization? There are a number of good reasons:

Flexibility: When you’re launching a new web product, it can be hard to predict what pieces of the application will need more resources than you originally gave them, and which will need less. We’re able to change the amount of memory, the number of CPUs and the amount of disk space a server has quickly, easily and remotely.

Availability: Because we use an iSCSI SAN for most all of our storage, we can move virtual servers between pieces of physical hardware. So if we lose one of our physical servers, we can quickly bring up the virtual servers it hosted somewhere else.

Resource utilization: CPUs today are incredibly fast and powerful; far more so than most applications need. Similarly, RAM has become cheap enough that a server with 16 or even 32 gigabytes of RAM is not particularly unusual, or particularly expensive. Running a simple web server on such a system would be a waste of CPU and memory, and therefore a waste of electricity. If you can run several virtual servers on such a system, however, you can get the maximum return on your investment by making sure you’re fully utilizing all of the CPUs and all of the RAM. Which is all tied to…

Cost savings: Colocation is expensive, and electricity certainly isn’t getting any cheaper. Using virtualization means we can get the absolute greatest value out of the rack space and electricity we’re paying for.

As of right now, we’re running about 200 virtual servers on about 25 physical servers. Just a few years ago we would have needed scores of physical servers consuming thousands and thousands of watts of power to do the work we’re able to do in this relatively modest environment. For a startup that would mean a higher burn rate with a shorter runway, and greater stock dilution for the founding stakeholders because of the amount of capital needed to get the work done. If you’re trying to get a tech startup off the ground, you owe it to yourself to see if you can leverage virtualization in your IT architecture. You’d really be crazy not too.

If you managed to read this post without your eyes glazing over, you may be interested in my new Linux infrastructure blog at I’m keeping notes on my experiences there, in the hopes that what I’ve learned over the past year can benefit others who find themselves in the same boat.

Photo credit: Leonard John Matthews

Lijit will increase your page views


At Lijit we collect statistics on thousands of individual blogs, blog networks, and conventional online publications. For some time we have noticed that blogs that use the Lijit Search Wijit, specifically those that use the Re-Search feature, enjoy a page view lift as a result. Recently, we started digging into some of the usage statistics from our database and found some interesting trends.

In an earlier post, I spoke about the concept of “second click“. When a user enters search terms into Google or Yahoo and selects “search”, that is the first click. The large horizontal search engines own the first click.

Google Search and Results

When the reader selects a result from the list of items that Google or Yahoo has returned, that constitutes the second click. It’s the second click that leads readers to a publisher’s site. The average blog within in our network sees approximately 25% of its page views coming as a result of a referring horizontal search engine (or second click). The big search engines are a great source of new readers and page views if you can successfully hold on to that reader. The Lijit Search widget takes advantage of this situation to promote publishers’ content to readers when a blog page displays as a result of a referring search.

Todd’s Re-Search

In our study, on a daily basis, the Lijit Search Wijit promoted a publisher’s content an average of 874 times and the content items in the re-search box were clicked on average 4% of the time. This converted to real-world page view increases, which ranged between 0% and 10% with an average of a little over a 1.5% increase.

Page View Graph

These are all meaningful numbers in the world of publishers and subscribers. More page views results in more ad revenue and more engaged readers.

Some other interesting observations from the study… Publishers with the Lijit Search Widget above-the-fold (i.e. higher on the site) showed markedly higher second click conversion than those publishers with the Wijit installed below the fold. Publishers with information-centric sites optimized for SEO showed much greater second click impressions. Those publications that were more focused on Q&A or providing concrete information showed significantly higher second click conversions. Interestingly, higher page view publications were more likely to have a larger percent of users arrive from search engines, thereby setting up the second click scenario. Presumably, this is due to the higher page rank of these sites and the positive feedback loop that occurs as a result of this – basically, the big get bigger.

In the future I intend to dig into some of the interesting dynamics of blog networks. Blog Networks have specialty dynamics that make some of these results even more interesting and more pronounced. Stay tuned.

For now, make sure you have Re-Search turned on and your Lijit Search Wijit displayed above the fold to maximize your page views!

Socially Influenced Search


Venture Beat published an interview with Google’s Marissa Mayer that ran today written by Doug Sherrets. Andy@Lijit was nice enough to forward it on to me. I’m always interested in hearing what others think about socially influenced search (careful choice of words). Having lived in this world I know some of the more esoteric opportunities and problems of the space.

First, I have to clear the decks with something right out of the gate. The term “Social” has to be the most abused tech buzz word of the last couple years.

Todd’s list of what social isn’t…

  1. A person, doing something.
  2. A group of people, who don’t know each other, doing something.
  3. A group of people, who don’t know each other but behave the same, doing something.
  4. A group of people, who do know each other, but don’t interact, doing something.
  5. A group of people, paid by someone, doing something. (test: What search startup is that?)

Quotes from the article with Marissa about Social initiatives at Google:

“One thing we tried…is labeling – have users annotate the search results they see and have those annotations be shared with people on their social network or with people of like mind and interest

“Another classic thing to try is “other users like you”, where you build implicit social connections between users who are like each other

“Other users that did that search, also searched for”

“You could take annotations that people have entered in something like Google Coop and broadcast the annotations.”

You see what is happening here don’t you? What derails ideas about social search are very simple. The Internet world grew up with a one box, take a pill, mentality. If we can’t search the entire world’s data in one box and have that box know what we mean, it’s a non-starter because of what we have come to expect.

Yet, social behavior is just the opposite. Social behavior is about the people we “know”. It’s important to “know” people in order to validate the result set they help deliver to you. We don’t “know” everyone and that contradicts the “one box” expectation of social based search. It also contradicts the “best answer” expectation. No matter how grand the plan is for social search you need to “cross the chasm” of who you and your network know, and who you and your network does not know.

For social search to work in a global, one box world, it cannot be completely a social model. What does work is using a social network of people you “know and interact with” to find a “local expert” within a known network. That “local expert” will likely have pointers to high quality data produced by “global experts”.

The Social Chasm

The system is not perfect. But it does work quite nicely in practice within the world of general knowledge and day-to-day problem solving, around YOU. Where the system is does not work as well are in situations where the local network, as whole, has little basic knowledge of the item or concept being searched. Interestingly, these are the situations where more traditional information stores like Wikipedia work exceedingly well. This is why I feel quite satisfied when I look up Nuclear Reactor in Wikipedia, but a person in the Nuclear Power space would look to his network to find more detailed information.

The graphic above is really what we want to achieve at Lijit. Though our Search Wijit we facilitate and explore the connections, discussions, and searches of connected “real” social networks. These people interact, get to know each other, have a proxy (the blog) for the metadata that exists in real world physical relationships. In other words they do more than simply throw food at each other. When the network – of networks – hits a critical mass we should have global pointers needed to cross the social chasm – in one box.

Scoble on Social Graph based Search


Scoble has a great couple of podcasts today on Social Graph based search. He talks about how new social based search plays are resistant to SEO (Search Engine Optimization) and are eventually going to kick Googles ass.

This is exactly the basis of what Lijit was founded on. The idea of social trust being the major driver in judging the value of content is spot on.

Scoble talks about how Mahalo is a social graph based search because they have content editors that are part of their social graph. I think the example is a little broken in the Mahalo case in that the graph is not really naturally occurring. In other words the graph as selected based on social popularity, who wants the job, etc. perhaps not actual expertise in a specific domain. He also suggests Facebook is in a great place to capitalize on their large social graph to revolutionize search. I agree, however Facebook is still a walled garden of content. If Facebook would extend their social reach (and content reach) beyond their walls they would be in a great position.

Lijit is using the naturally occurring networks that already exist out on the web to create powerful search from the ground up. We don’t hire content editors and we don’t force editors (publishers) into any specific social network. Lijit uses the networks you have to build the world’s most powerful foundation for search.

Great post Robert, we agree.