Monday, June 4, 2007

Silobreaker "Truthiness"

At Silobreaker we talk a lot about “intelligent search” and providing more insightful results than traditional search engines. But what do we really mean by that and how does it translate into practical use?

Let me show you a simple example of how Silobreaker can be used for analytical purposes. This is not groundbreaking scientific research and may come with some obvious flaws, but it provides a quick glance of instant “truthiness” that otherwise may require days if not weeks of manual research (it took me about 20 minutes).

Let’s take look at what’s going in the upcoming presidential election in the US (covered quite extensively in Silobreaker here).

Q: Can Silobreaker analytically process the huge volumes of news around this topic to work out what issues are debated, which issues each candidate is associated with and what differences there are between the Democrats and the Republicans?

A: The simple answer is; yes it can!

By searching all news articles in Silobreaker for each of the candidates during the month of May and especially what keyphrases each candidate is associated with in such news flow, a picture starts to appear. I also searched on Al Gore and George W Bush to get similar perspectives for one Democrat and one Republican who are not in the race for the White House.

Real political commentators may interpret the news for the month of May very differently, but the Silobreaker analytics suggest that:
  • While Health Care is the most commonly associated keyphrase with the Democratic candidates, Abortion is the equivalent keyword associated with Republican Candidates (in fact it is the top keyword for every single Republican candidate).

  • Global Warming/Climate Change and references to War on TerrororWar in Iraq make the top 10 list for most Democratic candidates, but such keyphrases are noticeable only by their absence for most of the Republicans candidates.

  • Immigration is another keyword primarily associated with most Republicans, while Tax has similar associations in both camps.

  • While some observers seem to think that Al Gore may still announce his candidacy, the Silobreaker analysis suggest that he has some catching up to do in terms of policies – except for environmental issues, of course, which almost entirely populate his top 10 list.

  • The day-to-day issues for the current President, George W Bush, seems to have little in common with the policy-platform on which most of his fellow Republicans think they will replace him.

  • Judging by the percentage share each Democratic candidate received of the total news coverage for all Democratic candidates, John Edwards, Hillary Clinton, and Barack Obama are way ahead of the rest of their Democratic peers.

  • The equivalent percentages for the Republicans, i.e. each candidate’s share of all Republican candidates’ news coverage, suggest that John McCain, Rudy Giuliani and Mitt Romney have taken a lead in the Republican camp (in terms of news coverage – good and bad) .

In some more detail
Roughly 8,000 unique news articles were found in Silobreaker for the month of May in which one or several candidates were mentioned by name. All articles were in English from some 1,000 sources around the world. Blogs were not included in the analysis above.

The relationships or associations that Silobreaker extracts and visualizes are not pre-determined or manually configured in any way. Silobreaker deduces relationships between terms or entities statistically by computing several measurements, such as co-occurrences, text-proximity, spelling variations etc in the information flow. This enables Silobreaker to offer more insightful search results than just traditional lists of headlines (often too many) by showing people, companies, places, keyphrases, etc that are related or associated with the search results. This in turn enables the user to move away from linear analysis to multi-dimensional exploration and discovery.

Check it out at

Kristofer Mansson, CEO


Anonymous said...

There’s a macro issue of how to restore trust in American leadership. It is clear that the Bush administration is going to leave, at least in the short term, a negative legacy. All related to the War in Iraq. There are two themes that is worth to mention:

On the Republican side of the ledger the theme is support for the president and a robust endorsement of the global war on terror.

On the Democratic side you have strong opposition to the administration’s policy in Iraq and clearly stated calls for finding a new policy, which range from slow to rapid withdrawal from Iraq.

Anonymous said...

This is truly cool. It will surely change the way we search, and process search results. But when it comes to blogs- how are they weighted?

Chester said...

As far as Bush's legacy, it will be just fine. Dare I say, heroic?

The war in Iraq has been won and in light of the Mumbai terrorist attack the Republicans are correctly a war on global terrorism.

Too bad the Dems are fighting for the other side.