The Growing Search Space


Couple of search related stories here, following my recent posting on whether or not search is broken.

Mike Arrington announces that Euekster have managed to grab $5.5 million in funding.

One of their products is a system for creating site specific search engines called Swickis, which also allows some user contributions in the form of voting for links.

Noel Hatch has set up a local government flavoured Swicki here.


Arrington also notes the launch of Wikiseek, which searches Wikipedia in a community-edited kind of way. It sounds pretty interesting, and I will be having a play. Their homepage looks familiar though…

Wikiseek LGSearch

Still, it’s interesting that these new approaches to search are starting to appear, and are working albeit on a fairly limited scale.

Is Search Broken?

Tom Foremski over at Silicon Vallery Watcher points out the things that annoy him about search:

– There are many publishers that try to make sure their headlines catch the attention of the search engines rather than catch the attention of readers. The same is true for content, editors increasingly optimize it for the search engines rather than the readers.

– Why should I have to tag my content, and tag it according to the specific formats that Technorati, and other search engines recommend?  Aren’t they supposed to do that?

– Google relies on a tremendous amount of user-helped search. Websites are encouraged to create site maps and leave the XML file on their server so that the GOOGbot can find its way around.

– The search engines ask web site owners to mask-off parts of their sites that are not relevant, such as the comment sections,  with no-follow and no-index tags.

– Web sites are encouraged to upload their content into the Googlebase database. Nice–it doesn’t even need to send out a robot to index the site.

– Every time I publish something, I send out notification “pings” to dozens of search engines and aggregators. Again, they don’t have to send out their robots to check if there is new content.

– Google asks users to create collections of sites within specific topics so that other users can use them to find specific types of information.

– The popularity of blogs is partly based on the fact that they find lots of relevant links around a particular subject. Blogs are clear examples of people-powered search services.

It’s my view that web search has come as far as it can based on algorithms and sheer grunt alone. There needs to be a human element in terms of whether or not a result is actually a) relevant and b) useful to the searcher.

This is the thinking behind the Search Wikia project which Wikipedia and Wikia’s Jimmy Wales is running. I wrote a little about this on my personal blog here and here.

It’s also why I am working on a human generated ‘search engine’. The aim will be for people to submit links they have found useful, tag and categorise them, and allow others to vote them as useful. This database of links will then be searchable, producing fewer results, but ones which have been recommended by others. I think it is going to be really useful, but it will need the committment of other people to make it work.

Watch this space.

Google Tips no more

Google have removed their controversial ‘tips’ feature, according to FireFox dude Blake Ross:

Google has removed the tips feature. Perhaps it was always intended as a test; I don’t know the official reason for its removal. In any case, thanks to Matt Cutts and other Googlers who listened and responded to feedback with an open mind, and over the holidays no less.

Nice to know they’re listening.

Technorati Tags: , ,

Search Wikia Update

One of the earliest posts on hyprtext was about the proposed new community search engine proposed by Jimmy Wales. There was quite a lot of kerfuffle about it, largely due to confusion over Amazon’s involvement, and quite a lot of debate concerning the name of the project, which everyone assumed was wikiasari. Most people agreed that this was a terrible name for a search engine.

Since then, I have had a chance to have a good poke about the site where the engine will be designed, The first thing the site does is put right a few misunderstandings:

Amazon has nothing to do with this project. They are a valued investor in Wikia, Inc., but people are really speculating beyond the facts. This search engine project has nothing to do with Amazon’s A9, etc…

This project has also nothing to do with the screenshot TechCrunch are running (which belongs to Wikisearch), and this search project has nothing to do with Wikipedia…

Wikiasari is not and will not be the name for the free search engine we’re developing. It was the name of a former project.

So, search.wikia is the name of the wiki where the project is discussed. Wikiasari is the name of a previous attempt to get a wiki based search engine going, and WikiSearch is a search engine that searches Wikipedia and all sites that are linked from it.

The new engine as yet has no name. But it’s being talked about.

I have to admit, I’m interested in this project, and I’d like to see it work. So, I have done as Jimbo implores us to do, and joined the mailing list. The first thing to pop into my mailbox was a missive from Jimbo, entitled First steps to getting it right…

…For now I just want to point out that the largest amount of skepticism about what we are going to try to accomplish here is driven by the inherent issue of spammers. There are huge incentives for people to try to abuse our good will and we have to anticipate and expect that. But, unlike many of the skeptics who think that this is impossible, I am very confident that if we can build a genuine community and give ourselves as a community the tools we need, then we can deal with this issue without a lot of trouble.

Tomorrow I will write more about how I see the core design working.

I’m looking forward to it.

Personalised Google by Default?

For some reason, whenever I have visited Google today, it has bounced me straight to – in other words, the Personalised Google page. This happens whether or not I am logged into my Google account.

Is Google starting to abandon it’s ‘classic’ homepage? Clicking the ‘Classic Home’ link now takes me to this URL.

Update: this explains it.

[tags]google, google personalised[/tags]

Google: Good or Evil?

There has been much discussion of late about Google’s antics in providing ‘tips’ at the top of searches. For example, try running a search on the word ‘blogging‘. What’s that, just under the sponsored links? A tip telling us to give Blogger a go! Not, or LiveJournal, or TypePad, but Blogger, Google’s own blogging platform. Hmmm.

There have been loads of posts about this issue, but more attention was gathered when Blake Ross, FireFox founder and all-round good guy, blogged about it:

Google is now displaying “tips” that point searchers to Google Calendar, Blogger and Picasa for any search phrase that includes “calendar” (e.g. Yahoo calendar), “blog” and “photo sharing,” respectively. This is clearly bad for competitors, and it’s also a bad sign for Google. But I generally support anything that benefits users, even if it’s controversial. I believe, for instance, that shipping Internet Explorer with Windows was a good move. So why are tips bad for users?

I care that Google is doing it because the company’s integrity over the years has impressed me and earned my loyalty.

Dave Winer, who was at the very least heavily involved in the development and propogation of RSS and related technologies, makes a further point:

I stopped believing in Google fairplay when they added a Blog-This feature to their toolbar, and didn’t use open APIs so users could post with any blogging tool, not just Google’s. To be clear, I wouldn’t have objected if they had set the default to work with their tool, as long as users could change a preference to use it with other tools.

Absolutely right – the ‘Blog-This’ function on the Google Toolbar was enough to make me remove it in annoyance. Why add a feature if it turns users away? A similar example is the fact that Picasa will only upload photos to Google’s piss-poor web albums, and not other sites like Flickr or PhotoBucket.

So why are people pissed off with Google? After all, they are a corporate company – and a damn big one. Surely they can do whatever they like in the interests of themselves? That’s what business is about, right? It’s like the decision to collude with the Chinese government in the censorship of search results. Whatever helps Google get ahead in its chosen markets is ok.

But as Ross points out, Google was meant to be different. That mantra of ‘Don’t be evil’ meant something to a lot of people and played a part in the huge and rapid growth of the company. Google were seen as the good guys, and continue to position themselves in that way in their competition with Microsoft. But the tides appear to be turning in popular opinion.

Matt Cutts – who plays a similar role for Google that Scoble did for Microsoft when he worked there – has responded to the barrage of criticism the search giant has received on this issue, and generally, he’s in agreement with the criticisms. He notes several searches where unwanted and irrelevant tips have appeared, and concludes that,

In each of the previous cases, I was not in the market for a blog or calendar or photo sharing service. Furthermore, the triggers appear to match on substrings: if I type in “blogoscoped”, I’m looking for Philipp, not to create a blog. The poor targeting alone is enough reason to turn off these tips (if I had my way).

He also agrees that it is reasonable for users to expect more of Google:

…it’s a fact that people expect more from Google than other companies. People compare other search engines to Google, but people compare Google to perfection. We have such passionate users that they’ll complain loudly if they think Google is ever straying from the right path. If you’re a Googler, it may feel frustrating. Instead, I’d choose to be grateful, because that passionate feedback keeps our heads on straight. When our users yell at Google, they care and want us to do the right thing (for their idea of what the right thing is). What other company gets that kind of feedback? Besides, if Yahoo or Microsoft jumped off a building, would you jump off too? 🙂 So yes, if the decision were up to me, I’d remove these tips or scale them way back by making sure that they are very relevant and targeted.

Let’s hope that whoever is in charge of the introduction of these tips into Google’s results soon sees the light and gets rid of them. They aren’t tips, they are advertisements.

Technorati Tags: , , ,

Wikiasari : WikiSearch?


Jimmy Wales, it would appear, is planning a search engine, along with his chums at Amazon, through his for-profit venture, Wikia.

Mr Wales has begun working on a search engine that exploits the same user-based technology as his open-access encyclopaedia, which was launched in 2003.

The project has been dubbed Wikiasari — a combination of wiki, the Hawaiian word for quick, and asari, which is Japanese for “rummaging search”.

Mr Wales told The Times that he was planning to develop a commercial version of the search engine through Wikia Inc, his for-profit company, with a provisional launch date in the first quarter of next year.

Earlier this year he secured multimillion-dollar funding from and a separate cash injection from a group of Silicon Valley financiers to finance projects at Wikia.


Niall Kennedy admits that he’s

skeptical. Sites such as Google, Yahoo!, and Windows Live already have the crowds clicking on search results every day, submitting bookmarks, and, in some cases, flagging spam. Wikia would need a critical mass of users to maintain a useful search index and query analyzer to supply Britney Spears’ fans, medical research, and the many many other search queries submitted every day. The same same search engine pickpockets wandering through Google’s search index will continue to target any significant source of traffic and unlike Wikipedia, you can’t just lock down a contested (or heavily profitable) area and still maintain balance.

Too true.

Ionut over at the Google Operating System blog takes up Wales’ claim that people will be more efficient at deciding whether a page is good or not than a computer:

I think the main job of a search engine is to understand how relevant a page is for a particular query. To scale, a search engine should that algorithmically. While people have a better ability to decide if a page is relevant, that doesn’t mean spammers won’t try to push their sites.

Wales and Wikia are taking on quite a bit at the moment – for example, the OpenServing site isn’t up and running yet (and I’m not yet entirely sure about what it’s meant to do) – and the introduction of a new search engine is a massive task.

And the name is really lame – ‘Google’ has become a verb. Can you imagine anyone ever saying “Can you Wikiasari this for me?” Answer: NO!

Update: In the comments, Jimbo points out that Amazon aren’t involved in the project. More here.

Update 2: Mike Arrington has a screenshot.
[tags]search, wikiasari, wikia, jimmy wales, niall kennedy[/tags]

AskX looks cool


Ask have launched a preview of their new search page design, nominally called AskX. Appearing in a three column layout, which is similar to their AskCity product, it now displays results from other information sources inthe right hand section, as well as the usual results in the middle. The right column is reserved for narrowing or expanding your search.

It’s quite cool. I did a test search for my hero, Brian Clough, and as well as the web results, it dragged up stuff from Ask News search, their blog search and Wikipedia.

The main problem is the speed at the moment, but I guess this is something they can work on…

[tags]ask, askx, search[/tags]


One of the frustrations of my job is when I am searching the web for information on what other councils are up to in my field. The trouble is that, by and large, I only really want to find results from the websites of local authorities in the UK.

So, using Google Coop, I’ve created my own. The main difficulty I had was tracking down the websites of every council in the country, but once that was done (and I have saved the list in about five different locations, just in case), it was all downhill.

I’ve called it LGSearch. Hopefully others will find it useful too.

[tags]local government, search, lgsearch, google coop, cse[/tags]