open source

Update on the Knowledge Hub

Knowledge Hub

I spent an enjoyable afternoon at the advisory group for the Knowledge Hub (KHub) last Tuesday (sorry for the delay in writing this up…). Steve Dale chaired the day which featured a number of updates about the project, in terms of procurement and project management; technology platform and supplier; and communications and engagement.

Remember – the Knowledge Hub is the next generation of the Communities of Practice. Think of it as CoPs with an open API, plus some extra functionality.

The Knowledge Hub is going to be built by an outfit called PFIKS – who I must admit I had never heard of before. Their approach is heavily open source based and apparently they have about 80% of the Knowledge Hub requirements already working within their platform.

I’ve come away with a load of thoughts about this, most of which I have managed to summarise below.

1. Open platform

One of the strongest improvements that the Knowledge Hub will bring as compared to the current Communities of Practice platform is the fact that is will be open. This means that developers will be able to make use of APIs to use Knowledge Hub content and data to power other services and sites.

One compelling example is that of intranets – a suggestion was made that it would be possible to embed Knowledge Hub content in a council intranet – without the user knowing where the information came from originally. Later in this post I’ll talk about the engagement challenges on this project, but perhaps creative use of the API will enable some of these issues to be sidestepped.

Another aspect of this is the Knowledge Hub app store. I’m not quite sure whether this will be available within the first release, but it should come along pretty soon afterwards – it’s something Steve Dale seems pretty excited about. Developers will be able to create apps which make use of content and data stored within the Knowledge Hub to do cool stuff. I’m guessing it will be a two way thing, so content etc externally stored can be pulled into the Knowledge Hub and mashed up with other content.

It’s certainly something for Learning Pool, and I guess other suppliers to local gov, to consider – how can our tools and content interact with the Knowledge Hub?

2. Open source

The open source approach is to integrate various components into a stable, cohesive platform. This appears to be based on the Liferay publishing platform, with others bits added in to provide extra functionality – such as DimDim, for example, for online meetings and webinars; and Flowplayer for embedding video.

On the backend, the open source technology being used includes the Apache Solr search platform which is then extended with Nutch; and Carrot2, which clusters collections of documents – such as the results of a search query – into thematic categories. I think it is fair to say that the search bit of the KHub should be awesome.

What is also cool is that PFIKS publish their code to integrate all this stuff as open source as well – so not only are they using open source, they are also contributing back into the community. This is good.

Open Source, as I have written earlier, is not as simple a thing to understand as it might first appear. There are numerous complications around licensing and business models that have to be considered before a project commences. It certainly isn’t the case that by using open source tools that you can just rely on the community to do stuff for you for free – which seems a common misunderstanding.

Still, from the early exchanges, it appears that PFIKS get open source and are taking an active involvement in the developer communities that contribute to their platform. Hopefully the Knowledge Hub will end up as being a great example of collaboration between government, a supplier, and the open source community.

3. Data

One of the original purposes of the Knowledge Hub was that it would be a tool to help local authorities share their data. This was a couple of years ago, when Steve first started talking about the project, when data.gov.uk didn’t exist and the thought of publishing all purchases over £500 would have been anathema.

It would appear that the data side of things is taking a bit of a back seat at the moment, with the revamp of the communities taking centre stage. My understanding up until this point was that the Knowledge Hub would act as a repository for local government data to be stored and published. It would appear from some of the responses at the meeting that isn’t going to be the case now.

This is, in many ways, probably a good thing, as authorities like Lincoln, Warwickshire and Lichfield (amongst others) are proving that publishing data isn’t actually that hard.

However, all those authorities are those with really talented, and data-savvy people working on their web and ICT stuff. Are all councils that lucky? Perhaps not.

Hadley Beeman’s proposed project seems to be one that pretty much does what I thought the Knowledge Hub might do, and so again, maybe a good reason for the KHub not to do it.

When a question was asked about data hosting on KHub, the response was that it could be possible on a time-limited basis. In other words (I think), you could upload some data, mash it up with something else on the KHub, then pull it out again. Does that make sense? I thought it did, but now I have typed it up it seems kind of stupid. I must have got it wrong.

4. Engagement

You could count the number of people who actually came from real local authorities on one hand at the meeting, which for an advisory group is slightly worrying – not least because this was the big ‘reveal’ when we found out what the solution was going to be and who the supplier was. Actually – maybe that’s not of huge interest to the sector?

Anyway, it’s fair to say that there hasn’t been a huge level of interest from the user side of things throughout this project. Again, maybe that’s fair enough – perhaps in this age of austerity, folk at the coal face need to be concentrating on less abstract things. But now the KHub is becoming a reality I think it will become increasingly important to get people from the sector involved in what is going on to ensure it meets their needs and suits the way they work. By the sound of the work around the ‘knowledge ecology’ that Ingrid is working on, plenty of effort is going to be put in this direction.

It will also be vital for the Knowledge Hub to have some high quality content to attract people into the site when it first launches, to encourage engagement across the sector.

For all the talk of open APIs and the Knowledge Hub being a platform as much as a website, it still figures that for it to work, people need to actually take a look at it now and again. To drag eyeballs in, there needs to be some great content sat there waiting for people to find and be delighted by.

Much of this could be achieved by the transfer of the vast majority of the existing content on the Communities of Practice. There’s an absolute tonne of great content on there, and because of the way the CoPs are designed, quite a lot of it is locked away in communities that a lot of people don’t have access to. By transferring all the content across and making it more findable, the whole platform will be refreshed.

5. Fragmentation

The issue of fragmentation occurred to me as the day went on, and in many ways it touches on all of the points above. For while the Knowledge Hub both pulls in content from elsewhere and makes its own content available for other sites, there are still going to be outposts here and there which just don’t talk a language the KHub understands or indeed any language at all.

It’ll be great for dorks like me to automatically ping my stuff into Knowledge Hub, whether posts from this blog, or my Delicious bookmarks, shared Google Reader items, or videos I like. But those sites which publish stuff without

One striking example of this are the Knowledge Forums on the LG Improvement and Development website, which have continued despite the existence of the functionally richer Communities of Practice. My instinct would always to have been to close these forums and port them to the CoPs to both reduce the fragmentation of content and the confusion to potential users.

What about the content and resources on the rest of the LG Improvement and Development website – will that continue to exist outside of the rest of the platform, or will it be brought inside the KHub?

There are plenty of other examples of good content existing in formats which can’t easily be resused in the KHub, and for it to be the window on local government improvement, it’s going to need to drag this stuff in. Maybe a technology like ScraperWiki could help?

The State of Open Source

Stephen O’Grady has a great post analysing where the open source software movement is in the Startup, Growth, Maturity or Decline model.

Why would commercial organizations willingly cede the fruits of their labor to a market that might include their competitors? Because for software that is non-differentiating, that is not a competitive advantage – which for most non-technology firms is virtually all of their software – it will cost more over the longer term to author software privately than it would publicly. Facebook and Twitter demonstrate this quite adequately (coverage), true, but it’s not just the web firms. We see it when a hosting company (Rackspace) and space agency (NASA) jointly author a cloud computing stack that neither intends to create a software sales business around. We see it when Lockheed Martin launches an open source social networking project. And so on.

None of these can be characterized as decisions driven by idealism or emotion; they are simply the most logical means of developing software for companies that aren’t in the business of selling software. Make no mistake: we’re seeing a resurgence of roll your own software (coverage). The difference this time around is that by sharing the code developed internally as open source, it becomes possible to amortize the development costs across multiple organizations with similar needs. Worst case, you have the opportunity to lower your costs of talent acquisition; this, presumably, is one of the justifications for Google sharing details on its MapReduce and Pregel processing approaches.

Open source is something I’ve been reading and thinking about a lot recently and the more you dig into it, the more complicated it can get.

With government policy focusing more and more on using open source solutions, I wonder whether the understanding is there amongst those making buying decisions about the various licenses and business models that exist. Maybe it doesn’t matter.

It does strike me though that it is easy for a supplier to claim to provide an open source solution when in fact they don’t.

The advantages of open sourcing for both the supplier and customer, as Stephen notes in his post, are huge. But this really isn’t as simple as just saying that open source software is cheaper, or indeed ‘open’- there are lots of factors here and the implications of taking decisions around open source are potentially significant.

More on this in future posts.

Look and feel

If you visit this site in a browser, rather than just getting the content via your RSS reader, you’ll notice it looks a bit different. I have reverted the site to the new default theme for WordPress whilst I figure out how I want the site to look in the future.

Previously I was using the Thesis theme for this site. However, various issues, neatly encapsulated in this post, mean I don’t feel terribly comfortable with that choice any more.

Hopefully I will have something permanent sorted out soon enough.

Update: this is another good summary of the situation.

Adventures in open source land

UbuntuI had a load of fun yesterday being a total geek and installing Ubuntu on a netbook I’ve have for a little while and which doesn’t get used an awful lot. It’s a Samsung NC10, which, as I mentioned in this post, is a nice machine for social reporting due to its small size and light weight. Since I got the Macbook Air, though, I’ve tended to use that for general laptop use and for reporting at events – leaving the NC10 sat on the shelf.

I’ve wanted a Linux based machine for a few months just to play with, really. For the uninitiated, Linux is an open source operating system – in other words a replacement for Windows, or Mac OSX. It’s the bit of software that makes all the boring stuff work behind the scenes, and provides the launchpad for the applications on your computer to do their stuff, like surfing the web, or writing documents, or editing photos.

Now, Linux comes in many different flavours. Some you have to pay for, others you don’t. There’s Fedora, or Mandriva, or Suse, or Debian, or many, many others. I chose Ubuntu as it is one of the free (as in beer) ones, and because it seems to be one of the most accessible – ie it’s easy to install and easy to use. I do think that the plethora of choices is probably something that holds people back from trying Linux though. It’s a bit like trying to choose what to drink in a coffee shop!

Even better, there’s a sub-flavour of Ubuntu known as Netbook Remix, especially designed for use on small and slow laptops like the NC10. As you can see from the image above, open source doesn’t mean you lose out on eye candy – it’s a lovely looking system, with a netbook-friendly user interface that’s dead simple to use.

Installing it wasn’t too hard in the end, though I did run into problems. This is because the NC10 lacks a CD or DVD drive, meaning I had to install via a USB stick. I downloaded the Ubuntu software as an ISO file (which you would normally burn to a CD), then had to download another bit of software, recommended by Matt Jukes, called Unetbootin. This allowed me to ‘burn’ the ISO file to a USB stick. The next job was to tell the NC10 to boot from this USB stick – rather than the internal hard drive – when I restarted the machine. This proved tricky, and only worked when I completely removed the hard drive from the priority list of devices to boot from.

After I fixed that, though, installation was pain free, and the computer attached itself quite happily to my home wireless network – which was something I feared might go wrong. Other stuff like the built in webcam and microphone worked fine too, which was great.

Once Ubuntu was installed, it was a case of finding what extra software was needed to be added. Ubuntu comes with a great range of open source software out of the box, with everything most people would need, from Firefox for web browsing, Evolution as an email client, OpenOffice.org for productivity stuff etc etc. Indeed, the whole idea of netbooks is of course that you use web based tools as much as possible, so having lots of software installed on the system is kind of missing the point.

Point missing being a stock in trade of mine, I set about adding a bunch of tools to the computer. This can either be very simple, or a bit tricky. There are two ways you can do it simply: first by using the Ubuntu software centre to add open source software to the computer. This is great – you literally just search for what you want, and then in a couple of clicks, it is installed and ready to use. Some software isn’t available from the centre, but is still easy to install, usually just by downloading and running a package from the relevant website.

The tricky bit is when the software you want to install contains propriatory elements, and so doesn’t qualify to be a part of the Ubuntu software centre. I found this with Skype, and to install this, I had to get my hands dirty by using the command line – quite a strange experience in 2010 (I know there is a terminal available in Mac OSX, but I have never found the need to use it). However, one of the strengths of the open source community is the huge amount of documentation available, and Ubuntu is no exception. The support is generally excellent, and these beginners’ problems are covered in depth.

The extra software I have installed includes:

  • Google’s Chrome browser
  • Skype for voice-over-IP calls
  • Filezilla – FTP client
  • Dropbox for online file sharing across all my computers
  • Liferea – an RSS reader which can sync with Google Reader. This seemed to struggle with my subscription list though – perhaps due to a lack of processing grunt and memory on the NC10
  • Tweetdeck – which also needed Adobe Air installing first, which was another command line pain. Like Liferea, Tweetdeck ran quite slowly on the NC10, so I gave it up for a web based client
  • The GIMP for image editing
  • Quanta Plus for HTML and PHP editing

Overall, I’m pretty pleased with the results. This will have breathed a bit of new life into a machine I had little use for before, and it has been an interesting experience to find out how easy it is to use Linux based software. In many ways the operating system argument is irrelevant these days as more and more services are made available in the cloud. This is certainly the aim for Google, whose Chrome operating system will do little more than connect people to the web through a browser. But it is nice to know that you don’t need to have a high spec computer, or a load of expensive software, to have a mostly easy to use, and very nice to look at, computing experience.

Big thanks to Matt Jukes, Mark O’Neill, Harry Harrold, Tony Malloy, David Wenban, Adam McGreggor and others for their Twitter support throughout this process!

Load more posts!