The beauty of open data

Some news is tweetworthy and some is bloggable but the Live train map for the London Underground is definitely both!

Its a map that shows all trains on the London Underground network in approximately real time. How does it work

Live departure data is fetched from the TfL API, and then it does a bit of maths and magic. It’s surprisingly okay given this was done in only a few hours at Science Hackday and the many naming/location issues encountered, some unresolved. A small number of stations are misplaced or missing; occasional trains behave oddly; some H&C stations are missing in the TfL feed.

This is a beautiful example of the strength of open data. Its created by Matthew Somerville (with helpful hinderances from Frances Berriman and James Aylett). The Source code is available.

This is a static shot. But look at it live. Its a work of art… hypnotic!

Wikipedia and editorial control

Wikipedia continues to experiment with access and control in relation to the most frequently vandalized pages. Previously it would simply “lock” the pages, removing the ability of anyone to just edit the page. This loses the advantage of dynamic creation which made Wikipedia great and now changes are afoot to enable editing. Instead of locking the pages the organization will have a system of tighter editorial control. As Buzzblog points out there seems to be a paradox between freedom and control but the systems goal is to limit “bad” edits. In an interview Jimmy Wales explained the position “These (pages) have had to be semi-protected for years just because they are too tempting for naughty people to try something funny. But semi-protection has prevented thoughtful and sincere newcomers from making good changes.”

A blog post from Wikipedia’s Moka Pantages explains the changes:

Over the next few days, English language Wikipedia users may notice a small change on some articles: a little magnifying glass where a lock once was. The icon, on the upper right corner of the article, represents an important step that Wikipedia volunteers have taken to open up articles that were previously protected from editing. Starting Tuesday at 11pm UTC, the English Wikipedia community will begin a two-month trial of a new tool called “Pending Changes” (formerly known as Flagged Protection).

Articles that are frequently subjected to malicious edits have long been locked, sometimes for years, and protected from editing by new and anonymous users. Over the last year, the Wikimedia Foundation and volunteers from the community have been working to develop Pending Changes, a softer alternative to these editing restrictions. At present, only about 0.1 percent of the 3.3 million articles on the English Wikipedia are under edit protection. This tool should help reduce disruptive edits or errors to articles while maintaining open, collaborative editing from anyone who wants to contribute.

If it ain't online & open it may as well be dead

While Rupert Murdoch keeps threatening the world that he will move all his media behind paywalls it is time to recall the fascinating truth about information: If it ain’t online & open it may as well be dead. If it cannot be found via Google it may as well not exist. I know that this is shallow and that it fails to take into account quality print media but then again – it doesn’t matter how great you are if nobody has ever heard of you.

Encyclopaedia Brittanica had an excellent market lead, brilliant trademark and high quality product. After the web they began to die. After Wikipedia who cares?
One of the oldest online free journals that keep producing, providing & pushing excellent content is First Monday. Authors don’t have to pay & readers don’t have to pay. And yet, miraculously every month quality pours out. Here are my must read articles from the December issue (volume 14, number 12):

Political protest Italian-style: The blogosphere and mainstream media in the promotion and coverage of Beppe Grillo’s V-day
by Alberto Pepe and Corinna di Gennaro

The self-Googling phenomenon: Investigating the performance of personalized information resources by Thomas Nicolai, Lars Kirchhoff, Axel Bruns, Jason Wilson, and Barry Saunders

Public lives and private communities: The terms of service agreement and life in virtual worlds by Debora J. Halbert

Cat & Mouse of internet regulation

Regulating technology is (almost) hopeless. When giving a speech to the Cyberspace Law and Policy Centre Symposium on ‘Meeting Privacy Challenges’ in 2008 Senator John Faulkner  said

Trying to legislate to control technological development or the ways people use technology is not perhaps ordering the tide to not come in, but it is certainly like trying to empty a bathtub with a teaspoon.

And yet we keep digging away with the teaspoon. Take for example the latest developments on The Pirate Bay site (via Slashdot)

“The Pirate Bay has shut down their BitTorrent tracker. Instead TPB is now using Distributed Hash Table to distribute the torrents. The Pirate Bay Blog states that DHT along with PEX (Peer Exchange) Technology is just as effective if not better for finding peers than a centralized service. The Local reports that shutting down the tracker and implementing DHT & PEX could be due to the latest court rulings in Sweden against 2 of TPB’s owners, and may decide the outcome of the case.”

Check out warsystems for a better and more thoughtful analysis of tpb’s latest move.

And thats just it. No matter what the single state may attempt to do, technical individuals will find a way to evade the problem for a little while longer. It is doubtful whether this can go on forever, the individuals will still lose but the problems will remain and grow. At best any victory will be a Pyrrhic one.

Open Flight data

An interesting site collecting and displaying flight plans is up and runningm the site OpenFlights is a site for “flight logging, mapping, stats and sharing”. They have also decided to release their data under the Open Database License (ODbL):

One of OpenFlights‘ most popular features is our dynamic airport and airline route mapping, and today, we’re proud to release the underlying data in an easy-to-use form, up to date for October 2009. Behold 56749 routes between 3310 airports on 669 airlines spanning the globe.

The data can be downloaded from our Data page and is free to use under the Open Database License.

Top 10 Lies Newspaper Execs are Telling Themselves

SimsBlog has listed (and explained) the top 10 lies newspaper executives are telling themselves (and us). Here is the short version – but go to her blog and read the motivations:

Lie #1: We can manage this disruption from within an integrated organization.
Lie #2: Print advertising reps can sell online ads too
Lie #3:  Aggregators are killing my business
Lie #4: We can re-create scarcity by putting up pay walls
Lie #5: Our readers paid for news in the past, they will again
Lie #6: There will never be enough online revenue to support our newsroom
Lie #7: No one will ever cover crime/health/city hall the way we do
Lie #8: Our readers can’t be trusted/they are idiots/they are assholes
Lie #9 Democracy will collapse without us
Lie #10: I can compete with the best digital leaders/thinkers/creators in the world without becoming an active member of the online community.

All these are important and serious but in my book the most dangerous is the myth that democracy is dependent upon print newspapers (Lie #9). This will be one of the factors media will use to demand stronger protectionism through legislation.

An example of this idea recently appeared on the Becker Posner Blog:

Expanding copyright law to bar online access to copyrighted materials without the copyright holder’s consent, or to bar linking to or paraphrasing copyrighted materials without the copyright holder’s consent, might be necessary to keep free riding on content financed by online newspapers from so impairing the incentive to create costly news-gathering operations that news services like Reuters and the Associated Press would become the only professional, nongovernmental sources of news and opinion.

Disrespectful handling of digitalized cultural artifacts

On several occasions I have had the opportunity to discuss digitalization of traditional media. In particular to images that are no longer covered by copyright. Those who act as caretakers and gatekeepers for these cultural treasures have long been positive to digitalization – but were quick to discover that digitalization alone is not enough. The turning point of public opinion occurred when the Library of Congress began its pilot project with Flickr in the Flickr Commons. Read more about it on the Library of Congress blog or the report from the pilot.

Despite the anecdotal evidence, the gut feeling and the report some gatekeepers are still concerned about what will happen to “their” images if the plebeian mass can access them freely.

At first I thought their fears stemmed from a loss of income from selling prints, but this seems not to be their main concern even if some do refer to this. There main concern is the way in which the images will be treated.

They fear the disrespectful handling of digitalized cultural artifacts.

Now you may well ask yourselves how a digitalized artifact may be manhandled? Obviously it is not about destruction but there are concerns about use. The legal protection is long gone. The photographs are long since in the public domain and can be used and abused at will. This is of concern to the caretakers/gatekeepers since they have been entrusted with the images in physical form. In almost all cases they have received the photographs with a promise that they are preserving a part of cultural heritage. They believe that in their role as cultural preservers lies a duty to ensure that the photographers honor is not sullied by disagreeable online use.

And they know all too well that once digitalization and access has been granted there is no longer any control.

While I am a copyright minimalist and I think our protection terms are way too long I do feel there is a point here. How can museums and archives fulfill their duty to preserve what they have received in trust while maintaining their duty to provide access to culture?

Then I look at the work done by the Swedish National Heritage Board in relation to this question. They have put a small selection of their images on the Flickr Commons. A mere 274 photographs by Carl Curman (1833-1913).

The photographs have been accessed over 200  000 time since 17 March this year, that’s less than four months! Or 50 000 views per month (K-Blogg).

Besides pushing the almost unknown Carl Curman to a portion of internet fame the project at the Swedish National Heritage Board has brought back to life a set of dead photographs. Image how many times a photograph is seen in it’s lifetime. The average must be depressingly low. The most popular photograph in their project has been viewed 7805 times. Stop. Read the numbers and think. Seven thousand eight hundred and five times.

Stockholm by Carl Curman now seen by one more person: You…

Sure the photo will be ripped off. It will be posted on websites, stored on computers, used in presentations and the name of Carl Curman will be disassociated from the picture he took. Even more certain is that the Swedish Cultural Heritage Board will not be attributed enough for their thankless task of bringing this dead cultural artifact to life. But let us remember the old adage – no good dead goes unpunished.

The role of the caretaker/gatekeeper is, not a they once believed it to be, to prevent access. In the real world, grubby fingers and clumsy handling destroy the real artifact and lose it to the whole world. That is why we should be kept away from the real thing. But in the digital world the same is not true. What the flickr commons shows beyond a doubt is that while digitalization is good, it is nothing without access.

Ask Carl Curman.

Google books and Oscar I

King Oscar I of Sweden 1799-1859 was the son of one of Jean-Baptiste Bernadotte, one Napoleon’s marshals who became king Charles XIV John of Sweden. During Oscar’s time as crown prince and heir to the Swedish throne he was very socially active. Among other things he wrote a series of articles on popular education, and (in 1841) an anonymous work, “Om Straff och straffanstalter”, advocating prison reforms. The latter was translated in many languages and in English was given the title On Punishments and Prisons. More info on Oscar and photo on wikipedia.

This is not really common knowledge even in Sweden but was mentioned briefly in a documentary tonight and it sparked my curiousity. So I looked for the book, searching the online databases of second hand bookstores. No luck. Then, almost as a joke, I googled it. And there it was on google books. Cool but it was not like I was going to read it online. Then I saw the download button. Within minutes of hearing of the book for the first time I had a pdf of it on my computer – Google books is too cool!

The book seems quite interesting and I look forward to comparing it to Panopticon. Here is a quote:

It is undoubtedly, both the right and the duty of society, to punish every action which can disturb the public system of justice; it can even, if the offender has, by a relapse, shown himself incorrigible, or his offence is of a nature to endanger the public safety, render him incapable of again injuring the other members of the community. But does this right extend farther that to the loss of liberty, by which the object is gained? Every punishment, which goes beyond the limit of necessity, enters the jurisdiction of despotism and revenge.

Museums are doing it

Today I came across a notice that the Powerhouse Museum is adopting the attribution, non-commercial, no-derivatives Creative Commons license (for the material it owns)

This licence is used on some parts of our website. Examples are our own photography in the Photo of the Day blog and also for children’s activities on our Play at Powerhouse website. This licence means that you can republish this material for any non-commercial purpose as long as you give attribution back to the Powerhouse Museum as the creator and that you do not modify the work in any way. A more detailed explanation of this licence is available from Creative Commons.

And not long ago I found that the Brooklyn Museum was also using the same license.

This is in addition to the great collection of museums and institutions which have chosen to join the Flickr Commons.

The key goals of The Commons on Flickr are to firstly show you hidden treasures in the world’s public photography archives, and secondly to show how your input and knowledge can help make these collections even richer.

Among the 23 organisations in the Flickr Commons is the Swedish National Heritage Board which has begun putting photographs online. How about this photo from the small fishing town of Lysekil

Photograph: People in old Lysekil by Carl Curman (c:a 1870) uploaded RÄA

Why would sub-democratic leaders blog?

Listening to the radio this morning and heard that Karim Massimov, the Prime Minster of Kazakhstan started his private, yet official blog on 9th January and apparently has been so happy with the result that he has ordered his minsters to start personal blogs.

A politician starting a blog is hardly worth mentioning and starting in 2009 seems even to be a late starter but this one is a bit interesting.

According to the American State Department Country Report on Kazakhstan

The Government’s human rights record remained poor, and it continued to commit numerous abuses. The Government severely limited citizens’ right to change their government and democratic institutions remained weak. On some occasions, members of the security forces, including police, tortured, beat, and otherwise mistreated detainees; some officials were punished for these abuses. Prison conditions remained harsh; however, the Government took an active role in efforts to improve prison conditions and the treatment of prisoners. The Government continued to use arbitrary arrest and detention and to selectively prosecute political opponents; prolonged detention was a problem. Amendments to several laws governing the authority of procurators further eroded judicial independence. The Government infringed on citizens’ privacy rights.

Reporters sans frontières begin their 2008 report on Kazakhstan:

As well as the usual problems journalists get when they expose corruption or criticise President Nazarbayev, the media was the victim of power struggles inside the regime. Three opposition journalists died in suspicious circumstances and coverage of the August 2007 parliamentary elections was biased.

So the idea that the Prime Minister of Kazakhstan starting a blog and praising the way in which it allows citizens to communicate more directly with government is surprising to say the least. Either the whole thing is a propaganda attempt gone wrong or a total misunderstanding of the power of online communication.

Or maybe those in power just don’t get how bad they are?