Given the huge amount of data now available online, I am having great difficulty persuading my journalism students of the value of looking elsewhere (for example a library). One way to do so I thought might be to show them how little of what has been written in the pre and early web era is currently available online. I don’t have a good source of data to hand about this so I just put together this graph pulling figures out of my head– can anyone volunteer a better source of data for this? Someone from Google Books perhaps? [Update - Jerome McDonough came up with a great response which I have pasted below this graph]

If the question is restated as what percentage of standard, published books, newspapers and journals are not available via open-access on the web, the answer is pretty straightforward: an extremely small percentage.  Some points you can provide your students:

* The Google Books Project has digitized about 20 million volumes (as of last March); they estimate the total number of books ever published at about 130 million, so obviously the largest comprehensive scanning operation for print has only handled about 15% of the world’s books by their own admission.

* The large majority of what Google has scanned is still in copyright, since the vast majority of books are still in copyright — the 20th century produced a huge amount of new published material.  An analysis of library holdings in WorldCat in 2008 showed that about 18% of library holdings were pre-1923 (and hence in the public domain).  Assuming similar proportions hold for Google, they can only make full view of texts available for around 3.6 million books.  That’s a healthy number of books, but obviously a small fraction of 130 million, and more importantly, you can’t look at most of the 20th century material, which is going to be the stuff of greatest interest to journalists.  You might look at the analysis of Google Books as a research collection by Ed Jones (http://www.academia.edu/196028/Google_Books_as_a_General_Research_Collection) for more discussion of this.  There’s also an interesting discussion of rights issues around the HathiTrust collection that John Price Wilkin did you might be interested in : http://www.clir.org/pubs/ruminations/01wilkin [I wonder what the situation is like for Amazon's quite extensive "Look inside the book" programme?]

As for newspapers, I think if you look at the Library of Congress’s information on the National Digital Newspaper Program at http://chroniclingamerica.loc.gov/about/ you’ll see a somewhat different problem. LC is very averse to anything that might smack of copyright violation, so the vast majority of its efforts are focused on digitization of older, out-of-copyright material.  A journalist trying to do an article on news-worthy events of 1905 in the United States is going to find a lot more online than someone trying to find information about 1993.

Now, the above having been said, a lot of material is available *commercially* that you can’t get through Google Books or library digitization programs trying to stay on the right side of fair use law in the U.S.  If you want to pay for access, you can get at more.  But even making that allowance, I suspect there is more that has never been put into digital format than there is available either for free or for pay on the web at this point.  But I have to admit, trying to get solid numbers on that is a pain.

[Thanks again to Jerome, and thanks to Lois Scheidt for passing my query on around her Library Science friends...]

When I joined New Scientist in 1995 as Net Editor (and ever since) I wondered why it largely covers the natural sciences not the social sciences. I assumed this was something to do with the ongoing intellectual and ideological struggle between ‘hard’ sciences and ’soft’ sciences and the related divide between qualitative and quantitative research. Imagine my surprise when thanks to the 3rd October podcast of Thinking Allowed, I discovered that the same people who launched New Scientist had launched New Society as well (50 years ago yesterday), explicitly as a social scientific publication.

I just remember New Society – it was merged with the New Statesman in 1988, a year after I arrived here in the UK (more detailed memories can be found on the podcast and in this recollection in the THES). Wouldn’t it be nice if on the anniversary of New Society’s birth New Scientist might be inspired to broaden its remit and introduce a New Society section? After all there’s no reason to keep off New Society’s patch now…

I’m writing a book chapter at the moment about the use of “user generated content” by journalists from the traditional media and to justify why I concentrate on the traditional media I thought I’d dig up a statistic or two about how dependent the public remains on traditional media for its news. I went looking for an update of Robert W. McChesney’s “The Titanic Sails On: Why the Internet won’t sink the media giants” written in 2000 and found his 2011 updated book The Death and Life of American Journalism. On page 17 I found this striking statement, “Harvard’s Alex S Jones estimates that 85% of all professionally reported news originates with daily newspapers and that he has seen credible sources place that figure closer to 95%”. Thinking this sounded like an interesting study I looked up the source and found Alex Jones’ book Losing the News: The Future of the News That Feeds Democracy. On page 4 he says, “my own estimate is that 85% of professionally reported accountability news comes from newspapers, but I have heard guesses from credible sources that go as high as 95%” (emphases mine). In other words either Jones has failed to cite his own research or (more probably) McChesney is reporting second hand and third hand guesswork.

This kind of thing really annoys me particularly when it takes me several minutes to get to the bottom of what turns out to be nothing more than a guess, and particularly when I know that there are a number of studies that discuss the sources of news with a greater deal of rigour. For example, there is How News Happens which argues that in Baltimore in 2009 95% of original news stories came from traditional news outlets, particularly newspapers (although its methodology has come under fire), or Paterson’s fascinating 2007 study showing that the leading online news sources (and to a lesser extent newspapers) are heavily dependent on news agency copy.

Of course it has a role for easy, quick communication of relatively unimportant information but I fear that its very availability and ease means that like some kind of online kudzu it is expanding and driving out longer-form online discourse – particularly blogs. This is particularly problematic for academics like myself. It used to be that I would string together 500 words and more about an academic subject or something in the news and post it on my academic blog (okay, I admit I was a grad student with a bit more time on my hands) but now I tend to just tweet or Facebook post about it because the blog form implicitly demands more engagement than I feel I can give. It seems to me that possibly for similar reasons gradually nearly all of the blogs I used to read by fellow academics giving me their insights into trends and papers have died away*, replaced by tweets simply directing me to relevant web addresses.

Don’t get me wrong–I love to read and pass on the kinds of references to papers and to newspaper articles I get–see my twitter feed– but by the time a tweeter tells you who sent them a web address, very briefly summarizes why you might want to click and perhaps provides a hash tag to indicate its subject all that remains to be said is that said document or image is “enjoyable”, “provocative”, scary etc. A blog posting by contrast does not have to be that much longer but allows the writer to provide at least a little more context for the resource that they are talking about or indeed to provide a small but nonetheless useful addition to scholarly knowledge without all the psychic and administrative burden of turning out an academic paper.

Moreover, I have recently realised thanks to the news about Datasift providing companies with access to archives of tweets back to 2010 that although Twitter has kept everything, if I as a user ever did want to find an insightful tweet from even a week ago unless I had favourited it or I had been using third-party programs to archive a particular user or hashtag I would be out of luck. I always supposed that the limitations of search in tools like TweetDeck or Twitter.com itself were just a coding problem not reflective of an underlying technical problem.

* Mind you, this rant which I have been saving for a while now was inspired in part by the excellence of a Nathan Jurgenson blog post which reminded me that academic blog excellence is not yet dead.

The cartoon below (sorry have lost the original source) presents a number of other good reasons I dislike Twitter…

I’ve just been listening to a segment on TV and TVOD on the BBC’s Media Show and it has reminded me just how far outside the mainstream my media consumption practices are. The average British household apparently ‘watches’ four hours of TV a day – a record high figure. This probably includes ambient sporadically viewed ‘TV on in the corner’ but still how on earth do they find the time? I probably watch an average of an hour of TV a week. X Factor has been an extraordinary success for ITV – I have never watched it (and probably haven’t watched ITV at all in a year). The channel I view programmes from most is probably (you guessed it) BBC4. Even with the proliferation of DVRs, TVoD etc, people still watch 88% of their television ‘live’. I watch or listen to almost nothing in that way any more. By far the bulk of my audiovisual media consumption comes in (audio) podcast form – about 1.5 hours a day – because I can do it while doing other things eg cycling to and from work.
It’s really odd to realise just how far outside of the media consumption mainstream I am (and it’s hard for me to imagine myself into the heads of more typical media consumers).

It’ll be interesting to see whether the great British public falls in love with this in the same way that Americans seem to have done with the HuffPo on its home turf. I suspect that since we already have a vibrant “opinion sphere” in our National press and (perhaps as a result?) the blogosphere here is rather less influential, it may struggle. I would have hoped that they could produce and highlight a few exciting exclusives for their first day but the page I saw this morning was reliant on the Press Association for several of the top stories, and aesthetically I found the layout much too garish and busy. That said, Tom Zeller’s feature piece on air quality in London was admirably thorough, the article about how you can print your own newspaper was interesting, and the story about the council who paid £100,000 to help schoolchildren get to McDonald’s was entertainingly quirky.

It’s early days–I look forward to seeing what the site comes up with and how its competitors react.

1) I started my new job as Senior Lecturer in the Division of Journalism and Communication at the University of Bedfordshire this week and have enjoyed meeting my new colleagues (and collecting my new Macbook Pro).
2) I just met my editor at Palgrave and agreed to write a book (my first full-length academic one) provisionally titled “Sharing Our Lives Online: Risks and Exposure in Social Media” – likely to be delivered in 2013. I plan to blog about it as I write using the “Sharing Our Lives Online” category, so keep an eye on that…
3) On my way back from that meeting I discovered that my wife has also just found a position for when her current one finishes, which given the turbulent situation in the NHS where she works is a big relief.

Of course I would be open to receiving further good news but these three bits of news are certainly enough to be starting with!

This podcast interview by Jesse Brown with the creator of Dinosaur Comics and this web interview about the brief but dazzling success of a short story collection, ‘Machine of Death are interesting at a number of levels.

Briefly, a group of well-known web comic creators got together and found contributors from among their readers for this short story collection that they would then illustrate. No mainstream publisher would touch it because it didn’t contain material from authors they recognised, so they thought they would self-publish it. And they organized the fan base they had gathered from their web comic activity to buy the book all at once in order to get media attention. It worked and the book hit number 1 for several hours on Amazon US (though as they said it only took “thousands” of sales to do this – it’s now at #1192). A few days later, they released the full text of the book free as a downloadable PDF.

This phenomenon has naturally excited a number of the proponents of “new authorship” models and it is indeed an impressive achievement, but I would add a few cautionary notes to this tale:

Ryan North says he is able to make a ‘comfortable living’ from t-shirt sales driven by his free online comic strip but wouldn’t say how much this amounted to (and his standards of ‘comfortable’ may have been formed by his recent status as an impecunious grad student).

It benefited from promotion by the fan bases of several well-known web comics authors, was promoted on a number of very prominent sites like boingboing, and falls into the sci-fi/fantasy genre. It may even be a great read (I don’t know yet but I have started downloading the podcast). Taken together this constitutes a nearly ‘perfect storm’ in favour of this book.

The broader question for the future of this model has to be how replicable it is. At the moment this is newsworthy – the economic significance of online-driven publication will be proven when tens of thousands instead of (I’m guessing) a few hundred authors can earn enough in this way to afford to bypass the conventional publishing system.

Of course none of this should take away from the fact that even if this is not the start of an economic revolution for new authors it may well be the start of a cultural revolution enabling many more people to become published authors (even if with a rather different notion of what being ‘published’ means). It is this as much as anything else I intend to explore in my upcoming research.

Storyful is a news agency based on an interesting idea that a lot of journalism scholars are talking up – journalists as curators, bringing together and highlighting the best news from social media. It is still in beta, so it’s perhaps premature to criticize the product but when I registered and went to take a look at the first story which interested me it had some flaws which indicate some of the potential problems with this kind of service.

Having recently visited Cambodia, the story on Cambodian child prostitution caught my eye. So what do I get? A prominent photo and trailer from a documentary on the subject which is (as far as I can tell) a product of the mainstream media. An introductory paragraph of information and claims, some of them quite controversial but without sourcing of any kind. A tweet from a Chicago-based comedian pointing to a related story – from the mainstream media. “Some informed opinion on the Cambodian sex industry” is two comments selected out of 84 youtube comments found on a two year old Al Jazeera news item. And lastly there are links to and excerpts from the Factbook on Global Sexual Exploitation and Human Rights Watch.

Leaving aside problems of design and implementation (which can be fixed) this suggests two linked problems. First, that because of digital divide and linguistic difficulties, it can be hard to find social media sources for news from outside the industrialized world and that as a result a lot of what one can find eventually links back to the work of (more or less) mainstream journalists rather than citizen journalists. Also see Gonzalez-Bailon (2009) on how the mainstream news organizations and those they link to get most online buzz and Paterson (2007) on how the online news environment is still dominated by output from two major news agencies.

This is not in any way to denigrate the work of those behind storyful and other projects – it’s just to point out that social media does not (yet?) provide would-be news providers easy-to-process rich seams of raw news material unless such material is on subjects that appeal to social media users (see Thelwall 2010) and in countries where social media use is widespread. What’s needed first is more citizen journalistic capacity building in developing countries by organizations like the World Service Trust, OneWorld and Global Voices and more and cheaper internet there (eg you can’t get decent citizen journalism out of the Central African Republic if broadband costs 40 times the average salary there).

PS UK readers may be interested that there is an (as far as I know unrelated) BBC Three programme about sex trafficking in Cambodia coming up next Thursday at 21:00.