I tend to assume that for all its flaws The Economist gets its facts right – at least on technical issues. But this article on How Google Works in their technology section recently repeats a popular misconception about search. The article says, ‘Google is thought to have several complete copies of the web distributed across servers in California and Virginia’ – whatever they do have it is nothing close to a complete copy of the web. Even if they had a complete index of the text of the first 100Kb of each page on the publicly spidered web (the most they would even claim) this would still miss the huge volume of available information that is stored in web-accessible databases (like the “British Telecom phone book”:http://www2.bt.com/edq_busnamesearch).

I believe that a search engine that managed to do a good job of searching this ‘invisible web’ alongside the ‘surface web’ would have a good shot at the number one spot.

P.S. While on the subject of search, here’s a tip – to get a (small) discount on your next Amazon purchase, check out their new A9 search engine.

Plunder!

Originally uploaded by derb.

A once in a lifetime opportunity came up for me – well-known academic publishers Routledge moved from central London yesterday and rather than pack up all their books they selected some and left the rest for hungry scholars to grab (charities didn’t want most of them for some reason). This was my haul. But they aren’t exactly free – given a bookshelf six shelves high I figure the space they take up in our flat would still be worth about 55 pounds given the cost of London real estate these days.

Still I’m not nearly as much of a book hoarder as some friends of mine – and with easy access to ‘one of the largest libraries in the world devoted to the economic and social sciences‘ I don’t really need to be.

Hmm… I seem to be turning into a book stack photoblogger – something of a dull niche! I promise if I put up more pictures they will be a little more interesting. Meanwhile take a look the few pictures I “have made public so far”:http://www.flickr.com/photos/derb/…

P.S. On the whole ‘buy vs sign out from library’ issue, I just came across a terrific little (free) tool described and linked to on the “43 folders”:http://merlin.blogs.com/43folders/2004/09/request_a_libra.html weblog. It lets you look up a book on Amazon then check to see if it is available at your local librar(ies) before buying. Mind you if your library has the book but with a different ISBN it won’t turn up. Definitely worth trying though – particularly if you prefer Amazon’s search to your library’s search.

WebSM (where SM stands for survey methods), ‘is dedicated to the methodological issues of Web surveys, but it also covers the broader area of interaction between modern technologies and survey data collection.’ It chiefly provides a collection of bibliographic references and some full text. Though the site itself is academic a fair amount of the papers are produced by and/or aimed at marketers. It has an index of survey software suppliers but this isn’t very handy as it doesn’t seem to include free software and is only organized by country.

Free commercial hosts for online surveys include my3q (don’t be put off by the Korean – it offers up to 5 questionnaires without question number or respondent number limits!), SurveyMonkey (their free service handles 10 questions and 100 responses), Zoomerang (free up to 30 questions, 100 responses but results stored for limited period). QuestionPro has a particularly good student offer – you can conduct one survey free of charge with unlimited questions and up to 5000 responses as long as you cite them publicly and link to their site from your project.

Castle is a suite of quiz software (adaptable presumably to other survey use) which is created for UK higher academics to use (free of charge) but appears to generate CGI scripts which must then be uploaded to your own server. GetFAST is similarly designed to help teachers get assessments from their students and allows for up to 20 questions but could also be adapted for more broad use I imagine.

I recall learning about a service run by a US university somewhere that was also free for academic use but I can’t remember where it is.

Later… While my3q is tempting I just realised that it doesn’t appear to let you download the results- you have to rely on their web stats which limits its usefulness. Advanced Survey at $25 a month (approx) looks pretty good but it isn’t clear if they support branching – for example, if my respondents answer yes to question 1 then don’t show them questions 2-4.

Zoomerang has a discount for educators for its full version I see ($99 for 3 months) and appears to do branching and allow downloading. QuestionPro allows branching and downloading of data but the non-academic free trial option only captures 25 respondents over a single month.

David Wilcox “blogs here in detail”:http://partnerships.typepad.com/civic/2004/01/nonprofit_tech_.html about a research report from Jeremy Wyatt at a regeneration consultancy, “Hall Aitken”:http://www.hallaitken.co.uk/. It suggests UK Online centres should be less in libraries and more in community centres and integrated with the voluntary and community sector, but says nonetheless that they are largely successful in reaching those they target (disadvantaged people who wouldn’t have Internet access elsewhere).

By sheer coincidence on the same day I came across a paper by “Dr Neil Selwyn”:http://www.cf.ac.uk/socsi/selwyn/ in the September 2003 edition of the journal “Information, Communication & Society”:http://www.tandf.co.uk/journals/titles/1369118x.html – unfortunately not publicly accessible (unless you are an academic with a subscription – if so look “here”:http://www.ingenta.com/isis/searching/ExpandTOC/ingenta;jsessionid=3ok9ubgqnr53e.circus?issue=infobike://routledg/rics/2003/00000006/00000003&index=5)
The paper seems to be largely based on “a report Dr Selwyn did for BECTA”:http://www.becta.org.uk/page_documents/research/digidiv_selwyn.pdf in any case (which is publicly accessible).

Anyway here are some key findings:

…The survey data suggest that, in terms of people’s effective access to ICT, public access sites have a relatively slight profile when compared with household and wider family access – perceived to offer ready access to ICT by only a minority of respondents. Moreover, when the use of these public ICT sites is examined, there is little evidence of public ICT sites attracting those social groups who may otherwise be excluded or marginalized from the information age.”

Update:Jeremy Wyatt himself was good enough to comment on this post. He said:

“I can see where you have seen a contradiction in the two reports but actually they don’t conflict in any way. As as researcher you’ll forgive me for suggesting you read the whole of both reports…we actually quote the work Neil and his colleagues did in our report.

One of the thrusts of Neil’s report is that people don’t use public internet access points much. The thrust of ours is that UK online centres have helped to introduce the internet to many new users and helped many gain skills and confidence. Our report stresses the introduction and skills services. It does not confirm or deny that public internet access once you have these skills is a viable approach. It refers to Neil’s work to suggest that there is data to suggest the opposite.

But, and its a big but, things change fast in this field and maybe public internet access has a big future once its ubiquitous.”

Mea culpa! I don’t have time to go more into this at the moment but I do encourage people to read both reports. The Hall Aitken report is “here”:http://www.dfes.gov.uk/research/programmeofresearch/index.cfm?type=5&keywordlist1=0&keywordlist2=0&keywordlist3=0&andor=or&keyword=CMF+funded+uk+online+centres&x=94&y=15.

Like Wyatt, Selwyn suggests that siting UK Online centres out of libraries, schools, colleges and museums would help and suggests

“another alternative strategy would be to develop a shift in emphasis away from community sites towards developing systems of community resources, which can then be loaned into people’s houses, thus building upon and augmenting people’s existing access to and use of ICT in friends’ and relatives’ houses.”

However he concludes pessimistically, “Although proving useful for those that use them, it appears likely that such sites will only ever fulfil a limited social role and are certainly not a panacea to the perceived inequalities of the information age.”

I share his pessimism because I feel not enough is being done to explain to disadvantaged people how what is on the Internet is relevant to their needs (and particularly not enough is done to encourage them to contribute themselves – which would help in turn to narrow the relevance gap).

I hope at least that the common recommendation of both reports – moving public access closer to where the public actually likes to hang out – will be listened to. To its credit the Office of the e-Envoy in its “annual report”:http://www.e-envoy.gov.uk/assetRoot/04/00/60/69/04006069.pdf seems to be taking this on board to some extent. The Government is funding ‘get online’ initiatives with the voluntary and community sector and spending around £3m (not a lot admittedly!) through Culture Online to, “engage hard-to-reach audiences, encouraging them to discover the potential of new digital technologies (p. 11)

Daniel Drezner cites a “Chicago Tribune article”:http://www.chicagotribune.com/news/nationworld/chi-0312250267dec25,1,7299722.story?coll=chi-newsnationworld-hed about Xmas in Eastern Europe which notes in passing:

The biggest obstacle credit card marketers had to overcome in Hungary was fear of fraud. But consumer concerns about the safety of their cards have led to an important security innovation made possible by the explosive growth of mobile phones in Hungary.

Each time a card is used, the cardholder immediately gets a text message on his or her cell phone confirming the transaction and notifying the cardholder of the balance. Initially developed in Hungary, the messaging system is used widely in Poland, the Czech Republic and Slovakia. It is now being introduced in Western Europe.

Ingenious! Not an infallible system, however – around 2% of SMSes don’t get through I seem to recall so customers should be warned that there would still be a chance their credit card transactions could be un-confirmed. Also there is a small cost per message which would eventually be passed on to customers somehow through higher fees, lower rates or whatever.

Mark Davies, the founder of BusyInternet, Ghana’s biggest cybercafe, told the BBC World Service’s latest “Go Digital”:http://www.bbc.co.uk/newsa/n5ctrl/progs/03/go_digital/24nov.ram programme that Yahoo had threatened to block all purchases to “Yahoo-hosted stores”:http://smallbusiness.yahoo.com/index.php from Ghanaian or Nigerian addresses because of the widespread fraudulent use of credit cards from his cafe. To try to head off this problem, he simply blocked all shopping. It’s extraordinary that a major portal like Yahoo could consider redlining entire nations, and that the “solution” should be for a cybercafe to block all ecommerce – particularly in a country where cybercafes may represent the only accessible Internet connection with the outside world.

A search turned up an article in “Balancing Act”:http://www.balancingact-africa.com/news/back/balancing-act_158.html from May this year with much more detail. According to the Yahoo security consultant:

The point is, 99.999% of purchases from Ghana are fraud. At least 99% of Yahoo stores don’t ship internationally anyway. Our fraud orders are up literally about 1000 percent over last year, almost all from Ghana. The cost to us in time and effort has reached the breaking point.

While it is certainly understandable why the move was threatened, imagine the furore if Yahoo had unilaterally threatened to block, say, all ecommerce from Portugal. This reveals how much unaccountable power these organizations have.

I have started to notice increasingly how people tend to assume that everyone tends to be like them – I’m no different. Down our street there is a substantial minority of people using Ocado – probably two or three a day, and they’re who we use as well. When we aren’t using them we use Sainsbury’s Online. But our neighborhood is totally atypical of the UK at large.

According to this article Ocado only gets 8,000 orders a week from across the whole UK and is therefore much smaller than Tesco’s home delivery which gets > 110,000 orders a week. If I had to guess I would have supposed as many as ten percent of people in the UK get groceries delivered online, and a quarter of those got them through Ocado. In reality it’s very roughly 1% of people ordering groceries online and 1% of those ordering via Ocado.

Salon’s Farhad Manjoo recently produced an interesting piece on the battle between cable companies and big tech companies over equal access to content over broadband cable.

As I commented on Eszter Hargittai‘s blog entry this issue appears at first to be a straightforward one – cable industry bad, free access good. But there are sound business and technical reasons why some forms of discrimination between different forms of content may be useful. For example, for good video quality cable companies want to put stuff in servers directly connected to their networks. But they can’t afford to put all streaming video content there so they may want to cut deals with certain providers. Is that unfair to the other providers? Internet users would still be able to see their stuff – just not as well.

Cable companies might also want to charge users who want to stream stuff from their “non-preferred” suppliers but keep “preferred supplier” content free (or lower cost). But while discriminatory the practice would also be fair, since the cable cos would be incurring different costs depending on where the content they were streaming came from.

Perhaps all legislation should do is demand open bidding for content deals and that per-Gb charges should have some proven relationship to the cost of providing bandwidth.calculator loan table amortizationestate ag real loansloans amortization bankmortgage get amc loan outhome loans guardian americanok loan sacramento cash payday advance$88 car loansbaltimore loans 100 investoradversary proceeding student loansexpert loaned servant alabama issues doctrinealpena alcona unions creditcredit rating advantis union financialcredit abc warehouse appliance storeaenima creditscredit card blogspot com accept e2for accreditation center detention youthon abet accredited lineabc card credit appliance warehouse Map

An MSNBC investigation shows that although big companies themselves may not spam they don’t seem to do much to prevent affiliates from spamming on their behalf and passing the results on as sales leads at $10-20 per respondent.payday companies 6 4 advance loanloan payday 5 free 7loan 500 personal57 loans student6 payday 8 loan 123personal loan 6 easy loan paydaycalifornia officers loan certification 63 3fast pay payday loan 8 day Map

Tom “plasticbag” Coates mused a while back about webloggers getting into bed with marketing companies and asks what can/should be done about it.

In the interests of full disclosure, any books I mention – particularly now that I am using Blaxm (which alas does not seem to be taking off) – I link to Amazon using an Amazon affiliate link. So far 46 people have followed those links but none have actually bought books.

It’s not that I even particularly like Amazon or that I expect the money I might get from any transaction to do anything more than perhaps take me to dinner once a year – it’s just that the software exists that makes it easy to make such links thanks to Amazon’s market dominance and if I am linking to Amazon anyway why not get free money if it were offered me?

I also recently received a registration code for some email software because someone who read it noticed I have just written a book about email. I will review it shortly but I assure you that the free license did not influence my review (as you will see when it comes out!)

So far I don’t think weblogging is big enough to make it worth marketers making large scale efforts to co-opt people but it’s definitely something to watch out for – particularly with the avowedly commercial weblogs like Gizmodo.loan aafesloan 450 fico home equityscholarship p loan u advantage ocredit 25,000 unsecured poor loanstudent loan barred statute albertafor loans calculator amorization autoalberta student legal loanspayday loans 1hr$40,000 loan with interest adoption 0no cosigner all loans