Thanks to the publicity provided by Google’s move, lots of applications are coming out of the woodwork. Here’s the latest news in brief. I just learned about three more products:
* “ISYS”:http://www.isys-search.com/products/desktop/index.html (free to try but they don’t tell the price up front)
* “Filehand”:http://www.filehand.com/ (now free – has the embarassing motto, ‘It’s like Google for your computer’)
* “x-friend”:http://www.x-friend.de/en/start/introduction/ (written in Java and runs on ‘any operating system’ – produced in Germany)
I also just learned about two reviews:
* “PC World just reviewed ten desktop search applications”:http://www.pcworld.com/reviews/article/0,aid,117809,pg,4,00.asp not including Google Desktop Search and
* “CNet”:http://reviews.cnet.com/4520-3684_7-5536376.html has just reviewed six desktop search apps (rather briefly) including Google Desktop, though they concluded “Copernic is best”:http://reviews.cnet.com/Copernic_Desktop_Search/4505-3684_7-31087427-2.html?tag=tab.

For more on this stuff see “earlier”:https://blog.org/?cat=29&submit=view (and scroll down that page to see more search related stuff).

“Google Desktop”:http://www.desktop.google.com/index.html has arrived and two more desktop searching products are on their way. AOL is reportedly developing AOL Desktop Search and a new search engine, “Exalead”:http://beta.exalead.com/search also plans a “desktop search product”:http://beta.exalead.com/search/C=0MlQAMAA%3d/2p=5.

BBC World’s Click Online just did a “short report”:http://www.bbcworld.com/content/template_clickonline.asp?pageid=666&co_pageid=2 about hard disk indexing programs which covers some of the same ground as I have done earlier (for example “here”:https://blog.org/archives/cat_search_engines.html#001238).

The “Diskmeta”:http://diskmeta.com/ search engine ‘works on all Windows platforms (98 or higher)’ and ‘ is fast, intuitive and unfussy. You can also view the raw text in a special preview window but doesn’t have a preview facility like X1, dtSearch or the new Copernic Desktop Search’. Unlike some other desktop search engines it supports a variety of “boolean operators”:http://diskmeta.com/en/doc/request.asp.

The free version (for non-commercial use) only indexes txt, .doc and .html however – for indexing PDFs you need to pay, and Diskmeta doesn’t index Outlook email.

Thanks to Jeremy Wagstaff for the heads up.

Also see “here”:https://blog.org/archives/cat_search_engines.html#001230 and “here”:https://blog.org/archives/cat_search_engines.html#001202 for earlier coverage of hard disk searching programs.

I tend to assume that for all its flaws The Economist gets its facts right – at least on technical issues. But this article on How Google Works in their technology section recently repeats a popular misconception about search. The article says, ‘Google is thought to have several complete copies of the web distributed across servers in California and Virginia’ – whatever they do have it is nothing close to a complete copy of the web. Even if they had a complete index of the text of the first 100Kb of each page on the publicly spidered web (the most they would even claim) this would still miss the huge volume of available information that is stored in web-accessible databases (like the “British Telecom phone book”:http://www2.bt.com/edq_busnamesearch).

I believe that a search engine that managed to do a good job of searching this ‘invisible web’ alongside the ‘surface web’ would have a good shot at the number one spot.

P.S. While on the subject of search, here’s a tip – to get a (small) discount on your next Amazon purchase, check out their new A9 search engine.

Search engine guru “Greg Notess”:http://notess.com/ has produced a Search Engine Overview featuring in-depth “search feature comparisons”:http://searchengineshowdown.com/features/ and frequently updated reviews of individual services. It covers directories and news search engines as well as the major search engines and ones that are now defunct. Learn all the advanced features of each search engine without having to click around the ‘advanced search help’ pages.

He doesn’t yet include “jux2”:http://jux2.com/index.php which lets you search any two of the major search engines simeultaneously or most of the other metasearch sites. Mind you none of the other metasearch sites I just checked seemed to correctly deliver all Yahoo, Google and Teoma results. “Dogpile”:http://www.dogpile.com/ for example finds Yahoo and Google at once but seems to truncate the results and didn’t find any Ask or Teoma results for my name although I know they are there.

If you are thinking about analysing group behaviour by looking at links (particularly web-mediated group behaviour), you must check out the imaginatively-named Link Analysis by “Mike Thelwall”:http://www.scit.wlv.ac.uk/~cm1993/ which contains lots of relevant links from the book he is writing on the subject. Also check out “SocSciBot”:http://socscibot.wlv.ac.uk/ a free Windows link crawler created by his group for social scientists to use. It’s nice to see a fellow academic being so generous in sharing his resources with others.

John Battelle asks:

Imagine the ability to ask any question and get not just an accurate answer, but your perfect answer – an answer that suits the context and intent of your question, an answer that is informed by who you are and why you might be asking. The engine providing this answer is capable of incorporating all the world’s knowledge to the task at hand be it captured in text, video, or audio… What opportunities arise when knowledge can be so easily gathered? What threats? How might this change our social structures, our politics, our economy?

As far as I am concerned, the danger is not what would happen if such perfect search existed – the danger is that “good enough” search might exist that seemed to deliver near-perfect results but actually relied on still flawed or commercially biased algorithms and had an underlying database that was incomplete. People might forget to use other better but harder to use sources of information and those other sources might gradually disappear. They might also put too much trust in the results they get.

In fact I fear this is already beginning to happen with Google.

I’ve been hoarding lots of search engine related postings waiting to put them up but my list of un-posted and rapidly ageing postings is getting out of hand. Here then without (much) comment are some links:

*Google gives free ads to non-profits* – See “Google Grants”:http://www.google.com/grants/. Charities must be based in the United States (at the moment). Thanks to Aaron Swartz’s Google Blog for the link

*A search engine for discussion forums* – Pandia alerted me to this new “Lycos search feature”:http://discussion.lycos.com/default.asp. It’s good to see some search innovation coming from outside the ‘big two’. Seems to me that other search engine companies may be able to carve out a role delivering specialised searches that the big boys don’t do (or don’t do as well).

*A new search engine specialising in business information* – “find.com”:http://find.com/matchpoint.aspx
Thanks to Tales from the Terminal Room

Directory of Open Access Journals (free, full text, quality controlled scientific and scholarly journals) now has a search facility for 319 of the 1219 supported journals. Thanks for the heads-up Pandia

*Interesting source of Google-related info* – Google Metrics Watch – it ‘daily queries Google for a set of terms. The number of pages returned is stored in a database. The idea behind this is that an increase or decrease in the number of web pages refering to a subject COULD INDICATE (or will probably be associated to) the popularity of this subject.’

*How to find pages linking to your own site* – Link search with Yahoo! and Google

*Google Groups (finally) supports mailing list creation* – Check out the new “Google Groups”:http://groups-beta.google.com/ Thanks to Google Weblog for the link

*Yahoo search to access “deep web”* (for a price – selected partners only) – Yahoo crawls deep into the Web – News – ZDNet

Copernic has just made a free “desktop search tool for Windows”:http://www.copernic.com/en/products/desktop-search/index.html available that searches your files, email and the web. It has already been “reviewed favourably”:http://www.searchenginewatch.com/searchday/article.php/3401711. From the looks of it this is the best desktop search tool yet for many – especially at the price! If I ever get my email store back (see “this heartbreaking tale”:https://blog.org/archives/cat_personal.html#001218) I will be sticking with “X1”:http://www.x1.com because unlike Copernic’s product it indexes Eudora email (which I prefer).

Thanks to John Battelle for the link

See “earlier”:https://blog.org/archives/cat_search_engines.html#001202 for more coverage of hard disk indexing programs.

There are lots of photo sharing services around – (two years ago I did a “little comparison”:https://blog.org/archives/cat_useful_web_resources.html#000385 of several of them which offer photo printing as well) – but “Flickr”:http://www.flickr.com/ – which I started to try out yesterday – seems to be the Internet geek’s best choice (they’ve got Cory Doctorow, “renaissance geek”:http://www.craphound.com/bio.html advising them so it’s “turning up a lot on boingboing”:http://www.boingboing.net/cgi-bin/mt/mt-search.cgi?IncludeBlogs=1&search=flickr).

If after reading the description below, Flickr appeals to you and you want to try it out (basic membership is free), instead of visiting the site right away and signing up I would appreciate it if you could “email me”:http://davidbrake.org/contact.htm and I will invite you. There’s an offer available at the moment – if I successfully invite 5 new people to join Flickr I will receive a Flickr Pro Account (valid until September 15th, 2004). Yes I have therefore a small interest in selling you on the idea but I already have other photo library accounts so it’s not a huge deal for me one way or another. Anyway…

The Flickr feature that first caught my attention is that it has an automatic ‘post to your blog’ feature (which I used yesterday). It also lets you post photos to your Flickr site and/or weblog via email and directly from camera phones. What’s more intriguing though is that it has a number of creative ways of organizing photos. Most photo sites make you sort pictures into albums. This one lets you attach pictures to several different groups, tag them by keyword, lets you and your Flickr-using friends pool and organize your pictures in interesting ways etc etc.

Geekily enough it also supports RSS in different ways so people can automatically know you have added more pictures and they have built in chat and messageboard facilities so people with similar interests can share pictures (yes there are porn-related groups as you’d expect but also groups like “Bonsai lovers”:http://www.flickr.com/groups_view.gne?id=36521982934@N01). I’m a sucker for organizations like this one that just don’t seem to know when to stop adding new features on the off chance that someone will use them. “ICQ”:http://www.icq.com/ was a bit like that – it’s a pity the full version isn’t seeing much development any more. Anyway…

There’s a quick overview of Flickr’s features “here”:http://www.flickr.com/learn_more.gne and a longer “get the most out of Flickr”:http://www.flickr.com/get_the_most.gne guide but the best way to figure it out is to sign up and try out its features.

Signup is free. For the moment you can only sign up for their free account which lets you share either your most recent 100 photos or photos uploaded in the last 3 months (whichever comes first). It must be said this is not over-generous – “photo.net”:http://www.photo.net/ has a 100Mb quota, “Webshots”:http://daily.webshots.com/scripts/signup.fcgi lets you store 240 photos. Also at the moment the only software available for bulk uploading of photos is for Windows XP and MacOS X. Later they will have software available for more operating systems and premium accounts with more storage and capabilities (they are in beta testing at the moment).

P.S. I just discovered “Phil Gyford”:http://www.gyford.com/ has also recently “taken a shine to Flickr”:http://www.gyford.com/phil/writing/2004/08/25/seeing_the_light.php.