UPDATED 20 Mar 2002 ORIGINAL 12 Feb 2002
FINAL UPDATE? 8 Apr 2002: A Google search for Scientology now turns up astonishing results--a balanced representation of what is actually out there on the net, ranked by popularity. It appears that whatever remaining issues may remain, the substantive central issue--whether this search engine accurately represents web content--is at least for the moment resolved. The rest of this page should, therefore, be considered, if not obsolete, at least partially superseded by events.
23 Mar 2002: Yahoo! Directory on Scientology Opposing Views noted for linking to this page, which is odd, considering that they deleted their link to Xenu.net although they have, so far, escaped scrutiny for doing so. Interestingly, this only appears to have disappeared for the search term "Scientology" itself. On a search for "Operation Clambake" it shows up first.
Evening 22 Mar 2002: Kady O'Malley comments on How Ava Paquette Hoodwinked Google.
Morning 22 Mar 2002: Xenu.net itself takes over with more coverage of the news events, including the replacement of Xenu.net in a search for "Scientology" on Google. It should be noted that only the link http://www.xenu.net itself has been replaced. Other URLs on the list from Scientology legal beagle Ava Paquette remain missing. As an amusing aside, there's already a song called "Ava Gram" by Subgenius slackrockers El Queso or "Enturbulator 009" about the habitual DMCA abuse of Scientology lawyer Ava Paquette.
Evening 21 Mar 2002: Xenu.net gives a full news update on the events of today and yesterday.
21 Mar 2002: Slashdot weighs in. Shortly thereafter, so does Heise.de, a German geek webzine fairly similar to Slashdot. Note: I have been corrected that Heise.de is, in fact, a publication of the German newsmagazine "Computer und Technik"
Xenu.net Comments on its removal from the Google search engine.
FLASH: This Usenet post from Andreas Heldal-Lund, webmaster of xenu.net, indicates that Google has removed links from its search engine based on a DMCA notification from the cult of Scientology. If true, this has huge repercussions for the entire Internet--sites will disappear from the Google search engine based upon a mere allegation that they MIGHT be infringing--in other words, based upon the mere opinion of a copyright holder. In light of AOL's recent smash victory in which it was proven that the DMCA provided it immunity, Google's decision seems bizarre, and even suicidal. Note that these Scientology allegations are nothing new. This cult has been shopping their laundry list of bogus allegations around to anyone who would listen for years. They have never to date filed an infringement suit, although they have had years to do so. The obvious reason for this is that Xenu.net contains no infringing content.
QUICK AND DIRTY LINK LIST: A few links to the developing story concerning the disappearance of Xenu.net from Google. Scientology conspiracy? Google revenge? Inadvertent fallout from overzealous supporters of the site? Only Google knows.
Daily Rotten article
Microcontent article on "Google Bombs"
Slashdot subthread on "Search Engine Payola"
Search for "Scientology Google search" on Google Groups (some but not all ongoing Usenet threads)
UPDATE: 20 March 2002: For some reason, some time between 02:55 PM EST and 07:25 PM EST yesterday, Xenu.net entirely disappeared from a Google search for Scientology. There are a variety of reasons this could have happened. Xenu.net could simply have been down at the exact moment Google spidered it. (However, it was up as late as 02:55 PM and was also up at 07:25 PM EST when I noticed its disappearance from the engine.) While other explanations are more suspicious, there's no reason yet to assume foul play. It could merely be an ironic quirk of fate that this has happened right now; but if it isn't, it will certainly be noted widely.
UPDATE: 6 March 2002. This update is to take note of some major changes in what I was reporting, as well as to clarify a few things that have resulted in misunderstanding. For the past week or so, the major situation I had been commenting on has changed. Xenu.net is at #4 on a Google search for Scientology. Another development is that the Scientology directory on the Open Directory Project is now listed as "This category needs an editor," indicating that for reasons I can not elaborate on, Scientologist editor andir is no longer editor of that category, or for that matter others for which she was previously listed.
These constitute rather major changes in the data reported below, which may become of only historical interest in the future. The second part of this update is to correct a few areas where I was vague. One of these was in seeming to blame the Open Directory Project for this situation. I should have mentioned several mitigating factors. One of them is that even when no critical sites were in the first page of results, due to Google's connection to the ODP, categories containing the search term were listed immediately above the search results, one of those categories being the Opposing Views category on Scientology. Another is that to the extent ODP influenced the PageRank of critical pages at all, that influence was positive. It was not my intention to suggest that either ODP, Google, or both, were somehow colluding with Scientology to give inflated rankings, nor to object to reasonable rankings like scientology.org being the first search result. Finally, these results apply only to the use of the single search term "Scientology" by itself. Even a relatively clueless search engine strategy like typing natural language questions into Google defeats all this very expensive spam. Try Should I join Scientology? or Is Scientology a cult? and you get almost entirely critical pages as results. (I'm aware Scientology could easily use these ideas in new spam attempts, but frankly I like the idea of them adding a rant denying being a cult to all their pages.)
In conclusion, Scientology is of course not the only entity using these strategies. As this Slashdot post shows, the porn industry uses tactics similar to Scientology's. While I suspected this might be the case, this revealed a similarly sophisticated strategy (as separated from a lot of the blatant spam that bounces off Google's well-designed search engine).
12 Feb 2002: This page was created to examine the reasons for Scientology's high placement in so many search engines despite its egregious unpopularity and highly popular sites opposed to this cult, such as Operation Clambake (also known as xenu.net) and Ron Newman's The Church of Scientology vs. the Net page. I created the page to discuss the interesting results one gets on Google on a search using "Scientology" as a search term.
Google is, of course, the king of all search engines, so it is with Google that I am primarily concerned, although the interaction between Google and the Open Directory Project is also interesting. The Open Directory Project is a Yahoo!-like hierarchically-organized directory which is widely mirrored by a variety of ISPs. The mirrors themselves are usually highly ranked in Google search results, so links from ODP tend to increase ranking.
It should be noted that this situation is worsened by the fact that the ODP directory related to Scientology is run by a Scientologist, while the ODP directory relating to Scientology opponents remains without an editor. ODP routinely refuses applications for editors of the directory, and some have accused them of round-filing user submissions, while every new Scientology-owned site is added immediately and without oversight. When sites critical of Scientology are added, it is often after a long delay, if the site is added at all. Despite numerous submissions to the ODP in the last three months, almost none have been added. This problem is also detailed on this thread on the newsgroup alt.religion.scientology.
Google prides itself on its PageRankô technology, which while largely proprietary, in the words of its creators "relies on the uniquely democratic nature of the web by using its vast link structure as an indicator of an individual page's value." Google continues, stating: "In essence, Google interprets a link from page A to page B as a vote, by page A, for page B. But, Google looks at more than the sheer volume of votes, or links a page receives; it also analyzes the page that casts the vote. Votes cast by pages that are themselves "important" weigh more heavily and help to make other pages 'important.'"
Unfortunately, in the case of this particular search term, the Church of Scientology, a large corporate entity, has figured out how to exploit this to their advantage by having large quantities of domains that are packed heavily with links to each other. Figure 1, generated by the VisIT software, is a stark illustration of this.
As you can see, Scientology-owned pages exclusively and exhaustively point to other Scientology-owned pages. Some, such as exactscientology.net, are little more than highly-linked link lists whose sole purpose is to point to other Scientology-owned pages which themselves link to exactscientology. Whois information for the vast majority of these indicate identical registrations such as this one for exactscientology.net. Therefore, Scientology-associated pages appear to be more highly ranked than critical pages, which appear to exist as islands, sparsely connected to each other.
The rankings are not representative of how broadly Scientology critical pages are linked. These search results checking links to scientology.org and xenu.net using the link: operand on Google indicate that despite comparable numbers of links, the search rankings vary widely. The links to scientology.org are almost exclusively from other Scientology owned pages, such as exactscientology.net and thousands of nearly identical template-generated "spam pages." In this case, in Google's language, the "uniquely democratic nature" of page ranking depends on how much money you have to buy duplicative domain registrations.
To be fair to Google, this effort by Scientology has been quite costly, involving huge numbers of domain registrations and the creation of a vast network of incestuously linked pages. Figure 1A shows this in more detail.
Looking at this in even more detail, one sees that outside the most obvious "core" Scientology sites, a galaxy of smaller domains also points to scientology.org and its sister sites, while Scientology-critical sites largely sit by themselves. While they are linked from other sites, those links do not add up to the sheer bulk of the incestuous network of links which creates Scientology's illusion of relevance--even though most of those links are, effectively, the result of Scientology linking to itself. Where critical pages do link to each other, the links are not effectively reciprocated.
The same general scheme prevails on the Open Directory Project, although with a slightly different focus, as Figure 2 shows.
Figure 2A takes a closer look at the trend.
Using both Google and the Open Directory Project as sources, the picture is complete.
Taking a closer look at this, we have the final result: scientology.org sitting atop a vast pyramid of links to and from it, boosting its ranking on search engines as well as the rankings of its sister sites, all owned by the same entity, yet appearing in the "uniquely democratic" ranking system as if they have been elected number one by universal acclaim, although the vast majority of this ranking is generated by what amounts to linking to itself.
While the Church of Scientology is merely one of thousands of entities attempting to manipulate its rankings in Google and other search engines, unlike the others, Scientology has managed effectively to subvert the PageRankô technology and elect itself supreme.
It doesn't seem entirely unreasonable for scientology.org to be the first hit on a search for "Scientology", but for it and sister sites to be the bulk of the search results is getting ridiculous. As a closing note, I should note that this analysis focuses almost exclusively on the "voting" aspect of the PageRankô technology and that other factors play a highly important role in page ranking, including relevance of content and how often and prominently the search term is mentioned on the page.
Note: All images on this page generated by the VisIT software created by the Beckman Institute for the University of Illinois, Urbana-Champaign.