[Stacy69]

You seem to know a lot about programming and the internet.

I recently did a search for VW on Google and not a single VW result comes up. The first link is wiki and the second is VC.

Why would this be? I see no reason for VW to be blacklisted from Google but since it's delisted, it's no longer getting SE traffic from Google, however VC is getting it all. Likewise if the name VW is being used to promote VC (the result shows VW's name with link to VC) isn't that an example of trademark infringement?

[zardoz1962]

Skip this if you don't care, but I am going to assume if a person asks, they want to know.


Nothing nefarious, just the way Google works.

The model for Google, since the time it was developed, is to gauge how important a webpage is by where and how the keywords appear in the page, how many other pages link to that page, and how important those linking pages are in turn. They assume that important pages link to other important pages. That is the big reason why taking a site down for more than a couple of days is completely idiotic.

If Google tries to visit a page and does not find it, is will usually let things slide for a while, but will eventually decide that page no longer exists and deletes it from the index. VW, since it was once a very searched for, dynamic, and linked to site, is visited frequently. The good thing is that new pages are quickly found by Google. The bad thing is that temporarily off-line pages are quickly lost by Google. The really bad thing is a lot of those pages that linked to VW were on other VW pages and in other VW domains. Google might now that voyeurweb.com and vnc.voyeurweb.com are the same site network, but it is unlikely they took into account that homeclips, redclouds, the playground, Igor's HOF, funbags, etc. were all start of the same family so a lot of those those interwoven links probably counted (it can detect that some connections can be ignored, but not all).

That is what happened. If you Google "site://voyeurweb.com" you will get a listing of all of the Voyeurweb.com site's pages in the Google index. Even my pathetic little company website which has maybe 20 visitors a day has 400+ pages in the Google database. A few months ago, VW would have thousands show on the first search.

Today it has six.

At the bottom of the Google search you will see a link reading "repeat the search with the omitted results included". If there are many pages very similar to each other, you will not see those on the first page as Google assumes they are just duplicates from dead copies, renamed pages, or multiple websites with the same content. Click that. Up at the top it shows "About 116,000 results". That is an estimate based on past searches of the database of various kinds. Google knows that most people will not go to page 2, so they don't bother with a complete search of the database until you ask.

Click on the "next" link. That will search a bit further in. Now you see the truth. Sixteen pages from voyeurweb.com and every child domain in the VW network.

That all that remains from a site that ranked in the few thousands and had millions of visitors a day. No nefarious plot. No government force. No legal threats. No blacklist. Just the natural consequence of letting your site stay off-line.

IF, and that looks like a big if, VW comes back online and IF they are smart enough to keep page URLs the same, the pages will make it back to the database over the course of a few months. Google "link://voyeurweb.com" and it will tell you it knows of 408,000 links into the VW site. As long as those stay, Google will keep looking for VW to return. The longer VW is down though, the less often Google will check and of course the linking pages will eventually be updated or deleted themselves and those links permanently lost.


Now about VC being #2. The notion that Google allows sites to show up in searches for other companies is an entire industry. There are SEO companies that specialize in getting Pepsi sites to appear when you search Coke (well, not that particular example, but you get the idea). Targeted companies have claimed infringement and tried to get Google to censor their links, but at least in the US there is no basis. Companies are free to refer to each other in their ads, as long as they stay within certain limits. VC can name Voyeurweb on their site, as long as they do not claim to be Voyeurweb. Claiming to be the people behind it or the successor to the community are 100% OK as advertising claims, as long as not patently false.

So you search Voyeurweb. Google checks its database for all webpages with the word "Voyeurweb" occurring on the page. ******** has the work voyeurweb on its homepage, so that counts. Now it checks how important the page is. Google only knows of 7,340 incoming links so it is not so important (even my puny 200 customer company has over 4,000). I would not be surprised if it also checks to see if the linking pages also contain the word "Voyeurweb" at or near the link, which ups the importance.

I assume you actually searched voyeurweb.com. That does have VC as link #2. just searching Voyeurweb is even worse. Link 1 is Wikipedia, link 2 is a listing of the 50 websites most like Voyeurweb (at least it is easy to fins a new home) and link #3 is for thecandidboard.com. VC is not until #6

So how did VC get to be so high? Easy, something has to be #2 and all those formerly important VW pages don't exist anymore. Yet another dumbass business decision. Look at what is below it. Torrents of pictures scraped off the VW site mostly, a few other Porn sites that snuck the work "Voyeurweb" into their text. Page #10 is an ad for 2006 looking for a programmer to write a VW clone. A month ago, I'll bet that was page 5000!


There is one hole in this - the main page of voyeurweb.com should not have fallen off. My bet is that either in those first few days with the site entirely down the main page was checked for frequently, was not found and was dropped. Another possibility is they may have accidentally asked not to be indexed (there are two ways to do this, one in the code on the page and one in a file on the server). Either way, I can find nothing in the page now that should prevent it from being listed once it gets found again. At that time, a lot of high ranking pages should reappear. Probably not above wikipedia, but at least above the competition.

[Stacy69]

Wonderful response and analysis, thx!