With the continuing proliferation of viral marketing efforts by pretty much everyone, we’ve seen a huge jump in the number of microsites over the last few years.

What is a microsite?

It’s generally a small website made up of just a few pages created by a company for the purpose of promoting a new product or ad campaign that resides on a domain name different from the company’s own domain.  A recent example is nomoretightywhities.com – part of the latest viral campaign by Canadian fashion underwear company, Ginch Gonch.

The concept is fun.  Protests against Tighty Whities are planned (the first was last weekend in Montreal).  There are fake newscasts and introspective videos.  A manifesto is presented.  Music is available for download and there’s a contest to gather email addresses.  Of course, there’s a link to the Ginch Gonch store.

The site does a lot of things right.  Its overall tongue in cheek tone fits the Ginch Gonch brand like a glove (or perhaps more like a snug boxer brief in this case).  The site is appealing to just the sort of target audience that Ginch Gonch wants – people who relish wearing colorful and humorous underwear.  The contest is a great way to obtain users’ permission to receive future emails.  52 pairs of great underwear is a fantastic prize, but the freely given targeted email addresses are a true goldmine for Ginch Gonch.

Unfortunately, this microsite (and many others) misses the mark from a search marketing perspective.  For one thing, there’s very little text.  The homepage actually has zero text.  Yes, you and I can read text on the homepage, but it’s contained within a flash file.  The search engine sees nothing.  Deeper links into the site are also contained in the flash file.  This makes it next to impossible for a search engine to find anything beyond the homepage.  A quick check of Google’s index shows that as of today, only the homepage is indexed.

Another problem is the separate domain.  Microsites are typically pretty good at attracting links from other websites.  This makes them important in the eyes of a search engine that’s trying to come up with reasons to rank one site higher than another.  Unfortunately, these links are wasted on nomoretightywhities.com instead of being directed to ginchgonch.com.  Links gained to the company’s domain would pay dividends in the form higher search engine rankings for underwear related keywords (and there were over 10 million searches for underwear related keywords on Google alone last month) long after the viral campaign has fizzled.

So, there’s a dilemma with microsites of having a separate domain name that’s easy to remember, easy to pass along and doesn’t look like an ad vs. gaining links to the website that does the actual selling.  There are many factors to weigh and there is no cut and dry solution to the dilemma.  If Ginch Gonch is simply trying to get their brand out in a fun way then they’ve probably accomplished that.  On the other hand, if they’re trying to get useful links in an effort to dominate searches for underwear related keywords, they’ve failed miserably.  My guess is that they’re hoping to attract traffic to the main website through the microsite.  I’m also guessing it’s not working as well as they’d like.

From a search marketing perspective, the best way to implement the “No More Tighty Whities” campaign would be to put it on ginchgonch.com.  Using a subfolder like ginchgonch.com/nomoretightywhities, or ginchgonch.com/nmtw would insure that all links to the campaign would count toward the authority of ginchgonch.com, thus helping that domain in its efforts to rank well for searches like “underwear”.  One of the downsides to this strategy, of course is that it’s obvious the whole thing is a publicity stunt by a company.  Another downside is that the web address is longer and more difficult to remember/pass on.

To get around this, Ginch Gonch could have purchased nomoretightywhities.com and 301 redirected it to ginchgonch.com/nomoretightywhities.  In this way, the domain (nomoretightywhities.com) can be easily passed around, but those who go there will be automatically taken to the subfolder of ginchgonch.com.  Likewise, anyone linking to files on the site would then be linking to ginchgonch.com.

Again, the situation is not cut and dry and we can’t really know what Ginch Gonch’s intentions were when developing the site.  Marketing strategy should always stem from a set of goals.  It is, however, a good illustration of what can be done wrong (and right) with a viral marketing campaign’s website.

EDIT:  It turns out that we can know Ginch Gonch’s intentions after all.  I was contacted this morning by Rebecca Wilkinson, the Director of Shmoozing and Carousing at Ginch Gonch.  She stated:

“…the purpose of the NMTW site is to entertain and to grab the attention of people not normally in our target audience and it is working well in that way. We also intend to drive people to the site via on-the-street outreach campaigns where we have GG brand ambassadors interact with people and hand out stickers and info that will drive them to the site.”

So, they weren’t targeting search traffic at all.  The campaign has been successful thus far in terms of their goals.

A few weeks ago, we were completing the development of a tracking application for a website.  Basically, this tracking application exists on dynamically generated pages of the website that have the following structure:

www.mydomain.com/products/track/

It’s basically just a little tool that logs the user’s ip address, the item they clicked on, and then automatically redirects them to a vendor that sells that product.  The user never even knows they’ve visited the page.  To them, it’s a seamless transition from the item they clicked on to the vendor’s website.  It’s there to help us track user behavior, learn how to make the website better, and keep the vendors who are paying us for those referrals honest.

Now, obviosuly there’s no reason for a search engine to need to index these pages.  There’s no useful content there at all.  So, we made use of robots.txt to tell the search engines that there is no reason to look at those pages.

What is robots.txt?

Robots.txt is simply a file that can be placed on a website to notify automated “crawlers” that there are certain parts of the site that should not be visited.  It makes use of the robots exclusion protocol.  When an automated crawler (such as Googlebot) visits a website, it looks at the robots.txt file to see if there are any pages it should not vist.  Not all automated crawlers pay attention to robots.txt, but the major search engines claim that they do.

Using robots.txt, we told the crawlers not to visit any pages in the “track” folder.  That worked out really well.  A few weeks later, we decided that if a user rolled over a link and saw the word “track”, they might get spooked and wouldn’t want to click on that particular link.  People don’t really like the idea of being tracked.  So, we decided to change the structure of the tracking application to the following:

 www.mydomain.com/products/buy

This naming convention seemed much more inocuous and was in line with what the user was trying to do.  We updated the robots.txt file to reflect these changes and uploaded the changes.

Here comes Googlebot!

Much to our surprise, a few hours later, we started to see a lot of clicks coming from the same ip address.  Thinking I had a rogue Chinese robot on my hands (that sounds silly but it has happened before), I looked up the ip address.  Lo and behold, it belongs to Google!

Throughout the day, I watched as Googlebot clicked on item after item with a frequency of roughly every 2 minutes.  I rechecked my robots.txt file.  It should have been blocking this activity.  I logged into my Google webmaster tools account and found the problem:

Google downloads robots.txt about once every 24 hours.

This particular website’s robots.txt file had been downloaded earlier in the morning.  Even though these were new files, the protocol is an exclusion protocol.  Since these files were not listed in the file Google had cached, they were fair game.  A few hours later, Googlebot called in reinforcements.  The website was now getting hit by two different Google ip addresses with a frequency of roughly every hour.  Unfortunately, they didn’t bring their credit cards.  They kept going until about 1:00 am the next morning when the new robots.txt file was finally downloaded and cached.  In total Google crawled and indexed a little over 1000 pages of content that was blocked using robots.txt.

The funny thing is that these pages were actually indexed.  I searched and found them a week later.  They were all indexed with the content of the landing pages on the vendor’s sites.  So, we inadvertantly pulled off a decent sized cloaking operation – something that is expressly against Google’s quality guidelines.  I sweated it for a while, but there doesn’t seem to be any negative effects on the site’s rankings.

So, the lesson is that if you’re going to upload pages that you don’t want a search engine to crawl, you should disallow those pages in the robots.txt file and make that file available at least 24 hours before you upload the actual files to the website.  If you have a Google webmaster tools account, it’d be a good idea to log in and see which version of the robots.txt file is in Google’s cache.

I thought the saga was over, but a few days later a few of the pages were crawled by Googlebot again.  In this case, it was only about 5 pages, so it may have been a small bug in the system, or perhaps even a Google employee hand checking things.  In any case, the pages are still in the index.