[FX.php List] Security Concerns with FileMaker Website

[Edward L. Ford]

Hello everyone,
In the past hour, I've done some analysis of various logs and emails,  
and I've come to a chilling realization that I've never had before  
about bots harvesting information from websites -- I knew it  
happened, but I never knew the scope of the problem until tonight --  
and this is a low traffic website!

So, I have a website which contains a public listing of email  
addresses and websites from a FileMaker database.  I want to stop  
unknown bots from crawling the site.  All of the data comes out of  
FileMaker, nicely formatted as links for the end user's clicking  
convenience.  I have a solution to fix email addresses from being  
harvested, but I was wondering if anyone knows of a way to prevent  
website addresses from being harvested, but still clickable as a  
hyperlink.

I thought maybe a PHP redirect link, like redirect.php?id=16 where  
redirect puts a user at the website listed in record 16, but once the  
PHP is all said and done, we're still at the linked website, so that  
doesn't really prevent anything from being harvested.

Is there a way to maybe detect is a link was actually clicked by a  
person, and not just passed through by an automated bot?  PHP is  
preferable for such a solution -- JavaScript is too easy to turn  
off.  Or, is there a way to specify that only bots from places like  
Google, Live, and Yahoo are allowed to crawl the site?

Hopefully my predicament is clear.  I need to solve this ASAP...

--Ed
---------------------
http://www.edwardford.net


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.iviking.org/pipermail/fx.php_list/attachments/20070124/941de619/attachment.html