[FX.php List] Security Concerns with FileMaker Website

[Edward L. Ford]

I didn't think about checking user agents.  I just did a cursory  
investigation of this, and I may personally implement this system.   
For the bots that say they're a real browser, I'll set up other  
roadblocks to keep the info on my site (relatively) protected from  
harvesting.

For others interested in checking user agents, I found the following  
to investigate:
o) look in the $_SERVER['HTTP_USER_AGENT'] string.  More info @  
http://us3.php.net/reserved.variables   There's a built in get_browser 
() function in PHP, which I never knew.

o) PEAR also has some tools:  Check out http://pear.php.net/package/ 
Net_UserAgent_Detect

--Ed

-----------------------------------
http://www.edwardford.net

On Jan 25, 2007, at 11:41 AM, Gjermund Gusland Thorsen wrote:

> Alternative 3 works in theory but still leaves some bots crawling,
> as the worst bots tells your webserver that it's the most popular  
> browser.
>
> ggt667
>
> On 1/24/07, Jason LEWIS <jasonlewis at weber.edu> wrote:
>> Ed,
>>
>> There are a few things you can do:
>>
>> 1. Sue to get them to stop (I don't like this option because it makes
>> enemies and takes a VERY long time.)
>> 2. use a robots.txt (This option is good for bots that actually honor
>> this.  Google honors the robots.txt.)
>> 3. Detect the type of browser before you give them anything and ship
>> out bad information for unwanted browser types.  (This is only  
>> good if
>> the bot owner does not imitate browser variables.)
>>
>> As for my options, I prefer #3 as I can give the bots something that
>> they are looking for, bad information.  In perl, I use
>> $ENV{HTTP_USER_AGENT}, but I am not sure how to call this in php.
>> Anyone else know?  A quick search returned nothing on this.  Could  
>> this
>> involve $HTTP_ENV_VARS?
>>
>> Jason
>>
>> >>> elford at cs.bu.edu 01/23/2007 10:18 PM >>>
>> Hello everyone,
>> In the past hour, I've done some analysis of various logs and emails,
>>
>> and I've come to a chilling realization that I've never had before
>> about bots harvesting information from websites -- I knew it
>> happened, but I never knew the scope of the problem until tonight --
>> and this is a low traffic website!
>>
>> So, I have a website which contains a public listing of email
>> addresses and websites from a FileMaker database.  I want to stop
>> unknown bots from crawling the site.  All of the data comes out of
>> FileMaker, nicely formatted as links for the end user's clicking
>> convenience.  I have a solution to fix email addresses from being
>> harvested, but I was wondering if anyone knows of a way to prevent
>> website addresses from being harvested, but still clickable as a
>> hyperlink.
>>
>> I thought maybe a PHP redirect link, like redirect.php?id=16 where
>> redirect puts a user at the website listed in record 16, but once the
>>
>> PHP is all said and done, we're still at the linked website, so that
>> doesn't really prevent anything from being harvested.
>>
>> Is there a way to maybe detect is a link was actually clicked by a
>> person, and not just passed through by an automated bot?  PHP is
>> preferable for such a solution -- JavaScript is too easy to turn
>> off.  Or, is there a way to specify that only bots from places like
>> Google, Live, and Yahoo are allowed to crawl the site?
>>
>> Hopefully my predicament is clear.  I need to solve this ASAP...
>>
>> --Ed
>> ---------------------
>> http://www.edwardford.net
>>
>>
>> _______________________________________________
>> FX.php_List mailing list
>> FX.php_List at mail.iviking.org
>> http://www.iviking.org/mailman/listinfo/fx.php_list
>>
> _______________________________________________
> FX.php_List mailing list
> FX.php_List at mail.iviking.org
> http://www.iviking.org/mailman/listinfo/fx.php_list

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.iviking.org/pipermail/fx.php_list/attachments/20070125/3302716d/attachment.html