[FX.php List] Web traffic versus native FileMaker use

Dale Bengston dbengston at tds.net
Mon May 10 07:38:10 MDT 2010 

Coincidentally, my client with the auto-enter calc fields has reached the point where reports and order entry into FMP via the web are nearly too slow to be usable any more. My resolution is also to split the web from the FMP client-server, but I'm following a different path. I'm putting the web records into MySQL. We're pushing a million sales history records, and the design of the system (lots of portal fields) and the volume of data is exposing FMP's weakness as a web data source. Why, oh why is the XML data pull from FMP so sloooooow?

I'll be using FX.php to work with the MySQL mirror, as well as sync up orders placed via the web with the client-server FMP solution. Wish me luck!

Dale

On May 10, 2010, at 1:27 AM, Gjermund Gusland Thorsen wrote:

> If you make layouts with only the needed fields, you should be able to do at least 200 000 records edited or insterted pr hour.
> 
> Scripts and additional fields will add time spent, not number of records pr minute to the execution of solution.
> 
> ggt
> 
> 2010/5/10 Jonathan Schwartz <jschwartz at exit445.com>
> Thanks for the great feedback, Bob.
> 
> In this case, there are 10 modules in the online system, each offering up a different slice of data for web customers.  Fortunately, the data for all but one of the modules can be separated off to a separate DB and needs updating only once per week.  The last module...and the busiest...needs to be live current data.
> 
> So for the first step of isolating the web modules away from the native system, 9 out of 10 can be done with no issues.  The last module will still need to talk to the mother ship....a step that I can address if and when it is needed.  I will keep Chris Hansen's number handy. ;-)
> 
> My original post was to try and work through issues of native FileMaker interfering with Web FileMaker on the same machine.  I have moved past that to work on HOW to separate the two...not IF.
> 
> Thanks again,
> 
> Jonathan
> 
> At 2:36 PM -0500 5/9/10, Bob Patin wrote:
>> Jonathan,
>> 
>> I'm late to the conversation, but have some input to share:
>> 
>> On the project I'm involved with now, we moved the database server into the museum for which it was designed. There's also a web app to go with it, and their IT guys have never been able to get the web server to work with the database server (you can imagine how impressed I am with these idiots).
>> 
>> So I finally created a 2nd database with only the tables that I use to sell tickets on the web, and it works fine. Here's the rub, and why I mention this here:
>> 
>> I couldn't find a way to synchronize these 2 databases; I wrote a script that would pull new orders into the "parent" db, but without opening the 2 databases on my machine (one on one FM server, the 2nd on another 700 miles away), I couldn't get the script to fire.
>> 
>> Finally I hired the good Mr. Chris Hansen to write a method for pulling the data using XML; as you might imagine, his solution works great, and it gets triggered everytime a sale occurs in the parent database, which is constant throughout the day.
>> 
>> So if you do go to a 2-database solution, where one is doing just web work, you may keep this in mind as something that will need to be done.
>> 
>> Of course, if pulling the web records can occur less frequently, you can either have someone do it as they work in the database, instead of requiring a solution like I used here...
>> 
>> Hope this helps,
>> 
>> Bob Patin
>> 
>> 
>> 
>> Longterm Solutions
>> bob at longtermsolutions.com
>> 615-333-6858
>> http://www.longtermsolutions.com
>> iChat: bobpatin
>> FileMaker 9 & 10 Certified Developer
>> Member of FileMaker Business Alliance and FileMaker TechNet
>> --
>> Expert FileMaker Consulting
>> FileMaker Hosting for all versions of FileMaker
>> PHP * Full email services * Free DNS hosting * Colocation * Consulting
>> On May 6, 2010, at 2:47 PM, Jonathan Schwartz wrote:
>> 
>> > Hey Joel,
>> >
>> > No PHP scripts calling FileMaker scripts.
>> >
>> > Since writing the original post...only minutes ago....my thinking has matured.  There is stuff going on in the  native solution (not under my control) that will continue to go on. I have now decided to isolate the web solution from it rather than attempt to fix the other side.
>> >
>> > So, the question now boils down to practical ways to isolate a web solution from a 2 machine deployment, where:
>> >     Machine1 = FMSserver 10 with MainDB and WebDB
>> >  Machine2 = FMSserver 10 Web Engine, serving WebDB
>> >
>> > This one just ocurred to me: Turn Machine 2 into a single machine deployment Web Publishing machine running the WEBDB. This will probably mean purchasing another copy of FMServer.
>> >
>> > Short of doing this, I was wondering if simply adding a new hard drive for the WebDB into machine 1 would be of any benefit.
>> >
>> > That's where I am right now.
>> >
>> > Jonathan
>> >
>> >
>> >
>> >
>> >
>> > At 12:12 PM -0700 5/6/10, Joel Shapiro wrote:
>> >> Hey Jonathan
>> >>
>> >> It seems that if your performance picks back up again in the evening, then it's probably not connected to slow record creation (I also had a recent web slowdown due to their FMP person having added auto-enter calc fields based on a bad relationship).
>> >>
>> >> Are those FM scripts connected to the PHP calls to the DB, or are they being run by users in FMP?  If they're called by the PHP, can you temporarily disable them to test whether or not they're the cause?
>> >>
>> >> HTH,
>> >> -Joel
>> >>
>> >> ~~~~~~~~~~~~~~~~~~~~~~~
>> >> Joel Shapiro
>> >> FileMaker Pro
>> >> database & web design
>> >> http://www.jsfmp.com
>> >> 415-269-5055
>> >> ~~~~~~~~~~~~~~~~~~~~~~~
>> >
>> > --
>> > Jonathan Schwartz
>> > Exit 445 Group
>> > jonathan at exit445.com
>> > http://www.exit445.com
>> > 415-370-5011
>> > _______________________________________________
>> > FX.php_List mailing list
>> > FX.php_List at mail.iviking.org
>> > http://www.iviking.org/mailman/listinfo/fx.php_list
>> Jonathan,
>> 
>> I'm late to the conversation, but have some input to share:
>> 
>> On the project I'm involved with now, we moved the database server into the museum for which it was designed. There's also a web app to go with it, and their IT guys have never been able to get the web server to work with the database server (you can imagine how impressed I am with these idiots).
>> 
>> So I finally created a 2nd database with only the tables that I use to sell tickets on the web, and it works fine. Here's the rub, and why I mention this here:
>> 
>> I couldn't find a way to synchronize these 2 databases; I wrote a script that would pull new orders into the "parent" db, but without opening the 2 databases on my machine (one on one FM server, the 2nd on another 700 miles away), I couldn't get the script to fire.
>> 
>> Finally I hired the good Mr. Chris Hansen to write a method for pulling the data using XML; as you might imagine, his solution works great, and it gets triggered everytime a sale occurs in the parent database, which is constant throughout the day.
>> 
>> So if you do go to a 2-database solution, where one is doing just web work, you may keep this in mind as something that will need to be done.
>> 
>> Of course, if pulling the web records can occur less frequently, you can either have someone do it as they work in the database, instead of requiring a solution like I used here...
>> 
>> Hope this helps,
>> 
>> Bob Patin
>> 
>> <new_logo_idea3_120w 32.jpg>
>> 
>> Longterm Solutions
>> bob at longtermsolutions.com
>> 615-333-6858
>> http://www.longtermsolutions.com
>> iChat: bobpatin
>> FileMaker 9 & 10 Certified Developer
>> Member of FileMaker Business Alliance and FileMaker TechNet
>> --
>> Expert FileMaker Consulting
>> FileMaker Hosting for all versions of FileMaker
>> PHP * Full email services * Free DNS hosting * Colocation * Consulting
>> On May 6, 2010, at 2:47 PM, Jonathan Schwartz wrote:
>> 
>>> Hey Joel,
>>> 
>>> No PHP scripts calling FileMaker scripts.
>>> 
>>> Since writing the original post...only minutes ago....my thinking has matured.  There is stuff going on in the  native solution (not under my control) that will continue to go on. I have now decided to isolate the web solution from it rather than attempt to fix the other side.
>>> 
>>> So, the question now boils down to practical ways to isolate a web solution from a 2 machine deployment, where:
>>>   Machine1 = FMSserver 10 with MainDB and WebDB
>>>    Machine2 = FMSserver 10 Web Engine, serving WebDB
>>> 
>>> This one just ocurred to me: Turn Machine 2 into a single machine deployment Web Publishing machine running the WEBDB. This will probably mean purchasing another copy of FMServer.
>>> 
>>> Short of doing this, I was wondering if simply adding a new hard drive for the WebDB into machine 1 would be of any benefit.
>>> 
>>> That's where I am right now.
>>> 
>>> Jonathan
>>> 
>>> 
>>> 
>>> 
>>> 
>>> At 12:12 PM -0700 5/6/10, Joel Shapiro wrote:
>>>> Hey Jonathan
>>>> 
>>>> It seems that if your performance picks back up again in the evening, then it's probably not connected to slow record creation (I also had a recent web slowdown due to their FMP person having added auto-enter calc fields based on a bad relationship).
>>>> 
>>>> Are those FM scripts connected to the PHP calls to the DB, or are they being run by users in FMP?  If they're called by the PHP, can you temporarily disable them to test whether or not they're the cause?
>>>> 
>>>> HTH,
>>>> -Joel
>>>> 
>>>> ~~~~~~~~~~~~~~~~~~~~~~~
>>>> Joel Shapiro
>>>> FileMaker Pro
>>>> database & web design
>>>> http://www.jsfmp.com
>>>> 415-269-5055
>>>> 
>>>> ~~~~~~~~~~~~~~~~~~~~~~~
>>> 
>>> --
>>> Jonathan Schwartz
>>> Exit 445 Group
>>> jonathan at exit445.com
>>> http://www.exit445.com
>>> 415-370-5011
>>> _______________________________________________
>>> FX.php_List mailing list
>>> FX.php_List at mail.iviking.org
>>> http://www.iviking.org/mailman/listinfo/fx.php_list
>> 
>> 
>> _______________________________________________
>> FX.php_List mailing list
>> FX.php_List at mail.iviking.org
>> http://www.iviking.org/mailman/listinfo/fx.php_list
> 
> 
> -- 
> Jonathan Schwartz
> Exit 445 Group
> jonathan at exit445.com
> http://www.exit445.com
> 415-370-5011
> 
> _______________________________________________
> FX.php_List mailing list
> FX.php_List at mail.iviking.org
> http://www.iviking.org/mailman/listinfo/fx.php_list
> 
> 
> _______________________________________________
> FX.php_List mailing list
> FX.php_List at mail.iviking.org
> http://www.iviking.org/mailman/listinfo/fx.php_list

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.iviking.org/pipermail/fx.php_list/attachments/20100510/032188a7/attachment.html

More information about the FX.php_List mailing list