[FX.php List] To data condition or not data condition. That is the question

[Jonathan Schwartz]

Guess I had to hear myself say it "outloud".  In years of 
development, this is the first time that I will not use data 
conditioning  due to the nature of the client's aggressive filtering 
process.

Had a little bit of separation anxiety to overcome. ;-)

Jonathan

At 10:41 AM -0600 12/6/09, Dale Bengston wrote:
>Hi Jonathan,
>
>I think you've answered your own question. It doesn't make sense to 
>condition data that's going to be filtered out to such an extent. I 
>can appreciate your desire to clean it up as much as possible as a 
>courtesy to the pairs of eyes doing the reviews.
>
>If this were my project, I'd probably stop at doing basic JavaScript 
>validation on emails and phone numbers and postal codes and the 
>like. You can also make your best attempt with on-screen 
>instructions to get people to type using proper case and syntax (for 
>what it's worth). Then the focus becomes managing the expectations 
>of the reviewers, and establishing editorial guidelines so the 
>humans are all following the same rules. Nothing worse than your 
>reviewers spending time re-editing records over and over because one 
>prefers St and another prefers St. and another writes out Street. 
>Having editorial guidelines starts with the up-front assumption that 
>the reviewers will be doing the data conditioning, which avoids 
>heartache when the site goes live.
>
>Dale
>
>On Dec 6, 2009, at 10:01 AM, Jonathan Schwartz wrote:
>
>>  You see, that is the reason not to even start with conditioning if 
>>95% the data is not going to flow upstream.  For the 5% that will 
>>flow upstream, only a small portion of those will need repair, so 
>>it boils down to not even starting.
>>
>>  I like your "Mc" tool below, but then we have all the other exceptions:
>>	deGalle
>>	d'Angello
>>	de la Cruz
>>	and on...
>>
>>  In this project, better not to data condition at all.  I have my answer.
>>
>>  Thx
>>
>>  Jonathan
>>
>>
>>
>>>  If you are to make proper() on names you would probably have to make
>>>  exceptions...
>>>
>>>  McDuck
>>>
>>>  case( left( $lastname; 2 ) = "mc" ; proper( left( $lastname; 2 ) ) &
>>>  proper( middle( $lastname; 3; -1 ) );
>>>  /* misc other exceptions */
>>>  proper( $lastname )
>>>  )
>>>
>>>  ggt
>>>
>>>  2009/12/5 Jonathan Schwartz <jschwartz at exit445.com>:
>>>>  Hi Folks,
>>>>
>>>>  To data condition or not data condition. That is the question
>>>>
>>>>  This is a philosophical question, I believe.
>>>>
>>>>  Up until now, most of my projects have dealt with collecting records where
>>>>  100% of the records would be used later in the client's FileMaker systems.
>>>>  It made sense to apply capitalization and phone number formatting and such
>>>>  to have the data comply with existing data standards.
>>>>
>>>>  In a new project, however, records are being submitted that will be stored
>>>>  in a remote database and undergo a series of reviews before potentially
>>>>  being accepted and copied to the main database.  Perhaps 5% will make it.
>>>>
>>>>  Because..
>>>>         - Data conditioning isn't a perfect process
>>>>         - It costs time to develop
>>>>         - It costs cycles to run
>>>>         - It "taints" the original data entry. (What would ee 
>>>>cummings think
>>>>  about having his name be capitalized?)
>>>>         - Only a small number of records will emerge from the process
>>>>
>>>>  ...does it makes sense to perform this step?
>>>>
>>>>  Also, is this a question best left to the client?
>>>>
>>>>  Would appreciate the insight of folks who have dealt with 
>>>>this....except Bob
>>>>  of course. ;-)
>>>>
>>>>  Jonathan
>>>>  --
>>>>  Jonathan Schwartz
>>>>  Exit 445 Group
>>>>  jonathan at exit445.com
>>>>  http://www.exit445.com
>>>>  415-370-5011
>>>>  _______________________________________________
>>>>  FX.php_List mailing list
>>>>  FX.php_List at mail.iviking.org
>>>>  http://www.iviking.org/mailman/listinfo/fx.php_list
>>>>
>>>  _______________________________________________
>>>  FX.php_List mailing list
>>>  FX.php_List at mail.iviking.org
>>>  http://www.iviking.org/mailman/listinfo/fx.php_list
>  >
>>
>>  --
>>  Jonathan Schwartz
>>  Exit 445 Group
>>  jonathan at exit445.com
>>  http://www.exit445.com
>>  415-370-5011
>>  _______________________________________________
>>  FX.php_List mailing list
>>  FX.php_List at mail.iviking.org
>>  http://www.iviking.org/mailman/listinfo/fx.php_list
>
>_______________________________________________
>FX.php_List mailing list
>FX.php_List at mail.iviking.org
>http://www.iviking.org/mailman/listinfo/fx.php_list


-- 
Jonathan Schwartz
Exit 445 Group
jonathan at exit445.com
http://www.exit445.com
415-370-5011