I really think
that there is no quicker path to crappy code then having to screen scrape data. I really do.
One of the things I have to do in my current job is to have software that connects to other sites (that are owned/used by our clients) and scrape data from those sites to pull into our database. It's somewhat a thankless task because it's ugly and awful and nobody really notices when it works but it can cause problems when it fails.
And the code. Well. I tried. But the fact is it's just ugly. There is some abstraction at least so that the parsing/scraping mess is an interface used by the sorting and loading code. But still...
And I'm not aided by the fact that some of these systems are truly dreadful from an user-interaction perspective. The kind of systems that rely on sessions for navigation thus resulting in unrecoverable errors should you hit back in your browser or attempt to see some data in any order but the one officially proscribed. I don't want to go off on a total rant here but it's 2010! Who writes web applications like that anymore? This isn't a new paradigm anymore guys, and frankly if your app can't handle a user hitting the back button on any page ever without complete collapse your app just plain out sucks.
I'll tell you, if it wasn't for fiddler I'd be totally, well, you know. (On an aside fiddler now supports HTTPS, I think it has for a little while now although it didn't used to and I hadn't used it before now, and while your browser will (rightly) complain about what is essentially a man-in-the-middle attack you can use it to fully trace HTTPS interactions)
|© 2008 Max Stocker|