Home     Show Politicians   Show Locations   Scrape your Own

Quick Summary for Reviewers

This is a database of politicians and their offices and locations, plus screen scraper for getting data.

Welcome to the Frog Scraper Atlas Project

During the campaign, they always try to convince us that they are princes and princesses. But, soon after election day, it becomes apparent that they're all frogs.

Regardless, there's a lot of them.

This program is an attempt to catalog politicians, and the tracts of land that they rule over, by grabbing information from other websites.

Glossary
Frog
n. a politician. Male or female, democratic or dictator, village dog catcher or UN Secretary General.

District
n. a piece of land, or an organization, that the frog rules over, or that the office is associated with.

Example: John McCain is a senator from Arizona. So frog = "John McCain" and district = "Arizona, United States". You can view the frogs or view the districts on this site.

It's a little bit like a wiki, in that the data comes from the people who use it. You can Scrape your Own information, if you know a website that has some sort of table of frogs and the districts where they hold office.

Scraping means to take data from a running program, while it's running. Imagine being in a computer, the old kind with a big video tube display. Looking at the screen from behind, that is, as if you were inside the video tube, you take out a paint scraper, and start scraping the data right off the back of the screen, like it was old wallpaper. You save the old wallpaper scrapings in a file that now has the data that used to be displayed on the computer screen. With modern software tools, we've even managed to make it work on flat-panel displays.

All of the data on this site comes from its own scraping; very little hand-editing has been done. Currently there are 1972 frogs holding office 2387 times in 621 districts on this site.

NO
Frog_scraper
Yes, this is a 'frog scraper' as well, but completely different and unrelated to this 'frog scraper' project. After sliding the baton out of the frog's teeth, you scrape it along the frog's jagged back, and it makes a sound remarkably like frogs croaking in the swamp, due to the frog's hollow body. Cute. Fun for children. Hand-carved by third-world artists. Available at your local internet gift shop. Now forget about that, and go scrape us up some frogs. Election day will be here before you know it.

Goals

The Frog Scraper Atlas is an experiment in user interface, to see if it's possible to make a highly technical tool usable by regular people. Web page scraping is a somewhat esoteric art where a programmer makes highly specialized code to extract information from a human-targeted HTML document. Details and exceptions complicate the task until it is seemingly impossible for the faint of heart, and the resulting scraper is usable only on the site it's designed for, often breaking as a result of small changes to that site.

By making intelligent choices where possible, and asking the human when it isn't, this scraper achieves a balance between scraping power, target website generality, and usability. You don't need to know HTML, and various text esoterica has been smoothed over.

This site is also font-size flexible; you should be able to change the font size (+/-) with your browser (usually the View menu) and everything formats correctly. Unlike most sites.

See the Bug List.


Have I told you that I'm looking for a job? I wrote this out of boredom between jobs. Actually by the time you read this I could have a job, check here. I've been learning Ruby on Rails off and on for years. This site is here to demonstrate the kind of work that I do. It is running in a virtual host with no Ruby support, but I got Rails working anyway. Click Here to see source. Click here to go see the rest of Allan's portfolio.     - Allan