Home     Show Politicians   Show Locations   Scrape your Own

Scraping: How to Do It

You, too, can contribute to the Frog Scraper Atlas by scraping your own favorite government page.

  1. Browse - use our embedded browser - just like in a real web browser - to find the page whose data you want to gather.
  2. Pick - point out the table on that page that has the data in it.
  3. Columns - tell us what kind of data is in which columns. There are also some editing controls.
  4. Edit - last-minute touch-up on data.
  5. Describe - give us more background information about which government these stuffed shirts are in, and what these people are called. Then we can start scraping.
  6. Finish - We commit the data to the database.

As it turns out, we have a tutorial that you can try. Just follow the directions in the red boxes like this one.

Ready to go?    




(for serious scrapers)

Have I told you that I'm looking for a job? I wrote this out of boredom between jobs. Actually by the time you read this I could have a job, check here. I've been learning Ruby on Rails off and on for years. This site is here to demonstrate the kind of work that I do. It is running in a virtual host with no Ruby support, but I got Rails working anyway. Click Here to see source. Click here to go see the rest of Allan's portfolio.     - Allan