I came across an open source crawler that recently hit my site. I was wondering, 1. How do you get a list of sites to crawl? 2. Can you get a list of sites to crawl in your city? 3. If you have all this information, where is this readily available from and what APIs are there to access this information?