FOIA Online Requests
Item Preview
There Is No Preview Available For This Item
This item does not appear to have any files that can be experienced on Archive.org.
Show all files
- Usage
- Public Domain Mark 1.0
This item contains a utility that was used for collecting data from foiaonline.gov which was shut down on September 30, 2023. The item also includes the collected data. Despite the fact that the website came online in 2012, searching in the interface found that there were records received going back as far as March 4, 2003.
The pull.py program interacted with the foiaonline.gov Advanced Search Form by requesting one week's worth of records at a time, starting on March 1, 2003. This was to get around a result size limit of 10,000 records. It retrieved the JSON from the API that was provided for paging through results. The records retrieved are then written to a JSONL file data.jsonl, which was gzipped on completion, and is present here as well.
The included Jupyter notebook provides an example of how to use the collected data. See https://wiki.archiveteam.org/index.php/FOIAonline and https://archive.org/details/archiveteam-fire?query=foiaonline.gov for more information about efforts to archive the web content itself.
I also wrote a blog post about this, which you can find at: https://inkdroid.org/2023/10/01/foiaonline/
Reviews cannot be added to this item.