Monitor URLs for changes
keepaneyeon is configured with YAML. The following configuration will
download the URLs http://httpbin.org/html and
http://httpbin.org/bytes/128. It will store the results in S3.
storage:
- &aws !storage/s3
access_key: <...>
secret_access_key: <...>
path: 's3://bucket/path'
downloaders:
- &http !downloader/http {}
targets:
- !target
name: static
downloader: *http
url: 'http://httpbin.org/html'
store: *aws
- !target
name: dynamic
downloader: *http
url: 'http://httpbin.org/bytes/128'
store: *awsIf config.yaml contains the above configuration, you can execute the
download with:
keepaneyeon config.yaml