Description
Having to schedule terminate_idle_job_flows locally is brittle. It would be nice if this could be automated somehow.
EDIT: tried the below, and it didn't work; see comments
It looks like the CloudWatch API:
[http://docs.aws.amazon.com/AmazonCloudWatch/latest/DeveloperGuide/UsingAlarmActions.html]
supports terminating job flows by idle time (using the IsIdle metric for EMR):
[http://docs.aws.amazon.com/ElasticMapReduce/latest/DeveloperGuide/UsingEMR_ViewingMetrics.html#UsingEMR_ViewingMetrics_MetricsReported]
I can't see a natural way to get the behavior "terminate the flow if it's idle and <5 minutes away from the end of the billing hour", but there might be one. Is it possible that we could set something like this up on the cluster itself as a cron job (via a bootstrap action)?