8000 GitHub - CSCfi/slurm-stats: Scripts for gathering SLURM statistics
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

CSCfi/slurm-stats

BranchesTags

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

12 Commits
srunning
srunning
 
 
README.md
README.md
 
 
helpers.R
helpers.R
 
 
sacct_stats.R
sacct_stats.R
 
 
sacct_stats_queue_dist.R
sacct_stats_queue_dist.R
 
 

Repository files navigation

slurm-stats

SLURM Stats are scripts for gathering SLURM statistics

Currently the scripts are

  • sacct_stats.R which generates simple user/monthly stats from sacct output
  • sacct_stats_queue_dist.R which generates a CSV with a binned distribution of how long jobs in each partition have queued
  • helpers.R helper functions to convert timestamps etc.

sacct_stats.R

Fetching data

To generate the data, you should use the following type of sacct command. You probably want to vary the start (and end) dates (-S and -E flags).

sacct --format JobID,JobIDRaw,JobName,User,Group,Partition,MaxRSS,MaxPages,AveCPU,MaxDiskWrite,MaxDiskRead,MaxVMSize,NTasks,AllocCPUS,Submit,Start,Elapsed,End,State,ExitCode,ReqMem,Timelimit -s BF,CA,CD,F,NF,PR,TO -P -a -S 08/15 > sisu

The example contains some extra fields which are not processed yet by the script but will likely be useful

Processing the data

  • Ensure that you have the data.table library installed in R
install.packages("data.table")
library(data.table)
  • Run the command and give the input file as an argument
R --no-save --args "taito-gpu" < sacct_stats.R
  • After the script completes you should have CSVs containing aggregations of per-month and per-user data
sisu_stats_per_user.csv
sisu_stats_per_month.csv
  • There are also some commented out lines at the end that generate other plots and statisics and can be used as basis for playing around with the data interactively

Interpreting the data

The resulting CSV files contain the following fields

  • User name or Date (Month/Year)
  • Count Number of jobs for the user or during the time period

For the following statistics, minimum, mean, maximum and standard deviation (min,mean,max,stddev) is calculated

  • AllocCPUS Allocated CPUs
  • QueueTime Time spent queued (in seconds)
  • Elapsed Time spent running (in seconds)
  • Timelimitaccuracy Difference of timelimit vs. actual runtime (Elapsed/Timelimit)

sacct_stats_queue_dist.R

Fetching data

You can use the same sacct output file as above but if you don't have it, the minimal set of data is the User, Partition, Submit and Start fields.

An example of the the very minimum feasible sacct command:

sacct --format User,Partition,Submit,Start -s BF,CA,CD,F,NF,PR,TO -P -a -S 08/15 > sisu

Processing the data

  • Ensure that you have the data.table library installed in R
install.packages("data.table")
library(data.table)
  • Run the command and give the input file as an argument
R --no-save --args "taito-gpu" < sacct_stats_queue_dist.R
  • After the script completes you should have a CSV containing a table with the suffix _out and the queue time distribution.
sisu_out.csv

About

Scripts for gathering SLURM statistics

Resources

Readme
Activity
Custom properties

Stars

22 stars

Watchers

37 watching

Forks

8 forks
Report repository

Releases

No releases published

Packages

No packages published

Contributors 2

  •  
  •  

Languages

0