Queueit is a CLI interface tool which helps to integrate beanstalkd[1] queue service into my shell scripts. This could be usefull for the cloud processing or task paralelling.
[1] http://kr.github.com/beanstalkd/
I am also using redis-server in one of the projects so there are scripts which will do the same for redis backend. Please check redis/redis-queue script.
You can redefine variables in a such way:
export QUEUEIT_HOST=btd.local
export QUEUEIT_PORT=11300
export QUEUEIT_TIMEOUT=100
export QUEUEIT_TTR=5000
-
Run the beanstalkd server
beanstalkd -d
-
Push new tasks to process
q-put todownload "http://site.com/video1.avi" "http://video2.avi"
-
Run workers in the cloud nodes
while true; do
url=$(q-get todownload);
wget -c $url -O $$.out
q-put toencode $$.out
done
while true; do
in_file=$(q-get toencode)
HandbrakeCLI -i $in_file -o ${in_file%.avi}.mp4
one
-
Run beanstalkd queue server:
beanstalkd -d
-
Push new task
q-put download "http://site.com/video1.avi"
-
Run q-wrapper to process the tasks
q-wrapper download encode wget {}
q-stat shows nubmer of ready and buried tasks per queue
tube watching buried ready
default 24 +1 0 0
t-start 3 124 570 -3
stats 1 3 0
t-ir 1 6 67 +2
t-ip 1 6 0
t-upl 0 0 9907
t-t350 1 0 0
t-t1350 1 0 0
