run commands via a queue immediately

W

weberjn

When my door motion detector sends an event over MQTT, I want to read a camera and transform the image with imagemagick. It can happen that there is another motion event while the previous image is still being processed, but the new one should be process not until the previous one is finished.

So the naive idea is to use a queue
Bash: 
echo 'process_motion_event' | at -q a now

Unfortunately

Note that at is implemented through the cron(8) daemon by calling atrun(8) every five minutes. This implies that the granularity of at might not be optimal for every deployment. If a finer granularity is desired, the /etc/cron.d/at file can be edited
Click to expand...

Even the minimal one minute granularity is not good enough. Actually, I'd expect at now to run now.

Any ideas how to run queued jobs immediately?
 
A queue is certainly what you need here, but at(1)'s queues aren't "general purposed" but designed for scheduling work at a given time.

sysutils/nq looks like it could do what you need here (I didn't look in detail though).
 
weberjn said:
[…] It can happen that there is another motion event while the previous image is still being processed, but the new one should be process not until the previous one is finished.
Click to expand...
Out of curiosity, what is the reason concurrent processing of images is undesired?​
weberjn said:
[…] Actually, I'd expect at now to run now.

Any ideas how to run queued jobs immediately?
Click to expand...
Well, have you considered manually triggering atrun(8)? Right after you have added your job to the queue?​
Bash: 
at -q a now << EOT
process_motion_event
EOT
sudo /usr/libexec/atrun
There is also sysutils/ts. ts(8) says:​
ts by default offers a queue where each job runs only after the previous finished.
Click to expand...
 
Last edited:
How about implementing the functionality in a simple scripting language?

Create a named FIFO, in a well-known directory; I used /tmp/fifo. Then have the process that receive motion events deposit something in the fifo; that could be as easy as "echo hallo > /tmp/fifo". Then write a tiny script for receiving things from the fifo:
Code: 
while true
do
  echo Starting to wait for a task at `date`
  task=`cat /tmp/fifo`
  echo Received task $task at `date`
  do_whatever_it_takes_to_process $task
done

where "do_whatever_it_takes" is the program or script that handles an image retrieval / transform task. One of the nice things about this approach is that the program that receives events can actually put some meaningful information into the task name, as long as it fits in one line.
 
ralphbsz said:
How about implementing the functionality in a simple scripting language?
Click to expand...

I thought of that, but I'd rather avoid doing synchronization myself (and having to clean up fifos and locks if the script crashes), hence the idea with the queue and having the operating system doing the work.

The at implementation via cron is quite a bad hack, if at now +1 minute can happen in 5 minutes, this is not what a user expects, and this in a central area of the os.

One could very well implement an at daemon that reacts immediately and still doesn't burn cpu.
 
You must log in or register to reply here.
Back
Top