Rotating custom logs. How?

A

Alexandr Kirilov

Hello all!

On my server started multiple ffmpeg processes, every process writing output from stderr and stdout to file with pipe. Something like this:

nohup ffmpeg -nostdin -nostats -hide_banner -y -loglevel debug -threads 0 -rtsp_transport tcp -skip_initial_bytes 70000000 -analyzeduration 10000000 -i rtsp://192.168.1.10:554 -reconnect 1 -reconnect_at_eof 1 -reconnect_streamed 1 -reconnect_on_network_error 1 [some other ffmpeg parameters] -b:a:3 500k -var_stream_map 'v:0,a:0,name:1080 v:1,a:1,name:720 v:2,a:2,name:480 v:3,a:3,name:320' 2>&1 &> /logs/192.168.1.10/ffmpeg.log &

How to make this logfile "/logs/192.168.1.10/ffmpeg.log" rotatable with cleaning old ones? Server might be handling about 600 - 1000 ffmpeg processes. What kind of options in FreeBSD to do it?
 
Have tried it. Not working for my case properly. My logs for each of process generating logbook in different directories, directories generates dynamically. Haven't found any solution to use dynamically on exact file without using global logrotate config.
 
Rotating /logs/*/*.log might actually work. But there's a chance ffmpeg keeps the log files opened, and might need a signal to release them and reopen a new log file. Or else ffmpeg will continue writing to the 'old' filehandles.
 
or you can pipe ffmpeg output to a shell script (tee like) that rotates the logs first
then make ffmpeg to run a for fixed time or input length and restart it
 
SirDice said:
Rotating /logs/*/*.log might actually work. But there's a chance ffmpeg keeps the log files opened, and might need a signal to release them and reopen a new log file. Or else ffmpeg will continue writing to the 'old' filehandles.
Click to expand...
The file opened by FFMPEG - global problem. The FFMPEG might be crashing. Tested it.
 
Do you happen to have Apache installed on that machine? For the longest time they included a rotatelogs executable. You can pipe output to it and it'll take care of rotation, without interrupting the sending process.
 
SirDice said:
Do you happen to have Apache installed on that machine? For the longest time they included a rotatelogs executable. You can pipe output to it and it'll take care of rotation, without interrupting the sending process.
Click to expand...
Apache isn't acceptable for me. My project written with Erlang. Erlang taking care of starting FFMPEG processes. There are option to handle it with Erlang VM, but it's expensive, require a lot of resources. Been trying to reduce it by using pure system solution. Just starting FFMPEG with piping it to something that rotating automatically. This surveillance server supposed to handle +600 cameras. And every camera has own unique log.
 
Code: 
#!/bin/sh
LOG=$1
N=${2:-10}
COUNT=${3:-1000000}
LIST=$(jot $N $(($N-1)) 0)
while true
do
dd  bs=$COUNT count=1   of=$LOG
[ $? -ne 0 ] && break
GOT=$(wc -c < $LOG)
[ "$GOT" -ne "$COUNT" ] && break
PREV=""
for num in $LIST
do
file=${LOG}.$num
[ -r "$file" -a "$PREV" != "" ] &&  mv "$file" "$PREV"
PREV=$file
done
mv "$LOG" "$file"
done
pipe ffmpeg thru this script
first arg is logfile, second is how many logs to keep (default 10), third log size (default 1000000)
had to use dd which kind of sucks because you get logs truncated in the middle of the line but all default line based utils like head, awk will read more and will cause holes in the output
(like head -1000) will read 1200 lines and discard 200 which wont show anywhere.
probably you can add read T echo "$T" >>$LOG after the wc test passes to get around broken lines
 
You must log in or register to reply here.
Back
Top