Multiple processes died instantly. Why?

A

Alexandr Kirilov

Hello all!

On my server using ffmpeg that is storing RTSP camera on tmpfs-disk After 3-4 hours of perfect working the ffmpeg processes just disappeared. In logs of ffmpg stream - nothing, even no notice about stopping it by FFMPG. Is there any way kind of tools or solution that allow me to investigate the processes lifetime and cause of killing it? Where to look in FreeBSD logging? Is there any extended mode of FreeBSD logging?

For now 4 fmmpeg processes stopped instantly at one exact time, even by microseconds without any reasonable cause.
 
SirDice said:
The obvious reason here seems to be running out of memory after writing several hours worth of streaming data to a memory disk. Processes need memory too.
Click to expand...
There are constant memory using, last 20 minutes only. Memory usage not increased. Every camera about 200Mb. There are 4 cameras only. tmpfs - 24Gb
Code: 
$ df
Filesystem  1K-blocks    Used     Avail Capacity  Mounted on
tmpfs        25165824       4  25165820     0%    /tmp
 
SirDice said:
The obvious reason here seems to be running out of memory after writing several hours worth of streaming data to a memory disk. Processes need memory too.
Click to expand...
How to setup tmpfs to avoid this kind of collisions? My /etc/fstab is
Code: 
# Device        Mountpoint      FStype  Options Dump    Pass#
/dev/ada0p3     /               ufs     rw      1       1
/dev/ada0p7     /home           ufs     rw      2       2
/dev/ada0p8     /space          ufs     rw      2       2
/dev/ada0p5     /tmp            ufs     rw      2       2
/dev/ada0p6     /usr            ufs     rw      2       2
/dev/ada0p4     /var            ufs     rw      2       2
/dev/ada0p1     none            swap    sw      0       0
tmpfs           /tmp  tmpfs   rw,mode=1777,size=24G 0       0
 
SirDice said:
Right, I figured they might be continuously storing the stream, that'll quickly fill up your memory.
Click to expand...
No they not. Just constant amount of data. That is why started asking about it. FFMPEG processes just died without any reasonable cause and 4 different streams from different cameras died instantly. Is there any way to monitor this processes to get the cause of dying?
 
i used to run a similar setup (rtsp to hls) and never had any problem (only when one of the cameras died) or the nvr rebooted or something (ffmpeg lost input)
 
SirDice said:
You could turn up the log level of ffmpeg(1) (verbose or even debug) and see if that provides any additional information.

Code: 
       -loglevel [flags+]loglevel | -v [flags+]loglevel
       Set logging level and flags used by the library.
ffmpeg(1).
Click to expand...
It's already working with loglevel. My FFMPEG command about 1000 symbols. There were just stopping without any messages.

Issue closed. You were right - it's about memory. There were mistakenly defined hls_delete_threshold parameter that been not cleaning outdated fragments. But!!! For me it's success - found the behaviour of the FreeBSD on early stage of development and really like it. It's stable and just stopped process and keep all data safe, but Ubuntu been restarting whole system and loosing everything in memory.

The solution:
-- keep logs at another location, defined special partition for the logs
-- checked FFMPEG hls_delete_threshold parameter
 
You must log in or register to reply here.
Back
Top