Shell Strange behaviour from named pipe / FIFO?

[nuclearpidgeon]

Hi all, hopefully someone can help me out with this...

I am looking into using a FIFO / named pipe on FreeBSD for getting some sensitive output out of a program that takes a file output argument. So far I've been having no luck, and I ran some simple testing in a shell and observed what seemed to be really odd behaviour:

In Shell 1:

mkfifo testfifo
tail -f testfifo

In Shell 2:

cat 'a' > testfifo

No output comes through in Shell 1. However, if I cat again....

In Shell 2:

`cat 'a' > testfifo

...two a lines come out through the tail in Shell 1 - i.e. the shell ends up looking like this:

[root@machine /home/swebb]# tail -f testfifo
a
a

Additionally, if I start a fresh tail -f and instead try piping in sysctl -a:
In Shell 1:

tail -f testfifo

In Shell 2:

sysctl -a > testfifo

...only the last 10 lines (of 14,636!) of the sysctl come out through tail.

Am I missing something here or is something busted?
The tests were all run on FreeBSD 13.2-RELEASE-p8 with a ZFS filesystem.

 

[Hiroo Ono]

In Shell 1:

mkfifo testfifo
tail -f testfifo

In Shell 2:

cat 'a' > testfifo

No output comes through in Shell 1. However, if I cat again....

cat 'a' means outputting the content of file 'a'. Did you mean echo 'a'?

In Shell 2:

`cat 'a' > testfifo

...two a lines come out through the tail in Shell 1 - i.e. the shell ends up looking like this:

[root@machine /home/swebb]# tail -f testfifo
a
a

Did a file named 'a' appeared?

Additionally, if I start a fresh tail -f and instead try piping in sysctl -a:
In Shell 1:

tail -f testfifo

In Shell 2:

sysctl -a > testfifo

...only the last 10 lines (of 14,636!) of the sysctl come out through tail.

Am I missing something here or is something busted?
The tests were all run on FreeBSD 13.2-RELEASE-p8 with a ZFS filesystem.

tail outputs the last 10 lines of what it first read, so it is normal behavior.
Try: $ echo 'a' > testfifo; sysctl -a > testfifo in shell 1 and tail -f testfifo in shell 2.

 

[zirias@]

In short, what you see is behavior of tail(1) unrelated to the fifo.

To make tail read from the very beginning, add a -c+0 flag. Why it doesn't immediately read a first line I don't know, this might be a bug in tail itself.

But why do you need tail at all here? With the -f flag, it will ignore EOF (which occurs when the writing end closes the fifo) and continue reading, but this is most likely not what you want in a script anyways, because, how would the script ever stop reading then? Just read from the fifo with an appropriate redirection in your script (or, depending on what exactly you're doing, use cat(1)).

 

[nuclearpidgeon]

cat 'a' means outputting the content of file 'a'. Did you mean echo 'a'?

Oh yes sorry I just mis-typed. My actual tests were with echo not cat

tail outputs the last 10 lines of what it first read, so it is normal behavior.

Oh yes of course... good catch thanks.

Using -c+0 makes all the sysctl output come through.
However the echo 'a' case still only sees output come through once echo is run twice. Strange...

I was mainly just using tail -f because it's what I'm used to from Linux for watching stuff coming through log files.

I've now run some testing with the program sending sensitive output. If I start a cat on the fifo before running the program in another shell, then the output value comes through as expected.

The real issue I'm facing I guess is that I am ultimately trying to wrangle everything from a perl script. If I make a FIFO then open it before running the other program, it seems the perl script blocks on the open() call (and thus line in the script that starts the other program further down never gets run).
The current test perl code is more or less as follows:

my $fifo_path = "/tmp/secret-out";
say "making fifo";
system "mkfifo", $fifo_path;
say "opening fifo";
open my $fifo_handle, '<', $fifo_path;

say "Executing secret wizard spell";
system "my-secret-prog", "-o", $fifo_path;
if ($? != 0) {
        say "my-secret-prog returned non-zero result code $?";
        exit $?;
}
say "reading output";
my $output = do { local($/); <$fifo_handle>; };
# my $result = read $fifo_handle, $output, 128;
# say "read $result bytes OK";
close $fifo_handle;
system "rm", $fifo_path;


say "done - got " . length($output) . " chars";
say "---";
print $output;
say "---";
say "exiting";

It seems I may have to do some fork/wait fiddling to get what I want here... (or threading if Perl can do that, but I shudder to think about what might happen there...)

(Thanks to both of you for your replies/help!)

 

[nuclearpidgeon]

Oh and I forgot to mention. I ran the same mkfifo/echo/tail -f test on a Linux system and the first 'a' came through the first time, as expected.

Weirdly, I also just ran the same tests on macOS 11.7.10 (which has a BSD mkfifo according to the man page...) and it behaved as expected (text came through each time).

 

[Hiroo Ono]

On stable/14-52d7bd8c4: Mon Feb 19 00:24:43 JST 2024, it seems to work as expected. So, it seems to have been fixed in between.

 

[zirias@]

I was mainly just using tail -f because it's what I'm used to from Linux for watching stuff coming through log files.

This has nothing to do with Linux. The feature tail -f provides is NOT exiting on EOF but instead keep watching the file and reading anything that's added. Of course, the usecase is "watching logfiles". It still makes little sense to ever use this in a script, because you'll have no way to know when to stop tail. You most certainly want to read whatever input exactly until there's an EOF.

However the echo 'a' case still only sees output come through once echo is run twice. Strange...

This is almost certainly a little glitch in FreeBSD's tail(1) itself, but IMHO not very relevant here.

In general, if I understand you correctly, all you want to do is read some output of some external command from some script. I wonder why you'd need a FIFO for that at all. Even if the command has no builtin way to write to stdout, you can always give it /dev/stdout as the file to write to. See this little example:

$ cat pipetest.sh
#!/bin/sh

# simulate some command writing to given file:
writesomething()
{
        test -z "$1" && return 1
        exec 3> "$1" # open output file
        echo >&3 foo
        echo >&3 bar frob
        echo >&3 baz
        exec 3>&- # close output file
        return 0
}

{
        read line1
        read line2
        read line3
} <<EOF
$(writesomething /dev/stdout)
EOF

echo line1: ${line1}
echo line2: ${line2}
echo line3: ${line3}

$ ./pipetest.sh
line1: foo
line2: bar frob
line3: baz
$