Solved tail -f file | grep search term not working on FreeBSD

reddy

reddy

I am trying to track the changes of a file and filter the output using grep, but the below command:

Code: 
tail -f /var/log/file.log | grep search_term

Gives no output. It keeps waiting for updates, but nothing is displayed even though when I then tail or cat I can see that updates that should have been displayed weren't displayed.
 
Okay, so actually the problem only occurs when I pipe additional greps:

Code: 
tail -f /var/log/file.log | grep search_term | grep filter2 | grep filter3

I am pretty sure my filters are valid since the output is right if I cat or tail without the -f option.

Edit: the solution is to use --line-buffered on the greps in the middle:

Code: 
tail -f /var/log/file.log | grep --line-buffered search_term | grep --line-buffered filter2 | grep filter3
 
I'm getting similar issue, when using grep, nothing happens...

screenshot.jpg



Entering:
grep .idea/workspace.xml | grep --line-buffered bindarknet | grep --line-buffered configuration

Doesn't do anything either...

I think Linux's grep is different than FreeBSD's grep.
 
It is quite possible that by the time you do "tail -f ... | grep ...", the last 10 lines (which is what tail shows by default) don't have the grep pattern. Try something like "tail -1000 -f ... | grep ...". Or if you want to start from the very beginning but continue watching, try "tail +1 -f ... | grep ..."
 
First_Law_of_Unix said:
Entering:
grep .idea/workspace.xml | grep --line-buffered bindarknet | grep --line-buffered configuration
Click to expand...
Read the documentation. The first argument to grep is the search pattern. All the remaining ones are input files. For example, if you are searching for elephants in a file called foo, you will type "grep elephant foo". If you give no input files (they are optional, see the documentation), grep will search stdin instead. That's common for most commands.

Since the first grep here has only one argument, what it really does: It search stdin for the search pattern ".idea/workspace.xml". If you actually typed something into the console now, then grep would search what you type. If you were to type ".idea/workspace.xml" into the console, it would get echoed by grep.

I think what you really mean: The first command should be cat. Or maybe you are just missing the first search term, and you really mean "grep elephant .idea/workspace.xml | ...".

Also, in normal usage, the "--line-buffered" switch is not needed, and it can be performance killing. It is only needed if you are trying to read the output in real time (like when doing grep on "tail -f ..."), which is not what's going to be happening here.
 
reddy said:
Edit: the solution is to use --line-buffered on the greps in the middle:
Click to expand...
put it on the last one too, save any future head scratching
Code: 
tail -f /var/log/file.log | grep --line-buffered search_term | grep --line-buffered filter2 | grep --line-buffered filter3
also, learn some basic egrep stuff you'll wish you knew it earlier (like covacat mentioned)
 
Thanks everyone for their advice. The grep command which I used was actually provided by a professional Software Architect.

I assumed the command he provided should work on FreeBSD:
grep .idea/workspace.xml | grep bindarknet | grep configuration

However it did not and I did a google and landed to this thread.

I then checked the grep man page and it's completely different on how it's syntaxes are used than what I was provided.
I ended up using the command below, which did the trick:
grep 'text_search' file.txt

So either the original grep command I was provided is either specific for Linux or MacOS since I know he uses both.

Thanks.
 
First_Law_of_Unix said:
I then checked the grep man page and it's completely different on how it's syntaxes are used than what I was provided.
Click to expand...
Note the differences between FreeBSD's grep(1) and textproc/gnugrep (which is what most Linux distributions use).

You're going to find subtle (and sometimes not so subtle) differences between BSD's implementation of various tools and their GNU counterparts.
 
I observe the behavior change of the "tail -F" itself recently.
In my application, I save the pid of a daemon process to a "pidfile", and each time the daemon is restarted, the "pidfile" is truncated and updated with the new pid. However, with the recent BSD merge, I found the "tail -F" doesn't reflect the new pid when the daemon is restarted, which fails my existing test cases.

I looked up the git commit history of usr.bin/tail/tail.c, and see there're some recent changes, e.g., fileargs_init() and fileargs_fopen() are introduced and freopen(3) was replaced with fileargs_open(3) and fclose(3). But I don't see the cause of the “tail -F" behavior change.
 
cchen said:
I observe the behavior change of the "tail -F" itself recently.
In my application, I save the pid of a daemon process to a "pidfile", and each time the daemon is restarted, the "pidfile" is truncated and updated with the new pid. However, with the recent BSD merge, I found the "tail -F" doesn't reflect the new pid when the daemon is restarted, which fails my existing test cases.

I looked up the git commit history of usr.bin/tail/tail.c, and see there're some recent changes, e.g., fileargs_init() and fileargs_fopen() are introduced and freopen(3) was replaced with fileargs_open(3) and fclose(3). But I don't see the cause of the “tail -F" behavior change.
Click to expand...
Yeah, certainly looks broken with a simple:
Code: 
$ rm pid
$ tail -F pid
...another terminal...
$ echo 123 > pid
$ echo 456 > pid
$ echo 789 > pid
...getting only 123 displayed, and correctly showing 123 456 789 lines on other OSes.
 
You must log in or register to reply here.
Back
Top