Solved Removing ANSI characters from log

I assume by “ANSI characters” you mean control sequences used by the shell, editors and similar, right?

That's quite a difficult thing to do by software, because in order to remove them, they have to be interpreted – that is, the software has to implement a terminal renderer. The ansi2txt tool does exactly that, but it only supports a subset of VT100 control sequences. It probably won't work with other kinds of emulated terminals.

Depending on your actual use case, there are a few alternatives. For example, you can set the TERM environment variable to dumb inside your script session. Then your shell (and other programs) won't use any control sequences at all. However, in this case you cannot use screen-oriented programs like editors (ed(1) will work, though), and the editing functions of your shell will be very limited. Also, if your shell prompt does fancy things (like using colors or bold font), you'll have to reset it to something simpler.
 
I assume by “ANSI characters” you mean control sequences used by the shell, editors and similar, right?
That's exactly what I'm trying to do.
That's quite a difficult thing to do by software, because in order to remove them, they have to be interpreted – that is, the software has to implement a terminal renderer. The ansi2txt tool does exactly that, but it only supports a subset of VT100 control sequences. It probably won't work with other kinds of emulated terminals.

Depending on your actual use case, there are a few alternatives. For example, you can set the TERM environment variable to dumb inside your script session. Then your shell (and other programs) won't use any control sequences at all. However, in this case you cannot use screen-oriented programs like editors (ed(1) will work, though), and the editing functions of your shell will be very limited. Also, if your shell prompt does fancy things (like using colors or bold font), you'll have to reset it to something simpler.

I did try using sed() but couldn't figure things out...

The TERM=dumb idea sounds useful, though not sure how to use it... Do I run env TERM=dumb before starting script?
 
In case your notion of control chars is just ASCII code < 32 (the space char), then you could use the following sh; sed sequence for replacing it with a single space character. Let's assume you want to convert the contents in file test.txt.
sh -- this is needed, because AFAIK the tcsh shell does not allow for specifying control chars by octal or hex code
sed -e "s|["$'\x01'"-"$'\x1F'"]| |g" -i ".back" test.txt
exit -- to the normal interactive shell, presumably tcsh
 
The TERM=dumb idea sounds useful, though not sure how to use it... Do I run env TERM=dumb before starting script?
It has to be a single command: env TERM=dumb script
Note that there are programs that ignore the terminal setting and use hard-coded control sequences nonetheless. Also, as alrady mentioned, you might want to reset the shell prompt. For bourne-compatible shells (sh, zsh, bash), it can be done with export PS1='> ', for example.
 
After being unable to find a solution to extracting ANSI escape sequences from logfiles, I thought I'd see if the problem was due to using miniterm. After replacing miniterm on Linux with cu on FreeBSD I no longer have this problem.
 
Back
Top