No newline in stream redirection

C

cootue

Hi everybody,
First of all I'm sorry for probably really trivial issue.
I have problem with curl not adding newlines to it's log file which makes it quite hard to read...
code looks like this:

Code: 
curl -o file.dat http://damain.com/file.dat 2> logfile.log &

I'm not sure where the problem lies, as on debian each message gets own line...

Could anybody advise me how to fix this?
 
-w is for displaying one message after download has been completed and I'd like to be logging progress in between as it's "infinite" file which download I'm stopping with kill.
 
If you are using a sh compatible shell try
Code: 
curl -o file.dat http://damain.com/file.dat [color=blue]> logfile.log 2>&1[/color] &
 
J65nko,
nope, result is same.

I noticed that in fact it's adding something at end of each line, it shows as ^M in editors.
vi, less, and mcedit don't recognize it as new line.
But nano does!
It's kind of workaround, but it'd be cool to know how to add "proper" newline there...
 
Actually the redirection command that I suggested redirects stderr (fd 2) to stdout (fd 1).
If it adds a CNTRL-M or "\r", you can use tr(1) to replace it with a Unix newline, a "\n'

Code: 
curl -o file.dat http://damain.com/file.dat 2>&1 | tr '\r' '\n' >logfile &

BTW are you aware that in FreeBSD fetch(1) and ftp(1) can do auto fetching too?
 
cootue said:
I noticed that in fact it's adding something at end of each line, it shows as ^M in editors.
Click to expand...
That's a "carriage return" (ascii 0x0D). The carriage return just sets the cursor back to the beginning of the line, linefeed actually advances one line. Similar to an old typewriter. Most *nix however only use "Line Feed" (ascii 0x0a) for a new line. Windows uses CR+LF.

As you've noticed, this pops up with 'progress' type output. What this actually does is print a line, do a carriage return, print next line, carriage return etc. This has the effect of the second print overwriting the first.
 
curl without '-o' argument outputs file to stdout, so "progress info" is in stderr stream, right?

Trick with tr leaves logfile empty, probably cause it lost data somewhere between streams or something, ehh.
I like java because you don't have to use magic to make things work.

At first I had wget in my script, but I didn't like using external variables (or how 'http_proxy' is called) for proxy support. I changed it to curl for its '-x' argument, but when it comes to logging this program seriously lacks key features. I think I'll try to find yet another alternative.

Thanks guys for helping me.
 
I've tried it, but curl outputting progress to stderr makes it too much for me.

I went back searching for right tool, checked lftp, aria2, fetch, pavuk, but none of those got what I need. Then I found that there's 'http_proxy' option in .wgetrc, and with '-e' it's all I'll ever need.

Over and out from contented wget user!
 
You must log in or register to reply here.
Back
Top