Solved tar errors, when file content extracted to STDOUT is piped

Seeker

Aspiring Daemon

Reaction score: 17
Messages: 865

Extracting one file to stdout passes without errors. tar -xOf src.txz usr/src/UPDATING But when I append a pipe to the above command to get part of the text I get what I wanted and an error:

Rich (BB code):
 tar -xOf src.txz usr/src/UPDATING | head
./usr/src/UPDATING: Write error
tar: Error exit delayed from previous errors.
It returns exit code 1.

This started to happen after a migration from 9.* to 10.*.
 

ShelLuser

Son of Beastie

Reaction score: 1,715
Messages: 3,537

I don't know if it helps but I'm missing an option up there: the compression being used. Commonly used options are either gzip (using -z) or bzip2 (using -j) but these "distribution files" (I don't know the official name) are compressed as well: using XZ. You'd need -J for that. So I'd use $ tar xJf src.txz.
 

DutchDaemon

Administrator
Staff member
Administrator
Moderator
Developer

Reaction score: 2,789
Messages: 11,281

tar(1) recognizes the major compression types automatically, so defining -j, -z, etc. is no longer a necessity nowadays.

Code:
-J, --xz
(c mode only) Compress the resulting archive with xz(1). In
extract or list modes, this option is ignored. Note that, unlike
other tar implementations, this implementation recognizes XZ com-
pression automatically when reading archives.

-j, --bzip, --bzip2, --bunzip2
(c mode only) Compress the resulting archive with bzip2(1). In
extract or list modes, this option is ignored. Note that, unlike
other tar implementations, this implementation recognizes bzip2
compression automatically when reading archives.
 

ShelLuser

Son of Beastie

Reaction score: 1,715
Messages: 3,537

DutchDaemon; You're fully right there, but the thing is; when it comes to debugging I tend not to rely on automated procedures in order to rule out as much as possible. Still, in this case the inclusion of those options turned out to be useless indeed.

Things like these intrigue me so I tried to reproduce, and I can. I get the impression that this might be a timing related issue. ktrace shows me this on FreeBSD 10 (running in VirtualBox):
Code:
(skip header (reading libs))
   933 bsdtar   CALL  open(0x28c34054,0x100000<O_CLOEXEC>,<unused>0x2805c6e1)
   933 bsdtar   NAMI  "src.txz"
   933 bsdtar   RET   open 3
   933 bsdtar   CALL  fcntl(0x3,F_GETFD,<invalid>0x3)
   933 bsdtar   RET   fcntl 1
   933 bsdtar   CALL  fstat(0x3,0xbfbfda20)
   933 bsdtar   STRU  struct stat {dev=72, ino=33073, mode=0100644, nlink=1, uid
=0, gid=0, rdev=71064, atime=1414070063, stime=1414069395, ctime=1414069395, birthtime=1414069296, size=112364972, blksize=32768, blocks=219584, flags=0x0 }
   933 bsdtar   RET   fstat 0
   933 bsdtar   CALL  read(0x3,0x28c8e000,0x10000)
   933 bsdtar   GIO   fd 3 read 4096 bytes
(skip data dump)
   933 bsdtar   RET   read 65536/0x10000
   933 bsdtar   CALL  mmap(0,0x800000,0x3<PROT_READ|PROT_WRITE>,0x1002<MAP_PRIVATE|MAP_ANON>,0xffffffff,0,0)
   933 bsdtar   RET   mmap 687865856/0x29000000
   933 bsdtar   CALL  clock_gettime(0xd,0xbfbfda90)
   933 bsdtar   RET   clock_gettime 0
   933 bsdtar   CALL  umask(0<><invalid>0)
   933 bsdtar   RET   umask 18/0x12
   933 bsdtar   CALL  umask(0x12<S_IWGRP|S_IWOTH>)
   933 bsdtar   RET   umask 0
   933 bsdtar   CALL  geteuid
   933 bsdtar   RET   geteuid 0
   933 bsdtar   CALL  fstat(0x1,0xbfbfda78)
   933 bsdtar   STRU  struct stat {dev=58, ino=4, mode=010000, nlink=0, uid=0, gid=0, rdev=0, atime=1414071152, stime=1414071152, ctime=1414071152, birthtime=0, size=0, blksize=4096, blocks=0, flags=0x0 }
   933 bsdtar   RET   fstat 0
   933 bsdtar   CALL  write(0x1,0x28ca7000,0x9000)
   933 bsdtar   GIO   fd 1 wrote 4096 bytes
       "Updating Information for FreeBSD current users

        This file is maintained and copyrighted by M. Warner Losh <imp@freebsd\
        .org>.
(skip rest of file)
So far, so good.

The second block of 4096 bytes is also read, but then this happens:
Code:
   933 bsdtar   RET   read 65536/0x10000
   933 bsdtar   CALL  write(0x1,0x28ca0000,0xbad5)
   933 bsdtar   RET   write -1 errno 32 Broken pipe
   933 bsdtar   CALL  write(0x2,0xbfbfd8e0,0x10)
   933 bsdtar   GIO   fd 2 wrote 16 bytes
       "usr/src/UPDATING"
   933 bsdtar   RET   write 16/0x10
   933 bsdtar   CALL  write(0x2,0xbfbfd8e0,0xd)
   933 bsdtar   GIO   fd 2 wrote 13 bytes
       ": Write error"
   933 bsdtar   RET   write 13/0xd
   933 bsdtar   CALL  write(0x2,0x2842fd8f,0x1)
   933 bsdtar   GIO   fd 2 wrote 1 byte
       "
       "
   933 bsdtar   RET   write 1
At first it seems to me as if head isn't capable to process the incoming data. However, when I started experimenting a bit more I noticed that using less produces the same error messages, with the exception that the file also gets displayed.

So; instead of head you use less, you'll see the first page of the file appear and then press q to quit the pager. You'll get the same write error.

This is as far as my expertise on this goes, but as said; I get the impression that it might be timing related. For some reason the original program (tar in this case) doesn't receive a signal that the pipe has (already) been cut by an (sometimes pre-mature) exit.
 
Last edited:
OP
OP
Seeker

Seeker

Aspiring Daemon

Reaction score: 17
Messages: 865

Even in its cleanest form it hangs: tar -xOf src.txz usr/src/UPDATING output the complete content immediately to stdout and then hung for 56 seconds!

To cut it short, this is a non-critical bug fixed in 9.*, a.k.a. tar regression.
 
Last edited by a moderator:
OP
OP
Seeker

Seeker

Aspiring Daemon

Reaction score: 17
Messages: 865

It's solved! Look at freebsd-hackers mailing list with the subject: "tar behavior 9.* -> 10.*".
 
Top