All freezes during extraction and checksum stages of port's build

Y

YuryG

It's rather annoying you can do nothing during extraction and checksum checking stages of a port build. The bigger downloaded file, the longer it takes (and it's done in a single CPU thread, so it can be long). For example, huge www/chromium take several minutes on these stages on my system. System freezes and for some seconds even mouse pointer is not reacting, windows switching in GNOME/ MATE is almost unresponsive, not to say about other activities.
May be some system options? Or is it a known problem?
 
You can force a higher or lower scheduling priority. There's two things you can do here. First you can add kern.sched.preempt_thresh=224 to /etc/sysctl.conf if you don't have that set already. This should improve the desktop responsiveness quite a bit while compiling ports. If this isn't enough, you can also use the nice(1) utility to run a port build with a lower scheduling priority( e.g. # cd /usr/ports/www/chromium && nice -n 10 make install clean).
 
Good catch! I tuned the scheduler(9) to avoid this common freeze and give more responsiveness on desktop.

Indeed, I think kern.sched.preempt_thresh is too low, default is only 80.
 
Thanx, I'll try. But why it is a noticeable problem only during extract–checksum stage?
 
What kind of system do you have? Even my lowly Raspberry Pi doesn't exhibit this behavior during extraction. Yes, the CPU load goes up, as does I/O. But the system is still responsive.
 
The biggest CPU hogs if they can be called that during extract and checksum phases are xz(1) decompression and sha256(1) checksumming. You don't happen to be using the VESA driver with Xorg? Otherwise I have hard time imagining that those would freeze your graphical desktop no matter how low end system you have. Of course if your system is low on available memory from running applications such as the web browser then it's a different matter.
 
SirDice said:
What kind of system do you have? Even my lowly Raspberry Pi doesn't exhibit this behavior during extraction. Yes, the CPU load goes up, as does I/O. But the system is still responsive.
Click to expand...
I have AMD FX-8300 CPU with 16GiB of RAM with not too fast traditional HDDs. And during extract–checksum for big files I see only no more than 6–15% of one core usage by bsdtar, but this freeze occurred. And thanks to kern.sched.preempt_thresh concerning advice, now it's not that obvious (but still noticeable).
 
YuryG said:
Thanx, I'll try. But why it is a noticeable problem only during extract–checksum stage?
Click to expand...

Code: 
Thu, 27 Aug 2015, 10:21:27                                                    
<FreeBSD 10.2-RELEASE-p2> [bsd:/usr/ports/www/chromium]                        
cjpm@pts/0 # time make extract

To build Chromium, you should have around 2 GB of memory
and a fair amount of free diskspace (~ 3.7GB).

Make sure you have Python build with the SEM option ON
(default in python27-2.7.8 since r361735)
===>  License BSD3CLAUSE LGPL21 MPL accepted by the user
===>  Found saved configuration for chromium-44.0.2403.155
===>   chromium-44.0.2403.155 depends on file: /usr/local/sbin/pkg - found
===> Fetching all distfiles required by chromium-44.0.2403.155 for building
===>  Extracting for chromium-44.0.2403.155
=> SHA256 Checksum OK for chromium-44.0.2403.155.tar.xz.
40.735u 10.628s 1:22.07 62.5%    159+174k 21150+6886io 16pf+0w

Thu, 27 Aug 2015, 10:22:53
<FreeBSD 10.2-RELEASE-p2> [bsd:/usr/ports/www/chromium]
Note that xz(1) is more efficient in terms of compression ratio. So it saves disk space and optimizes the transfer through the network. You can see this Quick Benchmark so as to discover the difference by practical tests.
 
Code: 
time make patch

To build Chromium, you should have around 2 GB of memory
and a fair amount of free diskspace (~ 3.7GB).

Make sure you have Python build with the SEM option ON
(default in python27-2.7.8 since r361735)
===>  License BSD3CLAUSE LGPL21 MPL accepted by the user
===>  Found saved configuration for chromium-44.0.2403.155
===>  chromium-44.0.2403.155 depends on file: /usr/local/sbin/pkg - found
===> Fetching all distfiles required by chromium-44.0.2403.155 for building
===>  Extracting for chromium-44.0.2403.155
=> SHA256 Checksum OK for chromium-44.0.2403.155.tar.xz.
===>  Patching for chromium-44.0.2403.155
===>  Applying extra patch /usr/ports/www/chromium/files/extra-patch-gcc
===>  Applying FreeBSD patches for chromium-44.0.2403.155
27.733u 7.955s 5:37.23 10.5%   166+178k 12245+6987io 161pf+0w
5:37.23
 
This kind of freeze occurred on my system when I was using UFS filesystem. I haven't anymore freeze with ZFS.
To avoid this problem when using UFS on my desktop computer, I have the habit to launch compilation of ports with idprio(1) like this :
# idprio 31 make install clean
 
Yes, it's annoying that it consumes a lot of amount time to extract the source tarball.
Code: 
# time idprio 0 make extract

To build Chromium, you should have around 2 GB of memory
and a fair amount of free diskspace (~ 3.7GB).

Make sure you have Python build with the SEM option ON
(default in python27-2.7.8 since r361735)
===>  License BSD3CLAUSE LGPL21 MPL accepted by the user
===>  Found saved configuration for chromium-44.0.2403.155
===>   chromium-44.0.2403.155 depends on file: /usr/local/sbin/pkg - found
===> Fetching all distfiles required by chromium-44.0.2403.155 for building
===>  Extracting for chromium-44.0.2403.155
=> SHA256 Checksum OK for chromium-44.0.2403.155.tar.xz.
40.935u 11.404s 1:17.17 67.8%    198+174k 19042+6778io 11pf+0w
It seems that using idprio(1) improved substantially the extraction time.
 
dlegrand said:
...To avoid this problem when using UFS on my desktop computer, I have the habit to launch compilation of ports with idprio(1) like this :
# idprio 31 make install clean
Click to expand...
Well, for me it's a bad advice. Yes, freezing is almost gone, but extraction takes ages!
Code: 
/usr/ports/www/chromium # time idprio 0 make extract

To build Chromium, you should have around 2 GB of memory
and a fair amount of free diskspace (~ 3.7GB).

Make sure you have Python build with the SEM option ON
(default in python27-2.7.8 since r361735)
===>  License BSD3CLAUSE LGPL21 MPL accepted by the user
===>  Found saved configuration for chromium-44.0.2403.155
===>  chromium-44.0.2403.155 depends on file: /usr/local/sbin/pkg - found
===> Fetching all distfiles required by chromium-44.0.2403.155 for building
===>  Extracting for chromium-44.0.2403.155
=> SHA256 Checksum OK for chromium-44.0.2403.155.tar.xz.
27.688u 7.699s 5:30.79 10.6%   159+180k 15276+6968io 630pf+0w

/usr/ports/www/chromium # time idprio 0 make clean
===>  Cleaning for chromium-44.0.2403.155
2.246u 3.121s 0:32.37 16.5%   1881+312k 301+11108io 171pf+0w

/usr/ports/www/chromium # time idprio 31 make extract
To build Chromium, you should have around 2 GB of memory
and a fair amount of free diskspace (~ 3.7GB).

Make sure you have Python build with the SEM option ON
(default in python27-2.7.8 since r361735)
===>  License BSD3CLAUSE LGPL21 MPL accepted by the user
===>  Found saved configuration for chromium-44.0.2403.155
===>  chromium-44.0.2403.155 depends on file: /usr/local/sbin/pkg - found
===> Fetching all distfiles required by chromium-44.0.2403.155 for building
===>  Extracting for chromium-44.0.2403.155
=> SHA256 Checksum OK for chromium-44.0.2403.155.tar.xz.
27.729u 8.774s 21:57.91 2.7%   166+178k 20588+6978io 25pf+0w
Almost 22 min! 4 times longer than usual.
 
YuryG said:
Well, for me it's a bad advice. Yes, freezing is almost gone, but extraction takes ages!
Code:
Click to expand...

I tried some extraction on my file server with UFS filesystem and it takes less than twice the time to extract when using idprio 31.
This computer used for the test have a 2 cores Intel cpu and sata HD:
Code: 
Intel(R) Celeron(R) CPU G1610 @ 2.60GHz (2594.17-MHz K8-class CPU)
Intel Panther Point AHCI SATA controller
ada0: <ST2000DM001-1CH164 CC27> ATA-9 SATA 3.x device
Code: 
/usr/ports/www/chromium # time make extract
To build Chromium, you should have around 2 GB of memory
and a fair amount of free diskspace (~ 3.7GB).

Make sure you have Python build with the SEM option ON
(default in python27-2.7.8 since r361735)
===>  License BSD3CLAUSE LGPL21 MPL accepted by the user
===>  Found saved configuration for chromium-44.0.2403.155
===>  chromium-44.0.2403.155 depends on file: /usr/local/sbin/pkg - found
===> Fetching all distfiles required by chromium-44.0.2403.155 for building
===>  Extracting for chromium-44.0.2403.155
=> SHA256 Checksum OK for chromium-44.0.2403.155.tar.xz.
32.762u 8.829s 2:25.69 28.5%  333+175k 16841+6805io 72pf+0w

/usr/ports/www/chromium # time make clean
===>  Cleaning for chromium-44.0.2403.155
1.971u 2.988s 0:13.09 37.8%  4443+252k 196+3832io 462pf+0w

/usr/ports/www/chromium # time idprio 31 make extract
To build Chromium, you should have around 2 GB of memory
and a fair amount of free diskspace (~ 3.7GB).

Make sure you have Python build with the SEM option ON
(default in python27-2.7.8 since r361735)
===>  License BSD3CLAUSE LGPL21 MPL accepted by the user
===>  Found saved configuration for chromium-44.0.2403.155
===>  chromium-44.0.2403.155 depends on file: /usr/local/sbin/pkg - found
===> Fetching all distfiles required by chromium-44.0.2403.155 for building
===>  Extracting for chromium-44.0.2403.155
=> SHA256 Checksum OK for chromium-44.0.2403.155.tar.xz.
34.192u 15.653s 4:18.32 19.2%  333+174k 18561+6783io 10pf+0w

My desktop computer is an AMD 2 cores cpu :
Code: 
AMD Athlon(tm) II X2 270 Processor (3415.40-MHz K8-class CPU)
And it is more responsive since I am using ZFS file system. Furthermore I am using my file server to build ports with poudriere, so UFS high throughput doesn't hurt anymore.
 
As I see, there's something to do with my filesystem.
Code: 
ahci0: <AMD SB7x0/SB8x0/SB9x0 AHCI SATA controller>
ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada1: <ST3160813AS CC2H> ATA8-ACS SATA 2.x device
ada1: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada2: <WDC WD5000AADS-00S9B0 01.00A01> ATA8-ACS SATA 2.x device
ada2: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
Where to look for? Or is there anything to expect?
 
As wblock@ mentioned in Thread 52970, using a non-default compiler can cause mysterious issues. Possibly, this is one of those issues. Disk I/O by itself shouldn't freeze your entire desktop so the problem lies somewhere else. If you were pegging the CPU at 100% on all 8 cores that could freeze the desktop but as you noted above, that isn't the case. You have 16GB of RAM installed so I find it hard to believe that could be part of the problem. Maybe try rebuilding your system and ports with the default compiler settings if you haven't already and see if that makes the issue go away.
 
You can try this to discard the disk io subsystem on write and see what data rate you can expect extracting the data :

# time sh -c 'xzcat /usr/ports/distfiles/chromium-44.0.2403.155.tar.xz | tar xf - | cat > /dev/null'

If data rate is good, then you can investigate why the disk io rate is slow (I/O errors, disk almost full, bad sectors, sata cable defect, ...)

EDIT: please, test raw read data rate before with :
# dd if=/usr/ports/distfiles/chromium-44.0.2403.155.tar.xz of=/dev/null bs=1m
Please, forget the command on top, because some data is written down to disk with it.
 
I made a test with tmpfs(5). I replaced /tmp mount point with a tmpfs in my /etc/fstab :
Code: 
tmpfs  /tmp  tmpfs  rw,mode=1777  0  0
Results :
Code: 
# cd /tmp
# time sh -c 'xzcat /usr/ports/distfiles/chromium-44.0.2403.155.tar.xz | tar xf - '
29.822u 4.565s 0:30.25 113.6%  74+188k 0+0io 0pf+0w
 
Shame on me !

I just seen that tar(1) is supporting .xz compressed files. Now it's far better :
Code: 
# time tar xf /usr/ports/distfiles/chromium-44.0.2403.155.tar.xz --to-stdout > /dev/null
28.964u 0.251s 0:29.22 99.9%  66+172k 0+0io 0pf+0w

It's laborious. Please excuse my numerous posts.
 
protocelt said:
As wblock@ mentioned in Thread 52970, using a non-default compiler can cause mysterious issues. Possibly, this is one of those issues. Disk I/O by itself shouldn't freeze your entire desktop so the problem lies somewhere else. If you were pegging the CPU at 100% on all 8 cores that could freeze the desktop but as you noted above, that isn't the case. You have 16GB of RAM installed so I find it hard to believe that could be part of the problem. Maybe try rebuilding your system and ports with the default compiler settings if you haven't already and see if that makes the issue go away.
Click to expand...
No, I don't think so. Just the same freezing was with 9.3, and earlier, when GCC was the default compiler. Due to top, most of the time bsdtar process is in biord state (and in wdrain in the end). And definately, bsdtar is a part of main system (which I had compiled either with base clang or with base gcc with no difference). And with kern.sched.preempt_thresh big enough it's not freezing any more, just lagging.
 
dlegrand said:
Shame on me !

I just seen that tar(1) is supporting .xz compressed files. Now it's far better :
Code: 
# time tar xf /usr/ports/distfiles/chromium-44.0.2403.155.tar.xz --to-stdout > /dev/null
28.964u 0.251s 0:29.22 99.9%  66+172k 0+0io 0pf+0w

It's laborious. Please excuse my numerous posts.
Click to expand...
Mine.
Code: 
24.040u 0.244s 0:24.28 100.0% 66+172k 0+0io 0pf+0w
By the way, what all these numbers mean?
 
YuryG said:
Mine.
Code: 
24.040u 0.244s 0:24.28 100.0% 66+172k 0+0io 0pf+0w
By the way, what all these numbers mean?
Click to expand...
It means that that your system is spending a lot of time writing data to disk.
Does your system is freezing, lagging during those 24 seconds ? I don't think so.
 
dlegrand said:
It means that that your system is spending a lot of time writing data to disk.
Does your system is freezing, lagging during those 24 seconds ? I don't think so.
Click to expand...
You are right. This disc is not too big and 80% full, but I thought that 22GB is enough for less than 4GB chromium.
 
YuryG said:
Mine.
Code: 
24.040u 0.244s 0:24.28 100.0% 66+172k 0+0io 0pf+0w
By the way, what all these numbers mean?
Click to expand...

The exact format depends on which shell you're using because time(1) is often a shell builtin(1). It looks like you're using tcsh(1) judging by the format. In that shell there's a time variable that controls the output:

Code: 
time
If set to a number, then the time builtin (q.v.) executes auto- 
matically after each command which takes more than that many
CPU seconds. If there is a second word, it is used as a format
string for the output of the time builtin. (u) The following
sequences may be used in the format string:

%U The time the process spent in user mode in cpu seconds.
%S The time the process spent in kernel mode in cpu seconds.
%E The elapsed (wall clock) time in seconds.
%P The CPU percentage computed as (%U + %S) / %E.
%W Number of times the process was swapped.
%X The average amount in (shared) text space used in Kbytes.
%D The average amount in (unshared) data/stack space used in
Kbytes.
%K The total space used (%X + %D) in Kbytes.
%M The maximum memory the process had in use at any time in
Kbytes.
%F The number of major page faults (page needed to be brought
from disk).
%R The number of minor page faults.
%I The number of input operations.
%O The number of output operations.
%r The number of socket messages received.
%s The number of socket messages sent.
%k The number of signals received.
%w The number of voluntary context switches (waits).
%c The number of involuntary context switches.

Only the first four sequences are supported on systems without
BSD resource limit functions. The default time format is `%Uu
%Ss %E %P %X+%Dk %I+%Oio %Fpf+%Ww' for systems that support
resource usage reporting and `%Uu %Ss %E %P' for systems that
do not.
 
You must log in or register to reply here.
Back
Top