I have two servers with nearly identical CPU's (Dual Xeon E5530 @2.40GHz and Dual Xeon E5540 @2.53GHz). The E5540 is slightly "beefier" with 64 GB of RAM.
I would expect Poudriere to preform the same or better on the E5540, but I am observing the opposite. The difference is significant. I currently have a build for the entire ports tree running. The E5540 has built 4011 packages in 43 hours and the E5530 has so far built 19,591 packages in 57 hours.
Here's some additional details on the hardware and configuration:
E5540
E5530
Poudriere is configured (mostly) the same on both systems (E5540, E5530). One difference is that the E5530 doesn't have ccache. The E5540 saw a slight improvement when ccache was enabled.
I can't imagine that ZFS has that much of a performance advantage? There must be something else that's slowing down Poudriere on the HP server.
I would expect Poudriere to preform the same or better on the E5540, but I am observing the opposite. The difference is significant. I currently have a build for the entire ports tree running. The E5540 has built 4011 packages in 43 hours and the E5530 has so far built 19,591 packages in 57 hours.
Here's some additional details on the hardware and configuration:
E5540
- HP DL180 G6 (The FreeBSD version isn't a variable since the same performance difference was observed with 12.2)
- 64 GB of RAM
- /var/run/dmesg.boot
- HP Smart Array RAID controller with a RAID5 configuration resulting in a single logical volume.
Code:
# gpart show
=> 34 3906918765 da0 GPT (1.8T)
34 1024 1 freebsd-boot (512K)
1058 419430400 2 freebsd-ufs (200G)
419431458 3355443200 3 freebsd-zfs (1.6T)
3774874658 132044140 4 freebsd-swap (63G)
3906918798 1 - free - (512B)
# zpool status
pool: poudriere
state: ONLINE
status: Some supported features are not enabled on the pool. The pool can
still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
the pool may no longer be accessible by software that does not support
the features. See zpool-features(5) for details.
scan: scrub repaired 0B in 02:27:01 with 0 errors on Sun Feb 7 01:16:21 2021
config:
NAME STATE READ WRITE CKSUM
poudriere ONLINE 0 0 0
da0p3 ONLINE 0 0 0
E5530
- IBM HS22 Blade
- 48 GB of RAM
- /var/run/dmesg.boot
- 4 SAS drives in a JBOD configuration. RAID is done with ZFS.
Code:
# gpart show
=> 40 62499920 da0 GPT (30G)
40 409600 1 efi (200M)
409640 58310656 2 freebsd-ufs (28G)
58720296 3125248 3 freebsd-swap (1.5G)
61845544 654416 - free - (320M)
# zpool status
pool: storage
state: ONLINE
scan: none requested
config:
NAME STATE READ WRITE CKSUM
storage ONLINE 0 0 0
raidz1-0 ONLINE 0 0 0
da1 ONLINE 0 0 0
da2 ONLINE 0 0 0
da3 ONLINE 0 0 0
da4 ONLINE 0 0 0
Poudriere is configured (mostly) the same on both systems (E5540, E5530). One difference is that the E5530 doesn't have ccache. The E5540 saw a slight improvement when ccache was enabled.
I can't imagine that ZFS has that much of a performance advantage? There must be something else that's slowing down Poudriere on the HP server.