To a certain degree I indeed think that people who want to compile the elephant ports should buy a semi-beefy machine (say $1000). I know this isn't possible for everyone, but machines that are cheap or otherwise crippled (some laptops) will naturally lead to high compile times.
FWIW I just built rust in 59 minutes on fairly low-end desktop hardware. The same machine does buildworld in 42 minutes, so it would be a significant difference for the base system, but 59 minutes doesn't seem excessive for ports - at least not compared with the time desktop ports took when I still used ports.
It's a AMD Ryzen 5 5600G with 64GiB of DDR4, the CPU was over 4 years old when I bought it and DDR4 was much cheaper than DDR5 at the time , so it was pretty cheap.