Broke my Installation - Could use advice for starting over

J

jardows

So I broke my FreeBSD installation last night. This is my fault primarily, so this is more of an "experience shared" rather than a request for help. I had some performance issues with some applications in my FreeBSD 12.1 installation on my new Ryzen 3600 desktop. I am dual-booting with Windows, so could compare application performance cross-platform. I was using /multimedia/openshot to process some videos - using CPU rendering in Windows I was achieving nearly 55 fps, while in FreeBSD I was only achieving 18 fps processing the exact same video file. I couldn't find any information regarding this, so I started troubleshooting myself. I wondered if perhaps using devel/llvm10 to compile programs from ports would make a difference, as I could use some of the optimizations for the Zen 2 processor. I figured it was also a good time to try out ports-mgmt/synth as there were some program defaults I wanted to change in a few packages. Of course, too many changes at one time is a recipe for disaster, and that's what seems to have happened. 3 of the ports failed to build, but a number of ports were removed, and I went from a very functional (having installed everything previously from packages) albeit not ideal for me system, to a broken system that would not properly load X or my DE. Now this isn't a big loss for me, as it was a fresh installation so there were few configurations and no data, I can just wipe it out and start new again. Just a word to the wise - don't try to change too much at one time!

I do have a question as a start to rebuild my FreeBSD installation. The options I am looking at are 1.) starting over with a 12.1 installation, immediately installing LLVM10 and then proceeding to build everything from ports or synth with clang10 set as the default compiler in make.conf, or 2.) install -CURRENT which already has LLVM10 in the base and install from there. Which path do you think would work better?
 
llvm/clang is in the base and it's default. There's no need to explicitly install llvm from ports. Don't switch to CURREN, it's mostly targeted to kernel devs, etc. Go CUURENT and the next update/compile you may have a broken system, and you have to seek help in mailing list (no CURRENT support in Forums). If you stay with default options in ports collection, there's no need to build from the ports, use pre-build packages. If you are going to change ports options, you have to build from the ports. Don't mix ports and packages. If you want to build ports, consider using ports-mgmt/poudriere
 
  • next time please use boot environments: sysutils/beadm (1st choice) or bectl(8)
  • consider to set up a system running 12-STABLE: 12.1-RELEASE boot env, create an empty new 12-STABLE boot env (use DESTDIR to fill it), once it's running stable for a month or so, you can delete the old boot env. EDIT 2: Although IMHO it's a good idea to keep a -RELEASE boot env with packages. Then you can check any errors in your -STABLE+ports setup against that.
EDIT: before every major change, create a clone boot env & recursive ZFS snapshot, delete afterwards once all went well.
 
  • Like
Reactions: a6h
It sounds like only your ports/packages got screwed up. Not really a reason to do a complete reinstall. Just do pkg delete -af to delete everything that's installed and start over from there. Remember, the OS and third-party software (ports/packages) are completely separated.
 
Thanks all for the tips.
vigole said:
llvm/clang is in the base and it's default. There's no need to explicitly install llvm from ports. Don't switch to CURREN, it's mostly targeted to kernel devs, etc. Go CUURENT and the next update/compile you may have a broken system, and you have to seek help in mailing list (no CURRENT support in Forums). If you stay with default options in ports collection, there's no need to build from the ports, use pre-build packages. If you are going to change ports options, you have to build from the ports. Don't mix ports and packages. If you want to build ports, consider using ports-mgmt/poudriere
Click to expand...
Base llvm/clang in 12.1 is 8.0, which does not have optimizations for the Zen 2 CPU. 10 has those optimizations, so I want to try that for compiling the programs to see if it makes a difference. For managing and building the ports, synth seems to be a much simpler solution for a single workstation than poudriere.
Zvoni said:
This sounds more like as if Openshot isn't using all cores/threads available
Click to expand...
Thus my motivation for wanting to try compiling it with llvm/clang10 for the cpu optimization.
mjollnir said:
  • next time please use boot environments: sysutils/beadm (1st choice) or bectl(8)
  • consider to set up a system running 12-STABLE: 12.1-RELEASE boot env, create an empty new 12-STABLE boot env (use DESTDIR to fill it), once it's running stable for a month or so, you can delete the old boot env. EDIT 2: Although IMHO it's a good idea to keep a -RELEASE boot env with packages. Then you can check any errors in your -STABLE+ports setup against that.
EDIT: before every major change, create a clone boot env & recursive ZFS snapshot, delete afterwards once all went well.
Click to expand...
Interesting idea, but it looks like it only applies to ZFS. I typically use UFS for single-drive desktop installations, and have here.
SirDice said:
It sounds like only your ports/packages got screwed up. Not really a reason to do a complete reinstall. Just do pkg delete -af to delete everything that's installed and start over from there. Remember, the OS and third-party software (ports/packages) are completely separated.
Click to expand...
A complete destruction of the packages and ports would probably be the easiest solution. I wasn't sure if that would work on the ports and packages at the same time, or just the packages. I certainly don't want to have to go into each port and make deinstall on anything not installed by pkg.
 
jardows said:
Base llvm/clang in 12.1 is 8.0, which does not have optimizations for the Zen 2 CPU. 10 has those optimizations, so I want to try that for compiling the programs to see if it makes a difference.
Click to expand...
Please search the web to see how big a change you expect. Typically, re-compiling for a specific CPU is a small effect, often 1% or 10%. The only way recompiling will make a more significant difference is if you start using specialized instructions for vector-type operations, and I don't know whether those have changed with your CPU, but this is something where reading documentation or searching the web might help. You are looking for a factor of 3 (18 to 55 fps), so recompiling for 10% effects isn't a good investment of time.
 
jardows said:
Interesting idea, but it looks like it only applies to ZFS. I typically use UFS for single-drive desktop installations, and have here.
Click to expand...
If you intend to reinstall anyway, strongly consider ZFS if you have a reasonable large HDD (let's say >50 GB). ZFS makes admin's & developer's life much easier. If you decide to stay with UFS, which is fine but sub-optimal, have a look at the service script to insert the geom(4) I/O scheduler here: /etc/rc.d/geom_sched.
 
SirDice said:
Ports build packages. So in the end everything that's installed is a package.
Click to expand...
Good to know. I'll probably take this path next then.
ralphbsz said:
Please search the web to see how big a change you expect. Typically, re-compiling for a specific CPU is a small effect, often 1% or 10%. The only way recompiling will make a more significant difference is if you start using specialized instructions for vector-type operations, and I don't know whether those have changed with your CPU, but this is something where reading documentation or searching the web might help. You are looking for a factor of 3 (18 to 55 fps), so recompiling for 10% effects isn't a good investment of time.
Click to expand...
At this point, I'll be re-installing all packages/ports anyway, so it's worth a shot! For my purposes, there are just a few to many configuration options I want to change, so ports are going to be my installation method. I didn't have time this morning to try to find out why the three ports failed to install, but as I was breaking one of the main rules of troubleshooting (changing more than 1 thing at a time) I cannot say if it had anything to do with using clang10 and znver2 optimizations. Due to the nature of Zen2, I feel that there can be some significant improvements with the optimizations, and some of the benchmarks I've seen bears this. Maybe not a 3-fold increase, but no need to leave performance on the table.
https://www.phoronix.com/scan.php?page=news_item&px=AMD-Znver2-Clang-10-Tests
This is comparing llvm 9 to llvm 10, but in the comparisons of 8-9 that I have seen, llvm 9 consistently has better performance, so I feel using llvm/clang 10 to compile ports, as long as they work, is worth it.
mjollnir said:
If you intend to reinstall anyway, strongly consider ZFS if you have a reasonable large HDD (let's say >50 GB). ZFS makes admin's & developer's life much easier. If you decide to stay with UFS, which is fine but sub-optimal, have a look at the service script to insert the geom(4) I/O scheduler here: /etc/rc.d/geom_sched.
Click to expand...
Interesting options there. Looking through the handbook, it looks like journaling is the primary benefit outside of RAID applications. I mainly choose UFS for my desktop installation for simplicity (I have ZFS for my server). Looking at the options, I think I'd rather use ZFS than mess with GEOM too much, unless I am wrong and there are other significant benefits to using GEOM on UFS.

All in all, this is still a learning experience for me, so I'm open to trying new things. Eventually I'll have an installation that is fully "the way I want it!"
 
jardows said:
Interesting options there. Looking through the handbook, it looks like journaling is the primary benefit outside of RAID applications.
Click to expand...
Actually, in my mind from a data safety and durability standpoint the main advantage to ZFS is checksums even if one doesn't use RAID. It just prevents data corruption. Well, and management flexibility and RAID, but for a laptop with a single disk drive, that's not so relevant.

A few years ago (by now, probably more than a decade ago), the CTO of NetApp famously said that "selling a storage system that is only single fault tolerant is akin to professional malpractice" or something to that effect. Now, in your case you're not buying your storage system (so there is nobody to blame for malpractice other than yourself), and with a single disk there is little you can do for fault tolerance (see footnote). But not even having checksums to validate that the disk actually returned the correct data is just unnecessary recklessness.

Footnote: The moment you start storing data that is not trivial to replace, please start thinking about backups. As long as your computer is a device for playing with, and you are happy to reinstall if it breaks, that's not so necessary. The moment the data on it becomes valuable, the game changes. And remember: It's not a backup until you have successfully restored it.
 
Personally I use UFS for / . This makes life easier for upgrade and booting. Also there are not many writes to /bin and /usr/bin.
Where speed is needed I mount ZFS on the UFS e.g. for poudriere.
 
  • Like
Reactions: a6h
ralphbsz said:
Actually, in my mind from a data safety and durability standpoint the main advantage to ZFS is checksums even if one doesn't use RAID. It just prevents data corruption. Well, and management flexibility and RAID, but for a laptop with a single disk drive, that's not so relevant.
Click to expand...
My statement was in response to GEOM, not ZFS, specifically the implementation of GEOM according to the FreeBSD handbook.

ralphbsz said:
Footnote: The moment you start storing data that is not trivial to replace, please start thinking about backups. As long as your computer is a device for playing with, and you are happy to reinstall if it breaks, that's not so necessary. The moment the data on it becomes valuable, the game changes. And remember: It's not a backup until you have successfully restored it.
Click to expand...
I have a server with ZFS for storing important stuff. The most important stuff stored on my desktop computer are config files ;)
 
You must log in or register to reply here.
Back
Top