Solved `make buildworld` does not build base llvm on FreeBSDDesktop/freebsd-base-graphics

m4r1n

m4r1n

Hi,

I'm in the process of testing the drm-next-4.7 branch from the FreeBSDDesktop github repository on several computers. As I want to avoid building world on every machine, I decided to set up a dedicated build machine running FreeBSD 11.0-STABLE. The plan was to build world, kernel and some parts of the ports tree with ports-mgmt/poudriere, and then generate FreeBSD release images and a pkgng repository for binary packages.

I was able to build both the world and the kernel successfully. I was also able to generate working install images with the make release command. However, it seems that the whole build process skips building the base versions of llvm39 and clang: as a consequence, all systems installed from the release images lack a compiler. Installing the new world and kernel on the build machine does not install llvm39: instead, llvm38 is left untouched, until flagged obsolete by the make remove-old-libs command, and deleted.

Does anybody know why make avoids building base llvm39 as a bootstrap compiler even while the current base version is llvm38 ? Is there a way to force the build of base llvm and clang ?

I do not use any src.conf or make.conf file. I read the src.conf(5) man page and tried to pass custom flags to make (like WITH_CLANG, WITH_CLANG_BOOTSTRAP, WITH_CLANG_FULL), without any success.

Here is how to reproduce:
  1. Install FreeBSD 11.0-RELEASE or a snapshot of 12.0-CURRENT with default options. I tried the following with both versions and base llvm was not built in both case.
  2. Install the devel/git package with pkg.
  3. Clone the github repository to /usr/src.
  4. Try to build everything:
    # cd /usr/src/
    # make buildworld buildkernel
  5. The build stops immediately because make cannot find llvm39:
    You need to install devel/llvm39 to build or set XCC
    That error is new to me: I never saw it when I ran make buildworld in previous releases. As a workaround, install devel/llvm39, or run make with explicit environment variables:
    # make XCC=clang XCXX=clang++ XCPP=clang-cpp buildworld buildkernel
    llvm39 will not be built whatever workaround you chose.
  6. After the build finishes, make a release:
    # cd /usr/src/release
    # make XCC=clang XCXX=clang++ XCPP=clang-cpp release WITH_DVD=1
  7. Install the release to another computer, or install the new world and kernel on the current machine as you would normally do, with mergemaster and make remove-old / make remove-old-libs.
  8. Boot the new install or the new kernel, then run:
    cc -v
  9. No compiler found.
Thanks a lot for your help!
 
I asked this same question over on the GitHub repo - Matt Macy says they explicitly disabled building the new compiler as part of buildworld in their tree, as it triples the build time and the assumption is the user should just install it with pkg afterwards. Does this help?
 
Looking at how that was done, it looks like MK_CLANG and its variants are set to no in various places throughout the tree (bear in mind I'm a BSD newb). I'm gonna try setting this back and seeing if it builds Clang. I like experimenting after all - I'll report back with my findings :-D
 
Ok the way to re-enable from my experimentation so far seems to be to add the following (paths relative to base of drm-next source tree):
Code: 
./lib/Makefile: SUBDIR.${MK_CLANG}+= clang
./usr.bin/Makefile: SUBDIR.${MK_CLANG}+= clang

Add these with the other SUBDIR.${} declarations in each case. That said, LLVM build in drm-next buildworld is failing for me when building libllvm so will need to see about that next :-)
 
SirDice said:
This is not an official tree but a fork. No guarantees are made regarding forks. I suggest you fetch the official tree.
Click to expand...

Yes, I'm aware of that. Yet, I can't use the official tree this time, as the new graphics stack relies on kernel changes which were not commited to CURRENT.

PhilPotter said:
Ok the way to re-enable from my experimentation so far seems to be to add the following (paths relative to base of drm-next source tree):
Code: 
./lib/Makefile: SUBDIR.${MK_CLANG}+= clang
./usr.bin/Makefile: SUBDIR.${MK_CLANG}+= clang

Add these with the other SUBDIR.${} declarations in each case. That said, LLVM build in drm-next buildworld is failing for me when building libllvm so will need to see about that next :)
Click to expand...

Thanks a lot for your posts! Yes, I read the whole (now closed) issue on the GitHub repository. I'm afraid we won't be able to build the compiler at all as of now.
 
The attached patch reverts everything in those two commits except for the libc tests and the svn building. It is based against the drm-next branch rather than drm-next-4.7. That said, the build still fails further down the line for me with a clang header error regarding DiagnosticCommonKinds.inc, so it is possible there is further divergence from 12-CURRENT there.

I am currently in the process of building world from 12-CURRENT, then I will just take the kernel from drm-next - this is confirmed to work by a contributor on the github repo as all of the relevant changes are kernel-side. It may be better for you to take this approach rather than going with my patch. Happy to help with anything I've learnt although you are almost certainly more knowledgeable about FreeBSD than I am :-D
 

Attachments

PhilPotter said:
Happy to help with anything I've learnt although you are almost certainly more knowledgeable about FreeBSD than I am :-D
Click to expand...

Or am I? Afterall, you made the patch! ;)

LLVM build failure in drm-next is unfortunately not a surprise: I read about it on Github, so this was something I expected. I shall probably do the same as you and build world from CURRENT, then kernel from drm-next. This shouldn't hurt as the kernel ABI is unlikely to have changed since the last drm-next MFC. I'll let you know if I succeed in some way!

Thanks again for taking the time to deal with this.
 
I made the patch using my Linux skills :-p I know very little about FreeBSD other than what I've gleaned from the excellent handbook and some Google-fu. I just find it fascinating and am drawn to it despite the fact that it doesn't support my graphics card properly. Just seems like a well thought out system that with enough time and patience I could actually understand in some fashion from the bottom up, and systems excite me :-)

Yeah as for drm-next, there is a guy on the github repo that is doing this so it should work. I'm currently getting towards the end (I hope!) of a CURRENT world build in a hyper-v VM, kernel build will be lightning fast in comparison! Speaking of which, the VM is running in legacy mode as the EFI loader craps out on gen 2 hyper-v VMs - I found a patch for that from some Microsoft dude which I've applied to my CURRENT copy. The idea is I'm gonna boot the new OS up from a USB HDD when it is setup with xorg-server 1.18 etc. and see if it works - I can do all the setup and minimise downtime that way. I'll let you know likewise if I succeed :-)
 
I can confirm this approach results in a bootable system - the VM is running quite nicely with kernel from drm-next and world from CURRENT, although no keyboard driver for Gen2 VMs - not an issue as I'm using ssh anyway - maybe that could be a task to cut my teeth on, a new keyboard driver :-D I will be testing it on the real system next, to see how it gets on with my Radeon HD 7850.
 
Yes, I confirm this too. I just finished to build the same mixed system, also on Hyper-V. I've built iso, img and tar install media with:
I'll upload the images to a public server. This could be a great time saver for people who wish to try the new drm stack. I already started to build some parts of the port tree with ports-mgmt/poudriere. I'll try to share the resulting repository too, since the one located at http://www.bsddesktop.com has been offline for some time.
 
Hope it goes well - I am anticipating my AMD experiment may not. Still needs another few months of cooking me thinks :-D Anyhow, I'll find out when I finish work.
 
Good news first: install images are online. They can be found at http://sites.olivarim.com/freebsd/drm-next/ with boring bla-bla. Maybe we should be sure it works as expected before posting the link on the mailing list? Or maybe not; I don't know.

Now the bad news: build of devel/llvm37 failed in ports-mgmt/poudriere. It is needed to build x11/xorg. However, it seems that this was fixed by a recent patch which haven't made it yet to the freebsd-ports-graphics ports tree. I just started a new build, and I'll let you know if it succeeds. If I'm able to build a decent pkgng repository, I'll upload it to the public web too.
 
Nice one, thanks for that :-) Weirdly my install boots on hyper-v but not on my real system - loader doesn't load the kernel properly. I suspect that patch I applied has something to do with it :-p
 
What do you mean when you say that your card doesn't work? In theory, Southern Islands-based GPUs are supported by stock FreeBSD, without graphics acceleration though. The drm-next kernel may include better support for your GPU, but shouldn't make things worse! If there is a regression, you may consider posting a word on the freebsd-x11 mailing list (or the repo issue tracker) to let the devs know about it.

I'm in the middle of building packages and drivers. I used default options, so the ATI driver was part of the build. Did you build your own packages? If so, what ports tree did you use?
 
I mean the system boots but monitor just goes black as soon as I load amdgpu module. If I load it at boot, system panics too. My system is now covered by this module as amdgpu now handles southern islands cards too. I tried radeonkms too with similar issues. I can get it to work with scfb without loading any modules. My card works admirably on 11.0-RELEASE, minus acceleration which I can't do without for my workload (I'm writing a PlayStation emulator).

I didn't build my own packages, just used the ones from offical 12-CURRENT repo, as per new guidance on drm-next repo - this now includes updated xorg server and drivers.
 
Well, that's bad news. You should try to ask for advice on the mailing list; maybe someone knows how to make it work.

I didn't know they updated 12.0-CURRENT. This obsoletes my own repository. Thanks for letting me know!
 
Yeah I may just do that, or post on the drm-next repo. I do know that at the moment AMD support is way behind Intel though. Seems a shame to give up on FreeBSD now so I will monitor :-)
 
You must log in or register to reply here.
Back
Top