Frame.work 12th gen update

[cmoerz]

Anyone who's been waiting like me - it appears we've reached the point where 12th gen frame.work notebooks do work with FreeBSD 14.

I've compiled the following branches:

• https://github.com/dumbbell/freebsd-src/tree/linuxkpi-updates-for-drm (f30894f)
• https://github.com/dumbbell/drm-kmod/tree/update-to-v5.17 (de9e9ea)

and after doing a full installworksinstallworld/installkernel/install and kldload i915kms, it appears I've got full graphics support, backlight works and I just did a test of sleep and suspend via zzz and it works!

dumbbell@ has done an amazing job, from what I can tell.

I suppose, even a regular stable/14 might do the trick with 5.15-lts from the usual sources. I haven't tested that yet, however.

Looks like it's time for me to switch my 11th gen for the 12th gen now. It's work, but it's so worth it. Now I just need to fiddle around with the cpu sets and see if I can make use of these low power cores to extend the battery life even more.

If anyone is interested, I can look at scraping everything together and posting my configs from my 11th gen, which helped me to get a very decent battery runtime. This should be a good starting point for the 12th gen as well, I think.

 

[smithi]

If anyone is interested, I can look at scraping everything together and posting my configs from my 11th gen, which helped me to get a very decent battery runtime. This should be a good starting point for the 12th gen as well, I think.

Yes please, post them to this thread for future reference.

 

[cmoerz]

Will do. I'm in the middle of installing 14-BETA2 from scratch to retest at the moment and will post everything in the next few days.

 

[cmoerz]

Update on this: I failed to get it running reproducibly with releng/14.0. Unfortunately, I didn't jot down my steps as I was a little too optimistic that things will work.

I have now reinstalled 14-BETA2, which did not work (it freezes up when loading i915kms) and now retracing my steps to get things working again with dumbbell@'s codebase.

Once I have that up, I'll re-add my loader.conf, sysctl and rc.conf and see whether things keep working and finally post my results with the configs.

 

[cmoerz]

Well, this sucks. I've managed to get everything working with the commits as above. Unfortunately, once I remove invariants, witness, dbd, deadlock resolution et al from the kernel, it breaks.

That means, unfortunately, I was probably a little too early with my announcement.

Not sure if I want to run on a debug kernel. I'll try to disable invariants and witness only and will check whether keeping debug capabilities fixes things. If that breaks stuff as well, we might be facing some timing issues that will be very hard to fix.

 

[cmoerz]

Ok, it still works after removing witness and invariants. Now to pick things out bit by bit and see where it breaks.
It's a git bisect for poor people!

 

[cmoerz]

Finally managed to get everything working. I did merge BETA3 into dumbbell@'s code and I am now running on BETA3 with i915kms loaded successfully.

I've managed to reduce power draw down to 3-5W at idle by limiting CPU cores to two low-power cores. This way, it's actually more efficient than the 11th gen frame.work.

Here's my configuration files - feedback and suggestions are welcome.

GitHub - christian-moerz/framework12: FreeBSD on frame.work 12th gen

FreeBSD on frame.work 12th gen. Contribute to christian-moerz/framework12 development by creating an account on GitHub.

github.com

Addtionally, here's the merged base code for buildworld+buildkernel - merged BETA3 with dumbbell@'s updates:

GitHub - christian-moerz/freebsd: FreeBSD mainline for drm-kmod with patches for frame.work 12th gen

FreeBSD mainline for drm-kmod with patches for frame.work 12th gen - christian-moerz/freebsd

github.com

 

[ko56]

Finally managed to get everything working. I did merge BETA3 into dumbbell@'s code and I am now running on BETA3 with i915kms loaded successfully.

I've managed to reduce power draw down to 3-5W at idle by limiting CPU cores to two low-power cores. This way, it's actually more efficient than the 11th gen frame.work.

Here's my configuration files - feedback and suggestions are welcome.

GitHub - christian-moerz/framework12: FreeBSD on frame.work 12th gen

FreeBSD on frame.work 12th gen. Contribute to christian-moerz/framework12 development by creating an account on GitHub.

github.com

Addtionally, here's the merged base code for buildworld+buildkernel - merged BETA3 with dumbbell@'s updates:

GitHub - christian-moerz/freebsd: FreeBSD mainline for drm-kmod with patches for frame.work 12th gen

FreeBSD mainline for drm-kmod with patches for frame.work 12th gen - christian-moerz/freebsd

github.com

This is great work. I have a Dell Optiplex Micro with an i5-13600T CPU (6P + 8E cores) and UHD 770 graphics that has been running 13.2-RELEASE. All cores run, but I'm unclear how much they're "managed". But the UHD 770 graphics are not supported.

How much of your work do you think will be in 14.0-RELEASE?

 

[cmoerz]

How much of your work do you think will be in 14.0-RELEASE?

That's an interesting question. I actually had assumed that all of dumbbell@'s linuxkpi changes would have made it into base already.

I had already previously attempted a "head-through-the-wall" approach of upgrading drm-kmod some time last year when I met with dumbbell@ and he showed me the way more convenient and standard approach to getting to a newer linux version for the drivers. We realized eventually that the official upgrade work ended up at the same non-working spot as my previous attempt; Jean-Sebastien made out vt as the culprit - the base terminal driver. We would have to redirect vt to call into the non-base drm-kmod driver somehow.

I suppose this vt change might still be under review - I haven't checked yet; neither phabricator nor my own diffs. If it's really vt still being in limbo, then I assume we won't see it in 14 because it's already pretty late in the release cycle.

Update: Yup, it's vt. I just compared commits on vt from base to dumbbell@; there's a bunch of stuff that is simply not in base yet. Actually, after looking at this a second time, I'm not so sure if it's just vt... I'll have to check the diffs to releng/14 to say for sure.

History for sys/dev/vt - dumbbell/freebsd-src

FreeBSD src tree. Contribute to dumbbell/freebsd-src development by creating an account on GitHub.

github.com

about P- and E-cores; those aren't managed to matter what. There's no support for differentiating those yet AFAIK. They work like regular cores and get a unique id which allows you to treat them like any other core (i.e. forcing them into idle or loading them depending on your preferences). In theory, one could write a driver to idle the power cores when on battery. I don't have the time to do this at the moment, even though this would certainly be a very interesting exercise.

 

[cmoerz]

And I've updated my source to BETA4 just now. Seems to work just as well as BETA3.

I did experience a few crashes on BETA3, but I believe the culprit is not in drm-kmod or anywhere in the graphics stack but with USB and USB ethernet...

 

[cmoerz]

And merged BETA5 just now. I'll keep updating until the final release is out. I'm still hoping that everything will work without my patch work then.

 

[ko56]

And I've updated my source to BETA4 just now. Seems to work just as well as BETA3.

I did experience a few crashes on BETA3, but I believe the culprit is not in drm-kmod or anywhere in the graphics stack but with USB and USB ethernet...

I put BETA5 on my (above) Optiplex Micro with the i5-13600T. Everything seems OK, except when I put kld_load="i915kms" in my /etc/rc.conf, the boot hangs at

Mounting local filesystems Loading kernel modules

I have drm-515 installed. Any ideas?

 

[gpw928]

For anyone interested, delivery of the 16" Framework models is now out to Q2 next year. From Tom's Guide:

"The laptop should start shipping at the end of 2023. However, if initial pre-orders are sold out, the next batch of units won't ship until the first quarter of 2024".​

I was waiting to see how FreeBSD went, but my old notebook is dead, and the Framework web site is suggesting that the initial pre-orders are sold out.

Maybe there's some smart marketing here... but I just paid the (fully refundable) deposit to get into the queue for batch 13 in Q2.

 

[cmoerz]

I put BETA5 on my (above) Optiplex Micro with the i5-13600T. Everything seems OK, except when I put kld_load="i915kms" in my /etc/rc.conf, the boot hangs at

Mounting local filesystems Loading kernel modules

I have drm-515 installed. Any ideas?

I'm afraid the 13th gen is still a bit too far out. My source branch just about works for 12th gen. Mine is a 13" 12th gen frame.work; I might have caused confusion because the most current 13" also comes with 13th gen. Apologies for that.

What I should probably also add: if you're running stock BETA5 and stock drm-kmod, it wouldn't even work with 12th gen - at least according to my test. That's why I did my patched version of base and drm-kmod in the first place. You could try my patched base and dumbbell@'s https://github.com/dumbbell/drm-kmod/commit/de9e9ea9f9bac04722ecf36549f6df99cef5d274 drm-kmod.

 

[ko56]

I'm afraid the 13th gen is still a bit too far out. My source branch just about works for 12th gen. Mine is a 13" 12th gen frame.work; I might have caused confusion because the most current 13" also comes with 13th gen. Apologies for that.

What I should probably also add: if you're running stock BETA5 and stock drm-kmod, it wouldn't even work with 12th gen - at least according to my test. That's why I did my patched version of base and drm-kmod in the first place. You could try my patched base and dumbbell@'s https://github.com/dumbbell/drm-kmod/commit/de9e9ea9f9bac04722ecf36549f6df99cef5d274 drm-kmod.

Thanks. I'm out of my depth here. Doing what you suggest above would require building a custom kernel, and a modified drm-kmod port, is that correct?

 

[cmoerz]

It's really just running a few commands - in the end you get a stock-ish kernel because you're building everything that you'd get in a binary installation.

You simply install git, clone my framework12 branch into /usr/src; there you do a make buildworld && make buildkernel && make installworld && make installkernel && reboot

You can add a -j<CPU/thread count> (i.e. -j8, -j16...) to your make commands, depending on how many CPU threads/cores you've got.

Once you have that running, clone dumbbell@'s drm-kmod into a directory of your choice and do a make && make install; here again, you can add a -j8 or -j16

You can try a kldload i915kms and should see whether it freezes up or works. If it works, add your kldlist to rc.conf.

PS. Corrected because I wrote "i916kms" instead of "i915kms"

 

[cmoerz]

I've updated to RC1 just about now. I believe I'll stop pinging this thread now, because I'll just keep rolling the code forward until 14 is finally released. No need to bump this anymore until then, unless any questions or issues arise.

 

[ko56]

I've updated to RC1 just about now. I believe I'll stop pinging this thread now, because I'll just keep rolling the code forward until 14 is finally released. No need to bump this anymore until then, unless any questions or issues arise.

I also just updated to RC1, on the chance that things would improve w.r.t. drm-515-kmod. I don't feel comfortable rebuilding the whole system as you described above. Anyhow, after the update, kldload i915kms still freezes the system, so no luck.

But given that you are rolling your code forward as you say, is there some reason that it's not being pushed into the release?
(Again, I'm an ordinary user, and don't understand all the considerations involved.)

 

[cmoerz]

That's a really good question; I just diffed my code again against the main branch of releng/14.0 - the majority of changes is indeed in vt. That likely explains why you're seeing freezes with mainline code. Those are a must for newer linux code to work at all.

I suppose Jean Sebastien is still polishing the code from what I can tell from his pull request - AMD GPUs apparently were creating issues and some people were getting blank screens.

Update to Linux 5.17 drivers by dumbbell · Pull Request #236 · freebsd/drm-kmod

This is the backport of the DRM drivers from Linux 5.17. Progress: Changes in Linux 5.17 You can read this Phoronix article to learn about the changes in the DRM drivers in Linux 5.17: https://www...

github.com

There are also pending reviews for his linuxkpi additions:

⚙ D42054 linuxkpi: `GFP_KERNEL` equals `M_NOWAIT` now instead of `M_WAITOK`

reviews.freebsd.org

and I almost forgot this one:

⚙ D42056 vt(4): Skip vt_flush() for nested panics

reviews.freebsd.org

that's necessary to handle kernel panics in the graphics driver which would break vt in the process unless his code is merged... that one is also related, I think:

⚙ D42055 kern_shutdown: Always request a core dump in `vpanic()` even for a panic triggered while handling another one

reviews.freebsd.org

Unfortunately, this whole thing has grown to such big proportions that it isn't easy to get everything to a stable state that works for everyone. You have to align drm-kmod as well as OS code.

That's why I chose to focus my efforts on the frame.work notebook - it allows me to publish a working state and I simply accept that it might not work at all for other graphics devices but the integrated Intel I'm working with.

The good thing is - you can verify my repository pretty easily by cloning the repo and diffing it against the mainline repo. Something like this should do the trick:

mkdir framework12
git clone -b framework12 https://github.com/christian-moerz/freebsd framework12/
cd framework12
git remote add bsd https://github.com/freebsd/freebsd-src
git pull bsd releng/14.0
git diff bsd/releng/14.0

I completely understand that you're not feeling comfortable in compiling things yourself. Unfortunately that means you'll either have to wait or someone could provide binaries, which will again create the dilemma of trusting someone's foreign binary code.

I suppose I could look into building an update server (there seems to be pretty good documentation on it at https://docs.freebsd.org/en/articles/freebsd-update-server/); I unfortunately don't have the time to do that at the moment. I'm afraid, it's probably too much of a niche effort that anyone else will consider it any time soon.

 

[Jose]

I've updated to RC1 just about now. I believe I'll stop pinging this thread now, because I'll just keep rolling the code forward until 14 is finally released. No need to bump this anymore until then, unless any questions or issues arise.

Your time is your own, of course, but FWIW I've been enjoying these.

 

[ko56]

That's a really good question; I just diffed my code again against the main branch of releng/14.0 - the majority of changes is indeed in vt. That likely explains why you're seeing freezes with mainline code. Those are a must for newer linux code to work at all.

I suppose Jean Sebastien is still polishing the code from what I can tell from his pull request - AMD GPUs apparently were creating issues and some people were getting blank screens.

Update to Linux 5.17 drivers by dumbbell · Pull Request #236 · freebsd/drm-kmod

This is the backport of the DRM drivers from Linux 5.17. Progress: Changes in Linux 5.17 You can read this Phoronix article to learn about the changes in the DRM drivers in Linux 5.17: https://www...

github.com

There are also pending reviews for his linuxkpi additions:

⚙ D42054 linuxkpi: `GFP_KERNEL` equals `M_NOWAIT` now instead of `M_WAITOK`

reviews.freebsd.org

and I almost forgot this one:

⚙ D42056 vt(4): Skip vt_flush() for nested panics

reviews.freebsd.org

that's necessary to handle kernel panics in the graphics driver which would break vt in the process unless his code is merged... that one is also related, I think:

⚙ D42055 kern_shutdown: Always request a core dump in `vpanic()` even for a panic triggered while handling another one

reviews.freebsd.org

Unfortunately, this whole thing has grown to such big proportions that it isn't easy to get everything to a stable state that works for everyone. You have to align drm-kmod as well as OS code.

I read through these 4 threads, it's a tremendous amount of work by a lot of people! Like you say, this whole issue is much bigger than I (naively) thought. I hope for the best, that it will end up in 14.0-RELEASE.
Thanks for all your postings.

 

[cmoerz]

By popular demand, or by lack of complaints... RC2 just landed in my repo as well.

I've had a few crashes recently in combination with USB networking. I do not believe this is due to the drm-kmod changes but I did not manage to troubleshoot in detail yet. So YMMV.

 

[cmoerz]

And, we're up to RC3.

Word of warning: I experienced a very weird issue with RC2. Everything was working fine, then I had my machine on standby and battery for a while. Opened the lid, everything came up (screen lit up with lock screen) and then it crashed/rebooted without a warning, with no error indicating anything in the logs.

After rebooting, i915kms froze - no matter whether I booted with the current or with the previous kernel. I had to recompile drm-kmod, reinstall and then things started to work again, which totally makes no sense to me at all, to be honest. Feels like this still isn't as stable as I had hoped, I guess.

This is the major issue with having new vt and drm-kmod code; once something breaks, you usually can't properly acquire a dump because you won't see anthing and the logging is useless. I once tried adding logging code into graphics code and lets just say you realize how often that code is getting called...

 

[ko56]

Apart from this thread, is this issue of i915, or drm-kmod more generally, discussed anywhere on the mailing lists? It's an important issue, but I've looked, and the only (slight) mention I could find is on the freebsd-x11 list, which is updated rather infrequently.

 

[cmoerz]

Not that I'm aware of. The thing is, FreeBSD will work just fine without i915kms.ko - it's X11 that's going to be primarily affected. Hell, you might even be able to get things working with the framebuffer driver. Not having the right driver means having no backlight control on a notebook, not having graphics acceleration, not being able to put the machine to S3 sleep and so on.

The people working on the issue certainly are aware - at least from what I know/can tell. I haven't talked to dumbbell@ in a while, but you can see on his Github that he's very much active and continues to work on things. The vt code is being reviewed last I checked...

If you're interested in knowing more or want to help out, I recommend reaching out to dumbbell@ - IRC is probably the best way (on LIbera chat if I remember correctly). I was fortunate originally to get connected via a contact at the FreeBSD Foundation.

I eventually gave up because I time-wise I did not live up to the requirements for getting my kernel knowledge to the required level. Graphics driver development really is the absolute top level of development because it's not only hard to do right but even harder to troubleshoot and debug. If you have a workstation with serial it's all fine and dandy, working with a notebook that's got nothing but USB-C, you're SOL. Any time anything breaks, you have to mash the keyboard with dump commands and hope for the best.

In retrospect, I should have started my kernel programing stint with some lighter food for thought, I guess

Still, I learned a lot in the process and I gained tremendous respect for the guys doing it and for the automation they have put into place to make things easier for new developers. While there's still room for more documentation, FreeBSD is doing something right here - being welcoming and lowering the entrance barrier as much as possible and also making it fun to develop.