Unreal Engine 4.20

malavon

Member

Thanks: 21
Messages: 28

#1
It's that time of the year again, with a new Unreal Engine 4 released by Epic and I'd like to take the opportunity to inform you about some
of the changes.

What has changed
I've been busy taking on a a huge amount of loose ends, although that hasn't really translated in much more commits due to the way I work.
Most of these I won't talk about because they're rather boring, but here are some highlights:

3rd party builds
Context: I'm not talking about 3rd party libraries here, I'm talking about building UE4 on your personal computer or build farm!

You read that right, I've been putting a lot of time in removing many manual steps I still had. It means you no longer have to trust me and
can build your own binaries (after reviewing all code changes of course ;)).
See the next post in this thread for more information.

Single-channel sound issue
Context: In the previous version sound was played on a single channel when using stereo speakers/headphones.

This has been since identified as both an issue with the code and possibly an issue or incompatibility with FreeBSD's channel mixing.
In short (more details, see sources or ask me):
Unreal Engine tries to set up 6-channel audio, which is converted into 4-channel audio by SDL's OSS code.
These 4 channels are then mixed (presumably by FreeBSD) where both the left and right channel get mapped to the left stereo channel,
with the right stereo channel getting nothing but silence.

Solution (as root, with # replaced by your pcm number):
> sysctl dev.pcm.#.bitperfect=1
Setting this sysctl to 1 means "The pure sound stream will be fed directly to the hardware." without any conversion/matrixing.
Note: I suspect that playing 5.1 content on stereo headphones with this setting like this will result in sub-optimal (or garbled) audio.
Set back to 0 after playing.

Broadcast messaging
Context: Broadcasting works differently on FreeBSD compared to Linux, Mac and Windows. This meant that it was not possible to do LAN broadcasting.

Since previous version I have modified the code in such a way that this now works on FreeBSD as well. The only catch is that only the first interface with
broadcasting enabled will be used, however this should be ok for most people.
Check your ifconfig to see if broadcast is enabled in case you need to troubleshoot this:
> ifconfig
em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 9000
...


Executable size
This isn't on my side, but it's important enough to mention it.
Epic has made some changes to the build system for Linux and I've done the same on the FreeBSD port.
In short: executables are now stripped, with debugging symbols in a separate file. Gone are the days of 1Gb+ executables (although they're still rather big).
I have tested that this means I can remove the debugging symbols to make the downloads slightly smaller.

Vulkan support
Technically there is no reason vulkan shouldn't work right now, although I haven't yet tested it (assuming I can get it to work on FreeBSD 11.2).
If you have it running already, try adding the "-vulkan" flag (without quotes) when starting the applications. It would be nice to know if it actually works.

Marketing ;)
Yes, you read that right. While it completely doesn't matter, the previous version didn't really advertise it was running on FreeBSD.
This has now been changed, although it won't be visible in the precompiled demos I supply since the logging is disabled for performance reasons.

A ton of bugs and small improvements
Not much comment here, too many to remember anyway.

Demo & games
Demos have been compiled on FreeBSD 11.2-RELEASE #0, amd64 architecture. No guarantees that they'll run on 11.1 or 11.0.
I have removed all debug symbols from the binaries to reduce size and most likely you won't need those. Should you experience crashes or something,
you can download the necessary files by appending "-debug" to the filename (before the extension).

InfiltratorDemo
Rather heavy tech demo to showcase some rendering features of the engine. Always nice to use as a little gpu roaster and on cold winter nights.
The official Epic documentation has more information on how it came to be.

PlatformerGame
More information can be found in the official Epic documentation.

ShooterGame
Thanks to the multicast code changes mentioned above, this game is now multiplayer in LAN as intended. Up to 8 players and bots can fight to the death!
The official documentation has more information about this sample.

StrategyGame
Rather fun little tower defense game, more information in the official documentation.

VehicleGame
Offroad vehicle game, beat your own time record over and over again! Or not ;)

SHA512 (UE4.20-FreeBSD11.2-InfiltratorDemo-debug.tar.xz) = a59270d2d1bf12ea0565187cdbd94277bb5c3db58f9984c442b4200b9dcad6e7da4b16399db6b4496e7d8d4832e439527fd20fa961021336603a31ad7a3efcf3
SHA512 (UE4.20-FreeBSD11.2-InfiltratorDemo.tar.xz) = aad49c0d1d150d4bbaffaa5cd03c5f2865b50639ce19590f52ef68d118c54a9d3efa2a6fd1f581728f99a0201ced7c404bf343829c10d1df057c7a3b358974cf
SHA512 (UE4.20-FreeBSD11.2-PlatformerGame-debug.tar.xz) = da60bf5edd8f92cf9e739c191cebac2532ab10da388eb55afb0eb50e47a91c500bea9d6a74d4a8dd4d4306f50bb94615e9dbabc0e035d978142f968d9b8476f7
SHA512 (UE4.20-FreeBSD11.2-PlatformerGame.tar.xz) = 9e79b51532fd04244debff7518b3bd84487c22999446feb1ffd88aff30d5985bc98785de25c46254e39dc7e30491cbb266bc4974decb2eb30a7b7b7b2d895e56
SHA512 (UE4.20-FreeBSD11.2-ShooterGame-debug.tar.xz) = 8a9b6b24355a6f23058af4da105bf8b323d7799438650c515d5610abdd973036b0a16ae6cdd6c69eda4f7d614ff76d0705a1fa3348acc57d546093c55a417d3e
SHA512 (UE4.20-FreeBSD11.2-ShooterGame.tar.xz) = c42dd16078c7a1fc3fbb218c28ff1312795dbea2fa02e0dd9e167323d7f6fcd27de3ee611905f50f71964cc04a76f0914883d10b95938bcb2e30de111b4314ba
SHA512 (UE4.20-FreeBSD11.2-StrategyGame-debug.tar.xz) = 16f78e6eedd4e55fd64f40e5248517aaf2499f8ab4f3681ac42b41f0df44f21d03694a27809af4c5e3bfba6e671af97cb88e917161a236464d61c4674d186cfe
SHA512 (UE4.20-FreeBSD11.2-StrategyGame.tar.xz) = 189185bf09c91a8f3134efd7b07541466fb571a63d6d7c6c03bf7b9ab45ab0837062a8b5c88711a16907801be57d68028cc704bd07a45e23dfc35eefb7d3ff79
SHA512 (UE4.20-FreeBSD11.2-VehicleGame-debug.tar.xz) = 47d1fb37d9be80b87428741b3011d0e7e2c65fb6a5f927cdce4c63db39196d64bf12b87d9997e64e86b5a284535144fbbbb9e7f8d7c732cab6a9a2dd4056329c
SHA512 (UE4.20-FreeBSD11.2-VehicleGame.tar.xz) = 4d6692486884cd5e2468e6b9a0d92cba3aa36dfd2386901fc1014909d468c5f2dbf0eb7bc5c6876ed9a29fdd27a375c5227df0b3dea14852c726db4cb632aa47

Practical stuff
In case you need to change keymappings (I don't use Qwerty but tried setting them back to original), change in:
<name>/Config/DefaultInput.ini e.g. ShooterGame/Config/DefaultInput.ini

* You will probably need an OpenGL 4.3-capable GPU and driver, although I've cooked for OpenGL 3 and vulkan as well
* If a demo starts on only part of your screen, try alt+enter twice (once to get out of fullscreen mode, second time to go back in)

Code on GitHub
 
Last edited:
OP
OP
M

malavon

Member

Thanks: 21
Messages: 28

#2
How to build Unreal Engine 4 on FreeBSD
Note: I have hardcoded /usr/local as prefix in several places. If you're using a different $PREFIX, you'll have to modify the code.

Prerequisites
Both running and building require at least 16Gb of ram (with 20Gb of swap space preferrably on an SSD). More ram is always better.
Do not skip any steps or you will have a hard time debugging why you're not getting anywhere.
OS version: FreeBSD 11 for amd64, tested on 11.2
I have upgraded my FreeBSDs to 11.2, which uses Clang 6.0 vs 4.0 in 11.1. Nevertheless I'm pretty sure that there shouldn't be any
problems doing all this on 11.1. Most of the development time has been done on 11.1

Get setup using the official UE4 GitHub documentation.

Install the necessary packages to build the engine and it's dependencies.
You might be able to get away without installing git and working with the GitHub zip, although I haven't tested this. Otherwise install it.
> pkg install git
Install the necessary packages to compile and run the engine tools.
> pkg install autoconf automake bash cmake coreutils gmake mono libinotify libtool unix2dos xdg-user-dirs xdg-utils xorg
From this point on you don't need root privileges anymore, and you have to relinquish them or the tools won't work!

Clone & first setup
Note: if your shell doesn't support pushd/popd you can use cd and just return to the previous directory on each popd.
Start by cloning the repository (unless you're working with the zip).
> git clone git@github.com:malavon/UnrealEngine.git
Do the setup, this will download third party libraries and engine content.
Note: check this forum thread to see if there are any newer versions available than the one in the git command.
> cd UnrealEngine
> git checkout 4.20.2-freebsd
> ./Setup.sh


Build all thirdparty libraries.
> pushd Engine/Build/BatchFiles/Linux/
> ./BuildThirdParty.sh --all
> popd

If this step doesn't end in "Success", check BuildThirdParty.log to see what went wrong. Building separate libs can be done using
> ./BuildThirdParty.sh -b <name>
You may encounter errors if you try rebuilding all dependencies a second time, it's advisable to rerun Setup.sh before doing this.

One manual step
The engine doesn't take care of building this binary itself, which is probably an oversight on Epic side after adding this in UE4.20.
> pushd Engine/Source/Programs/BreakpadSymbolEncoder
> ./BuildBreakpadSymbolEncoderFreeBSD.sh
> mv BreakpadSymbolEncoder ../../../Binaries/FreeBSD
> popd


Generate makefile
Feel free to omit the -cmakefile. You can also omit all arguments, which means the generator will create all make & editor files.
Supported are e.g. cmake, codelite, kdev4 (kdevelop) and some others. Check the documentation or code for these flags.
>./GenerateProjectFiles.sh -makefile -cmakefile

Build the editor and tools
> make
Yes, that's all :-D
This command has built the most important tools, using the "development" build. Check the makefile for other things you might build or for debug/shipping builds.

Start the editor the first time
The editor will spawn several ShaderCompileWorker processes and use them to compile the default materials/shaders in the engine.
The same will happen when you use the editor to load an existing project for the first time.
> pushd Engine/Binaries/FreeBSD
> ./UE4Editor


Enjoy!

Notes:
* if you want to use a branch (e.g. 4.20) instead of the tag, you'll notice that I regularly rebase my repo. In that case you'll have to do a
> git fetch origin && git reset --hard origin/4.20
* if you update your branch or checkout a different one (e.g. master), you'll most likely have to recompile your tools and shaders.
 
Last edited:

ShelLuser

Son of Beastie

Thanks: 1,586
Messages: 3,440

#3
Why not consider to adopt irc/unreal? It's unmaintained right now and I get the feeling that you have a bit of an affinity with it and FreeBSD which could make you a perfect candidate to keep it updated.

Maybe food for thought?
 

shkhln

Well-Known Member

Thanks: 94
Messages: 295

#4
there is no reason vulkan shouldn't work right now, although I haven't yet tested it (assuming I can get it to work on FreeBSD 11.2).
Relevant bits: ICD loader and Mesa patches. Nvidia doesn't provide a native ICD for FreeBSD at the moment and it's not clear whether they are planning to do it; there is a possible workaround (more on that later).
 

kpedersen

Daemon

Thanks: 396
Messages: 1,243

#5
Why not consider to adopt irc/unreal? It's unmaintained right now and I get the feeling that you have a bit of an affinity with it and FreeBSD which could make you a perfect candidate to keep it updated.
I don't think that port is what you think it is. It has nothing to do with 3D engines. It is actually an (The most popular?) IRC server ;)

I did get excited though for at least 3 seconds whilst the freshports page was loading. So thank you for that :)
 
OP
OP
M

malavon

Member

Thanks: 21
Messages: 28

#6
Relevant bits: ICD loader and Mesa patches. Nvidia doesn't provide a native ICD for FreeBSD at the moment and it's not clear whether they are planning to do it; there is a possible workaround (more on that later).
Yeah, I've been looking into it and I get the feeling that my (Haswell) gpu isn't really supported. I'll do some more digging, but if that's the case I won't be able to test vulkan; I'm not counting on Nvidia for releasing a vulkan-ready driver soon (although I would definitely appreciate it).

I don't think that port is what you think it is. It has nothing to do with 3D engines. It is actually an (The most popular?) IRC server ;)
I was wondering the same and more or less waiting for an error to be rectified. Or if not was trying to understand it humoristically/sarcastically somehow.
 

shkhln

Well-Known Member

Thanks: 94
Messages: 295

#7
Nvidia doesn't provide a native ICD for FreeBSD at the moment ... there is a possible workaround (more on that later).
So, about the promised workaround... Have you noticed FreeBSD's dynamic linker makes no effort to differentiate native FreeBSD shared libraries from Linux (glibc) shared libraries? It merely complains about missing symbols on accidental mixup. More so, libmap.conf man page actually contains this curious usage example (a reference to the defunct linuxpluginwrapper port):
Code:
     # Glue for Linux-only EPSON printer .so to be loaded into cups, etc.
     [/usr/local/lib/pips/libsc80c.so]
     libc.so.6               pluginwrapper/pips.so
     libdl.so.2              pluginwrapper/pips.so
I cleaned up and dumped on GitHub my crappy attempt at a similar compat library targeting specifically Nvidia's Linux Vulkan ICD: https://github.com/shkhln/nvshim.
 

NuLL3rr0r

Active Member

Thanks: 14
Messages: 174

#9
@malavon thank you so much for the effort. UE4 game dev here. The only reason I moved away from FreeBSD on desktop was the lack of official support for UE4 on FreeBSD (I'm running Funtoo Linux on desktop at the moment). If I recall correctly there was another GitHub repo up to UE 4.16. But, it looks like you made a great progress. I should definitely find some time and check this out.

Good old FreeBSD + i3wm (+ UE4) would be an exciting development machine. Thank you and keep up the good work!
 
OP
OP
M

malavon

Member

Thanks: 21
Messages: 28

#10
I cleaned up and dumped on GitHub my crappy attempt at a similar compat library targeting specifically Nvidia's Linux Vulkan ICD: https://github.com/shkhln/nvshim.
I'm going to play around with this somewhere in the following days. It's really a great idea and if it allows using vulkan on FreeBSD with Nvidia cards it'll help me get that part of UE4 right as well.
Many many thanks shkhln for creating this!
 
OP
OP
M

malavon

Member

Thanks: 21
Messages: 28

#11
UE4 game dev here. The only reason I moved away from FreeBSD on desktop was the lack of official support for UE4 on FreeBSD (I'm running Funtoo Linux on desktop at the moment). If I recall correctly there was another GitHub repo up to UE 4.16. But, it looks like you made a great progress. I should definitely find some time and check this out.
Great to see there's a game dev in the forum. Wish I could say I was one myself, but who knows what the future holds.
If you test it out, it'd be great if you could report what actually works and what doesn't. Some features of the editor won't work (FBX import for one) and there are bound to be many like that since I don't test them.
Feel free to create issues in the GitHub, with guidelines on how to reproduce the problem and I'll take a look at it when I can spare the time.
 
OP
OP
M

malavon

Member

Thanks: 21
Messages: 28

#12
shkhln I've been able to use your shim to run with vulkan support! There seem to be some graphical issues (mainly with the UI),
not sure if they're caused by UE4 or the Nvidia driver.
Nevertheless I know now that it works and I'll try to find out if the issues are also present on other systems.

You might actually want to start another thread about this specifically, I'm sure it'll be interesting to more people than the ones interested in UE4.

Interesting note: framerate under vulkan more than doubles. I'm not sure if it's inherent to vulkan or that there are simply certain
graphical effects that are disabled and thus improving the framerate.
 

shkhln

Well-Known Member

Thanks: 94
Messages: 295

#13
shkhln I've been able to use your shim to run with vulkan support!
Glad to hear that! I'm now reviewing binary compatibility for structs I'm currently passing to FreeBSD's libc without conversion and I'm myself a bit surprised it actually works :) I'll fix it though.

Interesting note: framerate under vulkan more than doubles. I'm not sure if it's inherent to vulkan or that there are simply certain
graphical effects that are disabled and thus improving the framerate.
Twice the OpenGL framerate is clearly too much. Something must be botched with regard to that measurement.
 

shkhln

Well-Known Member

Thanks: 94
Messages: 295

#14
There seem to be some graphical issues (mainly with the UI),
not sure if they're caused by UE4 or the Nvidia driver.
Vulkan is still under heavy development, you should try the 396.x branch. (Don't download the driver directly, update the version number in the nvidia-driver port and recompute the checksum with make makesum.)
 
OP
OP
M

malavon

Member

Thanks: 21
Messages: 28

#15
Vulkan is still under heavy development, you should try the 396.x branch. (Don't download the driver directly, update the version number in the nvidia-driver port and recompute the checksum with make makesum.)
Tried these, no changes. That makes me think that it might still be a UE4 issue.
Also, it seems that vulkan uses the mobile renderer instead of the desktop one, supporting less features. This might explain why the framerate is so different.

Found a future task in the UE4 roadmap, it'll be interesting to see when this actually happens. I think it's been there quite some time already.
 
OP
OP
M

malavon

Member

Thanks: 21
Messages: 28

#16
Just a quick update. I try to keep up with releases made by epic, and that's also true for 4.20.1 and 4.20.2.
Both have freebsd ports as well, with 4.20.2 released (for FreeBSD) today.
 
Top