Go for it man. If you can do it, I'd try it. I don't know though, sounds like there's some big hurdles to overcome. For one there's no Kodi Leia port. Hardware video acceleration would probably be out of the question since it relies on a blob from AMLogic. They barely have any interest in the one for Linux, let alone FreeBSD.
I don't know how much those CoreELEC guys get down to the coding level, so yeah, they might just be naysaying. Seems they mostly just package things, more like techs than engineers. Consider all that stuff is already out there for Linux, just needs to be put into a package. For example the kernel comes from HardKernel which gets their video drivers from AMLogic. The "just enough Linux" userland is based on Busybox which again is already out there.
It might be possible to run stuff under FreeBSD's Linux emulator, might save some work, but I don't know how feasible that is.