It seems to me that if you have PF in the kernel, then VIMAGE is extremely unstable when tearing down the vnet network, and it will almost always panic when the jail is killed.
But I just commented out the PF part in my kernel config, (basically all that is left is just VIMAGE, NULLFS and include GENERIC now) and while I still see the Freed UMA like described above, it finally does not panic anymore.
I've tried both epair, using a heavily modified jail script, and netgraph using the vimage script from DruidBSD. While I like the simplicity of epairs, the sequential numbering annoys me, whereas with netgraph the interfaces are named using the jail name, nicer and simpler.
But anyway, for me, panics went away after removing PF from the kernel. It wasn't supported in 2010 and apparently still isn't properly working.