Ok, I just tested. I actually have xpdf set as default for chromium and I didn't want that, I wanted zathura. First, to answer my own question, I changed $HOME/.config/mimeapps.list to use zathura.desktop instead of xpdf.desktop.
I then went to /usr/local/share/applcations and copied xpdf.desktop to zathura.desktop, substituting zathura for xpdf where it occurred in the file. I then went to open a pdf in chromium and it used zathura. To test for the OP, i also tried with mupdf and it works.
So, to sum up.
Edit your $HOME/.config/mimeapps.list so that the application for pdf says mupdf.desktop instead of xpdf.desktop. Go to /usr/local/share/appplications/ and copy the xpdf.desktop file to mupdf.desktop. In that new mupdf.desktop file, anywhere that you see xpdf, change it to mupdf. You shouldn't have to restart anything for it to work. I don't have icons--I use either dwm or openbox. If you use a desktop that uses icons, I don't know what you would edit to change the desktop icon, but hopefully, it's not overly complicated.