I'm not sure if it has recently changed, but Xfce4 did use the gstreamer framework and users needed to also install the appropriate (oss or pulseaudio) gstreamer plugin. I do not have Xfce4 installed, so I would check your installed packages for the gstreamer version and try installing the gstreamer-pulseaudio-plugin.
