If it is not set, then it uses the default load library path. So to add something else to it, the common syntax is: "export LD_LIBRARYPATH=${LD_LIBRARY_PATH}:/mnt/my_usb_stick/lib/libfoobar.so" (that example is in sh / bash syntax, it will be different for other shells). If it hasn't been set yet, you end up with ":/mnt/my_usb_stick/...", which is IMHO the correct way to set it: search standard paths first, then my own.
EDIT: No, don't do that, it exposes you to dangerously reading shared libraries from the current directory. Read the article that SirDice linked to below for how to do it right (with an if statement or a conditional string statement in the shell).
Look at "man ldconfig", it explains it.