Considering the large number of service ports you're likely scanning, and the fact that hosts on the receiving ends of your scan may be dropping tcp connect requests... you could be waiting awhile.
There is a "timing template" option that allows you to tweak timeout behavior, but that may be a can of worms in of itself depending on what you're doing and whom you're scanning.