How do the military, top-level governmental & financial institutions etc. handle this issue?
Let's look at typical "spooks", meaning government agencies, typically intelligence, law enforcement, and military.
Networking, traditional answer: There are no communication wires going into the data center. None, zip, zilch. Modern answer: the only communications wires going in and out use dedicated circuits that go to other data centers of the same organization, and use very strong hardware encryption on those circuits. For example, all major cloud/hosting providers (those that have more than one data center) encrypt their dedicated circuits that leave buildings, after they found that certain government agencies were wiretapping their circuits (yes, the NSA was spying on the likes of Amazon, Facebook and Google). The big ones in the industry all own their own cables (yes, the likes of Amazon, Google and Microsoft own their own terrestrial and undersea cables). If the site has to be connected to the public network, this will be done with carefully designed firewalls; the slang term for that is "spanning the air gap" (a bridge that goes across the air gap that separates internal and public networks).
Personal communications, traditional answer: while inside the building, nobody will have a cell phone, radio, or camera. The only laptops that can be brought in and out are those where WiFi can be turned off in hardware (many name-brand laptops have switches for that). All communication is via desk phones, usually some encrypted IP phone (they tend to look like Cisco hardware). Modern answer: Agency-issued cell phones can be brought into certain parts of the building, for example conference rooms, but typically not personally owned ones. Typically, there will be sniffers and jammers to prevent communication from happening. Forget Bluetooth and WiFi. And USB sticks are a complete no-no.
Access: At the gate, there is a set of guards. They have assault weapons. They check your credentials, and only let you in if your are authorized to enter. That includes personnel of the organization itself (so for example NSA employees who have a good reason to be in the data center, and 99.5% do not), and a very small set of employees from contractors. Typical contractors would be SRI, IBM, General Dynamics, Oracle, Lockheed-Martin, EMC. The term "contractor" explicitly does not include things like electricians and plumbers, that's handled by in-house staff. Everyone who has access to the building will have government security clearances, even contractors. The organization has its own emergency response systems (fire, internal law enforcement such as military police, ambulances).
Anecdote: My wife has worked at one of those government labs. Her office neighbor (who was quite elderly and of ill health) walked into her office slowly, and said: "I'm sorry to bother you, but I'm feeling very bad, I think I'm having a heart attack, can you please call an ambulance for me." My wife (who is knowledgeable about heart disease) obviously immediately grabbed the phone, and called the emergency number, telling them that someone is having serious heart problems. About half a minute later, she heard heavy steps in the hallway, and outside saw a military-camouflaged HumVee, and then a half dozen marines with machine guns got into her office. That's the organizations standard response too any emergency. Fortunately, some of them were trained as EMTs (paramedics). Another minute later, an ambulance showed up with sirens, and brought a stretcher into her office. Another minute later, the on-site staff doctor showed up. Her colleague did fine.
In some government sites, visitors who need to be present but have no security clearance can enter certain buildings, but are escorted. At Livermore for example, in certain buildings they are escorted by an armed security guard who brings a colored red light on a small stand with him, which he carries in the hallway, then places in front of whatever room the uncleared visitor is in. I hear that this is a rare occurrence, as these facilities typical have a "visitor center" near the entrance where non-cleared people can have meetings. Been there, done that. For example, the NSA today has a publicly accessible museum and gift shop (yes, random people without security clearance can buy a coffee mug or sweatshirt that says "National Security Agency", a few friends of mine have those). In the old days, the bigger vendors had their own secure rooms in these facilities. For example in mainframe days, a typical data center would have a little office for IBM personnel, where they could hang out, drink coffee, and store tools and frequently used field service parts.
In some installations (typically military intelligence), all people in the building are armed. Meaning a typical "sys admin" (today's term for that job description is "SRE") will have an assault weapon. For a military site, that makes some sort of sense: their intelligence data processing would be among the places at the highest risk for being attacked. Yes, I've been in (phone- and video-) meetings with people who are all in green uniforms, all armed, and all use assumed names, typical by alphabet in order they go around the table. We all laugh about that, when people get confused: "As my colleague Fred just said ... oops sorry, as my colleague Charlie just said, I got confused because he was Fred in yesterday's meeting ...".
Hardware: Only hardware that has been ordered and vetted can enter the building. I've heard horror stories about field service spare parts being delayed by days (while systems are down), because they need to be shipped to a receiving organization in a non-secure facility (where UPS and FedEx drivers can enter), then checked that they are the real thing, then internally shipped to the data center.
How does technical support work? Typically suppliers of hardware and software have a small set of field service people who are security cleared, and can work on the hardware. There are always protocols to be followed: Do not look at actual user data, do not communicate about the size, design or content of the system with your colleagues, do not take photographs, only take handwritten notes, and all handwritten notes need to be checked by a censorship organization before leaving the building. Typically systems are compartimentalized: The service person from IBM looking at servers obviously is not allowed to get near the Cisco networking hardware, and vice versa. Debugging under these circumstances is ... difficult.
Typically, no hardware ever leaves the building. Typically, sites have shredders, and hardware that is obsolete or broken is run through the shredder, and not sold used or returned to the vendor. If hardware has to leave to be diagnosed off-site (for example a disk platter needs to be put under a scanning tunneling microscope to see why so many read errors happened), special protocols are put in place (like a few engineers at the vendor will be specially security cleared). Some sites insist on all persistent storage devices (disks, flash) also implementing hardware encryption (SED or FIPS-140), although that's not actually very common.
By the way, most of the the large vendors of cloud and hosting services work pretty much the same way, except typically not with soldiers as security guards, and typically using subcontractors for things like plumbing and electrical work. This does not apply to small banks or small medical services though, they don't have the scale to be secure.
And just to be clear: I've never had a security clearance, I've never been in a production data center, but I've worked closely with folks who do this kind of stuff. For example being on the phone with them to help, while they're in a conference room with agency staff and a censorship person.
Final anecdote: In a previous job, we had a meeting (at my company's office) with a senior staff member from one of those agencies. Very smart, friendly, and intelligent person. He was kind enough to give me his business card, and it said something like "Government of Elbonia, Central Security Agency, Dr. Adam Bob, Strategic IT design planning", with the flag of Elbonia in the corner. But it had no address, phone number, e-mail, or fax!
Why would you consider a hoster? Cause it will save you a lot of money for things like mentioned above. And sure there is a price: trust.
