Why US government IT fails so hard, so often

[EC]

A long and very good and in depth analysis on why the government can't get computer systems right.

The rocky launch of the Department of Health and Human Services' HealthCare.gov is the most visible evidence at the moment of how hard it is for the federal government to execute major technology projects. But the troubled "Obamacare" IT system—which uses systems that aren't connected in any way to the federal IT infrastructure—is just the tip of the iceberg when it comes to the government's IT problems.

Despite efforts to make government IT systems more modern and efficient, many agencies are stuck in a technology time warp that affects how projects like the healthcare exchange portal are built. Long procurement cycles for even minor government technology projects, the slow speed of approval to operate new technologies, and the vast installed base of systems that government IT managers have to deal with all contribute to the glacial adoption of new technology. With the faces at the top of agency IT organizations changing every few years, each bringing some marquee project to burnish their résumés, it can take a decade to effect changes that last.

That inertia shows on agency networks. The government lags far behind current technology outside the islands of modernization created by high-profile projects. In 2012, according to documents obtained by MuckRock, the Drug Enforcement Agency's standard server platform was still Windows Server 2003.

Magnifying the problem is the government's decades-long increase in dependency on contractors to provide even the most basic technical capabilities. While the Obama administration has talked of insourcing more IT work, it has been mostly talk, and agencies' internal IT management and procurement workforce has continued to get older and smaller.

Over 50 percent of the federal workforce is over 48 years old—and nearly a quarter is within five years of retirement age. And the move to reliance on contractors for much of IT has drained the government of a younger generation of internal IT talent that might have a fresher eye toward what works in IT.

But even the most fresh and creative minds might go numb at the scale, scope, and structure forced on government IT projects by the way the government buys and builds things in accordance with "the FAR"—Federal Acquisition Regulations. If it isn't a "program of record," government culture dictates, it seems it's not worth doing.
Hell is a 1.5 million user installed base

Slow technology adoption is nothing new. In 1992, I was working as a contractor for the US Army Test Lab at Aberdeen Proving Ground, installing a batch of new PCs purchased under the Department of Defense's Desktop IV contract from Unisys. The PCs, built by Unisys specifically for the contract, came loaded with Windows 3.1; it was my job as the network engineering support manager to ensure that the hard drives were reformatted and loaded with Microsoft DOS 3.31—the standard supported operating system.

The hazards of running an operating system more than 10 years old, however, go beyond having a pretty user interface. The government has kept using Windows XP and Server 2003 despite warnings from the National Security Agency that "Windows XP and Windows Server 2003 lack critical security features and are near the end of their extended support lifecycle." Still, the federal government (much like most of the business world) largely took a pass on Windows Vista. And even though the National Institute of Standards and Technology added Windows 7 to the US Government Configuration Baseline a year after its release, most agencies didn't start migrating off Windows XP until 2011 or later. While the Army and Air Force had adopted Windows Vista, XP was still fairly widespread when the Army began migrating to Windows 7 in July of 2012.

More at link: http://arstechnica.com/information-technology/2013/10/why-us-government-it-fails-so-hard-so-often/

[Oceander]

I really liked XP - particularly the fact that it stuck around for so long and was a very mature OS - but it was leaky as h*ll in anything other than a tightly-controlled enterprise environment and I cannot fathom running anything internet-accessible on it other than a personal website run just for funsies.  I have an ersatz home server I built on top of an old XP-based desktop, but it never, ever gets to see the internet, or anything outside of my local intranet.

[ABX]

If you want to see an absolute mess that looks like first year programmers hacked together on a drunken weekend by copying and pasting chunks of script they found on a dozen different sites, go to healthcare.gov > right click > view page source.

[truth_seeker]

I was appointed to manage a study for a multinational corporation's "new" Payroll-Personal system. We were to start with domestic entities, and later to international locations.

I had no experience, so they first sent me to a seminar at IBM Hq. in NYC. I was chosen, because I was from the Finance user side. Other users, IT Dept. leads, were under me. We also retained a few "experts" from Arthur Andersen (a complete story in itself).

It was highly political, with each entity wanting to play a big role, as opposed to a top down approach. After a couple of years, we had largely failed.

System development to replace existing systems is really hard to do. We  investigated off-the-shelf systems from leading companies of the era, like Peachtree. But each was found not sufficient for the specifics of our business. Did I say we failed, I failed.

A few years when I was gone, a friend was given a similar assignment for Financial systems. He failed, too.

I hated it, and wanted to be done with it. I can imagine doing it in a government environment is beyond comprehension, considering the infighting, responsibility avoidance, risk aversion, etc.

Obama and his supporting team of hyper-political flunkies have their hands full. They picked an inexperienced firm, but the system architecture is now set.

They will probably patch the turkey, tell us it is running fine, but it won't be fine.

[ABX]

.....and the idiots didn't lock down their databases. Personal data can be accessed.

https://data.healthcare.gov/dataset/Navigators/qyne-xyvd

Pretty much all the raw data that goes through the site can be accessed via here:

https://data.healthcare.gov/

[truth_seeker]

I'd like to listen in to Obama lecturing the heads of major health insurance firms.

[Rapunzel]

I'd like to listen in to Obama lecturing threatening the heads of major health insurance firms.

Fixed it - this is more like it

[Rapunzel]

Our IT was in-house.  Everytime they tweaked something we ended up with more holes than you could shake a fist at. One time error reports stopped generating.  Everyone was giving themselves high-fives - gee we're doing such a great job, no one is making any mistakes.  I insisted this was not possible that something was wrong.. IT insisted nothing was wrong -- finally after I kept making a fuss they went in and looked and oops... they had a stop point in the system for errors and we'd reached it so .. ta da... quit kicking out error reports there was errors alright, just in a black hole.

Then wesuddenly had some major inventory out of balance and I insisted something had changed in the system and it was not calculating correctly between the different journals at month-end...  again they said "not possible."   I finally set down and literally tracked the movement of individual part numbers from beginning to end and proved to them that one of the journals which was supposed to be acting as a credit to inventory was debiting inventory so inventory was getting debited twice for the parts...  one thing about IT - their system tweaks are NEVER the problem  /s