Author Topic: HPE storage crash killed ATO online services  (Read 2763 times)

0 Members and 1 Guest are viewing this topic.

Offline Suppressed

  • Hero Member
  • *****
  • Posts: 12,921
  • Gender: Male
    • Avatar
HPE storage crash killed ATO online services
« on: December 14, 2016, 02:54:03 am »
HPE storage crash killed ATO online services
Revealed: Agency scrambles to restore 1PB of data.

By Allie Coyne | Dec 13 2016 1:55PM

A HPE storage network that crashed on Monday caused the loss of 1 petabyte of data and the downing of the Australian Tax Office’s online services.

Early on Monday morning the ATO advised that its online portals, digital services and website were offline following an unspecified “hardware issue”.

The ATO confirmed to iTnews that the issue was triggered by the catastrophic collapse of a hardware storage solution provided by HPE. It is understood the ATO acquired two new HPE 3PAR SANs late last year.

The collapse of the SAN caused the loss of 1PB of data.

The issue was compounded by the failure of back-up systems which did not kick in immediately following the initial outage: iTnews understands the data corruption issues from the primary SAN were replicated onto the second SAN.

The ATO is currently in the process of rebuilding both and restoring from back-up, it is understood.

The tax office emphasised that no sensitive taxpayer data had been compromised.

Excerpt.  Read more at http://www.itnews.com.au/news/hpe-storage-crash-killed-ato-online-services-444490

 :facepalm2:
+++++++++
“In the outside world, I'm a simple geologist. But in here .... I am Falcor, Defender of the Alliance” --Randy Marsh

“The most effectual means of being secure against pain is to retire within ourselves, and to suffice for our own happiness.” -- Thomas Jefferson

“He's so dumb he thinks a Mexican border pays rent.” --Foghorn Leghorn

Oceander

  • Guest
Re: HPE storage crash killed ATO online services
« Reply #1 on: December 14, 2016, 03:53:12 am »
:facepalm2:

Offline Frank Cannon

  • Hero Member
  • *****
  • Posts: 26,097
  • Gender: Male
Re: HPE storage crash killed ATO online services
« Reply #2 on: December 14, 2016, 03:58:45 am »
I read the words. I know what the words mean. I have no idea what the article said.

Offline Suppressed

  • Hero Member
  • *****
  • Posts: 12,921
  • Gender: Male
    • Avatar
Re: HPE storage crash killed ATO online services
« Reply #3 on: December 14, 2016, 04:35:37 am »
I read the words. I know what the words mean. I have no idea what the article said.

Storage network crashed, and redundancy failed.

Lesson learned: backup is like voting... Backup early and often, and it's still futile!
+++++++++
“In the outside world, I'm a simple geologist. But in here .... I am Falcor, Defender of the Alliance” --Randy Marsh

“The most effectual means of being secure against pain is to retire within ourselves, and to suffice for our own happiness.” -- Thomas Jefferson

“He's so dumb he thinks a Mexican border pays rent.” --Foghorn Leghorn

Offline InHeavenThereIsNoBeer

  • Hero Member
  • *****
  • Posts: 4,127
Re: HPE storage crash killed ATO online services
« Reply #4 on: December 14, 2016, 06:45:08 am »
I read the words. I know what the words mean. I have no idea what the article said.

Imagine you kept a little black book with the names and numbers of all the hotties you had ever met.

Imagine you have a secretary (also hot, I did say "Imagine you have a secretary" after all) whose job it is to keep a second copy of your little black book in case anything happens to it.

One day, you spill coffee on your little black book, ruining all of the information in it.  Fortunately, your secretary has a second copy of all if that information.  Unfortunately, I said she was hot, not smart, and she dutilfully spills coffee on her copy as well.

Keeping a second copy is a useful way of protecting data.  But if the first copy is corrupted, and you simply replicate that corruption to your backup....
My avatar shows the national debt in stacks of $100 bills.  If you look very closely under the crane you can see the Statue of Liberty.

Offline Frank Cannon

  • Hero Member
  • *****
  • Posts: 26,097
  • Gender: Male
Re: HPE storage crash killed ATO online services
« Reply #5 on: December 14, 2016, 06:54:28 am »
Imagine you kept a little black book with the names and numbers of all the hotties you had ever met.

Imagine you have a secretary (also hot, I did say "Imagine you have a secretary" after all) whose job it is to keep a second copy of your little black book in case anything happens to it.

One day, you spill coffee on your little black book, ruining all of the information in it.  Fortunately, your secretary has a second copy of all if that information.  Unfortunately, I said she was hot, not smart, and she dutilfully spills coffee on her copy as well.

Keeping a second copy is a useful way of protecting data.  But if the first copy is corrupted, and you simply replicate that corruption to your backup....

Well at least I still have the hot secretary.

Offline InHeavenThereIsNoBeer

  • Hero Member
  • *****
  • Posts: 4,127
Re: HPE storage crash killed ATO online services
« Reply #6 on: December 14, 2016, 07:22:53 am »
Well at least I still have the hot secretary.

I'm not seeing any photo evidence of that.
My avatar shows the national debt in stacks of $100 bills.  If you look very closely under the crane you can see the Statue of Liberty.

Offline Frank Cannon

  • Hero Member
  • *****
  • Posts: 26,097
  • Gender: Male
Re: HPE storage crash killed ATO online services
« Reply #7 on: December 14, 2016, 07:48:26 am »