Jump to content
chrisg

This Census

Recommended Posts

Well apparently adequate load testing was performed by RevIT. They ran it at 150% capacity for 8 hours and the system "didn't even flinch."

 

 

150% of an unstated number is an unstated number.

 

We don't know what that means, so it's a worthless claim by the ABS.

What distribution of the millions of entries over what time period.

What modelling of DOS/hack attempts.

What modelling of equipment failure.

What modelling of sustained overload vs peak overload was modelled. 150% overload for 8 hours is a fine is probably a fine test of sustained testing. Completely worthless as an test of surge. It's the nature of transaction systems that once they hit a certain point, they stop degrading performance and fail spectacularly. Almost every architecture has some degree of positive and negative feedback . Basic control systems theory - Negative feedback damps runaway, positive feedback will either lead to oscillation or runaway. Surge it for long enough, and the positive feedback overtakes the negative feedback for so long that it reaches a tipping point, then it's usually uncontrolled failure towards splat.
You can stick your had in the sand and deny positive feedback is present, or you can accept it, model it, and build a system to handle it for known levels and duration. Simple fact is their system failed, because they did not correctly model/test the scenario. the 150% test they claim is nice for one scenario, but they made no claim of having covered the other scenario - surge driving positive feedback > negative feedback.

What was weird is the conflicting definitions of when it's supposed to be done. The old days of 'Census night' and even using that term in the current mail-out materials implied doing it after dinner on Tuesday. That's going to pretty much drive the entire country to log on around 7pm-9pm. But apparently it was always going to be fine if you got in in over the next couple of days. so some simple PR and advertising to break down the 'Census night' myth might have changed their requirement significantly.

Share this post


Link to post
Share on other sites

Those videos they made with that weird looking bot thingy told you so little about how to fill out the form.

Share this post


Link to post
Share on other sites

Simple fact is their system failed, because they did not correctly model/test the scenario.

Underlined part correct, remainder is conjecture.

 

For all you know, they tested to [expected capacity]*1.5 but the alleged attack imposed [expected capacity]*3 or *5 or *20.

Edited by SquallStrife

Share this post


Link to post
Share on other sites

If you go check out the ABC News Twitter feed, Michael McCormack explains the situation in an 11 minute video. I can't find a link to the video anywhere other than Twitter.

 

Go watch that, then discuss :)


https://twitter.com/abcnews


@@abcnews

 

 

How do you embed this shit below into the forums?

Anyone? It's a link to the actual video.

 

<blockquote class="twitter-video" data-lang="en-gb"><p lang="en" dir="ltr">Minister responsible for <a href="https://twitter.com/hashtag/census2016?src=hash">#census2016</a> <a href="https://twitter.com/M_McCormackMP">@M_McCormackMP</a> explains ABS online form outage <a href="https://t.co/XKiGAcXddL">https://t.co/XKiGAcXddL</a></p>— ABC News (@abcnews) <a href="https://twitter.com/abcnews/status/763178528124776448">10 August 2016</a></blockquote>
<script async src="//platform.twitter.com/widgets.js" charset="utf-8"></script>
Edited by twinair

Share this post


Link to post
Share on other sites

If you go check out the ABC News Twitter feed, Michael McCormack explains the situation in an 11 minute video. I can't find a link to the video anywhere other than Twitter.

 

Go watch that, then discuss :)

Summary - 3 things happened.

 

First, a DoS attack which lead to a hardware failure (router overloaded) and that was subsequently followed by a 'false positive', in their monitoring systems. Those three events in a short timeline lead the ABS to shut the system down to ensure the data collected was safe.

 

That's my quick summary of what he said anyway :)

Share this post


Link to post
Share on other sites

That's about right as far as the summary goes.

 

What I found interesting is their strong denial of an attack, yet still used the term DoS.
DoS does not equal hack.

DoS does equal attack, though. Generally speaking.

 

So if they're saying a DoS occurred but it wasn't an attack, what is one left to assume other than it was just overloaded?

Unless they're just trying to get everyone to calm their farms. It's okay people, it was just a silly DoS, wasn't an attack!
Attack is such a negative word, let's not use it!


I'm actually dropping a D now. I see no distributed denial of service attack, if the attack maps are anything to go by.

I'm still very much intrigued as to exactly what happened. I fear we will never know the actual technical details.

Edited by twinair

Share this post


Link to post
Share on other sites

Makes one question the "load testing" done!

Any app and it's ecosystem needs to be tested against functionality, performance and security. Does the app behave as users expect? Is it consistent? Does it provide the right functions to meet it's purpose?

Can it perform? What's an acceptable level of performance given the expected load? And is the data and user's privacy secure?

 

The census site would have been tested against these three groups of requirements.

 

A DoS attack initially impacts on the performance of a site by gumming up the works. If this causes a failure of any sort then it also impacts on the functionality by reducing it to no functionality. As for security, a DoS attack doesn't necessarily cause a failure in this area.

 

There are lots of questions.

 

IBM & RevIT have the expertise to handle such a project which only makes the failure more curious.

 

Grab a big box of popcorn, it could take a while for the facts to leak out :)

 

What I found interesting is their strong denial of an attack, yet still used the term DoS.

DoS does not equal hack.

DoS does equal attack, though. Generally speaking.

Yeah. he said there was no attack or hack then said there was a DoS attack :/

 

I think he is trying to calm the horses and didn't use the best terminology.

Edited by Mac Dude
  • Like 1

Share this post


Link to post
Share on other sites

Sounds like an IT stuff up.

They load tested it for 1 million users per hour, but failed to realise that peak loading would be over 10 million users per hour during the 'the kids are in bed, lets do this stupid census crap, so we don't get fined' window of about 2 hours between 8pm and 10 pm.

A DDOS attack wouldn't have helped things either but they should have expected that with all the media publicity about the privacy issues the changes in the census was raising.

Edited by sebbyreddan

Share this post


Link to post
Share on other sites

Bit of bullshit flying around. In that first linked 11 minute video he's talking of overseas traffic being blocked to stop further Dos.

And in that context you instantly know the noob in the room because they automatically think Dos from overseas means the people instigating it are there as well.

  • Like 2

Share this post


Link to post
Share on other sites

 

What I found interesting is their strong denial of an attack, yet still used the term DoS.

DoS does not equal hack.

DoS does equal attack, though. Generally speaking.

Yeah. he said there was no attack or hack then said there was a DoS attack :/

 

I think he is trying to calm the horses and didn't use the best terminology.

 

It wasn't an attack, it was just a disruption!

 

Share this post


Link to post
Share on other sites

Bit of bullshit flying around. In that first linked 11 minute video he's talking of overseas traffic being blocked to stop further Dos.

And in that context you instantly know the noob in the room because they automatically think Dos from overseas means the people instigating it are there as well.

What do you make of these DoS heat maps? Bling or actual factual?

If they're to be believed, there was no DoS.

 

Fuck knows what is going on.

Share this post


Link to post
Share on other sites

Attack maps are generally pretty accurate, the security industry puts a lot of effort into them.

 

Cheers

Share this post


Link to post
Share on other sites

Attack maps are generally pretty accurate, the security industry puts a lot of effort into them.

 

Cheers

Okay cool.

 

 

 

Could it be a DoS from locals?

Well, potentially. But then you have Michael McCormack saying they were blocking international traffic, yet the maps showed zero international DoS activity.

 

 

So if these maps are to be believed, then we are now (speculation, once again) looking at a coverup for poor planning and implementation.

 

 

Who the fuck knows.

 

Share this post


Link to post
Share on other sites

Could it be a DoS from locals?

Yep that is what it was in the form of a large chunk of the population all trying to do their Census at the same time.

Share this post


Link to post
Share on other sites

 

Could it be a DoS from locals?

Yep that is what it was in the form of a large chunk of the population all trying to do their Census at the same time.

 

L

:)

 

Exactly, but that was just sooo unexpected... :)

 

Cheers

Share this post


Link to post
Share on other sites

 

Attack maps are generally pretty accurate, the security industry puts a lot of effort into them.

 

Cheers

Okay cool.

 

 

 

Could it be a DoS from locals?

Well, potentially. But then you have Michael McCormack saying they were blocking international traffic, yet the maps showed zero international DoS activity.

 

 

So if these maps are to be believed, then we are now (speculation, once again) looking at a coverup for poor planning and implementation.

 

 

Who the fuck knows.

 

 

Yeah good point, as I said yesterday I couldn't access the site through a VPN, not sure how that would affect hackers and ddos attacks? (They probably have a way around it?)

Share this post


Link to post
Share on other sites

 

I'm actually dropping a D now.

 

cant say i am that alarmed about it.

 

but yeah, it is concerning.

 

I drop Ds all the time. I gag for them.

I need D on a daily basis. Some times, hourly.

 

Edited by twinair

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×