Pages

Friday, January 29, 2010

Technical analysis of the bank america downtime.

Bank of Americas most recent downtime has stirred up the internet with all kinds of questions about what is really going on.  Everything from speculated ddos(distributed denial of service) attacks to some stating that Bank of America claims to be upgrading RAM.

Lets look at the issue from a technical standpoint and see what we can come up with.

This blog post is being updated in real time as I gather more information.  If this was a ram upgrade Bank of America would have had zero to minimal downtime(short spurts) as thier web servers operate in a load balanced/clustered setup.  This means that the website is hosted on many servers spread out so if one goes down, other servers take on the load of the down server.  So if they were increasing ram to prevent downtime it would be best to do this server by server,  letting the other servers pickup the load while each server gets upgraded.  So is this downtime caused by a ram upgrade?  I think not and if this is the case Bank of America should not be allowed to be a bank until they get a more competent I.T team.

So if that is not the case….is it in fact an attack?  Let’s look at the trace route.

root@web01 [~]# tracert -T www.bankofamerica.com
traceroute to www.bankofamerica.com (171.161.161.173), 30 hops max, 40 byte packets
 1  xxxxx
 2  xxxxx
 3  xxxxx
 4  xxxxx
 5  xxxxx
 6  ae-3.ebr3.Dallas1.Level3.net (4.69.132.78)  37.095 ms  35.937 ms  35.905 ms
 7  ae-4-90.edge5.Dallas1.Level3.net (4.69.145.202)  33.926 ms ae-2-70.edge5.Dallas1.Level3.net (4.69.145.74)  34.175 ms ae-3-80.edge5.Dallas1.Level3.net (4.69.145.138)  34.002 ms
 8  BANK-OF-AME.edge5.Dallas1.Level3.net (4.78.230.2)  35.708 ms  35.525 ms  35.518 ms
 9  171.161.191.248 (171.161.191.248)  35.961 ms  35.776 ms  35.467 ms
10  www.bankofamerica.com (171.161.161.173)  35.971 ms  36.069 ms  35.871 ms
11  www.bankofamerica.com (171.161.161.173)  36.157 ms  35.881 ms  35.964 ms
12  www.bankofamerica.com (171.161.161.173)  35.643 ms  35.382 ms  35.510 ms
13  www.bankofamerica.com (171.161.161.173)  35.869 ms  36.331 ms  38.235 ms
14  www.bankofamerica.com (171.161.161.173)  38.507 ms  35.909 ms  36.351 ms

Judging by this trace route I would have to say no this is not a DDOS attack.  How DDOS attacks work is they take the resources of many many computers and flood all those resources at a single target if this was the case we would not be getting a response back from Bank of Americas website (we could have also checked this with ping but if its a ddos its best to get a ping response from the upstream router to determine the attack size vrs pipe into the router)  or if we did get a response it would be very lagged,  these times are normal.  However one thing I did notice between this trace route and one I did earlier in the day is that I got more responses from Bank of America so it looks like they added more computers into the load balanced environment to combat whatever is going on.  It could mean they had a major surge of traffic today which is bad because its common knowledge to have resources to serve 75% more traffic than your average traffic and be able to serve 50% more than your average peak traffic.  And it’s unlikely a large organization like Bank of America would not have any idea of the amount of traffic it gets or to expect.

Could it be a targeted DOS attack? With the response times from the webserver being over 8-15 seconds(20-50 milliseconds being normal) It is very possible that some kind of resource starvation attack could be used to spike the cpu usage up to 100% which would produce very similar results to what is being seen.

Many customers have found ways to get around the issues by accessing the website via the nodes directly.  Connecting to the mobile site etc etc it is a mix and match as to what will work as users flood to newly discovered entrances into the online banking portion of the site and in turn only increase the load on the banking servers making it harder on the I.T staff to isolate problems.

Another speculation is a cyber attack/hack.  While this is very possible,  it is scary and unlikely that a bank would leave any part of its contaminated infrastructure on both the internet or on their own private network which makes me think that this is not a hack attempt/cyber terrorism attack on Bank of America.  However as you have seen in the post right under this one,  Cyber terrorism attacks are being brought to a whole new level of complexity and rather those attacks can succeed or not depends on the intelligence and training level of everybody who is a part of the company being attacked,  so I am not quite ready to write off that this is not a cyber terrorism attack off quite yet..

So what is the cause of the website being down?  It could be many different things and I would have to be onsite to figure it out.





More to come later as I do more tests.




Ill be fighting to legalize freedom tell the day I die.

Friday, January 15, 2010

The chinese hacking attacks on google, adobe etc etc

Well it seems to me that this is a wake up call to ANY AND ALL major backbone corporations.
Its time to rethink security.
There never will be any policing the internet at least not effectively,   the only great policing we have for our networks is to block large portions or even entire countries from accessing networks at backbone levels. and even then this is simple for anyone to get around.  Internet2 is just flat out designed wrong and provides central points of failure from the exact same kind of corporations that have just been attacked.  These attacks from the chinese government on cybersitter, google, adobe and all the others is a major wake up call.  The sophistication of these attacks is much beyond the target a server and scan the ip for vulnerabilities  The attackers knew what operating system,  and what browser version the computer they was attacking,  the email was crafted to that person and was able to get them to a: click a link, b: exploit the vulnerability or c: get them to open a file attachment that has the payload and attack. The emails also appeared to be from coworkers i do believe.  With attacks this targeted,  I want you to stop and think,  if you run a large corporation with thousands of employees each with their own email address,  their own computer, a vpn connection from their home to your network. Everything being a gateway to your data  and just about everything else in your company. This leaves you as vulnerable as the security of each individual employee.   Every company should have a very strong security policy for both technical level and the workplace.  While these attacks are very sophisticated attacks,  that does not mean they are hard to pull off.
China and these other places cannot be allowed to condone business this way in stealing information The fact of the matter is,  most of the botnets you hear about on the news are 80% asian computers.  and the reason those numbers are so high is because the economy there cannot afford better computers,  so they are stuck with some very old insecure computers usually running pirated versions of windows.  or old 486`s running linux in these tiny datacenters all over the place.  i remember back in the day when hacking was in the scan and hack days when people targeted the 211.x.x.x range(korea and whatnot)  because it had more insecure networks than any other range on the internet,  and still leads true to this day judging by all the recent ddos attacks i have had the fun experience of trying to stop this year working for various places.  So basicly what im getting at is china the u.s and major corporations and other big entities have placed themselves in a position to where there entire infrastructure can be compromised by only a handful of people.  Do we really want this out of our leaders?  Google has the biggest database on everything in the world,  Adobe controls software installed on a very very large portion of the internets computers(shockwave flash).  Oracle they make database software not really to sure why they was attacked only thing that comes to mind would be the fact they can stream updates to every company that uses there database software(that number is massive) and posibly allow remote attackers to grab any database from any company receiving the updates or possibly stream a trojan with the update and have full access.  There are more companies that was attacked these companies need to come forward and let people know what these attackers are after, what is in place to protect it?. Till that happens it is impossible for other companies to harden there own security policy.

Ill be fighting to legalize freedom tell the day I die.