TECH Massive AWS outage
- Thread starter Macgyver
- Start date
Hfcomms
EN66iq
We've been talking about it on Fung's thread today.
www.timebomb2000.com
CRISIS - FUNG ADVISORY: ***Incoming Economic Collapse***
Mainstream Economists Are Struggling To Hide The Incoming Economic Collapse TUESDAY, DEC 07, 2021 - 08:16 AM Authored by Brandon Smith via Alt-Market.us, For many years now there has been a contingent of alternative economists working diligently within the liberty movement to combat...
IceWave
Veteran Member
bassaholic
Veteran Member
Amazon down for thousands worldwide
An outage at Amazon Web Services (AWS) is causing cascading problems across the internet, with numerous websites displaying a “bad gateway” error. Downdetector shows problems started shortly after 8 am Pacific time.
Amazon down for thousands worldwide
7 Dec, 2021 16:45
Get short URL
© Global Look Press / Pavlo Gonchar
An outage at Amazon Web Services (AWS) is causing cascading problems across the internet, with numerous websites displaying a “bad gateway” error. Downdetector shows problems started shortly after 8 am Pacific time.
There have been over 20,000 outage reports for the main Amazon site and 11,000 for AWS. The Downdetector heatmap shows the problem is most acute in the eastern US.
As a lot of web infrastructure uses AWS to operate, outages have been reported on other services, from the McDonald's app and Disney+ streaming service, to the payment network Venmo.
Dating app Tinder, sports gambling app FanDuel, and Facebook are also showing spikes in outage reports.
By 8:26 am Pacific Time, Amazon said it had identified the cause of the outage as “API and console issues” in the eastern US region. The global console landing page is also located in the same region, and was likewise affected. The company said it has “identified the root cause” and was “actively working towards recovery.”
bassaholic
Veteran Member
I placed a couple orders an hour ago, still not showing up in my orders.
onetimer
Veteran Member
API Error Rates in US-EAST-1
[9:37 AM PST] We are seeing impact to multiple AWS APIs in the US-EAST-1 Region. This issue is also affecting some of our monitoring and incident response tooling, which is delaying our ability to provide updates. We have identified the root cause and are actively working towards recovery.
[10:12 AM PST] We are seeing impact to multiple AWS APIs in the US-EAST-1 Region. This issue is also affecting some of our monitoring and incident response tooling, which is delaying our ability to provide updates. We have identified root cause of the issue causing service API and console issues in the US-EAST-1 Region, and are starting to see some signs of recovery. We do not have an ETA for full recovery at this time.
[11:26 AM PST] We are seeing impact to multiple AWS APIs in the US-EAST-1 Region. This issue is also affecting some of our monitoring and incident response tooling, which is delaying our ability to provide updates. Services impacted include: EC2, Connect, DynamoDB, Glue, Athena, Timestream, and Chime and other AWS Services in US-EAST-1. The root cause of this issue is an impairment of several network devices in the US-EAST-1 Region. We are pursuing multiple mitigation paths in parallel, and have seen some signs of recovery, but we do not have an ETA for full recovery at this time. Root logins for consoles in all AWS regions are affected by this issue, however customers can login to consoles other than US-EAST-1 by using an IAM role for authentication.
[12:34 PM PST] We continue to experience increased API error rates for multiple AWS Services in the US-EAST-1 Region. The root cause of this issue is an impairment of several network devices. We continue to work toward mitigation, and are actively working on a number of different mitigation and resolution actions. While we have observed some early signs of recovery, we do not have an ETA for full recovery. For customers experiencing issues signing-in to the AWS Management Console in US-EAST-1, we recommend retrying using a separate Management Console endpoint (such as https://us-west-2.console.aws.amazon.com/). Additionally, if you are attempting to login using root login credentials you may be unable to do so, even via console endpoints not in US-EAST-1. If you are impacted by this, we recommend using IAM Users or Roles for authentication. We will continue to provide updates here as we have more information to share.
[2:04 PM PST] We have executed a mitigation which is showing significant recovery in the US-EAST-1 Region. We are continuing to closely monitor the health of the network devices and we expect to continue to make progress towards full recovery. We still do not have an ETA for full recovery at this time.
mistaken1
Has No Life - Lives on TB
Someone fat fingered the CIDR on a router change .... LOL
Ta-wo-di
Veteran Member
It's also affecting other platforms that use their services. We use ConnectWise Manage for ticket scheduling and tracking. It went down about 2:30 this afternoon and is still down. I figure its their turn to be hit by DDoS. We were affected back in September by a DDoS on our VoIP provider that we resell.
Macgyver
Has No Life - Lives on TB
Apparently that's unaffected. But I did read the orders won't show up in your history that you just placed.
The DoD and the clowns in America both use AWS as their platform. This is more than a disruption of Christmas shopping orders. Somebody is testing something
The Snack Artist
Membership Revoked
Asking if they still smell like urine will do that.
TxGal
Day by day
Interestingly, I got one of those scam amazon emails yesterday. One of those 'your account has been suspended' fake things and the email address wasn't from amazon.com. The timing sure is interesting.
Meemur
Voice on the Prairie / FJB!
Is the problem fixed, yet?
I'm having issues at one of my jobs due to this mess. The goofball IT guy is no help at all.
Last AWS report:
[4:35 PM PST] With the network device issues resolved, we are now working towards recovery of any impaired services. We will provide additional updates for impaired services within the appropriate entry in the Service Health Dashboard.
I'm having issues at one of my jobs due to this mess. The goofball IT guy is no help at all.
Last AWS report:
[4:35 PM PST] With the network device issues resolved, we are now working towards recovery of any impaired services. We will provide additional updates for impaired services within the appropriate entry in the Service Health Dashboard.
bassaholic
Veteran Member
I think this all has to do with testing for when they shut things down, you know, due to a cyber hack.
Knighttemplar
Veteran Member
I'm going to make a prediction and give you the reason why. You will see more and more of these outages. The networks use to be hardware(custom chips) with some software, now the network is mostly software run on generic hardware. Said another way the custom chips ran custom software that was well tested now the work is done by generic server hardware running linux with some custom software that is still in the infant stage. Very complex programing both on the engineer side and the backend, so many places for something to go wrong or for unintended consequences.
meandk0610
Veteran Member
As with all things, if you don’t own it, it’s not really yours. IMO, businesses that rely on their data should own their own in-house servers.
Meemur
Voice on the Prairie / FJB!
True! But I think our problem had to do more with the 3rd party aps that AWS runs. We had some functionally. I could open files, but they were "read only," so I couldn't edit or save the changes.
In any case, I ran hard copy before I left so that we're not totally up a creek in the morning if this problem isn't fixed.
I didn't even know we had anything to do with AWS. I though our stuff was all Microsoft crap but it isn't.
Jez
Veteran Member
I've said this for years. But corporate types only see the money saved and hope an outage is short and cheap. When things go TU they suddenly get all panicy and never want to discuss the path that led there. It's always interesting how they're not interested in blame if the path leads to their door.
Seeker22
Has No Life - Lives on TB
Why are they not using something military grade?
Jez
Veteran Member
Off the shelf is cheaper and easier to maintain. The "good stuff" isn't likely to be shared around the various agencies.
Don't think of the government as one big corporation that has a centralized IT infrastructure. Each agency is more like a little kingdom and they are not likely to share their toys. More than likely each agency thinks the others are incompetent and wouldn't deserve access to the good stuff anyway.
Besides, if it was all centrally managed it would be easier to breach because everyone would use close to standard equipment. Then there would be the "fun" part of having one big contractor who had the complete keys to the kingdom.
The hacks they have suffered (Info stealing) over the last ten years would scare you. Some are public if you search for them, most are not.
I'd call that progress.
So..wondered if this was related or coincidence, and if anyone else experienced it. My regular USPS mail delivery was about 5 hours late yesterday. Yes, she did have a package for me from Amazon. Hope the USPS boots-on-the-ground isn't also intertwined with these systems. Good grief.
Don't know. I've heard it from a number of sources though. Probably high end skills from the staff and cost.
Regarding AWS last night... Usually my fire tablet will go to sleep after a short time. Last night it didn't so I closed the cover. It went to sleep. A few minutes later I heard screaming in the room. I looked over and could see that the tablet was on even though the cover was closed. I jumped out of bed to check it out. The tablet was playing some Amazon prime horror movie. I powered the tablet down completely. Ten minutes later it came back on with Alexa loudly asking "are you there? Hello. Are you there?" My tablet has a demon. I put it in another room.
wvstuck
Only worry about what you can control!
Look at the systems that have had hiccups or problems over the past year or two. If every hack leaves behind an invisible switch hidden in a little code that is never discovered, then one day the switch is triggered and nearly every system goes down at the same time. no banking, no surfing, no TB2k, no medical records, no utilities, no anything would work. Game over in one day.
I just don't think there are unrelated events.
I just don't think there are unrelated events.
pauldingbabe
The Great Cat
Holy water and fire just to be sure. Sweat Lodge for four days. Plant a tree.
You'll be fine.
