Google cloud outage takes down banks

(neowin.net)

100 points | by paweladamczuk 2 hours ago

17 comments

  • nickcw 1 hour ago
    From Google's status page https://status.cloud.google.com/incidents/Xs2XTHzAVC9YnafnPy...

        We are experiencing an issue with multiple services in
        europe-west2 beginning on Monday, 2026-10-19 23:08 PDT.
        
        Due to a power event we had suffered an outage within
        europe-west2-b zone. The power event is mitigated. Partial
        mitigation to infrastructure services will be observed within the
        next 30 minutes and the remaining services will be mitigated there
        after.
    
    So some kind of power failure happened to bring down one zone. Unfortunate, but this kind of stuff happens. That is why all the cloud providers have multiple zones, and I'm amazed the big companies reported as down weren't using multiple zones.

    (Also reporting the times in PDT is quite unhelpful for users of this zone who are likely based in the UK or EU)

    • crumby 1 minute ago
      Often there are irritating legal geographic limitations on where cloud hardware can exist for a given country. I can understand why, but there is this particular drawback of having all eggs baskets close together.
    • bsuvc 56 minutes ago
      I'm not saying it's the case here, but sometimes the cloud providers themselves have inter-zone dependencies, or even inter-region dependencies that can cause outages in other zones and regions.

      For example, AWS us-east-1 outages have caused global outages on multiple occasions, such as https://www.networkworld.com/article/4168878/aws-hit-by-us-e...

    • HelloUsername 5 minutes ago
    • robinson-wall 1 hour ago
      > I'm amazed the big companies reported as down weren't using multiple zones

      Much as with backups - if you don't test them, they don't exist.

      Everywhere I've worked has had multi-AZ infrastructure. I have never worked anywhere where the failure of an AZ has been simulated regularly, with second-order effects mapped out.

    • cavoirom 31 minutes ago
      I know some banks out-sourced most of its infrastructure to external operators, and some operator I worked with didn't seem competent.
  • nfrankel 3 minutes ago
    Migrate to the Cloud, they said. It will be more resilient and less expensive, they said.
  • gadders 1 hour ago
    >>Google cloud outage takes down HSBC, Revolut, Bank of Scotland, Llyods, Equifax, more

    You will probably have issues logging into Llyods even when the cloud is back up.

    • venzaspa 33 minutes ago
      It's crazy. My main account is with Starling and I have a Lloyds account for a saving an account and some free cinema tickets. I don't think I've ever had an issue with Starling, it's great - Lloyds fails to sign in about 50% of the time consistently.
      • cowlevel 10 minutes ago
        Do you have an account with Llyods though?
    • michaellee8 38 minutes ago
      cannot agree more
  • bcvvvy 1 hour ago
    I bank with HSBC. Everything is up and working.

    A lot of the information people are leveraging here comes from Downdetector which is mostly users reporting that stuff is down, not whether they are actually down. It's a really bad source.

    • m11a 1 hour ago
      It's a pretty good source ime (at least, massive spikes tend to correlate well with issues). Traditional organisations aren't as transparent as SF tech companies about service downtime. Plus, if the service isn't working for its users, I wouldn't say it's actually up.
    • 4ndrewl 53 minutes ago
      Alternatively, and from the top of the HSBC homepage

      "We would like to apologise for any inconvenience and continue to monitor the situation, which is also affecting other organisations."

  • paweladamczuk 2 hours ago
    For me it was Revolut. No access to my main account, no withdrawals, no card payments since the morning. App doesn't work, there's no phone number to call, nobody to talk to. Dark side of neobanking, I guess.
    • Arbortheus 1 hour ago
      Not all neobanks. I use Monzo, and they had an outage a while back. I was impressed that it barely affected their functionality.

      They have a stand-in system that uses a separate codebase and separate cloud provider. They have a blog post about it here: https://monzo.com/blog/tolerating-full-cloud-outages-with-mo...

      The stand-in system has a limited subset of features, i.e. no bells and whistles, but enough that you can get any actual banking done.

      (Not an ad / I have no connection to Monzo, I just had some appreciation for a bank doing this properly as normally banking IT is a disaster. Plus the idea of keeping a separate codebase for almost an entire bank's functionality seems very invested.)

      I have always got bad vibes from Revolut...

      • m11a 1 hour ago
        Starling also has a disaster recovery system across cloud providers, although it is implemented differently to Monzo's (I used to work there on multi-cloud).

        Multi-cloud is really tough, tbh. It's easier said than done, especially if you care about data consistency at all.

      • ndsipa_pomu 1 hour ago
        > I have always got bad vibes from Revolut...

        I opened up a Revolut account a couple of years ago with the intention of switching my current account to it (i.e. wages going into it etc), but I got spooked after hearing that they weren't registered as a UK bank and thus subject to the various regulations and protections. I tried using Chase instead and have been happy with it so far.

        • misstuned 1 hour ago
          I used to use Starling as my main and they were fine, until I started getting issues with random card declines due to "fraud prevention" and nobody to talk to. I switched to First Direct and like the peace of mind of having a human on the phone if needed (when I wanted to make a huge transfer for my house deposit it was nice to have!) and I still get perks like free foreign card transactions.
        • altmanaltman 59 minutes ago
  • gentlerain 1 hour ago
    Countries talking about AI sovereignty, when they forgot compute sovereignty
    • octocop 4 minutes ago
      I think people have been fighting Googles monopolies for quite some time
    • Arbortheus 1 hour ago
      I think sovereignty might be the _cause_ of the banks' outages...

      This is affecting companies that put everything into a single GCP region in the UK for their entire business - aka most likely British companies or government with a data sovereignty requirement.

      • gentlerain 1 hour ago
        That's not sovereignty.

        Sovereignty is when there is a UK company with multiple data centres in the country.

        With that if one region goes down, another can carry on.

        • derektank 1 hour ago
          Redundancy does not seem like an inherent component of sovereignty
          • pwagland 43 minutes ago
            The UK, at least, is big enough that you can have multiple regions within the same country. There's a lot of distance between Glasgow and London, for example. Maybe not enough if both are targeted with nuclear warheads, but at that point, we have bigger fish to fry I suspect.

            Most European countries have enough space within the country to do that, and, for most things, except banking typically, hosting within the EU is enough, it doesn't have to be within your own country.

            • pumplekin 30 minutes ago
              Microsoft Azure has two entirely seperate regions.

              A "full function" one in/around London. And a "tickbox" one for multi-region requirements near Cardiff.

              The tickbox one has constant capacity constraints and a large number of Azure services aren't in it, but if you need to tick the "two regions and in country" requirement, you kinda/sorta can.

          • gentlerain 1 hour ago
            True on first glance.

            But considering what these sovereignty rules aim to achive, and as someone mentioned, that the problem may be due to companies relying on a single region provided by a hypersclaler due to data sovereignity rules.

            Then redundancy has to be considered by both the company and the rulers.

        • philipwhiuk 19 minutes ago
          That's not sovereignty, that's redundancy.

          Sovereignty is just where stuff is, not how many place it is there.

  • gaiagraphia 4 minutes ago
    It's a little worrying that banking with an institution from one of the world's financial capitals is so vulnerable. I would've thought London would've been priding itself on security, stability, safety, etc. Confidence in their systems is their bread and butter

    Afaik, even Russia and Ukraine aren't having such issues while dealing with data centres getting droned, lol.

    It's really quite poor performance.

  • misstuned 1 hour ago
    I couldn't unlock a shared city bike this morning on the Beryl system, probably the same outage.
  • diamondDrill 22 minutes ago
    how can cloud inf go down? that was the main selling point, it makes no sense
    • layer8 16 minutes ago
      The main selling points were auto-scaling and not having to self-host. Or, on a higher level, outsourcing accountability, being able to point at someone else if something goes wrong, as exemplified by the present case.
    • dist-epoch 19 minutes ago
      It makes no sense only if you believe in magic and marketing
  • BlackRabbit1 59 minutes ago
    Most likely it knocked out the "Ing" bank in Germany.
  • mistic92 50 minutes ago
    Ah again London region :D
  • josefritzishere 1 minute ago
    [dead]
  • pravex 39 minutes ago
    [flagged]
  • Heronsunny 1 hour ago
    [dead]
  • onetokeoverthe 52 minutes ago
    [dead]
  • rvz 1 hour ago
    Since there are no humans working on outages here is the test for superintelligence:

    Can someone ask Claude, Codex, Copilot, Antigravity, Devin to resolve this outage?

    No Googling or help from humans this time.

  • dangoodmanUT 24 minutes ago
    At this point, the way I see it is:

    - There's AWS

    - There's Azure

    - There's everything else

    And I don't even use Azure, I just hear it's good (and they don't charge cross-AZ traffic)

    • esseph 7 minutes ago
      [delayed]
    • jiggawatts 21 minutes ago
      Azure’s design is elegant and even — dare I say it — beautiful.

      Its implementation is clumsy and ugly.

      Its documentation is worse.

      Its support… isn’t.