Reading the tea leaves: given that the outage started with "connectivity issues affecting the File Manager", which was an install of Monsta FTP [1] - a commercial PHP application - I suspect the file manager got popped by one of the recently published PHP vulnerabilities, and users' FTP credentials may have been exfiltrated.
20 year customer here. This morning I asked their support chatbot if this is related to the PHP vulnerabilities and it assured me that's not the case. I'm skeptical.
They're being awfully vague about this issue so far. 8 hours ago I submitted a ticket requesting more information and they've not responded yet. I guess that they're still in the thick of it.
Having worked in support for years quite often higher layers of the company will give T1 support almost no information on what happened intentionally. That's pretty standard corporate CYA policy in litigious societies.
Internally, sure. But the application that was (probably) compromised was an FTP client, so it would have received the plain text of those passwords from users as they logged in.
> the application that was (probably) compromised was an FTP client
Thank you, this actually makes some sense. It's disturbing to get these emails with zero explanation what's going on, and very frustrating that user community has to guess. But PHP vuln in a web client intercepting user logins is the only thing that makes sense, otherwise that would mean passwords in the clear or some other 1990s-era nonsense.
I came across such a thing recently. The counterparty was a bank, and they point blank refused to update their processes so the people I talked to had no choice but to continue to use something they knew was hopelessly outdated. SFTP was released what, 3 decades ago or so? There is a good chance that whatever was on the other side of that upload was not capable of any kind of secure connection.
This has been a real annoyance for me lately since I run a lot of my music production business out of my website lexwhite.net hosted on Dreamhost and Dreamhost VPS. I love and trust Dreamhost a lot, though, as they have been an exceptionally solid company that I have worked with for, dang, nearly 20 years now I think! All of this new AI stuff has been exposing vulnerabilites at a rate that is hard to keep up with for anyone. We just have to really harden our cybersecurity as much as possible now.
Super displeased with DreamHost's handling of this. I have multiple master accounts. One of them got the email around 12pm today (over 48 hours after the incident started) saying...
"As a precaution, we have expired the passwords for some email and FTP/SFTP users on your DreamHost account. Their old passwords no longer work, and no new password has been set."
But none of those email passwords were actually expired - all were working. This is confusing - did they only expire passwords that were under threat? If a password wasn't expired, should it be trusted?
The same email suggests you change all email passwords you control.
Another account received no email warning from DreamHost so I assumed it was fine, but 7 hours later it also received the same generic email.
My complaints:
- sucks that it took them 48 hours to identify the breach - perhaps understandable
- but once the breach was identified, how the fuck does it take them over 7 rolling hours to email their customers? That's terrible.
- confusing messaging: DreamHost are claiming that passwords are expired, but in testing multiple accounts, all of them are still working. Do I need to rotate these? (I probably will just for safety).
I'm a 25+ year DreamHost customer and generally satisfied, but this feels like amateur hour.
I used them for entirely too long, and only continued to do so because a classmate in high school used my referral code to sign up for the account that would become his career of selling and managing websites for local churches.
I could not believe what the hell they were charging for a year of hosting with a dedicated ip (which they at the time required if you wanted https, real scumfuck move), so I didn't pay.
I guess I hope they actually deleted my shit because I sure didn't get this email
i wouldn't be so sure. I know I move in privacy and "anti big tech" circles, but I (and a lot of people I know) am moving a lot of my personal/recreational internet time to rss feeds of personal websites, mailing lists, and Mastodon. And there's a fair bit of discussion and small but growing use of private and very small group federated Mastodon hosts.
Having said that, very little of that is being done by the sort of people who'd host it on a service like Dreamhost.
Pricing wise they don’t beat Herzner anymore but are somewhat competitive with Digital Ocean.
I am gradually moving everyone to my own self hosting mostly for supportability reasons. We want to react in seconds, not hours, when a customer has an issue.
Just be warned Hetzner will delete your account at the slightest sign of controversy. It's fine for Minecraft but I wouldn't want to host a political website there.
I've been a user for more than 20 years. With their shared hosting plan, which was about $5/mo. when I joined and is now $13/mo, I've been able to host my personal blog and professional site, as well as probably 3 dozen other sites over the years, including several that have hit the HN front page and never got hugged by the squeeze of death.
Honestly, compared to some of the professional environments I've worked in, it's a joy to have an idea, create a few PHP files, and in a matter of seconds have that idea live.
It's cheap and reliable with predictable costs. It hasn't caused me any problems in a decade. Most of the websites I want to host don't need cloud anything, they're statically generated.
[1]: https://www.monstaftp.com/
These status updates from the PHP incident would also have us believe that they were never directly affected: https://www.dreamhoststatus.com/pages/incident/575f0f6068263...
They're being awfully vague about this issue so far. 8 hours ago I submitted a ticket requesting more information and they've not responded yet. I guess that they're still in the thick of it.
Having worked in support for years quite often higher layers of the company will give T1 support almost no information on what happened intentionally. That's pretty standard corporate CYA policy in litigious societies.
Thank you, this actually makes some sense. It's disturbing to get these emails with zero explanation what's going on, and very frustrating that user community has to guess. But PHP vuln in a web client intercepting user logins is the only thing that makes sense, otherwise that would mean passwords in the clear or some other 1990s-era nonsense.
As a precaution they reset everyone’s email passwords, presumably from an assumption people use the same ones for both.
I’m surprise they still let people use FTP.
"As a precaution, we have expired the passwords for some email and FTP/SFTP users on your DreamHost account. Their old passwords no longer work, and no new password has been set."
But none of those email passwords were actually expired - all were working. This is confusing - did they only expire passwords that were under threat? If a password wasn't expired, should it be trusted?
The same email suggests you change all email passwords you control.
Another account received no email warning from DreamHost so I assumed it was fine, but 7 hours later it also received the same generic email.
My complaints:
- sucks that it took them 48 hours to identify the breach - perhaps understandable
- but once the breach was identified, how the fuck does it take them over 7 rolling hours to email their customers? That's terrible.
- confusing messaging: DreamHost are claiming that passwords are expired, but in testing multiple accounts, all of them are still working. Do I need to rotate these? (I probably will just for safety).
I'm a 25+ year DreamHost customer and generally satisfied, but this feels like amateur hour.
I might have been willing to stick it out, but their panel/website-management is so slow.
May I ask to where you migrated?
I could not believe what the hell they were charging for a year of hosting with a dedicated ip (which they at the time required if you wanted https, real scumfuck move), so I didn't pay.
I guess I hope they actually deleted my shit because I sure didn't get this email
Having said that, very little of that is being done by the sort of people who'd host it on a service like Dreamhost.
Support is primarily via chat these days.
Pricing wise they don’t beat Herzner anymore but are somewhat competitive with Digital Ocean.
I am gradually moving everyone to my own self hosting mostly for supportability reasons. We want to react in seconds, not hours, when a customer has an issue.
Honestly, compared to some of the professional environments I've worked in, it's a joy to have an idea, create a few PHP files, and in a matter of seconds have that idea live.