If it is true that eventually a general approach that scales with compute/data will supersede a bespoke expert solution, it is not necessarily true that today's expert solution should be thrown away at the first sign of a data-driven model.
It says nothing about what you should use today to build things, it says everything about how you should do research that lasts.
If you're building for billions of users in 10 years, sure go with whatever you think is the bitterest.
But if you're building for right now, you probably mashup whatever works best. That might still be a PID loop, EKF, and Homographies sometimes. But when you hit the frontier of what "works", then you have to reach for new things anyway.
Chesterton's baby/bathwater
WRT humanoids - I am making no statements.
It's an entirely optional addon added by google.
can someone smarter than me explain in simpler words, how is that literally not the same thing?
Everyone complained there should be a way to disable AI, and that it should be on by default. For some reason (curiosity still makes sense here), Google did some 'malicious compliance' and added the AI switch-off without making a new version of search, and disabled AI by default.
My personal theory is some sort of moral liability thing, where they're hoping you blame your school district and tell them to switch platforms, so that if/when they don't, it's "their fault", not Google's.
I have a list somewhere I need to dig out of these types of link modifications available from when I was deployed. They were the only way to read news.
I play music by sight (poorly) and can improvise a little bit on guitar, and would love to understand music theory better but so far do not.
Is this a useful device to get a feel for creating music, or is it only really useful if you already understand music theory or improvisation?
> Importantly, we do not claim in this paper to have created or demonstrated the equivalent of a virus that primarily operates in weight space.
In fact if I have to ship in a different language, I'll often do a Haskell version first to steer towards isolating IO, enforcing referential transparency, and generally picking up other functional patterns that are a good idea no matter what. Then I'll port the Haskell version to whatever the solution demands.
Token efficiency is specifically the place where strict languages get a little trickier. With dynamic languages where the model has a ton of training data, like Python, the agents are really good at writing code that just works. You still have challenges there, but they're just different than the challenges you hit with a language like Haskell/Lean/Agda etc. With Lean, for example, the amount of things the agent decides it needs to prove for a relatively trivial operation can lead to an absurd amount of code. And the way it decides to prove them can be borderline labyrinthian. It sort of plays perfectly towards current agent weaknesses, where agents can run off and reinvent a lot of machinery and build unnecessarily complex abstractions.
And those iterations have a cost. Even if the final artifact is more terse, the path to getting there can involve a huge amount of backtracking and rewriting and off-by-one syntactic corrections.
Ultimately, a lot of this comes down to developer experience. Getting useful error messages for the agent, having fast compile times so that the agent can iterate quickly, providing useful utilities for debugging, etc. There was a popular blog post specifically about Haskell that generally makes this point--the Haskell language is, in a vacuum, ideal in a lot of ways for AI, but the Haskell developer experience and ecosystem made it hard for this team to build with it via agents: https://avi.press/posts/2026-07-10-after-7-years-in-producti...
Note: I am not advocating against using these kinds of language. Obviously, I still work in Lean and a little in Agda and some related languages/projects, so I think this direction is right. Probably biased in that I love them with or without LLMs, but I do think they're also more ideal for AI development. There are still just challenges in getting agents productive in them, and most of the challenges are not about the type systems but the actual ergonomics and dev experience.
An ideal language for agentic AI I would think is extremely strict, where the AI can iterate over the tooling for correctness. It would probably be less verbose and more terse to allow the AI to be more efficient with tokens.
I think that is an unfair characterization of the video and the intent of Haskell's co-creator, based on the rest of your comment. Simon Peyton Jones makes a graph with two dimensions, useful/useless and safe/unsafe. The question he gets at is: how can one distinguish between effectful and effect-free computations in a programming language? He proceeds to characterize most useful languages as unsafe, and Haskell as useless but safe. He then spends the majority of the video talking about how useful languages are becoming safer, and at the same time how Haskell must become useful.
> we are going to build the best onboarding experience in the programming language ecosystem
> This can only be done if we commit to unix principles > write programs that do one thing and do it well
Rust is so approachable because Cargo does not subscribe to "do one thing". In fact Cargo does a bunch of things. Which is very convenient for newcomers, because they only need to learn one thing instead of a bunch (and how they interact).
The same holds for a GUI like a file explorer with buttons and interactions like drag / drop or marquee select, which is a bunch simpler than chaining 3 programs together to delete 30 old files and move some folder into a subdirectory.
It also explains why the most successful tools within the land of Unix philosophy are stuff like AWK, which really don't just do one thing. Or why PowerShell is less usable than GNU. Or why Emacs is an 'operating system'.
It's also a reason why the installation experience for Haskell is so bad: there are a bunch of programs and users don't know where to start and what they need.
> Cabal and Stack: which one should I install? > We recommend installing both.
One thing and one thing well.
Atleast it seems they've added a combined installer for these tools since the last time I've checked.
In my opinion, the quality of MS Word peaked some time in the early 2000s.
So today, given the choice between LibreOffice and MS Word, I would always choose LibreOffice, even if MS Word were free, and despite the fact that LibreOffice Writer has some annoying bugs that have not been fixed for many years, but I find easier to work around those bugs than to cope with the user interface of modern MS Word.
On the other hand, I agree that MS Excel is clearly superior to LibreOffice Calc. Nonetheless, there already are decades since I switched to LibreOffice Calc and I did not need the extra features of MS Excel. For things beyond the capabilities of Calc, I use a real database, though that would probably not be the best solution for a typical office worker.
In the distant past I liked a lot MS Access, but I have no idea whether today it is still useful. I abandoned it many years ago, after I had some problems with the migration of data between different versions of MS Access and even more with the migration of data from MS Access to another database manager.
“ What a fascinating bug!! My wife has complained that open office will never print on Tuesdays!?! Then she demonstrated it. Sure enough, won't print on Tuesday. Other applications print. I think this is the same bug. Here is my guess:
Print to a postscript file. Observe the line: %%CreationDate: (Tue Mar 3 19:47:42 2009)
Change "Tue" to anything else: %%CreationDate: (XTue Mar 3 19:47:42 2009)
Save the file and it prints. Tools like evince work because they simply omit the "CreationDate" tag to begin with.
Now something odd happens when my cups script (I am using the Brother MFC420CN) copies the file to a temp file. Some of the code is rearranged, not sure how or why, but it uses a command called "file" to identify the file as "PostScript". This check would work on the original file you printed, but by the time it runs the check on the temp file, it misidentifies. Normally it would return:
PostScript document text conforming at level 3.0
But there is another check that happens before the PostScript check. If it finds "Tue" at the fourth byte of the file, it identifies it as: Jan 22 14:32:44 MET 1991\011Erlang JAM file - version 4.2
So it's not a problem w/ openoffice.org, cups, or the brother printer drivers. It is a bug in the `file` utility, and documented at https://bugs.launchpad.net/ubuntu/+source/file/+bug/248619.
”https://bugs.launchpad.net/ubuntu/+source/cupsys/+bug/255161...
Not true. There's inertia and network effects in play that often prevent better products from succeeding against established ones.
Blender is a little different because every 3D modeling program is odd in its own way, and the new UI was close enough that it was easy to learn after the transition (I do still use the legacy keybinds though).
But Blender is very niche hobby software. To "succeed" (if we define success as capturing a significant user share from closed source spyware), Libre and GIMP need to offer workflows that are close to what the average non-coder technical person expects. Maybe that isn't valued by the teams, or maybe they don't have enough resources to make it happen, but the end result is a reputation of jank.
I dislike vibe coded clones because of concerns about the engineering quality. But the reason they're making waves when traditional FOSS hasn't is that the UX is leagues ahead.
[0]: https://www.libreoffice.org/donate/
[1]: https://www.microsoft.com/en-us/microsoft-365/buy/compare-all-microsoft-365-products (DO NOT REDEEM)My guess: At some point in the past, a firmware was developed without plans to include firmware update functionality. The team that did this was later burned when the client tried to use a new certificate that the hardcoded, unchangeable system didn't recognize.
Angry meetings occurred, developers were chastised, and someone realized they could avoid that problem by fetching an updated root CA on demand. They did it over HTTP to avoid a chicken and egg problem.
In hardware manufacturing, everything is about crossing items off the list of problems. A problem was raised, someone came up with an idea to solve that problem, and it was implemented. The practice might have spread to other products because it became institutional knowledge about how to avoid a problem that made a customer angry once.
This is from my experience in the consumer hardware industry. If you're not watching the outsourced development closely, they will take the easiest route to solve the problem in front of them and nobody will consider any other consequences.
I'm seeing some 3d printed projects popping up that look to be filling this gap though.
1 20" box fan.
4 20" mirv 13 filters (cheapest is likely costco or lowes or home depot - buy in a 4 pack to save money).
This is still expensive but the amount of air processing per hour is incredible.More info Search "Corsi-Rosenthal Box" or click https://filterbuy.com/resources/air-filter-basics/how-to-bui...
Don't go hepa because it slows down the filtering too much. I use this technique in my house and it cuts down on pollen, smoke and mold. You'll feel a lot better than a tiny expensive hepa that has proprietary filters.
The most fascinating part of this wave of LLM reverse engineering is seeing how many weird and poor choices were lying in plain sight.
Any number of people could have discovered and exploited this, but it’s tedious work. Most people who can do this are busy with their job and other things. Now LLMs enable people who understand these things to quickly get through the tedious parts, so they can do a lot more of this work in their free time than they could before.
For all the talk about digital sovereignty, EU still builds the fundamental element of EU citizenship, the digital ID, on American companies. Baffling.
Changing stupid energy policy that is propping up oil demand.
When China is selling batteries and solar below cost or at tiny profit, the federal government is effectively giving the subsidy to other countries.
When wind is the cheapest with the highest ROI we should be doing more of it.
It is common to be asked for your CPF for trivial things like purchasing anything on a drugstore, and for any and all activity with government branches or banking. This combined with several breaches in government and private institutions in Brazil over the years has made this information essentially public.
More concerning is when more private information is leaked. For example, police security camera live feed and license plate or CPF-based tracking of anyone was sold online in the past, and has been used by criminal gangs in an attempted murder of a judge: https://www.poder360.com.br/justica/pcc-tinha-acesso-a-siste...
This highlights the importance of reducing the amount of data collected, but in Brazil and the rest of the world, both governments and private institutions have pushed for the contrary, more so in recent years.
https://blog.codinghorror.com/falling-into-the-pit-of-succes...
Defer will also appear in C compilers soon, but personally I still think goto-error handling is clearer.
(Rust has an advantage for temporal memory safety, but this does apply to Zig.)
I was much more interested once I saw the actual website.
Long story short, it's basically a fret at the bridge end of the guitar, so using it will bring the 'root' note of the bass strings up a semitone or few.
I can see this being pretty useful for drones when fingerpicking. Though combining this with normal frets on the top string will necessarily be inaccurate because it shortens the string length, so the normal frets will be slightly misplaced.
After watching the trailer I must say that the overall vibe is substantially off. Very pretty, but it lacks the perpetually disoriented, if you will, atmosphere that dominates the narration in the book. It's all too clean and predictable in the trailer.
But it is a difficult book to adapt, I will give them that.
EVs were the future then and especially now. German car manufactures placed poor bets.
People and the law keeps treating this as "just a sales pitch", but it's really theft in disguise and should come with evidence on an S1 filing.
However I do think there are opportunities to greatly improve all these fields with AI, I just don't think we should lead with an overpromise that sounds Musk-like.
https://en.wikipedia.org/wiki/Read%E2%80%93eval%E2%80%93prin....
Honestly, the way he lists them and even pauses before vaguely saying "mental illness" gives a strong impression that he's choosing whatever illnesses he can think of as he goes.
In a nutshell, nobody wants to be the first.
(possibly obligatory disclaimer: I’m not saying this to minimize harassment and abuse claims. More the opposite: I worry that perceiving this phenomenon as something specific to harassment and abuse allegations can become a pretense for dismissing them out of hand.)
For me, this is a lapse of judgement for the CEO and one of the founders of one of the biggest corporation's globally.
His security detail is also no longer worried about retribution and are likely complying with an internal investigation.
It's very likely his affairs were simply open secrets within the C suite.
https://www.berkshirehathaway.com/letters/1989.html
"In a difficult business, no sooner is one problem solved than another surfaces - never is there just one cockroach in the kitchen."
We have to figure out where this comes from! It's so recent: https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...
https://news.ycombinator.com/item?id=49828285 references https://x.com/_NathanCalvin/status/2084856543561093535 - that's the earliest I've seen so far. Others?
Edit: thanks ChatGPT: https://barrypopik.com/blog/there_is_never_one_cockroach_bus...
Proverb #232 of https://www.asau.ru/files/pdf/2759983.pdf says it's attested from 1991:
232 There is never just one cockroach. (1991)
Variant: There is never just one cockroach in the kitchen
Ants are nicer though.
At least we’ve now passed the tipping point where the majority of affairs are domestic. Some country out there is going to achieve HSM (Huang Second Marriage) and it’s vital to future prosperity that we make sure it’s the US.
Not all, very likely not even most, of course, do this, and it's not always truly mutual or healthy, but surprise surprise, not every couple wants strict monogamy. They just don't want society to judge them for it.
This feels so common that I would not even be quick to assume this is some sort of unknown betrayal to his family. One of the revelations about the Gates' lives was that Bill's promiscuity was apparently baked into their marriage agreement. At this level, your marriage might just be like a fixed mailing address for someone who has multiple homes.
Not to excuse this in any way. But these are the personalities of medieval lords and kings.
That’s what is going on.
Three different news orgs have reported the different affairs.
The problem was never the lack of a way to pay you, they just don't care if the transaction is fair.
EDIT: And that's without mentioning the value of machine-readable payment requests, which is what x402 actually brings.
I do want it to work, so just curious does anyone have any positive anecdotes? Can this naturally get picked up by crawlers?
People do this intuitively with payments (negative). They'll buy a timeshare or a boat (expensive, "lump" cost) when it would be much cheaper on net to stay at resorts or rent a boat and pay by usage.
They'll pay for subscriptions (automatic payments) rather than pay per experience. Many people will pay $20/mo for Netflix, watch it twice a month, but would never rent movies for $3 on Amazon.
They'll pay a big premium for "unlimited" data plans when this actually makes little sense for anyone vs. paying by usage.
The actual quote from the actual article: “ Then he went on to describe the situation the company was in. Major cloud providers were running away from Nvidia, making plans to use alternative chips and lessen their reliance on the firm. Even worse, Huang said, the vast supply of chip wafers and memory Nvidia had committed to buying were coming at him like an "avalanche."”
Obviously that's not the case for everybody. 3 years ago, LLMs were very far from producing the quality of code I wanted in my domain.
It's just one of those things, you'd think the economy or country would collapse effectively but things just tick on by they way they always did. Imagine the banks in the US closing for 6 months?
https://en.wikipedia.org/wiki/Irish_bank_strikes_(1966%E2%80...
That's a scary thought.
But I find it difficult to motivate it from the "productivity" angle. Like if I work for a day without an LLM today, it feels like I could go skiing instead, and the LLM will do what I would have done today in 10min tomorrow...
94 points by cloudbreaker 22 hours from now | flag | hide | 64 comments
I have experienced, half a decade ago during some more frequent but less lengthy O365 errors that there are some higher ups who get when this is not something you can fix and some you very much can't. Wishing strength to any "but it's our mail" and "but we bought the Claude" meetings that happen today.
OAuth request failed Temporarily unable to authenticate. Please retry.
Can this be fall-out from the Microsoft outage?
That itself wouldn't be a problem - email is very decentralised.
The problem is that we thought that putting like 2 companies in charge of most email communication is a good idea.
As an aside, I worked on Lotus Domino—IBM's competitor to MS Exchange—before it was sold it off in 2018.
Is this what it feels like to be old?
Note that the infra for M365, Azure etc is different. They would be running on servers hosted in MSFT DCs but have completely different CI/CD pipelines. M365 services have their own deployment pipelines and it takes on the order of weeks to deploy your changes to production.
Considering the fact that Outlook is down, clearly email doesn’t seem to be working. So email uses something called Microsoft Exchange Server, there used to be an on-premises version of this but MS stopped supporting it to run the cloud version hereby called M365. Email is one big “macro-service”, if you’re running Windows Server, look at the process called “EdgeTransport.exe” this is responsible for sending/receiving email on Windows Servers. There are probably multiple 100s of services supported by 100s of teams that are run while email is routed from sender to receiver via MSFT Exchange servers. My recollection is that even if one of these services goes down or introduces latency in the pipeline, it caused a major incident. When I was working in M365 Security in 2024, this Exchange Server pipeline was called “legacy” because it’s one big .NET Framework service. I don’t know if they upgraded to .NET core but developing on this was as HN comments say, very janky.
The reason Teams doesn’t seem to be down is that Teams services are completely different and are written using .NET Core and have their own separate release pipeline. It was much easier to interact with Teams services. I actually worked closely with the team that was responsible for ingesting Teams messages to the customer’s data store. Their code base was much more modern and easy to understand. Exchange Server not so much.
I don’t know if what I said is accurate but know that Exchange server code was like a fossil that didn’t change as much, over the ~5 years I worked on it from 2019-2024.
Story time over.
This absolutely predates AI.
But this is certainly exactly what I'd expect after indiscriminately laying off half your workers, then immediately offering a buyout to make sure any experienced people still standing will find their way to the door.
If this incident, or the difficulty they're having with remediation (update: holy shit, they still haven't even identified the problem), are somehow not caused by any of that, then it's at least a preview of what things are going to be like.
We use Azure at work and it’s wild how much stuff is just manual labor behind the scenes on their platform.
Seems like it’s always on fire somewhere, ripe for burnout and MBOs aren’t aligned with reality from what I hear.
The problem is amateur hour is the entire industry. Nobody specs anything, nobody tests overload conditions, nobody thinks failure modes. The stuff that every engineer does when building anything else other than software doesn't happen. The orgs are bloated. Too many teams. Too many silos. Incentives are stacked the wrong way. People are motivated to not think about systems but stay in their lane. Responsibilities are split in weird ways.
Yes. Big systems are hard. But we can build, manage and understand big systems with the right approach and people. Even one computer is complicated and many people get stuck trying to debug a piece of software running on one machine. At the end of the day it's not magic. All of humanity is big systems but when I have a power failure the power company is generally able to understand why and resolve it and the water in my tap doesn't stop flowing and the cell towers still work (generators, who would have thunk).
A friend who is really good had to leave Microsoft because of how AI was pushed. Before that he was telling me some other stories that make the culture seem pretty not great. Heard similar stories from other people. It used to be a great place filled with great people until at some point it's just another big company.
It's been hours, and the status updates still feel like there is an undertone of frantic panic (understood if you can't determine the root cause). When I think of past major outages, they may have been down for hours but usually folks knew what the root cause was relatively early, even if remediation took some time.
I wouldn't be surprised if AI coding and release pressure was a primary cause, so I really hope the future post-mortem is made publicly available.
I feel for those stuck in there. The outage going on for this long has got to be fucking brutal.
Everyone has occasional outages but this is the worst 365 outage our org has seen since starting with M365 in 2019.
In retrospect, the cause and cure are obvious to everyone.
Current status: While the servers we've returned to service initially remain healthy, we've observed that they can become unresponsive again once they begin receiving traffic. We're analyzing diagnostic data from these servers with our networking and platform experts to determine what's triggering this behavior.
Building on the protective traffic management measures we've implemented, we're also temporarily limiting which types of traffic can reach the affected infrastructure in specific locations. This will help us determine whether a particular type of request is contributing to the impact and allow us to restore service incrementally. Additionally, we're rebuilding a subset of affected servers using a previous software configuration to validate whether this prevents them from becoming unresponsive, and we'll expand this approach to additional servers if we observe positive results.
---
The 11:05:31 PM UTC update is is the first since the outage began that feels hopeful.
From the 11:06:31 PM UTC update:
Current status: We've begun returning a portion of the affected servers to service, and they're remaining stable so far. We've also shifted traffic and connections to alternative pathways wherever possible and have observed limited recovery in some regions.
We've streamlined our recovery process to expedite returning the remaining affected servers to service. To further support recovery, we're implementing protective traffic management measures in a specific location to reduce load on the affected infrastructure, and we'll expand these measures to additional locations if we observe positive results. As capacity is restored, users may experience a gradual recovery as their clients reconnect to the service.
---
The 10:11:39 PM UTC update is a little chilling. They don't know why what's happening is happening so they're doing experiments.
I do that with closed-source software when I don't want to bother trying reverse engineer it, but these guys have the code! Yikes.
I recognize that even w/ the code available there are likely behaviors that aren't likely to be anticipated because of the scale of traffic. Complex systems fail in complex and non-intuitive ways.
From the 10:11:39 PM UTC update:
Current status: We're continuing to pursue all available pathways to restore service, as our actions taken thus far have not provided the anticipated relief. Our investigation has found that a core process on the affected servers is becoming overwhelmed, which causes the servers to stop responding and contributes to the capacity loss.
To validate a potential recovery approach, we're fully isolating a portion of our service infrastructure, restarting the affected components, and gradually reintroducing traffic to confirm our services remain stable before applying this approach more broadly. We're also analyzing traffic patterns across our entire service infrastructure and evaluating whether isolating traffic within each region may allow affected regions to recover independently.
You know a lot of talented people who work there also post here, and it's not fair to insinuate that they aren't talented because some vastly complex system somewhere in the world had a bad day.
Stuff happens, things break. That's why lots of nines cost a lot of money. It doesn't automatically mean that anyone is incompetent.
"We're also preparing an automated recovery process to continually detect and restart unresponsive servers, to help keep them in service while this change progresses."
https://www.abc.net.au/news/2026-11-13/microsoft-outage-affe...
Current industry best practice is to use LLMs improperly.
From the 10:11:39 PM UTC update:
Current status: We're continuing to pursue all available pathways to restore service, as our actions taken thus far have not provided the anticipated relief. Our investigation has found that a core process on the affected servers is becoming overwhelmed, which causes the servers to stop responding and contributes to the capacity loss.
To validate a potential recovery approach, we're fully isolating a portion of our service infrastructure, restarting the affected components, and gradually reintroducing traffic to confirm our services remain stable before applying this approach more broadly. We're also analyzing traffic patterns across our entire service infrastructure and evaluating whether isolating traffic within each region may allow affected regions to recover independently.
From the 9:41:46 PM UTC update:
Current status: We're continuing to work through restarting and validating the affected servers to safely return them to service, and we're exploring opportunities to expedite this process. In parallel, we're assessing whether reverting the small number of recent configuration updates we identified could help accelerate recovery.
From the 9:01:14 PM UTC update:
Current status: We've validated an alternative health check method on a subset of affected infrastructure, which allowed the servers in that subset to successfully return to service, and we're now applying this change more broadly. We're also preparing an automated recovery process to continually detect and restart unresponsive servers, to help keep them in service while this change progresses. Additionally, our review of recent changes has identified a small number of configuration updates that we're investigating as potential contributors to the issue.
From the 8:44 PM UTC update:
Current status: While validating the targeted change, we found that many of the affected servers have become unresponsive, which is preventing us from confirming whether the change is effective. We're restarting these servers so they can come back online with the change in place, and we're applying the same approach to additional affected portions of our infrastructure to validate the results more broadly.
Previous Status: While we haven't identified any recent changes that correlate with the start of impact, our investigation indicates the failed health checks may originate from a specific component on the affected servers that's responsible for processing them. We're deploying a targeted change to bypass this component on a subset of servers to validate whether this restores availability, and we've taken steps to prevent additional servers from being removed from service while we work to return the affected servers to service. We're also investigating why affected servers are experiencing elevated processor usage while handling requests.
I can conjecture some reasons. Datamining would be high on the list, since MS can read your mail and thus market to you more effectively. Governments might also be happier if MS can provide them with one-stop access (like Prism) to otherwise self-hosted e-mail accounts proxied through a central server.
For reasons that I cannot fathom, Microsoft wants to act as the relay to my self-hosted e-mail account, if I add the account in Outlook.
The Status page is not working well, but I managed to subscribe to their RSS feed and it has been sending out updates periodically. I'll post some of them here.
9:02 PM UTC: We've validated an alternative health check method on a subset of affected infrastructure, which allowed the servers in that subset to successfully return to service, and we're now applying this change more broadly. We're also preparing an automated recovery process to continually detect and restart unresponsive servers, to help keep them in service while this change progresses. Additionally, our review of recent changes has identified a small number of configuration updates that we're investigating as potential contributors to the issue.
Previous status: While validating the targeted change, we found that many of the affected servers have become unresponsive, which is preventing us from confirming whether the change is effective. We're restarting these servers so they can come back online with the change in place, and we're applying the same approach to additional affected portions of our infrastructure to validate the results more broadly.
8:52 PM UTC: While validating the targeted change, we found that many of the affected servers have become unresponsive, which is preventing us from confirming whether the change is effective. We're restarting these servers so they can come back online with the change in place, and we're applying the same approach to additional affected portions of our infrastructure to validate the results more broadly.
Previous Status: While we haven't identified any recent changes that correlate with the start of impact, our investigation indicates the failed health checks may originate from a specific component on the affected servers that's responsible for processing them. We're deploying a targeted change to bypass this component on a subset of servers to validate whether this restores availability, and we've taken steps to prevent additional servers from being removed from service while we work to return the affected servers to service. We're also investigating why affected servers are experiencing elevated processor usage while handling requests.
7:52 PM UTC: We've identified that affected servers are failing automated health checks, which is causing them to become unavailable, and this may be related to a recent change to a component involved in processing those checks. We're testing a configuration change on a subset of affected infrastructure to address this and, if successful, will apply it more broadly. In parallel, we've initiated a restart of all affected servers and are validating whether they're successfully returning to service.
7:30 PM UTC: We're aggressively pursuing all pathways to reduce impact. As a precaution, we've paused all deployments and configuration changes while we review recent changes for any correlation to this event. We've determined that the elevated load isn't driven by an overall increase in traffic, but by a large number of servers being unexpectedly unavailable, which is shifting load onto the remaining infrastructure. We're investigating what's causing these servers to become unavailable while we work to safely return them to service.
7:16 PM UTC: We're aggressively pursuing all pathways to reduce impact. As a precaution, we've paused all deployments and configuration changes while we review recent changes for any correlation to this event. We've also moved traffic for some services off the affected infrastructure to further reduce load, with additional traffic moves in progress.
6:59 PM UTC: We've completed redistributing connection traffic across our worldwide infrastructure, and we're moving additional traffic off the affected infrastructure to further reduce load. We're also restarting affected servers to return them to service. In parallel, we're investigating whether the elevated traffic reflects a genuine increase in requests or a shift in how traffic is being routed across our infrastructure.
7:52 PM UTC: We've identified that affected servers are failing automated health checks, which is causing them to become unavailable, and this may be related to a recent change to a component involved in processing those checks. We're testing a configuration change on a subset of affected infrastructure to address this and, if successful, will apply it more broadly. In parallel, we've initiated a restart of all affected servers and are validating whether they're successfully returning to service.
7:30 PM UTC: We're aggressively pursuing all pathways to reduce impact. As a precaution, we've paused all deployments and configuration changes while we review recent changes for any correlation to this event. We've determined that the elevated load isn't driven by an overall increase in traffic, but by a large number of servers being unexpectedly unavailable, which is shifting load onto the remaining infrastructure. We're investigating what's causing these servers to become unavailable while we work to safely return them to service.
7:16 PM UTC: We're aggressively pursuing all pathways to reduce impact. As a precaution, we've paused all deployments and configuration changes while we review recent changes for any correlation to this event. We've also moved traffic for some services off the affected infrastructure to further reduce load, with additional traffic moves in progress.
6:59 PM UTC: We've completed redistributing connection traffic across our worldwide infrastructure, and we're moving additional traffic off the affected infrastructure to further reduce load. We're also restarting affected servers to return them to service. In parallel, we're investigating whether the elevated traffic reflects a genuine increase in requests or a shift in how traffic is being routed across our infrastructure.
And the other part is the "goodwill" you mention - if you jettison people for nothin en masse, word gets out, and make it hard to hire in the future
Or, rather, it's acting like it usually does.
Status page is down. Much of our org is halted.
Fingers crossed that Azure isn't impacted (seems fine so far)
(https://www.geekwire.com/2026/microsoft-makes-new-voluntary-...)
Well said, thank you.
It saddens me that people can't understand this. Already there is the fallacy of "I can make apps with AI, therefore no one needs to code anymore," as if all coding was was making apps. But even the apps that people think are passable feel so fake, clunky and sludgy.
This is very, very eye-opening, at least to me.
"Jensen Huang summoned his top finance executives to his office this summer. The AI economy, backstopped by Nvidia’s NVDA -4.34%decrease; down pointing triangle increasingly outrageous bets, was teetering, and Huang wanted to make sure they were getting it right.
Over the course of the meeting, he proceeded to tear into the woman who is now taking over for him as Nvidia CEO.
Colette Kress was “a deer in the headlights,” he said, interrupting her presentation, according to a person familiar with the comments.
He had summoned Kress and his senior finance team to diagnose a ballooning problem: Nvidia was in talks to provide billions of dollars of guarantees to companies that couldn’t afford its chips. That included deals led by Huang’s son-in-law, Nico Caprez, who had been offering upstart cloud providers and data-center developers financing in the hopes of turning them into long-term customers.
Huang had launched a dealmaking spree and was now starting to question whether it had gone too far, according to people familiar with the matter.
Kress was a “reporting CFO,” not someone who could help him define a strategy or make decisions, Huang said during the August meeting.
Then he went on to describe the situation the company was in. Major cloud providers were running away from Nvidia, making plans to use alternative chips and lessen their reliance on the firm. Even worse, Huang said, the vast supply of chip wafers and memory Nvidia had committed to buying were coming at him like an “avalanche.” (...) Amid the rollout of the program, Kress raised questions including how to decide which deals to pursue, how to value the guarantees, and how to calculate the return on investment, a person familiar with the matter said. Behind the scenes, she tried to rein in Huang’s son-in-law, who had been promoted to a vice president in January, people who worked with her said. "
The board backed him after "sleeping with a spy" came out.
The board did not back him when, days later, "sleeping with a staff member who reports to him" came out.
Seeing this as the top story everywhere makes me want to throw away my phone and become a monk.
Not really sure why I’ve written this comment, other than as some reassurance to anyone else that feels the same way.
NVIDIA (NVDA): The China Problem
https://culperresearch.com/wp-content/uploads/2026/05/Culper...
It uses careful language to avoid implying/defaming anything that couldn't be supported by publicly verifiable info but clearly found their relationship questionable to say the least.
Any investigative journalist worth their salt would have latched onto that possibly salacious hint and started digging. If anything it's surprising this had been quasi-public for months given the trillions at stake from either side of the trade.
It certainly didn't need a strategic CIA leak or House of Cards style knife in the back to doom Jensen. Embarrassing in retrospect how poorly they concealed it.
Your comment above implies that the answer to "Why now?" is: "Because covert powers so decided." There is nothing in the story that disproves that, but also nothing that requires it. The initial tip could have been ordered from above, but it could just as easily have come from any one of dozens if not hundreds of people, motivated by just about anything: grievance, envy, high-mindedness, malice, vanity.
Or perhaps it's just the sheer accumulation of chance. The longer it's happening as an open secret that no one wants to talk about, the more likely that someone who thinks it's wrong will find out and speak out.
There is almost certainly some level of coordination between said "intelligence officers" and these reporters to make this a public story. They are choosing to comment and provide details on this particular issue, and they are doing so strategically.
Based on the photos that have been popping up, the tech industry contact could have been any one of the comparatively large number of people in position to observe what was going on.
[1]: https://www.nytimes.com/2026/11/10/podcasts/the-daily/nvidia-jensen-huang-affair-chinese-spy-trump.html?showTranscript=1Obviously we can only come up with conjecture, but it should at least be discussed that this is not happening out of thin air due solely to his reckless actions but represents some kind of hidden power struggle. It's quite clear to me folks like Secretary of State Marco Rubio very likely knew about these things taking place for quite some time.
"Jensen and Nvidia were planning to go hard at producing frontier class open models. They wanted to verticalize their AI presence, and not depend on OAI or ANT to drive the industry forward. OAI, ANT, and GOOG are simultaneously competing with NVDA by designing AI chips."
"At the same time, Jensen had all but gone to war with NatSec hawks in DC over Nvidia sales in China. As I've noted over the last two years, NVDA sold huge numbers of GPUs (~20% of total sales?) to SE Asian buyers and I always thought many of those chips ended up physically in China, or available for remote use by Chinese companies."
For real? Anthropic sounds more evil that my wildest imagination. If true, I hope this kind of evil company gets destroyed sooner than later.
https://www.reuters.com/technology/former-nvidia-ceo-huang-h...
"Oh who cares who he slept with, he's been great for Nvidia. Nothing has been proven."
He's one of the most visible CEO's in the most visible industry in the world, and the entire AI economy is dependent upon them. The perception of him as stable is incredibly important.
Merely the suggestion that he is having affairs with people with close ties to China's government, given export controls, is a huge PR problem. Adding in an additional affair with someone under him. And the likelihood that there may be even more out there.
I doubt the Board wants to get rid of Jensen Huang. He's done incredibly well for them. But right now, there's a huge PR shitstorm swirling around the company.
Maybe in six months it all dies down and he comes back, who knows. But right now, you really can't just ignore the problem and hope the media doesn't care about the whole "well she was only sort of a spy" situation.
The lies may be part of it, but if you didn't fire them for the COC/SH violation, you would be in for near infinite liability because it ends up as evidence that the culture from the top was tolerant of violations, etc. THis is not conjecture, it's what has basically always happened in the legal realm when companies try to work around it.
* Megaspeed was evading US sanctions against China
* Megaspeed CEO was a Chinese state agent
* Huang was sleeping with Megaspeed CEO
Therefore, one of the following must be true:
(1) [Unlikely] Huang knowingly supported the evasion of sanctions. If so, he should be fired for breaking the law.
(2) Huang was tricked. If so, his judgement is now suspect.
Huang's argument is (2). In this case, the evalaution of his judgement is:
(A) Is this a one-off mistake? If so, he should be retained - he's human.
(B) Is this a pattern? If so, he should be fired - he's an ongoing risk (e.g., via regulatory action).
Huang resigned because the second affair indicated (B) - he's an ongoing risk.
As a result, the question of "who knew what when" is actually beside the point. Retaining Huang risks ongoing, overwhelming government action for evading sanctions - and the board members want 0 personal liability in that investigation.
After they caught him in a lie, they may not have been able to not fire him.
This stuff has probably been going on behind the scenes for a while. The leak about the affair is just the catalyst that accelerated it all.
Nvidia is a profitmaxxing company in a profitmaxxing economy and so it is going to profitmaxx; the criticism should lie with the US government: https://www.reuters.com/world/china/republican-senator-urges...
Exactly my read as well. "If you find two ants in your kitchen, the best estimate of the total number of ants in your kitchen is not two."
If you dissect the timeline of the events and various statements, I think the 2nd affair with a subordinate employee K Powell was the reason to fire Jensen Huang -- more so than the allegations of secretly selling to China.
This is the timeline:
Nov 8 : Jensen issues apology. NVIDIA allows him to use nvidia.com for this
Nov 8 : The Board of Directors publicly backs him up with a "vote of confidence" by stating: *"no reason to believe that either national security or NVIDIA secrets were compromised" and that "The Board has confidence in Jensen and his leadership team."* -- from https://nvidianews.nvidia.com/news/statement-from-nvidia-board-of-directors
Nov 11 : 2nd affair gets exposed in the news
Nov 11 evening : emergency meeting of Board of Directors is called to fire Jensen Huang
Before Nov 11, it looked like NVIDIA might try to ride out the storm and keep Jensen at the helm especially if they saw that the China accusations and evidence wasn't strong enough to be a problem. Again, it was totally unnecessary for the board to publicly back him up if they had plans to get rid of him because of China allegations. However, the Board of Directors emergency Nov 11 meeting is inconsistent with the Nov 8 statement affirming Jensen's leadership.Nov 8 was coordinated with carefully crafted communications showing that Jensen and BOD were aligned. Contrast with Nov 11 being chaos and an emergency meeting convened.
Reading between the lines, what probably happened is that Jensen didn't come totally clean with the board on Nov 8 and tell them about the 2nd K Powell affair. Instead, the board got totally blindsided and lost trust in him. (Is there another 3rd secret affair he's hiding from us?!?) It was impossible to keep Jensen under those circumstances.
Money is like oxygen. If you deny oxygen/money to the Chinese chip market, they can't grow as fast. By supplying China with chips, all the money will go back to the US and not enough money will go to Chinese chip manufacturers to help them advance the way Nvidia can.
This is what Jensen was saying and he is 100% correct here. The chip ban to China will really help them catch up because the money will recirculate since they have no choice but to buy Chinese chips, and in the end it will make them much stronger.
The second one -- affair with a direct report -- is what he is being fired for. This one is hard for the board to ignore as it directly violates HR rules.
As for the ban for national security reasons, the ban is not super useful because it only prevents hardware from being physically imported into China. If the hardware is physically located in Singapore and Chinese researchers use those machines remotely, the US does not ban that it. If so how meaningful is the ban?
The quote about succession planning is also a little out of context, he phrased it more that there should be continual information and responsibility sharing such that succession planning is effectively redundant or unnecessary.
I always wondered why investors were ok with not having any control due to dual structure shares and board composition. A "Great Men" theory of the world is a convenient and satisfying narrative, but it's usually not the best at explaining things. As the saying goes, "The cemeteries are full of indispensable men".
It was illegal, and I'm sure if this comes to light there will be (major) repercussions, but I don't see it as an immoral move.
Cheating on your wife of 40 years OTOH...
I don't see it. Nvidia has a lot to answer for, and the fact that Jensen was very successful as a CEO does not shy away from the cracks in the structure.
Shareholders don't care that he's having sex outside his marriage.
Jensen messed up by splitting the power between all board members equally, and not getting a bunch of his family on a board.
The lesson is that if you do the right, non-authoritarian thing, you get punished
Did he? A singular CEO with 40+ direct reports and no succession plan... that alone feels like a very short term mindset to me. He could have been hit by a car tomorrow, what's the contingency plan there? Not to mention the whole 'violating export restrictions' thing, that's typically not great for the long term viability of a company either.
When Cambridge Analytica happened, no one kicked Mark out of Facebook/Meta.
Not making excuses for Jensen, at his status and power level, this stuff goes with the territory. But I cant help but see a double standard here.
Watched the interview and as far as I see he was getting 'heated' because he was speaking the obvious truth and being frustrated by poorly informed questions.
This is entirely expected from somebody like him who has a long history of being on the right side of technical argument.
He is a huge loss to Nvidia and the US - replaced by a ex-CFO.
This whole saga has been a ridiculous waste of everybody's time and energy. There's no compelling argument that points to these new innovations not otherwise happening if RAM prices weren't artificially inflated by the fever dream of private equity.
Why? He caused this. He chose to have affairs with a spy and a subordinate. Now he has to deal with the consequences.
Parent is oblivious to this and has already made up their mind that these policies don't exist
> The fact the he did it, and that she got promoted seals his fate as a predator in my mind.
Musk has done, and continues to do, the most horrific stuff, including things far more questionable from a US security perspective. Altman's reputation is not one of honesty and fair dealing. We've had very clear evidence of Microsoft's leadership being caught in honey-traps by foreign governments. No heads rolled.
Tin foil hat take: I think Jensen's been sniped.
This is a terrible decision for NVIDIA and the broader economy. Jensen gets to retire with $100B+ and not deal with the crap.
These aren't just affairs.
Also, fraud remains to be seen, and shopping chips to China sure seems like questionable judgement.
I think you are too narrowly focused.
I had only heard about a consensual relationship.
"It's not the crime, it's the coverup."
I think the real risk for the economy, the AI industry, and the US is that Nvidia becomes JUST another company. Nvidia losing 50% of its value from its peak means it’s still a $3 trillion company. What if it loses 90%? Then it’s the size of Intel which we generally agree is an important part of the industry.
Nvidia can afford to invest in companies because the stock is so high and capital is incredibly abundant. Those companies buy Nvidia products at the 50% NET margins. Nvidia gets a ton of cash. Earnings are crazy. But if this cycle stops and efficiency wins in the market (lower margins, more competitors, more options), then Nvidia goes back to where they were which was a Tier-1/2 supplier battling for margin and market share. That is not a $6T company.
Now without him there will be a vacuum for the ideologies of Dario / Elon / Sam to fill the void.
Apparently I am able to, what was that?
I assume leaning on Nvidia to integrate chip verification and tracking to not let any unmonitored compute exist in the world is next, which is also in line with what the leading US labs want.
Not a single inch of empathy for his wife and family, nor the misled investors and staff.
Having these people saying otherwise is pure hypocrisy.
And: the public only knows what's made it out. Who knows what else has come out in private.
Like, maybe him stepping down is the right decision. But it feels like a hit job and something that was pushed through, rather than something that was properly considered as what is actually best for the company.
From all the reporting I've read, it is hard for me to avoid the conclusion that NVIDIA sold to China despite the export ban. I'd love to be wrong
The example are contrived, but still very real.
Wish me luck!
It obviously isn't always the case, but at least where I live knowing the right people is extremely important if you want to get a job in the IT/software industry. I wish the importance of networking and getting to know the right people was stressed a lot more during my education and formative years.
The fact that the government can get taken down by a central server is mental.
Is it some sort of spying thing? Incompetence?
Of course I haven't received a single email because every important email is sent via Microsoft but I still feel superior because I'm not affected
I hope this sort of stuff pushes us to a distributed model rather than a centralised mono-provider one.
Along with the daily Github outages recently, it's making me do an audit of my dependence on Microsoft.
It’s not the first time of course. Something similar happened with banking, where the US got stuck in the mid-20th century with people still using paper checks, no access to instant digital transfers, etc.
Learn to read and don't spread these editorial title lies.
A better title: "TSMC reports employee data found in public to SEC" or something like that.
TSMC has found that some of our current and former employee data was made available on an external website without authorization. Currently, we have found no evidence that TSMC’s systems have been compromised. This information has been verified to match data stored in the system services of a cloud service provider, but is unrelated to the company’s operations or customer-related data. We have been informing impacted personnel and are providing them with necessary support. TSMC has engaged with both the cloud service vendor, the relevant investigating authorities, and a third-party cybersecurity agency to jointly ascertain the facts and take necessary actions. We remain firmly committed to protecting employee privacy and safeguarding data security.
I'm guessing they used 3rd party HR software and that software got hacked.It's been 18 hours since the start of the outage. If service isn't restored in the next 18 hours (36hrs+ outage), you can get the entire fee waived.
Microsoft gives a 99.9% monthly uptime guarantee, which means maximum allowed downtime is 7hrs.
> Engineering teams are cycling affected infrastructure through traffic drain, restart, remediation, and staged return-to-service while monitoring closely.
so they have no clue what is going on...
I’ve seen the chats about cascading failures and it’s impossible to not expect AI is involved somehow (likely brain drain and what not), but what are the options?
It sure looks like “fired everyone who knew what they were doing” but I feel like at this point you’re breaking open the check book and calling them, and even if they did that I’m assuming this shouldn’t be a “normal” event?
Now I will say there is more to embroidery than one might first expect. It's got some overlap with image editing but only to a point. The stitching really matters and to make it stitch out well requires sort of a lot of rules about how to do that. Most software is either $1000+ and requires you take a course, or there's a subscription but you're really just downloading designs and sending them to your machine. There's a fairly healthy market of people selling embroidery designs on Etsy in part I believe because digitizing for embroidery is not that easy and requires expensive software. So people often just buy designs rather than make them themselves.
To make things worse, basically no embroidery digitizing software was available on Android or iOS and it usually was just Windows. So I set out to build the easiest digitizing software which would also work on mobile devices which means you have to design around things like no key combos, no right click, etc. So while I expected it wouldn't be that hard to write embroidery digitizing software, I learned that just being able to send a PES/DST/etc file to a machine was really not the hard part. So kudos to the OP writing his own software. Gotta break that grip of Big Embroidery.
The PE-150 is a great example of the absurdity/sadness of the embroidery world: today, you can have those machines for $100 on eBay, completely functional, but being sold/abandoned because they use a proprietary cartridge holding proprietary data that no one wants to deal with.
Even novel Brother machines that have USB connectors still force you to use terrible apps and lock you into subscriptions. This entire world - which is/should be full of twee and about making beautiful things with your hands - is somehow corrupted with this merchandising/everything-software-should-have-a-price mentality which is just so sad.
Nothing to announce announce, but I 100000000000000000000000000000000000% want to make embroidery machines more accessible, open, and free of this balagan.
A few more links on this topic:
- https://github.com/CreativeInquiry/PEmbroider !! Check the "Motivation" section of the readme!
- https://emberdesign.net/ - a startup by another friend, with a really great online editor, and which exports a variety of formats
[0] https://owentrueblood.com/blog/2024/12/10/reverse-engineerin...
I'm sure if the field hadn't been moving so fast they would have tried, but it's kind of already decided unless they take away general computation.
Claude now refuses to reverse engineer games to fix compatibility issues on Linux/Proton, particularly in relation to anti-cheat software. Codex still works, and while Claude is still useful for other types of work, I am considering a migration to GLM and DeepSeek.
When I'm doing stuff like this, I always have this thought in the back of my head like "man, this is awesome- I bet they try to stop people from doing stuff like this soon". Like there'll be an in-built "do not assist in reverse engineering or circumventing subscription-based services" instruction.
Seems very promising. I have a home recording studio, and there's a small revolution going on with people cloning subscription-based stuff. Some brands are notorious for making expensive equipment which they just stop supporting after 10 years, sometimes rendering them obsolete. Or tacking ridiculous licensing mechanism onto the equipment.
Codex did it and flashed it in 1 minute.
I'm all for AI safety, but this sounds more like corpo preference safety to me.
The more people do stuff like this, the better. Subscription based e-waste is awful.
Your app represents a future that I, an old, actually love. When I see projects like this, I don't think about ongoing support or if the project will be abandoned. I think, "Woah, I hadn't thought of that. Now I'm going to try it. Because why wouldn't I try to get all the value I can out of my $20/month Claude subscription?"
Even though I've been a developer for 30+ years (and a nerd for longer), I've never felt so motivated by the potential of a technology to make an individual's life richer. Thanks for sharing it.
And if you wrote it by hand, my hat's off to you for sharing that, too.
The lesson most people learn from this sort of outage is that things bounce back afterwards, and the risk is minimal enough that a day out of every few years isn't enough of a problem to pay more.