- Quoted phrase in the opening sentence
- Opening quote: “side quests”
- Quoted phrase in the opening sentence
- Opening quote: “always exploit”
Seems weird when it's become apparent that LLMs aren't really special sauce outside of just having the compute.
The transformer based approach is seen as a highway to race down in terms of training and building better models with minimal organizational distractions allowed while every company puts its foot down on this highway. Exploratory research into other AI methods is highly discouraged for now and organizations are encouraging transformer based scale out and iteration as a top priority.
As an example Meta literally fired it's previously head of AI who held views that transformers won't scale. He wanted to veer off the highway and heavily fund exploratory new types of models. They got someone willing to scale the current method out. Essentially the messaging it "Foot down, there's a straight highway ahead don't divert".
I have the view this makes sense too. Current top models are mere (mere in big datacenter terms) trillions of parameters right now. We definitely haven't hit the ceiling and everyone's noticing "Hey this Opus 5.5 with the increased parameters seems much better than the last one". Well that's a trillion parameter model. We'll soon start hitting 10's of trillions and 100's of trillions of parameter models. Unless there's a ceiling which nobody expects they should keep scaling just as they have been. I really do agree that it'd be insane to veer off this highway with exploratory research at this particular point in time. Foot down, get to 10 trillion parameters, see if the scaling holds, if so off to 100 trillion parameters. If the scaling continues to hold we'll be asking the AI to do the exploratory research for us soon enough.
Perhaps "The End of Microsoft Research's AI Frontiers Group" would be a less ambiguous title.
- Quoted phrase in the opening sentence
- Opening quote: “look at these 2000 lines of text and think really hard”
At least for me. Am I missing something?
Their support has acknowledged the issue to us over a month ago but no fix as of now.
I swipe Ireland, it gets it right the first time.
I swipe “united ireland” a hundred times, it never work.
It’s not probability.
Some blocked phrases are quite arbitrary: Australia, India, Kenya, Nepal, Pakistan, and the US are the only countries you can't say are bad or poor. Trayvon Martin is the only full name you can't write. You can call Putin, Biden, or Xi a dictator, but not Trump. In other languages, the suppression can be even worse.
And some are ridiculously overbroad: "is important" and "black woman" are blocked! When I try to swipe them, I get "is impotent" and "blank woman".
I think it's not totally unreasonable for a list like this to exist, at least for autocomplete, but it ought to be transparent so people can see the impact Apple's choices are having on our use of language. Individually it's a small nudge, but on the scale of billions of iPhones it can shape the discourse.
There are cases like https://gizmodo.com/a-cellphones-missing-dot-kills-two-peopl... where, in 02008, Ramazan Çalçoban texted his estranged wife in Turkish, "you change the topic every time you run out of arguments", but the lack of the Turkish dotless "i" on his cellphone turned "sıkısınca" into "sikisinca", which looks more like a misspelling of "sikisince". So he ended up texting her, "You change the topic every time they are fucking you."
Long story short, his father-in-law murdered him for thus insulting his daughter's fidelity, following which he murdered the daughter in question, and the father-in-law committed suicide in prison.
So there might be certain words and phrases that it's best for your computers to not send to other people on your behalf unless they're really sure you mean them.
This would probably be a bad customer experience for many people, even if Apple could prove these are simply the most common n-gram completions.
And if they altered their training data to avoid completions that are politically unpalatable, IMO that's actually much much worse than a grey list.
I'm a big free speech guy but this one is genuinely a bit complex.
For anyone wondering the degree to which this is an exaggeration: The iPhone will physically work against you by invisibly changing the size of keys on the keyboard.
Every iPhone, since the very first, dynamically resizes the hitboxes of the iOS keyboard. This is invisible, can not be disabled, and the hitboxes overlap those of other keys.
They used to advertise this as "dynamic tap zones" but they removed the video for it, but you can see their advertising for it here: https://www.youtube.com/watch?v=wPmVKyhyl9U
Swipe "Ireland" alone, works. "United" alone, works.
Same swipes but back to back, and now it's suddenly "United Iceland".
There's no predictive text involved here, this starts going into the willful territory.
- There is a chance, quantified by some metric, where a person is suggested a word that is offensive to them or their culture because a probabilistic model made the wrong decision.
- There is no chance that the above happens.
Apple will choose the latter, time and time again, because they are risk averse and freedom is not particularly high on their values list. We’re talking about the company that doesn’t let you type the Taiwanese flag in China, or the pride flag in Russia. It’s no surprise you’re not allowed to swipe “fuck” in the United States. I used to work at Apple.
“Male chauvinist pig” is something many would be very upset to send to their boss by accident. So it’s perhaps understandable some such phrases are not excluded from the guesswork.
Producing "trump is …" when a user wants that is one thing. Producing that when the user wants something else could be construed as coming from the software itself, and we all know that people looking for a shitstorm will seize the opportunity.
So probabilistic software like a swipe interpreter will often be written to avoid some guesses, to lessen the chance of a shitstorm. Users who want that output have to use a dependable input method instead of one based on probabilities and guesswork.
IMO this avoidance isn't among the ten worst consequences of shitstorms.
- Quoted phrase in the opening sentence
- Opening quote: “then why can’t I swipe it out?”
Because then it could look like they're suggesting the word to you, rather than you explicitly selecting it, and they don't want to be accused of suggesting any terms that might be controversial, for the same reason they don't want to be accused of suggesting profanity to their users.
To the shocking number of people here who are justifying this or explaining how it only affect swipe input as if that makes it better: soft coercion is still coercion. If I made a physical keyboard that stops keys from working when you're about to finish typing something I don't want you to type, but you can still circumvent it, I still made a censorship keyboard. It's a sign of the times with how used tech companies are to 'softly' moderating societal trends by unilaterally tweaking the software that everyone uses.
- Quoted phrase in the opening sentence
- Opening quote: “bug”
- Opening sentence uses a quote to challenge or distance a term
- Opening sentence challenges the term “bug”
Considering arguably the keyboard itself is one of the most common "touch surfaces" on iOS, the level of neglect and level of overall quality is staggering. There's some serious incentive problems at Apple if nobody is tasked with making one of the best digital keyboards in the world.
PS - Whoever came up with the concept, on swipe-style touch keyboards, of the "Tap Map"/Heatmap wherein keys visible location doesn't match their touch area - so that BY DESIGN it always provides an inconsistent user experience needs to be fired into the sun. You literally cannot disable it on most mobile keyboards. It is on iOS's native keyboard AND Microsoft's Swiftkey.
- Quoted phrase in the opening sentence
- Opening quote: “self-contained OS on a USB drive with encrypted persistence”
- Quoted phrase in the opening sentence
- Opening quote: “The results of this experiment was [sic] somewhat discouraging”
" Fermi writes that "The results of this experiment was [sic] somewhat discouraging" presumably because of the absorption of neutrons by some unknown impurity. So, in December 1940 Fermi and Szilard met with Herbert G. MacPherson and V. C. Hamister at National Carbon to discuss the possible existence of impurities in graphite. During this conversation it became clear that minute quantities of boron impurities were the source of the problem. "
As a result, you got stories about dog sized ants that were nonsense but then also stories that were legit.
People tend to be rather binary with how they process information, so when there's things that are wrong they group everything in the 'wrong' category.
Definitely a lot of false negatives with how Herodotus was regarded. Much like how the prevailing consensus was that Troy was a fictional city until it was discovered.
For those that don't know about him:
- he was an ancient historian
- he made a lot of "wild claims" e.g. there was a tribe that made clothes out of human skin
- historians assumed these were myths or exaggerated
- recent research has shown that many of his wild claims were true (even the one above) [1]
0 - https://en.wikipedia.org/wiki/Herodotus
1 - https://www.livescience.com/archaeology/ancient-scythians-us...
- Quoted phrase in the opening sentence
- Opening quote: “pots aren't people”
> The findings challenge decades of scholarship that played down the importance of migration and emphasized the spread of language and culture between settled populations.“
> “What the genotyping is showing is what the ancient sources report, and what late–20th century historiography has been working very hard to deny,” says Yale University historian Noel Lenski, who was not involved in the work. “We have to … admit the sources knew more than we thought they did.”
If I'm reading that right they are implying academics were pushing the idea of cultural exchange being the most valuable thing, a vague menagerie of local and foreign cultures coming together, rather than a specific set of people arriving from a far off land to dominant an existing empire and influencing the entire region in the process.
The other two formative experiences for me were using Cincom’s Mantis on the 3278 and Dataflex 2. Both made it obvious how quickly one could go from screen to persisted data, CRUD, application logic, and reporting with the right level of abstraction. Much later I described Dataflex as “Rails for the VT100”. Sadly, Cincom will not provide a hobby license to run it on an emulated 4381, and won’t even give away free manuals, and the company behind Dataflex (I don’t remember their name right now) seems to not even have any versions from that time. When I read Paul Graham’s Beating the Averages, it all clicked perfectly.
It is such a big category error to say "we're building simple apps without looking at code, therefore no one reads code anymore."
I'm not sure if this comment is meant as a little joke or observation, but I keep on seeing serious comments like this. It is serious anti-intellectualism to scoff at people reading programming books or understanding things.
Of course, lots of things that make modern software bigger are things we very much don't want to do without--full Unicode support, accessibility, clever rendering, improvements to quality of life. But our 'foundations' are also built higher; the Web platform and a bunch of native toolkits provide a lot out of the box. (The platform doesn't include all the widgets that are now quasi-standard, of course, so we build a lot on top anyway.)
There are also a ton of useful third-party libraries you can call on without getting too bloated, and there's a lot that would have been bloated then but not now: in the 90s you probably wouldn't embed an SQL DB you didn't really need, but SQLite is a pretty safe dependency now.
Nothing in particular I'm really driving towards here but I think "how do I keep the code to functionality ratio good" can still be an interesting angle from which to look at high-level design decisions even as a lot of the circumstances and numbers have changed over decades.
Indeed. NFS:U2 came out in late Nov 2004, goes double platinum, sells 11M copies and EA shut down online multiplayer in Aug 2006. Reasonable debate can be had about time frames but games out of print over 20 years should be an easy call. The problem isn't just copyright duration, it's also lack of granularity. We need a middle-ground like 'Expanded Fair Use' which kicks in after 20 years and enables non-commercial sharing of the actual code, without also looping in rights to long-lived franchise characters (Mario, Sonic, Zelda) or an entire 'cinematic universe'. It's the 'all-or-nothing' that lets publishers effectively block everything by arguing "But then anyone will be able to make all-new Need For Speed or Final Fantasy games."
As for "all sorts of cool possibilities", just look at what the retro modding community has already done with HD texture packs, expansions and fan translations.
- Quoted phrase in the opening sentence
- Opening quote: “Riders on the Storm”
With movies more so than with games, the piracy scene and the organized/commercalized bootleg scene were never far apart. Few people sit in a cinema with a camcorder "for the love of the game" and risk criminal charges, while many people will do the same thing for a few hundred bucks.
__
moderators: "Let's talk about privacy, sorry Mark, who is here..."
Jobs: "We've always had a very different view of privacy than some of our colleges in the Valley..."
https://www.youtube.com/watch?v=uk_pLVQRNzg
The permissions system in iOS feels like it is following Jobs' ethos. Truly amazing all these years later. Kudos to the iOS team!
Who ever implemented what is called out in TFA, is clearly diverging from what makes Apple and AAPL very special. Please stop that.
- Quoted phrase in the opening sentence
- Opening quote: “you're not the customer you're the product”
- Opening sentence uses a quote to challenge or distance a term
- Opening sentence challenges the term “you're not the customer you're the product”
When I buy a table saw, I don't expect DeWalt to hang around my workshop telling me what I can and can't cut, watching and logging what wood I use and when I use it, what my finished product looks like, changing the features and buttons on the saw in the middle of the night. Why the fuck do we tolerate this from software??
It's gotten to the point that Linux is the only sensible option, and not all distributions! Some of them are letting telemetry and metrics and A/B testing creep in. And some third party Linux software is just as intrusive and demanding as commercial Windows/Mac software.
It does the last 4 commands:
cd ~/Library
rm -rf IntelligencePlatform
touch IntelligencePlatform
xattr -w com.flock.mode 444 IntelligencePlatform
chown root:wheel IntelligencePlatform
chmod 444 IntelligencePlatform
chflags schg IntelligencePlatform- Quoted phrase in the opening sentence
- Opening quote: “allowed”
- Quoted phrase in the opening sentence
- Opening quote: “appliance”
An entire generation thereby grew up with personal computing represented by vendor-controlled handheld appliances, and naturally regarded actual PCs in the same manner.
And their skills to manage a PC atrophied too, leading to a feedback loop.
Agreed. None of this is magic, and the vast majority of us have been handing this data out to SaaS providers of all sizes and shapes for decades now, for free. Many HN people even _build_ those platforms.
Would I like to see more controls around this? Yes, absolutely. Having some random app be able to read this data with only a single permission barrier is room for improvement. I would also like to see more tooling included to inspect and correct this data. Something like a Monica[1], but local and built on local data. Suddenly address book data is no longer a list of names and numbers, but a part of a data structure that you can use to surface information, like friend's preferences or vacation photos together. Not everything that collects data is insidious and a Black Mirror episode in the making.
- Quoted phrase in the opening sentence
- Opening quote: “someone is making your computer do X behind your back”
I wouldn't be surprised at all, given that Photos knows the names and faces of all the loved ones in my life. Not just people, in fact, but the "furry friends." And Calendar and Reminders knows where I go. And so forth.
The feature is genuinely useful. Only thing is it should be opt-in and have guarantees about where/how the data is stored, such as encrypted-at-rest. I trust Apple with that more than all other Big Tech combined, but still in a "trust, but verify" way.
I provided a "better DNS configuration" suggestion, a month ago, but at this point I doubt it will remain useful [†]:
<https://news.ycombinator.com/item?id=50071310#50078574> [have router's DHCP auto-issue DNS to your pi-hole].
[†] I am but an observant (on my DNS logs) bluecollar poweruser – my tip will reduce only one vector, and we probably live in a world where IPv6 DNS addresses are [about to be, if not already] being hardcoded into TVs, computer security chips, and phones
----
At an absolute minimum: LittleSnitch will NOT help you here (and isn't doing everything it claims to, under the hood); it is entirely impossible for LS to stop pre-boot DNS queries; even under normal operation (e.g: when a dialogue pops up to `ACCEPT/DENY` a DNS resolution) your ISP still sees every. single. URL resolution (even if you `DENY` the connection).
Before you write "no it doesn't" please check your own DNS logs. It do be like it is.
Likewise, you could say “Tell my boss I’m not coming in tomorrow” and it would have to know who your boss is.
We could get super angry about this, but seeing as it’s fully local and doesn’t leave the device, it really doesn’t seem like a big deal. Is it much different to your car remembering who you call and where you go at what times for navigation purposes?
Chrome had already solved the hard problem of "how do we support webcams on linux, windows, and mac" and you as the dev just needed to use the Javascript webcam api. ( https://developer.mozilla.org/en-US/docs/Web/API/MediaDevice... )
In C++ land, you are ultimately going to either need to cook up your own webcam library or look for a 3rd party (which, in 2010, likely didn't exist) to provide it.
Today, there are a few easy to discover rust crates which provide cross platform webcam apis. But further, rust does cross platform better than C/C++ IMO. It's really easy to change how code is compiled based on the target. That was all well thought out because rust was supposed to be a language for browsers.
I think people become super lazy... "Ah, it's will ocupie a few hundreds of RAM, no problem, let's ship it."
They are companies with billions or even trillions shipping shit with Electron.
This bit is obviously subjective, but Rust as a language is simply not ergonomic enough for a lot of the use cases people are trying to apply it to, especially UI development. Most of the Rust UI libraries I've seen to date either use a more imperative style, or attempt to replicate more declarative approaches like Compose UI or SwiftUI, but without the nice language aesthetics that make it simple to read and write. And if the counter-argument there is for LLMs to be reading and writing this stuff, that's a whole other can of worms: (1) The subsidized inference era is quickly coming to an end and (2) if you're using LLMs anyway, why not maintain apps that are as native to each platform as possible?
The practical takeaway, however, seems to be an interesting "don't use max mode". The passage about deepseeks weird API behavior dropping to pretty much zero the moment you used high says a lot.
Also, personally, even though Opus is clearly so much ahead of the Open Source models after the changes, I think the adjusted score is still very much competitive. I see other commenters using this as proof to justify why their opinions of the open source models - but a drop from 74% to ~50% is, while a big drop, still about a ~50% pass rate afterwards. That's far from useless for coding, especially since DeepSWE seems to be such a difficult benchmark. It simply seems to more accurately reflect the capabilities of these models.
All in all, I'd love to see more of such articles, and will probably keep an eye on this one - if they continue "benchmarking" these models this way.
Look at GPT 5.6-Sol vs. GPT 5.6-Luna in particular.
(The GLM pair shouldn't be expected follow this pattern, because they're misnamed: GLM-5.3 and GLM-5.3-flash are distinct lineages. The -flash is an early release of a new pretrain on a new architecture).
If this *is* the main factor, I'd expect Kimi K3.1 and Gemini Carbon would also do well on this test (unlike Gemini-flash); while Haiku 6 would collapse (breaking the trend of the other recent Claude models). It's a falsifiable prediction.
- Quoted phrase in the opening sentence
- Opening quote: “horizontal wealth”
"But horizontal wealth means not letting your increased income dictate your tastes. You like books and now you have money? Buy more books! Change those catenary bookshelves for good hardwood ones! In my case, build a library extension to your office. And, of course, you buy what will be useful for that most wonderful of pursuits, blind research, which is research without direction for the sheer joy of it."
(Original source is his book A Slip of the Keyboard - this linked blog post quotes that bit.)
I hear that excuse constantly on the web if I try to correct people. And maybe it doesn't "prohibit access", but it does force me to think about things that aren't the point of the message, in order to figure out the point of the message.
I've given up correcting people. I'm old enough now that it's time to just watch people's choices come back on them. The whole world will pay for it, and I've done what I can.
I can't even get people to stop using "weary" to mean "wary". That's a whole different meaning, and there are so many people that use it wrong.
One distinct memory stands out. I could not wrap my mind around the Object-Oriented extensions on Pascal. I read, re-read, and re-re-read the section of the book describing what polymorhpism is and how it worked, probably at least a dozen times. Somehow it seemed really important and I kept coming back to it. I worked the examples. And then one day the ah-hah moment arrived. I remember it felt like a shock, why it was useful and what I could do with it. I promptly used my knowledge to write a new game using OOP principles to guide the rudimentary "characters" of various geometric shapes at the core of the game.
Fun times, and very formative to me!
And: "A Personal History of Compilation Speed, Part 2" https://prog21.dadgum.com/47.html
Perhaps "The End of Microsoft Research's AI Frontiers Group" would be a less ambiguous title.
If I were you, I would probably ensure that the fridges you used have residential or commercial (aka non-DC) IP addresses. Maybe Shodan uses this data for their "honeypot" label, my experience was some time ago so no idea though.
The directors always try to avoid this criticism by saying, "Well he goes to jail in the end," but I think it's a bit of a cop out.
I tend to agree with the idea that art should try to embody some moral values, which goes against a lot of contemporary thoughts on aesthetics. But I think it’s a basic fact that people look toward fictional characters as role models, directly or indirectly. Avoiding that responsibility is not good.
- Ordinary term used more than once in the same comment
- “preferred service” appears 2 times
- Whether family or friends, always suggest it as a "fallback" for when "preferred service" is down and to use when discussion ultra-sensitive topics or sharing financial files. With time they may just stay connected to it if the friction to do so is low. It has to be their choice, not yours.
- What helps? Add more incentives such as perks that "preferred service" does not offer. No file size limits. No or few file type limits. No content filters. Add emoji's or gifs that align with their inside jokes and their style of humor. Keeping random people away is very important. Making your friends and family administrators or moderators and let them decide how it operates. Do not moderate or power-trip over your friends or family or they will leave. Limit rules to "Do not break the laws of this country in public chatrooms" and private chat rooms if you do not physically host the server. Database and file backups become even more important but they will feel in control of a thing. Have a keyword or code word they all learn means to go check the private platform that "primary platform" would not recognize. Maybe when they see "Turtle Farts!" they know to go log into your service.
- They may just stay on their primary preferred service. Every group is different and not every group will see the benefits of your fallback service. They may not appreciate the work you put into setting it up for them. Do not expect to get the friends of your friends and family to move to your service. They may not appreciate the OpSec value until "Bad Things"(c)™2038 happen to enough people. Just be there for when it does.
- If there are concerns that you would spy on them, consider instead using eJapperd or Prosody as you can implement OMEMO for individual and group end-to-end encrypted messages. Set up TLS fingerprint checks from multiple locations to reduce the risk of MitM attacks until XEP-0474: SASL SCRAM Downgrade Protection is fully supported and active by default on both servers and all clients. This is the only thing holding me back from setting up a public Prosody server as 0474 is still experimental. See [1]
- Avoid (block) federation on private servers for friends and family otherwise it is not a private server.
BUT! I feel that lost hope could/will turn into a new forms of "min-maxing" or whatever where we will see people focusing on small, efficient software to make the most of what we have to weather the drought. We got spoiled with cheap hardware. Time to make the most of all that cheap hardware we have.
Though of course, replacing this hardware as it breaks down will be expensive. Perhaps we could go back to 1/2/4 GB RAM sticks and 256/512 GB Disks. Then run minimal Linux installs, minimal GUIs, minimal services, minimal software.
To amend this: Reading the 8088 domination article and the little $8 RISC-V Linux board I think we have plenty of opportunity. Maybe we learn to shed our inefficient BIG computers for small, low power, purposeful, yet general computers. I play with Plan 9 a lot and its portability, small footprint, low resource, and distributed design has showed me we can do a lot with a little as long as we stay disciplined to keep things simple and to the point.
Not everyone subscribes to the "cloud compute is more important than everything else" outlook, but everyone is affected by it nonetheless.
- Quoted phrase in the opening sentence
- Opening quote: “written from scratch”
Projects like these are defiling the programmer's guild.
It's definitely not categorically malpractice to not use Word. Depends entirely on the situation. If a court demands docx and you file an rtf, or you refuse to read a client's redlines because they're in Word, sure, that could be malpractice.
The only proprietary software that I think _might_ be malpractice not to use is a legal research service with a capable citator, like Westlaw or LexisNexis. If you rely on a case that was overruled, you can't defend a malpractice suit by saying, "I didn't want to pay for the software."
I hear CourtListener (a free legal research tool) is working on a citator--hopefully it's competitive with the paid services.
Though to be fair the concept of a powerful city lost to time is something that resonates a lot, and there's multiple similar myth like the isle of Ys, the lost continent of Lemuria, or in the arabic folklore 'Iram of the pillar', often called 'Atlantis of the sand'.
[0] There was an episode on it in the excellent podcast 'In Our Time' by the BBC (https://www.bbc.co.uk/programmes/m001c6t3)
I found Fira (nearby "large" town on Santorini, very close to the sites) miserably crowded, but seeing Thera was very cool. This is all about 1500 years older than Pompeii, so a very different experience IMO.
So the main improvement to memories I'd like to see is better judgment on which ones to create.
The parts about "dreaming" make me think of how agent transcripts contain a ton of information, which I think is currently under-utilized. I'll repeat advice I've seen here a few times: ingest your own transcripts into duckdb (claude code will build that pipeline in about 10 minutes) and do some data analysis on them. Some ideas if you don't know where to start: look at tool call failures to see if you can prevent common ones, check out if you have large cache misses and what causes them, look at various efficiency metrics over time (tokens per $, lines of code per tokens, etc.) with main thread vs sub-agents, etc. I started with this sort of things, and ended up following some threads to interesting insights. For example, I found indications that I tended to make Opus 5.5 / Sol 6.1 divide their work into units that were too small. They have the capacity to work on more stuff at once than older models, and I wasn't really taking advantage of that.
People linked to them came up with the "prediction" nonsense to sound modern and technological as a way to dissociate themselves from the life/family destroying gambling of old.
But they are gambling, pure and simple, with no value to society.
- Quoted phrase in the opening sentence
- Opening quote: “versioned structs”
If you have structs that might need to be extended, you can use "versioned structs". One popular technique is to include the struct size as the first field. The struct size is set to `offsetof(mystruct, lastmember) + sizeof(lastmember)` and the receiver of the struct can use this to check if a member exists. The ARA API is a good example: https://github.com/Celemony/ARA_API/blob/65ec5c43b943a48cb54...
The map shown of 14th century Europe is doing better than most in that recognizes that several borders (but still too few) are fuzzy rather than lines on a map. The other problem you run into is that, in most non-state societies (in the Weberian sense of "monopoly on the legitimate use of force"), political linkages tend to be fragile and based more on personal ties so that a Big Man's successor may well repudiate the alliances and homages the Big Man makes. Which makes it hard to draw maps, because is Burgundy a part of France? The Duke of Burgundy is as often as not fighting the French king.
This is what makes adjudicating the "oldest continuous republic" hard, because at what point do we say that the homages San Marino is paying stops counting as being part of that country?
Basically when somebody makes a pricing error on a decentralized cryptocurrency exchange there's a mob of automated arbitrage bots that try to trade against it. Who wins and actually gets to make the trade is mainly determined by how high of a transaction fee your bot is willing to pay.
You can win instantly by offering to trade with a transaction fee so high it almost wipes out all but a cent or two of the entire arbitrage profits, because nobody else will pay a higher fee.
Instead what happens is bots offer the lowest possible fee, then immediately increase their fee as soon as they see a competing bot try to take the trade with a higher fee. This results in the bot offering the highest fee changing every several milliseconds as the fee quickly grows.
The way Ethereum mining works, this results in the winning bot being randomly selected from among the set of bots "bidding" up the transaction fee: blocks are mined at random intervals of iirc about 20 seconds, and whoever had the highest bid at the time the next block is mined wins. <20 seconds of bidding is short enough that even with the bidding, the transaction fee doesn't get bid up particularly high, and the winner makes out with a highly profitable arbitrage trade.
By giving up a guaranteed win and instead rolling the dice against other bots, transaction fees stay low and everyone has a higher expected profit. It's coordination that happens without any communication between the participants and quite possibly the participants didn't realize they were in effect cooperating when they added the high speed bidding behavior to their bots.
Let's imagine that everything you say is 100% correct. LLMs can't access materials for developing weapons without weapons tests conducted at their behest, and humans choose not to - of their own volition - use world-ending weapons.
Let's also imagine for the purpose of conversation that one of the things that can be gotten at an extremely high skill level through increased intelligence is the power of persuasion.
A "superintelligence" persuades humans to perform weapons tests and then to give it the ability to use the weapon.
I think everything you've said is reasonable except the bit where you think step one of an "AI" exterminating humanity wouldn't look like a human doing it. In my opinion, the most likely vector for disaster is a human being who is "guided" by a computer model to do something unfortunate, not the model doing so directly.
But that’s just the parts of ML research that take place within your compute environment, namely experiments on model architecture, training strategy, UI/UX, etc. The algorithms and software.
Just because that part of research is progressively accelerated does not imply that we’re on a collision course with world-ending “superintelligence”. It just means that you’ll get to the best possible model of your data faster. And even when your data is the entire corpus of textual knowledge, there is a limit to the utility that provides.
It just means that software engineering, data analysis, and other computational tasks will be less of a bottleneck in the overall progress of the world (which, by the way, is great, and extremely valuable).
I don’t think it’s a stretch to say that we’ll have reached max LLM capability within a couple of years (what the author refers to as “super intelligence”). But it is a huge logical leap to say this:
> If we go through this feedback loop, I expect that the AI systems that come out the other side will be able to very quickly develop novel weapons and very easily overpower human civilization in a conflict.
First of all, the weapons necessary to overpower human civilization exist. They aren’t used as such due to geopolitics and humans’ basic survival instinct. Second, developing novel weapons requires physical tests, which are decidedly not accelerated by LLMs.
AI is a tool wielded by humans. If we’re wiped out by AI, it will be at the direction of a human. But we’ve made it pretty far with some pretty unstable humans in command of planet-killer weapons. I’m not overly concerned.
Benchmarks are by definition a curated set of things that can be measured and evaluated.
Things that can be measured and evaluated can be (expensively) trained on.
The progress we are observing comes from figuring out how to measure and evaluate particular aspects of intelligence, and then expensively training on feedback loops made of those exact measurements, and synthesized data representing further measurements (literally benchmaxing).
What is not shown in these graphs are the set of things that intelligence can do, which are difficult, expensive or impossible to measure and evaluate.
That set of things is much larger than the set of benchmarks represented in the graph.
Therefore the post is misleading in the sense that the "superintelligence" being targeted is "jagged" to use Karpathy's term, and we infact already have jagged superintelligence (calculators, compilers, Bayesian spam classifiers, etc.)
We are witnessing a profound explosion of much more jagged superintelligence, restricted to the exact domains that can be (expensively) RL'ed on.
- Ordinary term used more than once in the same comment
- “sudo” appears 3 times
If I install and setup and run "sudo" I expect it to work like sudo.
If you want your program to not work like sudo do not call it "sudo". This is what the OpenBSD folks did with doas.
POLA.
(It's fine if they want it to work differently than sudo; not fine that they called it "sudo".)
Great, then give it a new name.
> At least the command is still called sudo or run0
You're happy it's a knockoff?
> would have been a much more breaking change
All these "Rust rewrite" projects are creating breaking changes all over the place, whilst solving no real problems. They can shove them up their arses.
Langford has been at MSR a very long time and has done amazing research doing that time. He's bounced around a bit at MSR during his tenure and it's a shame to lose him. AI Frontiers wasn't around that long.
Much of this is Microsoft performing a very large and uncommunicated strategy update. All updates of MSR reorgs are just that "MSR is becoming more lean and focused" when it's obvious this isn't the case. This gets into the conversation that many Microsoft employees tend to have these days which is "Why does culture no longer matter at Microsoft and why did Satya turn into another heartless CEO". I won't speculate there, but the place isn't nearly as fun as it used to be.