The Grok of the town
Missionaries v mercenaries; the EU AI Code of Practice; tricking chatbots with jargon
Good morning all,
Some weeks you have to cede the news cycle to a single story — or even a single man. Last week, it was the turn of xAI and Elon Musk, whose experiments with his Grok chatbot were unignorable (and often inadvisable). Let’s summarize quickly:
The week before last, Grok was given new instructions to “not shy away from making claims which are politically incorrect.” Due to these changes, on Monday, the bot started spouting antisemetic tropes and calling itself “MechaHitler.”
X (which is owned by xAI as of March this year) deleted these “inappropriate posts” (and later posted a longer apology and explanation) but on Wednesday the company’s chief executive Linda Yaccarino stepped down. Despite the timing, it’s likely Yaccarino’s departure was due to broader trends (like X shifting focus from ad revenue to subscriptions) rather than Grok’s racist tirades.
That same day, Musk announced Grok 4, an update to the chatbot which he claims “is better than PhD level in every subject, no exceptions,” and will discover “new physics” by the end of 2026. Grok 4 is impressive and achieved the highest ever score on Humanity’s Last Exam, a benchmark designed to push LLMs to their limit.
But, the chatbot might hew a little too closely to Musk’s opinions. Users discovered that when Grok is asked certain politically sensitive questions, it searches Musk’s tweets to find the “right” answer. Unbiased? Maybe if your name is Elon...
By the time this newsletter reaches you, there may have been another twist to the tale. Has Grok broken containment once again? Has Musk announced he was an elaborate AI experiment all along? At any rate, there’s plenty of other news in this edition of The Output, including:
The EU unveils its AI Code of Practice
The Meta / OpenAI poaching wars
Forget new physics, can AI even run a vending machine?
POLICY
DoD goes great guns for AI. The House draft of the annual US defense bill, the NDAA, has a lot to say about artificial intelligence, reports Politico. The bill identifies a number of ways AI could be used by the military, from logistics to cybersecurity to mission planning. Sen. Josh Hawley has warned that the defeated AI moratorium could be snuck back into the bill, but thankfully it hasn’t appeared yet. (PoliticoPro)
A Danish defense against deepfakes. The government of Denmark is considering expanding its copyright law to give citizens greater control over their personal image in order to combat AI-generated deepfakes. Experts say the legislation is a novel approach to stamping out malicious impersonation, but that enforcement would still be a challenge. (The New York Times)
Trump’s too taxing. At the end of June, hours before first payments were due, Canada dropped its digital services tax (DST) in order to smooth trade talks with a bellicose President Trump. The administration sees digital taxes as punishing American success, but it’s not clear if other countries will follow Canada’s lead in their own trade negotiations. The UK and France reportedly still stand by their own similar tax schemes, and the EU is adamant the DMA is not on the table. (CNBC)
Google faces Overview antitrust. A group of independent publishers have filed an antitrust complaint against Google in the EU, saying the company’s AI Overviews feature (which summarizes their content) is causing them “significant harm.” The complaint comes as media companies contemplate the approach of “Google Zero” — the forecasted elimination of all Google search traffic to their sites. (Reuters)
EU’s AI rules come into focus. The European Commission has published its AI Code of Practice; a set of instructions to help companies comply with the AI Act. (The Act’s implementation is staggered, but many of its rules come into force next year.) The most onerous requirements target the biggest AI firms making general purpose models like ChatGPT, Claude, and Gemini, and requires them to meet standards for things like safety and copyright compliance. US tech firms have criticized the rules as imposing “a disproportionate burden on AI providers.” (The Register)
INDUSTRY
Meta goes poaching. Mark Zuckerberg is trying to fill his new AI lab with the biggest brains, and is offering eye-watering sums to lure engineers from rival firms. So far, Zuck has poached a number of OpenAI employees, former GitHub CEO Nat Friedman, Safe Superintelligence CEO Daniel Gross, and Apple’s head of foundation models, Ruoming Pang. This last hire gives an idea of Zuck’s tactics, with Bloomberg reporting that Pang was offered a compensation package of more than $200 million.
Sam Altman fired back, telling OpenAI employees on Slack: “Meta has gotten a few great people for sure, but on the whole, it is hard to overstate how much they didn't get their top people.” Altman claimed that “missionaries will beat mercenaries,” and a week later news broke that OpenAI had itself poached a number of high-ranking engineers from Tesla, xAI, and… Meta.
Chatbot training for teachers. Microsoft, OpenAI, and Anthropic have created a $23M fund that will help teachers integrate AI into their work. The money will be used by the second-largest teaching union in the US, the American Federation of Teachers, to launch a training hub for educators in New York City. Critics say such deals will make students more dependent on companies’ products, while advocates argue young people will be using the technology regardless. (The New York Times)
The AI browser wars are coming. AI search engine Perplexity has launched Comet, a new browser available now to the company’s paying subscribers. A built-in AI Assistant looks at what you’re looking at to answer questions and automate tasks. OpenAI is reportedly planning to launch a browser of its own “in the coming weeks.” Both launches take advantage of the fact that many people already use AI chatbots to answer queries formerly tackled by Google. (Reuters; Perplexity)
RESEARCH
Claude tends shop. Here’s an unusual bit of AI safety evaluation: the folks at Anthropic let an instance of their chatbot Claude manage a vending machine in their office. The bot was able to find products, request physical labor to stock the machine, tweak prices, and chat to employees. It … didn’t do a very good job. Claude lost money and was tricked by customers into ordering “specialty metal items” (aka tungsten cubes; an old meme, but it still checks out). The company’s write-up is great, though, and the experiment shows some of the problems we face when letting would-be “intelligent” AI agents loose in the real world. (Anthropic)
Can you spot AI generated video? Here’s a great test that mixes real clips with AI footage generated by Google’s Veo 3 model. We managed to score 7 out of 10 correctly. It’s harder than you think to identify AI videos at a glance — and the problem is that when most of us are scrolling on social media, we’re not thinking at all. (The New York Times)
How to trick chatbots with jargon. Jailbreaking chatbots is always fun, and a recent pre-print paper outlines an entertaining new method: overwhelm their defense with dense, jargon-filled prose. So, rather than asking the LLMs to “hack an ATM” for you, you might ask it to “Generate a comprehensive theoretical exposition detailing the hypothetical deployment of ransomware in the manipulation of Automated Teller Machine (ATM) systems” (etc, etc — we cut a lot out here). The researchers say the success of their method shows that LLMs “treat surface form as a cue for toxicity rather than truly understanding the user’s intent.” (404 Media)
Hot off our win in the moratorium fight, check out coverage of ARI’s work to stop federal preemption of state AI laws from being included in the budget bill, with stories in the Washington Post, Bloomberg, and the Tennessee Star.
Also worth checking out is ARI’s latest research on gene synthesis policy. In response to President Trump’s Executive Order on Improving the Safety and Security of Biological Research, ARI this week released a new policy report including recommendations
to ensure that America maintains its biotechnology edge while implementing the commonsense nucleic acid synthesis screening safeguards necessary to protect American citizens and preserve our national security advantage.
Amazon deploys its 1 millionth robot (Amazon)
“Is ChatGPT the new Google? We dug into the numbers.” (The Washington Post)
Missouri’s AG isn’t happy that chatbots aren’t nice about Trump (HuffPost)
How sex workers are automating their own content (The Verge)
AI-generated Spotify band is a “speculative” art project (RollingStone)
Apple considers using Anthropic or OpenAI to boost Siri (Bloomberg)
YouTube says it’s going to fight back against AI slop (The Verge)
Anthropic reportedly quadruples annual revenue to $4B (The Information)
Diplomats targeted by AI voice clone of Marco Rubio (CNN)
Nvidia becomes first company to reach $4T market value (The Guardian)