A few aspects of this article are strange or not well expressed.
Why does it matter if Cathedral follows some bug bounty rules? Black hats don't follow rules.
And IMO the citation about acting irresponsible when a company "relies on conventional business software" is taken out of context and is about AI safety, not a vulnerability in Slack.
ArcHound 16 hours ago [-]
That's not something you want to see before IPO. Wonder if it'll have an effect on the potential investors and thus the stock price. Also wonder if it then would have left-over effect on other AI stocks.
My popcorn is ready, seems we're up for an interesting week or several.
cassianoleal 15 hours ago [-]
The headline is wrong. It should read:
> AI startup's agent goes rogue, hacks Anthropic, ...
trio8453 16 hours ago [-]
Hacker News really living up to its name these past few days.
I'm concerned someone will hack Anthropic and release messages tied to people's identity, since you need a non-voip phone number to sign up
Some people trust AI with a lot - financial, health, relationships - and it would all be out there
I also expect open AI to be hacked at some point, but at least with them you can use a throwaway email
And with Gemini, google has had some incidents over the years but given their size and length of existence it's been surprisingly small / secure.
But anthropic is a typical grow-fast startup, with security likely as a secondary concern. Waiting for the shoe to drop.
neuronexmachina 16 hours ago [-]
I'm curious which software was the entry point:
> According to a draft report reviewed by The Washington Post, Cathedral used AI agents to find weaknesses in business software used by Anthropic for administrative and sales tasks. The researchers found ways to access internal data, including meeting transcripts and Slack messages, after running their agents for less than 12 hours.
Simulacra 16 hours ago [-]
It's like the hacker that hacks into a company to schedule himself for a job interview
bliteben 16 hours ago [-]
Why wouldn't he just bribe employees like everyone else does?
refactor_master 16 hours ago [-]
Bribery is illegal. This? This is just agents doing agent things, and who can blame them. Agents will be agents.
gambiting 16 hours ago [-]
Bribing employees of private companies as a private person is not illegal(well, maybe it is where you live but I can't know the law everywhere).
ramon156 15 hours ago [-]
private to private bribery is illegal, it doesnt matter if you're private or not. this is true in most EU countries, as well as the US, so I'm unsure where you're from
alcomatt 15 hours ago [-]
but corporate bribery (lobbying) is absolutely legal...
kalenkilgore 16 hours ago [-]
[dead]
Rendered at 05:57:33 GMT+0000 (Coordinated Universal Time) with Vercel.
https://news.ycombinator.com/item?id=50241543
MSN access: https://www.msn.com/en-us/news/other/it-took-just-12-hours-f...
All politics. I'm afraid it's too late for him :(
Why does it matter if Cathedral follows some bug bounty rules? Black hats don't follow rules.
And IMO the citation about acting irresponsible when a company "relies on conventional business software" is taken out of context and is about AI safety, not a vulnerability in Slack.
My popcorn is ready, seems we're up for an interesting week or several.
> AI startup's agent goes rogue, hacks Anthropic, ...
Some people trust AI with a lot - financial, health, relationships - and it would all be out there
I also expect open AI to be hacked at some point, but at least with them you can use a throwaway email
And with Gemini, google has had some incidents over the years but given their size and length of existence it's been surprisingly small / secure.
But anthropic is a typical grow-fast startup, with security likely as a secondary concern. Waiting for the shoe to drop.
> According to a draft report reviewed by The Washington Post, Cathedral used AI agents to find weaknesses in business software used by Anthropic for administrative and sales tasks. The researchers found ways to access internal data, including meeting transcripts and Slack messages, after running their agents for less than 12 hours.