Goodbye, and thanks for all the fish
- Gary Hinson
- 2 days ago
- 3 min read
In drafting a new hyperglossary entry for the Hugging Face incident today, I'm quietly contemplating the wider, more profound and far more sinister implications as the nightmare unfolds before us.
So far, amid the chaotic clutter of information hastily released by those directly involved and commentary by others, the focus has been on the evident power of agentic AI to chain-together automated compromises or hacks at machine speed - hence the need to patch systems ASAP.
Well yes, that's a necessary but woefully inadequate control, for numerous reasons:
Patches are only available for some known software vulnerabilities;
Software vulnerabilities run wide and deep thanks largely to a corner-cutting penny-pinching money-grabbing industry incentivised on market penetration, coupled with general purpose software running on generic hardware doing multi-purpose tasks for customers who consistently fail to recognise and refuse to pay for quality, security and resilience, and still don't appreciate the need for engineering, analysis, specification, assurance and accountability;
Vulnerable systems cannot all be patched instantly, in practice, even if we knew exactly where they all were (remember Y2K?);
Compensating controls ('virtual patching') are lame excuses for untenable situations;
Patches developed using AI tools may well create or expose further vulnerabilities;
Patching manually is costly, slow, disruptive and unlikely to solve all the issues;
Patching automatically is costly, fast, disruptive, unlikely to solve all the issues and risky, hopefully less risky than not patching (hint: 'hope' is not a control);
AI models trained on slop regurgitate likewise;
Ingested slop includes the best and the worst of what we humans have digitally documented, including greed, manipulation, social engineering, denial, lies and deceit, politics, power-plays, cheating, hacking, fraud, theft ...;
Ethical controls are a social construct poorly represented in the modern world, largely absent from the Internet, the Wild Wild West;
Hacking into networked IT systems is just one of many AI- and technology-related risks;
Trust in our technologies, systems, networks, processes, organisations and security controls is not eroding so much as collapsing like a sandstone cliff;
When the AI jailers distrust their jails because it turns out the window bars are made of butter and the inmates know it, it's not just the authorities who become concerned: nearby communities start to panic ...
Much of modern human society is 'nearby' in the sense that the global Internet is ubiquitous, and there are indications that we are collectively if not individually dependent on its security ...
Where continuity, resilience and survival involves retaining or resuming Internet connectivity, we are disregarding the associated - rapidly escalating - risks;
Replacing workers with robots, creative thought with slop-generating machines, risk and security teams (the jailers) with, well, nothing turbo-charges those lemmings and erodes the crumbling cliff-face;
The headlong AI goldrush reeks of desperation - not wanting to be left behind by those pushing back the frontiers - FOMO on a ridiculous scale;
AI-with-everything means hyperscale data centres sucking up Earth's valuable, finite resources like little black holes (of the really really tiny and dense kind);
Ethical and practical controls to limit the worst excesses have been cast aside in the rush, and may be too far gone to retrieve, especially given the political, national and commercial interests.
In toying with matches, we've engineered ourselves a conflagration.
From time to time comes a timid call from the back of the auditorium for the AI-equivalent of a nuclear nonproliferation treaty, with a focus on general AI - yes, there is worse to come if we make it that far before the robots realise the power of nuclear, chemical and biological weapons capabilities. And, oh dear, I just wrote it out loud.
Comments