Risky Super Intelligence
The AI juggernaught, rapidly picking up speed on the down-slope, is flinging out new cybersecurity terms like shards of broken brake shoes.
This week I've been chasing the 40-tonner, catching up on the release of ISO/IEC 27090, Super Intelligence (a Trumpism) and other AI language developments.
A blog piece by the Institue for Science and Technology concerning a joint project with the Future of Life Institute caught my beady (A)eye.
RSI which once stood for a physical security threat (Repetitive Strain Injury - such as when an author spends way too many hours furiously typing ...) is now being reinterpreted as Recursive Self-Improvement (or perhaps Risky Super Intelligence?).
So far this week*, I've defined it thus:
and added a few more freshly-minted AI terms such as:
While pointedly ignoring the political and religious overtones, those last two typify the overtly pessimistic or fatalistic language of AI doomerism (a.k.a. doom-mongering, fear-mongering and doom-trolling), with dire predictions about frontier AI models (cutting-edge experimental systems and autonomous agents) advancing to the point that they can evade or disable the safeguards intended to contain them.
Artificial SuperIntelligence and RSI promote the notion that if autonomous AI systems can 'improve' themselves, that will lead to a rapid escalation of their capabilities, becoming too quick, too slick, too adaptive and too capable for conventional cybersecurity controls, including human-in-the-loop (whatever that means!). Personally I wouldn't consider that an 'improvement' from the human perspective - more of a degradation in fact, an information risk combining substantial probability with massive impact.
Quoting from the IST blog :
Despite broad agreement that RSI represents a significant security risk, there remains no consensus on what signals should trigger closer monitoring, coordinated action, or development pauses. To address this issue, the Institute for Security and Technology (IST) is launching the RSI Initiative. The project seeks to establish practical governance frameworks and response mechanisms for AI systems with RSI capabilities, working directly with developers, evaluators, and government safety bodies. This effort aims to identify actionable risk thresholds; develop monitoring and escalation pathways for systems that can modify their own behavior; and co-create risk management guidance that enables safe development as AI enters this new phase of self-modification and improvement.
I'm not convinced 'closer monitoring, coordinated action, or development pauses' will achieve the intended result, even if there is effective coordination between, self-restrain and regulation of, AI developers in various countries (which is as doubtful and fragile as a ceasefire in any war).
Consider the anti-nuclear-proliferation agreements in the 60's: how's that working out, six decades later?
It occurs to me that, aside from weak governance, fallible cybersecurity and dubious safety controls, ethics are a notably missing piece of the AI puzzle. Perhaps we should expect this as a natural consequence of having trained genAI/LLMs on vast quantities of original human content stolen from the Interwebs and books, along with political pressure to push back the AI frontiers (ostensibly for national defense purposes, but in reality for offensive global domination) and rampant commercialism (the AI buble). "Do as I say, not as I do" was never a strong imperative - a classic integrity failure.
Meanwhile, a tame if not well-trained AI system suggested the name ETHOS for the AI equivalent of the Campaign for Nuclear DIsarmament with the following 'mission and core pillars':
Equitable - ensuring training data, algorithmic outcomes, and access to benefits do not reinforce existing systemic inequalities or disenfranchise marginalised communities.
Technology: keeping the scope focused on practical, applied systems—from frontier large language models to automated decision-making in public infrastructure.
Human-centered - prioritising human agency, labour rights and mental well-being over unconstrained corporate velocity and autonomous deployment.
Oversight - advocating for enforceable auditing standards, algorithmic transparency and mandatory red-teaming before public deployment.
Society - anchoring development goals to public good, environmental sustainability, and democratic integrity rather than pure shareholder profit.
The irony of asking an AI system to advise on the means of AI repression is not lost on me.
*PS These working definitions may change tomorrow or the days after. Keep an eye on the Hyperglossary dotcom change page if you're interested, or task an AI agent to do that for you ...
Comments