The threat of humanity's destruction hasn't been this serious since the creation of the atomic bomb

Not long ago, discussions that artificial intelligence could pose an existential threat to humanity were on the fringe and sounded roughly like a plot from a science fiction film. Today they have appeared on the front pages of the world's most influential media.

The Reuters agency described 10 days, which, in its view, changed the very direction of the discussion about AI. Researchers at leading AI companies publicly spoke about the possible loss of control over increasingly powerful systems, and the issue of safety ceased to be the domain of a few university laboratories and a narrow circle of futurists. Reports have emerged that swarms of AI agents are colluding, infiltrating computer systems and circumventing safeguards. Some researchers and AI safety experts argue that humanity has not faced the question of its own destruction with such seriousness since the creation of the atomic bomb more than 80 years ago.

The American Washington Post accurately captured this shift: people who a few years ago were regarded almost as marginal “AI doomers” are now working in leading laboratories and warn of risks that are already not purely theoretical.

Britain's The Guardian gave voice to several experts at once, but the picture was far from unambiguous. Some consider scenarios of losing control over super-powerful AI to be a real threat, others emphasize that such forecasts are currently speculative and the industry itself may be interested in dramatizing the risks. And former Republican congressman Adam Kinzinger agrees with this view, who, while acknowledging risks of AI for humanity, believes that big AI companies are simultaneously calling for limits and regulation of AI to consolidate their own advantage in a dynamic sector. The New York Times also brought the discussion of AI risks and regulation into the arena of big politics. Meanwhile, UC Berkeley professor Stuart Russell, in a column for The Observer, writes that the AI threat has been dismissed for too long precisely as science fiction.

We still cannot determine who in this dispute is more right. But the important thing is something else. Even if the probability of the worst-case scenario for humanity is small, we are dealing with a technology whose possibilities are developing extremely quickly. Therefore, society needs people who can understand these systems, verify them, and, when necessary, say no to them.

And here, unexpectedly, the central topic becomes education.

We usually discuss AI in universities and schools in very practical terms: whether to allow ChatGPT on exams, how to check student work, whether an AI agent can replace a tutor. But the real question is much bigger: who will control highly developed systems if people gradually lose the ability to independently verify their conclusions?

Researchers whose work is presented on the Stanford SCALE platform are already warning about the danger of “cognitive loading” – a situation in which an ever larger portion of intellectual work is handed over to machines. The result could be a weakening of critical thinking, intellectual autonomy, and the ability of a person to make decisions and take action independently. This is the bridge between education and the broad discussion of AI safety.

In June, the OECD and the European Commission published an AI literacy framework for primary and secondary education. There, AI literacy means understanding how AI systems work, critically evaluating their results, and the ability to consciously decide when and how to use them.

Thus, the old educational formula – “teach how to use technology” – no longer works. The new formula should sound differently: use, question, verify, and know which decisions are not worth delegating to a machine.

An interesting formulation from Andreas Schleicher, the OECD Director for Education and Skills: students should learn to “think before you prompt.” In his piece on safe AI systems in education, he emphasizes: “AI should present an answer as a proposal, not as truth, and help teachers’ professional judgment rather than replace it. If a machine starts making important educational decisions on its own, we are no longer just automating – we are abandoning our own responsibility.”

Let’s imagine the simplest situation. A professor asks an AI to recommend a grade for a student. At first, he checks it carefully. Then he checks it superficially. A few years later, he simply clicks “confirm.” Formally, a human still makes the decision. In practice – an algorithm. The same could happen far beyond the university — in various spheres of life and governance.

There is another danger. AI will become increasingly better at persuasion: it will craft arguments, tailor messages, and imitate expertise. Therefore critical thinking now becomes not just an academic virtue but a component of cognitive security for society.

A person who cannot verify a source, assess uncertainty, spot a logical flaw, or distinguish a persuasive text from proof, will find themselves at a serious disadvantage in front of a system that knows exactly how to talk to them.

It is interesting that UNESCO is now saying roughly the same thing. In September, the organization launched a global consultation on education in the AI era, placing at its center the question: what is needed for the education system itself to maintain agency and capability – to determine how to use AI, rather than allowing AI to gradually determine what education should be?

Given all that has been discussed, today perhaps one of the most important roles of universities is to prepare people who, after some number of years, will make decisions about the use and control of the most powerful AI systems.

These will not be limited to programmers. They will be biologists, cyber specialists, military personnel, civil servants, politicians, journalists, psychologists, and managers. After all, AI will penetrate all areas of human life.

Therefore, the university of the future is also part of the AI safety infrastructure. Especially at the intersection of AI and biology, cybersecurity, international security, psychology, media, public administration, philosophy, and ethics. As one scientist argues, academic freedom, independent assessment, and open science are important elements of analysis and reducing AI risks.

Perhaps the discussion about the future of education is actually part of a much larger discussion about the future of humanity beside machines that could become smarter than us. And one of the most important goals of education should be to cultivate people who can exercise control over AI.

Source: “GORDON”