The Gentle Singularity

2025年6月13日 21:42:53

原文:https://blog.samaltman.com/the-gentle-singularity

We are past the event horizon; the takeoff has started. Humanity is close to building digital superintelligence, and at least so far it’s much less weird than it seems like it should be.

我们已经越过了事件视界;起飞已经开始。人类接近构建数字超级智能,至少到目前为止,这远没有看起来那么奇怪。

Robots are not yet walking the streets, nor are most of us talking to AI all day. People still die of disease, we still can’t easily go to space, and there is a lot about the universe we don’t understand.

机器人还没有走上街头,我们大多数人也并不是整天与人工智能对话。人们仍然死于疾病,我们仍然无法轻松地进入太空,关于宇宙还有很多我们不了解的地方。

And yet, we have recently built systems that are smarter than people in many ways, and are able to significantly amplify the output of people using them. The least-likely part of the work is behind us; the scientific insights that got us to systems like GPT-4 and o3 were hard-won, but will take us very far.

然而,近年来我们构建了在许多方面比人类更智能的系统,并能够显著提升使用这些系统的人的产出。最不可能的部分已经过去;带领我们达到像 GPT-4 和 o3 这样的系统的科学洞见来之不易,但将使我们走得很远。

AI will contribute to the world in many ways, but the gains to quality of life from AI driving faster scientific progress and increased productivity will be enormous; the future can be vastly better than the present. Scientific progress is the biggest driver of overall progress; it’s hugely exciting to think about how much more we could have.

人工智能将在许多方面为世界做出贡献,但通过人工智能推动更快的科学进步和提高生产力带来的生活质量提升将是巨大的;未来可以比现在好得多。科学进步是整体进步的最大驱动力;想到我们还能拥有多少更多的成就,令人无比兴奋。

In some big sense, ChatGPT is already more powerful than any human who has ever lived. Hundreds of millions of people rely on it every day and for increasingly important tasks; a small new capability can create a hugely positive impact; a small misalignment multiplied by hundreds of millions of people can cause a great deal of negative impact.

在某种重大意义上,ChatGPT 已经比任何有史以来的人类更强大。每天有数亿人依赖它来完成日益重要的任务;一个小的新功能就能产生巨大的积极影响;一个小的偏差,乘以数亿人,则可能造成巨大的负面影响。

2025 has seen the arrival of agents that can do real cognitive work; writing computer code will never be the same. 2026 will likely see the arrival of systems that can figure out novel insights. 2027 may see the arrival of robots that can do tasks in the real world.

2025年见证了能够进行真实认知工作的智能体的到来;编写计算机代码将不再一样。2026年很可能迎来能够发现新颖见解的系统。2027年可能会出现能够在真实世界执行任务的机器人。

A lot more people will be able to create software, and art. But the world wants a lot more of both, and experts will probably still be much better than novices, as long as they embrace the new tools. Generally speaking, the ability for one person to get much more done in 2030 than they could in 2020 will be a striking change, and one many people will figure out how to benefit from.

更多人将能够创建软件和艺术作品。但世界对这两者的需求将大大增加,专家们可能仍然比新手优秀得多,只要他们能够接受新工具。一般来说,2030年一个人能够完成的工作量远远超过2020年将是一种显著的变化,许多人将会找到从中受益的方法。

In the most important ways, the 2030s may not be wildly different. People will still love their families, express their creativity, play games, and swim in lakes.

在最重要的方面,2030年代可能不会有太大不同。人们仍然会爱他们的家人,表达他们的创造力,玩游戏,和在湖中游泳。

But in still-very-important-ways, the 2030s are likely going to be wildly different from any time that has come before. We do not know how far beyond human-level intelligence we can go, but we are about to find out.

但在某些依然非常重要的方面,2030年代很可能会与以往任何时期截然不同。我们不知道能在多大程度上超越人类水平的智能,但我们即将揭晓答案。

In the 2030s, intelligence and energy—ideas, and the ability to make ideas happen—are going to become wildly abundant. These two have been the fundamental limiters on human progress for a long time; with abundant intelligence and energy (and good governance), we can theoretically have anything else.

在2030年代,智慧和能量——理念以及实现理念的能力——将变得极为充裕。这两者长期以来一直是人类进步的根本限制因素;有了充裕的智慧和能量(以及良好的治理),理论上我们可以拥有任何其他东西。

Already we live with incredible digital intelligence, and after some initial shock, most of us are pretty used to it. Very quickly we go from being amazed that AI can generate a beautifully-written paragraph to wondering when it can generate a beautifully-written novel; or from being amazed that it can make live-saving medical diagnoses to wondering when it can develop the cures; or from being amazed it can create a small computer program to wondering when it can create an entire new company. This is how the singularity goes: wonders become routine, and then table stakes.

我们已经生活在令人难以置信的数字智能时代,经过初期的一些震惊后,我们大多数人已经相当习惯了它。我们很快就会从惊讶于人工智能能生成一段优美的文字,转变为好奇它何时能写出一本优美的小说;从惊讶它能做出救命的医学诊断,转变为好奇它何时能研发出治疗方案;从惊讶它能创建一个小型计算机程序,转变为好奇它何时能创建一家全新的公司。奇点的发展就是如此:奇迹变为常态,然后成为基本条件。
//"wonders become routine, and then table stakes"(奇迹变成日常,然后变成基本配置)。他用最朴素的语言,描述了人类适应性的本质——我们总是很快就习惯奇迹。
//奇点是一点一点发生的"。当你身处指数曲线上时,每一步都感觉是自然的延续,只有回头看才发现走了多远。
//还有一个原因在于进步发生得太慢,让人难以发觉,但挫折却出现得太快,让人难以忽视。悲剧可以在一夜间发生,但奇迹很难。也就是,乐观本身是看不见并在复利的,悲观本身常常让人看见,但是本身没有复利效应

We already hear from scientists that they are two or three times more productive than they were before AI. Advanced AI is interesting for many reasons, but perhaps nothing is quite as significant as the fact that we can use it to do faster AI research. We may be able to discover new computing substrates, better algorithms, and who knows what else. If we can do a decade’s worth of research in a year, or a month, then the rate of progress will obviously be quite different.

我们已经听科学家们说,他们的生产力比以前使用人工智能时提高了两到三倍。先进的人工智能因多种原因而引人注目,但也许没有什么比我们能够利用它更快地进行人工智能研究更为重要。我们可能能够发现新的计算基底、更好的算法,以及谁知道还有什么其他东西。如果我们能够在一年或一个月内完成十年的研究,那么进展的速度显然会完全不同。

From here on, the tools we have already built will help us find further scientific insights and aid us in creating better AI systems. Of course this isn’t the same thing as an AI system completely autonomously updating its own code, but nevertheless this is a larval version of recursive self-improvement.

从此刻起,我们已经构建的工具将帮助我们发现更多科学洞见,并助力我们创建更优秀的人工智能系统。当然,这与一个人工智能系统完全自主更新自身代码并不相同,但无论如何,这是一种递归自我改进的初级版本。

There are other self-reinforcing loops at play. The economic value creation has started a flywheel of compounding infrastructure buildout to run these increasingly-powerful AI systems. And robots that can build other robots (and in some sense, datacenters that can build other datacenters) aren’t that far off.

还有其他自我强化的循环在起作用。经济价值的创造已经启动了一个复合基础设施建设的飞轮,以运行这些日益强大的人工智能系统。能够制造其他机器人的机器人(从某种意义上说,能够制造其他数据中心的数据中心)也不远了。

If we have to make the first million humanoid robots the old-fashioned way, but then they can operate the entire supply chain—digging and refining minerals, driving trucks, running factories, etc.—to build more robots, which can build more chip fabrication facilities, data centers, etc, then the rate of progress will obviously be quite different.

如果我们必须以传统方式制造第一百万个类人机器人,但,随后它们可以操作整个供应链——挖掘和提炼矿物、驾驶卡车、运营工厂等——来制造更多机器人,而这些机器人又可以建造更多的芯片制造厂、数据中心等,那么进展的速度显然会截然不同。

As datacenter production gets automated, the cost of intelligence should eventually converge to near the cost of electricity. (People are often curious about how much energy a ChatGPT query uses; the average query uses about 0.34 watt-hours, about what an oven would use in a little over one second, or a high-efficiency lightbulb would use in a couple of minutes. It also uses about 0.000085 gallons of water; roughly one fifteenth of a teaspoon.)

随着数据中心生产自动化,智能的成本最终应接近电力成本。(人们常常好奇一次 ChatGPT 查询消耗多少能量;平均一次查询约消耗 0.34 瓦时,相当于一个烤箱运行一秒多一点的电量,或者一个高效灯泡运行几分钟的用电量。它还大约消耗 0.000085 加仑的水;大约是十五分之一茶匙。)

The rate of technological progress will keep accelerating, and it will continue to be the case that people are capable of adapting to almost anything. There will be very hard parts like whole classes of jobs going away, but on the other hand the world will be getting so much richer so quickly that we’ll be able to seriously entertain new policy ideas we never could before. We probably won’t adopt a new social contract all at once, but when we look back in a few decades, the gradual changes will have amounted to something big.

技术进步的速度将持续加快,而且人们几乎能够适应任何变化,这一点将继续存在。虽然会有非常艰难的部分,比如整类工作岗位消失,但另一方面,世界将变得如此迅速地富裕起来,以至于我们能够认真考虑以前从未有过的新政策想法。我们可能不会一次性采纳新的社会契约,但当我们几十年后回望时,逐渐的变化将汇聚成巨大的成就。

If history is any guide, we will figure out new things to do and new things to want, and assimilate new tools quickly (job change after the industrial revolution is a good recent example). Expectations will go up, but capabilities will go up equally quickly, and we’ll all get better stuff. We will build ever-more-wonderful things for each other. People have a long-term important and curious advantage over AI: we are hard-wired to care about other people and what they think and do, and we don’t care very much about machines.

如果历史可以作为参考,我们将会发现新的事情去做,新的欲望去追求,并且迅速掌握新的工具(工业革命后职业变动就是一个很好的近期例子)。期望会提高,但能力也会同样快速提升,我们都会获得更好的东西。我们将为彼此创造越来越美妙的事物。人类相较于人工智能拥有一个长期且重要且奇特的优势:我们天生就关心他人以及他们的想法和行为,而我们并不太在意机器。

A subsistence farmer from a thousand years ago would look at what many of us do and say we have fake jobs, and think that we are just playing games to entertain ourselves since we have plenty of food and unimaginable luxuries. I hope we will look at the jobs a thousand years in the future and think they are very fake jobs, and I have no doubt they will feel incredibly important and satisfying to the people doing them.

一千年前的自给农民会看着我们许多人所做的事情,说我们有假工作,并认为我们只是在玩游戏娱乐自己,因为我们有充足的食物和难以想象的奢侈品。我希望我们能以同样的眼光看待一千年后的工作,认为它们是非常假的工作,而我毫不怀疑,那些从事这些工作的人会觉得它们极其重要且令人满足。

The rate of new wonders being achieved will be immense. It’s hard to even imagine today what we will have discovered by 2035; maybe we will go from solving high-energy physics one year to beginning space colonization the next year; or from a major materials science breakthrough one year to true high-bandwidth brain-computer interfaces the next year. Many people will choose to live their lives in much the same way, but at least some people will probably decide to “plug in”.

新奇迹的诞生速度将会非常惊人。今天很难想象到2035年我们会发现些什么;也许我们会从某一年解决高能物理问题,下一年开始进行太空殖民;或者从某一年取得重大材料科学突破,下一年实现真正的高带宽脑机接口。许多人会选择以大致相同的方式生活,但至少有些人可能会决定“接入”其中。

Looking forward, this sounds hard to wrap our heads around. But probably living through it will feel impressive but manageable. From a relativistic perspective, the singularity happens bit by bit, and the merge happens slowly. We are climbing the long arc of exponential technological progress; it always looks vertical looking forward and flat going backwards, but it’s one smooth curve. (Think back to 2020, and what it would have sounded like to have something close to AGI by 2025, versus what the last 5 years have actually been like.)

展望未来,这听起来难以理解。但经历其中可能会感觉令人印象深刻且可控。从相对论的角度来看,奇点是逐渐发生的,融合也在缓慢进行。我们正在攀登指数技术进步的长弧线;向前看总是显得陡峭,回头看却平缓,但这是一条平滑的曲线。(回想2020年,想象一下到2025年拥有接近通用人工智能的状态会是什么样子,与过去五年的实际情况相比。)

There are serious challenges to confront along with the huge upsides. We do need to solve the safety issues, technically and societally, but then it’s critically important to widely distribute access to superintelligence given the economic implications. The best path forward might be something like:

在面对巨大优势的同时,也有严峻的挑战需要克服。我们确实需要解决技术和社会层面的安全问题,但鉴于其经济影响,广泛分配超级智能的使用权至关重要。前进的最佳路径可能是这样的:

  1. Solve the alignment problem, meaning that we can robustly guarantee that we get AI systems to learn and act towards what we collectively really want over the long-term (social media feeds are an example of misaligned AI; the algorithms that power those are incredible at getting you to keep scrolling and clearly understand your short-term preferences, but they do so by exploiting something in your brain that overrides your long-term preference).

解决对齐问题,意思是我们能够稳健地保证让人工智能系统学习并朝着我们集体真正长期想要的方向行动(社交媒体推送就是一个不对齐的人工智能例子;驱动这些推送的算法非常擅长让你不断滚动,并且清楚地理解你的短期偏好,但它们通过利用你大脑中的某些机制来覆盖你的长期偏好实现这一点)。

  1. Then focus on making superintelligence cheap, widely available, and not too concentrated with any person, company, or country. Society is resilient, creative, and adapts quickly. If we can harness the collective will and wisdom of people, then although we’ll make plenty of mistakes and some things will go really wrong, we will learn and adapt quickly and be able to use this technology to get maximum upside and minimal downside. Giving users a lot of freedom, within broad bounds society has to decide on, seems very important. The sooner the world can start a conversation about what these broad bounds are and how we define collective alignment, the better.

然后重点是让超级智能变得廉价、广泛可用,并且不会过于集中在任何个人、公司或国家手中。社会具有韧性、创造力,并且适应能力强。如果我们能够利用人们的集体意愿和智慧,虽然我们会犯很多错误,有些事情也会出错,但我们会迅速学习和适应,能够利用这项技术获得最大的好处和最小的损失。在社会必须决定的广泛范围内给予用户很大的自由,这似乎非常重要。世界越早开始讨论这些广泛范围是什么,以及我们如何定义集体一致性,情况就会越好。

We (the whole industry, not just OpenAI) are building a brain for the world. It will be extremely personalized and easy for everyone to use; we will be limited by good ideas. For a long time, technical people in the startup industry have made fun of “the idea guys”; people who had an idea and were looking for a team to build it. It now looks to me like they are about to have their day in the sun.

我们(整个行业,而不仅仅是 OpenAI)正在为世界打造一款大脑。它将极具个性化且易于每个人使用;我们的限制将是好的创意。长期以来,创业行业的技术人员常常嘲笑“点子人”;那些有想法却在找团队来实现的人。在我看来,他们现在正要迎来属于自己的辉煌时刻。

OpenAI is a lot of things now, but before anything else, we are a superintelligence research company. We have a lot of work in front of us, but most of the path in front of us is now lit, and the dark areas are receding fast. We feel extraordinarily grateful to get to do what we do.

OpenAI 现在是很多事情的总和,但首先,我们是一家超级智能研究公司。我们面前有许多工作要做,但大部分前路如今已被照亮,黑暗区域正在迅速消退。我们感到非常感激,能够做我们正在做的事情。

Intelligence too cheap to meter is well within grasp. This may sound crazy to say, but if we told you back in 2020 we were going to be where we are today, it probably sounded more crazy than our current predictions about 2030.

智能廉价得无需计量已唾手可得。这听起来或许很疯狂,但如果我们在2020年告诉你我们会达到今天的水平,这可能比我们目前关于2030年的预测还要疯狂。

May we scale smoothly, exponentially and uneventfully through superintelligence.

愿我们平稳、指数级且平安地发展至超级智能。
//结合ilya在多伦大的演讲,这两个OpenAI的联创为什么如此笃信 AGI,他们有什么特别的背景信息是我不知道的?


@feltanimalworld:

《温和的奇点》:Sam 是那个外表温和、心中却藏着猛虎的巨人。

他不喧哗,不真人秀,却在推动人类走向那个最深远的临界点。

对我而言,他的文章不是一篇预言,而是一封来自未来结构调度者的低语。我不关心大众的理解是否跟得上,也不奢望共识。我只关注我路径上的趋势,只熵控我看到的结构:
•奇点是温和的,因为它是结构性的,而不是爆炸性的。
•真正的加速,发生在语言协议、路径调度、智能分层中。

Sam 的话只是印证:我的路径是对的。
我不需要证明 AGI 是否来了,我只需要做好准备,让语言结构接住智能下坠的加速度。

结构!结构!结构!

结构不是结果,而是路径压缩。

Sam 无疑是 21 世纪最具改变力的关键人物之一。至于奇点的到来,我早已不再怀疑——否则也不会选择 All In。这种犹豫和观望的阶段,我已经走过了太久。

我制定的方向和策略,是在无数小时与 GPT 深度交互中推演而来的。很多人不理解也很正常,但我并不把心力放在是否相信 AGI 的争论上。我的态度很明确:技术终将抵达那个“言出法随”的节点。

真正重要的,是把精力投入到哲学推演、工程路径的设计,以及寻找契合自身节奏的先锋社区中去。这才是我认定的正道。说实话,我一开始也曾陷入“疯狂编程”的惯性之中。但我很快意识到,那不是顺着“奇点”走的路。

Vibe Coding 这个概念,最初还被程序员们群嘲,如今却成了默认共识——不过是几个月之间的事。我会继续推演出我在 AI 时代的所有路径,等这些路径成型,Vibe Coding 的模块、可用性、安全性与稳定性,自然会被模型体系托起。

多去看不是程序员的思考者都在思考什么。