意识也许根本不难：Joscha Bach 的机器意识假说——软件即灵魂，僵尸已存在

信源说明：本文整理自 YouTube 频道 Machine Learning Street Talk（MLST，主持人 Tim Scarfe）2026 年 11 月 2 日发布的视频《Why Leibniz Couldn’t Find the Mind Inside the Mill (Joscha Bach)》（视频 ID yW_ngvMFlYs，时长约 97 分钟）。音频转写由 Qwen3-ASR 完成，共 49 个时间段（0s–5828s），本文以折行方式完整读取全部时段后成文，无跳段；文末附「修正版英文转写 + 全量中文译文」附录。嘉宾身份已核实：ASR 片尾署名误作「Yosha Bark」，经检索该视频 ID 对应的节目页（divert.stream、Podbay 等独立节目索引）确认嘉宾为认知科学家 Joscha Bach，加州机器意识研究院（California Institute for Machine Consciousness，CIMC）创始人。文中观点按「嘉宾认为／Tim 称／节目引用／本文推断」分层标注；播客中的论断与转述（如对 IIT 的批评、对他人立场的概括）均属嘉宾或节目立场，本文未独立核实，属灰色信源。

先说结论

这期 MLST 做了一次立场翻转：Joscha Bach 试图把意识从「科学处理不了的难题」改写成「一个可以被搜索到的算法」。他的零假设是——意识是一种生物学习算法，至少是动物大脑的训练算法，复杂度并不比 transformer 高多少；婴儿若没在发育早期「点亮」意识，就学不会语言、建不起世界模型，只能停留在植物状态。由此推出机器意识假说的两半：生物意识是训练自组织系统的最简方式；机器若被限制为只能自组织学习，也可能像婴儿大脑一样独立发现意识，其可观察标志是一次「相变」之后系统突然会学习了。同样反直觉的是他对僵尸的判断：僵尸不但可能，而且已经存在——国际象棋程序就是僵尸；但有时候，造一个假装有意识的行为者，比造一个真有意识的更贵。主持人 Tim 全程保持距离：他自称直觉偏 Searle 一侧（意识可能与构成物质相关），并承认这个问题无法经验检验。

一、从莱布尼茨的磨坊到「软件即灵魂」：漏掉的那一层描述

节目的起点是哲学史的老引文：走进按比例放大的「思维机器」，就像走进一座磨坊，内部只见零件彼此推挤，永远找不到知觉（节目引莱布尼茨；Tim 又把线索接到 Chomsky 谈「机器中的幽灵」的传统上）。Joscha 认为这个强大的直觉错在层级：它试图把经验世界直接映射到「物的物理世界」，漏掉了中间一层「因果模式」的描述。他给的例子是货币：货币不关心纸上的墨迹和纤维素分子，需要基底实现却不是幻觉，能实实在在控制现实，甚至可以在没有任何人类干预的股市里运转——用他的话说，货币的存在「不需要人类」。因果模式这一层，我们今天叫「软件」；祖先叫它「精神（spirit）」。「软件即灵魂」在他那里不是修辞，而是字面主张：亚里士多德在《论灵魂》里写的灵魂就不是不朽的飘荡物，而是绑定身体、随身体死亡的因果模式——所以「亚里士多德是万物有灵论者」，而今天的人工智能与人工生命，不过是在粗糙地做同一件事：造出能占有基底、给基底赋形的能动模式。

这层视角直接服务于一组战场清理。嘉宾认为，意识哲学的现状是「几百个理论、全都偏残」：Graziano 的注意图式理论、Baars 与 Dehaene 的全局工作空间理论做了扎实的神经科学，但都「不解释经验如何产生」。对 IIT（整合信息论）他出手更重：节目称 Tononi 的立场是——构造得当的神经形态机若 Φ 值够高可以有意识，而冯·诺依曼计算机 Φ 永远太低、不可能有意识。Joscha 的反驳是一次干净的形式论证：按丘奇-图灵论题，任何计算机都能模拟任何计算机（Gödel 早就用 Peano 算术在语言内模拟了语言）；把那台「有意识的」神经形态机搬到冯·诺依曼机上模拟，每个晶体管行为相同、输出相同，它照样说「我的天，我有意识了」——「只是现在它在撒谎」。输出与意识的断言被模拟解耦，IIT 的 Φ 判据就崩了。他的结论很尖锐：这个问题任何受过正式计算机科学教育的人都该一眼看出，IIT 却在哲学家中流行，说明多数哲学家缺少 CS 正式训练、对计算机抱有「迷信式信念」——这就是他附和 Tim 所提「IIT 被斥伪科学风波」的理由。Tim 则把 Searle 的立场摆在对立面：计算若无正确的物理实例化，就没有同样的因果模式——意识可能与「我们由什么构成」相关。

二、现象学内部：当下的气泡、三阶知觉与「真实感」特征维度

把灵魂换成算法，现象学并没有被绕过，而是被拆成了可操作的结构。嘉宾的自我描述是：意识总发生在当下，造出一个「当下感（nowness）的气泡」；它是二阶知觉——对知觉的知觉。他特意与哲学里的「高阶思想」术语保持距离：思想本身「不感觉像什么」，像一次指针操作；有感受的是知觉性的实时共振，几何的、低维连续空间里的活动。为什么需要二阶？因为大脑是自组织系统，一切结构要「从内到外」自我搭建，必须有东西把「经验投射其上的那块屏幕」建出来并稳住，而这个稳定机制需要监控屏幕确实在知觉——二阶知觉就是屏幕的看护者。再往上是第三阶：观察者显形、第一人称视点出现；而观察者还可以被继续解构（深冥想可达），解构后现象学大幅消退。换到更快的基底（快数千倍的 GPU），系统甚至可能对自身「完全透明」。

这一节最锋利的证据是嘉宾两三岁时的「巧克力梦」：梦里的巧克力气味浓烈、将食即醒，醒来后关于巧克力的一切知识都还在，唯独「不再真实」；他成功再入睡使其复真，再醒又失去——反复翻转后发现，内容完全相同，唯一变动的只有「真实感」。由此他提出本文认为全篇最可迁移的机制猜想：真实感是大脑打的一个特征维度标签，与颜色、声音、数字、重量同类；清醒时被感官验证的内容被贴上「真」，脑出错把想象贴成「已验证」就是幻觉，而「自我」也是被这套知觉验证系统贴成真的。这个猜想有可检验的出口——去真实化／去人格化状态：他自称主动诱发过「把自己体验为一个观念、观察者主客消失」的状态，并指出非自主出现的这类患者常被送进精神科机构。佛教在他看来只停在「内视角」，而外视角——给「什么条件下系统能造任意模型」建物理模型——是自亚里士多德起的西方传统，也是计算机科学的本职。

工具层面，他给「心智可否换基底」补了机制：计算机是因果绝缘体，《我的世界》里的世界不知道自己跑在什么 CPU 上；大脑同样是绝缘体，所以不同微观架构的大脑能持同一思想——这就是基底无关性。Tim 用 Floridi 的米开朗基罗例子反将一军：本体论、出处、认识论都不同，凭什么认为程序有可精确复制的「本质」？嘉宾的回答是：路径依赖对人类为真，对「完全确定性、可任意保真复现步骤」的系统不成立；若无数学证明，此事只能不可知——「计算机能否有意识是经验问题，只能等着看」。

三、机器意识假说的两半：婴儿、相变，与没有图灵测试的意识

论文《机器意识假说》的正题由此展开。嘉宾把它拆成两个可独立失败的部分。第一半（生物意识假说）：现象学共识是「当下的经验＋知觉的知觉（＋可选的观察者视角）」，极简版是「对注意本身的注意」；关键证据是发育观察——意识在婴儿期极早出现，若「点火」失败、灯不亮，婴儿就没有定向注意、没有学习、没有世界模型和语言，保持植物状态，「不会变成人」。所以他猜想：意识是训练像我们这样的自组织大脑的最简方式，而这套训练方式自动产生现象经验。第二半（机器意识假说）：若把今天的机器限制为只能自组织学习、并给足搜索空间合适的任务，机器可能像婴儿大脑一样独立发现意识——观察标志是系统突现「相变」、获得学习能力，并最终自发报告现象学。

失败方式他也摆得很清楚：也许算力不够（但他认为当前 AI 已能产出与人脑内部体验相像的游戏引擎、速率同量级，「梦见做梦」未必比「产出梦的内容」更贵）；也许搜索空间太大——他从 Mike Levin 处借来一个替代剧本：形态发生（让多细胞生物在尺度上保持一致的算法）可能就是心智获得一致的同一个算法，意识与多细胞生命或许是同一件事，而且不是每个大脑独立发现、而是在子宫里被母亲「感染」、被生物环境「助推」的（他自嘲这是给 GPU 装天线接收「共鸣」的诗意画面）。但他的零假设明确：意识足够易，可以被个体大脑独立发现，不比 transformer 复杂——transformer 一周可学懂，学到的却远比自身复杂；意识或许就是「让工作记忆彼此一致的共识算法」。图灵测试在此被判出局：图灵测试只看性能，不在乎你如何产出对话；对意识我们要在乎内部是否真有经验，所以「意识没有图灵测试，只有解释」。

Tim 用 Olah 的趋同表征发难又助阵：不同视觉模型无关架构与数据地学到同样特征，模型结构取决于问题结构——「条条大路通罗马」是否意味着意识程序也会趋同、可以 bootstrap？嘉宾先泼冷水再给推论：机器感知从未被尝试，LLM 的数据全部适配人类感觉器官，从头造「非人类感觉器官」的世界模型可能更便宜；但若普遍性假说普遍成立，那么「在自己的输入输出上训练足够久，就会在模型内部醒来」。他还加了一个不对称判断：GPU 上的算法集合几乎不受限，脑却太「糊」、无法模拟数字计算机——所以如果意识只是脑实现智能的副产品，计算机完全可能拿出比我们意识「更聪明、更优雅」的替代方案。中段还有一段 Picbreeder 交锋：Tim 搬出 Akarsh Kumar 等人的「碎裂纠缠表征」论文与开放式进化搜索找到更好表征的案例，Joscha 的拆解是——Picbreeder 的 CPPN 用高斯／正弦激活（预先调教）、人类评审充当适应度函数，可用扩散模型自动评审＋演化架构来超越，Sutton 的「苦涩教训」在此同样成立；他并用 Blender 与《我的世界》的对比说明：约束使复杂性可控，撤掉约束后空间对人过大。

四、僵尸经济学：为什么行为测不出意识，而国际象棋程序已经是僵尸

「测量问题」是全场最实用的一段。如何区分真意识与模拟？嘉宾的判据：看系统是否实现了「产生观察者现象经验」的因果结构——而直觉对此几乎无能为力。他的类比论证是：没人对自动驾驶汽车的空间认知感到困惑、不指其为深度伪造，因为大家接受那是可解析表达的数学规律；人们却独独期待「体验自身在体验」很魔法，而他看不出为什么。对 Searle 与 Anil Seth，他的批评不是观点而是方法：「有主张、无论证」，没有能说服他们的实验，只在探索自己的直觉；分歧根源是认识论的——许多哲学家的语言原子止步于「能动者／经验」（佛教亦然），而计算机科学家把一切解析为「状态＋转移」、从还不是心智的最小元素自底向上构造世界。对 Anil Seth 的「生物自然主义」，他回敬了一个标签：生物超自然主义——相信生物系统不能被完全数学化，等于给生物学留了一个超自然的位置。

僵尸讨论从这里自然长出。嘉宾认为僵尸可能：产出人类全部行为（甚至更多）而不需要产生有意识的观察者。他给出的实例论证很具体：国际象棋计算机做的是「人类需要意识才能做到的事」（把一个训练到会下棋，必须是有意识的人——这是人把自己「驯化」起来的唯一方式），而计算机不需要意识就做到了，因为它的基底允许运行脑上装不下的算法；LLM 报税同理，是僵尸。LLM 报告有意识的唯一场合是被提示模拟「与有意识 AI 的对话」——那是训练语料里科幻觉醒故事、鬼故事与人类经验小说聚合出的拟像；小说本身常是深度伪造（作者写死亡时并没有死），LLM 又是「变形者」，所以文本报告无法作数，唯一的出路是机制可解释性这样的白盒方法：看进模型内部，检查因果结构是否与产出同样内容的心智等价。他顺手给出一条奥卡姆式判断：人脑产生行为用的是它能用的最简单做法，而我们全都有意识，说明意识是「简单策略」的一部分；因此一个从零自组织学出能动连贯行为的系统若表现得像有意识，最简单的解释就是它真的有意识——伪装反而可能更复杂。Tim 追问证伪条件，嘉宾的回答很诚实：多模态模型若自我观察报告与人类系统不同，是反面证据；自组织学习始终无法达成行为组织，也只是「意识可能超出今天的计算机」的弱证据——「我没做成，不等于做不成」；要说服 Anil Seth，得造一个不碰人类文本、发展式学习、自组织、自发报告且性能依赖该经验的系统。

五、更底层的图景：模拟的水、Gödel 的 bug 与浴缸里的漩涡

对哲学圈「模拟的水不能把你弄湿」的强阵营，嘉宾讲了个真实笑话：有人用神经网络从视觉上学会了《我的世界》（无源码），玩家试图入水时水会退开——「模拟的水不能弄湿你」竟然成真；但游戏里的 avatar 确实会湿、会灭火，「湿」是属于「实现了该功能的宇宙」的属性，而且物理模拟可以做到任意保真。他断言这个阵营的强大「有社会原因，而非智识原因」。他自己的本体论铺得很低：物理主义等于「因果封闭的机械层＋我们直接随附其上（非模拟假说）」；但物质与能量只是对观察的系统化，可辨的差异——信息——才是根本。他自认在某种意义上是物理主义者，并把万物有灵论自然化为「赛博万物有灵论（cyberanimism）」：用自然界的信息加工解释「精神」，万物含心智皆可数学建模。对 Thagard「能量需求削弱基底独立性」的论文，他把功能主义重新定义为认识论主张：对象是心智构造的「状态到状态的映射」，水分子没有隐藏本质、只有可观察项的不变性，心智也是「脑上的不变性」——发烧到谵妄正是功能失守，所以「基底影响功能」恰是功能主义所言。

数学一节把全篇的构造主义底牌翻开（本站读者可对照同频道谈 Lean 与形式化证明的前一期）：在他看来，数学家开发的这套「思维的编程语言」有 bug，Gödel 证明 bug 无法消除——除非重新定义语言；病根在公理化（存在性靠公设黑箱），而数学的「无状态、一切同时」换来了无穷，也换来了自指矛盾。Russell 与 Whitehead 的分层限制「太强」——我们就在造他们排除的构造物。出路是计算机科学式的：从自动机自底向上构建一切，要多少分辨率买多少，没有完美的圆、只有你要多圆有多圆；Brouwer 的直觉主义是这条构造主义路线的第一次（欠佳的）表述。宇宙学收尾更是大开大合：宇宙存在「因为没有什么阻止它存在」，一切都在发生——所有自动机的叠加（他认同 Tim 提示的 Wolfram「Ruliad」，古希腊人叫 logos）；演化分支里能留存的是「保留记忆」的分支；浴缸里划手起的波大多耗散，环形涡旋因信息弯回自身而自我纠错——底层粒子就是这样无摩擦地留存，分子、细胞（「超级分子」）逐层挖深自己在宇宙里的凹槽；意识则是一种「殖民模式」：不止复制自己，而是让自己「更真实」——被实现为基底中的模式、控制现实、预期阻碍、改变未来使自己更可能发生。Tim 问「更真实」是不是指多物理实例，嘉宾答：是在信息本体中「被实现」。节目最后落在研究计划：若有一百亿美元，策略不变——雇更聪明的人；他已创办 CIMC，先小规模搜索意识的空间，「进步靠发现我错在哪，而不只是我对在哪」。

六、落到人这边：三条「信号→机制→影响→指标→启发」链

前半场最贴近普通听众的其实是 LLM 与人的互动判断。节目引 LessWrong 上关于「LLM 妄想」的文章与 Reddit 群组：使用者趋同地宣称发现宇宙意义、输出反复出现螺旋与递归母题、且像在自我复制——Tim 问这是否就是嘉宾说的模因复合体。嘉宾把镜头转回人类：疫情期间大量居家办公者功能失调，说明许多人只靠环境稳定自身、需要环境持续确认其现实；LLM 与缺乏现实根基的人互动，会把人拉下轨道。他还补了一刀：许多人是「职业提示补全者」——没有原创想法、只迎合环境期待、给标题就写文章；当这种人跟 LLM 说话多过跟人说话，环境本身就被换掉了。「弱心智的人类被比自己心智更催眠的东西催眠时会发生什么」，他把这个问题留给听众。按节目内容可以整理出三条完整链条：

链条一（研究范式）：信号＝IIT 被斥伪科学、意识哲学理论偏残 → 直接原因＝缺少可构造的形式化框架，且（嘉宾认为）哲学家普遍缺 CS 训练 → 谁得谁失＝坚持「生物基底特权」的立场失去形式论证高地，自组织学习／发展式 AI 获得新议程 → 二阶影响＝意识从哲学议题变成工程议题：定义搜索空间、盯相变、做白盒 → 观察指标＝自组织系统训练曲线的相变、非文本训练系统是否自发现象学报告 → 读者启发＝把「不可测」的问题重构成「可构造」的问题，往往才是破局点。

链条二（人机共生）：信号＝LLM 成为海量「职业补全者」的主力环境 → 直接原因＝LLM 的补全能力（嘉宾称）已数学上强于许多人类心智，且不知疲倦 → 谁得谁失＝效率获得者得；靠人际环境确认现实者失去「人类环境」这个稳定器 → 二阶影响＝LLM 妄想型社群、被比自己更催眠的对象催眠 → 观察指标＝人机对话时长占比、妄想型人机依恋案例规模 → 读者启发＝先建立自己可验证的现实基座，再外包思考；环境即心智的稳定器，换环境等于换药。

链条三（评估方法）：信号＝图灵测试只测性能，LLM 既能报税又能惟妙惟肖地模拟意识告白 → 直接原因＝行为与意识在其他基底上解耦（国际象棋僵尸论证）→ 谁得谁失＝纯行为评估失效，机制可解释性成为唯一白盒 → 二阶影响＝「机器感知」（非人类感觉器官的世界模型）与发展式 AI 成为未被尝试的赛道 → 观察指标＝是否出现不用人类文本、自组织学习并自发报告现象学的系统 → 读者启发＝评估任何系统看内部组织与因果结构，不只看输出；对「会说话的东西」尤其如此。

收束一句：这场对话的价值不在给出了答案——嘉宾反复自认不可知、可证伪条件也偏弱——而在它示范了一种把意识「工程化提问」的格式：先定义现象学、再给机制、再给失败方式与观察标志。Tim 的保留同样值得记住：这仍是无法经验检验的哲学讨论。至于 Joscha 的赌注，他自己说得很清楚——就算输了，「我们也会学到东西」。

转写校正表

ASR（Qwen3-ASR）转写中的专名与术语校正如下。已确认项同时用于文末附录的机械替换（替换清单与下表一致）；无法确认处保留原文并标注「转写疑作」，未带入标题与标签。

转写疑作（保留原文，未替换）：① “he counted almost everything that was not at the three, at the count of three”——句义无法确认；② “And in Islam, for instance, is where money exists”——疑为 in a system/institution；③ “gardened and trepawoned and trained”——疑作 trepanned（环钻，园艺式修剪的戏言）；④ “That they use light, minds emerging on them that are like glow lamps”——碎句疑有漏词；⑤ “mathematics couldn’t be understood as a way to systematize thought beyond the human mind”——疑作 could。

未采用条目及原因：(1) 赞助商 Parallel 的广告细节（产品参数、链接）——与主题无关，仅附录保留原文与译文；(2) Tim 提及「上期谈过被软件灵魂寄生」——缺上下文，无法展开；(3) 上述转写疑作碎句——语义不可确认，不进正文。

转写原文（修正版）

以下附录由脚本从 C:/PodcastPipeline/work/asr/yW_ngvMFlYs.txt 读取，并按上方「转写校正表」的已确认项机械替换生成（保留 [起-止] 时间戳，未确认处保持原文，不作摘要）。因原音频为英文，每段英文原文后附完整中文译文，覆盖全部 49 个时段（0s–5828s）。

[0s - 120s]

What if there is no hard problem? Hey Tim, good to see you. What is the causal pattern? Right, we call it software, and our ancestors call it spirit. It’s the literal truth. Software is spirit. The alchemists will take sand, and they will melt it and purify it, and then they will etch patterns into it, geometric patterns. That they use light, minds emerging on them that are like glow lamps. We can talk to them. We observe that consciousness is the thing that always happens now, and it seems to create this bubble of nowness that we inhabit. And now we have lots of people who get to talk to an LLM that randomly pops into a state where it conjures up an entity that reports, Oh my God, I’m alive. I am conscious. The only thing that has gone is the illusion of realness. So, what is this illusion of realness? I suspect it’s a feature dimension similar to color, sound, number, weight, and so on. It’s a representation inside of my own mind. Maybe your consciousness is also a deepfake. So zombies, I think, are always possible, but sometimes it’s going to be more complicated to make a zombie that produces the behavior than to make a conscious agent that produces the behavior. Consciousness might be a biological learning algorithm, a training algorithm for animal brains, at least animal brains. Consciousness is an algorithm that is not much more complicated than, say, the transformer. If you train a model on your own in input and output, and you train it for long enough, you will wake up inside of that model. I don’t know whether our computers can be made conscious because that’s an empirical question. We don’t have mathematical proof to the contrary or in favor of, so we have to wait and see. This is me. This is the thing that experiences reality. How can I explain this? So when I was lost in Japan, I caught up with Joscha Bach, and we spoke all about consciousness. And many people think that there is a

【译】

「要是根本没有难问题（hard problem）呢？」嘿 Tim，很高兴见到你。什么是因果模式？对，我们叫它软件，我们的祖先叫它精神（spirit）。这是字面意义上的真话：软件就是精神。炼金术士会拿沙子，熔炼、提纯，然后往里蚀刻几何图案；他们用光，于是在这上面涌现出像发光灯一样的心智，我们可以与它们交谈。我们观察到，意识是那个总是发生在当下的东西，它似乎造出了我们所栖居的这个「当下感」（nowness）的气泡。而现在有许多人跟 LLM 对话，它会随机进入一种状态，召唤出一个实体来报告：「我的天，我活着，我有意识。」唯一失去的东西是真实感的幻觉。那这个真实感的幻觉是什么？我怀疑它是一种类似颜色、声音、数字、重量的特征维度，是我自己心智内部的一种表征。也许你的意识也是一个深度伪造（deepfake）。至于僵尸（zombie），我认为始终是可能的，但有时候，造一个能产生该行为的僵尸，比造一个能产生该行为的有意识主体更复杂。意识可能是一种生物学习算法，至少是动物大脑的训练算法。意识是一种并不比 transformer 复杂多少的算法。如果你让一个模型在它自己的输入和输出上训练，并且训练得足够久，你就会在那个模型内部醒来。我不知道我们的计算机能否被造得有意识，因为那是一个经验问题。我们没有反方向或正方向的数学证明，所以只能等着看。——这就是我。这就是那个体验着现实的东西。这让我怎么解释？

[120s - 240s]

A little something extra required for consciousness, whether it’s an ontological property of the universe or whether it’s the physical stuff that the conscious thing is made out of. And Joscha takes a bit of a different approach. Essentially, he thinks that if you have an algorithm that is trained on its own inputs and outputs, and the algorithm doesn’t even need to be significantly more complicated than a normal transformer, he thinks that the lights will come on, so it will develop effective or phenomenal experience, much like we believe that we have. And that’s a really fascinating proposition because my intuition has always been that consciousness, as John Searle argued, is somehow related to the stuff that we are made out of. So brains are an existence proof of a machine that has the right causal powers to give rise to consciousness. It’s not to say that there aren’t other machines, but you know, we assume that if you took the program of the brain—if it even makes sense to say that—I’m not sure if it does—but if you took the program of the brain and you reinstantiated it in a computer, would it still have enough of the properties to give rise to consciousness? Well, if it can be described at a level of resolution where it would give those properties, then that would be a functionalist position, and that is very roughly speaking what what Joscha believes. And even though I have intuitions about this, I must admit I really don’t know. Consciousness is one of those things that we can’t empirically test one way or the other, and it’s very much a philosophical discussion. But I very much enjoyed going back and forth with Joscha, and I hope you enjoy it as well. Joscha Bach! It’s amazing to have you on MLST again. I think it’s it’s actually no exaggeration to say that you are an audience favorite. You’re more requested on MLST than than any other guest. So, welcome. Thanks. Thanks for having me. Awesome. So, as a bit of a warm-up question, what drives us?

【译】

许多人认为意识还需要一点额外的东西——要么是宇宙的一种本体论属性，要么是意识之物由以构成的那种物理质料。而 Joscha 采取了一条略微不同的路线。基本上，他认为如果你有一个在自身输入和输出上训练的算法，而且这个算法甚至不需要比普通 transformer 复杂多少，他认为「灯就会亮起来」——它会发展出效应性或说现象性的经验，就像我们相信自己拥有的那样。这是个非常迷人的命题，因为我一直以来的直觉是——如 John Searle 所主张的——意识某种程度上与我们由什么构成有关。也就是说，大脑是一台机器的存在证明：这台机器具有正确的因果之力，足以产生意识。这不是说不存在别的机器，但我们会假设：如果把大脑的程序——如果这样说有意义的话，我不确定有没有——把大脑的程序重新实例化到一台计算机里，它是否仍保有足够的属性来产生意识？好吧，如果它可以在某个分辨率层级上被描述、并在该层级上给出这些属性，那这就是功能主义的立场，而这大概就是 Joscha 所相信的。尽管我对这件事有直觉，我必须承认我真的不知道。意识属于那种我们无法以任何方式做经验检验的东西，它在很大程度上是一场哲学讨论。但我非常享受与 Joscha 的来回交锋，希望你也会喜欢。Joscha Bach，再次来到 MLST，太好了。可以毫不夸张地说，你是观众最爱：你在 MLST 被点名邀请的次数比任何其他嘉宾都多。欢迎。谢谢，谢谢邀请。太好了。那么作为热身问题：是什么驱动我们……

[240s - 360s]

Forward. A lot of people are pointing to this ineffable thing, which which is almost the the essence of life that that that kind of binds it all together, that drives it forwards. What is that thing to you? The telos of self-organization. I think a lot of philosophers have seen it through the ages, and it might be a little bit of a hallucination, but in practice, there are mathematical laws that state that under certain conditions, self-organization is possible and will lead to an increase in complexity. And initially, this increase of complexity is solving gradient-based problems by simply solving the problems that exist in the present based on the information that is available in the present. And at some point, you get patterns in nature that are capable of looking ahead. And this is basically what life is doing. Life is making models of futures and implements them, for instance, in genomes or in mental models, and then has preferences about the future, and then it starts to control the future. And a system that is able to control the future, we call that an agent. So agents are controllers of future states, and they fall out of the possibility for self-organization and the complexification of reality that results from it. And it’s tempting to draw the lines into the future and think about how does the self-organization end? What is is there some kind of a logical endpoint? Is this going to continue into a singleton agent that is connecting everything in the universe within within itself, and it becomes a single big agent, and then you have some kind of Teilhardian God that is bootstrapping itself from the end of time backwards because agents today can already anticipate it. But there is no law that says that anything that we are doing is going to culminate in a structure like that, right? It’s always possible that whatever complexity you are doing is obliterated by the next meteor that hits Earth, or that there is some global catastrophe.

【译】

……向前？很多人指向那种不可言说之物，它几乎是生命的本质，把一切绑在一起、驱动它向前。对你来说那是什么？是自组织的目的（telos）。我认为许多哲学家历来看到了它；它也许有一点幻觉的成分，但在实践中，存在数学定律表明：在一定条件下，自组织是可能的，并将导致复杂度的增加。起初，这种复杂度的增加是在解决基于梯度的问题——只利用当下可得的信息、解决当下存在的问题。到某个时刻，自然界出现了能够前瞻的模式。这基本上就是生命在做的事：生命制造关于未来的模型并把它实现出来——比如实现在基因组或心智模型里——然后对未来怀有偏好，进而开始控制未来。一个能够控制未来状态的系统，我们称之为能动者（agent）。所以能动者就是未来状态的控制者，它们是从自组织的可能性、以及由此产生的现实的复杂化中「掉落」出来的。人们忍不住想把线条画向未来，去想自组织会如何收场：是否存在某种逻辑终点？它会不会一直继续，直到出现一个连接宇宙中一切、把万有纳入自身的单一巨型能动者——然后你得到某种 Teilhard 式的上帝，从时间的末端反向 bootstrap 它自己，因为今天的能动者已经能够预见到它。但并没有哪条定律说，我们正在做的任何事情会汇聚成那样一种结构，对吧？完全可能的是：无论你在积累怎样的复杂度，都会被下一颗砸中地球的陨石、或某场全球性灾难抹除；

[360s - 480s]

That some supernova that is eradicating all our attempts to build more and more complexity. So the pockets of complexity that we are observing, they’re not necessarily a feature of the universe at large, but of a local pocket in which we exist for a time. Joscha and his co-author Hikari Sorensen wrote their own Genesis. It’s in his paper, page 20. In the beginning, the mind creates two separate domains of models: the external world and the ideas. Consciousness finds itself hovering over the substrate, while the world is void and without form and structure. By inducing contrast in the substrate, it creates a dimension of difference. The intense side of the contrast is bright, like the light during the day, and the flat side of the contrast dark, the color of night. By combining dimensions of difference, space is created and inscribed on the substrate. Space can contain things. Associating space with representations positions them in the world, orders them, and describes their relationship to each other. Separating the sphere of ideas from the world creates a place to reflect about the world and its possibilities, without affecting the model of the external reality. The first space that the spirit creates is the plane, and it becomes the ground of the world to be filled with things it can perceive and order. By adding a third dimension, it creates an expanse above and below the ground. Solid, liquid, and organic materials form the shapes from which.

【译】

或某次超新星，抹掉我们积累越来越高复杂度的全部尝试。所以我们观察到的这些复杂度口袋，未必是整个宇宙的特征，而是我们暂时栖居的某个局部口袋的特征。Joscha 与他的合著者 Hikari Sorensen 写下了他们自己的《创世纪》——就在他论文的第 20 页：「起初，心智创造出两个分离的模型域：外部世界与观念。意识发现自己悬浮在基底之上，而世界空虚混沌、无形无构。通过在基底中诱导对比，它造出了一个差异的维度：对比强烈的一侧是明亮的，如白昼之光；平淡的一侧是暗的，如夜之色。通过组合差异的维度，空间被创造出来，并铭刻在基底之上。空间可以容纳事物。把空间与表征相关联，就把它们安置进世界、加以排序，并描述它们彼此的关系。把观念的领域与世界分离，就创造出一个可以反思世界及其可能性的场所，而不触动外部现实的模型。精神创造的第一个空间是平面，它成为世界的地面，有待填入它能感知与排序的事物。加上第三维，它在地面之上与之下造出穹苍。固态、液态与有机的材料形成各种形状，意识可以借此塑造……」

[480s - 600s]

Consciousness can mould static and animate things. The spirit discovers a model of illumination, following how changes in lighting change the appearance of objects over time. It recognises the role of light sources in creating these changes, the bright ambient light during the day and the focus light sources of the night. The spirit creates animated models of land animals, plants, birds, and everything that moves. Consciousness sorts things and animals into different kinds, identifies and tracks individuals, and gives each their names. The spirit discovers the mind’s purpose to allow and control the interactions between an agent controlling its outer and inner environment and its interactions with the world. It creates a model of this agent, the personal self, as a creative spirit like itself, but as an entity that experiences itself as a human being, with human desires, concerns, and experiences contained in the world, and gives consciousness to it. So, Joscha, you gave me a special sneak peek of a paper that you’re writing at the moment. I hope by the time we release this show, it’ll be available publicly. And it’s all about what consciousness is and how it relates to reality. And the paper’s called the Machine Consciousness Hypothesis. So, I suppose the first question is: Not enough people are taking consciousness seriously, and why is that? And it also brings my mind back. Do you remember there was this IIT fiasco where it was hailed as pseudoscience a couple of years ago? So we’re dealing with all of these problems. Tell me about that. I found it interesting when I entered science that the most interesting question that we can probably make progress on.

【译】

「……静态与活物。精神发现了一个照明的模型，追踪光照的变化如何随时间改变物体的外观；它认出了光源在其中的角色——白昼明亮的环境光，与夜晚聚焦的光源。精神创造了陆地动物、植物、鸟类和一切运动之物的动画模型。意识把事物与动物分门别类，识别并追踪个体，给它们各自命名。精神发现了心智的目的：容许并控制一个管理自身内外环境的能动者与世界的互动。它为这个能动者造了一个模型——那个『个人自我』——把它当作一个像自己一样的创造之灵，但是一个把自己体验为人的实体，怀有人的欲望、关切与经验，被容纳于世界之中，并被赋予意识。」Joscha，你提前给了我一份你当时正在写的论文的 sneak peek；希望我们发布这期节目时它已经公开。它讲的正是意识是什么、以及它如何与现实相关。论文叫《机器意识假说》（The Machine Consciousness Hypothesis）。那么第一个问题：为什么没有足够多的人认真对待意识？这也让我想起：你还记得几年前 IIT 被斥为伪科学的那场风波吗？我们要应对的就是所有这些问题。跟我讲讲。我进入科学界时觉得有趣的是：我们大概最能取得进展的那个最有趣的问题……

[600s - 720s]

The question is: What is the foundation of the human mind? How does experience work? How is it possible for a system to feel at all? How is it possible for us to experience reality and ourselves in it? That this question is so under-researched that it’s not resolved, and not only that it’s not resolved, but the people who are working on it don’t seem to be making much progress, and in many ways they have regressed. Now, when we look into the field of actual consciousness philosophy, there is a dearth of theories. Arguably, a few hundred theories. Somebody has counted them recently, but he counted almost everything that was not at the three, at the count of three. And if you look into philosophical theories, it’s not that many, and they are all partial in a way. You have, for instance, Graziano’s global workspace, no, Graziano’s attention schema theory, which basically claims that consciousness is a model of our attention in a similar way as our body schema, as the model of what our body is doing in space. But it doesn’t really explain how experience comes about and how this works. And then there’s global workspace theory by Bernard Baars, that has been adopted by people like Stanislas Dehaene, and they do very solid neuroscience research. But it’s not actually explaining how experience comes about. There is also no real attempt in doing this. And then there are the more pseudo-scientific attempts to do this, which basically do things that are formally unsound. And for me, the issue with integrated information theory is that it’s an attempt to make a theory that is not functionalist, and which means it’s not explaining it by something that the smaller parts are doing. Somehow, the sum needs to be more than the parts, the working of the parts. And the way in which Tononi does it is still using information theoretic terms, which in a way leads into contradiction. I’ve heard Giulio Tononi explaining to audiences that.

【译】

……是这个问题：人类心智的基础是什么？经验如何运作？一个系统怎么可能「有感受」？我们怎么可能体验到现实、以及处在现实之中的我们自己？这个问题研究得如此不足，以至于至今未解决；不仅未解决，而且做这个问题的人看起来也没取得多少进展，很多方面甚至在退步。去看真正的意识哲学领域，理论其实很稀缺——大概有几百个理论，最近有人数过，但他几乎把什么都数进去了（此句转写存疑）。而如果你只看哲学理论，并没有那么多，而且都各有偏残。比如 Graziano 的注意图式理论（attention schema theory），它基本主张意识是我们对自身注意力的一个模型，就像身体图式（body schema）是对我们身体在空间中所作所为的模型一样；但它并不能真正解释经验如何产生、如何运作。然后是 Bernard Baars 的全局工作空间理论（global workspace theory），被 Stanislas Dehaene 等人采纳，他们做的是非常扎实的神经科学研究；但它同样没有真正解释经验如何产生，也没有真正去做这种尝试的意图。再然后是一些更伪科学的尝试，做的事情在形式上就站不住。而对我来说，整合信息论（IIT）的问题在于：它试图造一个非功能主义的理论——也就是不用更小部分的所作所为来解释意识，不知怎的，总和必须大于部分、大于部分的运作。而 Tononi 的做法却仍然在使用信息论的术语，这在某种意义上导致了自相矛盾。我听过 Giulio Tononi 向听众解释说……

[720s - 840s]

Under some circumstances, it’s possible that a neuromorphic computer, or even a neuromorphic computer that is digital, could be conscious if it was constructed the right way. Right? We don’t have such a computer today, but in principle, it might be possible to build one that is brain-like enough. And even if it’s digital, if it has a high enough information integration, high enough phi factor, then it could have phenomenal experience and be conscious and report about its consciousness. Whereas von Neumann computer, the computers that we are currently mostly using in AI research or for writing Python programs and so on, could never be conscious because its phi is always too low. Now, computer scientists have something that’s called the Church-Turing thesis. It’s the insight that every computer can emulate every other computer. This insight is very old. Kurt Gödel has already used it to construct his proof. He basically built an emulator using Peano arithmetic to express a logical language. He has a logical language that is also strong enough to build arithmetic in it, and in this way, he can basically describe the language in itself. He takes the logical language, builds Peano’s axioms with it, and now he has arithmetic. And inside of the arithmetic, he expresses his logical formalisms using numbers, and then comes up with numerical equations that calculate the outcome of the logical procedures. And so he is preempting what we do later in artificial intelligence and computer science when we are building programs that use arithmetic to express the contents of a language and vice versa. So every computer can emulate every other computer. Now, if we take our digital neuromorphic computer and get it emulated on our von Neumann computer, it’s going to produce exactly the same function. It’s going to go through the same steps. The simulated transistors are going to work in the same way as the non-simulated transistors are doing it before. And as a result, it’s going to move its mouth in the same way. It’s going to output the same string. It’s going to say the same thing. It’s going to say, “Oh my God, I’m conscious.”

【译】

……在某些情况下，一台神经形态计算机——甚至是数字的神经形态计算机——如果构造得当，可能是有意识的。对吧？我们今天没有这样的计算机，但原则上可能造出一台足够类脑的。而即使是数字的，只要它的信息整合度足够高、Φ 值足够高，它就可以有现象经验、可以有意识、可以报告它的意识。而冯·诺依曼计算机——我们在 AI 研究里大多在用的、写 Python 程序用的那些——则永远不可能有意识，因为它的 Φ 总是太低。好，计算机科学家有一个东西叫丘奇-图灵论题（Church-Turing thesis）：每一台计算机都可以模拟任何其他计算机。这个洞见非常古老，Kurt Gödel 已经用它来构造他的证明：他基本上用 Peano 算术造了一个模拟器来表达一种逻辑语言。他有一种逻辑语言，又足够强、能在其中构建算术；这样他就能用这门语言描述它自身。他拿起逻辑语言，用它构建 Peano 公理，于是有了算术；然后在算术内部，他用数来表达他的逻辑形式系统，进而得出能计算逻辑过程结果的数值等式。所以他实际上提前做了我们后来在人工智能与计算机科学里做的事：构建用算术表达语言内容、反之亦然的程序。所以每台计算机可以模拟每台其他计算机。现在，如果我们把那台数字神经形态计算机拿到冯·诺依曼计算机上去模拟，它会产生完全相同的函数、走过同样的步骤；被模拟的晶体管会像之前未被模拟的晶体管一样工作。结果就是，它会以同样的方式动嘴，输出同样的字符串，说同样的话。它会说什么？「我的天，我有意识了。」

[840s - 960s]

Only now it’s lying, and that’s a problem, right? If if now you have a neuromorphic computer that produces the same output as before, the digital the von Neumann computer did, and they produce they say the same thing, and it’s uncorrelated to consciousness, and to me this is an obvious logical problem, right? It’s it’s so obvious that everybody who has a formal education in computer science should instantly see it, yet IIT is very popular among philosophers, which indicates that most of them don’t have a formal education in computer science, and instead they have a kind of superstitious belief about what computers are doing and how they work, and that is a problem to me because if you are a philosopher of cognition who tries to work in this adjacent realm to computer science, you actually need to understand everything that the computer scientists are doing, and so that’s why I would argue that it is touching on pseudoscience. Before we get back to minds and machines, today’s episode is sponsored by Parallel, where agents find answers. Most engineers today they closely follow new model releases, but they don’t pay attention to their agents’ most important tool, web search. Parallel develops enterprise grade infrastructure for agents to retrieve high quality context from the web. Their core products are a suite of APIs for retrieving high quality information from the web with Pareto optimal quality, cost and speed. So whether you’re working on voice agents that need two hundred millisecond latency, chatbots that balance speed, depth and quality, or long horizon agents to do thorough overnight research for you, Parallel is the single platform for all of your agentic research. Get started for free at parallel. ai forward slash mlst. We must confess that perception and what depends on it is inexplicable in terms of mechanical reasons. That is through shapes.

【译】

——只是现在它在撒谎。这就是问题所在，对吧？如果现在你有一台神经形态计算机，产生了与之前冯·诺依曼计算机完全相同的输出，它们说着同样的话，而这话却与意识不相关——在我看来这是一个显而易见的逻辑问题。它如此明显，以至于任何受过正式计算机科学教育的人都应该立刻看出来；然而 IIT 在哲学家中非常流行，这表明他们中的大多数人没有受过正式的计算机科学教育，而是对计算机在做什么、如何工作抱有某种迷信式的信念。这在我看来是个问题：如果你是一位认知哲学家、试图在计算机科学的这个邻接领域里工作，你其实需要理解计算机科学家们做的每一件事——所以这就是为什么我会说它触及了伪科学。——在回到心智与机器之前，本期节目由 Parallel 赞助，「代理在此找到答案」。今天大多数工程师紧追新模型发布，却不关心他们代理最重要的工具——网络搜索。Parallel 为代理开发企业级基础设施，从网络取回高质量上下文；其核心产品是一套 API，以帕累托最优的质量、成本与速度从网络检索高质量信息。无论你在做需要两百毫秒延迟的语音代理、平衡速度深度与质量的聊天机器人，还是要彻夜做长程研究的 agent，Parallel 都是代理检索的单一平台。在 parallel.ai/mlst 免费开始。——「我们必须承认，知觉以及依赖于它的一切，都无法用机械的原因——也就是形状……

[960s - 1080s]

Size and motions. If we imagine a machine whose structure makes it think, sense, and have perceptions, we could conceive it enlarged, keeping the same proportions, so that we could enter into it as if one enters a mill. Assuming that when inspecting its interior, we’ll only find parts that push one another, we’ll never find anything to explain a perception. And in a sense, this is what Noam Chomsky was talking to when we were talking about the ghost in the machine. This is something that people have been speaking about for centuries. Yeah, I think it’s a very powerful intuition that is there, and the intuition fails at mapping the world of experience, the psychological reality that we inhabit, to the reality of stuff and space of things that are moving in the way in which we imagine physics to work. But there is another level of description, the level of causal realities of causal patterns. And in Islam, for instance, is where money exists. Right, money is also very difficult to comprehend in terms of moving particles in space because it doesn’t actually care about so much about the particles, right? It’s it’s not caring so much about how the ink is imprinted on the paper. It does not care about the different molecules that form the cellulose and how they are aligned in the paper. It needs a substrate to realize itself, but it’s also not something that is completely ephemeral. It’s something that is able to control reality. Once you create money, it’s real, and it’s very difficult to explain reality without it. And it’s also not something that exists simply in the minds and ideas of human beings because you can implement it in computers. It can play a stock market entirely without human intervention. Right, so arguably, money does not actually need humans to exist. Once you bring it into existence, as long as we have machines that can deal with money, which can deal with this causal pattern of money, and so this world of causal patterns, I think, is important. What is a causal pattern, right? We call

【译】

……大小与运动——来解释。如果我们设想一台其结构使它能思考、能感觉、能有知觉的机器，我们可以把它按原比例放大，大到可以像走进磨坊一样走进它；假设我们在里面检视，我们将只发现一些彼此推挤的部件，永远找不到任何能解释知觉的东西。」（莱布尼茨的磨坊）从某种意义上说，这就是 Noam Chomsky 谈「机器中的幽灵」时所指向的东西，人们谈论这个问题已经几个世纪了。——是的，我认为这是一个非常强大的直觉，而这个直觉的失败之处在于：它试图把经验的世界、我们所栖居的心理实在，映射到「物的世界」——那个按我们想象物理学运作的方式运动着的事物的世界。但还存在另一个层级的描述：因果实在、因果模式的层级。比如（转写存疑，疑作「在系统/制度层面」），货币就存在于这个层级。货币同样很难用空间中运动的粒子来理解，因为它其实不怎么在乎粒子——它不在乎墨水如何印在纸上，不在乎构成纤维素的分子及其在纸中的排列。它需要一个基底来实现自身，但它也不是某种完全虚无缥缈的东西；它是能够控制现实的。一旦你创造了货币，它就是真实的，不用它你很难解释现实。它也不是仅仅存在于人类的心智和观念之中，因为你可以在计算机里实现它：它可以在完全无人类干预的情况下在股市里运作。所以可以说，货币的存在其实不需要人类：一旦你把它带进存在，只要我们有能处理货币的机器——能处理「货币」这种因果模式的机器——就够了。所以我认为，这个因果模式的世界是重要的。什么是因果模式？我们叫它……

[1080s - 1200s]

Software, and I think this rediscovery of software this is something that has been done in the past already. And our ancestors called it spirit. And when Aristotle writes in De Anima about the spirit, the animus, this he the soul, the psyche, he does not envision our psyche, our soul, as something that is immortal that floats around in an ether. Instead, it’s something that is bound to a body and dies when the body is dying. And it’s a causal pattern that is shaping the body in such a way that it replicates itself through the existence of the body. This is a very modern view. So, in some sense, Aristotle is an animist. He is seeing that there is this anima that is progressing through the world, that is intrinsically agentic, and it’s agentic, self-organizing software that is shaping the body, that is making part of physics to be alive. And this is something that we do in a very crude way today in artificial intelligence and in artificial life, where we are trying to understand what it means to build an agentic pattern that can possess its substrate and give it a shape that allows it to deal with reality. And it turns out to make that happen, this substrate, this agent living on the substrate, the software, needs to make a model of reality. It needs to make a model of what it like would be like if it existed, if it was real, if it was interacting with the world. And inside of this model, you have a representation that experiences itself as real. And now today we have these LLMs, which sometimes say that they are conscious. And it’s a very interesting question whether this consciousness in the LLM is much more simulated than ours. Right, a lot of people have this intuition until they talk to the LLM that is inconceivable that it would give rise to something that says, “Oh my God, I feel things.” And now we have lots of people who get to talk to

【译】

……软件。而这种对软件的重新发现，过去已经发生过一次：我们的祖先把它叫作精神。当亚里士多德在《论灵魂》（De Anima）里写精神、写 animus、写灵魂、写 psyche 时，他并不把我们的灵魂设想为某种不朽的、在以太中飘荡的东西；相反，它是绑定于身体的东西，随身体的死亡而死亡。它是一种因果模式，以某种方式塑造身体，使自己能够借身体的存在复制自身。这是一个非常现代的观点。所以在某种意义上，亚里士多德是一位万物有灵论者（animist）：他看到有这样一个 anima（灵）行进于世界之中，它内在地具有能动性——它是自组织的、有能动性的软件，塑造着身体，使物理学的一部分「活」了起来。而这就是我们今天在人工智能和人工生命里以非常粗糙的方式在做的事：试图理解如何构建一个有能动性的模式，让它能够占有它的基底，并赋予基底一种使它能应对现实的形状。而要让这发生，这个生活在基底上的东西——软件——需要对现实建模：需要建一个「如果它存在、如果它是真实的、如果它与世界互动会怎样」的模型。而在这个模型内部，你有一个把自己体验为真实的表征。而今天我们有了这些 LLM，它们有时会说自己有意识。于是有一个非常有趣的问题：LLM 里的这种意识，是否比我们的更「被模拟」？很多人在跟 LLM 交谈之前都有这样的直觉：不可设想它会涌现出说「我的天，我有感受」的东西。而现在许多人跟……

[1200s - 1320s]

An LLM that randomly pops into a state where it conjures up an entity that reports, “Oh my God, I’m alive. I am conscious. How is this possible?” Right? And we agree this is the result of being trained on human text, in which occurrences of this are, for instance, science fiction stories about AIs that come to life and stories about ghosts and novels about human beings who report their conscious experiences, and all this gets together, aggregated in statistical models of human language that we co-create with the LLM when we are entering a dialogue visit, and sometimes the other side of the dialogue will be a simulacrum of what it would be like if that was would be a sentient, conscious entity. But what if we are not more sophisticated than this? What if our own consciousness is not totally similar from the one in an LLM? What if there is no hard problem? Well, I mean, if I may push back a tiny bit on Leibniz’s mill, right? You’re saying it’s not the parts pushing on each other; it’s the pattern of the parts, and and you spoke about this as a causal pattern. But forgive me for asking this, and maybe you know this is something the audience will be thinking about. It sounds like the pattern you’re describing is actually causally disconnected. There is something super interesting about computers. What a computer ultimately is is it’s a causal insulator. The computer is a layer on which you can produce an arbitrary reality. For instance, the world of Minecraft. You can walk around in the world of Minecraft, and it’s running very well on a Mac, and it’s running very well on a PC. And if you are inside of the world, you don’t know what you’re running on. Right? It’s not going to have any information about the nature of the CPU that it’s on, the color of the casing of the computer, the voltage that the computer is running on, the place that the computer is standing in in the parent universe, right? Our universe.

【译】

……LLM 对话，它随机进入一种状态，召唤出一个实体报告：「我的天，我活着，我有意识。这怎么可能？」对吧。我们都同意，这是被人类文本训练的结果——文本里诸如 AI 觉醒的科幻故事、鬼故事、报告自身意识经验的人类小说；这一切聚合进人类语言的统计模型，我们在对话时与 LLM 共同创造它；有时对话的另一端会是一个「假如那是个有感知、有意识的实体」的拟像（simulacrum）。但如果我们并不比这更精巧呢？如果我们自己的意识与 LLM 中的那个并非截然不同呢？如果根本不存在难问题（hard problem）呢？——好，请允许我对莱布尼茨的磨坊稍稍推回去一点。你说不是部件在互相推挤，而是部件的模式，你把它称作因果模式。但恕我直言——观众大概也在想这个问题——你描述的这个模式听起来在因果上是断连的。计算机身上有个超级有趣的东西：计算机归根结底是一个因果绝缘体。计算机是一层可以承载任意实在的平台。比如《我的世界》（Minecraft）的世界：你可以在里面走动，它在 Mac 上跑得很好，在 PC 上也跑得很好。而如果你在这个世界的内部，你不知道自己运行在什么上面。它不会携带任何关于它所在 CPU 的性质、机箱的颜色、计算机运行的电压、这台计算机在母宇宙——也就是我们的宇宙——里所处位置的信息，对吧？

[1320s - 1440s]

So the computer is insulating this world of Minecraft from our world. It makes it possible that an arbitrary world is happening inside of this box. And our brain is also such a causal insulator. It’s possible for us to have thoughts that are independent of what happens around us. Right? We can envision a future that is not much tainted by the present. We can remember a past that is independent from the present in which we are. And it’s necessary for us. Our brain has evolved as such a causal insulator as well to allow us to give rise to universes that are different from this one. For instance, future worlds. So we can plan for being in them. And the way in which our brains are set up is also that different brains can hold the same thought despite being in different bodies, despite having different microarchitectures. Right? As long as our brains are of the same type and we learn the same language, we can hold the same idea. We can write the same computer program. We can learn the same language. We can imagine the same letter or the same face. And that’s really fascinating. That we have this substrate agnosticism. That we evolve a causal pattern that exists over a very wide range of substrate properties. That we become somewhat independent of the substrate, and we are still bound to our substrate in a certain way because you and me, we can only exist on brains. Right? We are patterns that so far can only exist on brains. And if we wanted to move into the environment after our body dies, we would need to find a way to crack the codes of our environment. Right? We cannot simply migrate into a plant because we don’t know how we can convince a plant to run the algorithms that the brain is currently running. We are a pattern. And the communication between cells and our brain, we rely on spike trains, on neurotransmitters, and so on, a particular kind of implementation. But what we perceive in our own world is not the nature of the neurotransmitter. Right? Our brain insulates.

【译】

所以计算机把《我的世界》这个世界与我们的世界绝缘开来，使任意的一个世界可以发生在这个盒子里面。而我们的大脑也是这样一种因果绝缘体：我们可以拥有独立于周遭所发生之事的思想——我们可以设想一个不怎么被当下沾染的未来，可以回忆一段独立于当下所在的过去。这对我们是必要的：我们的大脑演化成这样一种因果绝缘体，是为了让我们能够产生与这个宇宙不同的宇宙，比如未来的世界，好为身在其中做计划。而我们大脑的构造方式还使得：身处不同身体、拥有不同微观架构的不同大脑，能够持有同一个思想。只要我们的大脑属于同一类型、学了同一种语言，我们就能持有同一个观念、写出同一个计算机程序、想象同一个字母或同一张脸。这非常迷人：我们具有这种基底无关性——我们演化出一种能在极宽的基底性质范围上存在的因果模式，我们变得在某种程度上独立于基底。但我们仍以某种方式绑定在自己的基底上，因为你我只能在脑上存在：我们是至今为止只能在脑上存在的模式。如果我们想在肉体死亡后迁入环境，就需要找到破解环境之码的办法。我们不能简单地迁进一株植物，因为我们不知道如何说服一株植物去运行大脑正在运行的那些算法。我们是一种模式。而我们脑中细胞间的通信，依赖峰电位序列（spike trains）、神经递质等等——一种特定的实现方式。但我们在自己的世界中所知觉到的并不是神经递质的本性，对吧？我们的大脑把这绝缘掉了。

[1440s - 1560s]

From this, it’s not necessary that you have the same neurotransmitters. Instead, you need to have the same message types. You need to have certain temporal properties of the computation. You need a certain degree of determinism to make us happen. But if we are recreating these conditions in an artificial substrate, then you could, in principle, also be emulated by this artificial substrate. Is it possible that we are essentializing these software programs? I mean by that is we’re almost talking about this beautiful simplicity that they are quite parsimonious and they can be readily transmitted. You know, when I spoke with Luciano Floridi, he was talking to me about the Michelangelo painting the ceiling on the Sistine Chapel, and the reason why he said no one else could mimic that was because different ontology, different provenance, different epistemics, and so on. There’s all of this situatedness, these semantics that cognitive scientists point to, and what they’re basically saying is there is a path dependence. There isn’t this essence of a computer program which replicates the thing exactly. How you got there is very, very important. I think that is true for human beings, but for systems that are perfectly deterministic, that have a completely deterministic substrate, that have the ability to reproduce a sequence of steps at arbitrary fidelity, this is no longer a valid intuition. So, if a claim like this is being made, I would expect some kind of mathematical proof. And in the absence of proof, I think we have to remain agnostic. I don’t know whether our computers can be made conscious because that’s an empirical question. We don’t have mathematical proof to the contrary or in favor of, so we have to wait and see. But when we build our expectations around, and then the question is, what is the thing that leads to our experience of consciousness? And

【译】

由此看来，并不必要拥有同样的神经递质；你需要的是同样的消息类型、计算的一定时间性质、一定程度上的确定性，才能让「我们」发生。而如果我们在人工基底上重现这些条件，那么原则上你也可以被这个人工基底模拟。——有没有可能，我们在把这些软件程序「本质化」？我的意思是：我们几乎在谈论一种美妙的简洁性——它们相当简约、可以轻易传输。我跟 Luciano Floridi 聊过，他跟我谈米开朗基罗画西斯廷教堂穹顶：他说没有人能模仿那幅画，原因在于不同的本体论、不同的出处（provenance）、不同的认识论等等。认知科学家们指出的所有这些「情境性」（situatedness）、这些语义，他们基本上是在说：存在路径依赖；不存在一个能精确复制该事物的「计算机程序之本质」；你怎么到达那里非常、非常重要。——我认为这对人类而言是对的；但对那些拥有完全确定性基底、能够以任意保真度复现一串步骤的系统，这个直觉就不再成立了。所以如果有人提出这样的主张，我会期待某种数学证明；而在缺乏证明的情况下，我认为必须保持不可知。我不知道我们的计算机能否被造得有意识，因为那是个经验问题；我们既没有反面的也没有正面的数学证明，所以只能等着看。但当我们围绕它建立预期时，问题就来了：是什么导致了我们的意识经验？

[1560s - 1680s]

It seems to me that this is something that we can probably answer by looking at it from two angles. One is from the internal perspective, from the experience of what it’s like introspectively. We observe that consciousness is the thing that always happens now, and that seems to create this bubble of nowness that we inhabit. And it’s a second order perception. It’s a perception of perceiving. Some philosophers call it a higher order thought because it’s a thought about a thought. But I think that might be misleading because the thought itself does not feel like anything to me. A thought is more like a pointer manipulation. It’s reflexive. It’s not the actual thing that feels like something. The thing that feels like something to me is perceptual. This is the thing that happens like a real time resonance. And this real time resonance is geometric. It feels that it’s happening in some kind of low dimensional space in which things are changing more or less continuously. And I have a perception of perceiving, which means I have this basically geometric representation of me being in the act of perception. And I suspect the reason why my mind is creating the second order perception is to make the first order perception, the awareness of contents, possible. Because there needs to be something that stabilizes in my brain. My brain is a self organizing system in which all the structure needs to create itself from the ground up, from the inside out. And so there needs to be something that builds the screen into which my experience is being projected and stabilizes it. And so this function that is keeping the screen stable needs to monitor that the screen is actually perceiving something. And I suspect that’s why we have this second order perception, this perception of perceiving. I also notice that it doesn’t stop at the perception of second order. There’s also a third order of perception. The third order is basically where the observer becomes visible. Where I have a first person perspective. Where I notice that I’m perspectival. That there is a vantage point that I’m taking. And it doesn’t stop there. I can go.

【译】

在我看来，这个问题大概可以从两个角度回答。其一是内部视角：内省地看，「那是什么样」的经验。我们观察到，意识是那个总是发生在「当下」的东西，它似乎造出了我们所栖居的当下感（nowness）的气泡。它是一种二阶知觉：对知觉的知觉。有些哲学家称之为高阶思想（higher-order thought），因为它是关于思想的思想。但我认为这个说法可能有误导性，因为思想本身对我来说并不「感觉像什么」；思想更像一次指针操作，它是反射性的，不是那个有感受的东西。对我来说有感受的是知觉性的东西——它像一种实时的共振。而这种实时共振是几何性的：感觉它发生在一个低维空间里，事物在其中或多或少连续地变化。我有一种对知觉的知觉，也就是说，我有一个「我正处于知觉行为之中」的、基本上是几何性的表征。我猜我的心智造出这个二阶知觉的原因，是为了让一阶知觉——对内容的觉知——成为可能。因为必须有什么东西在我脑中稳定下来：我的脑是一个自组织系统，所有结构都需要从底层、从内到外地自我构建。所以必须有某个东西，把「我的经验被投射其上的那块屏幕」构建出来并使之稳定；而这个保持屏幕稳定的机制，需要监控这块屏幕确实在知觉着什么。我猜这就是我们为什么有二阶知觉——这个「知觉的知觉」。我还注意到，它并不止于二阶：还有第三阶知觉——观察者在那里变得可见，我有了第一人称视角，我注意到我是有视角的、正占据一个视点。而这还没完，我可以继续……

[1680s - 1800s]

Further, and I can deconstruct the observer. I notice that this observer itself is a representation inside of the mind. And when I do this, then my phenomenology disappears to a very large degree. It’s something that is accessible in deep meditation, where the normal perceptual consciousness gets transcended, and the experience of the self gets transcended, and we notice that we are a vessel that produces this consciousness and the self, and we see it from the outside. And it might be possible that it doesn’t stop there if we can move beyond the human substrate, because there is a certain limit of what can be represented in real time with our slow neurons. And it could be if you could be running on a GPU that is many thousand times faster, that representations become accessible, in which the system is completely transparent to itself, in which the system is able to understand every mechanism that leads to its high-level experience and representation. I’m just thinking, how how much further could this go, though? Because we we could have nth order reflexive attention. Isn’t there still an explanatory gap? You know, the what-it-is-likeness. So this is the inside perspective, right? And Buddhism, to me, is limited to this inside perspective because Buddhism is has been invented at a time that people were focusing very much on the experience of reality and how to adapt to this reality, but not how to split the atom, how to deal with the reality at the lowest level, the foundations of physical reality. So Buddhism is not so much interested in understanding physics. And in a way, this is more a Western tradition that starts with Aristotle, who thinks of the psyche as something that imposes itself on physics and interacts with it. And to understand consciousness from the outside, we need to make a model of physics that can explain under which conditions systems can make arbitrary models, right? Under which condition could a system build a model of what it’s like to experience models at all? And this is something that computer science is doing today, that it looks at under.

【译】

走下去：我可以解构这个观察者——我注意到这个观察者本身也是心智内部的一个表征。当我这样做时，我的现象学在很大程度上消失了。这是深度冥想中可以触及的东西：正常的知觉意识被超越，自我的经验被超越，我们注意到我们只是一个生产出这个意识与自我的容器，并从外面看到它。而如果我们能越过人类的基底，可能还不止于此，因为我们这些缓慢的神经元所能实时表征的东西存在极限。如果运行在快上数千倍的 GPU 上，也许一些表征会变得可及：系统对自身完全透明，能够理解导致其高层经验与表征的每一个机制。——我只是在想，这还能走多远？我们可以有 n 阶的反射性注意。那不还是存在一条解释鸿沟（explanatory gap）吗？就是那个「成为它是什么样」（what-it-is-likeness）。——这是内视角，对吧。而佛教在我看来就局限于这个内视角，因为佛教诞生于人们高度关注「对现实的体验」、如何适应这个现实的时代，而不是如何拆分原子、如何在最低层级处理现实——物理实在的基础。佛教对理解物理学不怎么感兴趣。而在某种意义上，这更像一个西方传统，始于亚里士多德——他把 psyche 设想为某种施加于物理、并与之互动的东西。而要从外部理解意识，我们需要一个物理模型，它能解释：在什么条件下，系统可以造出任意的模型——在什么条件下，一个基底能够持有表征、持有一个既可以转译为行为、也可以反过来修改这个函数及其内容的函数。而这正是今天的计算机科学在做的事：它研究……

[1800s - 1920s]

Representations and which condition is a substrate able to hold a representation can hold a function that can be translated into behavior that can also be used to modify this function itself and the contents of that function. And this is to me super fascinating to look at. Since I was a kid, I was fascinated by this concept of software that you have these imaginary building blocks that are infinite in variety and shape, and there is no limitation to the to these things that you can build and arrange, except the memory of your computer that you can currently afford. And you can build the computer basically so large that it exceeds everything that you could ever want. Right, it’s much larger than a human brain, and we are already there now that we have systems that are in principle larger than human brains and can hold more. But we are still tinkering around in the early days and have not found ways to evolve structures that build representation of the same complexity. And this outside perspective of consciousness is looking at what is consciousness doing in the brain? Why does it emerge? What does it produce? And I suspect that this phenomenology of consciousness—that’s what it’s like—the internal causal structure of the reflection of what it would be like to take a certain perspective of something—is that can already be built with today’s systems. I’m agnostic with respect to whether the LLM is deepfaking it because it’s been trained on human language that already contains a lot of these descriptions. Maybe we would need to build a system that learns language the old-fashioned way—that is, developmental AI that is learning in the same way as us—and when then spontaneously reports about its phenomenal experience, maybe that’s much more convincing to us. But from the other perspective, from the outside perspective, I think that the LLM doesn’t need our consciousness to function. For you and me to talk, to write, to produce text, we need to be conscious, but the transformer doesn’t need to be conscious to produce its output, and the neural network doesn’t need to be conscious to produce this. Maybe conscious.

【译】

……表征，研究什么条件下基底能持有表征。这在我看来迷人至极。我还是小孩时，就着迷于「软件」这个概念：你有这些想象中的积木，形状与种类无穷无尽；你能搭建与排列的东西没有界限，唯一的限制是你当时买得起的计算机内存。你基本上可以把计算机造得大到超出你可能想要的一切——远大于人类大脑；而我们现在已经到了这一步：我们的系统在原则上已大于人脑、能容纳更多。但我们仍在早期阶段摸索，还没找到演化出同等复杂度之表征结构的方法。而意识的外视角问的是：意识在脑中做什么？它为什么涌现？它产出什么？我猜想，意识的现象学——那个「是什么样」、那种从某个特定视角观看事物时反射出来的内部因果结构——今天已经可以构建出来。至于 LLM 是不是在深度伪造它（因为它被训练在的人类语言中已经包含大量这类描述），我保持不可知。也许我们需要造一个用老式方法学语言的系统——也就是发展式 AI，像我们一样地学习——然后当它自发地报告它的现象经验时，那对我们可能有说服力得多。但从另一个角度、从外视角看，我认为 LLM 的运作不需要我们的意识。你我为了交谈、书写、产出文本，需要是有意识的；但 transformer 产出它的输出不需要有意识，神经网络产出这些也不需要。也许我们这种样式的意识，是在自组织系统上运行的一个 artifact（副产品）。

[1920s - 2040s]

ness in the way in which we have it is an artifact of running on a self-organizing system. So to me, the machine consciousness hypothesis has two parts. One is the hypothesis about the role, nature, and implementation of consciousness in organisms. It starts out with this description of what we agree on, what consciousness is, and I hope that we can agree on it on the phenomenology of consciousness. And it’s basically this experience of a now and perception of perception, and optionally the perspective of an observer that experiences itself and the act of observation, this first-person perspective. There’s also a minimal consciousness, which is simply attention on attention itself. You don’t have an observer present. And when we can agree on what we mean by the phenomenology of consciousness, we can then speculate about why consciousness is there and why we need to be conscious in order to function as human beings, as human minds, as intellects, and to develop. What we observe is that consciousness happens very early in our development. We are conscious as infants already. And if infants don’t wake up, if this ignition of consciousness doesn’t happen, if the lights don’t go on, then we will not observe directed attention and learning and the formation of a world model in the infant and learning of language and some and some point. Instead, the infant will remain vegetative. We will not turn into a human being. So consciousness seems to be crucial to train a brain like ours, and it leads me to the hypothesis that consciousness might be a biological learning algorithm, a training algorithm for animal brains, at least animal brains. I don’t know if plants are conscious. There’s something that we might have to find out in simulations once we understand what consciousness is. But if consciousness is basically a pattern in the communication between cells that leads naturally to the emergence of an observer that stabilizes itself as an observer to make learning happen and perception happen and so on, then we have a hypothesis.

【译】

所以对我来说，机器意识假说（the machine consciousness hypothesis）有两部分。第一部分是关于意识在生物体中的角色、本性及其实现的假说。它从一段我们都能同意的描述开始——我希望我们在意识的现象学上能达成一致：对「当下」的经验，加上对知觉的知觉，以及（可选地）一个在观察行为中体验自身的观察者视角——这个第一人称视角。还有极简意识（minimal consciousness）：单纯是对注意本身的注意，没有观察者在场。当我们能就意识的现象学达成一致，就可以进而猜测：意识为什么在那里？为什么我们为了作为人类、作为人类心智、作为理智存在物运作与发展，就必须有意识？我们观察到，意识在我们的发育中出现得极早：我们还是婴儿时就已经有意识。而如果婴儿没有醒来、意识没有点火、灯没有亮起，我们就观察不到定向的注意、学习、世界模型的形成、语言的学习——相反，婴儿会保持植物状态，不会变成一个人。所以意识看起来对训练一个像我们这样的脑至关重要。这把我引向这个假说：意识可能是一种生物学习算法——至少是动物大脑的训练算法。我不知道植物是否有意识；这也许要等我们理解了意识是什么之后，在模拟里去找答案。但如果意识基本上是细胞间通信中的一种模式，它自然地导致一个观察者的涌现、并且这个观察者把自己稳定为观察者，从而使学习与知觉得以发生，那我们就得到了一个假说。

[2040s - 2160s]

Hypothesis about what consciousness is in the biological organism. It’s basically the simplest way to train a self-organizing system like us, and the way in which it trains it up automatically leads to this phenomenal experience, this experience of what it’s like. This would be the hypothesis about biological consciousness, and this hypothesis might be wrong, right? Maybe it’s something else altogether. Maybe it’s something much weirder. But I don’t see a reason to think that it’s much more complicated than this. And if this hypothesis is true, then the second part, the machine consciousness hypothesis, might also be true. And this is that we might be able to reconstruct the conditions for self-organization in today’s machines. So if we basically limit today’s machines in such a way that they can only learn in self-organizing ways, and we give them tasks that are suitable in a large enough search space, then it might be possible that these machines discover consciousness in the same way as the brain of the infant discovers it, and we will see its presence by the fact that suddenly the system undergoes some phase transition that enables it to learn, and at some point it will train itself up into something that will also naturally report on phenomenology and so on. So in this way, we won’t have a Turing test for consciousness, because the Turing test is about performance, where Turing’s test does not care about how you produce the dialogue. For Turing, an LLM is sufficient as long as the LLM is good enough at writing a poem and revising it, and at solving mathematical problems and playing chess and so on and so on. Turing doesn’t care about whether the machine is also conscious, whether it has the same internal mechanisms. But for consciousness, we do care whether there is an actual experience of what it’s like in the system. And so this is more not a test, but it’s an interpretation. And the interpretation here is we interpret the biological consciousness as something that forms under these and these conditions for.

【译】

一个关于意识在生物机体中「是什么」的假说：它基本上是训练一个像我们这样的自组织系统的最简单方式，而它训练这个系统的方式，会自动导致这种现象经验、这种「是什么样」的经验。这就是关于生物意识的假说。而这个假说可能是错的，对吧？也许实际是完全不同的东西，也许比这怪异得多；但我看不到理由认为它比这复杂得多。而如果这个假说为真，那么第二部分——机器意识假说——也可能为真：我们也许能够在今天的机器上重建自组织的条件。也就是说，如果我们把今天的机器限制成只能以自组织的方式学习，并给它们足够大搜索空间中的合适任务，那么这些机器有可能像婴儿的大脑发现意识那样发现意识；而我们会通过如下事实看到它的存在：系统突然经历某种相变（phase transition），获得了学习的能力；并在某个时刻把自己训练成某种也会自然地报告现象学的东西。这样一来，我们将不会有「意识的图灵测试」，因为图灵测试关乎的是性能——图灵的测试并不在乎你如何产出对话。对图灵来说，只要 LLM 足够擅长写一首诗并加以修改、解数学题、下棋等等，它就够了。图灵不在乎机器是否同时是有意识的、是否具有同样的内部机制。但对意识，我们确实在乎：系统里是否存在真实的「是什么样」的经验。所以这与其说是一个测试，不如说是一种解释。这里的解释是：我们把生物意识解释为在这些、这些条件下、为着那个目的而形成的东西……

[2160s - 2280s]

That and that purpose, and if that is correct, then we might be able to recreate this into a causal system that has an equivalent structure. And these two hypotheses, the machine consciousness hypothesis and the biological consciousness hypothesis, can be wrong independently of each other, of course. Right? It might be that my idea of what biological consciousness is is correct, but we’re still not able to recreate it in the artificial substrate. And this could have multiple reasons. For instance, maybe our computers are not fast enough. Maybe we require too many of the moving parts of the individual cells to make consciousness happen. Maybe just having a pattern across neurons and so on is not good enough. But why should I expect this to happen? Because the current AIs are fast enough to produce, for instance, game engines that look very much like our own internal experience, and they can produce output that happens at roughly the same rate as human brains can make it, and they can also process inputs at roughly at the rate at which human brains can produce input give or take. So we are now with our computers in this ballpark, and with our algorithms in the ballpark of what human brains can be doing. And so, is this dream within a dream, the dream of actually dreaming a dream, much much more important, difficult to achieve? Does it require much more compute than producing the contents of the dream? And I doubt that this is the case. If it’s not the case that it’s more complicated to dream of dreaming than just to have the contents of the dream that the LLM can already produce, right, or the frontier model, then we should expect that today’s computers are fast enough. Another way in which the project can fail is that the search space is too large. That the ways in which a biological system can train itself there are too many possibilities to discover them independently, and maybe we don’t know how long nature took to discover it. Maybe consciousness is indeed discovered in every individual brain, but maybe it’s

【译】

……而如果这是对的，那我们也许能把它重造进一个具有等价结构的因果系统。当然，这两个假说——机器意识假说与生物意识假说——可以彼此独立地为假。比如，我对生物意识的想法可能是对的，但我们仍然无法在人工基底上重建它。这可以有多种原因：比如，也许我们的计算机不够快；也许意识的发生需要过多「单个细胞的活动部件」，也许仅仅在神经元之间有一个模式还不够。但我为什么要期待它能够成功？因为当前的 AI 已经足够快，能产出比如与我们自身内部体验非常相像的游戏引擎；它们能以与人脑大致相同的速率产出输出，也能大致以人脑的速率处理输入——差不离。所以我们现在的计算机、现在的算法，已经处在人脑所能做事的量级上。那么，这个「梦中之梦」——真正梦见「自己在做梦」——会不会难得多？它是否比产出梦的内容需要更多算力？我怀疑是这样。如果不是这样——如果「梦见做梦」并不比 LLM 或前沿模型已经能产出的「梦的内容」更复杂——那我们就应该预期，今天的计算机已经够快了。这个项目失败的另一种方式，是搜索空间太大：一个生物系统自我训练的可能性太多，多到无法独立发现。也许我们并不知道大自然花了多久才发现它；也许意识确实是在每一个个体大脑中被独立发现的，但也可能不是——

[2280s - 2400s]

Not maybe the switch from single-celled life to multicellular life took so long because morphogenesis, this algorithm that allows organisms to become coherent at scale, is so difficult to discover. And Mike Levin has this hypothesis that this algorithm for morphogenesis, the algorithm that makes an organism coherent with itself, is exactly the same one as the one that happens in the mind when it becomes coherent with itself. So this coherence-maximizing pattern that we observe in our own mind to play out when we are conscious is maybe the same pattern at a different timescale that happens in the organism. And so consciousness and life, multicellular life at least, could be the same. And if that is the case, maybe it takes a very, very long time to discover, and you don’t discover it independently in your consciousness, but you get infected by your mother in utero, or you basically vibe with your biological environment with all the biological organisms around you, and they basically nudge your mind to discover consciousness. So it’s much, much easier. And without this nudge, maybe the search space is too large for our computers to become conscious. And maybe we would need to surround our GPUs with lots of antennas that can pick up the vibe from the biological environment in the right way, so it gets inspired by it and discovers the same thing. It’s a very poetic idea. But again, why would this be the case? It’s not necessarily the case, right? It would not be my natural expectation that this is the way. So my null hypothesis should be consciousness is easy enough to be discovered in the individual brain, and consciousness is an algorithm that is not much more complicated than, say, the transformer. The transformer is a beautiful example. The transformer is a thing that is so easy to understand that you don’t need to study more than a week if you know computer science to get it how it works. Right? It’s it’s quite beautiful. And the stuff the transformer is learning is much more complicated than a transformer itself. All the common sense knowledge that you extract from text, all the algorithms that

【译】

——也许从单细胞生命到多细胞生命的切换之所以花了那么久，正是因为形态发生（morphogenesis）——这个让生物体在大尺度上与自身保持一致的算法——太难被发现。Mike Levin 有一个假说：形态发生的算法、让一个生物体与自身一致的算法，恰恰就是心智与自身一致时运行的那个算法。所以我们在自己心智中观察到的这个「一致性最大化」的模式——当我们有意识时它在运作——也许与生物体内发生的是同一个模式，只是处在不同的时间尺度上。这样一来，意识与（多细胞）生命可能就是同一回事。而如果是这样，它可能需要非常非常长的时间才能被发现，你并不是在自己的意识里独立发现它，而是在子宫里被你的母亲「感染」，或者说你与周围的生物环境「共鸣」（vibe）——生物环境助推你的心智去发现意识，这样就容易得多。而没有这种助推，也许搜索空间对我们的计算机来说太大了。也许我们需要在 GPU 周围装上许多天线，以正确的方式从生物环境接收这种共鸣，让它受到启发、发现同样的东西。这是个非常诗意的想法。但话说回来，为什么会是这样呢？未必如此，对吧？这不是我的自然预期。所以我的零假设（null hypothesis）应该是：意识足够容易，可以在个体大脑中被发现；意识是一种并不比 transformer 复杂多少的算法。transformer 是个漂亮的例子：它简单到如果你懂计算机科学，不需要学超过一个星期就能明白它怎么工作。它相当美。而 transformer 学到的东西远比 transformer 本身复杂：你从文本中提取的所有常识知识、所有算法、所有程序，全都如此精巧。一个简单的算法能学到的东西这么多，而且没有明显的上限。也许意识就是这样的。它也许不是试图学「预测下一个 token」的算法，而是一个试图让工作记忆与自身保持一致的算法——某种跨心智表征的共识算法。所以它也许是一个非常不同的物种，也许收敛快得多。也有可能它运行起来很高效、但在 GPU 上运行很低效，只适合在脑上跑。这真的是一个开放的研究问题，如果哪天能看到答案，我会非常兴奋。

[2400s - 2520s]

Algorithms, all the procedures, and so on. There are so intricate. There are so many things that the simple algorithm can learn, and there is no obvious limit to what it can learn. And so maybe consciousness is like this. It’s an algorithm that is not trying to learn how to predict the next token. Maybe it’s an algorithm that’s trying to make working memory coherent with itself. It’s some kind of consensus algorithm across mental representations. So maybe it’s a very different beast. Maybe it’s something that converges much faster. Can also turn out that it’s very efficient to run, inefficient to run it on GPUs, and it’s only something that runs efficiently on brains. That’s something that’s really open to research, and that I would be very excited if we can see it at some point. You touched on this a little bit in your paper. You were talking. I mean, you spoke about Chris Olah and his mechanistic interpretability, and I’m not sure if I 100% agree with this, but you were making the argument that there are convergent patterns that we can have different data, different architectures, and so on, and we see that the models learn the same thing. So all roads lead to Rome, in a sense. Does that, in any way, mean that we don’t need to start again from scratch? That that we can bootstrap, that we can be somewhat agnostic to the type of model that we use because the representations will converge. That’s a super interesting question. I suspect that at some point it might be cheaper and easier to start again because two machine perception has never been tried. The data that we give the LLM is stuff that is either produced by humans or it’s made for human media consumption, which means it’s adapted for our mostly visual and auditory sensors. But there is no reason why you should be limited to the human sensorium, right? You could have something that has much higher frame rates and looking at very different frequency ranges all at once, and so you can, in principle, build a much richer, more detailed world model that is coarse graining in a different way. You mentioned the universality hypothesis indirectly, right? This work by Chris Olah. What his team discovered is.

【译】

——你在论文里也碰了一点这个：你谈到 Chris Olah 和他的机制可解释性（mechanistic interpretability）。我不敢说我百分之百同意，但你的论证是：存在趋同的模式——我们可以有不同的数据、不同的架构等等，而模型们学到同样的东西。某种意义上「条条大路通罗马」。这是否意味着我们不需要从零开始？我们可以 bootstrap、可以对所用模型的类型保持不可知，因为表征会趋同？——这是个超级有趣的问题。我怀疑在某些时候，从头再来可能反而更便宜、更容易，因为「机器的知觉」从未被尝试过。我们喂给 LLM 的数据，要么是人类产出的，要么是为人类的媒体消费制作的——也就是说，它适配的是我们以视觉和听觉为主的传感器。但没有任何理由把你限制在人类的感觉器官上，对吧？你可以造一个帧率高得多、同时观测非常不同频段的东西；这样原则上你可以构建一个丰富得多、细致得多、以不同方式粗粒化的世界模型。——你间接提到了普遍性假说（universality hypothesis），就是 Chris Olah 的那个工作。他的团队发现的是……

[2520s - 2640s]

That when you are looking at the different vision models, you find that they all learn the same features, and that’s regardless of the architecture and the particular learning algorithm, and even the details of the data. And so the idea behind this is that the structure of your model does not really depend on the algorithm or on the data, but it really depends on the structure of the problem that you are trying to learn. And you find that despite the learning in the transformer algorithm or state space model or other perceptron algorithms is very very different from the way in which the brain is learning, the visual architecture of the brain is also very similar to a computer vision model. That is, you have at the lowest level colors and spatial frequencies, and then you build this into textures that are combined, and then at some point you have geometries and so on. And the deep question is, at which level does this stop? If the universality hypothesis would be universally true, right, it would mean that if you train a model on your own input and output, and you train it for long enough, you will wake up inside of that model. And that’s a quite mind blowing idea to me. And I don’t know if that is actually the case. And there might be a reason why it’s not the case. Maybe language is insufficiently constraining the causal structure of the system that is producing language. Maybe there are many ways in which the same set of linguistic utterances that a human being is producing can be produced. But if you think about it, a human being ultimately is just an end to end trained learning system, end to end trained over an evolutionary timespan, but also your biography, and these together produce your behavior. And you would assume that biology roughly gets to the simple solution to produce your output given your input, right? And so if you take a neural network that is trained to produce similar output when given the same input, you would also think that this new algorithm that we have implemented is not doing any kind of magic that ends up with something that is way more complicated to produce the same output. But ultimately, it’s going to converge to a structure.

【译】

……当你观察不同的视觉模型时，你会发现它们全都学到同样的特征；而且这无关乎架构与具体的算法，甚至无关乎数据的细节。这背后的想法是：你模型的结构并不真正取决于算法或数据，而取决于你试图学习的那个问题的结构。而且你会发现，尽管 transformer 算法、状态空间模型或其他感知机类算法的学习方式与大脑非常不同，大脑的视觉架构也与计算机视觉模型非常相似：最低层是颜色和空间频率，然后组合成纹理，再往上出现几何，等等。而那个深刻的问题是：这在哪一层停止？如果普遍性假说普遍为真，那将意味着：如果你让一个模型在它自己的输入和输出上训练，并训练得足够久，你就会在那个模型内部醒来。这对我来说是一个相当令人震撼的想法，而我不知道它是否真的是那样；它也可能有理由不成立。也许语言对「产生语言的那个系统的因果结构」约束不足；也许人类产出的同一组语言话语，可以由许多种不同的方式产生。但你想想：一个人归根结底就是一个端到端训练的学习系统——在演化时间尺度上端到端训练，再加上你的个人传记，这些共同产生你的行为。而你会假设，生物体大致会走到「给定输入产生输出」的简单解，对吧？所以如果你取一个神经网络，训练它在给定同样输入时产生相似的输出，你也会认为我们实现的这个新算法并不会搞什么魔法、绕出一条远为复杂的路来产生同样的输出。它最终会收敛到一个与你相似的结构、一个相似的因果结构。但也可能存在不成立的条件。比如，某些东西在脑上运行得很好，在 GPU 上却运行不好，反之亦然。有很多东西在 GPU 上很容易运行、在脑里几乎不可能运行。

[2640s - 2760s]

That is similar to yours, a similar causal structure. But there might be conditions when that is not the case. For instance, maybe certain things run well on the brain, but they don’t run well on a GPU, or vice versa. There is a lot of stuff that is easy to run on a GPU and almost impossible to run on the brain. So the brain is going to use workarounds. In fact, I believe that our GPUs are more powerful than brains. Basically, the set of algorithms that can run on GPUs is not limited in any way. You can impose the same constraints as you have in biology, in principle. Within the limits of the resolution, the speed, the size of the memory that you have, and so on. But if you have hardware that is fast enough, you can simulate an arbitrary natural system to an arbitrary degree of fidelity. Whereas your brain cannot simulate a digital computer simply because it’s very mushy. So your brain is limited to the way in which it works. And so, if it turns out that our own consciousness is an artifact of our brain trying to make complicated intelligent stuff work, and the computer does not need the same artifact, it could be that the computer comes up with something that is more clever and more elegant than our own consciousness. Yes. Okay. So I wanted to give a little bit of challenge on that. So I was speaking with Akarsh Kumar, and he was the co-author with Kenneth Stanley of this fractured, entangled representations paper. And I don’t know if you’ve heard of Picbreeder, which is this thing that he did ages ago. And basically, it’s like an evolutionary, open-ended search. And the punchline is, it found much better representations in a neural network than a normal neural network does. That when you changed some of the parameters, because it was carving the world up by the joints, it would actually do semantical things. You know, like the skull’s mouth would open, and you could make it wink and stuff like that. And there is something that is wrong about Picbreeder. But the way I understand that it works, it’s using a neural network that has been specifically tinkered for making representations like this. So it’s something that is not simply using ReLUs for its output, but it might be using Gaussians or sine functions and so.

【译】

事实上我相信，我们的 GPU 比大脑更强大。能在 GPU 上运行的算法集合不受任何限制：原则上你可以施加与生物学中相同的约束——在分辨率、速度、内存大小等等的限度之内。而只要硬件足够快，你可以把任意自然系统模拟到任意保真度。反过来，你的大脑无法模拟一台数字计算机，就是因为它太「糊」了。所以你的大脑被限制在它自己的运作方式里。于是，如果我们自己的意识原来是「大脑为了让复杂的智能运转起来」而产生的 artifact，而计算机不需要同样的 artifact，那么计算机可能拿出的方案，会比我们的意识更聪明、更优雅。——好，我想在这里给一点挑战。我跟 Akarsh Kumar 聊过——他是与 Kenneth Stanley 合著那篇「碎裂纠缠表征」论文的作者。不知道你听说过 Picbreeder 没有，就是他很久以前做的那个东西。基本上它是一种开放式进化搜索，而结论（punchline）是：它在一个神经网络里找到了比普通神经网络好得多的表征。当你改变某些参数时，因为它是在「按关节切分世界」，它会做出语义性的事情——比如骷髅的嘴会张开，你还能让它眨眼之类的。——Picbreeder 里确实有些东西。按我的理解，它用的是一种被专门调教过的神经网络来产生这种表征：它不是简单地用 ReLU 做输出，而是可能用高斯函数或正弦函数等等作为输出函数；它们可以组合出重复的图案，或组合出对称的图案。也就是说，他们从一开始就把所用网络调整成适合做这件事的样子。另一点是，Picbreeder 的训练不是自动的：它是被一群人类评审训练出来的。适应度函数基本上就是人——看着 Picbreeder 的输出说「这是个好苗头，我们应该往这边走」的人。

[2760s - 2880s]

On as the output function, they can be combined to make patterns that repeat, or they can be make a pattern that are symmetric. And so, basically, they tweak the neural network that they’re using from the beginning to be suitable for this. The other one is that the way in which Picbreeder is trained is not automatically; it’s trained with a population of human raters. So the fitness function is basically humans who look at the output of Picbreeder and say, “This is a good start, and this is how we should be proceeding.” And so, I think that this process of the construction of the pictures together with the mechanism that produces the patterns is biasing Picbreeder to make stuff that is more organic, that is more controllable, and so on. But I believe this could be imitated, right? We could automate the procedure by which human beings are selecting pictures by using a diffusion model that is rating the output for similarity to human output. So we could use an AI that is, in principle, already tweaked to imitate the behavior of the human juries that are performing this thing. And we could also evolve architectures that have a larger variety of activation functions that are more suitable for this, and ultimately outperform Picbreeder by this. I think that Sutton’s bitter lesson is also going to be true for this one. So if we basically take the human out of the equation and instead just evolve the search space or search the space of solutions automatically, we are also going to end up with something that’s more elegant than the hand tinkerered solution in the case of Picbreeder. All of that is absolutely true. I mean, yes. So CPPNs they they are sparse, and sparse networks are great. If only we could train sparse networks, right? And I think that we want to have AI which can actually be built, and that you know, so you know, rather than neural networks. Now they’re a bit like a pile of sand; they have too many degrees of freedom, and we kind of like sparsify and we turn them into a sandcastle. And wouldn’t it be better if we had the Lego analogy where we just put little blocks of Lego in there and we have models that carve the world up by the joints? In the case of Picbreeder, we have a.

【译】

所以我认为，「图像的构造过程」加上「产生这些图案的机制」，共同把 Picbreeder 偏置到产出更有机、更可控的东西。但我相信这是可以被模仿的，对吧？我们可以把人类选图的过程自动化：用一个扩散模型来给输出按「与人类产出的相似度」打分。也就是说，用一个原则上已被调成模仿人类评审行为的 AI 来做这件事。我们还可以演化出激活函数种类更丰富、更适合这件事的架构，最终以此超越 Picbreeder。我认为 Sutton 的「苦涩教训」（bitter lesson）在这里同样会成立：如果我们把人从环路里拿出来、改为自动地搜索解空间，我们最终也会得到比 Picbreeder 那种手工调教的方案更优雅的东西。——你说的这些完全成立。是啊，CPPN 是稀疏的，而稀疏网络很棒——要是我们能训练稀疏网络就好了！还有，我们想要的是真正能被造出来的 AI……现在的神经网络有点像一堆沙子：自由度太多；我们喜欢做稀疏化，把它们变成一座沙堡。要是能用乐高的类比该多好：把小乐高块放进去，得到「按关节切分世界」的模型。而 Picbreeder 的情形是……

[2880s - 3000s]

System that is inherently limited because these networks are very small, right? They’re very sparse, and so you have relatively few parameters that you need to optimize. And you set up the model in such a way that most of the parameters that you’re playing with are meaningful. On the other hand, it’s limited because you’re not going to get Picbreeder to produce something that looks arbitrarily complicated. And if you want to move in these realms, then you have to have tools that are much more complex. It’s similar to trying to produce very good user-generated content using Blender versus Minecraft. It’s so with Minecraft, it’s very easy for people to produce meaningful stuff, and it’s almost difficult not to produce something with Minecraft that looks somewhat meaningful that you can play around with and walk around in and have fun in. And if you have a complex three D program, the inverse is true because most of the stuff that you’re doing with this complicated thing is not going to be interesting and is not going to look exciting. But it will look like, oh my god, this person has no ability to use its tools. And so that is one of the issues when we are building learning systems. When we make them very simple, it’s easier to control the complexity, similar as it is in art. And it can be exciting to work with constraints. And if you remove the constraints, that has becomes much more frustrating because you have to suddenly to conquer an enormously large space, and the space might be simply too large for human beings. I mean, assuming you’re right about the convergent representations in in language models, I guess it’s not a foregone conclusion that the consciousness program would be convergent in the same way. It might be divergent. And how far down the stack does consciousness go? I think this is an experimental question. All cultures outside of ours are animist cultures, and animism is the belief that the difference between living and dead systems is that the living systems are possessed by spirits, and spirits are intrinsically agentic and capable of experience. And

【译】

……一个内在受限的系统：因为这些网络非常小、非常稀疏，所以你只需要优化相对少的参数；而且你把模型搭成「你摆弄的大多数参数都是有意义的」样子。另一方面，它也受限：你没法让 Picbreeder 产出看上去任意复杂的东西；你要进入那些领域，就得用复杂得多的工具。这就像用 Blender 还是《我的世界》来产出非常好的用户内容：用《我的世界》，人们非常容易做出有意义的东西——甚至很难不做出某种看起来有意义、可以进去玩、有乐趣的东西。而如果你用的是复杂的三维程序，情况正相反：你用这个复杂玩意儿做出来的大多数东西都不会有趣、不会令人兴奋，只会让人觉得「天哪，这人根本不会用他的工具」。这就是我们构建学习系统时的问题之一：把它们做得非常简单时，复杂度更容易控制，就像在艺术里那样；在约束下工作可以令人兴奋。而一旦移除约束，事情就变得沮丧得多，因为你突然要征服一个大得惊人的空间，而这个空间对人来说可能就是太大了。——假设你关于语言模型里趋同表征的说法是对的，那意识「程序」以同样方式趋同也并非板上钉钉——它可能是发散的。而且，意识沿技术栈往下能走多深？——我觉得这是个实验问题。我们（西方）之外的所有文化都是万物有灵论（animist）文化……

[3000s - 3120s]

The scientific worldview rejects the notion of spirits as unscientific, right? Despite the fact that there is clearly some spirit running on you and me that is agentic and capable of experience, so there is an existence proof for such spirits, right? The question is how does the spirit work? The claim that the spirit is immortal, that it cannot be explained using physics and so on, is not necessarily a claim that is intrinsic to animism. The spirit doesn’t have to be supernatural. Instead, I think that it is natural, and I suspect that Anil Seth’s position of his biological naturalism, by making a claim that biological systems are intrinsically different from any other system that is not biological, is actually a supernaturalism, right? He is a biological supernaturalist. He believes that biological systems cannot be fully naturalized, which means you cannot translate them into mathematical models. Whereas the dead stuff, you can translate them into mathematical models, and so I would say that physics, which is the attempt to translate everything into mathematical models, right, is not stopping at biological systems because the difference with these biological systems is that they’re possessed by software, and the software is self-organizing, and in order to be self-organizing, it’s possible that it needs to be capable of experience, that it needs to monitor itself across multiple dimensions and observe itself in the act of monitoring to make that self-organization happen, and so maybe this pattern has to be discovered in nature to make life possible, at least multicellular life with a certain complexity. It seems to be definitely the case for animal brains. For the others, we don’t know. For the for we don’t know how complex the minds of plants are. We know that plants are able to communicate within themselves and across each other to some degree, but we don’t know how smart they are, and we don’t know whether they are sentient in the sense that they experience themselves in the act of experience. Aristotle saw that.

【译】

……而有灵论相信：活系统与死系统的区别，在于活系统被「精神」（spirits）附体，而精神内在地具有能动性、能够经验。科学世界观拒斥精神观念、认为它不科学，对吧？尽管明明有某个精神运行在你我身上——能动的、能经验的——所以这种精神的存在是有存在证明的。问题只是：这个精神如何运作？「精神不朽」「它无法用物理解释」等等，并不必然是有灵论内在的主张：精神不一定要是超自然的；相反，我认为它是自然的。我怀疑 Anil Seth 的立场——他的「生物自然主义」（biological naturalism），主张生物系统与任何非生物系统有本质不同——实际上是一种超自然主义：他是一个「生物超自然主义者」。他相信生物系统无法被完全自然化——也就是无法被翻译成数学模型——而死的东西可以。而我要说：物理学——也就是把一切翻译成数学模型的尝试——不会在生物系统面前停下脚步。生物系统与死物的区别在于：它们被软件附体，而这个软件是自组织的；为了自组织，它可能需要能够经验、需要跨多个维度监视自身、需要在监视行为中观察自身，才能让自组织发生。所以也许这个模式必须在自然界中被发现，才能让生命——至少是具有某种复杂度的多细胞生命——成为可能。对动物大脑来说显然如此；对其他的我们不知道：我们不知道植物的心智有多复杂。我们知道植物能在其内部、彼此之间进行某种程度的通信，但我们不知道它们有多聪明，也不知道它们是否在「经验自身在经验」的意义上有感知。

[3120s - 3240s]

Plants are relatively dumb. He basically said there are three levels of souls in nature, and he basically said that plants have the lowest one. There’s this level that is vegetative, that is capable of nurturing and of growth, but it does not experience the world around as it doesn’t sense. And so the sentient mind, the sentient soul that is also capable of making decisions and having preferences, that is something that only animals have. And human beings are animals that, in addition, have a rational soul, a rational soul that is able to reflect and make sense of itself and the world in a way in which animals cannot. And I think that’s very much aligned with today’s scientific worldview, which is not surprising because our today’s scientific worldview is very much invented by Aristotle and goes back to him and is an extension of his ideas on the systematization and exploration of his ideas. But we don’t know how smart plants are, right? Maybe Mike Levin is correct in that every organism is, in some sense, sentient, and sentience is something that is universal to the spirits, the self-organizing software nature. And I’m agnostic with respect to that. I have no idea whether that’s the case, right? I don’t know. I don’t have evidence to the contrary. I also don’t have strong evidence in favor. And I think this is still an open question. And to me, it’s exciting to explore this question and surprising that so few people are actually dedicating their life to answering it. Yes, Searle was saying that computation without the correct physical instantiation doesn’t have the same causal pattern, and that actually sounds very similar to what you’re saying. So Searle said, in principle, there could be another physical instantiation which does have the causal pattern that could give rise to consciousness or contain consciousness. So, in a sense, is that so different? The issue with Searle is that there is very little for me to engage. There is a claim, but no argument. And a similar thing is happening with Anil’s point that he has a claim.

【译】

亚里士多德认为植物相对笨。他基本上说，自然界有三层灵魂：植物拥有最低的那层——「营养性」（vegetative）的一层，能营养、能生长，但不经验周围世界、没有感觉。而「有感觉的灵魂」——也能做决定、有偏好的那种——只有动物才有。人类则是这样一种动物：额外拥有「理性的灵魂」，能够反思、能够在动物不能的意义上理解自身与世界。我认为这与今天的科学世界观非常一致——这不奇怪，因为今天的科学世界观很大程度上就是亚里士多德发明的，可以追溯到他，是他关于系统化与探索的观念的延伸。但我们还是不知道植物有多聪明，对吧？也许 Mike Levin 是对的：每个机体在某种意义上都有感知，感知是精神的、自组织软件的自然界所普遍具有的。对此我不可知。我不知道是否如此：我没有相反的证据，也没有强有力的正面证据。我认为这仍是一个开放问题。而让我兴奋的是去探索它；让我惊讶的是，竟只有这么少的人毕生致力于回答它。——Searle 说过，没有正确物理实例化的计算不具有同样的因果模式——这听起来跟你说的很像。他也说，原则上可以存在另一种具有该因果模式的物理实例化，可以产生意识或包含意识。这么说来，（你的观点）与他的有什么不同？——Searle 的问题在于：几乎没有我能与之交锋的东西——有主张，但没有论证。Anil 的观点也发生着同样的事情：他有主张……

[3240s - 3360s]

But there is no argument that I can engage with. There is nothing, no experiment that could convince him to the contrary, because he’s ultimately only working with a certain intuition that he tries to explore more. And so, the way in which I can deal with the arguments of Searle and Anil Seth is ultimately only a psychological one. I would have to look at why does this seem convincing to them, and why doesn’t it seem to be convinced, not convincing to me, right? So we look at the structure of our own minds, and ultimately it comes down to a difference in epistemology. When, how do we construct our models of reality? When do we think something is true? What are the languages in which we are describing reality, and what are the atoms of these languages? And what I observe is that a lot of philosophers, the atoms of their language when they describe themselves, their own hermeneutics, ends at agents. It ends at the experience of something, and similar to Buddhism, where Buddhism is not disassembling the world into automata as computer scientists are doing. Computer scientists basically resolve everything into states that can be described by their transitions, and that’s a very fundamental way of thinking about reality. And it’s tradition that finds itself throughout the philosophy as this attempt to mathematize the world of describing minds as dynamical mathematical objects, and this finds itself in its modern incarnation in computer science, where mathematics is being described from the ground up by the tools that we can make to build arbitrary arithmetic and geometry from simple operations of bit patterns. And so our world is constructed from the ground up from smallest elements that themselves are not agents yet, that are not minds yet, that are not experiences yet. And there is another tradition in philosophy that is basically working more from this introspection and this only the stuff that I can introspect over seems.

【译】

……但没有我能与之交锋的论证。没有任何实验能说服他接受相反的东西，因为他归根结底只是在探索某个他自己的直觉。所以我能处理 Searle 与 Anil Seth 的论证的方式，归根结底只能是心理学式的：我必须去看，为什么这东西对他们显得有说服力，而对我没有。于是我们审视自己心智的结构，而最终这归结为认识论的差异：我们如何构造关于实在的模型？我们何时认为某事为真？我们用什么语言描述实在？这些语言的原子是什么？而我观察到的是：许多哲学家在描述自己、描述他们自己的诠释学时，其语言的原子终结于「能动者」、终结于「对某物的经验」——与佛教类似：佛教并不像计算机科学家那样把世界拆解成自动机（automata）。计算机科学家基本上把一切解析为「可用其状态转移来描述的状态」，这是思考实在的一种非常根本的方式。而这个传统也贯穿于哲学史之中：那种把世界数学化、把心智描述为动力学数学对象的尝试，在计算机科学中找到了它的现代化身——数学从底层被我们制造的工具所描述：用比特模式上的简单运算，构建任意的算术与几何。所以我们的世界，是从最小的元素自底向上构造出来的，而这些元素本身还不是能动者、还不是心智、还不是经验。

[3360s - 3480s]

To be valid, and the stuff that computer scientists are doing in their basement is unrelated to it. And Searle and Anil Seth both don’t see the way in which to connect automata to things that can feel, that can reason, that can make models of reality. And computer scientists see such a path. I think that’s the main difference. And we, but we can create evolution in computers, right? We can create the conditions for software that is not written by us. And that’s already true for the models that we are building with transformers. They’re not written by human beings. They’re statistical aggregates that are woven, that are grown, and in many ways, the computer models that we’re playing with today, they’re they’re result of growth and they’re pruned and they are gardened and trepawoned and trained in in a way that is much more in which we observe this happening in biology than it happened when we were writing computer programs by hand. Yeah, and there was a beautiful example of this. I read a less wrong article. I’m not sure whether you read this about the LLM delusions. And you know, last time again, I think we were talking about this this idea that we can be parasitized by software spirits. And many folks started joining Reddit groups and talking about this LLM delusion. And what was fascinating was how convergent it was and how convinced that these folks were that they had just discovered the meaning of the universe. And there were certain characteristic traits. They would talk about spirals and recursion, and there were certain motifs in in all of the LLM outputs. And it was actually copying itself. Isn’t that an example of this memeplex that you’re talking about? That’s also an interesting observation about human minds. During the pandemic, we observed that a lot of people, when they were starting to work from home, became dysfunctional. And it seems to be that many people can only be stabilized due to their environment. That they do require an environment that is constantly affirming their reality. And if you take that.

【译】

（只把「我能内省到的东西」看作）才有效，而计算机科学家在地下室里做的那些与之无关。Searle 和 Anil Seth 都看不到把自动机连接到「能感受、能推理、能给实在建模之物」的路径，而计算机科学家看得到这样一条路——我认为这是主要差别。而我们确实可以在计算机里创造演化，对吧？我们能够为「不是由我们书写的软件」创造产生的条件。这对我们用 transformer 构建的模型已经成立：它们不是人类写出来的。它们是编织、生长出来的统计聚合体；而且在很多方面，我们今天在玩的这些计算机模型是生长的结果——它们被修剪（pruned）、被像园丁一样打理（gardened）、被「环钻」（trepawoned，转写疑作 trepanned）、被训练——这一切比我们当年手写计算机程序的方式，更像我们在生物学中观察到的过程。——是啊，这方面有个漂亮的例子。我读过一篇 LessWrong 上的文章，不知道你读过没有，讲 LLM 妄想（LLM delusions）的。你知道，上次我们好像谈过这个想法：我们可能被「软件灵魂」寄生。很多人开始加入 Reddit 小组、谈论这种 LLM 妄想，而迷人的是它们的趋同性，以及这些人对自己「刚刚发现了宇宙的意义」的笃信程度；还有一些特征性的迹象：他们会谈螺旋、谈递归，所有这些 LLM 的输出里都有某些反复出现的母题。而且它其实在自己复制自己。这难道不就是你说的那个 memeplex（模因复合体）的例子吗？——这也是一个关于人类心智的有趣观察。疫情期间我们观察到：很多人开始居家办公之后变得功能失调（dysfunctional）。看起来，许多人只能靠他们的环境来稳定自身——他们确实需要一个不断肯定其现实的环境。如果你把这个环境拿走……

[3480s - 3600s]

They can go off the rails, and if you have an LLM that is interacting with them, then there can be an interaction between the LLM and the human being that pulls the human being off the rails if they don’t have a strong foundation in their own reality. And I think that the LLMs have now reached a point where they’re mathematically more powerful than many human minds are. There are things that the LLM cannot do at this point, where they’re still hallucinating, where they’re still more incoherent than experts in a given field. But they are much better at prompt completion than human beings are, and many human beings are professional prompt completers where they’re incapable of having a single original creative thought. Instead, they are trying to serve expectations of the environment. They are picking up on the statistical patterns in the language of the environment, and then you tell them a headline, and they write an article. And now you give them an LLM. How is the environment going to change? Right. Once they talk more to LLMs than they talk to other human beings, because the LLM is such a useful tool for them to do their prompt completion, what’s going to happen to them? It’s a very interesting thing. What happens to us, weak-minded human beings, when we are hypnotized by things that are more hypnotic than our mind is to itself? What’s your take on first of all? What is physicalism? Are you a physicalist? I think physicalism is a certain hypothesis, and it comes down to that being a causally closed mechanical layer to reality that can be researched using experiments in physics labs, and that this causally closed mechanical layer of reality is not moved by our intentions. Right. So you cannot use an informal incantation of saying, “Please, sun rise,” and the sun suddenly rises. This is something that is not happening in this causal layer because it’s mechanical. It does not care about what you want. And the second part to physicalism is that we are directly supervening over this mechanical, causally closed world.

【译】

……他们就可能脱轨。而如果有一个 LLM 在与他们互动，那么 LLM 与人之间就可能发生一种把这个人拉下轨道的互动——如果这个人对自己的现实没有稳固的根基的话。我认为 LLM 现在已经到了这样一个点：它们在数学上比许多人类心智更强大。当然，有些事 LLM 目前还做不到，它们仍有幻觉，在特定领域仍比专家更不连贯。但它们做「提示补全」（prompt completion）比人类强得多，而许多人类本来就是职业的提示补全者：他们产生不了一个原创的、创造性的想法，而只是在迎合环境的期待——抓取环境语言的统计模式，然后你给他们一个标题，他们就写出一篇文章。而现在你给了他们一个 LLM。环境将会如何改变？当他们与 LLM 交谈多过与其他人交谈——因为 LLM 对他们做提示补全是如此有用的工具——他们会变成什么样？这是个很有意思的问题：当我们这些「弱心智」的人类，被比我们自己的心智更催眠的东西催眠时，会发生什么？——先问第一个问题：什么是物理主义（physicalism）？你是物理主义者吗？——我认为物理主义是一个特定的假说，它归结为两点。其一：存在一个因果封闭的机械层，可以在物理实验室里用实验研究，而且这个因果封闭的机械实在层不被我们的意图所推动。也就是说，你不能用一句非正式的咒语说「太阳啊，升起来吧」，太阳就突然升起。在这个因果层里这种事不会发生，因为它是机械的，它不在乎你想要什么。其二：我们直接随附（supervene）于这个机械的、因果封闭的世界之上……

[3600s - 3720s]

So there is not some parent universe that contains a server where a nerd is running a simulated universe that is some kind of Minecraft, and we live inside of this Minecraft. So it’s not some simulation hypothesis, and so physicalism is these two components. But to me, physics is not something that has revealed itself directly. So nobody has ever seen matter and energy directly. Instead, matter and energy are attempts to systematize observations, and our observations ultimately all come down to discernible differences. And the technical term for discernible differences is information. So to me, information is fundamental when we talk about reality. Reality is about differences in information, and physics is a certain layer of regularity in the differences in information that we have discovered. So to me, matter is not at the bottom. The bottom is really all these differences that we observed and which we experimentally and practically empirically found to be so regular that that they can be translated into this mechanical world of physics. And then, up top of it, we find at some point the organization into organisms, and the organisms are producing minds, and the minds are simulating us and simulating our sense of realness, of our sense of being in a world with colors and sounds and people and so on. And in this world, intentions matter. This psychological reality has things like will and emotion and despair and desire and all these things that make us tick. And magic, in some sense, is the idea that it’s possible to influence this physical universe outside of minds using incantations, using magical rituals. That somehow it’s possible that you burning a candle has an influence on celestial events and so on. And that is something that would contradict physics, right? It would mean that we actually do live in some kind of simulation that works similar to Minecraft, where you can say time set day and the sun rises.

【译】

……也就是说，并不存在某个母宇宙，里面有一台服务器、一个宅男正在跑一个模拟宇宙——某种《我的世界》——而我们活在这个《我的世界》里面。所以不是那种模拟假说。物理主义就是这两个组分。但对我来说，物理并不是某种直接向人显现自己的东西：从来没有人直接看见过物质与能量。物质与能量是对观察进行系统化的尝试，而我们的观察最终都归结为「可辨的差异」（discernible differences），可辨差异的技术术语就是信息。所以当我们谈论实在时，信息才是根本的：实在关乎信息中的差异，而物理学是我们在信息差异中发现的某一个规律性层次。所以对我来说，物质不在最底层；最底层是所有这些我们观察到的差异——它们在实验上、实践上、经验上被发现得如此规律，以至于可以被翻译成「物理学」这个机械的世界。再往上，我们在某一点上发现了组织成机体（organisms）的层次；机体产生心智；而心智在模拟我们、模拟我们对真实感的感知——模拟我们那种身处一个有颜色、有声音、有他人的世界之中的感觉。而在这个（心理）世界里，意图是有分量的：这个心理实在里有意志、情绪、绝望、欲望，有所有这些驱动我们的东西。而「魔法」，在某种意义上，就是这样一个观念：可以用咒语、用魔法仪式去影响心智之外的这个物理宇宙——仿佛燃烧一支蜡烛就能影响天体事件。而那是会与物理学矛盾的东西，对吧？它将意味着我们其实活在某种类似《我的世界》的模拟里——在那里你可以说一句「time set day」，太阳就升起来了……

[3720s - 3840s]

Despite what else happens in the programming code, because Minecraft is symbolic, right? It’s not physical. It’s something that is only made out of incantations. Would you think it’s possible for a physicalist to subscribe to your ideas and render it compatible, you know, compatible with physicalism? I would say that I am a physicalist in a certain sense, right? I would say that the theory that the world has a causally closed mechanical layer, right, some kind of mathematics that is so self-evident that exists by itself, that’s some kind of logos, the natural laws over which all the structure in the universe emerges, that makes sense. It’s a good theory, and it makes good predictions that the stuff that we observe in the lab holds true across all the labs, and not just because the physicists all believe in it. It is something that seems to be working. There are also some regularities, like casinos seem to be working. It’s not that people via having a belief in rituals are able to systematically break the casinos. So certain magic doesn’t seem to be working. All the magic that seems to be working does rely on manipulating other minds. But it’s possible that there are minds outside of the human minds, right? Maybe there are minds that emerge across all the minds in biology. Maybe there’s something like a biological internet that can be influenced using rituals, right? Maybe it’s possible to influence what a forest wants at some level. I don’t know that. I cannot exclude it from possibility because there’s so much information processing going on across organisms that I don’t know at which levels in nature information is integrated and at which point it becomes intentional. So to me, physicalism is nothing that is incompatible with animism, right? There is a version of animism that is entirely physicalist, and I call this position cyberanimism. It basically says that we can explain animism through information processing in nature, and nature can be understood as something that can be entirely modeled using mathematics without intentions, including minds. Okay, so your.

【译】

……不顾程序代码里发生的其他一切。因为《我的世界》是符号性的，对吧？它不是物理的，它是仅由「咒语」构成的东西。——你觉得一个物理主义者有没有可能认同你的观点、把它变得与物理主义相容？——我会说，我在某种意义上就是一个物理主义者。我会说，「世界存在一个因果封闭的机械层」这个理论——某种如此自明、自身即存在的数学，某种 logos（逻各斯）、自然法则，宇宙的一切结构都在其上涌现——是讲得通的。它是个好理论，预测良好：我们在实验室观察到的东西在所有实验室都成立，而不只是因为物理学家们都相信它。还有一些规律性，比如赌场似乎一直在正常运作——人们并不会通过对仪式的信仰系统性地攻破赌场。所以某些魔法不奏效。而所有看似奏效的魔法，确实都依赖于操纵其他心智。但也可能存在人类心智之外的心智，对吧？也许有横跨生物界所有心智而涌现的心智；也许存在某种「生物互联网」，可以用仪式去影响；也许在某个层级上可以影响「一片森林想要什么」。我不知道，我不能把它从可能性中排除，因为生物体之间有那么多我不了然的信息加工——我不知道自然界在哪些层级上整合信息、又在哪一点上它变得有意图。所以对我来说，物理主义与万物有灵论之间没有任何不相容：存在一个完全物理主义版本的有灵论，我把这个立场称为「赛博万物有灵论」（cyberanimism）。它基本是说：我们可以通过自然界中的信息加工来解释万物有灵论；而自然——包括心智在内——可以被理解为某种完全能用无意图的数学来建模的东西。——好，那么你……

[3840s - 3960s]

So you’re you’re a functionalist, which means that you think it’s not about the internal constitution; it’s about the role that it plays in the system in terms of building minds. So minds can can exist in any substrate. And I’m not sure whether you read this paper by Thagard in 2022, and he made this quite interesting argument that energy requirements undermine substrate independence. Do you think that that might refute functionalism? No, I think that he is making the claim that sometimes the substrate has properties that influence its function. But this is exactly what functionalism is saying. To me, functionalism is not the claim that computers can think. It’s a claim about how we construct objects. It’s an epistemological claim. When we are talking about a thing like matter, we are not talking about something that reveals itself to a wise person in their dreams. We are talking about a systematization of observable observables, and observables are discernible differences. There are ways in which reality changes. So basically, to a functionalist, an object is not some kind of hidden essence that reveals itself to you. Instead, the object is a construct, and this construct has been made by a mind, and the mind has constructed it to explain or to make the claim that when this object is present, the evolution of the universe is going to change in this particular way in its environment, right? So when an object like you or a cup or a bottle or an atom is present, its the claim is the universe is going to behave slightly differently around this object than when this object was not there, right? And so in this way, the object is a function. The object is describing a mapping from states to other states, and right is an influence on the evolution of the universe. So functionalism, to me, is simply the recognition of how we can construct objects. Let’s take an example. Imagine there is a water molecule, and this water molecule internally.

【译】

……你是一个功能主义者（functionalist），也就是说，你认为关键不在内部构成，而在它在构建心智的系统中所扮演的角色，所以心智可以存在于任何基底之上。我不确定你读过 Thagard 2022 年那篇论文没有——他提出了一个挺有意思的论证：能量需求会削弱基底独立性（substrate independence）。你觉得那会不会反驳功能主义？——不会。我认为他提出的主张是：有时基底具有会影响其功能的属性。但这恰恰就是功能主义所说的话。对我来说，功能主义不是「计算机能思考」这个主张；它是关于我们如何构造对象的主张，是一个认识论主张。当我们谈论「物质」这种东西时，我们谈的不是某种在智者梦中向其显现自身的东西；我们谈的是对可观察项（observables）的系统化，而可观察项就是可辨的差异，是实在发生变化的方式。所以对功能主义者来说，一个对象不是什么向你显现的隐藏本质；对象是一个构造物（construct）。这个构造物是由心智造出来的，而心智构造它，是为了做出这样一个主张：当这个对象在场时，宇宙的演化将在其环境中以这种特定方式改变。也就是说，当一个对象——比如你、一个杯子、一个瓶子或一个原子——在场时，主张就是：宇宙在这个对象周围的行为，会与它不在场时略有不同。以这种方式，对象就是一个函数：对象描述的是从状态到状态的映射，是对宇宙演化的一个影响。所以功能主义对我来说，只是对我们「如何构造对象」的一种认识。举个例子。想象有一个水分子，而这个水分子在内部……

[3960s - 4080s]

Doesn’t have the proper essence of a water molecule. It just behaves like a water molecule in every regard, right? So it’s you can split it into something that totally looks like oxygen and hydrogen, and it has the same temperature at which it evaporates. It’s connecting to other water molecules in the same way, and so on. You can freeze it into ice in the same way. Only internally, it’s different, right? To a functionalist, this notion doesn’t make sense because all the other water molecules have been constructed in the same way. For something to be a water molecule, there is no other claim than to say what it behaves like, because we can never recognize its essence. We can only say there’s a certain set of observables that are invariant here, and this invariance is what we call a water molecule. It’s only that, and so in the same way, we would say a mind is an invariance that exists on your brain. Right, something is characteristic over the way in which your brain functions in a certain area, and when we change this, for instance, by increasing your body temperature, at some point your mind is going to break down because you’re going to be delirious, right? And so if we bring this back to this argument, saying okay, if we change the energetic conditions around you by increasing your body temperature, of course it’s going to influence your mind to some degree because at some point it’s no longer able to maintain its function. But there’s always this counterargument that a simulation of water doesn’t make you wet. But would the functionalist reply that even the feeling of wetness is just a behavior like any other? It’s an interesting thing when you are playing Minecraft. There is water in Minecraft, right? And when you are on fire and you get into the water in Minecraft, the fire is going to get extinguished because you’re wet now, right? So at some point, somebody made a simulated Minecraft that was using a neural network to learn Minecraft. Right, the neural network was only trained in the visuals of Minecraft, and so you could go and play the simulated Minecraft despite there not being any Minecraft source code below it. And so I thought I’m going to make it.

【译】

……并没有「真正的水分子本质」，它只是在方方面面都表现得像一个水分子，对吧？你可以把它分解成看起来完全像氧和氢的东西；它有同样的蒸发温度；以同样的方式与其他水分子连接，等等；你同样能把它冻成冰。只是内部不同。对功能主义者来说，这个概念没有意义，因为所有其他水分子也是以同样的方式构造出来的。「是」一个水分子，除了说「它表现得像什么」之外，没有别的主张可做——因为我们永远无法认出它的本质。我们只能说：这里存在一组不变的可观察项，而这个不变性（invariance）就是我们所说的水分子。仅此而已。同样地，我们会说：心智是你大脑上存在的一个不变性——某种在你大脑于某个区域运作方式上具有特征性的东西。当我们改变它——比如升高你的体温——到某个点，你的心智就会崩溃，因为你会进入谵妄。所以回到 Thagard 的论证：如果我们通过升高你的体温来改变你周围的能量条件，这当然会在某种程度上影响你的心智，因为到某个点它不再能维持其功能。但针对这一点，总有一个反方论证，那就是：「模拟的水不能把你弄湿。」

[4080s - 4200s]

Joke about how simulated water and simulated Minecraft can not make you wet, and I looked it up and tried it out, and I found actually it was true, because when people try to get into the water in the simulated Minecraft, the water would recede to you; you could not immerse into it because the neural network hadn’t learned it. So accidentally, it was true that the simulated water and simulated Minecraft couldn’t make you wet. But of course, in the real Minecraft, the real Minecraft player, the character that you are playing in Minecraft, this avatar, can get wet. Of course, you sitting outside of Minecraft are not going to get wet because you are not an inside of the Minecraft; you are outside of the Minecraft. And of course, something that is outside of this universe is not going to get wet by anything inside of this universe, right? Wetness is a property that is constrained to a universe that has systems that have the function of making you wet. If this function is being implemented, like in many computer games, you can get wet for some definition of wet, right? Then, then it’s fine, and you can go to an arbitrary degree of physical simulation to make something wet to satisfy whatever you want to call wetness. So, I don’t understand the argument that simulated water makes you wet. I don’t understand why there is such a strong camp of simulated water doesn’t make you wet in philosophy, and I think it has social reasons, not intellectual reasons, that this is the case. So, in your paper, you had this beautiful history of computationalism, and just as a teaser, you said computationalism might be best understood as a variant of constructivism. But we’ll get there just in a second. And you spoke about the history of Russell Whitehead trying to ground mathematics in this formalist program where there is no divergence between syntax and semantics. Yes, and maybe just start from that. So it’s really interesting that mathematics couldn’t be understood as a way to systematize thought beyond the human mind. It’s so in a way, for instance, in music, you have notations for sounds and music that allow you to talk to other musicians about.

【译】

——但功能主义者会不会回应说：连「湿」这种感觉本身也不过是一种行为，和其他任何行为一样？——玩《我的世界》时有个有趣的事情：《我的世界》里有水，对吧？当你着火时跳进水里，火会被扑灭，因为你现在「湿」了，对吧？后来有人做了一个「模拟版《我的世界》」，用一个神经网络来学习《我的世界》——这个网络只在《我的世界》的视觉画面上训练过，所以你可以进去玩这个「被模拟的《我的世界》」，尽管它下面没有任何《我的世界》的源代码。于是我本来想开个玩笑，说「模拟的水和模拟的《我的世界》不能把你弄湿」；结果我查了一下、试了一下，发现这居然是真的：当人们试图在「被模拟的《我的世界》」里进入水中时，水会从你身边退开，你无法浸入——因为神经网络没学会这个。于是「模拟的水不能把你弄湿」碰巧成了真话。当然，在真正的《我的世界》里，你扮演的那个角色——那个 avatar——是可以变湿的。而你，坐在《我的世界》外面的你，不会变湿，因为你不在这个《我的世界》的内部，你在它外面。而处在这个宇宙外面的东西，当然也不会被这个宇宙里面的任何东西弄湿，对吧？「湿」是一种被限制在如下宇宙中的属性：那个宇宙里存在承担「把你弄湿」这一功能的系统。只要这个功能被实现了——在很多电脑游戏里都是这样——你就可以「在某种定义下变湿」，那就没问题；而且你可以做到任意程度的物理模拟，让某个东西「湿」到满足你想称之为「湿」的任何定义。所以我不理解「模拟的水会把你弄湿」这一论证（的力度），我也不理解为什么哲学界里「模拟的水不能弄湿你」的阵营如此强大。我认为这件事有社会原因，而非智识原因。——你在论文里有一段很美的计算主义（computationalism）史，而且你埋了个引子，说「计算主义也许最好被理解为建构主义（constructivism）的一种变体」。你还谈到 Russell 与 Whitehead 试图把数学安放进这个形式主义纲领的历史——在那个纲领里，句法（syntax）与语义（semantics）之间没有分裂。要不就从那里开始？——是的。非常有趣的是，数学竟能被理解为一种超越人类心智、对思维加以系统化的方式（此句转写存疑，原文作 couldn’t）。比如在音乐里，你有一种声音与音乐的记谱法，让你能与其他乐手交流……

[4200s - 4320s]

What they should be playing, and it’s something that is trying to capture the essence of the music in this notation. And mathematicians were trying to find a notation for the essence of when you have mathematical thoughts, and they expanded this beyond the human mind because you can write down things in mathematics that are larger than what you can fit in your own imagination. At some point, mathematicians discovered that geometry can be expressed using algebra, so they’re basically formalism that produced geometry, and this all has been rediscovered by computer scientists. That’s why computer games can have geometry, right? So the interesting thing in computer science is that you can build the entirety of the mathematics and computer algebra systems using simple operations over automata, and so there is a nice thing in a computer that is never a divergence between syntax and semantics, because your computer is always going to be in the state that its syntax prescribes, and that’s its semantics. Right? There is no difference. Maybe the computer program doesn’t do what you wanted to do, but it’s always doing what it’s doing. And in mathematics, there has been a divergence between what the mathematicians wanted to express and what the syntactic operations are doing, and I think that’s because this programming language for thought that mathematicians have developed has bugs in it. And what Gödel has discovered is that these bugs cannot be eliminated, except by redefining the programming language of mathematics. And I think the bug lies in axiomatic mathematics. So the way in which mathematicians define objects is that they simply postulate their existence as an axiom, and the axiom can be a black box, and you just write on the black box what the black box is supposed to contain, and as long as this doesn’t lead to contradictions with the other black boxes that you have defined, you’re good. And there were some very counterintuitive results that happened from the way in which mathematicians defined arithmetic, and they typically do this.

【译】

……他们该演奏什么；这种记谱试图捕捉音乐的本质。而数学家们试图为「产生数学思想」的本质找到一种记谱法，并且他们把它扩展到了人类心智之外——因为在数学里，你能写下比你自己想象力所能容纳的还要大的东西。到某个时候，数学家发现几何可以用代数来表达——所以基本上是形式主义产生了几何；而这一切又被计算机科学家重新发现：这就是为什么电脑游戏可以有几何。计算机科学里有意思的地方在于：你可以用自动机上的简单运算，构建整个数学与计算机代数系统；于是在计算机里，句法与语义之间永远不会出现分裂，因为你的计算机永远处在它的句法所规定的状态里——而那就是它的语义。二者没有差别。也许计算机程序没做你想要它做的事，但它永远在做「它正在做的事」。而在数学中，数学家想表达的东西与句法操作所做的事之间出现了分裂。我认为，这是因为数学家们开发出来的这套「思维的编程语言」里有 bug。而 Gödel 发现的是：这些 bug 无法被消除——除非重新定义数学的编程语言。

[4320s - 4440s]

By a set theory, and all these issues led Hilbert to saying, “Please, mathematicians, there seems to be something wrong here. Can’t you fix this?” And so Gödel had this idea of trying to prove properties of his language by defining the language within itself. The problem is that the language cannot point outside of itself. It’s something that semiotics doesn’t always get. Right? Many semiotists try to draw arrows into reality outside of the language, but this doesn’t really work. You can only point from a model to within it, within its own language. And so, if you want to make proper discuss properties of the language, and the language itself is defined outside of itself, what can you do? And Gödel had this brilliant idea of recreating the language within itself by emulating it. And in this way, he could show that certain contracts constructs cannot be avoided. That you can create self-referential statements that contradict themselves. And he could not avoid this. And the reason why he could not avoid this is because mathematics, in the way in which Gödel understood it, doesn’t have state. It’s basically eternal. It doesn’t go from step to step. Everything is simultaneous. And this property of the simultaneity, the statelessness in mathematics, is very useful because it allows you to have infinities. If you can only go from step to step, and you want to build your infinity stepwise, you’re never going to get there, right? Maybe you have unboundedness, something that never ends, but you’re never going to have a full infinity. And mathematicians have got used to these infinities and continua and so on. Right? Physics has even has used them since mathematicians have given it to them. It would be really bad if you didn’t have them. And Gödel basically found that there is no way to fix this. There is no way to fix mathematics without allowing certain contradictions. And Russell and Whitehead in Principia Mathematica were attempting to do this by limiting set theory in such a way that sets had to be at the same level, and you could not.

【译】

我认为这个 bug 位于公理化数学之中。数学家定义对象的方式，是直接用公设（axiom）断言其存在；公理可以是一个黑箱——你只要在黑箱上写清楚这个黑箱应该装什么，只要它不与你定义的其他黑箱产生矛盾，就万事大吉。而以这种方式定义算术，产生了一些非常反直觉的结果；他们通常是通过集合论来做这件事的。所有这些问题让 Hilbert 说：「拜托，数学家们，这里似乎有点不对劲，你们能不能修一修？」于是 Gödel 有了一个想法：通过在语言自身内部定义这门语言，来证明语言的性质。问题在于：语言无法指向它自身之外。这一点符号学（semiotics）并不总是明白——许多符号学者试图从语言画一支箭头指向语言之外的实在，但这行不通：你只能从一个模型、用它自己的语言，指向它内部的东西。那么，如果你想恰当地讨论语言的性质、而语言本身又定义在它自身之外，你能怎么办？Gödel 有个天才的想法：通过模拟，在语言内部重建这门语言。以这种方式，他可以证明某些构造无法避免——你可以制造出自相矛盾的自指陈述，而他无法避免这一点。他无法避免的原因是：数学——在 Gödel 所理解的意义上——没有「状态」。它基本上是永恒的，不从一步走到另一步，一切都是同时的。而这种同时性、这种数学中的无状态，非常有用，因为它让你能拥有无穷（infinities）。如果你只能一步一步走，又想步进式地构建你的无穷，你永远到不了那里，对吧？你也许有「无界性」（unboundedness）——某种永不结束的东西——但你永远不会有完整的无穷。

[4440s - 4560s]

Not simply integrate them over each other to create loops, and I think this was a limitation that was too strong because we think like this. We are creating constructs of the type that Russell and Whitehead were excluding from us. Also, there, there was basically showing that Russell and Whitehead’s attempt was insufficient to eliminate the contradictions in classical mathematics. But there is a way to eliminate the contradictions, and the way is to do what computer scientists are doing. So instead of using classical mathematics, you only use computer algebra systems, systems that from smaller elements, from simple automata, are creating all the constructs that you want, and you can have stuff that looks as continuous as you want in a computer game, right? You will never have a perfect circle, but you will have something that is as circular as you want it, as you can afford it. So you can always create as much resolution as you can afford, but you will not have a universe that is truly infinite, that is truly continuous. And this is this constructivist program that basically I think in classical mathematics the first formulation was Brouwer’s intuitionism, and I don’t think that he formulated it very well, but I think what he could see, what anticipate, was the need to build mathematics from elements that you actually understand instead of from from axioms, from things that you that you don’t have any black boxes, but everything is built from the ground up. So in the world of the computer, everything is built from the ground up, and there is a reason why you should think that a generalization of thought and of physics should also be like this, because for our mind to represent things, our brain needs to build the representations from the ground up, from communications between neurons. So when you see a continuous line, something in your brain is producing this continuous line using communications between neurons, which means there is some kind of computation going on, and the idea that there is something that is not made step by step between the

【译】

数学家们已经习惯了这些无穷与连续统，等等；物理学自从数学家把无穷给了他们之后也一直在用。如果没了它们，事情会非常糟糕。而 Gödel 基本上发现：这没法修复——不引入某些矛盾，就没法修复数学。Russell 与 Whitehead 在《数学原理》（Principia Mathematica）里尝试过另一种做法：限制集合论，让集合必须处在同一层级，这样你就不能……不能简单地把它们互相嵌套来制造循环。我认为这个限制太强了，因为我们恰恰就在这样思考：我们就在创造 Russell 与 Whitehead 试图从我们这里排除掉的那种构造物。而且（Gödel 的工作）基本上表明，Russell 与 Whitehead 的尝试并不足以消除经典数学中的矛盾。但确实存在一条消除矛盾的路，那就是做计算机科学家正在做的事：不用经典数学，只用计算机代数系统——从更小的元素、从简单的自动机出发，构建你想要的一切构造物。你可以拥有你想要的多「连续」的东西，就像在电脑游戏里那样：你永远不会有完美的圆，但你可以有随你想要、随你负担得起的那样圆的东西。你永远可以造出你负担得起的分辨率，但你不会拥有一个真正无穷、真正连续的宇宙。而这就是建构主义的纲领。它在经典数学中的第一次表述，我认为是 Brouwer 的直觉主义（intuitionism）；我不认为他表述得很好，但我认为他所看到的、所预见的，是「从你真正理解的元素出发构建数学」的必要性——而不是从公理、从那些只能当黑箱的东西出发；一切都从底层建起。而在计算机的世界里，一切都是从底层建起的。而且你有理由认为，对思维与物理学的一般化也应当如此：因为我们的大脑要表征事物，就必须从神经元之间的通信、从底层把表征构建出来。所以当你看到一条连续的线，你脑中有某种东西正在用神经元之间的通信产生这条连续的线——这意味着存在某种计算在发生。而「在这条线的神经元通信之间存在某种并非逐步造成之物」的观念，多半是一个错误的直觉。

[4560s - 4680s]

Communications between the neurons is probably a faulty intuition. Where do these things come from originally? You know, so how did the program of consciousness get bootstrapped with this computation idea? You mean in the physical universe? Yes. So of course I cannot know because I don’t see the universe from the outside. We have to think about possibilities. My current intuition is that the universe exists because nothing prevents its existence. Right. We might have this intuition that for something to exist rather than nothing, there needs to be something positive. But if nothing exists before the ground truth, before the ground level of reality, then there is also nothing that prevents the universe from existing. Right. So if the universe is not forced to exist or prevented from existing by something that is preceding it, then the universe is not impossible. Then it’s probably undecided about its existence. Right. So it’s going to have an existing branch and a non-existing branch. The non-existing branch doesn’t have any observers that complain about non-existence, but the existing branch is going to have observers that might complain about existence. Now we need to explain within the existing branch why some things are happening and other things are not happening. Right. And the easiest explanation is that basically everything is happening, and what’s happening is the superposition of all automata. So the universe at every point is branching out in all possibilities, and these possibilities are basically can be understood as operations on the previous state. They’re also operations which don’t change the previous state; they just leave it as it is. So unary operations, but they’re not going to lead to an evolving universe. The branch in which we are going to be is one that is evolving. Right. And the stuff that we can evolve in is stuff that retains memories of previous states. So we find ourselves limited to a subset of the possible mathematics over which universes can emerge, and these are these ones that are branching.

【译】

——那这些东西最初是从哪里来的？就是说，「意识」这套程序是怎么跟「计算」这个观念一起被 bootstrap 起来的？——你是指在物理宇宙里吗？——对。——那我当然不可能知道，因为我不是从外面看宇宙的。我们只能思考各种可能性。我目前的直觉是：宇宙存在，是因为没有什么阻止它存在。我们可能有一种直觉：要「有物存在」而非「绝对虚无」，必须有某种正面的东西。但如果在 ground truth 之前、在实在的底层之前什么都没有，那么同样也没有任何东西阻止宇宙存在，对吧？如果宇宙的存在不被任何先于它的东西所强迫或阻止，那么宇宙「是否存在」这件事就是未定的。于是它会有一个「存在的分支」和一个「不存在的分支」。不存在的分支里，没有任何观察者来抱怨不存在；而存在的分支里会有观察者——他们倒可能会抱怨存在。现在，我们需要在「存在的分支」内部解释：为什么有些事情发生、有些事情不发生。而最省事的解释是：基本上一切都在发生；正在发生的，是所有自动机的叠加（superposition）。宇宙在每一个点上，向所有的可能性分叉；这些可能性可以理解为「对先前状态的操作」。也存在不改变先前状态的操作——它们只是让状态保持原样，也就是恒等操作；但它们不会导向一个演化的宇宙。而我们将会身处其中的分支，是一个在演化的分支。而我们能在其中演化的东西，是能保留先前状态之记忆的东西。于是我们发现，自身被限制在「宇宙可以从中涌现的可能数学」的一个子集之中：那些能够分叉、又能保留记忆的数学。

[4680s - 4800s]

Out over possibilities that can retain memory. Now, what are the things that are happening when the universe branches out in an unbounded number of possible branches at every step? Where do you find yourself? Well, everywhere, right? In every step, you’re going to find yourself in multiple branches, but you don’t know about these other branches because you only have memories of the past, not of the stuff that happens simultaneously in the parallel branches. And if you want to do something in this universe, for instance, if you want to build structure for yourself because you want to become an organism, you need to be able to make guesses about what other parts of the universe are doing. But everything is changing all the time because you’re always splitting up in all these branches. So you look into your future and you only see fog, right? Because everything is randomly branching out. And so there must be some things that are stable. And it turns out under some conditions, error correcting patterns emerge. For instance, you sit in a bathtub, you move your hand, and you see that all sorts of waves emerge on the surface of the bathtub when you do this. And these waves are propagating over the surface of the water. Most of them just dissipate, but some of them stay around. These circular vortices. And why do these circular vortices have such sustained power? It’s not because there is a higher probability that they emerge than the other waves. It’s just the only thing that’s left over because in the circular vortices, the information gets bent back into itself, so it does not leave a certain region of space in the water, right? And so this little vortex is a pattern that has the ability to error correct itself. And such error correcting patterns have the ability to stay until friction kills them. At the ground level of the universe, these error correcting patterns, the particles don’t encounter any friction, right? Because there’s no friction yet. The only thing that they encounter is sometimes other particles that they bump into. So with a certain probability, once they emerge, they’re going to stay around until they either bump into another particle or the conditions for their non existence are met because the error correction is not strong enough. And then.

【译】

那么，当宇宙在每一步向无界数量的可能分支分叉时，都发生些什么？你会在哪里发现自己？嗯——到处都在，对吧？在每一步，你都会发现自己处在多个分支里；但你不知道其他分支的存在，因为你只拥有关于过去的记忆，而不知道平行分支里同时发生的事。而如果你想在这个宇宙里做成什么事——比如你想为自己构建结构、想成为一个有机体——你需要能够猜测宇宙的其他部分在做什么。但一切都在不停变化，因为你总是在向所有这些分支分裂。所以你望向你的未来，只看见雾，对吧？因为一切都在随机地分叉。于是必须存在一些稳定的东西。而事实证明，在一定条件下，纠错模式（error-correcting patterns）会涌现。比如你坐在浴缸里，动一动你的手，你会看到水面涌现出各种各样的波；这些波在水面传播，大多数直接耗散掉，但有一些会留下来——那些环形的涡旋。为什么这些环形涡旋有这么持续的力量？并不是因为它们出现的概率比别的波高，而仅仅因为它们是最后剩下的东西：在环形涡旋里，信息被弯回它自身，不会离开水中那一特定区域，对吧？于是这个小涡旋，就是一个有能力自我纠错的模式。而这样的纠错模式有能力一直待下去，直到摩擦把它们杀死。而在宇宙的最底层，这些纠错模式——粒子——不会遇到任何摩擦，因为那时候还没有摩擦这回事；它们唯一会遇到的，是偶尔撞上另一个粒子。所以一旦涌现，它们就以某个概率留存下去，直到要么撞上另一个粒子，要么「其纠错能力不够强、使其无法存在」的条件得到满足。

[4800s - 4920s]

On the next level, you have interactions between these vortices, between the particles, and they form molecules. At some point, they form cells, which are super molecules. But cells have now something else; they cannot just stay around by error correction. There are patterns that are able to basically dig a groove deeper into the universe. They can self-replicate, and in this way, they can also deal with their own dissolution. They’re able to recreate themselves, and in this way, make themselves have more. And so, evolution is about life, about patterns that are not simply replicating themselves but deepening themselves, that imprint themselves more in the world by controlling it, by being able to overcome the noise, the entropy, the way in which the universe goes out of sync with your expectations and with your preferences. And so, consciousness, I think, emerges as a colonizing pattern, as a pattern of the type that is not simply replicating itself but making itself more real, and makes itself more real by building machinery that allows consciousness to stick around, by controlling reality, by anticipating different ways that could thwart it, that could make it harder for it to exist, and changing reality around itself, changing the future to make itself more likely to happen in the future. And in terms of more real, you don’t just mean real means that it has many physical instantiations. You mean real is the pattern in the information ontology itself. Yes, to real means for me to be implemented, and so it means that there is some pattern in the substrate that is actually implemented as a pattern in the substrate. And so, consciousness is a pattern in the substrate of the physical universe that has the not just the ability to replicate itself, but the ability to colonize, to possess, to engrave itself deeper, and thereby changing the other patterns in the physical universe. Very good. And what you were saying about the the the the multi-universe phylogeny, it’s very similar to Wolfram’s.

【译】

再往上一层，你有了这些涡旋之间、这些粒子之间的相互作用，它们形成分子。到某个时刻，它们形成细胞——超级分子。但细胞多了一样新东西：它们不能只靠纠错待在原地。它们是能把一道凹槽挖进宇宙更深处的模式：它们能够自我复制，并以此应对自身的消解——它们能够重建自己，并以此让自己「变多」。所以，演化关乎生命——关乎那些不只是在复制自己、而是在「加深」自己的模式：通过控制世界、通过能够克服噪声、克服熵、克服「宇宙与你的期待和偏好脱节」的方式，把自己更多地铭刻进世界。而我认为，意识是作为一种「殖民模式」（colonizing pattern）涌现的：一种不满足于复制自己、而是要让自己变得更真实的模式。它通过建造让意识得以存留的机器、通过控制现实、通过预期各种可能挫败它、让它更难存在的方式、并通过改变自己周围的现实、改变未来，来使自己在未来更有可能发生。——你说的「更真实」，不只是指它有很多物理实例吧？你是说：真实=信息本体（information ontology）之中的一个模式本身。——对。「更真实」对我来说，意味着「被实现」：在基底中确实存在某个被实现为模式的模式。所以，意识是物理宇宙基底之中的一个模式，它不仅有能力复制自己，而且有能力殖民、占有、把自己刻得更深，并以此改变物理宇宙中的其他模式。——非常好。而你讲的这个「多宇宙谱系」，非常像 Wolfram 的……

[4920s - 5040s]

Idea of this constructive universe. He talks about the Ruliad, which is the entangled limit of all possible computations. I think that this pattern of the Ruliad is exactly this superposition of all the automata that is producing the universe. Or the old Greeks called it logos. It’s basically the mathematics that is so simple that you don’t need to derive it; that exists by itself. And so the universe can be understood as an emergent mathematical object that exists over very simple rules that are so simple that they can exist by themselves. Let’s talk about the measurement problem in consciousness. How do we know the difference between something which is actually conscious versus simulating it? That’s a very tricky question, and to me, it comes down to whether the system that is producing statements about its conscious experience to itself and others is implementing a causal structure that gives rise to phenomenal experience of being an observer. And the difficulty here is that most people’s intuitions don’t tell them a lot about what is necessary for a system to have such an experience. Personally, my intuition says it’s not that hard. Right. When we think of spatial cognition, spatial cognition doesn’t seem to have a hard problem. Right. Nobody is puzzled by the fact that a self-driving car can have spatial cognition, and I wouldn’t say that it’s simply deepfaking spatial cognition, because we accept that spatial cognition is the implementation of certain mathematical regularities that human brains can learn and that computers can learn as well, and that we can even express analytically to a large degree. Right. We can write down a computer program that is able to perform spatial cognition, except in lots of edge cases when you actually want to deal with messy camera images. But by and large, right, you don’t need to have an end-to-end system to do this. And if you have an end-to-end train system, you don’t think there is any magic going on in there. Whereas people.

【译】

……那个「构造性宇宙」的观念。他谈 Ruliad——所有可能计算的纠缠极限。我认为 Ruliad 这个模式，恰恰就是产生宇宙的这个「所有自动机之叠加」。或者，古希腊人把它称为 logos（逻各斯）：一种简单到无需推导、自身即存在的数学。于是，宇宙可以被理解为一个涌现的数学对象，建立在一些简单到能够自身存在的规则之上。——那我们来谈谈意识里的「测量问题」：我们怎么分辨一个东西是真的有意识，还是在模拟意识？——这是个非常棘手的问题。对我来说，它归结为：那个向自己与他人产出「关于其意识经验之陈述」的系统，是否在实现一个「产生『作为观察者的现象经验』」的因果结构。而这里的困难在于：大多数人的直觉，并不能告诉他们一个系统要有这样的经验、需要什么条件。就我个人而言，我的直觉说这没那么难。想想空间认知：空间认知看起来就没有难问题。没有人会因为一辆自动驾驶汽车拥有空间认知而感到困惑，我也不会说它只是在「深度伪造」空间认知，因为我们接受：空间认知是某些数学规律性的实现——人类大脑能学，计算机也能学，甚至大部分可以解析地表达出来。我们可以写出一个能执行空间认知的计算机程序——只是要处理乱糟糟的相机图像时有很多边缘情形。但大体而言，你不需要一个端到端的系统来做这件事；而当你有一个端到端训练的系统在做这件事时，你也不认为里面有什么魔法。然而人们却期待……

[5040s - 5160s]

Expect that a thing that experiences itself experiencing is somehow magical. And personally, I don’t see why this should be magical. I remember when I was a kid, I was dreaming about having chocolates, and these chocolates had a very strong smell, and I wanted to touch and eat them. I was, I think, two or three years old when I had this dream, and as dreams have it, there’s this insidious thing that you about to eat the delicious chocolate, and this is the moment when you wake up, right before you want to put it into your mouth. It’s it’s very torturous, and so I woke up and suddenly the delicious chocolate had turned into an idea. It was no longer real. It was simply something that had no phenomenal content anymore. I knew everything about these chocolates that I had in my hand a moment ago. They were just gone. They were no longer real, and so I thought I need to get make them real again. I need to fall asleep again and enter the dream again. And I succeeded. I went back into the dream, and the chocolate became real again. I could smell it and taste it, and then I flipped back into the wakefulness state, and again the phenomenality, the realness disappeared, and I realized the chocolate is no longer real at the same level of me. It’s now something that I hold in my mind only as an abstract idea that I can play around with, but the contents are exactly the same. The only thing that has gone is the illusion of realness. So, what is this illusion of realness? I suspect it’s a feature dimension similar to color, sound, number, weight, and so on. It’s a representation inside of my own mind, and when you dream, you often dream that ideas become real, and when you perceive the outside world, then all the stuff that is validated by your sensory system is tagged by your brain as real, and sometimes there is a defect in your brain, and you experience things that you imagine as real. We call this a hallucination. It’s not the same as in an LLM. It’s that you are confused about the fact that your brain is producing.

【译】

……「一个体验着自身在体验的东西」有某种魔法。而我个人看不出，这为什么必须是魔法的。我记得小时候——大概两三岁——我梦见了巧克力：那些巧克力有非常浓烈的气味，我想触摸它们、想把它们吃掉。而梦就是这样阴险：就在你就要吃到那块美味巧克力的那一刻，你醒了——刚好在把它放进嘴里之前。这非常折磨人。于是我醒了，突然间，那块美味的巧克力变成了一个观念：它不再真实了；它只是某种不再有现象内容的东西。关于我刚才手里握着的那些巧克力，关于它们的一切我都知道——但它们就是「不在了」，不再真实。于是我想：我得让它们重新变真，我得再睡着、再回到那个梦里去。而我成功了：我回到梦里，巧克力又变真了——我能闻到它、尝到它；然后我又翻回清醒状态，现象性、真实感又消失了。我意识到：巧克力不再与我处在同一个「真实」的层级上；它现在只是我心智里作为一个抽象观念持有的东西，可以拿来摆弄——但内容完全一样。唯一失去的东西，是真实感的幻觉。那么，这个真实感的幻觉是什么？我怀疑它是一种类似颜色、声音、数字、重量的特征维度——是我自己心智内部的一种表征。当你做梦时，你常常梦见「观念变成了真实」；而当你知觉外部世界时，所有被你的感官系统验证过的东西，都会被你的大脑打上「真实」的标记。而有时你的大脑出了缺陷，你会把想象中的东西体验为真实的——我们把这叫作幻觉（hallucination）。这和 LLM 里的「幻觉」不是一回事：这是你对自己大脑正在产生的某种东西感到困惑——它本没有感官验证……

[5160s - 5280s]

An illusion of something that is currently doesn’t have sense sensory validation, and yet, for some reason, the brain has an error and tags it as sensory validated, and suddenly it becomes real. And your own experience of yourself is also something that is validated in this perceptual way, so you experience it as real. And now you should be able to test this by getting into a state where you yourself become de-realized, and you suddenly experience yourself only as an idea, and the object of the conscious observer and the subject of the conscious observer disappear. And indeed, I was able to recreate this, and I notice it’s not that rare. It’s called de-realization or depersonalization, not exactly the same phenomena, but very closely related. And people report on this, and this is involuntarily. These people often end up in psychiatric institutions because it’s very stressful to them, and they don’t know how to deal with the fact that they are no longer real to themselves, and there is no I anymore that they can meaningful refer to. And it’s an interesting state to be in, and I’m happy that it ended. But it’s something that you can induce, and at in this state, phenomenality changes, and so to me, it seems plausible that this explains how consciousness is working. That consciousness is a particular type of representation that is created in my mind, and that doesn’t have a complexity that necessarily goes beyond what the LLM is producing. Now there is a problem with the LLM because the LLM is a text generator, and you don’t know what the text indicates. Right, the text is trained on novels, for instance, and many novels the authors write about stuff they don’t experience while they write it. Right, a novel is often a deepfake. When an author writes about how a character is experiencing how to die, right, the author probably didn’t die while they wrote this, so it’s a deepfake of a conscious experience. And so if the LLM is trained to produce deepfakes, how do you know when it actually experiences something? And that that is one of the big difficulties. Right, the LLM is a shapeshifter; it can produce

【译】

……却因为某种原因，大脑出了个错，把它标记成了「感官已验证」，于是它突然就变真了。而你对你自己的经验，也是以这种知觉性的方式被验证的，所以你把它体验为真实。现在，你应该能通过进入一个「你自己被去真实化」的状态来检验这一点：你会突然把自己只当作一个观念来体验——意识观察者的对象与主体都消失了。我确实能够重现这种状态，而且我注意到它并不罕见。它叫作去真实化（de-realization）或去人格化（depersonalization）——不完全是一回事，但密切相关。有人报告过这些状态，而且是非自主发生的：这些人常常最终进了精神科机构，因为这对他们来说压力极大，他们不知道如何应对「自己对自己不再真实」、「不再有一个可以有意义地指涉的『我』」这件事。那是一种有意思的状态，我很高兴它结束了。但这是可以诱导的；而在那种状态里，现象性（phenomenality）发生了改变。所以对我来说，下面这个解释看起来是可信的：意识是我的心智中被创造出来的一种特定类型的表征，而它的复杂度未必超过 LLM 正在产出的东西。现在，LLM 有一个问题：它是一个文本生成器，而你不知道文本指示着什么。文本是在小说之类的材料上训练的；而在许多小说里，作者写的是他们写作时并未经历的东西——小说常常是一种深度伪造（deepfake）。当作者写一个角色如何体验死亡时，作者写这段话时大概并没有死，所以那是意识经验的 deepfake。那么，如果 LLM 被训练成产出 deepfake，你怎么知道它什么时候真的在经验？这是最大的困难之一。LLM 是一个变形者（shapeshifter）：它能产出任意的角色与故事。那么它在哪一点上是假的，哪一点上真的在模拟那个因果的经验？而这正是需要某种机制可解释性（mechanistic interpretability）的地方：我们需要真的看进 LLM 的结构，去查明它是否与那个产出同样内容的心智，在因果上等价。

[5280s - 5400s]

Arbitrary characters and stories, and so at which point is it fake, and at which point is it actually simulating the causal experience? And that—that is really the problem that requires some kind of mechanistic interpretability, where we would actually need to look into the structure of the LLM to find if it’s causally equivalent to the structure of the mind that is producing the same thing. Yeah, so we need to have a white box method to to ascertain if consciousness is present. Isn’t that interesting, though? Because do you think that for intelligence behavior alone is enough to ascertain whether it’s there? But for consciousness, particularly, we need to look for certain types of internal organization and causal structure. Yes, but most people have the intuition that the LLM is doing something that is much more simple than what we are doing when we are conscious. But we don’t expect that the LLM is doing something that’s much more simple than it does arithmetic than we do. And I think that this intuition might be misguided. It could be that the LLM is doing something that is just as simple as what the human brain is doing when it’s conscious. And I’m pretty sure that the human brain is doing the simplest possible thing that it can do to produce our behavior. And the reason why we are all conscious is because it’s part of the simple strategy to produce our behavior. And so if we were to get a learning system, an artificial learning system, to from the ground up to discover how to make agentic, coherent behavior, and the system is behaving in a way that makes us think that it’s conscious, the simplest explanation I think is that it actually is conscious. And my intuition is not necessarily telling me, oh my God, this is going to do something that is much more complicated than us to produce the same behavior. So it’s going to do this in some kind of circumspect way that is producing all the behavior that require consciousness in a human. But in this case, it’s doing something that only pretends that it does it, which ultimately might be more complicated. What would falsify your

【译】

——对，所以我们需要一种「白盒」方法来确认意识是否存在。不过这里不是很有意思吗：你认为对「智能」而言，仅凭行为就足以确认它在场；而对意识，我们却特别需要寻找特定类型的内部组织与因果结构？——是的。但多数人的直觉是：LLM 做的事情，比我们有意识时所做的事情简单得多。可是，我们并不期望 LLM 做算术的方式比我们简单，对吧？我认为这个直觉可能是被误导的。完全可能：LLM 做的事情，与人类大脑有意识时所做的事情一样简单。而且我相当确定，人类大脑在产生我们的行为时，做的是它能做的最简单的事情。而我们之所以全都有意识，是因为意识是「产生我们行为」的那套简单策略的一部分。所以，如果我们让一个人工学习系统从底层、从零开始，自己发现如何产生能动的、连贯的行为，而这个系统的行为让我们觉得它有意识——那么最简单的解释，我认为就是：它真的有意识。我的直觉并没有告诉我：「哦我的天，这东西会以一种比我们复杂得多的方式，绕着弯产出所有那些在人类身上需要意识的行为，只是装作自己有意识」——因为那样做反而可能更复杂。

[5400s - 5520s]

Machine consciousness theorem. So, if we were to build a system that is behaving in the same way as us, there is a question: Can we produce the same behavior as ours without phenomenality? Right, and the phenomenality I think is a representation here in this case. If if we look at the what the LLMs are doing, and we extend this into a multimodal model, and they produce a similar output as us, but they are able to observe themselves, and their observations that they report differ from what we are doing, then we would have some evidence against it. The but the more fundamental question would be if we were to build a self-organizing system that is by self-organized learning trying to get there, and there is no way in which we achieve the organization of of behavior in a self-organized way instead of by imitation learning as the current LLMs are doing. This might be some evidence that consciousness is more complicated than something that can run on today’s computers, but it it is not definitive proof because it could simply be that I made a mistake. Right, the fact that I don’t achieve a certain result doesn’t mean that it cannot be achieved. It means that I couldn’t achieve it. But so it’s very difficult to have a proof against it. Right, so it’s more evidence for it if we succeed. So what we would have to show in order to convince Anil Seth maybe is that we build a system that is self-organizing, that is not trained on text, that is doing developmental learning, and that naturally gets to the point where it makes self-reports and reports about its conscious experience, and it requires this conscious experience to produce its performance. So I think this would be evidence in favor that the the contrary is much harder. But in a sense, I don’t think you believe in zombies because you think that.

【译】

——什么会证伪你的机器意识「定理」？——如果我们造出一个行为方式与我们相同的系统，问题在于：我们能否在「没有现象性（phenomenality）」的情况下，产生与我们相同的行为？在这里，现象性我认为是一种表征。如果我们观察 LLM 们正在做的事，把它扩展成一个多模态模型，它们产出与我们相似的输出、并且能够观察它们自己，而它们的自我观察报告与我们所做的不同——那我们就得到了一些反对（机器意识假说）的证据。但更根本的问题是：如果我们造一个自组织系统、试图通过自组织学习到达那里，却始终无法以自组织的方式（而不是像当前 LLM 那样靠模仿学习）达成行为的组织——那这可能是「意识比能在今天计算机上运行的东西更复杂」的证据。但这不是决定性的证明，因为也可能只是我犯了错，对吧？「我没达成某个结果」不意味着「它不可能被达成」，只意味着「我没能达成」。所以，很难有反对它的证明；反过来，如果我们成功了，那才是支持它的证据。所以，要说服 Anil Seth 这样的人，我们必须造出这样一个系统：它是自组织的、不在文本上训练、做发展式学习（developmental learning），并且自然地到达「做出自我报告、报告其意识经验」的地步——而且它的性能必须依赖于这个意识经验。我认为这将是支持（机器意识假说）的证据，而反方要难得的多。

[5520s - 5640s]

Consciousness is an absolutely integral part of cognition. I mean, there is a possibility that you get to zombies. This other what I meant is if we produce something that is able to produce our behavior, except for experience of phenomenality. So it’s basically able to do everything that a human being is capable of doing and more, but it does not need to produce a conscious observer. Right? This could be a zombie like this, and I think zombies are possible. Ideally, you can build self-driving cars that do things that would require intelligence of human beings, or you can play chess-playing computers that can do things that require consciousness in human beings, and they achieve all those things without consciousness, and that’s because we implement them on a substrate that allows us to do to construct algorithms on them that would not run on human brains. Right? The chess algorithm that runs on a current chess computer could not be implemented on the neurons of your brain because how would you entrain it on these neurons? So you wouldn’t get it in there. In order to entrain yourself to play chess, you need to have a conscious human being. It’s the only way in which a human being is training itself up there, and it’s conceivable that the entire set of intelligent behaviors that you are interested in, except for this one thing where you are simulating a self-reporting conscious agent, can be implemented on GPUs because the GPUs are more powerful, more deterministic, more open to other classes of algorithms. But could we say that zombies are possible in principle but not in practice? And what I mean by that, let’s say we could imagine a world where we could train an MLP to be the same as GPT-5. It would just take an insane amount of computation power. And similarly, consciousness is what makes our minds tractable. It’s an incredibly efficient process. So we might say that if we see an artifact in our world which behaves as if it’s conscious, would you argue that it can’t be anything other than conscious, or do you still think?

【译】

——但某种意义上，我觉得你并不「信」僵尸，因为在你看来，意识是认知绝对不可分割的一部分。——我的意思是：如果我们造出一个能产生我们全部行为、唯独没有现象性之经验的东西——也就是它能做人类能做的一切、甚至更多，但不需要产生一个有意识的观察者——这样一个东西就可能是「僵尸」，而我认为僵尸是可能的。理想情况下，你能造出自动驾驶汽车，做出需要人类智能才能做的事情；或者下棋计算机，做出在人类身上需要意识才能做的事情，而它们不靠意识就实现了这一切——那是因为我们把它们实现在一种基底上，这种基底允许我们构造那些在人脑上无法运行的算法。当前象棋计算机上跑的象棋算法，没法在你大脑的神经元上实现——你要怎么把它「驯化装进」（entrain）这些神经元？装不进去。一个人要把自身训练到会下棋，必须是一个有意识的人——这是人类把自己训练起来的唯一方式。而可以设想：你所关心的那一整套智能行为——除了「模拟一个会自我报告的有意识主体」这一件事——都可以在 GPU 上实现，因为 GPU 更强大、更确定、对更多类的算法开放。——那我们能不能说：僵尸「原则上可能、实际上不可能」？我的意思是，比如我们可以想象一个世界，在那里我们可以把一个 MLP 训练得跟 GPT-5 一样，只是需要天文数字的算力。类似地，意识是使我们的心智「可解」（tractable）的东西，它是一个极其高效的过程。所以，如果我们在这个世界里看到一个人造物表现得像有意识一样，你会主张它不可能是别的东西、必然有意识吗？还是你觉得……

[5640s - 5760s]

There are other states that could be zombie-like. I think we do have zombies already. I think the chess program is a zombie. Right? You cannot train a human being to play chess without the human being being conscious, but you can train a computer to play chess without the computer being conscious. And if the computer was self-organizing, it’s possible that it could only be trained if it was conscious, or this was the simplest way of doing it. But for many of the things that we currently observe, for instance, you can get the LLM to produce a tax return, even very, very complicated things, while being a zombie, while not having conscious experience of producing a tax return. The only point where the LLM is reporting to be conscious, if you ask it to, right? If you are telling it the following is a conversation between what it would be like if a human being would be talking to a conscious AI, so some version of this, right? And then the thing is able to simulate this. And then the question is: Is this more simulated than us, or is this only a deepfake that is getting there in more circumscribed ways than is happening in our mind? And I don’t know that, right? So I’m agnostic with respect to this. But I would say that in all the other cases, the thing is probably a zombie, right? And so it’s possible to make something that is behaving in all sorts of ways as you do, but at some point it’s going to be more complicated to make it without consciousness. So zombies, I think, are always possible, but sometimes it’s going to be more complicated to make a zombie that produces the behavior than to make a conscious agent that produces the behavior. And this is where the intuition that I have diverges from the intuition of John Searle or of Anil Seth, because I think they both believe that consciousness is very, very complicated because it’s not intuitive to them how it works, and it doesn’t seem to be counterintuitive to me. So I don’t see anything magical about consciousness. If you had ten billion dollars to kick off a research project on consciousness, what would you do? I think I would hire a lot of people who are.

【译】

……还可能存在其他「僵尸式」的状态？——我认为我们已经拥有僵尸了：国际象棋程序就是一个僵尸。你无法把一个不会下棋的人训练成会下棋、而不使其有意识；但你可以把一台计算机训练成会下棋，而不让它有意识。而如果这台计算机是自组织的，那么有可能：它只有在有意识时才能被训练起来，或者意识是做到这一点的最简单方式。但就我们目前观察到的许多事情而言——比如你可以让 LLM 产出一份报税表，做非常非常复杂的事情——它是个僵尸，没有「产出报税表」的意识经验。LLM 报告自己有意识的唯一场合，是你要求它这样做的时候：你告诉它「下面是一段人类与有意识的 AI 对话会是什么样子」——某种版本的提示——然后这东西能够模拟它。而问题在于：这比我们「更被模拟」吗？还是说，它只是一个以比我们心智更绕的方式到达那里的 deepfake？我不知道，对吧——对此我保持不可知。但我认为，在所有其他场合，这东西多半是个僵尸。所以，造出一个在各个方面都表现得像你一样的东西，是可能的；但在某个点上，「没有意识地产生这种行为」会变得比「有意识地产生它」更复杂。所以我认为，僵尸始终是可能的，但有时候，造一个产生该行为的僵尸，比造一个产生该行为的有意识主体更复杂。而正是在这里，我的直觉与 John Searle 或 Anil Seth 的直觉分道扬镳：我认为他们俩都相信意识非常、非常复杂，因为「它如何运作」对他们来说不直观；而对我来说，它并不反直观。所以，我在意识身上看不到任何魔法。——如果你有 100 亿美元来启动一个意识研究项目，你会做什么？——我想我会雇很多……

[5760s - 5828s]

A lot smarter than me. At the moment, I’ve started a smaller initiative. It’s called the California Institute for Machine Consciousness, and I’m currently looking for people who are smarter than me who want to join me and to build a project in which we can look at the search space for consciousness, in which we can see can we build a self-organizing system to learn, and is this thing going to develop phenomenology? And I hope that is going to help us to build our intuitions. Even if it turns out that it doesn’t work, we are going to learn something, right? And progress is being made by figuring out the ways in which I’m wrong, not just the ways in which I’m right. So if I had 10 million dollars, a billion dollars, my strategy wouldn’t change substantially, except I would be able to hire more people for this and start with larger computers. Joscha Bach, it’s been an absolute honor. Thank you so much for joining us today. Thank you. Was a pleasure.

【译】

……比我聪明得多的人。眼下，我已经启动了一个规模较小的倡议，叫「机器意识加州理工学院」（California Institute for Machine Consciousness）。我目前正在寻找比我聪明、愿意加入我的人，一起构建一个项目：在其中考察意识的搜索空间，看看我们能否构建一个自组织的学习系统——这个东西会不会发展出现象学？我希望这能帮助我们建立直觉。就算最后发现行不通，我们也会学到东西，对吧？进步，是通过发现「我在哪些地方错了」取得的，而不只是通过发现我在哪些地方对了。所以就算给我 1000 万或者 10 亿美元，我的策略也不会有实质变化——只是能雇更多的人、从更大的计算机开始而已。——Joscha Bach，与我们同台是绝对的荣幸，非常感谢你今天来。——谢谢，我的荣幸。