压缩是为了节约，也是为了对比 既要有显著特征，也要有可以对比特征

通过compression，创造节点 通过association，创造链接

wrong feature extraction wrong feature association

我觉得大模型，压缩也可以，链接也可以 但是，压缩效率比较低，链接的能力也比较弱 大模型估计可以实现了某种意义的AGI，但是效率会比较低

deduction and reduction

压缩就算是你脑力无边，也不行 因为你要predict，partial or variantional concept has to be detected

Deep networks = Compression + Association Multi-Layer = Compression Self-Attention = Association Layers = semantic compression (turn raw data into abstract representations) Attention = contextual association (relate parts to make sense of the whole)

挖词在于重复的观察 连词也在于重复的观察

特征要salient 链接要明确 都是没有噪音

挖词 和 连词

一共四件事 1. 学习，压缩和链接 2. 预测，解压和链接 3. 压缩，信息的抽象 4. 链接，计算和推理

lexeme trigger distritution of next lexeme the high prob lexemes association is a lexeme as well

p(predict | prompt), based on prompt to select right asscociate p(model | data), what to compress and what to associate

有没有值得压缩的抽象 有没有值得链接的概念

lexemes and their co-occurs

Narrow down, focus to direction p(y|prompt / x_i)

势利眼就是恶人

多层之间有抽象，就会有不同层的抽象概念 任何概念之间都可以有链接

Exists f for all d f(d) Given t, f(t)

for all f (a is A and f(A) => f(a)

人类真有趣，既要贪心 少付出，又大回报 但也要使劲付出，得到一堆垃圾

费挺大劲，吃一口屎

我自己不成熟时 希望看几分钟，几个案例 就能捕捉到全部，想迅速得到结论 What You See Is All There Is

就是看问题角度 之前就是能看懂，做上题

真·思考结束

认知越浅薄 需求越原始 价值越扭曲

我这一生，希望能理解智能的工作原理 把花书吃透，将是我的第一个目标，虽然我学了五门机器学习相关的课，但还是不够内化

虚荣就是情绪幼稚

既要有因认知而造成的限制 也要有认知福利 也就是特殊通道

过分的迁就，就是在激励，白痴的胜利 要想办法减少低认知，对群体利益的损伤

这就是研究智能的悖论 如果做到的是人类可以轻易做到的，缺乏经济动力 如果做到的是人类不可以轻易做到的，可能和智能就不搭边 比如，大模型 imo 拿金牌，比如，统计学解决很多人脑不能解决的问题，就让我觉得，可能这些不是人脑的工作方式

Amalfi Coast 种草

给低能者开辟机会 就是在为作恶者提供舞台

evidence-based thinking and emotional regulation Humans are evolutionarily tribal, but modern societies need to downplay tribalism It’s not things themselves, but our interpretations of them that disturb us

Approximately, error, if sample is good, how much your hypothesis is wrong, how wrong you are about the world Probably, confidence, if model is good, probability you got a bad sample

痛苦拉满，快速无情

大脑算的时候也会更新模型 即使没有 error 也更新

贝叶斯的词汇 P(token | prompt, context, model)

P(model|prompt, context, token)

上面是计算 下面是更新

上面是 params 下面是 features

贝叶斯 既给了你 compressed path 也给了你 meaningful path

5.9 teaching not as transmitting all knowledge but as guiding the learner toward useful learning opportunities What new concept, skill, or idea is most beneficial for the learner right now, given what they already know. give labeled examples

Heat is generating new structures to solve problems of stable existing

Of course, we are trapped in specific space time frames Of course, death means eternal black Of course, you are the only observer that matters So what? You still have to enjoy Have to learn and solve Have to progress and win

No need to be perfect But how well it works If it acts like causation Consider as the causation

Local maximization

Causation is simply the removal of the irrelevant context

再看一遍这本书，六年前看的很开心，做一个好学生

提高不评价，进展不对比 痛苦，快速，无情绪