diff --git a/archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter/2026-07-17-DaleCloudman-ya-compress-the-ast-size-not-the-character-count.md b/archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter/2026-07-17-DaleCloudman-ya-compress-the-ast-size-not-the-character-count.md new file mode 100644 index 0000000..03d366a --- /dev/null +++ b/archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter/2026-07-17-DaleCloudman-ya-compress-the-ast-size-not-the-character-count.md @@ -0,0 +1,94 @@ +--- +title: "@VictorTaelin Ya compress the AST size not the character count.." +type: archive +source: twitter +source_url: "https://x.com/DaleCloudman/status/2078134295768486176" +author: "Dale Cloudman" +handle: DaleCloudman +post_id: "2078134295768486176" +date: 2026-07-17 +archived: 2026-08-25 +draft: false +tags: + - archive + - twitter + - VictorTaelin +description: "@VictorTaelin Ya compress the AST size not the character count.." +in_reply_to: "" +parent_post_id: "2078110851123286452" +--- + +## Source + +- URL: https://x.com/DaleCloudman/status/2078134295768486176 +- Author: Dale Cloudman (@DaleCloudman) +- Posted: 2026-07-17 15:06:40 + +## Branch + +**1/** **@DaleCloudman** ^2078134295768486176 + +**@VictorTaelin** + +Ya compress the AST size not the character count.. + +Not just branches + +Foo(a(1,2)) +Bar(a(1,2)) + +Is worse than + +Z = a(1,2) +Foo(z) +Bar(z) + +**2/** **@VictorTaelin** ^2078137866752864328 + +**@DaleCloudman** + +no. number of branches. + +ast size is not the true measure of complexity because non branching ast nodes can be inferred in linear time + +**3/** **@DaleCloudman** ^2078142390586499488 + +**@VictorTaelin** + +Didn’t follow, what it means to ‘infer’ an AST node in this context? + +**4/** **@VictorTaelin** ^2078145575774494751 + +I mean quadratic time * + +I just obsreve that any program without branching can be reconstructed in quadratic time given enough I/O pairs. So I believe measuring complexity from number of branches is more accurate + +You can't build a complex algorithm without branches so, the more branches you have, the more complex your file / algorithm is. If you remove all branches, it becomes trivial and compressible, even if there are still lots of characters + +Example: + +foo = λa. a + a + +and + +foo = λa. a + a + a + a + a + +have the same underlying complexity, despite one being larger. + +or so I think (: + +**5/** **@DaleCloudman** ^2078608651707625503 + +**@VictorTaelin** + +So I'm trying this for Python and there's lot of edge cases... how do I handle calling foreign code, it 'optimized' a max-finding algo by just calling np.max instead of doing the py version lol. and using np.where to avoid ifs... + +**6/** **@DaleCloudman** ^2078619365839192104 + +**@VictorTaelin** + +Ok I think it’s more that a function call is not a branch… + +## Related + +- Spine: [[archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter]] diff --git a/archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter/2026-07-17-Jatin_exe-i-love-how-well-articulated-this-post-is-counter.md b/archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter/2026-07-17-Jatin_exe-i-love-how-well-articulated-this-post-is-counter.md new file mode 100644 index 0000000..e73d97f --- /dev/null +++ b/archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter/2026-07-17-Jatin_exe-i-love-how-well-articulated-this-post-is-counter.md @@ -0,0 +1,69 @@ +--- +title: "I love how well articulated this post is! " +type: archive +source: twitter +source_url: "https://x.com/Jatin_exe/status/2078116506924171699" +author: "Jatin Khanna" +handle: Jatin_exe +post_id: "2078116506924171699" +date: 2026-07-17 +archived: 2026-08-25 +draft: false +tags: + - archive + - twitter + - VictorTaelin +description: "I love how well articulated this post is! " +in_reply_to: "" +parent_post_id: "2078110851123286452" +--- + +## Source + +- URL: https://x.com/Jatin_exe/status/2078116506924171699 +- Author: Jatin Khanna (@Jatin_exe) +- Posted: 2026-07-17 13:55:59 + +## Branch + +**1/** **@Jatin_exe** ^2078116506924171699 + +I love how well articulated this post is! + +> ...Counter that by counting the NUMBER OF BRANCHES. A branch is: an "if", a "match", a "case". That's THE complexity of your program.. + +Could we do this better by measuring the properties of the IR graph from the code + +Maybe we need to aim for kolmogorov compression + +**2/** **@VictorTaelin** ^2078120435762213196 + +**@Jatin_exe** + +no + +number of branches. period. + +**3/** **@Jatin_exe** ^2078121769073074250 + +**@VictorTaelin** + +What if it starts doing branchless programming as a reward hack + +**4/** **@henryzom** ^2078136211604906066 + +**@Jatin_exe** **@VictorTaelin** + +Yay performance + +**5/** **@Jatin_exe** ^2078139019859550484 + +**@henryzom** **@VictorTaelin** + +You can have a very bad performance code with zero branches compared to good perf with branches (I think ) + +If you take it to the extreme and try to minimize only the no of branches to the least you can get, will be interesting to see how it works + +## Related + +- Spine: [[archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter]] diff --git a/archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter/2026-07-17-LeeeeeeeeeT_-a-related-issue-that-bothers-me-is-that-the.md b/archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter/2026-07-17-LeeeeeeeeeT_-a-related-issue-that-bothers-me-is-that-the.md new file mode 100644 index 0000000..a5e6f50 --- /dev/null +++ b/archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter/2026-07-17-LeeeeeeeeeT_-a-related-issue-that-bothers-me-is-that-the.md @@ -0,0 +1,43 @@ +--- +title: "@VictorTaelin a related issue that bothers me is that the changes ai makes to your project are always continuous." +type: archive +source: twitter +source_url: "https://x.com/LeeeeeeeeeT_/status/2078172260771201471" +author: "LeeeeT" +handle: LeeeeeeeeeT_ +post_id: "2078172260771201471" +date: 2026-07-17 +archived: 2026-08-25 +draft: false +tags: + - archive + - twitter + - VictorTaelin +description: "@VictorTaelin a related issue that bothers me is that the changes ai makes to your project are always continuous." +in_reply_to: "" +parent_post_id: "2078110851123286452" +--- + +## Source + +- URL: https://x.com/LeeeeeeeeeT_/status/2078172260771201471 +- Author: LeeeeT (@LeeeeeeeeeT_) +- Posted: 2026-07-17 17:37:32 + +## Branch + +**1/** **@LeeeeeeeeeT_** ^2078172260771201471 + +**@VictorTaelin** + +a related issue that bothers me is that the changes ai makes to your project are always continuous. i've never seen ai deleting the entire modules along with the ideas behind them, and then building new modules based on new abstractions. + +**2/** **@VictorTaelin** ^2078177095990096274 + +**@LeeeeeeeeeT_** + +exactly + +## Related + +- Spine: [[archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter]] diff --git a/archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter/2026-07-17-Rafa_Schwinger-i-think-anthropic-is-already-doing-something.md b/archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter/2026-07-17-Rafa_Schwinger-i-think-anthropic-is-already-doing-something.md new file mode 100644 index 0000000..7b6feff --- /dev/null +++ b/archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter/2026-07-17-Rafa_Schwinger-i-think-anthropic-is-already-doing-something.md @@ -0,0 +1,81 @@ +--- +title: "@VictorTaelin I think Anthropic is already doing something similar because they are compressing intelligence as much as possible for pretraining." +type: archive +source: twitter +source_url: "https://x.com/Rafa_Schwinger/status/2078114345477603691" +author: "Rafa Schwinger 🇻🇦" +handle: Rafa_Schwinger +post_id: "2078114345477603691" +date: 2026-07-17 +archived: 2026-08-25 +draft: false +tags: + - archive + - twitter + - VictorTaelin +description: "@VictorTaelin I think Anthropic is already doing something similar because they are compressing intelligence as much as possible for pretraining." +in_reply_to: "" +parent_post_id: "2078110851123286452" +--- + +## Source + +- URL: https://x.com/Rafa_Schwinger/status/2078114345477603691 +- Author: Rafa Schwinger 🇻🇦 (@Rafa_Schwinger) +- Posted: 2026-07-17 13:47:24 + +## Branch + +**1/** **@Rafa_Schwinger** ^2078114345477603691 + +**@VictorTaelin** + +I think Anthropic is already doing something similar because they are compressing intelligence as much as possible for pretraining. + +**2/** **@VictorTaelin** ^2078115018294341680 + +**@Rafa_Schwinger** + +how so + +**3/** **@Rafa_Schwinger** ^2078115738422824982 + +**@VictorTaelin** + +You need minimum descriptions to maximize intelligence density. That implies that your internal data factory needs a model that is about to compress information losslessly or minimally lossy. They could adapt something similar for posttraining + +**4/** **@VictorTaelin** ^2078116059035349109 + +**@Rafa_Schwinger** + +I mean why do you think they're training on erasure seriously + +**5/** **@Rafa_Schwinger** ^2078120963351187604 + +Because if you want to maximize intelligence density, let's say pre-training or in thinking traces, you of course need the least amount of tokens that represents the information. Suppose that you want to feed Wikipedia as a dataset to train a LLM. You can probably get 99% of the effective quality by trimming it at 10% of its size. You compress the text and you leave the most important articles a bit longer and the rarest articles shorter. The whole definition of entropy from info theory is about minimum length anyway so anyone working with synthetic data can have a preprocessing layer like this. + +It is more difficult to do that at scale because you need to QA reward hacking, but since GPT was a bit behind specifically in reward hacking despite being good at grind seems to suggest that Anthropic takes this point more seriously. In a way, elegant solutions is basically the same thing as avoiding reward hacking. + +Of course I could be wrong about all of this, but it seems to be pointing in this direction. + +**6/** **@VictorTaelin** ^2078123440637194583 + +**@Rafa_Schwinger** + +my point is that I think they're doing that accidentally + +when you have 500000 B300, you will be training for erasure indirectly, because it IS needed for capabilities. but you're doing so innefficiently + +my point is that minding erasure will 100x their effective FLOPS + +**7/** **@Rafa_Schwinger** ^2078125069952315482 + +No, I think that everyone is relying heavily on synthetic data and internal RLVR pipelines rn and making sure that you have something closer to minimal descriptions would be a important part of this pipeline. So I think that they are aware of the issue even if they perhaps can't do that properly. + +The problem is also a matter of topology. Some operations are global in nature, but the training is still too local and iterative. So sometimes refactoring demands a global operation while adding is of course local. our current training methods. There are way too many physics systems where you would not be able to get to the minimum using naive gradient descent. An RL seems to enhance this issue. + +The deeper issue that you're alluding to is that some compressions can't be found just by using a sequence of small local compressions, which is a hard problem. that is physically equivalent to these transitions in memory-shaped alloys. You need to heat it up to cool it down to a different local minimum again. + +## Related + +- Spine: [[archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter]] diff --git a/archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter/2026-07-17-ShumingHu-dl-is-not-built-for-erasure-as-much-as-we-dont.md b/archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter/2026-07-17-ShumingHu-dl-is-not-built-for-erasure-as-much-as-we-dont.md new file mode 100644 index 0000000..ebde415 --- /dev/null +++ b/archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter/2026-07-17-ShumingHu-dl-is-not-built-for-erasure-as-much-as-we-dont.md @@ -0,0 +1,38 @@ +--- +title: "@VictorTaelin DL is not built for erasure as much as we don’t control what we forget." +type: archive +source: twitter +source_url: "https://x.com/ShumingHu/status/2078140561224278290" +author: "Shuming Hu" +handle: ShumingHu +post_id: "2078140561224278290" +date: 2026-07-17 +archived: 2026-08-25 +draft: false +tags: + - archive + - twitter + - VictorTaelin +description: "@VictorTaelin DL is not built for erasure as much as we don’t control what we forget." +in_reply_to: "" +parent_post_id: "2078110851123286452" +--- + +## Source + +- URL: https://x.com/ShumingHu/status/2078140561224278290 +- Author: Shuming Hu (@ShumingHu) +- Posted: 2026-07-17 15:31:34 + +## Branch + +**1/** **@ShumingHu** ^2078140561224278290 + +**@VictorTaelin** + +DL is not built for erasure as much as we don’t control what we forget. +It’s super important part of intelligence but it’s not the same as tasks like compacting text paragraphs. + +## Related + +- Spine: [[archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter]] diff --git a/archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter/2026-07-17-WilmsRolf-intelligence-is-not-just-the-ability-to-reason.md b/archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter/2026-07-17-WilmsRolf-intelligence-is-not-just-the-ability-to-reason.md new file mode 100644 index 0000000..ec0b860 --- /dev/null +++ b/archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter/2026-07-17-WilmsRolf-intelligence-is-not-just-the-ability-to-reason.md @@ -0,0 +1,39 @@ +--- +title: "Intelligence is not just the ability to reason, but the ability to spend reasoning resources wisely." +type: archive +source: twitter +source_url: "https://x.com/WilmsRolf/status/2078163825329668190" +author: "Rolf Wilms 🏴󠁧󠁢󠁥󠁮󠁧󠁿" +handle: WilmsRolf +post_id: "2078163825329668190" +date: 2026-07-17 +archived: 2026-08-25 +draft: false +tags: + - archive + - twitter + - VictorTaelin +description: "Intelligence is not just the ability to reason, but the ability to spend reasoning resources wisely." +in_reply_to: "" +parent_post_id: "2078110851123286452" +--- + +## Source + +- URL: https://x.com/WilmsRolf/status/2078163825329668190 +- Author: Rolf Wilms 🏴󠁧󠁢󠁥󠁮󠁧󠁿 (@WilmsRolf) +- Posted: 2026-07-17 17:04:01 + +## Branch + +**1/** **@WilmsRolf** ^2078163825329668190 + +Intelligence is not just the ability to reason, but the ability to spend reasoning resources wisely. Attention, memory, time, and computation are limited, so an intelligent system must identify what matters for the current goal, compress it into a useful abstraction, and keep irrelevant detail outside active attention. + +But relevance is contextual, compression is lossy, and ignored details may later prove decisive. The goal cannot be deletion. A robust system needs layered representations: a compact working model, links to supporting evidence, and the ability to recover omitted information when confidence drops, contradictions appear, risks rise, or goals change. + +This makes intelligence a problem of adaptive resource allocation and metacognition: deciding what to think about, at what level of detail, for how long, and when to reconsider. The core tension is efficiency versus fidelity, speed versus robustness, and compression versus recoverability. Intelligence lies in managing these trade-offs dynamically. + +## Related + +- Spine: [[archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter]] diff --git a/archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter/2026-07-17-asefweca-i-think-they-already-do-erasure-a-lot-on-every.md b/archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter/2026-07-17-asefweca-i-think-they-already-do-erasure-a-lot-on-every.md new file mode 100644 index 0000000..e47aa62 --- /dev/null +++ b/archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter/2026-07-17-asefweca-i-think-they-already-do-erasure-a-lot-on-every.md @@ -0,0 +1,37 @@ +--- +title: "@VictorTaelin I think they already do erasure a lot, on every token." +type: archive +source: twitter +source_url: "https://x.com/asefweca/status/2078153493148528732" +author: "asefweca" +handle: asefweca +post_id: "2078153493148528732" +date: 2026-07-17 +archived: 2026-08-25 +draft: false +tags: + - archive + - twitter + - VictorTaelin +description: "@VictorTaelin I think they already do erasure a lot, on every token." +in_reply_to: "" +parent_post_id: "2078110851123286452" +--- + +## Source + +- URL: https://x.com/asefweca/status/2078153493148528732 +- Author: asefweca (@asefweca) +- Posted: 2026-07-17 16:22:57 + +## Branch + +**1/** **@asefweca** ^2078153493148528732 + +**@VictorTaelin** + +I think they already do erasure a lot, on every token. The MLP deletes information because it's not a bijective function. The token then supplied with fresh information from its neighbors through attention + +## Related + +- Spine: [[archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter]] diff --git a/archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter/2026-07-17-capitalist_qol-ive-gotten-rid-of-compaction-in-my-loops-in-favor.md b/archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter/2026-07-17-capitalist_qol-ive-gotten-rid-of-compaction-in-my-loops-in-favor.md new file mode 100644 index 0000000..0c407bb --- /dev/null +++ b/archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter/2026-07-17-capitalist_qol-ive-gotten-rid-of-compaction-in-my-loops-in-favor.md @@ -0,0 +1,37 @@ +--- +title: "I've gotten rid of compaction in my loops in favor of a summary handoff to continue the given task because I've noticed that compacting is actually *worse*." +type: archive +source: twitter +source_url: "https://x.com/capitalist_qol/status/2078150266386567317" +author: "Pragmatic Developer" +handle: capitalist_qol +post_id: "2078150266386567317" +date: 2026-07-17 +archived: 2026-08-25 +draft: false +tags: + - archive + - twitter + - VictorTaelin +description: "I've gotten rid of compaction in my loops in favor of a summary handoff to continue the given task because I've noticed that compacting is actually *worse*." +in_reply_to: "" +parent_post_id: "2078110851123286452" +--- + +## Source + +- URL: https://x.com/capitalist_qol/status/2078150266386567317 +- Author: Pragmatic Developer (@capitalist_qol) +- Posted: 2026-07-17 16:10:08 + +## Branch + +**1/** **@capitalist_qol** ^2078150266386567317 + +I've gotten rid of compaction in my loops in favor of a summary handoff to continue the given task because I've noticed that compacting is actually *worse*. You end up with a history that's a mystery, which can lead to good results, or bad results, or crazy results where the agent overindexes on something that it wouldn't have otherwise. + +Definitely not a solved problem in the space! + +## Related + +- Spine: [[archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter]] diff --git a/archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter/2026-07-17-dkubb-ive-been-spending-a-lot-of-time-on-reducing-the.md b/archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter/2026-07-17-dkubb-ive-been-spending-a-lot-of-time-on-reducing-the.md new file mode 100644 index 0000000..ffca7ca --- /dev/null +++ b/archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter/2026-07-17-dkubb-ive-been-spending-a-lot-of-time-on-reducing-the.md @@ -0,0 +1,37 @@ +--- +title: "I’ve been spending a lot of time on reducing the state space of a program instead of just trying to remove branches, and then models are really good at it if you give them the right skills." +type: archive +source: twitter +source_url: "https://x.com/dkubb/status/2078142642177646741" +author: "Dan Kubb" +handle: dkubb +post_id: "2078142642177646741" +date: 2026-07-17 +archived: 2026-08-25 +draft: false +tags: + - archive + - twitter + - VictorTaelin +description: "I’ve been spending a lot of time on reducing the state space of a program instead of just trying to remove branches, and then models are really good at it if you give them the right skills." +in_reply_to: "" +parent_post_id: "2078110851123286452" +--- + +## Source + +- URL: https://x.com/dkubb/status/2078142642177646741 +- Author: Dan Kubb (@dkubb) +- Posted: 2026-07-17 15:39:50 + +## Branch + +**1/** **@dkubb** ^2078142642177646741 + +I’ve been spending a lot of time on reducing the state space of a program instead of just trying to remove branches, and then models are really good at it if you give them the right skills. + +I only try to prune branches that don’t directly correspond to an invariant that my program must follow. + +## Related + +- Spine: [[archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter]] diff --git a/archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter/2026-07-17-jakelevi_ml-i-think-code-golfing-minification-uglification-is.md b/archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter/2026-07-17-jakelevi_ml-i-think-code-golfing-minification-uglification-is.md new file mode 100644 index 0000000..c9b5505 --- /dev/null +++ b/archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter/2026-07-17-jakelevi_ml-i-think-code-golfing-minification-uglification-is.md @@ -0,0 +1,39 @@ +--- +title: "@VictorTaelin I think \"code-golfing / minification / uglification\" is okay, as long as the model can still recover an interpretable and human-readable version of the code." +type: archive +source: twitter +source_url: "https://x.com/jakelevi_ml/status/2078147579947180143" +author: "Jake Levi" +handle: jakelevi_ml +post_id: "2078147579947180143" +date: 2026-07-17 +archived: 2026-08-25 +draft: false +tags: + - archive + - twitter + - VictorTaelin +description: "@VictorTaelin I think \"code-golfing / minification / uglification\" is okay, as long as the model can still recover an interpretable and human-readable version of the code." +in_reply_to: "" +parent_post_id: "2078110851123286452" +--- + +## Source + +- URL: https://x.com/jakelevi_ml/status/2078147579947180143 +- Author: Jake Levi (@jakelevi_ml) +- Posted: 2026-07-17 15:59:27 + +## Branch + +**1/** **@jakelevi_ml** ^2078147579947180143 + +**@VictorTaelin** + +I think "code-golfing / minification / uglification" is okay, as long as the model can still recover an interpretable and human-readable version of the code. + +You can imagine a model maintaining 2 versions of every program: the minimal version and the human-readable version + +## Related + +- Spine: [[archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter]] diff --git a/archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter/2026-07-17-o1lo01ol1o-not-true-weight-decay-explicitly-forces.md b/archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter/2026-07-17-o1lo01ol1o-not-true-weight-decay-explicitly-forces.md new file mode 100644 index 0000000..3c444da --- /dev/null +++ b/archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter/2026-07-17-o1lo01ol1o-not-true-weight-decay-explicitly-forces.md @@ -0,0 +1,37 @@ +--- +title: "@VictorTaelin Not true, weight decay explicitly forces sparseification of parameters during training." +type: archive +source: twitter +source_url: "https://x.com/o1lo01ol1o/status/2078125981114433630" +author: "o1lo01ol1o" +handle: o1lo01ol1o +post_id: "2078125981114433630" +date: 2026-07-17 +archived: 2026-08-25 +draft: false +tags: + - archive + - twitter + - VictorTaelin +description: "@VictorTaelin Not true, weight decay explicitly forces sparseification of parameters during training." +in_reply_to: "" +parent_post_id: "2078110851123286452" +--- + +## Source + +- URL: https://x.com/o1lo01ol1o/status/2078125981114433630 +- Author: o1lo01ol1o (@o1lo01ol1o) +- Posted: 2026-07-17 14:33:38 + +## Branch + +**1/** **@o1lo01ol1o** ^2078125981114433630 + +**@VictorTaelin** + +Not true, weight decay explicitly forces sparseification of parameters during training. This for forces the “grokking effect” + +## Related + +- Spine: [[archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter]] diff --git a/archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter/2026-07-17-software_is_art-in-rust-ive-ended-up-making-a-code-garbage.md b/archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter/2026-07-17-software_is_art-in-rust-ive-ended-up-making-a-code-garbage.md new file mode 100644 index 0000000..4cbdd2b --- /dev/null +++ b/archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter/2026-07-17-software_is_art-in-rust-ive-ended-up-making-a-code-garbage.md @@ -0,0 +1,42 @@ +--- +title: "@VictorTaelin In Rust I’ve ended up making a code garbage compactor." +type: archive +source: twitter +source_url: "https://x.com/software_is_art/status/2078221834059915275" +author: "Callum Galbreath" +handle: software_is_art +post_id: "2078221834059915275" +date: 2026-07-17 +archived: 2026-08-25 +draft: false +tags: + - archive + - twitter + - VictorTaelin +description: "@VictorTaelin In Rust I’ve ended up making a code garbage compactor." +in_reply_to: "" +parent_post_id: "2078110851123286452" +--- + +## Source + +- URL: https://x.com/software_is_art/status/2078221834059915275 +- Author: Callum Galbreath (@software_is_art) +- Posted: 2026-07-17 20:54:31 + +## Branch + +**1/** **@software_is_art** ^2078221834059915275 + +**@VictorTaelin** + +In Rust I’ve ended up making a code garbage compactor. + +Before opening a PR the tool uses a bottom up module traversal technique that iteratively: +1. Changes all pub members to pub(mod) +2. Compiles +3. Build errors show what to keep / delete + +## Related + +- Spine: [[archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter]] diff --git a/archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter/2026-07-17-theproteindance-i-got-some-very-interesting-results-recently.md b/archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter/2026-07-17-theproteindance-i-got-some-very-interesting-results-recently.md new file mode 100644 index 0000000..b7014ba --- /dev/null +++ b/archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter/2026-07-17-theproteindance-i-got-some-very-interesting-results-recently.md @@ -0,0 +1,41 @@ +--- +title: "@VictorTaelin I got some very interesting results recently about erasure rates in vocabulary for codebases." +type: archive +source: twitter +source_url: "https://x.com/theproteindance/status/2078127413100269962" +author: "blue" +handle: theproteindance +post_id: "2078127413100269962" +date: 2026-07-17 +archived: 2026-08-25 +draft: false +tags: + - archive + - twitter + - VictorTaelin +description: "@VictorTaelin I got some very interesting results recently about erasure rates in vocabulary for codebases." +in_reply_to: "" +parent_post_id: "2078110851123286452" +--- + +## Source + +- URL: https://x.com/theproteindance/status/2078127413100269962 +- Author: blue (@theproteindance) +- Posted: 2026-07-17 14:39:19 + +## Branch + +**1/** **@theproteindance** ^2078127413100269962 + +**@VictorTaelin** + +I got some very interesting results recently about erasure rates in vocabulary for codebases. AI-generated codebases were much more conservative. + +Centaurs (AI + human code touchers) had the highest rates of vocab deletion and compaction. + +![](https://pbs.twimg.com/media/HNb9q6wWwAADT3R?format=jpg&name=orig) + +## Related + +- Spine: [[archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter]] diff --git a/archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter/2026-07-17-traits_reality-100-post-training-currently-rewards-one-shot.md b/archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter/2026-07-17-traits_reality-100-post-training-currently-rewards-one-shot.md new file mode 100644 index 0000000..46e4866 --- /dev/null +++ b/archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter/2026-07-17-traits_reality-100-post-training-currently-rewards-one-shot.md @@ -0,0 +1,42 @@ +--- +title: "@VictorTaelin 100%" +type: archive +source: twitter +source_url: "https://x.com/traits_reality/status/2078112131598225766" +author: "sin-ack" +handle: traits_reality +post_id: "2078112131598225766" +date: 2026-07-17 +archived: 2026-08-25 +draft: false +tags: + - archive + - twitter + - VictorTaelin +description: "@VictorTaelin 100%" +in_reply_to: "" +parent_post_id: "2078110851123286452" +--- + +## Source + +- URL: https://x.com/traits_reality/status/2078112131598225766 +- Author: sin-ack (@traits_reality) +- Posted: 2026-07-17 13:38:36 + +## Branch + +**1/** **@traits_reality** ^2078112131598225766 + +**@VictorTaelin** + +100% +post-training currently rewards one-shot solutions instead of long-term maintenance, because +1) it's extremely hard to quantify maintenance/long-term thinking +2) it's extremely subjective +3) survivorship bias +even your tweet need a refactor + +## Related + +- Spine: [[archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter]] diff --git a/archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter/2026-07-18-eshear-a-useful-elaboration-whether-something-should-be.md b/archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter/2026-07-18-eshear-a-useful-elaboration-whether-something-should-be.md new file mode 100644 index 0000000..da29096 --- /dev/null +++ b/archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter/2026-07-18-eshear-a-useful-elaboration-whether-something-should-be.md @@ -0,0 +1,37 @@ +--- +title: "@VictorTaelin A useful elaboration: whether something should be deleted goes with the square root of its age." +type: archive +source: twitter +source_url: "https://x.com/eshear/status/2078329276093604270" +author: "Emmett Shear" +handle: eshear +post_id: "2078329276093604270" +date: 2026-07-18 +archived: 2026-08-25 +draft: false +tags: + - archive + - twitter + - VictorTaelin +description: "@VictorTaelin A useful elaboration: whether something should be deleted goes with the square root of its age." +in_reply_to: "" +parent_post_id: "2078110851123286452" +--- + +## Source + +- URL: https://x.com/eshear/status/2078329276093604270 +- Author: Emmett Shear (@eshear) +- Posted: 2026-07-18 04:01:27 + +## Branch + +**1/** **@eshear** ^2078329276093604270 + +**@VictorTaelin** + +A useful elaboration: whether something should be deleted goes with the square root of its age. You should new delete stuff quite readily, and older stuff much less readily. Age of course being “number of times it has been loaded”. + +## Related + +- Spine: [[archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter]] diff --git a/archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter/2026-07-18-rom1504-yes-but-this-is-missing-what-you-compress-i-think.md b/archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter/2026-07-18-rom1504-yes-but-this-is-missing-what-you-compress-i-think.md new file mode 100644 index 0000000..816614d --- /dev/null +++ b/archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter/2026-07-18-rom1504-yes-but-this-is-missing-what-you-compress-i-think.md @@ -0,0 +1,40 @@ +--- +title: "@VictorTaelin Yes but this is missing *what* you compress." +type: archive +source: twitter +source_url: "https://x.com/rom1504/status/2078291890940653988" +author: "Romain Beaumont" +handle: rom1504 +post_id: "2078291890940653988" +date: 2026-07-18 +archived: 2026-08-25 +draft: false +tags: + - archive + - twitter + - VictorTaelin +description: "@VictorTaelin Yes but this is missing *what* you compress." +in_reply_to: "" +parent_post_id: "2078110851123286452" +--- + +## Source + +- URL: https://x.com/rom1504/status/2078291890940653988 +- Author: Romain Beaumont (@rom1504) +- Posted: 2026-07-18 01:32:54 + +## Branch + +**1/** **@rom1504** ^2078291890940653988 + +**@VictorTaelin** + +Yes but this is missing *what* you compress. +I think it should be about compressing future code while keeping constant or increasing the amount of purpose. + +Explore the full tree of possibilities and make is smaller to encode. + +## Related + +- Spine: [[archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter]] diff --git a/archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter/index.md b/archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter/index.md new file mode 100644 index 0000000..6fb684a --- /dev/null +++ b/archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter/index.md @@ -0,0 +1,80 @@ +--- +title: "> Read this and your next model will be 10x smarter." +type: archive +source: twitter +source_url: "https://x.com/VictorTaelin/status/2078110851123286452" +author: "Taelin" +handle: VictorTaelin +post_id: "2078110851123286452" +date: 2026-07-17 +archived: 2026-08-25 +draft: false +tags: + - archive + - twitter + - VictorTaelin +description: "> Read this and your next model will be 10x smarter." +in_reply_to: "" +--- + +## Source + +- URL: https://x.com/VictorTaelin/status/2078110851123286452 +- Author: Taelin (@VictorTaelin) +- Posted: 2026-07-17 13:33:30 + +## Thread + +**1/** **@VictorTaelin** ^2078110851123286452 + +> Read this and your next model will be 10x smarter. < + +Nobody knows what intelligence truly is. We just know models are converging to being smarter, as they train. Yet, we DO know some of the fundamental features of intelligence. And when one of these features is neglected or not trained for, then there is no way for a model to obtain it. Neglecting an aspect of intelligence hinders a model's general capabilities, in a way no amount of flops can compensate for. + +I'm making this whole post to convince you there is ONE fundamental aspect of intelligence that YOU are neglecting, underestimating, and under-training for. Anyone using models 24/7 can see this weakness. It is blinding, glaring, as clear as skylight. + +That feature is: ✨ erasure ✨ + +Removal. Compression. Garbage collection. + +Models are not sufficiently trained for that. +They are trained to ADD information. +Not to REMOVE it. + +You ask a question. They give you an answer. + +They work in a project. They write files. + +You post a bug. They craft a solution. + +They're only indirectly, if at all, rewarded for removing information, or compressing information. This is a huge mistake, because erasure is a cornerstone of intelligence. + +The human brain has several mechanisms entirely dedicated to removing information. Short term memory, long term memory, sleep, all mechanisms to throw garbage away. Furthermore, grokking is nothing but a compression event. An aha-moment happens when your brain is capable of expressing new information in terms of information you already posses stored. This is what allows that info to be stored. That is how you learn. + +Erasure isn't a small feature, erasure is *THE* underlying driver of intelligence. It is what allows us to keep absorbing tons of information and still managing to turn it into useful capabilities. Intelligence is not about producing good knowledge, it is about removing bad knowledge. So, erasure is half of it. + +So, my advice to you: take erasure seriously. Train on it. The architecture is fine. It can lead to AGI. But you won't be a complete athlete if you skip leg day. Reward your model on the other half of intelligence. Teach it how to erase and compress information competently. Make this a big program in your company. Have entire teams dedicated to this. + +"I'm already kinda doing that!" + +No you are not. And if you think you are, take this as a signal you should do 100x more of it. I want to be very clear here: erasure is HALF of intelligence. So, if not half of your FLOPS are flowing into erasure, you're wasting your GPUs, and no optimizer can compensate for that. + +"But how do I teach a model to erase?" + +Literally, just ask it to compress a text, then reconstruct it, and ask questions to assert how lossless the conversion was. That simply. You can do that in any dataset. + +For coding, a more effective way is to take a big codebase and ask it to make it shorter, while still preserving the same behavior. IMPORTANT: avoid code-golfing / minification / uglification. Removing comments or making variable names shorter IS reward hacking. Counter that by counting the NUMBER OF BRANCHES. A branch is: an "if", a "match", a "case". That's THE complexity of your program. + +Count it, and ask the model to reduce it. There's no way to do so other than building better abstractions. And a better abstraction is nothing more than a blob of information that lets you throw other information away, because it expands into the information that was just discarded. Train on that, and your model will be incentivized to build better abstractions. Do you know what we call humans capable of building better abstractions? Geniuses. + +So, please: appreciate the full nature of intelligence and give your models the rewards they need to train on all of it. Let erasure be a major part of your training programs. Do not skip leg day. + +Thanks for coming to my TED talk... + +Branches: [[archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter/2026-07-17-traits_reality-100-post-training-currently-rewards-one-shot]], [[archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter/2026-07-17-Rafa_Schwinger-i-think-anthropic-is-already-doing-something]], [[archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter/2026-07-17-Jatin_exe-i-love-how-well-articulated-this-post-is-counter]], [[archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter/2026-07-17-o1lo01ol1o-not-true-weight-decay-explicitly-forces]], [[archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter/2026-07-17-theproteindance-i-got-some-very-interesting-results-recently]], [[archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter/2026-07-17-DaleCloudman-ya-compress-the-ast-size-not-the-character-count]], [[archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter/2026-07-17-ShumingHu-dl-is-not-built-for-erasure-as-much-as-we-dont]], [[archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter/2026-07-17-dkubb-ive-been-spending-a-lot-of-time-on-reducing-the]], [[archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter/2026-07-17-jakelevi_ml-i-think-code-golfing-minification-uglification-is]], [[archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter/2026-07-17-capitalist_qol-ive-gotten-rid-of-compaction-in-my-loops-in-favor]], [[archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter/2026-07-17-asefweca-i-think-they-already-do-erasure-a-lot-on-every]], [[archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter/2026-07-17-WilmsRolf-intelligence-is-not-just-the-ability-to-reason]], [[archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter/2026-07-17-LeeeeeeeeeT_-a-related-issue-that-bothers-me-is-that-the]], [[archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter/2026-07-17-software_is_art-in-rust-ive-ended-up-making-a-code-garbage]], [[archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter/2026-07-18-rom1504-yes-but-this-is-missing-what-you-compress-i-think]], [[archive/threads/VictorTaelin/2026-07-17-read-this-and-your-next-model-will-be-10x-smarter/2026-07-18-eshear-a-useful-elaboration-whether-something-should-be]] + +**2/** **@VictorTaelin** ^2078167633904463995 + +I HAVE NOT SEEN ANY 3B1B VIDEO + +I've been ranting about this for years :|