Spaces:

CompactAI-O
/

Homepage

Running

App Files Files Community

CompactAI commited on 9 days ago

Commit

3180ab6

verified ·

1 Parent(s): 5209117

Create Claude Code Fixed My Script And I Published Haiku-2 Anyway.html

Browse files

Files changed (1) hide show

Claude Code Fixed My Script And I Published Haiku-2 Anyway.html +174 -0

Claude Code Fixed My Script And I Published Haiku-2 Anyway.html ADDED Viewed

	@@ -0,0 +1,174 @@

+<!DOCTYPE html>
+<html lang="en">
+<head>
+<meta charset="UTF-8">
+<meta name="viewport" content="width=device-width, initial-scale=1.0">
+<title>Claude Code Fixed My Script And I Published Haiku-2 Anyway | FMN-GPT - CompactAI</title>
+<link rel="preconnect" href="https://fonts.googleapis.com">
+<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
+<link href="https://fonts.googleapis.com/css2?family=Geist:wght@400;500;600;700&family=Geist+Mono&display=swap" rel="stylesheet">
+<style>
+    :root {
+        --blue-900: #0a1628;
+        --blue-800: #0f2240;
+        --blue-700: #142d54;
+        --blue-600: #1a3a6b;
+        --blue-500: #2250a0;
+        --blue-400: #3a7bd5;
+        --blue-300: #6ba3f0;
+        --blue-200: #a8c8f5;
+        --blue-100: #d4e4fa;
+        --white: #ffffff;
+        --white-soft: #f0f4fa;
+        --white-muted: #c8d8ec;
+        --grid-line: rgba(255, 255, 255, 0.08);
+        --grid-line-major: rgba(255, 255, 255, 0.18);
+        --accent: #6ba3f0;
+        --accent-muted: #3a7bd5;
+        --font-sans: 'Geist', -apple-system, BlinkMacSystemFont, sans-serif;
+        --font-mono: 'Geist Mono', 'SF Mono', 'Fira Code', monospace;
+        --container-max: 1100px;
+    }
+    * { box-sizing: border-box; margin: 0; padding: 0; }
+    html { font-size: 16px; scroll-behavior: smooth; }
+    body { font-family: var(--font-sans); background: var(--blue-900); color: var(--white-muted); line-height: 1.7; -webkit-font-smoothing: antialiased; }
+    a { color: var(--white); text-decoration: none; transition: color 0.15s ease; }
+    a:hover { color: var(--accent); }
+    .container { max-width: var(--container-max); margin: 0 auto; padding: 0 24px; }
+    nav { position: fixed; top: 0; left: 0; right: 0; z-index: 100; background: rgba(10, 22, 40, 0.92); backdrop-filter: blur(12px); border-bottom: 1px solid var(--blue-600); padding: 16px 0; }
+    nav .container { display: flex; justify-content: space-between; align-items: center; }
+    .nav-brand { font-size: 18px; font-weight: 600; color: var(--white); display: flex; align-items: center; gap: 8px; }
+    .nav-brand span { color: var(--accent); }
+    .nav-links { display: flex; gap: 32px; }
+    .nav-links a { font-size: 14px; font-weight: 500; color: var(--blue-200); }
+    .nav-links a:hover { color: var(--white); }
+    .post { padding: 140px 0 80px; }
+    .post-back { display: inline-block; color: var(--blue-200); font-size: 14px; margin-bottom: 32px; }
+    .post-back:hover { color: var(--accent); }
+    .post-back::before { content: '← '; }
+    .post-meta { display: flex; gap: 12px; margin-bottom: 20px; }
+    .post-date { font-size: 13px; color: var(--blue-200); font-family: var(--font-mono); }
+    .post-tag { font-size: 11px; font-weight: 600; text-transform: uppercase; letter-spacing: 0.05em; color: var(--accent); background: rgba(107, 163, 240, 0.1); padding: 4px 10px; border-radius: 4px; }
+    .post h1 { font-size: 36px; font-weight: 700; color: var(--white); margin-bottom: 32px; line-height: 1.2; letter-spacing: -0.02em; }
+    .post-body p { font-size: 17px; line-height: 1.8; margin-bottom: 24px; color: var(--blue-200); }
+    .post-body p:first-of-type { font-size: 20px; color: var(--white-muted); }
+    .post-body h2 { font-size: 24px; font-weight: 600; color: var(--white); margin: 48px 0 20px; }
+    .post-body blockquote { border-left: 3px solid var(--accent); padding: 20px 24px; margin: 32px 0; background: var(--blue-800); border-radius: 0 8px 8px 0; }
+    .post-body blockquote p { font-size: 16px; font-style: italic; color: var(--blue-200); margin: 0; }
+    .post-body hr { border: none; height: 1px; background: var(--blue-600); margin: 48px 0; }
+    .code-block { background: var(--blue-800); border: 1px solid var(--blue-600); border-radius: 8px; padding: 20px; margin: 24px 0; font-family: var(--font-mono); font-size: 13px; overflow-x: auto; }
+    .code-block .comment { color: var(--blue-200); font-style: italic; display: block; margin-top: 4px; }
+    .cta-box { background: var(--blue-800); border: 2px solid var(--accent); border-radius: 12px; padding: 24px; margin: 32px 0; text-align: center; }
+    .cta-box a { color: var(--accent); font-weight: 600; font-size: 18px; word-break: break-all; }
+    .cta-box a:hover { color: var(--white); }
+    .cta-box p { margin: 12px 0 0; color: var(--blue-200); font-size: 14px; }
+    .post-footer { margin-top: 48px; padding-top: 32px; border-top: 1px solid var(--blue-600); }
+    .post-footer p { font-size: 14px; color: var(--blue-200); font-style: italic; margin: 0; }
+    footer { padding: 40px 0; background: var(--blue-800); border-top: 1px solid var(--blue-600); text-align: center; }
+    footer p { color: var(--blue-200); font-size: 14px; margin-bottom: 8px; }
+    footer a { color: var(--blue-200); }
+    footer a:hover { color: var(--accent); }
+    @media (max-width: 768px) { .post h1 { font-size: 28px; } .nav-links { display: none; } }
+</style>
+</head>
+<body>
+    <nav>
+        <div class="container">
+            <a href="index.html" class="nav-brand"><span>/</span>FMN-GPT</a>
+            <div class="nav-links">
+                <a href="blog.html">Blog</a>
+                <a href="status.html">Model Status</a>
+                <a href="https://huggingface.co/CompactAI" target="_blank">HuggingFace</a>
+            </div>
+        </div>
+    </nav>
+    <main>
+        <article class="post">
+            <div class="container">
+                <a href="blog.html" class="post-back">Back to Blog</a>
+                <header>
+                    <div class="post-meta">
+                        <span class="post-date">2026-04-03</span>
+                        <span class="post-tag">Progress Reports</span>
+                    </div>
+                    <h1>Claude Code Fixed My Script And I Published Haiku-2 Anyway</h1>
+                </header>
+                <div class="post-body">
+                    <p>I asked Claude Code to fix my training script. It fixed almost every bug. Then it added SPIN. Then it made my models more efficient. Then I published Haiku-2. Then I added all the optimizations. Obviously that is what I needed to do.</p>
+                    <blockquote>
+                        <p>When an AI fixes your code better than you can, you have two choices: learn from it or publish first and optimize later. I chose the latter. Obviously.</p>
+                    </blockquote>
+                    <h2>Claude Code To The Rescue</h2>
+                    <p>My training script was a mess. NaN losses. Gradient explosions. Memory leaks that made my GPU cry. I pasted the error logs into Claude Code. It responded with fixes. Not suggestions. Fixes. Actual working code.</p>
+                    <div class="code-block">
+                        <span class="comment"># Before Claude Code</span><br>
+                        try:<br>
+                        &nbsp;&nbsp;&nbsp;&nbsp;train_model()<br>
+                        except:<br>
+                        &nbsp;&nbsp;&nbsp;&nbsp;cry()<br>
+                        <br>
+                        <span class="comment"># After Claude Code</span><br>
+                        try:<br>
+                        &nbsp;&nbsp;&nbsp;&nbsp;train_model()<br>
+                        except NanError:<br>
+                        &nbsp;&nbsp;&nbsp;&nbsp;clip_gradients()<br>
+                        &nbsp;&nbsp;&nbsp;&nbsp;lower_lr()<br>
+                        &nbsp;&nbsp;&nbsp;&nbsp;retry()<br>
+                        <span class="comment"># Still cries sometimes. But less.</span>
+                    </div>
+                    <p>It found bugs I did not know existed. It added error handling I forgot to write. It optimized memory usage I did not know could be optimized. I felt seen. I also felt embarrassed.</p>
+                    <h2>Enter SPIN</h2>
+                    <p>Then Claude Code suggested SPIN. Self-Play Fine-Tuning. An iterative training method that enables models to improve through self-play without additional data. The model generates responses, distinguishes its own outputs from human references, and iteratively refines its behavior.</p>
+                    <p>I had never heard of SPIN. Claude Code had. It explained the algorithm. It showed me the implementation. It added the code to my training loop. I watched as my loss curve started doing something other than screaming into the void.</p>
+                    <blockquote>
+                        <p>SPIN is like having your model debate itself until it gets less wrong. I am not sure if that is efficient or just deeply meta. Either way, it works.</p>
+                    </blockquote>
+                    <h2>Efficiency Gains</h2>
+                    <p>With SPIN enabled and the bugs fixed, my training became more efficient. The loss curve went down. Occasionally. When it felt like it.</p>
+                    <p>Haiku-2 benefited from these improvements. It learned faster. It forgot less. It still outputs pipe characters sometimes. But now it outputs words too. Sometimes. Do not count on it.</p>
+                    <h2>Haiku-2 Is Out</h2>
+                    <div class="cta-box">
+                        <a href="https://huggingface.co/CompactAI/TMLM-Haiku-2" target="_blank">https://huggingface.co/CompactAI/TMLM-Haiku-2</a>
+                        <p>1M parameters. Can speak sometimes. Do not count on it. Open weights. Open hopes. Open questions about why it still says "|fdish|".</p>
+                    </div>
+                    <p>Haiku-2 is live on HuggingFace. It uses the optimized training script. It uses SPIN. It uses everything I added after publishing it. Obviously that is what I needed to do.</p>
+                    <p>What can you expect? Sometimes it answers questions. Sometimes it outputs coherent sentences. Sometimes it remembers that Paris is a city and not a person named Pierre. Progress is slow. Progress is real. Progress is occasionally interrupted by pipe characters.</p>
+                    <h2>The Optimization Timeline</h2>
+                    <p>Let me be clear about the sequence of events:</p>
+                    <div class="code-block">
+                        <span class="comment"># My release strategy in chronological order</span><br>
+                        Step 1: Publish Haiku-2<br>
+                        Step 2: Realize script has bugs<br>
+                        Step 3: Ask Claude Code for help<br>
+                        Step 4: Claude Code fixes everything<br>
+                        Step 5: Add SPIN<br>
+                        Step 6: Optimize everything<br>
+                        Step 7: Wonder why I did not do Step 3 first<br>
+                        <span class="comment"># Obviously this is the correct order.</span>
+                    </div>
+                    <p>Would it have been smarter to optimize before publishing? Probably. Would that have been less fun? Definitely. I chose fun. I chose chaos. I chose publishing first and fixing later. Obviously that is what I needed to do.</p>
+                    <h2>What SPIN Actually Does</h2>
+                    <p>For those who care about the technical details: SPIN enables iterative self-improvement through a self-play mechanism inspired by game theory. The model generates responses, compares them to reference outputs, and updates its weights to favor human-like patterns.</p>
+                    <p>In practice this means Haiku-2 gets better at not being wrong. It still gets wrong. Just less often. Just slightly less confusingly. Just enough that you might mistake it for competence if you are not paying attention.</p>
+                    <h2>Why This Matters</h2>
+                    <p>I am not competing with frontier models. I am competing with my previous self. Haiku-1.3 said "| as the USA | fdish|||||!@|". Haiku-2 says that sometimes too. But it also says "hello" sometimes. That is progress.</p>
+                    <p>Claude Code helped me get there. SPIN helped me get there. Optimizations helped me get there. Publishing first and optimizing later helped me learn valuable lessons about workflow. Obviously that is what I needed to do.</p>
+                    <h2>Final Thoughts</h2>
+                    <p>Haiku-2 is out. It can speak sometimes. Do not count on it. The training script is fixed. SPIN is enabled. Optimizations are applied. All after publication. Obviously that is what I needed to do.</p>
+                    <p>If you download Haiku-2, please be kind. It is trying its best. Its best is not good. But it is trying. Just like me. Just like all of us. Just like obviously what we all need to do.</p>
+                    <hr>
+                </div>
+                <footer class="post-footer">
+                    <p>Current status: Haiku-2 published. Optimizations added. SPIN enabled. Claude Code thanked. Obviously that is what I needed to do.</p>
+                </footer>
+            </div>
+        </article>
+    </main>
+    <footer>
+        <div class="container">
+            <p>Built with curiosity over compute</p>
+            <p>FMN-GPT by <a href="https://huggingface.co/CompactAI" target="_blank">CompactAI</a> | 2026</p>
+        </div>
+    </footer>
+</body>
+</html>