152334Hhttps://152334H.github.io/152334H Personal BlogHugo -- gohugo.ioenSat, 14 Feb 2026 00:00:00 +0800Blog Refurbishmenthttps://152334H.github.io/blog/blog-refurbishment/Sat, 13 Aug 2022 05:30:32 +0100152334Hhttps://152334H.github.io/blog/blog-refurbishment/Under new managementAre exploits free?https://152334H.github.io/blog/kctf-eval/Sat, 14 Feb 2026 00:00:00 +0800152334Hhttps://152334H.github.io/blog/kctf-eval/<p>For various reasons, I had to develop a kernel n-day exploit last week.</p>Truncated Thoughts on 2025https://152334H.github.io/blog/2026/Thu, 01 Jan 2026 00:00:00 +0800152334Hhttps://152334H.github.io/blog/2026/<p>It has been, overall, a pretty shit year.</p>Time To Thinkhttps://152334H.github.io/blog/time-to-think/Tue, 04 Feb 2025 01:02:03 +0800152334Hhttps://152334H.github.io/blog/time-to-think/I don’t remember what it’s like to think with more than 60 seconds of context.

]]>Calculating the Cost of a Google Deepmind Paperhttps://152334H.github.io/blog/scaling-exponents/Tue, 30 Jul 2024 00:00:00 +0800152334Hhttps://152334H.github.io/blog/scaling-exponents/Recently, GDM released a great paper titled, Scaling Exponents Across Parameterizations and Optimizers, in which they conduct over 10,000 LLM training runs to obtain optimal hyperparameters under different regimes.

After reading it (it was great), I wanted to test my understanding of the paper by tallying up all experiments conducted within, calculating the total compute cost it would take to replicate the paper.

]]>DeepSeek Core Readings 0 - Coderhttps://152334H.github.io/blog/deepseek-0/Sun, 30 Jun 2024 00:00:00 +0800152334Hhttps://152334H.github.io/blog/deepseek-0/Paper summary: 1.3B to 33B LLMs on 1/2T code tokens (87 langs) w/ FiM and 16K seqlen. Strong effort in constructing pretraining data from Github from scratch, with repository-level samples. Evals beat OSS code models solidly + GPT-3.5 a bit; Coder-7B > CodeLlama-33B often.

They don’t spend much effort on Instruction tuning. They commit a continued pretrain of DeepSeek LLM -> Coder: I believe it underperforms; they don’t.

]]>DeepSeek Core Readings 1 - LLMhttps://152334H.github.io/blog/deepseek-1/Sun, 23 Jun 2024 00:00:00 +0800152334Hhttps://152334H.github.io/blog/deepseek-1/Paper summary: LLaMA-like 7B/67B pretrain (Base) + SFT&DPO (Chat). 2T tokens with strong CN/EN mix, >1mil SFT examples. Well-executed exploration of scaling laws. Good details about evals and safety. Not much described about their actual data.

]]>Basic tips for remaining conscioushttps://152334H.github.io/blog/remain-conscious/Fri, 12 Apr 2024 12:56:00 +0800152334Hhttps://152334H.github.io/blog/remain-conscious/I don’t think very often. These are some of the things I’ve tried in the past to rectify that.

]]>2023https://152334H.github.io/blog/2023/Sun, 31 Dec 2023 23:25:26 +0800152334Hhttps://152334H.github.io/blog/2023/<p>Let&rsquo;s keep this short.</p>Rough thoughts on Mixtral vs Open Sourcehttps://152334H.github.io/blog/mixtral-vs-oss/Wed, 13 Dec 2023 20:12:34 +0800152334Hhttps://152334H.github.io/blog/mixtral-vs-oss/Here’s a thesis (hypothesis, predicate, etc) to chew on:

The mixture-of-experts paradigm is fundamentally a hinderance to open source development, and mixtral-8x5B+2B will be summarily supplanted by a dense model like llama3/mistral-70b/yi/qwen/… in the near future.

]]>