Scaling Laws, Self‑Play, and Agentic Workflows: AI Club Highlights

The meetup opened with a reminder that training on human‑generated data limits models to the “typical set” of solutions. As one speaker put it, “If the full solution space is F, training on known human solutions will limit you to some typical set H… You won't feasibly sample F minus H.” This framing leads to two core efficiency questions: how to increase “intelligence per sample” and how to boost “intelligence per watt” without simply scaling compute forever.

AI for Biology

Protein sequences are treated as a 20‑letter alphabet language. Large‑scale language models such as the ESM Cambrian family have been trained on 2.8 billion metagenomic sequences, revealing log‑linear scaling laws that mirror those of text LLMs. Crucially, these sequence‑only models now rival AlphaFold 3 even without hand‑built Multiple Sequence Alignments (MSAs). The latent spaces of the models spontaneously organize hierarchical biological concepts—from individual amino acids up to functional roles—supporting the claim that “you’ll know a protein by amino acids it keeps.”

Self‑Play for LLMs

Traditional reinforcement learning plateaus quickly, prompting a shift toward self‑play. The discussion distinguished symmetric self‑play (e.g., AlphaGo) from asymmetric self‑play, where a Conjecturer creates problems and a Solver attempts them. Vanilla self‑play often produces “messy, artificially complex” tasks that do not aid learning. The proposed Self‑Guided Self‑Play (SGS) introduces a Guide that evaluates each generated problem, filtering out junk and keeping the difficulty aligned with target tasks. Using SGS, a 7 B‑parameter model matched the performance of a 70 B model on formal mathematics benchmarks.

Streaming Retrieval‑Augmented Generation (RAG)

Standard RAG adds unacceptable latency to voice assistants. The club presented a streaming RAG approach that processes audio in small blocks, launching retrieval as soon as the partial query reaches sufficient semantic relevance. This early‑trigger strategy reduced latency by 0.5 seconds in synthetic tests and by up to 1.5 seconds with real users, making voice AI feel more responsive.

Formal Math & Verification

Lean, a functional programming language and interactive theorem prover, enables “verifiable coding” where every piece of generated code is accompanied by a formal proof checked by the Lean kernel. This contrasts with “wide coding,” which merely produces large volumes of code without guarantees of correctness. Tools such as TorchLean extend verification to neural network components, allowing proofs of properties like attention‑mechanism invariants. The Mathlib library now contains roughly one million lines of formalized mathematics, illustrating the scale of verified knowledge that can be leveraged.

Agentic Engineering (RTS‑Style Development)

Software development was reframed as a real‑time‑strategy (RTS) game. Parallel agents, orchestrated by Claude, operate on a shared “work tree” that branches into many concurrent tasks. The focus shifts to “macro”—spawning many agents—to maximize Actions Per Minute (APM), while “micro” interventions occur only when critical. High‑visibility dashboards and audio cues provide early warnings, enabling rapid course correction. This RTS‑style workflow yielded a 3.5× increase in pull‑requests per engineer per month.

Closing Thoughts

Across biology, language modeling, and software engineering, the recurring theme is moving from brute‑force scaling toward smarter, more efficient mechanisms. Whether it is leveraging asymmetric self‑play, streaming retrieval, formal verification, or agentic parallelism, the goal remains the same: achieve higher intelligence per sample and per watt while keeping the learning signal clean and actionable.