The Pipeline Is the Moat: +24.5 Points on a Free, On-Prem Model

The ChipCraftX pipeline turns a free, open-weight model into a production RTL engine. Running Qwen3.6-35B on two consumer RTX 3090s - fully on-prem, $0 per run - we scored 86.4% across all 302 NVIDIA CVDP problems. The raw model on its own manages 61.9%. The architecture does the rest: +24.5 points of retrieval, hierarchical decomposition, orchestration, and EDA validation. That's the whole thesis - the model is a commodity you can swap; the pipeline is the product. It's model-agnostic by design, so every gain in open weights is one we inherit for free, at zero marginal cost, on hardware we own. Every result is scored on NVIDIA's own cocotb functional harness - real simulation, not lint, not self-grading.

“The model is the commodity. The pipeline is the product. A free model on hardware we own already does production-grade work - and as open weights get better, we get better for free.”

ChipCraftBrain Paper Published on arXiv

Our research paper — ChipCraftBrain: Validation-First RTL Generation via Multi-Agent Orchestration — is now published on arXiv. It formalizes the architecture behind our benchmark results: a hybrid symbolic-neural system that orchestrates six specialized agents over a 168-dimensional state space, combining pattern retrieval, hierarchical decomposition, and adaptive reinforcement learning to solve RTL generation at production accuracy.

Local Testbench Model: 100% semantic accuracy, zero API cost

We trained a dedicated testbench LLM that replaces all cloud API calls for testbench generation. When it produces a testbench that compiles, it passes simulation 100% of the time — zero semantic failures. Compilation-only failures account for the remaining 7.3% and are recoverable through iteration. This drops per-run testbench cost to zero and unblocks unlimited-budget preference training for the RTL model. The model was fine-tuned on a curated dataset of ~150K verified entries. Verify-before-train discipline is why the model achieves zero semantic failures on compiled outputs: it learned from exclusively correct signal and can expand to novel specs.

“Verification is the moat. Every piece we bring in-house makes the loop tighter.”

CVDP: 54.24% pass rate improvement over SOTA baseline while requiring only 1.28 iterations on average

The previous SOTA relied on fine-tuned models, massive compute budgets, and hundreds of retries. ChipCraftBrain solves the same problems in 1.28 iterations on average — lifting the pass rate by 54.24 percentage points through a fundamentally different approach.

“We invite all other competitors to publish their scores on CVDP and similar benchmarks for fair evaluation of the systems.”

VerilogEval 98.7%, #1 on the Benchmark

ChipCraftBrain achieves 98.72% pass rate on the full 156-problem VerilogEval functional benchmark, surpassing every published and commercial system, including MAGE (95.9%), ChipAgents (97.4%, closed-source), VFlow (83.6%), and CodeV (59.2%).