本文へ移動
cccskills
無料GitHub で公開

add-model-arch

Use when adding support for a new model architecture to imp, porting a model family, or debugging a model that loads but produces wrong output - "add support for <model>", "new arch", loader detection, chat template, tokenizer parity, RoPE variant, "outputs garbage", "prompt-blind", "digits scrambled", "NaN logits", "describes a different picture", "does it fit in VRAM". Do NOT use for kernel performance (sm120-cuda-expert) or quant-format questions (quant-formats).

kekzl/imp
インストール方法を見る

含まれるファイル（1）

  • SKILL.md7.5 KB

SKILL.md（原文）

インストールする前に、エージェントに与えられる指示の中身を確認できます。

Adding a Model Architecture - imp

First: is it a new arch at all?

Diff config.json against a supported sibling before estimating. Qwen3.8 shipped with zero enum members (loads as QWEN35); the work was tokenizer parity, template goldens, KV dtype default and MTP head (#1750). HF reference values come from curl, never from memory.

Integration checklist (gpt-oss #572 is the reference PR)

StepWhereNotes
1. Enum + registryModelArch in src/model/model_arch.h; parse_model_arch (GGUF general.architecture in gguf_loader.cpp, HF architectures/model_type in hf_config_loader.cpp; arch-only config.json keys: one file in src/model/hf_config/ + a kHfConfigHooks row in arch_registry.cpp), model_arch_name, apply_arch_defaults, sampling defaults in src/model/model.cppthen ModelProfile (src/model/model_profile.h/.cpp, SSoT since #622/#623; AttnVariant { STANDARD, GEMMA4_SWA, GPTOSS_SWA, NOPE, MLA }). No new cfg.arch == X in hot paths. Add the arch to the KV-dtype lists kv_nvfp4_default_safe, kv_fp8_hint_default_safe, kv_fp8_no_hint_default_safe (evidence per family in model.cpp): a missing entry silently gets FP16 KV, which on a GDN hybrid gates context
2. Loadersrc/model/tensor_kind_matcher.cpp, weight_map.cpp; SafeTensors safetensors_loader.cpp; NVFP4 via llm_compressor_loader.cpp (compressed-tensors) or Modelopt (hf_quant_config.json)the two NVFP4 layouts have RECIPROCAL tensor scales (quant-formats)
3. Arch configmodel_config.h + apply_arch_defaultsRoPE pair layout (NeoX vs GPT-J), YaRN/rope_freq_scale, SWA layer pattern, NoPE, sinks, softcap, norm placement, MoE router
4. Chat templatefamily in src/model/chat_template_families.cpp (ChatTemplateFamily in chat_template.h), rendering in chat_template.cpp (+ jinja.cpp)a new family needs a golden pin (make chat-goldens, tests/refs/chat_template_goldens.h, nine families since #1721, three more Jinja gaps fixed in #1701; Jinja fails SILENTLY). reasoning_effort must reach ChatTemplate::apply*/render_jinja + the server snapshot: identical prompt-token counts across efforts = not threaded (#1750: 67/67 before, 41/11/53 after)
5. Tokenizer paritytemplate tests/test_tokenizer_qwen38.cpp (32/32 encode+decode vs HF), tests/test_qwen38_chat_template.cpp; harness tools/tokenizer_parity/run.sh (from #2159, branch fix/tokenizer-hf-parity until merged)BERT-family GGUFs use SPM, not WordPiece. An HF reference built from a GGUF tokenizer can split special tokens (<think> -> 6 text ids instead of 151667): compare token ids before logits
6. Kernelsonly for genuinely new ops; check src/exec/ + src/compute/ firstnew RoPE variants go into src/compute/rope_yarn.cuh (shared with MTP heads, #913)
7. Verifyloads -> greedy coherent (check-degeneration) -> imp-cli --perplexity vs HF (within ~10-20%, often closer: gpt-oss 4.68 vs bf16 4.607, #663) -> decode/prefill sanity (benchmark-cuda)PPL with runtime.deterministic=true, speculative.mtp_k=0, ppl_corpus_45k.txt (tools/analysis/make_ppl_corpus.sh)
8. Docsdocs/MODELS.md row (+ docs/BENCHMARKS.md if hero); known gaps to docs/LIMITATIONS.md; perf baseline entry if gated

A new checkpoint is UNTRUSTED INPUT: SafeTensors/tokenizer.json parsers are hardened (#1660, #1694); fuzz targets under fuzz/; no parsing shortcuts around the bounds checks.

Wrong-output fingerprints

SymptomRoot-cause classCase
Fluent but ignores the prompt ("prompt-blind")RoPE pair layout: HF SafeTensors need rope_neox=true, GGUF pre-permutes Q/KSafeTensors Llama/Mistral, #503
Words fine, digits scrambledposition encoding (NoPE layer as RoPE or vice versa)Nemotron-H rope_attn_disabled, #518
Argmax always token 0NaN logits upstream (residual overflow, bad scale)gpt-oss FP16 residual
Coherent to ~1k ctx, then garbageYaRN/rope_freq_scale inverted or fused-rope path without YaRNgpt-oss 1024x error, #572
Wrong only with chunked prefill at long ctxcontinuation-chunk path#553
Wrong language / valid-but-wrong tokensweight upload / dequant layout (MoE: weight_upload.cpp expert promotion first)Qwen3.6-35B NVFP4, #925
Garbage from token 0 (!!!)silent VRAM-alloc failure in a decode fallbackMXFP4 GDN hybrids, #935
Multimodal: describes a DIFFERENT pictureM-RoPE per-token (t,h,w) layout, src/model/mrope_positions.cppQwen3-VL
Vision fluent but generictower loaded partly or embeddings never reach the sequence: tools/analysis/vision_sight_check.py
Drift only at very long positionsYaRN float trap: __sinf/__cosf on an unreduced argument; long-ctx tests run ext_factor=0 and cannot see it#1704
Coherent-ish, drifts vs HFMLA/YaRN rope_mscale on the wrong base (1.261x); pin the transformers oracle (4.44.2 for MLA)#880
Draft accept collapses, output correctMTP head RoPE differs from the target (NeoX vs YaRN)#913
CLI fine, server brokennot arch: server-api
Which block divergesdiagnostics.dump_hidden_dir + tools/analysis/layer_diff.py (vs llama.cpp llama-eval-callback) or layer_ab_diff.py (two imp runs); dump_gdn_state_dir, dump_logits_dir0 non-finite GDN states over a 46579-token prefill on Qwen3.8

Known traps

  • rope_neox: GGUF converters pre-permute Q/K; HF SafeTensors do not.
  • swa_layers was Gemma-only hardcoded once; verify per-layer attention type on any interleaved-SWA arch.
  • The fused rope+KV-write kernel must apply the same YaRN scaling as the standalone path.
  • Arch control tokens (Harmony channels) must not land on the banned list; the spec-verify argmax applies the mask since #1796.
  • Gemma-4: per-layer rope_freqs for non-SWA layers, n_rot=hd.
  • GDN/hybrid state: BF16 storage + FP32 arithmetic is the default (gdn.state_bf16, #1776/#1778); FP16 state NaNs at depth; the old "must be FP32" was a layout bug.
  • Multimodal checkpoints wrap the LM under model.language_model.* (Qwen3.5-VL, #647): strip in the loader or every tensor "is missing".
  • Encoder/embedding archs are supported (nomic-bert #867, cosine 0.999 vs HF).
  • VRAM arithmetic: card 32 607 MiB; CUDA primary context ~1680 MiB; library reserve is a per-model MEASURED value cached in src/memory/library_reserve_cache.h (#1119; vram.library_reserve_cache, override vram.library_reserve_mb); the kMeasuredLibraryReserveBytes ~3900 MiB constant in src/memory/plan.h is the first-run fallback (measured 0 MiB on Qwen3-4B-IQ4_NL, 7460 on Qwen3-8B-Q8_0). An MTP head adds ~0.79 GiB when speculative.mtp_k auto takes it. Peaks per config: docs/internals/MEMORY.md.
  • Getting a checkpoint onto the NVFP4 path: scripts/stage-model.sh (download + imp-quantize in one command; imp fetches nothing itself).

After it works

make verify-fast; a DegenerationTest-compatible probe prompt; think-channel archs through tools/analysis/degen_suite.py (think-leak); vision through vision_sight_check.py and make test-vision.

レビュー

まだレビューはありません。使ってみた感想をお寄せください。

ログインしてレビューを書く

同じリポジトリのスキル

概要と使いどころ

benchmark-cuda

無料

Use when benchmarking, profiling, or A/B-testing CUDA kernels or end-to-end perf in the imp inference engine on RTX 5090 (sm_120), including refreshing tests/perf_baseline.json or publishing numbers to docs/BENCHMARKS.md and the README. Triggers on "benchmark kernel", "profile cuda", "ncu", "nsys", "kernel timing", "kernel sum", "occupancy", "bandwidth bound", "compute bound", "roofline", "perf baseline", "is this regression real", "decode dropped", "aggregate throughput", "two-image A/B", "prefill kernel A/B". Do NOT use for writing/optimizing kernel code (sm120-cuda-expert) or output-quality checks (check-degeneration).

日本語の概要は準備中です。原文の説明を表示しています。

kekzl/imp442026年10月11日 更新

building-and-testing

無料

Use when building imp, running its test suite, checking CI status, or debugging build/test failures - "make build", "run the tests", "test-gpu", "verify-fast", GTEST_FILTER, Docker/CUDA toolchain, dependency bumps, "CI is red/blocked", "which gate failed", stale objects / segfault after a header edit, hook edits, docker-entrypoint env vars, determinism or perplexity checks. Do NOT use for benchmarking/profiling (benchmark-cuda) or output-quality batteries (check-degeneration).

日本語の概要は準備中です。原文の説明を表示しています。

kekzl/imp442026年10月11日 更新

check-degeneration

無料

Use when verifying that a model in the imp inference engine produces coherent output without repetition loops, token-stuck states, or state corruption across turns or streams. Triggers on "degenerates", "check degeneration", "repetition loop", "own own own", "stuck token", "empty content", "multi-turn regression", "does it still work", "NIAH", and after enabling CUDA graphs / changing forward pass / MoE routing / KV cache or KV dtype / GDN state or scan / sparse attention / PDL / speculation (MTP, n-gram) / ragged prefill / batched-decode kernels (smallm, producer quantize) / FA2 softmax.

日本語の概要は準備中です。原文の説明を表示しています。

kekzl/imp442026年10月11日 更新

code-graph

無料

Use when a question is about *structure* rather than text - who calls or launches a symbol, where it is defined, what a change would reach, whether something is dead, how a request gets from the API to a kernel. Triggers on "who calls", "who launches this kernel", "where is X defined", "what breaks if I change", "is this still used", "is this dead", "trace the path from X to Y", "blast radius", "what depends on this header". Do NOT use for free-text search (`rg` is better and cheaper) or to open a file whose path you already know.

日本語の概要は準備中です。原文の説明を表示しています。

kekzl/imp442026年10月11日 更新

codebase-audit

無料

Use when auditing the imp codebase for structural debt, dead code, god-objects/files, duplication, flag sprawl, or deciding whether a cleanup is worth shipping - "structure audit", "tech debt", "is this still used", "dead code", "refactor for clarity", "should we split this file", "remove this flag", "file size gate red", "alloc sites gate red". Do NOT use for build/test mechanics (building-and-testing), perf/kernel work (benchmark-cuda / sm120-cuda-expert), or output quality (check-degeneration).

日本語の概要は準備中です。原文の説明を表示しています。

kekzl/imp442026年10月11日 更新

docs-layers

無料

Use when writing, moving or auditing any .md in imp - deciding which file a paragraph belongs in, adding a doc, fixing a stale claim, or when docs_lint.py fails in CI. Covers the four reader layers (L0 README / L1 operators / L2 kernel devs / L3 agents), the HTML-comment metadata header, [PROV:] provenance, the single-source-of-truth map, generated perf blocks, which numbers may appear in prose, plan-doc closure. Triggers on "which doc does this go in", "docs lint failed", "add a doc", "this claim is stale", "update the README", "PROV block", "layer", "STALE.md". Do NOT use for the CHANGELOG or a release body (shipping-prs), or for code-comment accuracy (codebase-audit).

日本語の概要は準備中です。原文の説明を表示しています。

kekzl/imp442026年10月11日 更新

kekzl のスキルをすべて見る

このスキルの問題を報告する