本文へ移動
cccskills
無料GitHub で公開

property-based-testing

Writes, reviews, and debugs property-based tests — Hypothesis, fast-check, proptest, jqwik, rapid, and Echidna or Medusa for Solidity invariants. Use whenever tests should cover a whole input domain instead of a hand-picked list of examples: encode/decode and serialize/deserialize pairs, parsers, canonicalizers and normalizers, validators, numeric and Decimal types, comparators and sort order, data structures, and smart-contract state invariants. Also use when adding cases to an existing @given, fast-check, or proptest suite, when judging whether existing property tests assert anything real, and when a generator has shrunk a counterexample and you need to tell a wrong property from a genuine bug. Not for coverage-guided binary fuzzing (libFuzzer, AFL), mutation-testing campaigns, static analysis, benchmarking, or end-to-end UI tests.

インストール方法を見る

含まれるファイル(9)

  • SKILL.md4.2 KB
  • agents/openai.yaml254 B
  • assets/trail-of-bits-mark.svg3.0 KB
  • README.md15.0 KB
  • references/generating.md2.6 KB
  • references/interpreting-failures.md3.3 KB
  • references/libraries.md2.9 KB
  • references/refactoring.md4.7 KB
  • references/reviewing.md2.8 KB

SKILL.md(原文)

インストールする前に、エージェントに与えられる指示の中身を確認できます。

Property-Based Testing

An example test asserts one point. A property asserts a rule over the whole input domain and lets the generator hunt for the counterexample. That trade is worth making when the code has an algebraic shape — an inverse, an invariant, an oracle — and not otherwise. Code with no such shape gets example tests; saying so is a valid outcome.

Check first whether the shape is missing or merely buried. A calculation wrapped in I/O, a string built by concatenation, an in-place mutation — each has a property and no seam to assert it through. See references/refactoring.md before concluding there is nothing to assert.

Property catalog

PropertyFormulaWhere it applies
Roundtripdecode(encode(x)) == xSerialization, conversion pairs
Inversef(g(x)) == xencrypt/decrypt, compress/decompress
Oraclenew(x) == reference(x)Optimization, refactoring, reimplementation
Idempotencef(f(x)) == f(x)Normalization, formatting, sorting
InvariantHolds before and afterAny transformation, contract state
Easy to verifyis_sorted(sort(x))Complex algorithms with cheap checkers
Commutativityf(a, b) == f(b, a)Binary and set operations
Associativityf(f(a,b), c) == f(a, f(b,c))Combining operations
Identityf(x, e) == xOperations with a neutral element

Strength ordering, weakest to strongest: no crash → type preservation → invariant → idempotence → roundtrip / oracle.

Assert the strongest property the code supports. "No crash" alone rarely justifies the dependency — if that is all you can find, either a small rearrangement exposes something stronger, or the honest report is that this code is a poor PBT candidate. Rule out the first before settling for the second.

The two ways a property test asserts nothing

  • Tautology. assert add(a, b) == a + b restates the implementation; no bug they share can fail it. Pick a property that constrains the function without recomputing it. Note the exception: f(x) == f(x) is a genuine determinism property when f is not obviously pure — serializers over dicts or sets, hashing, anything reading the clock.
  • Vacuity. assume() that filters out nearly every input passes without exercising anything, and self-contradictory assume() passes having run zero cases. Push constraints into the strategy so the generator produces valid inputs directly.

Where to look next

Load the one that matches the task in front of you:

TaskFile
Writing new tests, designing strategiesreferences/generating.md
The code has no property to assert yetreferences/refactoring.md
Reviewing existing property testsreferences/reviewing.md
A property test just failedreferences/interpreting-failures.md
Library choice, Echidna and Medusareferences/libraries.md

Introducing PBT to a project that lacks it

If the project already uses a PBT library, just write the tests in it. If it does not, adding one is a dependency decision that belongs to the user — offer it once with the specific property you would write, and take the answer either way.

レビュー

まだレビューはありません。使ってみた感想をお寄せください。

同じリポジトリのスキル

概要と使いどころ

Builds and runs code under AddressSanitizer to catch buffer overflows, use-after-free, and other memory errors during fuzzing or tests. Covers -fsanitize=address builds, ASAN_OPTIONS, reading the crash report, LeakSanitizer, and the overhead and platform trade-offs. Use when fuzzing C/C++ or Rust that has unsafe blocks or FFI, when debugging a memory corruption crash, or when reading an ASan stack trace.

日本語の概要は準備中です。原文の説明を表示しています。

trailofbits/skills7,4712026年10月10日 更新

aflpp

無料

Sets up and runs AFL++ for multi-core fuzzing of C/C++ projects built with afl-clang-fast or afl-gcc-fast. Covers instrumentation modes, parallel main and secondary campaigns, persistent mode, corpus minimization, and crash triage. Use when scaling fuzzing across cores, fuzzing a mature C/C++ codebase, reading the afl-fuzz status screen, or moving on after libFuzzer has plateaued.

日本語の概要は準備中です。原文の説明を表示しています。

trailofbits/skills7,4712026年10月10日 更新

Audits GitHub Actions workflows for security vulnerabilities in AI agent integrations including Claude Code Action, Gemini CLI, OpenAI Codex, and GitHub AI Inference. Detects attack vectors where attacker-controlled input reaches AI agents running in CI/CD pipelines, including env var intermediary patterns, direct expression injection, dangerous sandbox configurations, and wildcard user allowlists. Use when reviewing workflow files that invoke AI coding agents, auditing CI/CD pipeline security for prompt injection risks, or evaluating agentic action configurations.

日本語の概要は準備中です。原文の説明を表示しています。

trailofbits/skills7,4712026年10月10日 更新

Scans Algorand smart contracts for 11 common vulnerabilities including rekeying attacks, unchecked transaction fees, missing field validations, and access control issues. Use when auditing Algorand projects (TEAL/PyTeal).

日本語の概要は準備中です。原文の説明を表示しています。

trailofbits/skills7,4712026年10月10日 更新

atheris

無料

Sets up and runs Atheris, the coverage-guided Python fuzzer built on libFuzzer. Covers TestOneInput harnesses, FuzzedDataProvider, instrumenting both pure Python and native C extensions, and running under AddressSanitizer. Use when fuzzing a Python package, hunting memory corruption in a Python C extension, or choosing between Atheris and Hypothesis for a Python target.

日本語の概要は準備中です。原文の説明を表示しています。

trailofbits/skills7,4712026年10月10日 更新

Augments Trailmark code graphs with external audit findings from SARIF static analysis results, weAudit annotation files, and version-gated Trailmark 0.4.x binary-analysis graph exports. Maps findings to graph nodes by file and line overlap, creates severity-based subgraphs, and enables cross-referencing findings with pre-analysis data (blast radius, taint, etc.). Use when projecting SARIF results onto a code graph, overlaying weAudit annotations, importing binary graph findings, cross-referencing Semgrep, CodeQL, or binary-analysis findings with call graph data, or visualizing audit findings in the context of code structure.

日本語の概要は準備中です。原文の説明を表示しています。

trailofbits/skills7,4712026年10月10日 更新

trailofbits のスキルをすべて見る

このスキルの問題を報告する