本文へ移動
cccskills
無料GitHub で公開

sam3sapiens2-pose

Use when working on the SAM3 plus Sapiens2 pose estimation pipeline, batch GPU processing, or VRAM and throughput issues.

インストール方法を見る

含まれるファイル(1)

  • SKILL.md3.0 KB

SKILL.md(原文)

インストールする前に、エージェントに与えられる指示の中身を確認できます。

SAM3-Guided Sapiens2 Pose Pipeline (vailá)

Use when working on SAM3 + Sapiens2 pose estimation, running batch CLI video pipelines (vaila/sam3sapiens2.py), tuning batch sizes for NVIDIA GPUs (RTX 4090 / 3090 / 5050), configuring headless GPU mode via gpumode, or diagnosing VRAM/throughput bottlenecks.

Overview

vaila/sam3sapiens2.py is a top-down markerless pose pipeline integrating:

  • SAM3 (Segment Anything Model 3): Serves as the authority for person detection, tracking, silhouette extraction, and persistent obj_ids.
  • Sapiens2 Pose (1B / 5B / 0.8b / 0.4b ViT): Runs top-down on SAM3-generated crops and masks (308 body/face/hand keypoints). DETR detector is disabled.

CLI Execution Pattern

# Standard batch run on directory:
uv run python -u vaila/sam3sapiens2.py \
    -i /path/to/videos \
    -o /path/to/output_dir \
    -t person \
    --model 1b \
    --stride 1 \
    --device 0 \
    --kpt-thr 0.3 \
    --bbox-padding 0.12 \
    --contour-margin 8 \
    --max-persons 1 \
    --pose-batch-size 16

# With Test-Time Augmentation (Flip-Test) for highest joint accuracy:
uv run python -u vaila/sam3sapiens2.py \
    -i /path/to/videos \
    -o /path/to/output_dir \
    -t person \
    --model 1b \
    --pose-batch-size 16 \
    --flip-test

GPU & Batch Sizing Benchmark (RTX 4090 24GB)

Measured on Sapiens2 1B (308 keypoints, 1024x768 crop resolution):

Batch SizeThroughput (crops/s)VRAM AllocatedVRAM Reserved (Peak)Safety Margin
42.45 crops/s7.93 GiB9.57 GiB~14.4 GiB free
82.67 crops/s10.33 GiB13.54 GiB~10.5 GiB free
122.66 crops/s12.72 GiB17.40 GiB~6.6 GiB free
162.67 crops/s15.11 GiB21.22 GiB~2.8 GiB free
202.61 crops/s17.51 GiB20.01 GiB~4.0 GiB free
242.58 crops/s18.68 GiB22.73 GiB~1.3 GiB free

Key Rules & Recommendations:

  1. Saturation: ViT 1B saturates GPU Tensor Cores at Batch 8 to 16 (~2.67 crops/s). Higher batch sizes (e.g. 24) do not increase throughput and risk CUDA OOM during long multi-video runs.
  2. Optimal Batch: Use --pose-batch-size 16 for high throughput with sufficient headroom.
  3. Single Athlete Videos: In videos with 1 athlete, batch size per frame is 1 crop (~7.19 GiB VRAM). --max-persons 1 prevents spurious multi-crop overhead.
  4. --flip-test Impact: Doubles forward passes (2x compute time), negligible VRAM change, provides higher anatomical accuracy for challenging sports movements.

Headless / CLI GPU Management (gpumode)

To free all VRAM from desktop compositors (Xorg/Wayland) on dedicated NVIDIA setups:

# Switch to CLI mode (multi-user.target, frees 100% VRAM):
sudo gpumode --cuda

# Return to GUI desktop:
sudo gpumode --desktop

レビュー

まだレビューはありません。使ってみた感想をお寄せください。

同じリポジトリのスキル

概要と使いどころ

Configures best-practice alerting policies for AI agents using OpenTelemetry (OTel) metrics, generating output as Terraform (.tf) configuration files. Use when analyzing, writing, or deploying alerting policies to monitor agent latency, error rates, token usage, and quality metrics. Don't use for standard infrastructure monitoring unrelated to AI agents, or when the agent is not instrumented with OpenTelemetry (for Reliability, Cost, Safety, Security alerts). NOTE: Reliability, Cost, Safety, and Security alerts use generic OTel metrics and work across runtimes (such as Cloud Run, Vertex AI). Quality alerts rely on Vertex AI Online Monitors and are strictly bound to Vertex AI deployments.

日本語の概要は準備中です。原文の説明を表示しています。

vaila-multimodaltoolbox/vaila192026年10月8日 更新

Deploy open models or custom weights from Model Garden to Agent Platform endpoints, check the status of an in-progress deployment operation, or clean up resources by undeploying models and deleting endpoints. Use when asked to actively deploy a model, list the Model Garden CATALOG of available models, check if a specific model is deployable (`gcloud ai model-garden models list-deployment-config`), query deployment cost, troubleshoot deployment errors (like quota limits), or undeploy/clean up endpoints. Also use when copying and deploying a 1P Tuned Model. Don't use for pure listing/discovery questions of the form "is X deployed?", "list my endpoints", or "which regions have models running?" — for those use `agent-platform-endpoint-management`. Don't use for public Vertex AI deployments (use the `vertex-deploy` skill) or for running model evaluations (use the `agent-platform-eval-flywheel` skill).

日本語の概要は準備中です。原文の説明を表示しています。

vaila-multimodaltoolbox/vaila192026年10月8日 更新

Manages Agent Platform serving endpoints. Use when you need to create, list, describe, update, or delete serving endpoints for model deployment on Agent Platform. Also use when troubleshooting endpoint permission, quota, or resource busy errors. Don't use for deploying models to endpoints or for running model evaluations.

日本語の概要は準備中です。原文の説明を表示しています。

vaila-multimodaltoolbox/vaila192026年10月8日 更新

Measures and improves the quality of AI models and agents on Google Cloud using the Eval Quality Flywheel methodology. Use when evaluating an agent or model, building an eval dataset, picking or writing evaluation metrics, analyzing failures, comparing results before and after a fix, or when guidance is needed on Agent Platform eval methodology — including dataset schema, LLM-as-judge scoring, and common failure causes. For fine-tuning, use agent-platform-tuning. For general production deployment, use agent-platform-deploy.

日本語の概要は準備中です。原文の説明を表示しています。

vaila-multimodaltoolbox/vaila192026年10月8日 更新

Connects to and performs inference with Google Cloud Agent Platform GenAI models, including First-Party Gemini models and Third-Party OpenMaaS models (Llama, DeepSeek, Qwen, etc.). Use when asked to perform inference, ask a model a question, run a test prompt, execute chat completions, or generate code for calling Gemini or OpenMaaS models, authenticate with GenAI SDK, OpenAI SDK, or legacy Agent Platform SDK, configure base URLs and global/regional endpoints, or troubleshoot 429 Resource Exhausted (DSQ), 400 User Validation, or 404 Not Found errors. Don't use for deploying models to endpoints or for running model evaluations.

日本語の概要は準備中です。原文の説明を表示しています。

vaila-multimodaltoolbox/vaila192026年10月8日 更新

Guides agents and users through migrating from Gemini API in Google AI Studio to Gemini Enterprise Agent Platform (formerly Vertex AI). Use this skill when moving applications to Google Cloud, to leverage Cloud credits, or to unify inferencing with other Cloud infrastructure (IAM, billing, telemetry).

日本語の概要は準備中です。原文の説明を表示しています。

vaila-multimodaltoolbox/vaila192026年10月8日 更新

vaila-multimodaltoolbox のスキルをすべて見る

このスキルの問題を報告する