本文へ移動
cccskills
無料GitHub で公開

train-sentence-transformers

Train or fine-tune sentence-transformers models across `SentenceTransformer` (bi-encoder, dense or static embedding model for retrieval, similarity, clustering, classification, paraphrase mining, dedup, multimodal), `CrossEncoder` (reranker, pair scoring for two-stage retrieval / pair classification), `SparseEncoder` (SPLADE, sparse embedding model for learned-sparse retrieval), and `MultiVectorEncoder` (ColBERT / late-interaction, per-token embeddings scored with MaxSim). Covers loss selection, hard-negative mining, evaluators, distillation, LoRA, Matryoshka, and Hugging Face Hub publishing. Use for any sentence-transformers training task.

インストール方法を見る

含まれるファイル(31)

  • SKILL.md10.3 KB
  • references/base_model_selection.md7.5 KB
  • references/dataset_formats.md6.3 KB
  • references/evaluators_cross_encoder.md5.5 KB
  • references/evaluators_multi_vector_encoder.md6.6 KB
  • references/evaluators_sentence_transformer.md5.8 KB
  • references/evaluators_sparse_encoder.md4.9 KB
  • references/hardware_guide.md5.9 KB
  • references/hf_jobs_execution.md7.1 KB
  • references/losses_cross_encoder.md9.5 KB
  • references/losses_multi_vector_encoder.md10.5 KB
  • references/losses_sentence_transformer.md10.7 KB
  • references/losses_sparse_encoder.md5.1 KB
  • references/model_architectures.md9.0 KB
  • references/prompts_and_instructions.md6.4 KB
  • references/training_args.md15.0 KB
  • references/troubleshooting.md12.9 KB
  • scripts/mine_hard_negatives.py8.3 KB
  • scripts/train_cross_encoder_distillation_example.py9.2 KB
  • scripts/train_cross_encoder_example.py9.0 KB
  • scripts/train_cross_encoder_listwise_example.py10.2 KB
  • scripts/train_multi_vector_encoder_example.py8.8 KB
  • scripts/train_sentence_transformer_distillation_example.py11.9 KB
  • scripts/train_sentence_transformer_example.py7.5 KB
  • scripts/train_sentence_transformer_make_multilingual_example.py11.5 KB
  • scripts/train_sentence_transformer_matryoshka_example.py7.1 KB
  • scripts/train_sentence_transformer_multi_dataset_example.py9.9 KB
  • scripts/train_sentence_transformer_static_embedding_example.py10.3 KB
  • scripts/train_sentence_transformer_with_lora_example.py10.3 KB
  • scripts/train_sparse_encoder_distillation_example.py9.4 KB
  • scripts/train_sparse_encoder_example.py8.1 KB

SKILL.md(原文)

インストールする前に、エージェントに与えられる指示の中身を確認できます。

Train a sentence-transformers Model

This SKILL.md is a router, not a manual. It tells you which references and example scripts to load for your task. The actual content (recommended losses, evaluators, training-script structure, model selection, training-arg knobs, troubleshooting) lives in references/ and scripts/.

Do not synthesize a training script from this file alone. Open the per-type production template (scripts/train_<type>_example.py) and copy it as your starting point. The templates contain load-bearing scaffolding (autocast helper, model-card class, logger silencing list, force=True, seed, TF32, version-compatible imports, named-evaluator metric handling) that prior agent runs have repeatedly missed when rolling their own from a synthesized snippet.

1. Identify the model type

TagClassWhat it doesWhen to pick
[SentenceTransformer]SentenceTransformer (bi-encoder)Maps each input to a fixed-dim dense vectorRetrieval, similarity, clustering, classification, paraphrase mining, dedup
[CrossEncoder]CrossEncoder (reranker)Scores (query, passage) pairs jointlyTwo-stage retrieval (rerank top-100 from bi-encoder), pair classification
[SparseEncoder]SparseEncoder (SPLADE)Sparse vectors over the vocabularyLearned-sparse retrieval, inverted-index backends (Elasticsearch / OpenSearch / Lucene)
[MultiVectorEncoder]MultiVectorEncoder (ColBERT)One embedding per token, scored with MaxSimLate-interaction retrieval, recall gains over bi-encoders at higher storage cost, multimodal (ColPali / ColQwen2)

Tiebreakers when the request is ambiguous: "embedding model" / "vector search" / "similarity" → [SentenceTransformer]. "rerank" / "ranker" / "two-stage" → [CrossEncoder]. "SPLADE" / "sparse" / "inverted index" → [SparseEncoder]. "ColBERT" / "late interaction" / "multi-vector" / "MaxSim" / "ColPali" / "ColQwen" → [MultiVectorEncoder]. If still unclear, ask.

2. Required reading

Read these in full before writing any code. Do not triage by perceived relevance.

Per-type: always required

[SentenceTransformer]

  • references/losses_sentence_transformer.md: loss-to-data-shape mapping, BatchSamplers.NO_DUPLICATES requirement for MNRL-family, Cached* ↔ gradient_checkpointing incompatibility.
  • references/evaluators_sentence_transformer.md: evaluator-to-task mapping, metric_for_best_model key construction (named vs unnamed), per-evaluator primary_metric values.
  • references/model_architectures.md: encoder vs decoder vs static vs Router pipelines, pooling rules (mean / cls / lasttoken), auto-mean-pooling behavior for fresh-start MLM bases.
  • scripts/train_sentence_transformer_example.py: production template. Copy this as your starting point.

[CrossEncoder]

  • references/losses_cross_encoder.md: pointwise / pairwise / listwise / distillation, pos_weight derivation, activation_fn=Identity() mandatory for non-BCE losses (silent eval-rank collapse otherwise).
  • references/evaluators_cross_encoder.md: CrossEncoderRerankingEvaluator recipe, named-evaluator key format eval_{name}_{primary_metric}.
  • scripts/train_cross_encoder_example.py: production template. Copy this as your starting point.

[SparseEncoder]

  • references/losses_sparse_encoder.md: SpladeLoss wrapper requirement, FLOPS regularizer weights, smoke-test active-dim ramp behavior.
  • references/evaluators_sparse_encoder.md: SparseNanoBEIREvaluator (English-only) and the in-domain alternative, eval_{name}_{primary_metric} key format.
  • scripts/train_sparse_encoder_example.py: production template. Copy this as your starting point.

[MultiVectorEncoder]

  • references/losses_multi_vector_encoder.md: MaxSim scoring, scale choice per scoring mode (scale=1.0 for MaxSim, roughly the average query length for MeanMaxSim), MNRL / CachedMNRL / MarginMSE / DistillKLDiv, XTR-vs-ColBERT scoring, CachedMNRL ↔ gradient_checkpointing incompatibility.
  • references/evaluators_multi_vector_encoder.md: MultiVectorNanoBEIREvaluator (English-only) and the in-domain alternative, eval_NanoBEIR_mean_maxsim_ndcg@10 key format, distillation-eval spearman variant.
  • scripts/train_multi_vector_encoder_example.py: production template. Copy this as your starting point.

Cross-cutting: always required (regardless of task)

  • references/training_args.md: TrainingArguments knobs, precision rules (load fp32 + autocast bf16/fp16, never torch_dtype=bfloat16), warmup_steps (float) vs deprecated warmup_ratio, save_steps must be a multiple of eval_steps for load_best_model_at_end, schedulers, HPO, tracker, resume, hub-push variants.
  • references/dataset_formats.md: column-matching rules (label name auto-detection, column-order-not-name), reshaping recipes, hard-negative mining options.
  • references/base_model_selection.md: discovery commands, per-type model namespaces, ModernBERT-family max_seq_length=8192 trap, datasets >= 4 script-loader rejection, non-English starting-point shortcuts.
  • references/troubleshooting.md: symptom-indexed failure recipes. Skim the section headings on every run, even a healthy one. The "Metrics don't improve" and "Hub push fails" entries cover bugs that bite frequently and are cheaper to recognize before they fire than to debug after.

Cross-cutting: load when applicable

  • references/hardware_guide.md: VRAM sizing, multi-GPU, FSDP / DeepSpeed, HF Jobs flavors. Required for >24GB models, multi-GPU, or HF Jobs runs.
  • references/hf_jobs_execution.md: required when running on HF Jobs.
  • references/prompts_and_instructions.md: required when using prompt-tuned bases (E5, BGE, GTE, Qwen3-Embedding, Instructor, Nomic, etc.) or adding query: / passage: style prefixes.

Variant scripts (open when the task matches)

  • [SentenceTransformer] scripts/train_sentence_transformer_<matryoshka|multi_dataset|with_lora|distillation|make_multilingual|static_embedding>_example.py.
  • [CrossEncoder] scripts/train_cross_encoder_<distillation|listwise>_example.py.
  • [SparseEncoder] scripts/train_sparse_encoder_distillation_example.py.
  • Hard-negative mining CLI: scripts/mine_hard_negatives.py.

3. Defaults

Override only if the user specifies otherwise:

  • Local execution. Pitch HF Jobs only if local hardware can't fit the job.
  • Single run. After it completes, propose experimentation if the user would benefit (weak/marginal verdict, "see how high you can push it" framing, etc.). Iteration rules in references/training_args.md (Experimentation section).
  • Public Hub push at end-of-run, wrapped in try-except. On HF Jobs (ephemeral env) ALSO enable in-trainer push (push_to_hub=True + hub_strategy="every_save"). Details in references/hf_jobs_execution.md.

4. Constraints the produced script must satisfy

These are non-negotiable contracts. Implementation lives in the production templates and references. Do not reinvent.

  • Capture the pre-training evaluator score as baseline_eval before trainer.train().
  • Emit a single end-of-run line: VERDICT: WIN|MARGINAL|REGRESSION | score=... | baseline=... | delta=.... A monitor scrapes for this.
  • Silence httpx, httpcore, huggingface_hub, urllib3, filelock, fsspec to WARNING (otherwise HF download URLs flood the agent's context).
  • Tee logs to logs/{RUN_NAME}.log.
  • End with model.push_to_hub(...) wrapped in try/except.
  • Smoke-test before any long run (max_steps=1 + tiny dataset slice). The production templates show one common pattern (SMOKE_TEST env var).
  • [CrossEncoder] Include EarlyStoppingCallback(patience>=3). CE rerankers often peak mid-training and regress.
  • [SparseEncoder] Log query_active_dims / corpus_active_dims on the verdict line. High nDCG with collapsed sparsity is not a win. The keys come back name-prefixed (e.g. ..._query_active_dims). Use suffix matching to pluck them. See the SPARSE production template for the exact pattern.
  • [MultiVectorEncoder] Match scale to the scoring mode on any MNRL-family loss: near 1.0 for unnormalized MaxSim (do not copy scale=20.0 from bi-encoder MNRL), roughly the average query length with length-normalized MeanMaxSim, since each score is divided by its query's token count. XTRScores is a train-only similarity_fct: the evaluators reject it, so evaluation always scores with MaxSim, including for XTR-trained models.

5. Workflow

  1. Identify the model type (§1). Ask if ambiguous.
  2. Load the §2 required-reading files for that type.
  3. Open scripts/train_<type>_example.py and copy it as your starting point.
  4. Replace MODEL_NAME, DATASET_NAME, RUN_NAME, the loss, and the evaluator with the user's task. Cross-check loss/data-shape match against references/losses_<type>.md. Cross-check the metric_for_best_model key against references/evaluators_<type>.md (named evaluators format the key as eval_{name}_{primary_metric}).
  5. Smoke-test (max_steps=1).
  6. Run.
  7. After the run, append to logs/experiments.md and propose iteration if the verdict is weak/marginal.

Prerequisites

pip install "sentence-transformers[train]>=5.0"        # add [train,image] / [audio] / [video] for [SentenceTransformer] multimodal
                                                       # [MultiVectorEncoder] requires >=6.0
pip install trackio                                    # optional tracker (or wandb / tensorboard / mlflow)
hf auth login                                          # or set HF_TOKEN with write scope (for Hub push)

GPU strongly recommended. CPU works only for demos and [SentenceTransformer] StaticEmbedding.

レビュー

まだレビューはありません。使ってみた感想をお寄せください。

同じリポジトリのスキル

概要と使いどころ

hf-cli

無料

Hugging Face Hub CLI (`hf`) for downloading, uploading, and managing models, datasets, spaces, buckets, repos, papers, jobs, and more on the Hugging Face Hub. Use when: handling authentication; managing local cache; managing Hugging Face Buckets; running or scheduling jobs on Hugging Face infrastructure; managing Hugging Face repos; discussions and pull requests; browsing models, datasets and spaces; reading, searching, or browsing academic papers; managing collections; querying datasets; configuring spaces; setting up webhooks; or deploying and managing HF Inference Endpoints. Make sure to use this skill whenever the user mentions 'hf', 'huggingface', 'Hugging Face', 'huggingface-cli', or 'hugging face cli', or wants to do anything related to the Hugging Face ecosystem and to AI and ML in general. Also use for cloud storage needs like training checkpoints, data pipelines, or agent traces. Use even if the user doesn't explicitly ask for a CLI command. Replaces the deprecated `huggingface-cli`.

日本語の概要は準備中です。原文の説明を表示しています。

huggingface/skills1.1万2026年10月9日 更新

Discover the user's local AWS context (active profile, region, account ID, caller identity) at the start of any AWS task. Use this skill before any other AWS work — deploying to SageMaker, creating resources, calling AWS APIs, or anything that touches an AWS account. Use it especially when the user has not specified a region or profile explicitly, when they say things like "use my AWS account", "deploy to AWS", "use my profile", or when about to make any AWS CLI or SDK call. Never guess the region or account ID — always use this skill to read it from the local configuration first.

日本語の概要は準備中です。原文の説明を表示しています。

huggingface/skills1.1万2026年10月9日 更新

Set up an isolated Python environment for SageMaker / AWS work, with the right Python version and current boto3. Use this skill whenever Python code will be executed for a SageMaker deployment, training job, or any AWS automation — including when about to run `pip install`, when about to invoke `boto3`, when creating or activating a virtualenv, or when the user asks to "set up the environment". Never use system Python and never `pip install` into it. Always isolate. This skill prevents the most common failure modes: wrong Python version, dependency conflicts, and stale SDKs.

日本語の概要は準備中です。原文の説明を表示しています。

huggingface/skills1.1万2026年10月9日 更新

Plan and coordinate the deployment of a model to Amazon SageMaker AI. Use this skill whenever the user wants to deploy, host, serve, or expose a model on SageMaker or AWS — including phrases like "deploy a model", "host this LLM on AWS", "serve this embedding model", "deploy a reranker", "deploy a text-to-image / diffusion model", "host this for async inference", "create an endpoint", "serve my fine-tuned model", or any request that involves making a model available for inference on AWS. Use this even when the user is vague (e.g. "I just want to get this running on AWS, you figure it out"). Works for text-generation LLMs, embedding models, rerankers, classifiers, text-to-image / diffusion models — picks the right serving stack and chooses between real-time and async inference. This is the entry-point skill for SageMaker deployment work — it asks clarifying questions, picks a deployment pathway, and coordinates the other deployment skills.

日本語の概要は準備中です。原文の説明を表示しています。

huggingface/skills1.1万2026年10月9日 更新

Ensure a usable SageMaker execution role exists before deploying or training. Use this skill whenever about to create a SageMaker endpoint, model, training job, or any resource that requires an execution role. Use it especially when the user has not provided a role ARN explicitly, when scripts are about to call `iam:CreateRole`, or when an AccessDenied error mentions an IAM action. Never blindly call `iam:CreateRole` — always check for existing roles first. This skill prevents the most common SageMaker deployment failure: trying to create IAM resources from an SSO principal that has no IAM write permissions.

日本語の概要は準備中です。原文の説明を表示しています。

huggingface/skills1.1万2026年10月9日 更新

Create a SageMaker endpoint (real-time, real-time scale-to-zero, or async) with autoscaling, CloudWatch alarms, and tagging enabled by default. Use this skill whenever about to create a SageMaker endpoint, write deployment code that calls `create_endpoint`, or finalize a deployment after the image URI and IAM role are known. Provides deploy.py for real-time endpoints, deploy_ic.py for real-time endpoints that scale to zero instances via inference components, and deploy_async.py for async endpoints (also scale-to-zero). This is the last step in the SageMaker deployment workflow. Never generate a bare `create_endpoint` call without these defaults — endpoints without autoscaling or alarms are demos, not deployments.

日本語の概要は準備中です。原文の説明を表示しています。

huggingface/skills1.1万2026年10月9日 更新

huggingface のスキルをすべて見る

このスキルの問題を報告する