「tokenization」の検索結果

Validate and implement Sinclair ZX Spectrum 48K ROM-visible behavior in the EMULATORS repository, including reset and restart vectors, IM1/NMI handling, keyboard scanning and repeat, beeper and tape routines, screen/printer output, channels and streams, BASIC tokenization and execution, expression evaluation, memory movement, system variables, calculator stack, floating-point formats, ROM traps, localized ROM selection, and firmware/hardware timing contracts. Use when Codex reviews or edits ZX Spectrum ROM loading, KERNAL-style traps, keyboard, tape, screen, printer, BASIC files, tokens, system variables, startup, interrupts, or compatibility tests.

日本語の概要は準備中です。原文の説明を表示しています。

Guide for using renderers — the bridge between chat-style messages and token sequences. Covers renderer setup, TrainOnWhat, vision inputs, model family matching, and custom renderers. Use when the user asks about renderers, tokenization, message formatting, or vision inputs.

日本語の概要は準備中です。原文の説明を表示しています。

Fast BPE/WordPiece tokenization and custom vocab training.

日本語の概要は準備中です。原文の説明を表示しています。

High-performance toolkit for genomic interval analysis in Rust with Python bindings. Use when working with genomic regions, BED files, coverage tracks, overlap detection, tokenization for ML models, or fragment analysis in computational genomics and machine learning applications.

日本語の概要は準備中です。原文の説明を表示しています。

Fast tokenizers optimized for research and production. Rust-based implementation tokenizes 1GB in <20 seconds. Supports BPE, WordPiece, and Unigram algorithms. Train custom vocabularies, track alignments, handle padding/truncation. Integrates seamlessly with transformers. Use when you need high-performance tokenization or custom tokenizer training.

日本語の概要は準備中です。原文の説明を表示しています。

Work with state-of-the-art machine learning models for NLP, computer vision, audio, and multimodal tasks using HuggingFace Transformers. This skill should be used when fine-tuning pre-trained models, performing inference with pipelines, generating text, training sequence models, or working with BERT, GPT, T5, ViT, and other transformer architectures. Covers model loading, tokenization, training with Trainer API, text generation strategies, and task-specific patterns for classification, NER, QA, summarization, translation, and image tasks. (plugin:scientific-packages@claude-scientific-skills)

日本語の概要は準備中です。原文の説明を表示しています。