6 tests covering the research skeleton:
Tokenizer (5 tests):
* loadHFTokenizer builds a valid WordPiece state (vocab size, special
token IDs)
* encodeWordPiece wraps output with [CLS] ... [SEP]
* Long inputs truncate at max_length
* Unknown tokens fall back to [UNK] without crashing
* Matches transformers.js AutoTokenizer on 4 fixture strings — the
correctness anchor. If our tokenizer drifts from transformers.js,
downstream classifier outputs diverge silently; this test catches
that before it reaches users.
Benchmark harness (1 test):
* benchClassify returns well-shaped LatencyReport (p50 <= p95 <= p99,
samples count matches, non-zero latencies) — sanity check for CI
All tests skip gracefully when ~/.gstack/models/testsavant-small/
tokenizer.json is missing (first-run CI before warmup).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Garry Tan
c257d72d7d