fix: solve vLLM->BLM misrecognition via glossary postprocessing

The bench and CLI kept emitting BLM for vLLM. Three layers:

1. rules.py: add default rule BLM -> vLLM (word-boundary, case-insensitive) so the default post_mode=rules path restores it everywhere.

2. benchmark/runner.py: the bench never ran postprocessing - it measured raw engine text, so entity retention (66.7%) never reflected rules/glossary. _transcribe_clip now builds Segments and runs run_postprocess(settings, glossary) before metrics; manifest top-level glossary {pattern: replacement} is supported and recorded in run_config (post_mode/glossary).

3. glossary plumbing: CLI transcribe gains --glossary KEY=VALUE (repeatable, parsed + validated); BatchPipeline.run accepts glossary= and passes it to run_postprocess; the in-proc worker forwards job.options.glossary or post_correction.

Notebook: transcribe cells pass --glossary BLM=vLLM, bench manifest carries glossary, postprocess section documents rules/glossary/hotword.

+ 9 unit tests (BLM rule, boundary/case behavior, bench postprocess + glossary entity retention). 147 tests pass, ruff clean.
This commit is contained in:
2026-08-12 21:06:42 +09:00
parent f171ce4992
commit 6bb89eb51f
9 changed files with 237 additions and 22 deletions
+12 -5
View File
@@ -192,9 +192,12 @@ def cells() -> list[dict]:
"# 첫 실행 시 Hugging Face에서 모델을 다운로드합니다 (turbo ≈ 1.6GB, 1~2분)\n"
"# v0.1: 'cuda:N' device를 분리해 CTranslate2 계약에 맞게 전달하도록 수정됨\n"
"# (기존: unsupported device cuda:0). 그래도 실패하면 CPU 폴백 안내가 출력됩니다.\n"
"# 후처리: 기본 rules가 vLLM→BLM 같은 흔한 오인식을 복원하고,\n"
"# --glossary 'BLM=vLLM'으로 도메인 용어를 명시적으로 보강할 수 있다.\n"
"import subprocess, json\n"
"r = subprocess.run(\n"
" ['luke-scribe', 'transcribe', 'samples/colab-ko-en.mp3', '--language', 'ko', '--device', 'auto'],\n"
" ['luke-scribe', 'transcribe', 'samples/colab-ko-en.mp3', '--language', 'ko', '--device', 'auto',\n"
" '--glossary', 'BLM=vLLM'],\n"
" capture_output=True, text=True,\n"
")\n"
"print(r.stdout[-2500:] if r.stdout else '')\n"
@@ -202,12 +205,14 @@ def cells() -> list[dict]:
),
md(
"### 5-2) 후처리 검증\n\n"
"glossary(오인식 용어 복원) + hotword(용어 사전 주입) 동작을 확인합니다.\n"
"`--hotword vLLM Kubernetes`를 주면 initial_prompt에 용어가 주입되어 보존률이 올라갑니다."
"후처리(rules/glossary) + hotword(용어 사전 주입) 동작을 확인합니다.\n"
"- **rules (기본)**: `BLM → vLLM`, `v l l m → vLLM` 같은 흔한 오인식을 결정적으로 복원.\n"
"- **glossary**: `--glossary '오인식=표준'`으로 도메인 용어를 명시적으로 보강 (반복 가능).\n"
"- **hotword**: `--hotword vLLM Kubernetes` → initial_prompt 주입으로 보존률 향상."
),
code(
"# 8) hotword 포함 전사 (용어 보존 강화)\n"
"!luke-scribe transcribe samples/colab-ko-en.mp3 --language ko --device auto --hotword vLLM --hotword Kubernetes\n"
"# 8) hotword + glossary 포함 전사 (용어 보존 강화)\n"
"!luke-scribe transcribe samples/colab-ko-en.mp3 --language ko --device auto --hotword vLLM --hotword Kubernetes --glossary BLM=vLLM\n"
),
md(
"## 6) REST API 스모크\n\n"
@@ -454,6 +459,8 @@ def cells() -> list[dict]:
" 'dataset_version': '1.0',\n"
" 'language': 'ko',\n"
" 'targets': {'entity_preservation': 0.95, 'cer': 0.15},\n"
" # 벤치도 후처리를 적용해 실사용 지표를 측정 (vLLM→BLM 복원 포함)\n"
" 'glossary': {'BLM': 'vLLM'},\n"
" 'clips': [\n"
" {'id': 'ko-en-tech', 'audio_path': 'samples/colab-ko-en.mp3',\n"
" 'reference_path': '/content/reference.txt',\n"