Self-host Presidio (Docker Compose, German + English)
AdminCompanyWizard’s PII detection uses Microsoft Presidio as an external service. If you run the free or evaluation edition, you can provide it yourself via Docker. This page offers two ready-made setups: a minimal quickstart (English, fast) and the multilingual production build (German + English, Stanza). What the PII shield does conceptually is explained in The PII shield; the in-app wiring is covered by Set up Presidio.
Variant A — minimal quickstart (English only)
Section titled “Variant A — minimal quickstart (English only)”For a quick trial, the two official images are enough — no custom build. The default analyzer ships an English spaCy model.
services: analyzer: image: mcr.microsoft.com/presidio-analyzer:2.2.362 restart: unless-stopped ports: - "127.0.0.1:6001:5001" anonymizer: image: mcr.microsoft.com/presidio-anonymizer:2.2.362 restart: unless-stopped ports: - "127.0.0.1:6002:5001"docker compose up -dcurl http://localhost:6001/health # -> "Presidio Analyzer service is up"Limitation: this setup only detects English names/places/organizations (NER). For German text you need variant B.
Variant B — production build (German + English, Stanza)
Section titled “Variant B — production build (German + English, Stanza)”This build replaces the default spaCy model with Stanza, adds curated recognizers for en + de and a robust Gunicorn configuration. Create the following directory layout:
presidio/├── docker-compose.yaml└── analyzer/ ├── Dockerfile ├── gunicorn.conf.py ├── patch_stanza_mwt.py └── conf/ ├── stanza_multilingual.yaml ├── analyzer.yaml └── recognizers.yamldocker-compose.yaml
Section titled “docker-compose.yaml”# Presidio text-only: analyzer (Stanza, en+de) + anonymizer — production setupservices: analyzer: build: context: ./analyzer restart: unless-stopped ports: # Safer (if a reverse proxy runs on the same host): "127.0.0.1:${ANALYZER_PORT:-6001}:5001" - "${ANALYZER_PORT:-6001}:5001" environment: - PORT=5001 - WORKERS=${ANALYZER_WORKERS:-1} # consumed by the image entrypoint.sh: gunicorn -w $WORKERS # Cap threads per inference: set to cores ÷ workers. Without a cap every # Stanza/PyTorch inference grabs ALL cores; parallel requests can then slow down # many times over. KMP_BLOCKTIME=0 stops the threads spin-waiting after inference. - OMP_NUM_THREADS=${ANALYZER_OMP_THREADS:-2} - KMP_BLOCKTIME=0 # Three matching configs (en + de). They must be consistent, otherwise: # "Misconfigured engine, supported languages have to be consistent". - NLP_CONF_FILE=presidio_analyzer/conf/stanza_multilingual.yaml - ANALYZER_CONF_FILE=presidio_analyzer/conf/analyzer.yaml - RECOGNIZER_REGISTRY_CONF_FILE=presidio_analyzer/conf/recognizers.yaml # Gunicorn config file with lifecycle hooks + faulthandler boot watchdog. - GUNICORN_CMD_ARGS=--config=/app/gunicorn.conf.py - ANALYZER_TIMEOUT=${ANALYZER_TIMEOUT:-300} - ANALYZER_GRACEFUL_TIMEOUT=${ANALYZER_GRACEFUL_TIMEOUT:-30} - ANALYZER_MAX_REQUESTS=${ANALYZER_MAX_REQUESTS:-0} - ANALYZER_MAX_REQUESTS_JITTER=${ANALYZER_MAX_REQUESTS_JITTER:-0} - ANALYZER_LOG_LEVEL=${ANALYZER_LOG_LEVEL:-info} - ANALYZER_BOOT_WATCHDOG=${ANALYZER_BOOT_WATCHDOG:-330} healthcheck: # curl is not guaranteed in the slim images -> use python3 test: ["CMD", "python3", "-c", "import urllib.request; urllib.request.urlopen('http://localhost:5001/health', timeout=5)"] interval: 30s timeout: 10s retries: 5 start_period: 180s # Stanza en+de takes much longer on first start deploy: resources: limits: memory: ${ANALYZER_MEMORY_LIMIT:-6g} # 8GB host: 6g analyzer + 1g anonymizer + ~1g OS/buffer reservations: memory: ${ANALYZER_MEMORY_RESERVATION:-3g} networks: - presidio
anonymizer: image: mcr.microsoft.com/presidio-anonymizer:2.2.362 restart: unless-stopped ports: - "${ANONYMIZER_PORT:-6002}:5001" environment: - PORT=5001 - GUNICORN_CMD_ARGS=--workers=${ANONYMIZER_WORKERS:-1} # reliable in the official image healthcheck: test: ["CMD", "python3", "-c", "import urllib.request; urllib.request.urlopen('http://localhost:5001/health', timeout=5)"] interval: 30s timeout: 10s retries: 5 start_period: 30s depends_on: - analyzer # start order only deploy: resources: limits: memory: 1g networks: - presidio
networks: presidio:analyzer/Dockerfile
Section titled “analyzer/Dockerfile”# 1) PIN the base image (verify the tag against MCR first!)FROM mcr.microsoft.com/presidio-analyzer:2.2.362
# 2) Create a fixed model location usable by 'presidio'USER rootENV STANZA_RESOURCES_DIR=/opt/stanza_resourcesRUN mkdir -p "$STANZA_RESOURCES_DIR" && chown -R presidio:presidio "$STANZA_RESOURCES_DIR"
# 3) Install stanza SYSTEM-WIDE with the app interpreter (no user-site)RUN python3 -m pip install --no-cache-dir stanza
# 4) Place conf files at the ACTUAL presidio_analyzer pathCOPY conf/stanza_multilingual.yaml /tmp/conf/stanza_multilingual.yamlCOPY conf/analyzer.yaml /tmp/conf/analyzer.yamlCOPY conf/recognizers.yaml /tmp/conf/recognizers.yamlRUN CONF_DIR="$(python3 -c 'import os,presidio_analyzer as p; print(os.path.join(os.path.dirname(p.__file__),"conf"))')" \ && cp /tmp/conf/stanza_multilingual.yaml "$CONF_DIR/stanza_multilingual.yaml" \ && cp /tmp/conf/analyzer.yaml "$CONF_DIR/analyzer.yaml" \ && cp /tmp/conf/recognizers.yaml "$CONF_DIR/recognizers.yaml" \ && rm -rf /tmp/conf
# 5) Bake models as 'presidio' into STANZA_RESOURCES_DIRUSER presidioRUN python3 -c "import stanza; stanza.download('en'); stanza.download('de')"
# 6) Point the default ENVs at our configs (may be overridden by docker-compose).ENV NLP_CONF_FILE=presidio_analyzer/conf/stanza_multilingual.yamlENV ANALYZER_CONF_FILE=presidio_analyzer/conf/analyzer.yamlENV RECOGNIZER_REGISTRY_CONF_FILE=presidio_analyzer/conf/recognizers.yaml
# 7) Gunicorn config with lifecycle hooks + faulthandler boot watchdog.USER rootCOPY --chown=presidio:presidio gunicorn.conf.py /app/gunicorn.conf.pyUSER presidio
# 8) Required for German: do not expand multi-word tokens (im/am/zum/zur/beim/vom),# otherwise the de leg drops every NER span (no PERSON) and indented text ends in# HTTP 500 "Did not find word ...". See the patch_stanza_mwt.py section.USER rootCOPY patch_stanza_mwt.py /tmp/patch_stanza_mwt.pyRUN cd /app && python3 /tmp/patch_stanza_mwt.py && rm /tmp/patch_stanza_mwt.pyUSER presidioanalyzer/conf/stanza_multilingual.yaml — NLP engine
Section titled “analyzer/conf/stanza_multilingual.yaml — NLP engine”nlp_engine_name: stanzamodels: - lang_code: en model_name: en - lang_code: de model_name: dener_model_configuration: labels_to_ignore: - O model_to_presidio_entity_mapping: # German – Stanza CoNLL03: PER, LOC, ORG, MISC PER: PERSON LOC: LOCATION ORG: ORGANIZATION MISC: NRP # English – Stanza OntoNotes: PERSON, GPE, ORG, NORP, DATE, TIME … PERSON: PERSON GPE: LOCATION LOCATION: LOCATION ORGANIZATION: ORGANIZATION NORP: NRP DATE: DATE_TIME TIME: DATE_TIME low_confidence_score_multiplier: 0.4 low_score_entity_names: - NRPanalyzer/conf/analyzer.yaml — languages
Section titled “analyzer/conf/analyzer.yaml — languages”# 'supported_languages' MUST be identical to recognizers.yaml and to the# lang_code in stanza_multilingual.yaml. Otherwise: "Misconfigured engine,# supported languages have to be consistent".supported_languages: - en - de
default_score_threshold: 0analyzer/conf/recognizers.yaml — recognizer registry
Section titled “analyzer/conf/recognizers.yaml — recognizer registry”# Custom registry for German (de) + English (en) PII detection with Stanza.supported_languages: - en - de
global_regex_flags: 26 # re.DOTALL | re.MULTILINE | re.IGNORECASE
recognizers:
# NLP recognizer (Stanza) - declared explicitly per language. - name: StanzaRecognizer type: predefined supported_languages: - en - de
# Language-sensitive recognizers with German + English context words - name: CreditCardRecognizer type: predefined supported_languages: - language: en context: [credit, card, visa, mastercard, cc, amex, discover, jcb, diners, maestro, instapayment] - language: de context: [kreditkarte, karte, kartennummer, visa, mastercard, amex, cc]
- name: DateRecognizer type: predefined supported_languages: - language: en context: [date, time, birthday, birthdate, dob] - language: de context: [datum, zeit, uhrzeit, geburtstag, geburtsdatum]
- name: EmailRecognizer type: predefined supported_languages: - language: en context: [email, mail, address] - language: de context: [email, "e-mail", mail, mailadresse, emailadresse]
- name: PhoneRecognizer type: predefined supported_languages: - language: en context: [phone, number, telephone, fax, mobile, cell] - language: de context: [telefon, telefonnummer, rufnummer, handy, mobil, mobiltelefon, fax, durchwahl]
# Language-agnostic recognizers (pure regex) -> apply to every language. - name: CryptoRecognizer type: predefined - name: IbanRecognizer type: predefined - name: IpRecognizer type: predefined - name: MacAddressRecognizer type: predefined - name: UrlRecognizer type: predefined - name: MedicalLicenseRecognizer type: predefinedanalyzer/gunicorn.conf.py
Section titled “analyzer/gunicorn.conf.py”This file makes the worker boot observable and gives Stanza enough time on a cold start (the default 30 s timeout is too short). Use it unchanged:
from __future__ import annotations
import faulthandlerimport osimport sysimport threading
def _env_int(name: str, default: int) -> int: raw = os.environ.get(name) if raw is None or raw.strip() == "": return default try: return int(raw) except ValueError: return default
bind = f"0.0.0.0:{os.environ.get('PORT', '5001')}"workers = _env_int("WORKERS", 1)worker_class = "sync"
timeout = _env_int("ANALYZER_TIMEOUT", 300)graceful_timeout = _env_int("ANALYZER_GRACEFUL_TIMEOUT", 30)
# Worker rotation OFF by default: with WORKERS=1 it forces a ~90s Stanza# reload and breaks the healthcheck. Enable via ANALYZER_MAX_REQUESTS.max_requests = _env_int("ANALYZER_MAX_REQUESTS", 0)max_requests_jitter = _env_int("ANALYZER_MAX_REQUESTS_JITTER", 0)
loglevel = os.environ.get("ANALYZER_LOG_LEVEL", "info")accesslog = "-"errorlog = "-"capture_output = Trueenable_stdio_inheritance = True
# Boot watchdog: dump a Python stack after this many seconds if the worker# hangs between post_fork and post_worker_init._BOOT_WATCHDOG_SECONDS = _env_int("ANALYZER_BOOT_WATCHDOG", timeout + 30)
def on_starting(server) -> None: server.log.info("[gunicorn-hook] on_starting master_pid=%s timeout=%ss", os.getpid(), timeout)
def when_ready(server) -> None: server.log.info("[gunicorn-hook] when_ready master_pid=%s workers=%s bind=%s", os.getpid(), workers, bind)
def post_fork(server, worker) -> None: server.log.info("[gunicorn-hook] post_fork worker_pid=%s -- arming boot watchdog (%ss)", os.getpid(), _BOOT_WATCHDOG_SECONDS) faulthandler.enable(file=sys.stderr, all_threads=True) faulthandler.dump_traceback_later(_BOOT_WATCHDOG_SECONDS, repeat=False, file=sys.stderr, exit=False)
def post_worker_init(worker) -> None: try: faulthandler.cancel_dump_traceback_later() except Exception: pass worker.log.info("[gunicorn-hook] post_worker_init worker_pid=%s -> READY", os.getpid())
def worker_abort(worker) -> None: worker.log.error("[gunicorn-hook] worker_abort (timeout) worker_pid=%s -- dumping stack:", os.getpid()) faulthandler.dump_traceback(file=sys.stderr, all_threads=True)The original file also defines
pre_fork,worker_int,worker_exit,child_exitandon_exitfor gap-free boot logs — those hooks are optional and only useful for diagnostics.
analyzer/patch_stanza_mwt.py — required for German
Section titled “analyzer/patch_stanza_mwt.py — required for German”#!/usr/bin/env python3"""Build-time patch for presidio_analyzer/nlp_engine/stanza_nlp_engine.py (cw-app #953).
Problem (measured 2026-09-12 against presidio-analyzer 2.2.362 + stanza 1.14): The vendored spacy-stanza tokenizer flattens Stanza multi-word tokens into their expanded words ("im" -> "in", "dem"). The expanded words no longer match the original text, the alignment check in _convert_doc raises, and the whole document text is REPLACED by " ".join(tokens). Consequences: Stanza NER spans (original offsets) hit no token boundary in the replaced doc and are dropped -> `de` returns NO PERSON for any German text containing am/im/zum/zur/beim/vom; and in indented text the token list ends before the text does, so LemmaContextAwareEnhancer raises "Did not find word ... in the list of tokens" -> HTTP 500.
Fix: keep multi-word tokens as ONE surface token (text = token.text, annotations ofthe first word, lemma = surface). Alignment then holds, doc.text == text, entitieskeep their original offsets. Verified in-container: minimal pairs im/in dem, zum/zudem, beim/bei dem, Am ... zur all yield PERSON with correct offsets; lemmas ofordinary tokens unchanged.
Fails the build loudly when the expected code is not found (a presidio bump must bere-verified, not silently unpatched)."""import importlibimport osimport pathlibimport sys
# run from /app: presidio_analyzer lives there as a plain package, not in site-packagessys.path.insert(0, os.getcwd())
module = importlib.import_module("presidio_analyzer.nlp_engine.stanza_nlp_engine")path = pathlib.Path(module.__file__)source = path.read_text(encoding="utf-8")
OLD = """ for token in sentence.tokens: for word in token.words: # Here, we're calculating the absolute token index in the doc, # then the *relative* index of the head, -1 for zero-indexed # and if the governor is 0 (root), we leave it at 0 if word.head: head = word.head + offset - len(tokens) - 1 else: head = 0 heads.append(head) tokens.append(word) offset += sum(len(token.words) for token in sentence.tokens)"""
NEW = """ for token in sentence.tokens: # cw-app #953: a multi-word token (German "im" -> "in" + "dem") is kept # as ONE surface token. Expanding it breaks the text alignment below, # which replaces the whole doc text and drops every NER span. if len(token.words) > 1: heads.append(0) tokens.append(_SurfaceToken(token)) continue for word in token.words: # Here, we're calculating the absolute token index in the doc, # then the *relative* index of the head, -1 for zero-indexed # and if the governor is 0 (root), we leave it at 0 if word.head: head = word.head + offset - len(tokens) - 1 else: head = 0 heads.append(head) tokens.append(word) offset += sum(1 if len(token.words) > 1 else len(token.words) for token in sentence.tokens)"""
HELPER = '''
class _SurfaceToken: """cw-app #953: stand-in for a Stanza multi-word token, unexpanded.
Carries the surface text and offsets of the token and borrows every other annotation (pos, feats, deprel, ...) from its first word. """
def __init__(self, token): self._word = token.words[0] self.text = token.text self.lemma = token.text self.head = 0 self.start_char = token.start_char self.end_char = token.end_char self.ner = token.ner
def __getattr__(self, name): return getattr(self._word, name)'''
MARKER = "# cw-app #953"if MARKER in source: print(f"already patched: {path}") sys.exit(0)
if source.count(OLD) != 1: print(f"ERROR: expected code block not found exactly once in {path} " f"(count={source.count(OLD)}) — presidio version changed, re-verify the patch.") sys.exit(1)
anchor = "\nclass StanzaTokenizer(object):"if source.count(anchor) != 1: print(f"ERROR: StanzaTokenizer class anchor not found exactly once in {path}") sys.exit(1)
patched = source.replace(OLD, NEW).replace(anchor, HELPER + anchor)path.write_text(patched, encoding="utf-8")
# compile check + smoke importcompile(patched, str(path), "exec")importlib.reload(module)print(f"patched: {path}")Verify after start — expected is exactly one PERSON “Thomas Bergmann” at offset 28–43; without the patch the same request returns []:
curl -s http://localhost:6001/analyze -H 'Content-Type: application/json' \ -d '{"text":"Wir treffen uns im Büro mit Thomas Bergmann.","language":"de","score_threshold":0.4,"entities":["PERSON"]}'Build & start
Section titled “Build & start”docker compose build # downloads Stanza en+de — takes a few minutesdocker compose up -ddocker compose logs -f analyzer # wait for "post_worker_init ... READY"Changing the language configuration
Section titled “Changing the language configuration”The supported languages live in three places and must be identical:
| File | Key |
|---|---|
analyzer.yaml | supported_languages |
recognizers.yaml | supported_languages |
stanza_multilingual.yaml | models[].lang_code |
If they diverge, Presidio aborts at startup with “Misconfigured engine, supported languages have to be consistent”. Adding a language therefore means: list it in all three files, download the matching Stanza model in the Dockerfile via stanza.download(...), and rebuild.
Not every detection is available in every language
Section titled “Not every detection is available in every language”| Recognizer type | Examples | Language-dependent? |
|---|---|---|
| NLP/NER (Stanza) | PERSON, LOCATION, ORGANIZATION, NRP, DATE_TIME | Yes — coverage varies per model |
| Language-sensitive (rule-based) | credit card, date, email, phone | Yes — own context words per language |
| Language-agnostic (pure regex) | IBAN, crypto, IP, MAC, URL, medical license | No — apply to all languages |
Key consequences for German vs. English:
- The English Stanza model (OntoNotes) additionally detects
DATE/TIMEvia NER and thus yieldsDATE_TIME; the German model (CoNLL03) only knowsPER/LOC/ORG/MISC— dates there are captured only by the rule-basedDateRecognizer(context words). NRP(nationality/religion/political group) is sourced fromNORPin English and fromMISCin German — detection quality differs.MedicalLicenseRecognizertargets US formats and is of little value for German text.
Which entity types the app actually masks is configured additionally in the system settings → Set up Presidio.
How many workers?
Section titled “How many workers?”The analyzer runs gunicorn sync workers: each worker handles one request, all others wait. A running index pass (full reindex, import, rebuild after an embedding change) scans section by section and, with two languages (de + en), keeps 2 workers busy the whole time. The scans of a chat turn — input, search query, knowledge context — queue up behind it as soon as no worker is free.
Rule of thumb: if the instance serves chat and index runs at the same time, plan for at least 4 analyzer workers. The default of 1 worker is fine for trying things out and as long as chat and index runs rarely coincide.
Measured with the production build (variant B, de + en) — duration of a chat turn’s PII scans, without and with a running index pass:
| Analyzer workers | no index pass | with index pass | factor |
|---|---|---|---|
| 4 | 8.3 s | 10.2 s | 1.2× |
| 2 | 4.8 s | 11.7 s | 2.4× |
| 1 (default) | 7.2 s | 30.8 s | 4.3× |
With 1 worker, scanning a large tool result (12,000 characters) additionally takes up to ~30 s. Method: p95 per measurement block, the higher of two blocks each; with 1 worker, one no-load block with a single outlier was not counted. The instances ran on different hardware — what matters is the factor, not the absolute time.
What 4 workers cost:
- Memory: each worker loads the models in full — guideline ~8–12 GB for the analyzer. Raise
ANALYZER_MEMORY_LIMITaccordingly (e.g.12g) and plan for a host with ≥ 16 GB. - CPU threads: set
ANALYZER_OMP_THREADSto cores ÷ workers (8 cores, 4 workers →2). Without this cap every inference grabs all cores; several simultaneous requests can then collapse drastically — measured up to 30× (2 workers on 8 cores).
Tuning (environment variables)
Section titled “Tuning (environment variables)”| Variable | Default | Effect |
|---|---|---|
ANALYZER_WORKERS | 1 | Parallel workers. Each loads the models in full → RAM ≈ workers × ~2–3 GB. For chat + index runs at the same time ≥ 4 → How many workers? |
ANALYZER_OMP_THREADS | 2 | CPU threads per inference. Set to cores ÷ workers; mandatory from 2 workers, otherwise drastic slowdowns under parallel load. |
ANALYZER_MEMORY_LIMIT | 6g | Memory limit of the analyzer container; raise it with more workers. |
ANALYZER_MEMORY_RESERVATION | 3g | Reserved memory. |
ANALYZER_PORT / ANONYMIZER_PORT | 6001 / 6002 | Host ports. |
ANALYZER_TIMEOUT | 300 | Hard timeout per request/worker boot (seconds). |
ANALYZER_MAX_REQUESTS | 0 | Worker rotation; 0 = off (recommended with 1 worker). |
ANALYZER_BOOT_WATCHDOG | 330 | Deadline for the stack dump during worker boot. |
ANONYMIZER_WORKERS | 1 | Anonymizer workers (lightweight). |
Securing it
Section titled “Securing it”Bind the ports to 127.0.0.1 when a reverse proxy terminates on the same host (prepared as a comment in the compose file): "127.0.0.1:${ANALYZER_PORT:-6001}:5001". Presidio itself ships no authentication — never expose it unprotected to the network.
Connecting CompanyWizard
Section titled “Connecting CompanyWizard”Enter the analyzer endpoint (http://localhost:6001) as PiiScanner:PresidioEndpoint or in Administration → System settings; there you can also adjust threshold, entity types and languages, and run a test connection. Full wiring and failure behavior: Set up Presidio.
Troubleshooting
Section titled “Troubleshooting”- Boot takes long / healthcheck red: Stanza en+de loads for tens of seconds on first start. The
start_period: 180scovers it; in the logs wait forpost_worker_init ... READY. - “Misconfigured engine …”: the three language lists don’t match (see above).
- Chat gets slow during an index run: the chat’s scans wait behind the index run for free workers. Check
ANALYZER_WORKERS— for chat + index runs at the same time ≥ 4 → How many workers? - Requests suddenly take 30 s or more as soon as several run at once: the inferences compete for all cores. Set
ANALYZER_OMP_THREADSto cores ÷ workers and restart the analyzer. - Worker hangs after fork: the boot watchdog dumps a Python stack to
stderr(docker compose logs analyzer). - German detects no persons / HTTP 500 “Did not find word …”: the image was built without step 8 (
patch_stanza_mwt.py). Add the step, rundocker compose build analyzer && docker compose up -d analyzer, then verify with thecurlfrom the patch section.