X-Git-Url: https://git.openstreetmap.org./nominatim.git/blobdiff_plain/5be320368c6695498c4fed7cbba44220d3c91b17..566ac7af993f487e249185ddae59bc38c38f84de:/docs/customize/Tokenizers.md diff --git a/docs/customize/Tokenizers.md b/docs/customize/Tokenizers.md index c563b201..11c27e38 100644 --- a/docs/customize/Tokenizers.md +++ b/docs/customize/Tokenizers.md @@ -102,7 +102,7 @@ Here is an example configuration file: ``` yaml normalization: - ":: lower ()" - - "ß > 'ss'" # German szet is unimbigiously equal to double ss + - "ß > 'ss'" # German szet is unambiguously equal to double ss transliteration: - !include /etc/nominatim/icu-rules/extended-unicode-to-asccii.yaml - ":: Ascii ()" @@ -128,7 +128,7 @@ The configuration file contains four sections: The normalization and transliteration sections each define a set of ICU rules that are applied to the names. -The **normalisation** rules are applied after sanitation. They should remove +The **normalization** rules are applied after sanitation. They should remove any information that is not relevant for search at all. Usual rules to be applied here are: lower-casing, removing of special characters, cleanup of spaces. @@ -213,6 +213,21 @@ The following is a list of sanitizers that are shipped with Nominatim. rendering: heading_level: 6 +##### clean-tiger-tags + +::: nominatim.tokenizer.sanitizers.clean_tiger_tags + selection: + members: False + rendering: + heading_level: 6 + +#### delete-tags + +::: nominatim.tokenizer.sanitizers.delete_tags + selection: + members: False + rendering: + heading_level: 6 #### Token Analysis