X-Git-Url: https://git.openstreetmap.org./nominatim.git/blobdiff_plain/ca149fb796b1c5e6705a25005683548b816d20f2..b0067a0345c2ab7793614925d6150141ef5f18f0:/docs/customize/Tokenizers.md?ds=sidebyside diff --git a/docs/customize/Tokenizers.md b/docs/customize/Tokenizers.md index 11c27e38..df336a71 100644 --- a/docs/customize/Tokenizers.md +++ b/docs/customize/Tokenizers.md @@ -176,58 +176,66 @@ The following is a list of sanitizers that are shipped with Nominatim. ##### split-name-list ::: nominatim.tokenizer.sanitizers.split_name_list - selection: + options: members: False - rendering: heading_level: 6 + docstring_section_style: spacy ##### strip-brace-terms ::: nominatim.tokenizer.sanitizers.strip_brace_terms - selection: + options: members: False - rendering: heading_level: 6 + docstring_section_style: spacy ##### tag-analyzer-by-language ::: nominatim.tokenizer.sanitizers.tag_analyzer_by_language - selection: + options: members: False - rendering: heading_level: 6 + docstring_section_style: spacy ##### clean-housenumbers ::: nominatim.tokenizer.sanitizers.clean_housenumbers - selection: + options: members: False - rendering: heading_level: 6 + docstring_section_style: spacy ##### clean-postcodes ::: nominatim.tokenizer.sanitizers.clean_postcodes - selection: + options: members: False - rendering: heading_level: 6 + docstring_section_style: spacy ##### clean-tiger-tags ::: nominatim.tokenizer.sanitizers.clean_tiger_tags - selection: + options: members: False - rendering: heading_level: 6 + docstring_section_style: spacy #### delete-tags ::: nominatim.tokenizer.sanitizers.delete_tags - selection: + options: members: False - rendering: heading_level: 6 + docstring_section_style: spacy + +#### tag-japanese + +::: nominatim.tokenizer.sanitizers.tag_japanese + options: + members: False + heading_level: 6 + docstring_section_style: spacy #### Token Analysis @@ -386,7 +394,7 @@ The analyzer cannot be customized. ##### Postcode token analyzer The analyzer `postcodes` is pupose-made to analyze postcodes. It supports -a 'lookup' varaint of the token, which produces variants with optional +a 'lookup' variant of the token, which produces variants with optional spaces. Use together with the clean-postcodes sanitizer. The analyzer cannot be customized.