X-Git-Url: https://git.openstreetmap.org./nominatim.git/blobdiff_plain/9a5f75dba759a3e52e013ef8c5db433cf4ef1b8f..75a77810d1248d21f40f6207d31e032694410b7a:/docs/customize/Tokenizers.md?ds=sidebyside diff --git a/docs/customize/Tokenizers.md b/docs/customize/Tokenizers.md index 11c27e38..2c7b6878 100644 --- a/docs/customize/Tokenizers.md +++ b/docs/customize/Tokenizers.md @@ -176,58 +176,66 @@ The following is a list of sanitizers that are shipped with Nominatim. ##### split-name-list ::: nominatim.tokenizer.sanitizers.split_name_list - selection: + options: members: False - rendering: heading_level: 6 + docstring_section_style: spacy ##### strip-brace-terms ::: nominatim.tokenizer.sanitizers.strip_brace_terms - selection: + options: members: False - rendering: heading_level: 6 + docstring_section_style: spacy ##### tag-analyzer-by-language ::: nominatim.tokenizer.sanitizers.tag_analyzer_by_language - selection: + options: members: False - rendering: heading_level: 6 + docstring_section_style: spacy ##### clean-housenumbers ::: nominatim.tokenizer.sanitizers.clean_housenumbers - selection: + options: members: False - rendering: heading_level: 6 + docstring_section_style: spacy ##### clean-postcodes ::: nominatim.tokenizer.sanitizers.clean_postcodes - selection: + options: members: False - rendering: heading_level: 6 + docstring_section_style: spacy ##### clean-tiger-tags ::: nominatim.tokenizer.sanitizers.clean_tiger_tags - selection: + options: members: False - rendering: heading_level: 6 + docstring_section_style: spacy #### delete-tags ::: nominatim.tokenizer.sanitizers.delete_tags - selection: + options: members: False - rendering: heading_level: 6 + docstring_section_style: spacy + +#### tag-japanese + +::: nominatim.tokenizer.sanitizers.tag_japanese + options: + members: False + heading_level: 6 + docstring_section_style: spacy #### Token Analysis