X-Git-Url: https://git.openstreetmap.org./nominatim.git/blobdiff_plain/e427712cb04baf001d41e34af46bb9fd083202a1..3a8c5df82d95b2b7346718e972b0d49b82514336:/docs/develop/ICU-Tokenizer-Modules.md diff --git a/docs/develop/ICU-Tokenizer-Modules.md b/docs/develop/ICU-Tokenizer-Modules.md index 2cf30a56..daadf899 100644 --- a/docs/develop/ICU-Tokenizer-Modules.md +++ b/docs/develop/ICU-Tokenizer-Modules.md @@ -53,8 +53,7 @@ the function. ### Sanitizer configuration ::: nominatim.tokenizer.sanitizers.config.SanitizerConfig - rendering: - show_source: no + options: heading_level: 6 ### The main filter function of the sanitizer @@ -62,12 +61,10 @@ the function. The filter function receives a single object of type `ProcessInfo` which has with three members: - * `place`: read-only information about the place being processed. + * `place: PlaceInfo`: read-only information about the place being processed. See PlaceInfo below. - * `names`: The current list of names for the place. Each name is a - PlaceName object. - * `address`: The current list of address names for the place. Each name - is a PlaceName object. + * `names: List[PlaceName]`: The current list of names for the place. + * `address: List[PlaceName]`: The current list of address names for the place. While the `place` member is provided for information only, the `names` and `address` lists are meant to be manipulated by the sanitizer. It may add and @@ -77,16 +74,14 @@ adding extra attributes) or completely replace the list with a different one. #### PlaceInfo - information about the place ::: nominatim.data.place_info.PlaceInfo - rendering: - show_source: no + options: heading_level: 6 #### PlaceName - extended naming information ::: nominatim.data.place_name.PlaceName - rendering: - show_source: no + options: heading_level: 6 @@ -134,7 +129,7 @@ sanitizers: !!! warning This example is just a simplified show case on how to create a sanitizer. It is not really read for real-world use: while the sanitizer would - correcly transform `West 5th Street` into `5th Street`. it would also + correctly transform `West 5th Street` into `5th Street`. it would also shorten a simple `North Street` to `Street`. For more sanitizer examples, have a look at the sanitizers provided by Nominatim. @@ -145,14 +140,12 @@ They can be found in the directory ## Custom token analysis module ::: nominatim.tokenizer.token_analysis.base.AnalysisModule - rendering: - show_source: no + options: heading_level: 6 ::: nominatim.tokenizer.token_analysis.base.Analyzer - rendering: - show_source: no + options: heading_level: 6 ### Example: Creating acronym variants for long names