Merge pull request #3491 from mtmail/faq-unicode-encoding

[nominatim.git] / docs / develop / ICU-Tokenizer-Modules.md

diff --git a/docs/develop/ICU-Tokenizer-Modules.md b/docs/develop/ICU-Tokenizer-Modules.md
index 2cf30a5699f7863db7db1f7eb04f7fa2ae3bf1b5..d189422a9d8e20b56cd7c53a18f3efcc6a51571b 100644 (file)
--- a/docs/develop/ICU-Tokenizer-Modules.md
+++ b/docs/develop/ICU-Tokenizer-Modules.md
@@ -52,9 +52,8 @@ the function.
 
 ### Sanitizer configuration
 
 
 ### Sanitizer configuration
 

-::: nominatim.tokenizer.sanitizers.config.SanitizerConfig
-    rendering:
-        show_source: no
+::: nominatim_db.tokenizer.sanitizers.config.SanitizerConfig
+    options:

         heading_level: 6
 
 ### The main filter function of the sanitizer
         heading_level: 6
 
 ### The main filter function of the sanitizer


@@ -62,12 +61,10 @@ the function.
 The filter function receives a single object of type `ProcessInfo`
 which has with three members:
 
 The filter function receives a single object of type `ProcessInfo`
 which has with three members:
 

- * `place`: read-only information about the place being processed.
+ * `place: PlaceInfo`: read-only information about the place being processed.

    See PlaceInfo below.
    See PlaceInfo below.

- * `names`: The current list of names for the place. Each name is a
-   PlaceName object.
- * `address`: The current list of address names for the place. Each name
-   is a PlaceName object.
+ * `names: List[PlaceName]`: The current list of names for the place.
+ * `address: List[PlaceName]`: The current list of address names for the place.

 
 While the `place` member is provided for information only, the `names` and
 `address` lists are meant to be manipulated by the sanitizer. It may add and
 
 While the `place` member is provided for information only, the `names` and
 `address` lists are meant to be manipulated by the sanitizer. It may add and


@@ -76,17 +73,15 @@ adding extra attributes) or completely replace the list with a different one.
 
 #### PlaceInfo - information about the place
 
 
 #### PlaceInfo - information about the place
 

-::: nominatim.data.place_info.PlaceInfo
-    rendering:
-        show_source: no
+::: nominatim_db.data.place_info.PlaceInfo
+    options:

         heading_level: 6
 
 
 #### PlaceName - extended naming information
 
         heading_level: 6
 
 
 #### PlaceName - extended naming information
 

-::: nominatim.data.place_name.PlaceName
-    rendering:
-        show_source: no
+::: nominatim_db.data.place_name.PlaceName
+    options:

         heading_level: 6
 
 
         heading_level: 6
 
 


@@ -134,7 +129,7 @@ sanitizers:
 !!! warning
     This example is just a simplified show case on how to create a sanitizer.
     It is not really read for real-world use: while the sanitizer would
 !!! warning
     This example is just a simplified show case on how to create a sanitizer.
     It is not really read for real-world use: while the sanitizer would

-    correcly transform `West 5th Street` into `5th Street`. it would also
+    correctly transform `West 5th Street` into `5th Street`. it would also

     shorten a simple `North Street` to `Street`.
 
 For more sanitizer examples, have a look at the sanitizers provided by Nominatim.
     shorten a simple `North Street` to `Street`.
 
 For more sanitizer examples, have a look at the sanitizers provided by Nominatim.


@@ -144,15 +139,13 @@ They can be found in the directory
 
 ## Custom token analysis module
 
 
 ## Custom token analysis module
 

-::: nominatim.tokenizer.token_analysis.base.AnalysisModule
-    rendering:
-        show_source: no
+::: nominatim_db.tokenizer.token_analysis.base.AnalysisModule
+    options:

         heading_level: 6
 
 
         heading_level: 6
 
 

-::: nominatim.tokenizer.token_analysis.base.Analyzer
-    rendering:
-        show_source: no
+::: nominatim_db.tokenizer.token_analysis.base.Analyzer
+    options:

         heading_level: 6
 
 ### Example: Creating acronym variants for long names
         heading_level: 6
 
 ### Example: Creating acronym variants for long names