X-Git-Url: https://git.openstreetmap.org./nominatim.git/blobdiff_plain/247065ff6f6f096c609729080b83896235aedfc8..d77aa7dfc9541bffa0109da89b95f296e751aea0:/docs/customize/Tokenizers.md diff --git a/docs/customize/Tokenizers.md b/docs/customize/Tokenizers.md index df336a71..30be170e 100644 --- a/docs/customize/Tokenizers.md +++ b/docs/customize/Tokenizers.md @@ -15,48 +15,6 @@ they can be configured. chosen tokenizer is very limited as well. See the comments in each tokenizer section. -## Legacy tokenizer - -The legacy tokenizer implements the analysis algorithms of older Nominatim -versions. It uses a special Postgresql module to normalize names and queries. -This tokenizer is automatically installed and used when upgrading an older -database. It should not be used for new installations anymore. - -### Compiling the PostgreSQL module - -The tokeinzer needs a special C module for PostgreSQL which is not compiled -by default. If you need the legacy tokenizer, compile Nominatim as follows: - -``` -mkdir build -cd build -cmake -DBUILD_MODULE=on -make -``` - -### Enabling the tokenizer - -To enable the tokenizer add the following line to your project configuration: - -``` -NOMINATIM_TOKENIZER=legacy -``` - -The Postgresql module for the tokenizer is available in the `module` directory -and also installed with the remainder of the software under -`lib/nominatim/module/nominatim.so`. You can specify a custom location for -the module with - -``` -NOMINATIM_DATABASE_MODULE_PATH= -``` - -This is in particular useful when the database runs on a different server. -See [Advanced installations](../admin/Advanced-Installations.md#importing-nominatim-to-an-external-postgresql-database) for details. - -There are no other configuration options for the legacy tokenizer. All -normalization functions are hard-coded. - ## ICU tokenizer The ICU tokenizer uses the [ICU library](http://site.icu-project.org/) to @@ -175,7 +133,7 @@ The following is a list of sanitizers that are shipped with Nominatim. ##### split-name-list -::: nominatim.tokenizer.sanitizers.split_name_list +::: nominatim_db.tokenizer.sanitizers.split_name_list options: members: False heading_level: 6 @@ -183,7 +141,7 @@ The following is a list of sanitizers that are shipped with Nominatim. ##### strip-brace-terms -::: nominatim.tokenizer.sanitizers.strip_brace_terms +::: nominatim_db.tokenizer.sanitizers.strip_brace_terms options: members: False heading_level: 6 @@ -191,7 +149,7 @@ The following is a list of sanitizers that are shipped with Nominatim. ##### tag-analyzer-by-language -::: nominatim.tokenizer.sanitizers.tag_analyzer_by_language +::: nominatim_db.tokenizer.sanitizers.tag_analyzer_by_language options: members: False heading_level: 6 @@ -199,7 +157,7 @@ The following is a list of sanitizers that are shipped with Nominatim. ##### clean-housenumbers -::: nominatim.tokenizer.sanitizers.clean_housenumbers +::: nominatim_db.tokenizer.sanitizers.clean_housenumbers options: members: False heading_level: 6 @@ -207,7 +165,7 @@ The following is a list of sanitizers that are shipped with Nominatim. ##### clean-postcodes -::: nominatim.tokenizer.sanitizers.clean_postcodes +::: nominatim_db.tokenizer.sanitizers.clean_postcodes options: members: False heading_level: 6 @@ -215,7 +173,7 @@ The following is a list of sanitizers that are shipped with Nominatim. ##### clean-tiger-tags -::: nominatim.tokenizer.sanitizers.clean_tiger_tags +::: nominatim_db.tokenizer.sanitizers.clean_tiger_tags options: members: False heading_level: 6 @@ -223,7 +181,7 @@ The following is a list of sanitizers that are shipped with Nominatim. #### delete-tags -::: nominatim.tokenizer.sanitizers.delete_tags +::: nominatim_db.tokenizer.sanitizers.delete_tags options: members: False heading_level: 6 @@ -231,7 +189,7 @@ The following is a list of sanitizers that are shipped with Nominatim. #### tag-japanese -::: nominatim.tokenizer.sanitizers.tag_japanese +::: nominatim_db.tokenizer.sanitizers.tag_japanese options: members: False heading_level: 6