From 751563644fe6572f7c63f67525ae56f4d9133e5c Mon Sep 17 00:00:00 2001 From: Sarah Hoffmann Date: Tue, 12 Oct 2021 21:25:13 +0200 Subject: [PATCH] docs: make customization chapter a separate section --- docs/CMakeLists.txt | 1 + docs/admin/Customization.md | 101 ------------------------ docs/customize/Overview.md | 16 ++++ docs/customize/Postcodes.md | 37 +++++++++ docs/customize/Settings.md | 27 +++++++ docs/customize/Special-Phrases.md | 34 ++++++++ docs/customize/Tiger.md | 28 +++++++ docs/{admin => customize}/Tokenizers.md | 0 docs/mkdocs.yml | 9 ++- 9 files changed, 150 insertions(+), 103 deletions(-) delete mode 100644 docs/admin/Customization.md create mode 100644 docs/customize/Overview.md create mode 100644 docs/customize/Postcodes.md create mode 100644 docs/customize/Settings.md create mode 100644 docs/customize/Special-Phrases.md create mode 100644 docs/customize/Tiger.md rename docs/{admin => customize}/Tokenizers.md (100%) diff --git a/docs/CMakeLists.txt b/docs/CMakeLists.txt index 6472722a..191c221c 100644 --- a/docs/CMakeLists.txt +++ b/docs/CMakeLists.txt @@ -10,6 +10,7 @@ set (DOC_SOURCES admin develop api + customize index.md extra.css styles.css diff --git a/docs/admin/Customization.md b/docs/admin/Customization.md deleted file mode 100644 index fc1d45cf..00000000 --- a/docs/admin/Customization.md +++ /dev/null @@ -1,101 +0,0 @@ -# Customization of the Database - -This section explains in detail how to configure a Nominatim import and -the various means to use external data. - -## External postcode data - -Nominatim creates a table of known postcode centroids during import. This table -is used for searches of postcodes and for adding postcodes to places where the -OSM data does not provide one. These postcode centroids are mainly computed -from the OSM data itself. In addition, Nominatim supports reading postcode -information from an external CSV file, to supplement the postcodes that are -missing in OSM. - -To enable external postcode support, simply put one CSV file per country into -your project directory and name it `_postcodes.csv`. `` must be the -two-letter country code for which to apply the file. The file may also be -gzipped. Then it must be called `_postcodes.csv.gz`. - -The CSV file must use commas as a delimiter and have a header line. Nominatim -expects three columns to be present: `postcode`, `lat` and `lon`. All other -columns are ignored. `lon` and `lat` must describe the x and y coordinates of the -postcode centroids in WGS84. - -The postcode files are loaded only when there is data for the given country -in your database. For example, if there is a `us_postcodes.csv` file in your -project directory but you import only an excerpt of Italy, then the US postcodes -will simply be ignored. - -As a rule, the external postcode data should be put into the project directory -**before** starting the initial import. Still, you can add, remove and update the -external postcode data at any time. Simply -run: - -``` -nominatim refresh --postcodes -``` - -to make the changes visible in your database. Be aware, however, that the changes -only have an immediate effect on searches for postcodes. Postcodes that were -added to places are only updated, when they are reindexed. That usually happens -only during replication updates. - -## Installing Tiger housenumber data for the US - -Nominatim is able to use the official [TIGER](https://www.census.gov/geographies/mapping-files/time-series/geo/tiger-line-file.html) -address set to complement the OSM house number data in the US. You can add -TIGER data to your own Nominatim instance by following these steps. The -entire US adds about 10GB to your database. - - 1. Get preprocessed TIGER 2021 data: - - cd $PROJECT_DIR - wget https://nominatim.org/data/tiger2021-nominatim-preprocessed.csv.tar.gz - - 2. Import the data into your Nominatim database: - - nominatim add-data --tiger-data tiger2021-nominatim-preprocessed.csv.tar.gz - - 3. Enable use of the Tiger data in your `.env` by adding: - - echo NOMINATIM_USE_US_TIGER_DATA=yes >> .env - - 4. Apply the new settings: - - nominatim refresh --functions - - -See the [developer's guide](../develop/data-sources.md#us-census-tiger) for more -information on how the data got preprocessed. - -## Special phrases import - -As described in the [Importation chapter](Import.md), it is possible to -import special phrases from the wiki with the following command: - -```sh -nominatim special-phrases --import-from-wiki -``` - -But, it is also possible to import some phrases from a csv file. -To do so, you have access to the following command: - -```sh -nominatim special-phrases --import-from-csv -``` - -Note that the two previous import commands will update the phrases from your database. -This means that if you import some phrases from a csv file, only the phrases -present in the csv file will be kept into the database. All other phrases will -be removed. - -If you want to only add new phrases and not update the other ones you can add -the argument `--no-replace` to the import command. For example: - -```sh -nominatim special-phrases --import-from-csv --no-replace -``` - -This will add the phrases present in the csv file into the database without -removing the other ones. diff --git a/docs/customize/Overview.md b/docs/customize/Overview.md new file mode 100644 index 00000000..b86a7164 --- /dev/null +++ b/docs/customize/Overview.md @@ -0,0 +1,16 @@ +Nominatim comes with a predefined set of configuration options that should +work for most standard installations. If you have special requirements, there +are many places where the configuration can be adapted. This chapter describes +the following configurable parts: + +* [Global Settings](Settings.md) - detailed description of all parameters that + can be set in your local `.env` configuration +* [Tokenizers](Tokenizers.md) - describe the configuration of the module + responsible for analysing and indexing names +* [Special Phrases](Special-Phrases.md) are common nouns or phrases that + can be used in search to identify a class of places + +There are also guides for adding the following external data: + +* [US house numbers from the TIGER dataset](Tiger.md) +* [External postcodes](Postcodes.md) diff --git a/docs/customize/Postcodes.md b/docs/customize/Postcodes.md new file mode 100644 index 00000000..96ab9d15 --- /dev/null +++ b/docs/customize/Postcodes.md @@ -0,0 +1,37 @@ +# External postcode data + +Nominatim creates a table of known postcode centroids during import. This table +is used for searches of postcodes and for adding postcodes to places where the +OSM data does not provide one. These postcode centroids are mainly computed +from the OSM data itself. In addition, Nominatim supports reading postcode +information from an external CSV file, to supplement the postcodes that are +missing in OSM. + +To enable external postcode support, simply put one CSV file per country into +your project directory and name it `_postcodes.csv`. `` must be the +two-letter country code for which to apply the file. The file may also be +gzipped. Then it must be called `_postcodes.csv.gz`. + +The CSV file must use commas as a delimiter and have a header line. Nominatim +expects three columns to be present: `postcode`, `lat` and `lon`. All other +columns are ignored. `lon` and `lat` must describe the x and y coordinates of the +postcode centroids in WGS84. + +The postcode files are loaded only when there is data for the given country +in your database. For example, if there is a `us_postcodes.csv` file in your +project directory but you import only an excerpt of Italy, then the US postcodes +will simply be ignored. + +As a rule, the external postcode data should be put into the project directory +**before** starting the initial import. Still, you can add, remove and update the +external postcode data at any time. Simply +run: + +``` +nominatim refresh --postcodes +``` + +to make the changes visible in your database. Be aware, however, that the changes +only have an immediate effect on searches for postcodes. Postcodes that were +added to places are only updated, when they are reindexed. That usually happens +only during replication updates. diff --git a/docs/customize/Settings.md b/docs/customize/Settings.md new file mode 100644 index 00000000..92afc8e0 --- /dev/null +++ b/docs/customize/Settings.md @@ -0,0 +1,27 @@ +This section provides a reference of all configuration parameters that can +be used with Nominatim. + +# Configuring Nominatim + +Nominatim uses [dotenv](https://github.com/theskumar/python-dotenv) to manage +its configuration settings. There are two means to set configuration +variables: through an `.env` configuration file or through an environment +variable. + +The `.env` configuration file needs to be placed into the +[project directory](../admin/Import/#creating-the-project-directory). It +must contain configuration parameters in `=` format. +Please refer to the dotenv documentation for details. + +The configuration options may also be set in the form of shell environment +variables. This is particularly useful, when you want to temporarily change +a configuration option. For example, to force the replication serve to +download the next change, you can temporarily disable the update interval: + + NOMINATIM_REPLICATION_UPDATE_INTERVAL=0 nominatim replication --once + +If a configuration option is defined through .env file and environment +variable, then the latter takes precedence. + +# Configuration Parameter Reference + diff --git a/docs/customize/Special-Phrases.md b/docs/customize/Special-Phrases.md new file mode 100644 index 00000000..3c49ef14 --- /dev/null +++ b/docs/customize/Special-Phrases.md @@ -0,0 +1,34 @@ +# Special phrases + +## Importing OSM user-maintained special phrases + +As described in the [Importation chapter](Import.md), it is possible to +import special phrases from the wiki with the following command: + +```sh +nominatim special-phrases --import-from-wiki +``` + +## Importing custom special phrases + +But, it is also possible to import some phrases from a csv file. +To do so, you have access to the following command: + +```sh +nominatim special-phrases --import-from-csv +``` + +Note that the two previous import commands will update the phrases from your database. +This means that if you import some phrases from a csv file, only the phrases +present in the csv file will be kept into the database. All other phrases will +be removed. + +If you want to only add new phrases and not update the other ones you can add +the argument `--no-replace` to the import command. For example: + +```sh +nominatim special-phrases --import-from-csv --no-replace +``` + +This will add the phrases present in the csv file into the database without +removing the other ones. diff --git a/docs/customize/Tiger.md b/docs/customize/Tiger.md new file mode 100644 index 00000000..3f1c3546 --- /dev/null +++ b/docs/customize/Tiger.md @@ -0,0 +1,28 @@ +# Installing TIGER housenumber data for the US + +Nominatim is able to use the official [TIGER](https://www.census.gov/geographies/mapping-files/time-series/geo/tiger-line-file.html) +address set to complement the OSM house number data in the US. You can add +TIGER data to your own Nominatim instance by following these steps. The +entire US adds about 10GB to your database. + + 1. Get preprocessed TIGER 2021 data: + + cd $PROJECT_DIR + wget https://nominatim.org/data/tiger2021-nominatim-preprocessed.csv.tar.gz + + 2. Import the data into your Nominatim database: + + nominatim add-data --tiger-data tiger2021-nominatim-preprocessed.csv.tar.gz + + 3. Enable use of the Tiger data in your `.env` by adding: + + echo NOMINATIM_USE_US_TIGER_DATA=yes >> .env + + 4. Apply the new settings: + + nominatim refresh --functions + + +See the [TIGER-data project](https://github.com/osm-search/TIGER-data) for more +information on how the data got preprocessed. + diff --git a/docs/admin/Tokenizers.md b/docs/customize/Tokenizers.md similarity index 100% rename from docs/admin/Tokenizers.md rename to docs/customize/Tokenizers.md diff --git a/docs/mkdocs.yml b/docs/mkdocs.yml index bc8a6cdd..c4579036 100644 --- a/docs/mkdocs.yml +++ b/docs/mkdocs.yml @@ -19,12 +19,17 @@ pages: - 'Import' : 'admin/Import.md' - 'Update' : 'admin/Update.md' - 'Deploy' : 'admin/Deployment.md' - - 'Customize Imports' : 'admin/Customization.md' - - 'Tokenizers' : 'admin/Tokenizers.md' - 'Nominatim UI' : 'admin/Setup-Nominatim-UI.md' - 'Advanced Installations' : 'admin/Advanced-Installations.md' - 'Migration from older Versions' : 'admin/Migration.md' - 'Troubleshooting' : 'admin/Faq.md' + - 'Customization Guide': + - 'Overview': 'customize/Overview.md' + - 'Configuration Settings': 'customize/Settings.md' + - 'Special Phrases': 'customize/Special-Phrases.md' + - 'Tokenizers' : 'customize/Tokenizers.md' + - 'External data: US housenumbers from TIGER': 'customize/Tiger.md' + - 'External data: Postcodes': 'customize/Postcodes.md' - 'Developers Guide': - 'Setup for Development' : 'develop/Development-Environment.md' - 'Architecture Overview' : 'develop/overview.md' -- 2.39.5