X-Git-Url: https://git.openstreetmap.org./nominatim.git/blobdiff_plain/f0088ca2be3bafc77993558d794715c652eb7b25..d3604b8594ce1c2ec01bac675fc35b49ab8fc7df:/docs/admin/Import-and-Update.md?ds=sidebyside diff --git a/docs/admin/Import-and-Update.md b/docs/admin/Import-and-Update.md index 179b729e..3d090597 100644 --- a/docs/admin/Import-and-Update.md +++ b/docs/admin/Import-and-Update.md @@ -58,13 +58,93 @@ Nominatim can use postcodes from an external source to improve searches that inv cd $NOMINATIM_SOURCE_DIR/data wget https://www.nominatim.org/data/gb_postcode_data.sql.gz +## Choosing the Data to Import + +In its default setup Nominatim is configured to import the full OSM data +set for the entire planet. Such a setup requires a powerful machine with +at least 32GB of RAM and around 800GB of SSD hard disks. Depending on your +use case there are various ways to reduce the amount of data imported. This +section discusses these methods. They can also be combined. + +### Using an extract + +If you only need geocoding for a smaller region, then precomputed extracts +are a good way to reduce the database size and import time. +[Geofabrik](https://download.geofabrik.de) offers extracts for most countries. +They even have daily updates which can be used with the update process described +below. There are also +[other providers for extracts](https://wiki.openstreetmap.org/wiki/Planet.osm#Downloading). + +Please be aware that some extracts are not cut exactly along the country +boundaries. As a result some parts of the boundary may be missing which means +that cannot compute the areas for some administrative areas. + +### Dropping Data Required for Dynamic Updates + +About half of the data in Nominatim's database is not really used for serving +the API. It is only there to allow the data to be updated from the latest +changes from OSM. For many uses these dynamic updates are not really required. +If you don't plan to apply updates, the dynamic part of the database can be +safely dropped using the following command: + +``` +./utils/setup.php --drop +``` + +Note that you still need to provide for sufficient disk space for the initial +import. So this option is particularly interesting if you plan to transfer the +database or reuse the space later. + +### Reverse-only Imports + +If you only want to use the Nominatim database for reverse lookups or +if you plan to use the installation only for exports to a +[photon](http://photon.komoot.de/) database, then you can set up a database +without search indexes. Add `--reverse-only` to your setup command above. + +This saves about 5% of disk space. + +### Filtering Imported Data + +Nominatim normally sets up a full search database containing administrative +boundaries, places, streets, addresses and POI data. There are also other +import styles available which only read selected data: + +* **settings/import-admin.style** + Only import administrative boundaries and places. +* **settings/import-street.style** + Like the admin style but also adds streets. +* **settings/import-address.style** + Import all data necessary to compute addresses down to house number level. +* **settings/import-full.style** + Default style that also includes points of interest. + +The style can be changed with the configuration `CONST_Import_Style`. + +To give you an idea of the impact of using the different style, the table +below gives rough estimates of the final database size after import of a +2018 planet and after using the `--drop` option. It also shows the time +needed for the import on a machine with 32GB RAM, 4 CPUS and SSDs. Note that +the given sizes are just an estimate meant for comparison of style requirements. +Your planet import is likely to be larger as the OSM data grows with time. + +style | Import time | DB size | after drop +----------|--------------|------------|------------ +admin | 5h | 190 GB | 20 GB +street | 42h | 400 GB | 180 GB +address | 59h | 500 GB | 260 GB +full | 80h | 575 GB | 300 GB + +You can also customize the styles further. For an description of the +style format see [the developement section](../develop/Import.md). ## Initial import of the data -**Important:** first try the import with a small excerpt, for example from +**Important:** first try the import with a small extract, for example from [Geofabrik](https://download.geofabrik.de). -Download the data to import and load the data with the following command: +Download the data to import and load the data with the following command +from the build directory: ```sh ./utils/setup.php --osm-file --all [--osm2pgsql-cache 28000] 2>&1 | tee setup.log @@ -98,15 +178,6 @@ you also need to enable these key phrases like this: Note that this command downloads the phrases from the wiki link above. -### Reverse-only Imports - -If you only want to use the Nominatim database for reverse lookups or -if you plan to use the installation only for exports to a -[photon](http://photon.komoot.de/) database, then you can set up a database -without search indexes. Add `--reverse-only` to your setup command above. - -This saves about 5% of disk space. - ## Installing Tiger housenumber data for the US