Merge pull request #3554 from lonvia/postcode-bbox

[nominatim.git] / docs / admin / Installation.md

diff --git a/docs/admin/Installation.md b/docs/admin/Installation.md
index fe4749e25ea215628112854037f7ff93dc1dad0e..78062908c9ca230179023a960008a6f04fd9deb1 100644 (file)
--- a/docs/admin/Installation.md
+++ b/docs/admin/Installation.md
@@ -4,9 +4,8 @@ This page contains generic installation instructions for Nominatim and its
 prerequisites. There are also step-by-step instructions available for
 the following operating systems:
 
 prerequisites. There are also step-by-step instructions available for
 the following operating systems:
 

-  * [Ubuntu 18.04](../appendix/Install-on-Ubuntu-18.md)
-  * [Ubuntu 16.04](../appendix/Install-on-Ubuntu-16.md)
-  * [CentOS 7.2](../appendix/Install-on-Centos-7.md)
+  * [Ubuntu 24.04](Install-on-Ubuntu-24.md)
+  * [Ubuntu 22.04](Install-on-Ubuntu-22.md)

 
 These OS-specific instructions can also be found in executable form
 in the `vagrant/` directory.
 
 These OS-specific instructions can also be found in executable form
 in the `vagrant/` directory.


@@ -16,156 +15,176 @@ and can't offer support.
 
   * [Docker](https://github.com/mediagis/nominatim-docker)
   * [Docker on Kubernetes](https://github.com/peter-evans/nominatim-k8s)
 
   * [Docker](https://github.com/mediagis/nominatim-docker)
   * [Docker on Kubernetes](https://github.com/peter-evans/nominatim-k8s)

+  * [Kubernetes with Helm](https://github.com/robjuz/helm-charts/blob/master/charts/nominatim/README.md)

   * [Ansible](https://github.com/synthesio/infra-ansible-nominatim)
 
 ## Prerequisites
 
 ### Software
 
   * [Ansible](https://github.com/synthesio/infra-ansible-nominatim)
 
 ## Prerequisites
 
 ### Software
 

-For compiling:
+!!! Warning
+    For larger installations you **must have** PostgreSQL 11+ and PostGIS 3+
+    otherwise import and queries will be slow to the point of being unusable.
+    Query performance has marked improvements with PostgreSQL 13+ and PostGIS 3.2+.

 
 

-  * [cmake](https://cmake.org/)
-  * [libxml2](http://xmlsoft.org/)
-  * a recent C++ compiler
+For running Nominatim:

 
 

-Nominatim comes with its own version of osm2pgsql. See the
-osm2pgsql README for additional dependencies required for compiling osm2pgsql.
+  * [PostgreSQL](https://www.postgresql.org) (9.6+ will work, 11+ strongly recommended)
+  * [PostGIS](https://postgis.net) (2.2+ will work, 3.0+ strongly recommended)
+  * [osm2pgsql](https://osm2pgsql.org) (1.8+, optional when building with CMake)
+  * [Python 3](https://www.python.org/) (3.7+)

 
 

-For running tests:
+Furthermore the following Python libraries are required:

 
 

-  * [behave](http://pythonhosted.org/behave/)
-  * [Psycopg2](http://initd.org/psycopg)
-  * [nose](https://nose.readthedocs.io)
-  * [phpunit](https://phpunit.de)
+  * [Psycopg3](https://www.psycopg.org)
+  * [Python Dotenv](https://github.com/theskumar/python-dotenv)
+  * [psutil](https://github.com/giampaolo/psutil)
+  * [Jinja2](https://palletsprojects.com/p/jinja/)
+  * [PyICU](https://pypi.org/project/PyICU/)
+  * [PyYaml](https://pyyaml.org/) (5.1+)
+  * [datrie](https://github.com/pytries/datrie)

 
 

-For running Nominatim:
+These will be installed automatically when using pip installation.

 
 

-  * [PostgreSQL](http://www.postgresql.org) (9.1 or later)
-  * [PostGIS](http://postgis.refractions.net) (2.0 or later)
-  * [PHP](http://php.net) (5.4 or later)
-  * PHP-pgsql
-  * PHP-intl (bundled with PHP)
-  * [PEAR::DB](http://pear.php.net/package/DB)
-  * a webserver (apache or nginx are recommended)
+When using legacy CMake-based installation:
+
+  * [cmake](https://cmake.org/)
+  * [expat](https://libexpat.github.io/)
+  * [proj](https://proj.org/)
+  * [bzip2](http://www.bzip.org/)
+  * [zlib](https://www.zlib.net/)
+  * [ICU](http://site.icu-project.org/)
+  * [nlohmann/json](https://json.nlohmann.me/)
+  * [Boost libraries](https://www.boost.org/), including system and file system
+  * PostgreSQL client libraries
+  * a recent C++ compiler (gcc 5+ or Clang 3.8+)

 
 For running continuous updates:
 
 
 For running continuous updates:
 

-  * [pyosmium](http://osmcode.org/pyosmium/)
+  * [pyosmium](https://osmcode.org/pyosmium/)

 
 

-### Hardware
+For running the Python frontend:

 
 

-A minimum of 2GB of RAM is required or installation will fail. For a full
-planet import 32GB of RAM or more strongly are recommended.
+  * [SQLAlchemy](https://www.sqlalchemy.org/) (1.4.31+ with greenlet support)
+  * [asyncpg](https://magicstack.github.io/asyncpg) (0.8+, only when using SQLAlchemy < 2.0)
+  * one of the following web frameworks:
+    * [falcon](https://falconframework.org/) (3.0+)
+    * [starlette](https://www.starlette.io/)
+  * [uvicorn](https://www.uvicorn.org/)

 
 

-For a full planet install you will need at least 700GB of hard disk space
-(take into account that the OSM database is growing fast). SSD disks
-will help considerably to speed up import and queries.
+For dependencies for running tests and building documentation, see
+the [Development section](../develop/Development-Environment.md).

 
 

-On a 6-core machine with 32GB RAM and SSDs the import of a full planet takes
-a bit more than 2 days. Without SSDs 7-8 days are more realistic.
+### Hardware
+
+A minimum of 2GB of RAM is required or installation will fail. For a full
+planet import 128GB of RAM or more are strongly recommended. Do not report
+out of memory problems if you have less than 64GB RAM.

 
 

+For a full planet install you will need at least 1TB of hard disk space.
+Take into account that the OSM database is growing fast.
+Fast disks are essential. Using NVME disks is recommended.

 
 

-## Setup of the server
+Even on a well configured machine the import of a full planet takes
+around 2.5 days. When using traditional SSDs, 4-5 days are more realistic.

 
 

-### PostgreSQL tuning
+## Tuning the PostgreSQL database

 
 You might want to tune your PostgreSQL installation so that the later steps
 make best use of your hardware. You should tune the following parameters in
 your `postgresql.conf` file.
 
 
 You might want to tune your PostgreSQL installation so that the later steps
 make best use of your hardware. You should tune the following parameters in
 your `postgresql.conf` file.
 

-    shared_buffers (2GB)
-    maintenance_work_mem (10GB)
-    work_mem (50MB)
-    effective_cache_size (24GB)
+    shared_buffers = 2GB
+    maintenance_work_mem = (10GB)
+    autovacuum_work_mem = 2GB
+    work_mem = (50MB)

     synchronous_commit = off
     synchronous_commit = off

-    checkpoint_segments = 100 # only for postgresql <= 9.4
-    checkpoint_timeout = 10min
+    max_wal_size = 1GB
+    checkpoint_timeout = 60min

     checkpoint_completion_target = 0.9
     checkpoint_completion_target = 0.9

+    random_page_cost = 1.0
+    wal_level = minimal
+    max_wal_senders = 0

 
 The numbers in brackets behind some parameters seem to work fine for
 
 The numbers in brackets behind some parameters seem to work fine for

-32GB RAM machine. Adjust to your setup.
+128GB RAM machine. Adjust to your setup. A higher number for `max_wal_size`
+means that PostgreSQL needs to run checkpoints less often but it does require
+the additional space on your disk.
+
+Autovacuum must not be switched off because it ensures that the
+tables are frequently analysed. If your machine has very little memory,
+you might consider setting:
+
+    autovacuum_max_workers = 1
+
+and even reduce `autovacuum_work_mem` further. This will reduce the amount
+of memory that autovacuum takes away from the import process.
+
+## Downloading and building Nominatim

 
 

-For the initial import, you should also set:
+### Downloading the latest release

 
 

-    fsync = off
-    full_page_writes = off
+You can download the [latest release from nominatim.org](https://nominatim.org/downloads/).
+The release contains all necessary files. Just unpack it.

 
 

-Don't forget to reenable them after the initial import or you risk database
-corruption. Autovacuum must not be switched off because it ensures that the
-tables are frequently analysed.
+### Downloading the latest development version

 
 

-### Webserver setup
+If you want to install latest development version from github, make sure to
+also check out the osm2pgsql subproject:

 
 

-The `website/` directory in the build directory contains the configured
-website. Include the directory into your webbrowser to serve php files
-from there.
+```
+git clone --recursive https://github.com/openstreetmap/Nominatim.git
+```

 
 

-#### Configure for use with Apache
+The development version does not include the country grid. Download it separately:

 
 

-Make sure your Apache configuration contains the required permissions for the
-directory and create an alias:
+```
+wget -O Nominatim/data/country_osm_grid.sql.gz https://nominatim.org/data/country_grid.sql.gz
+```

 
 

-    <Directory "/srv/nominatim/build/website">
-      Options FollowSymLinks MultiViews
-      AddType text/html   .php
-      DirectoryIndex search.php
-      Require all granted
-    </Directory>
-    Alias /nominatim /srv/nominatim/build/website
+### Building Nominatim

 
 

-`/srv/nominatim/build` should be replaced with the location of your
-build directory.
+#### Building the latest development version with pip

 
 

-After making changes in the apache config you need to restart apache.
-The website should now be available on http://localhost/nominatim.
+Nominatim is easiest to run from its own virtual environment. To create one, run:

 
 

-#### Configure for use with Nginx
+    sudo apt-get install virtualenv
+    virtualenv /srv/nominatim-venv

 
 

-Use php-fpm as a deamon for serving PHP cgi. Install php-fpm together with nginx.
+To install Nominatim directly from the source tree into the virtual environment, run:

 
 

-By default php listens on a network socket. If you want it to listen to a
-Unix socket instead, change the pool configuration (`pool.d/www.conf`) as
-follows:
+    /srv/nominatim-venv/bin/pip install packaging/nominatim-{db,api}

 
 

-    ; Comment out the tcp listener and add the unix socket
-    ;listen = 127.0.0.1:9000
-    listen = /var/run/php5-fpm.sock
+#### Building in legacy CMake mode

 
 

-    ; Ensure that the daemon runs as the correct user
-    listen.owner = www-data
-    listen.group = www-data
-    listen.mode = 0666
+!!! warning
+    Installing Nominatim through CMake is now deprecated. The infrastructure
+    will be removed in Nominatim 5.0. Please switch to pip installation.

 
 

-Tell nginx that php files are special and to fastcgi_pass to the php-fpm
-unix socket by adding the location definition to the default configuration.
+The code must be built in a separate directory. Create the directory and
+change into it.

 
 

-    root /srv/nominatim/build/website;
-    index search.php index.html;
-    location / {
-            try_files $uri $uri/ @php;
-    }
+```
+mkdir build
+cd build
+```

 
 

-    location @php {
-            fastcgi_param SCRIPT_FILENAME "$document_root$uri.php";
-            fastcgi_param PATH_TRANSLATED "$document_root$uri.php";
-            fastcgi_param QUERY_STRING    $args;
-            fastcgi_pass unix:/var/run/php/php7.2-fpm.sock;
-            fastcgi_index index.php;
-            include fastcgi_params;
-    }
+Nominatim uses cmake and make for building. Assuming that you have created the
+build at the same level as the Nominatim source directory run:

 
 

-    location ~ [^/]\.php(/|$) {
-        fastcgi_split_path_info ^(.+?\.php)(/.*)$;
-        if (!-f $document_root$fastcgi_script_name) {
-            return 404;
-        }
-        fastcgi_pass unix:/var/run/php7.2-fpm.sock;
-        fastcgi_index search.php;
-        include fastcgi.conf;
-    }
+```
+cmake ../Nominatim
+make
+sudo make install
+```

 
 

-Restart the nginx and php5-fpm services and the website should now be available
-at `http://localhost/`.
+Nominatim installs itself into `/usr/local` per default. To choose a different
+installation directory add `-DCMAKE_INSTALL_PREFIX=<install root>` to the
+cmake command. Make sure that the `bin` directory is available in your path
+in that case, e.g.

 
 

+```
+export PATH=<install root>/bin:$PATH
+```

 
 

-Now continue with [importing the database](Import-and-Update.md).
+Now continue with [importing the database](Import.md).