X-Git-Url: https://git.openstreetmap.org./nominatim.git/blobdiff_plain/3c7519444867cd9c6b3003c4c88729df21b8c265..4efad0bb95024e2082f73b4ae33613ef8f6ebef3:/docs/develop/Development-Environment.md diff --git a/docs/develop/Development-Environment.md b/docs/develop/Development-Environment.md index f0c36ba3..c6515d2c 100644 --- a/docs/develop/Development-Environment.md +++ b/docs/develop/Development-Environment.md @@ -1,6 +1,6 @@ # Setting up Nominatim for Development -This chapter gives an overview how to set up Nominatim for developement +This chapter gives an overview how to set up Nominatim for development and how to run tests. !!! Important @@ -26,16 +26,22 @@ following packages should get you started: ## Prerequisites for testing and documentation The Nominatim test suite consists of behavioural tests (using behave) and -unit tests (using PHPUnit). It has the following additional requirements: +unit tests (using PHPUnit for PHP code and pytest for Python code). +It has the following additional requirements: -* [behave test framework](https://behave.readthedocs.io) >= 1.2.5 -* [nose](https://nose.readthedocs.io) -* [phpunit](https://phpunit.de) >= 7.3 +* [behave test framework](https://behave.readthedocs.io) >= 1.2.6 +* [phpunit](https://phpunit.de) (9.5 is known to work) * [PHP CodeSniffer](https://github.com/squizlabs/PHP_CodeSniffer) +* [Pylint](https://pylint.org/) (CI always runs the latest version from pip) +* [mypy](http://mypy-lang.org/) (plus typing information for external libs) +* [Python Typing Extensions](https://github.com/python/typing_extensions) (for Python < 3.9) +* [pytest](https://pytest.org) The documentation is built with mkdocs: * [mkdocs](https://www.mkdocs.org/) >= 1.1.2 +* [mkdocstrings](https://mkdocstrings.github.io/) >= 0.16 +* [mkdocstrings-python-legacy](https://mkdocstrings.github.io/python-legacy/) ### Installing prerequisites on Ubuntu/Debian @@ -49,7 +55,8 @@ To install all necessary packages run: sudo apt install php-cgi phpunit php-codesniffer \ python3-pip python3-setuptools python3-dev -pip3 install --user behave nose mkdocs +pip3 install --user behave mkdocs mkdocstrings pytest pylint \ + mypy types-PyYAML types-jinja2 types-psycopg2 types-psutil ``` The `mkdocs` executable will be located in `.local/bin`. You may have to add @@ -78,65 +85,15 @@ echo 'export PATH=~/.config/composer/vendor/bin:$PATH' > ~/.profile ## Executing Tests -All tests are located in the `\test` directory. +All tests are located in the `/test` directory. -### Preparing the test database - -Some of the behavioural test expect a test database to be present. You need at -least 2GB RAM and 10GB disk space to create the database. - -First create a separate directory for the test DB and fetch the test planet -data and the Tiger data for South Dakota: - -``` -mkdir testdb -cd testdb -wget https://www.nominatim.org/data/test/nominatim-api-testdata.pbf -wget -O - https://nominatim.org/data/tiger2018-nominatim-preprocessed.tar.gz | tar xz --wildcards --no-anchored '46*' -``` - -Configure and build Nominatim in the usual way: - -``` -cmake $USERNAME/Nominatim -make -``` - -Create a minimal test settings file: - -``` -tee .env << EOF -NOMINATIM_DATABASE_DSN="pgsql:dbname=test_api_nominatim" -NOMINATIM_USE_US_TIGER_DATA=yes -NOMINATIM_TIGER_DATA_PATH=tiger -NOMINATIM_WIKIPEDIA_DATA_PATH=$USERNAME/Nominatim/test/testdb -EOF -``` - -Inspect the file to check that all settings are correct for your local setup. -In particular, the wikipedia path should point to the test directory in your -Nominatim source directory. - -Now you can import the test database: - -``` -dropdb --if-exists test_api_nominatim -./utils/setup.php --all --osm-file nominatim-api-testdb.pbf 2>&1 | tee import.log -./utils/specialphrases.php --wiki-import | psql -d test_api_nominatim 2>&1 | tee -a import.log -./utils/setup.php --import-tiger-data 2>&1 | tee -a import.log -``` - -### Running the tests - -To run all tests just go to the test directory and run make: +To run all tests just go to the build directory and run make: ```sh -cd test -make +cd build +make test ``` -To skip tests that require the test database, run `make no-test-db` instead. - For more information about the structure of the tests and how to change and extend the test suite, see the [Testing chapter](Testing.md). @@ -161,7 +118,7 @@ symlinks (see `CMakeLists.txt` for the exact steps). Now you can start webserver for local testing ``` -build> mkdocs serve +build> make serve-doc [server:296] Serving on http://127.0.0.1:8000 [handlers:62] Start watching changes ``` @@ -170,7 +127,7 @@ If you develop inside a Vagrant virtual machine, use a port that is forwarded to your host: ``` -build> mkdocs serve --dev-addr 0.0.0.0:8088 +build> PYTHONPATH=$SRCDIR mkdocs serve --dev-addr 0.0.0.0:8088 [server:296] Serving on http://0.0.0.0:8088 [handlers:62] Start watching changes ```