Merge remote-tracking branch 'upstream/master'

[nominatim.git] / docs / develop / Development-Environment.md

diff --git a/docs/develop/Development-Environment.md b/docs/develop/Development-Environment.md
index b5e5ceba6d40068099a58eed359209c86e28ddee..643064549cae9e3291fc9ecc9dca62ad8a687f0e 100644 (file)
--- a/docs/develop/Development-Environment.md
+++ b/docs/develop/Development-Environment.md
@@ -1,6 +1,6 @@
 # Setting up Nominatim for Development
 
 # Setting up Nominatim for Development
 

-This chapter gives an overview how to set up Nominatim for developement
+This chapter gives an overview how to set up Nominatim for development

 and how to run tests.
 
 !!! Important
 and how to run tests.
 
 !!! Important


@@ -26,15 +26,29 @@ following packages should get you started:
 ## Prerequisites for testing and documentation
 
 The Nominatim test suite consists of behavioural tests (using behave) and
 ## Prerequisites for testing and documentation
 
 The Nominatim test suite consists of behavioural tests (using behave) and

-unit tests (using PHPUnit). It has the following additional requirements:
+unit tests (using PHPUnit for PHP code and pytest for Python code).
+It has the following additional requirements:

 
 

-* [behave test framework](https://behave.readthedocs.io) >= 1.2.5
-* [phpunit](https://phpunit.de) >= 7.3
+* [behave test framework](https://behave.readthedocs.io) >= 1.2.6
+* [phpunit](https://phpunit.de) (9.5 is known to work)

 * [PHP CodeSniffer](https://github.com/squizlabs/PHP_CodeSniffer)
 * [PHP CodeSniffer](https://github.com/squizlabs/PHP_CodeSniffer)

+* [Pylint](https://pylint.org/) (CI always runs the latest version from pip)
+* [mypy](http://mypy-lang.org/) (plus typing information for external libs)
+* [Python Typing Extensions](https://github.com/python/typing_extensions) (for Python < 3.9)
+* [pytest](https://pytest.org)
+* [pytest-asyncio](https://pytest-asyncio.readthedocs.io)
+
+For testing the Python search frontend, you need to install extra dependencies
+depending on your choice of webserver framework:
+
+* [httpx](https://www.python-httpx.org/) (starlette only)
+* [asgi-lifespan](https://github.com/florimondmanca/asgi-lifespan) (starlette only)

 
 The documentation is built with mkdocs:
 
 * [mkdocs](https://www.mkdocs.org/) >= 1.1.2
 
 The documentation is built with mkdocs:
 
 * [mkdocs](https://www.mkdocs.org/) >= 1.1.2

+* [mkdocstrings](https://mkdocstrings.github.io/) >= 0.18
+* [mkdocstrings-python](https://mkdocstrings.github.io/python/)

 
 ### Installing prerequisites on Ubuntu/Debian
 
 
 ### Installing prerequisites on Ubuntu/Debian
 


@@ -48,7 +62,10 @@ To install all necessary packages run:
 sudo apt install php-cgi phpunit php-codesniffer \
                  python3-pip python3-setuptools python3-dev
 
 sudo apt install php-cgi phpunit php-codesniffer \
                  python3-pip python3-setuptools python3-dev
 

-pip3 install --user behave mkdocs
+pip3 install --user behave mkdocs mkdocstrings pytest pytest-asyncio pylint \
+                    mypy types-PyYAML types-jinja2 types-psycopg2 types-psutil \
+                    types-ujson types-requests types-Pygments typing-extensions\
+                    httpx asgi-lifespan

 ```
 
 The `mkdocs` executable will be located in `.local/bin`. You may have to add
 ```
 
 The `mkdocs` executable will be located in `.local/bin`. You may have to add


@@ -77,65 +94,15 @@ echo 'export PATH=~/.config/composer/vendor/bin:$PATH' > ~/.profile
 
 ## Executing Tests
 
 
 ## Executing Tests
 

-All tests are located in the `\test` directory.
-
-### Preparing the test database
-
-Some of the behavioural test expect a test database to be present. You need at
-least 2GB RAM and 10GB disk space to create the database.
-
-First create a separate directory for the test DB and fetch the test planet
-data and the Tiger data for South Dakota:
-
-```
-mkdir testdb
-cd testdb
-wget https://www.nominatim.org/data/test/nominatim-api-testdata.pbf
-wget -O - https://nominatim.org/data/tiger2018-nominatim-preprocessed.tar.gz | tar xz --wildcards --no-anchored '46*'
-```
-
-Configure and build Nominatim in the usual way:
-
-```
-cmake $USERNAME/Nominatim
-make
-```
+All tests are located in the `/test` directory.

 
 

-Create a minimal test settings file:
-
-```
-tee .env << EOF
-NOMINATIM_DATABASE_DSN="pgsql:dbname=test_api_nominatim"
-NOMINATIM_USE_US_TIGER_DATA=yes
-NOMINATIM_TIGER_DATA_PATH=tiger
-NOMINATIM_WIKIPEDIA_DATA_PATH=$USERNAME/Nominatim/test/testdb
-EOF
-```
-
-Inspect the file to check that all settings are correct for your local setup.
-In particular, the wikipedia path should point to the test directory in your
-Nominatim source directory.
-
-Now you can import the test database:
-
-```
-dropdb --if-exists test_api_nominatim
-./utils/setup.php --all --osm-file nominatim-api-testdb.pbf 2>&1 | tee import.log
-./utils/specialphrases.php --wiki-import | psql -d test_api_nominatim 2>&1 | tee -a import.log
-./utils/setup.php --import-tiger-data 2>&1 | tee -a import.log
-```
-
-### Running the tests
-
-To run all tests just go to the test directory and run make:
+To run all tests just go to the build directory and run make:

 
 ```sh
 
 ```sh

-cd test
-make
+cd build
+make test

 ```
 
 ```
 

-To skip tests that require the test database, run `make no-test-db` instead.
-

 For more information about the structure of the tests and how to change and
 extend the test suite, see the [Testing chapter](Testing.md).
 
 For more information about the structure of the tests and how to change and
 extend the test suite, see the [Testing chapter](Testing.md).
 


@@ -160,7 +127,7 @@ symlinks (see `CMakeLists.txt` for the exact steps).
 Now you can start webserver for local testing
 
 ```
 Now you can start webserver for local testing
 
 ```

-build> mkdocs serve
+build> make serve-doc

 [server:296] Serving on http://127.0.0.1:8000
 [handlers:62] Start watching changes
 ```
 [server:296] Serving on http://127.0.0.1:8000
 [handlers:62] Start watching changes
 ```


@@ -169,7 +136,7 @@ If you develop inside a Vagrant virtual machine, use a port that is forwarded
 to your host:
 
 ```
 to your host:
 
 ```

-build> mkdocs serve --dev-addr 0.0.0.0:8088
+build> PYTHONPATH=$SRCDIR mkdocs serve --dev-addr 0.0.0.0:8088

 [server:296] Serving on http://0.0.0.0:8088
 [handlers:62] Start watching changes
 ```
 [server:296] Serving on http://0.0.0.0:8088
 [handlers:62] Start watching changes
 ```