X-Git-Url: https://git.openstreetmap.org./nominatim.git/blobdiff_plain/139cea5720e3b1030ec9a10c678d723752d707f4..ad50016c498098d0f0850c3a9de60623a3bd7e1f:/docs/develop/Development-Environment.md diff --git a/docs/develop/Development-Environment.md b/docs/develop/Development-Environment.md index 20d02411..441556ff 100644 --- a/docs/develop/Development-Environment.md +++ b/docs/develop/Development-Environment.md @@ -4,7 +4,7 @@ This chapter gives an overview how to set up Nominatim for development and how to run tests. !!! Important - This guide assumes that you develop under the latest version of Debain/Ubuntu. + This guide assumes you develop under the latest version of Debian/Ubuntu. You can of course also use your favourite distribution. You just might have to adapt the commands below slightly, in particular the commands for installing additional software. @@ -26,12 +26,9 @@ following packages should get you started: ## Prerequisites for testing and documentation The Nominatim test suite consists of behavioural tests (using behave) and -unit tests (using PHPUnit for PHP code and pytest for Python code). -It has the following additional requirements: +unit tests (using pytest). It has the following additional requirements: * [behave test framework](https://behave.readthedocs.io) >= 1.2.6 -* [phpunit](https://phpunit.de) (9.5 is known to work) -* [PHP CodeSniffer](https://github.com/squizlabs/PHP_CodeSniffer) * [Pylint](https://pylint.org/) (CI always runs the latest version from pip) * [mypy](http://mypy-lang.org/) (plus typing information for external libs) * [Python Typing Extensions](https://github.com/python/typing_extensions) (for Python < 3.9) @@ -41,14 +38,19 @@ It has the following additional requirements: For testing the Python search frontend, you need to install extra dependencies depending on your choice of webserver framework: -* [httpx](https://www.python-httpx.org/) (starlette only) -* [asgi-lifespan](https://github.com/florimondmanca/asgi-lifespan) (starlette only) +* [httpx](https://www.python-httpx.org/) (Starlette only) +* [asgi-lifespan](https://github.com/florimondmanca/asgi-lifespan) (Starlette only) The documentation is built with mkdocs: * [mkdocs](https://www.mkdocs.org/) >= 1.1.2 * [mkdocstrings](https://mkdocstrings.github.io/) >= 0.25 * [mkdocs-material](https://squidfunk.github.io/mkdocs-material/) +* [mkdocs-gen-files](https://oprypin.github.io/mkdocs-gen-files/) + +Please be aware that tests always run against the globally installed +osm2pgsql, so you need to have this set up. If you want to test against +the vendored version of osm2pgsql, you need to set the PATH accordingly. ### Installing prerequisites on Ubuntu/Debian @@ -57,17 +59,22 @@ In particular, pylint tends to have a lot of breaking changes between versions. The easiest way, to handle these Python dependencies is to run your development from within a virtual environment. +```sh +sudo apt install libsqlite3-mod-spatialite +``` + To set up the virtual environment with all necessary packages run: ```sh virtualenv ~/nominatim-dev-venv ~/nominatim-dev-venv/bin/pip install\ - psycopg2-binary psutil psycopg[binary] PyICU SQLAlchemy \ - python-dotenv jinja2 pyYAML datree \ - behave mkdocs mkdocstrings pytest pytest-asyncio pylint \ + psutil psycopg[binary] PyICU SQLAlchemy \ + python-dotenv jinja2 pyYAML datrie behave \ + mkdocs mkdocstrings mkdocs-gen-files pytest pytest-asyncio pylint \ types-jinja2 types-markupsafe types-psutil types-psycopg2 \ types-pygments types-pyyaml types-requests types-ujson \ - types-urllib3 typing-extensions unicorn falcon + types-urllib3 typing-extensions unicorn falcon starlette \ + uvicorn mypy osmium aiosqlite ``` Now enter the virtual environment whenever you want to develop: @@ -76,28 +83,6 @@ Now enter the virtual environment whenever you want to develop: . ~/nominatim-dev-venv/bin/activate ``` -For installing the PHP development tools, run: - -```sh -sudo apt install php-cgi phpunit php-codesniffer -``` - -If your distribution does not have PHPUnit 7.3+, you can install it (as well -as CodeSniffer) via composer: - -``` -sudo apt-get install composer -composer global require "squizlabs/php_codesniffer=*" -composer global require "phpunit/phpunit=8.*" -``` - -The binaries are found in `.config/composer/vendor/bin`. You need to add this -to your PATH: - -``` -echo 'export PATH=~/.config/composer/vendor/bin:$PATH' > ~/.profile -``` - ### Running Nominatim during development The source code for Nominatim can be found in the `src` directory and can @@ -106,12 +91,12 @@ be run in-place. The source directory features a special script but executes against the code in the source tree. For example: ``` -me@machine:~$ cd Nomiantim -me@machine:~Nomiantim$ ./nominatim-cli.py --version +me@machine:~$ cd Nominatim +me@machine:~Nominatim$ ./nominatim-cli.py --version Nominatim version 4.4.99-1 ``` -Make sure you have activated the virtual environment that holds all +Make sure you have activated the virtual environment holding all necessary dependencies. ## Executing Tests @@ -124,7 +109,14 @@ To run all tests, run make from the source root: make tests ``` -There are also goals for executing parts of the test suite: mypy, lint, pytest, bdd. +There are also make targets for executing only parts of the test suite. +For example to run linting only use: + +```sh +make lint +``` + +The possible testing targets are: mypy, lint, pytest, bdd. For more information about the structure of the tests and how to change and extend the test suite, see the [Testing chapter](Testing.md). @@ -136,18 +128,14 @@ built using the [MkDocs](https://www.mkdocs.org/) static site generation framework. The master branch is automatically deployed every night on [https://nominatim.org/release-docs/develop/](https://nominatim.org/release-docs/develop/) -To build the documentation, go to the build directory and run +To build the documentation run ``` make doc -INFO - Cleaning site directory -INFO - Building documentation to directory: /home/vagrant/build/site-html ``` -This runs `mkdocs build` plus extra transformation of some files and adds -symlinks (see `CMakeLists.txt` for the exact steps). -Now you can start webserver for local testing +For local testing, you can start webserver: ``` build> make serve-doc @@ -159,7 +147,7 @@ If you develop inside a Vagrant virtual machine, use a port that is forwarded to your host: ``` -build> PYTHONPATH=$SRCDIR mkdocs serve --dev-addr 0.0.0.0:8088 +build> mkdocs serve --dev-addr 0.0.0.0:8088 [server:296] Serving on http://0.0.0.0:8088 [handlers:62] Start watching changes ```