X-Git-Url: https://git.openstreetmap.org./nominatim.git/blobdiff_plain/ab6f35d83a350ead4eb12b59c4cbc050e48f93ee..fa1c402b47bdd1f76b8528885c0e3a5e90731cdc:/docs/develop/Development-Environment.md diff --git a/docs/develop/Development-Environment.md b/docs/develop/Development-Environment.md index eba87c09..20d02411 100644 --- a/docs/develop/Development-Environment.md +++ b/docs/develop/Development-Environment.md @@ -1,22 +1,22 @@ # Setting up Nominatim for Development -This chapter gives an overview how to set up Nominatim for developement +This chapter gives an overview how to set up Nominatim for development and how to run tests. !!! Important - This guide assumes that you develop under the latest version of Ubuntu. You - can of course also use your favourite distribution. You just might have to - adapt the commands below slightly, in particular the commands for installing - additional software. + This guide assumes that you develop under the latest version of Debain/Ubuntu. + You can of course also use your favourite distribution. You just might have + to adapt the commands below slightly, in particular the commands for + installing additional software. ## Installing Nominatim The first step is to install Nominatim itself. Please follow the installation instructions in the [Admin section](../admin/Installation.md). You don't need -to set up a webserver for development, the webserver that is included with PHP -is sufficient. +to set up a webserver for development, the webserver that can be started +via `nominatim serve` is sufficient. -If you want to run Nominatim in a VM via Vagrant, use the default `ubuntu` setup. +If you want to run Nominatim in a VM via Vagrant, use the default `ubuntu24` setup. Vagrant's libvirt provider runs out-of-the-box under Ubuntu. You also need to install an NFS daemon to enable directory sharing between host and guest. The following packages should get you started: @@ -32,34 +32,54 @@ It has the following additional requirements: * [behave test framework](https://behave.readthedocs.io) >= 1.2.6 * [phpunit](https://phpunit.de) (9.5 is known to work) * [PHP CodeSniffer](https://github.com/squizlabs/PHP_CodeSniffer) -* [Pylint](https://pylint.org/) (2.6.0 is used for the CI) +* [Pylint](https://pylint.org/) (CI always runs the latest version from pip) +* [mypy](http://mypy-lang.org/) (plus typing information for external libs) +* [Python Typing Extensions](https://github.com/python/typing_extensions) (for Python < 3.9) * [pytest](https://pytest.org) +* [pytest-asyncio](https://pytest-asyncio.readthedocs.io) + +For testing the Python search frontend, you need to install extra dependencies +depending on your choice of webserver framework: + +* [httpx](https://www.python-httpx.org/) (starlette only) +* [asgi-lifespan](https://github.com/florimondmanca/asgi-lifespan) (starlette only) The documentation is built with mkdocs: * [mkdocs](https://www.mkdocs.org/) >= 1.1.2 -* [mkdocstrings](https://mkdocstrings.github.io/) +* [mkdocstrings](https://mkdocstrings.github.io/) >= 0.25 +* [mkdocs-material](https://squidfunk.github.io/mkdocs-material/) ### Installing prerequisites on Ubuntu/Debian -Some of the Python packages require the newest version which is not yet -available with the current distributions. Therefore it is recommended to -install pip to get the newest versions. +The Python tools should always be run with the most recent version. +In particular, pylint tends to have a lot of breaking changes between versions. +The easiest way, to handle these Python dependencies is to run your +development from within a virtual environment. -To install all necessary packages run: +To set up the virtual environment with all necessary packages run: ```sh -sudo apt install php-cgi phpunit php-codesniffer \ - python3-pip python3-setuptools python3-dev pylint - -pip3 install --user behave mkdocs mkdocstrings pytest +virtualenv ~/nominatim-dev-venv +~/nominatim-dev-venv/bin/pip install\ + psycopg2-binary psutil psycopg[binary] PyICU SQLAlchemy \ + python-dotenv jinja2 pyYAML datree \ + behave mkdocs mkdocstrings pytest pytest-asyncio pylint \ + types-jinja2 types-markupsafe types-psutil types-psycopg2 \ + types-pygments types-pyyaml types-requests types-ujson \ + types-urllib3 typing-extensions unicorn falcon ``` -The `mkdocs` executable will be located in `.local/bin`. You may have to add -this directory to your path, for example by running: +Now enter the virtual environment whenever you want to develop: +```sh +. ~/nominatim-dev-venv/bin/activate ``` -echo 'export PATH=~/.local/bin:$PATH' > ~/.profile + +For installing the PHP development tools, run: + +```sh +sudo apt install php-cgi phpunit php-codesniffer ``` If your distribution does not have PHPUnit 7.3+, you can install it (as well @@ -72,24 +92,40 @@ composer global require "phpunit/phpunit=8.*" ``` The binaries are found in `.config/composer/vendor/bin`. You need to add this -to your PATH as well: +to your PATH: ``` echo 'export PATH=~/.config/composer/vendor/bin:$PATH' > ~/.profile ``` +### Running Nominatim during development + +The source code for Nominatim can be found in the `src` directory and can +be run in-place. The source directory features a special script +`nominatim-cli.py` which does the same as the installed 'nominatim' binary +but executes against the code in the source tree. For example: + +``` +me@machine:~$ cd Nomiantim +me@machine:~Nomiantim$ ./nominatim-cli.py --version +Nominatim version 4.4.99-1 +``` + +Make sure you have activated the virtual environment that holds all +necessary dependencies. ## Executing Tests All tests are located in the `/test` directory. -To run all tests just go to the build directory and run make: +To run all tests, run make from the source root: ```sh -cd build -make test +make tests ``` +There are also goals for executing parts of the test suite: mypy, lint, pytest, bdd. + For more information about the structure of the tests and how to change and extend the test suite, see the [Testing chapter](Testing.md).