remove farms and isolated dwellings from computed addresses

[nominatim.git] / docs / develop / Development-Environment.md

diff --git a/docs/develop/Development-Environment.md b/docs/develop/Development-Environment.md
index 20d024114107f97448acc8b2445056fcc1f0356f..2425ec78c9315cbd5e83ca2391e9e84decbf2452 100644 (file)
--- a/docs/develop/Development-Environment.md
+++ b/docs/develop/Development-Environment.md
@@ -4,7 +4,7 @@ This chapter gives an overview how to set up Nominatim for development
 and how to run tests.
 
 !!! Important
 and how to run tests.
 
 !!! Important

-    This guide assumes that you develop under the latest version of Debain/Ubuntu.
+    This guide assumes you develop under the latest version of Debian/Ubuntu.

     You can of course also use your favourite distribution. You just might have
     to adapt the commands below slightly, in particular the commands for
     installing additional software.
     You can of course also use your favourite distribution. You just might have
     to adapt the commands below slightly, in particular the commands for
     installing additional software.


@@ -26,13 +26,10 @@ following packages should get you started:
 ## Prerequisites for testing and documentation
 
 The Nominatim test suite consists of behavioural tests (using behave) and
 ## Prerequisites for testing and documentation
 
 The Nominatim test suite consists of behavioural tests (using behave) and

-unit tests (using PHPUnit for PHP code and pytest for Python code).
-It has the following additional requirements:
+unit tests (using pytest). It has the following additional requirements:

 
 * [behave test framework](https://behave.readthedocs.io) >= 1.2.6
 
 * [behave test framework](https://behave.readthedocs.io) >= 1.2.6

-* [phpunit](https://phpunit.de) (9.5 is known to work)
-* [PHP CodeSniffer](https://github.com/squizlabs/PHP_CodeSniffer)
-* [Pylint](https://pylint.org/) (CI always runs the latest version from pip)
+* [flake8](https://flake8.pycqa.org/en/stable/) (CI always runs the latest version from pip)

 * [mypy](http://mypy-lang.org/) (plus typing information for external libs)
 * [Python Typing Extensions](https://github.com/python/typing_extensions) (for Python < 3.9)
 * [pytest](https://pytest.org)
 * [mypy](http://mypy-lang.org/) (plus typing information for external libs)
 * [Python Typing Extensions](https://github.com/python/typing_extensions) (for Python < 3.9)
 * [pytest](https://pytest.org)


@@ -41,33 +38,44 @@ It has the following additional requirements:
 For testing the Python search frontend, you need to install extra dependencies
 depending on your choice of webserver framework:
 
 For testing the Python search frontend, you need to install extra dependencies
 depending on your choice of webserver framework:
 

-* [httpx](https://www.python-httpx.org/) (starlette only)
-* [asgi-lifespan](https://github.com/florimondmanca/asgi-lifespan) (starlette only)
+* [httpx](https://www.python-httpx.org/) (Starlette only)
+* [asgi-lifespan](https://github.com/florimondmanca/asgi-lifespan) (Starlette only)

 
 The documentation is built with mkdocs:
 
 * [mkdocs](https://www.mkdocs.org/) >= 1.1.2
 * [mkdocstrings](https://mkdocstrings.github.io/) >= 0.25
 * [mkdocs-material](https://squidfunk.github.io/mkdocs-material/)
 
 The documentation is built with mkdocs:
 
 * [mkdocs](https://www.mkdocs.org/) >= 1.1.2
 * [mkdocstrings](https://mkdocstrings.github.io/) >= 0.25
 * [mkdocs-material](https://squidfunk.github.io/mkdocs-material/)

+* [mkdocs-gen-files](https://oprypin.github.io/mkdocs-gen-files/)
+
+Please be aware that tests always run against the globally installed
+osm2pgsql, so you need to have this set up. If you want to test against
+the vendored version of osm2pgsql, you need to set the PATH accordingly.

 
 ### Installing prerequisites on Ubuntu/Debian
 
 The Python tools should always be run with the most recent version.
 
 ### Installing prerequisites on Ubuntu/Debian
 
 The Python tools should always be run with the most recent version.

-In particular, pylint tends to have a lot of breaking changes between versions.

 The easiest way, to handle these Python dependencies is to run your
 development from within a virtual environment.
 
 The easiest way, to handle these Python dependencies is to run your
 development from within a virtual environment.
 

+```sh
+sudo apt install libsqlite3-mod-spatialite osm2pgsql \
+                 postgresql-postgis postgresql-postgis-scripts \
+                 pkg-config libicu-dev virtualenv
+```
+

 To set up the virtual environment with all necessary packages run:
 
 ```sh
 virtualenv ~/nominatim-dev-venv
 ~/nominatim-dev-venv/bin/pip install\
 To set up the virtual environment with all necessary packages run:
 
 ```sh
 virtualenv ~/nominatim-dev-venv
 ~/nominatim-dev-venv/bin/pip install\

-    psycopg2-binary psutil psycopg[binary] PyICU SQLAlchemy \
-    python-dotenv jinja2 pyYAML datree \
-    behave mkdocs mkdocstrings pytest pytest-asyncio pylint \
+    psutil psycopg[binary] PyICU SQLAlchemy \
+    python-dotenv jinja2 pyYAML datrie behave \
+    mkdocs mkdocstrings mkdocs-gen-files pytest pytest-asyncio flake8 \

     types-jinja2 types-markupsafe types-psutil types-psycopg2 \
     types-pygments types-pyyaml types-requests types-ujson \
     types-jinja2 types-markupsafe types-psutil types-psycopg2 \
     types-pygments types-pyyaml types-requests types-ujson \

-    types-urllib3 typing-extensions unicorn falcon
+    types-urllib3 typing-extensions unicorn falcon starlette \
+    uvicorn mypy osmium aiosqlite

 ```
 
 Now enter the virtual environment whenever you want to develop:
 ```
 
 Now enter the virtual environment whenever you want to develop:


@@ -76,28 +84,6 @@ Now enter the virtual environment whenever you want to develop:
 . ~/nominatim-dev-venv/bin/activate
 ```
 
 . ~/nominatim-dev-venv/bin/activate
 ```
 

-For installing the PHP development tools, run:
-
-```sh
-sudo apt install php-cgi phpunit php-codesniffer
-```
-
-If your distribution does not have PHPUnit 7.3+, you can install it (as well
-as CodeSniffer) via composer:
-
-```
-sudo apt-get install composer
-composer global require "squizlabs/php_codesniffer=*"
-composer global require "phpunit/phpunit=8.*"
-```
-
-The binaries are found in `.config/composer/vendor/bin`. You need to add this
-to your PATH:
-
-```
-echo 'export PATH=~/.config/composer/vendor/bin:$PATH' > ~/.profile
-```
-

 ### Running Nominatim during development
 
 The source code for Nominatim can be found in the `src` directory and can
 ### Running Nominatim during development
 
 The source code for Nominatim can be found in the `src` directory and can


@@ -106,12 +92,12 @@ be run in-place. The source directory features a special script
 but executes against the code in the source tree. For example:
 
 ```
 but executes against the code in the source tree. For example:
 
 ```

-me@machine:~$ cd Nomiantim
-me@machine:~Nomiantim$ ./nominatim-cli.py --version
+me@machine:~$ cd Nominatim
+me@machine:~Nominatim$ ./nominatim-cli.py --version

 Nominatim version 4.4.99-1
 ```
 
 Nominatim version 4.4.99-1
 ```
 

-Make sure you have activated the virtual environment that holds all
+Make sure you have activated the virtual environment holding all

 necessary dependencies.
 
 ## Executing Tests
 necessary dependencies.
 
 ## Executing Tests


@@ -124,7 +110,14 @@ To run all tests, run make from the source root:
 make tests
 ```
 
 make tests
 ```
 

-There are also goals for executing parts of the test suite: mypy, lint, pytest, bdd.
+There are also make targets for executing only parts of the test suite.
+For example to run linting only use:
+
+```sh
+make lint
+```
+
+The possible testing targets are: mypy, lint, pytest, bdd.

 
 For more information about the structure of the tests and how to change and
 extend the test suite, see the [Testing chapter](Testing.md).
 
 For more information about the structure of the tests and how to change and
 extend the test suite, see the [Testing chapter](Testing.md).


@@ -136,18 +129,14 @@ built using the [MkDocs](https://www.mkdocs.org/) static site generation
 framework. The master branch is automatically deployed every night on
 [https://nominatim.org/release-docs/develop/](https://nominatim.org/release-docs/develop/)
 
 framework. The master branch is automatically deployed every night on
 [https://nominatim.org/release-docs/develop/](https://nominatim.org/release-docs/develop/)
 

-To build the documentation, go to the build directory and run
+To build the documentation run

 
 ```
 make doc
 
 ```
 make doc

-INFO - Cleaning site directory
-INFO - Building documentation to directory: /home/vagrant/build/site-html

 ```
 
 ```
 

-This runs `mkdocs build` plus extra transformation of some files and adds
-symlinks (see `CMakeLists.txt` for the exact steps).

 
 

-Now you can start webserver for local testing
+For local testing, you can start webserver:

 
 ```
 build> make serve-doc
 
 ```
 build> make serve-doc


@@ -159,7 +148,7 @@ If you develop inside a Vagrant virtual machine, use a port that is forwarded
 to your host:
 
 ```
 to your host:
 
 ```

-build> PYTHONPATH=$SRCDIR mkdocs serve --dev-addr 0.0.0.0:8088
+build> mkdocs serve --dev-addr 0.0.0.0:8088

 [server:296] Serving on http://0.0.0.0:8088
 [handlers:62] Start watching changes
 ```
 [server:296] Serving on http://0.0.0.0:8088
 [handlers:62] Start watching changes
 ```