Merge remote-tracking branch 'upstream/master'

[nominatim.git] / docs / develop / Testing.md

diff --git a/docs/develop/Testing.md b/docs/develop/Testing.md
index 3c9d437f5b7be30b7bba5edb4b959a9eeb8a726b..be13d94983c795865334d63e13e72eb785f281f1 100644 (file)
--- a/docs/develop/Testing.md
+++ b/docs/develop/Testing.md
@@ -21,14 +21,15 @@ This test directory is sturctured as follows:
   | +-  api        Tests for API endpoints (search, reverse, etc.)
   |
   +-   php         PHP unit tests
   | +-  api        Tests for API endpoints (search, reverse, etc.)
   |
   +-   php         PHP unit tests

-  +-   scenes      Geometry test data
+  +-   python      Python unit tests

   +-   testdb      Base data for generating API test database
   +-   testdb      Base data for generating API test database

+  +-   testdata    Additional test data used by unit tests

 ```
 
 ## PHP Unit Tests (`test/php`)
 
 ```
 
 ## PHP Unit Tests (`test/php`)
 

-Unit tests can be found in the php/ directory. They test selected php functions.
-Very low coverage.
+Unit tests for PHP code can be found in the `php/` directory. They test selected
+PHP functions. Very low coverage.

 
 To execute the test suite run
 
 
 To execute the test suite run
 


@@ -36,11 +37,26 @@ To execute the test suite run
     UNIT_TEST_DSN='pgsql:dbname=nominatim_unit_tests' phpunit ../
 
 It will read phpunit.xml which points to the library, test path, bootstrap
     UNIT_TEST_DSN='pgsql:dbname=nominatim_unit_tests' phpunit ../
 
 It will read phpunit.xml which points to the library, test path, bootstrap

-strip and set other parameters.
+strip and sets other parameters.

 
 It will use (and destroy) a local database 'nominatim_unit_tests'. You can set
 a different connection string with e.g. UNIT_TEST_DSN='pgsql:dbname=foo_unit_tests'.
 
 
 It will use (and destroy) a local database 'nominatim_unit_tests'. You can set
 a different connection string with e.g. UNIT_TEST_DSN='pgsql:dbname=foo_unit_tests'.
 

+## Python Unit Tests (`test/python`)
+
+Unit tests for Python code can be found in the `python/` directory. The goal is
+to have complete coverage of the Python library in `nominatim`.
+
+To execute the tests run
+
+    py.test-3 test/python
+
+or
+
+    pytest test/python
+
+The name of the pytest binary depends on your installation.
+

 ## BDD Functional Tests (`test/bdd`)
 
 Functional tests are written as BDD instructions. For more information on
 ## BDD Functional Tests (`test/bdd`)
 
 Functional tests are written as BDD instructions. For more information on


@@ -67,17 +83,19 @@ The tests can be configured with a set of environment variables (`behave -D key=
                    the test databases (db tests)
  * `TEST_DB` - name of test database (db tests)
  * `API_TEST_DB` - name of the database containing the API test data (api tests)
                    the test databases (db tests)
  * `TEST_DB` - name of test database (db tests)
  * `API_TEST_DB` - name of the database containing the API test data (api tests)

+ * `API_TEST_FILE` - OSM file to be imported into the API test database (api tests)
+ * `API_ENGINE` - webframe to use for running search queries, same values as
+                  `nominatim serve --engine` parameter

  * `DB_HOST` - (optional) hostname of database host
  * `DB_PORT` - (optional) port of database on host
  * `DB_USER` - (optional) username of database login
  * `DB_PASS` - (optional) password for database login
  * `SERVER_MODULE_PATH` - (optional) path on the Postgres server to Nominatim
                           module shared library file
  * `DB_HOST` - (optional) hostname of database host
  * `DB_PORT` - (optional) port of database on host
  * `DB_USER` - (optional) username of database login
  * `DB_PASS` - (optional) password for database login
  * `SERVER_MODULE_PATH` - (optional) path on the Postgres server to Nominatim
                           module shared library file

- * `TEST_SETTINGS_TEMPLATE` - file to write temporary Nominatim settings to
- * `REMOVE_TEMPLATE` - if true, the template database will not be reused during
-                       the next run. Reusing the base templates speeds up tests
-                       considerably but might lead to outdated errors for some
-                       changes in the database layout.
+ * `REMOVE_TEMPLATE` - if true, the template and API database will not be reused
+                       during the next run. Reusing the base templates speeds
+                       up tests considerably but might lead to outdated errors
+                       for some changes in the database layout.

  * `KEEP_TEST_DB` - if true, the test database will not be dropped after a test
                     is finished. Should only be used if one single scenario is
                     run, otherwise the result is undefined.
  * `KEEP_TEST_DB` - if true, the test database will not be dropped after a test
                     is finished. Should only be used if one single scenario is
                     run, otherwise the result is undefined.


@@ -89,25 +107,22 @@ feature of behave which comes in handy when writing new tests.
 ### API Tests (`test/bdd/api`)
 
 These tests are meant to test the different API endpoints and their parameters.
 ### API Tests (`test/bdd/api`)
 
 These tests are meant to test the different API endpoints and their parameters.

-They require to import several datasets into a test database.
-See the [Development Setup chapter](Development-Environment.md#preparing-the-test-database)
-for instructions on how to set up this database.
+They require to import several datasets into a test database. This is normally
+done automatically during setup of the test. The API test database is then
+kept around and reused in subsequent runs of behave. Use `behave -DREMOVE_TEMPLATE`
+to force a reimport of the database.

 
 

-The official test dataset was derived from the 180924 planet (note: such
-file no longer exists at https://planet.openstreetmap.org/planet/2018/).
-Newer planets are likely to work as well but you may see isolated test
-failures where the data has changed.
+The official test dataset is saved in the file `test/testdb/apidb-test-data.pbf`
+and compromises the following data:

 
 

-The official test dataset can always be downloaded from
-[nominatim.org](https://www.nominatim.org/data/test/nominatim-api-testdata.pbf)
-To recreate the input data for the test database run:
+ * Geofabrik extract of Liechtenstein
+ * extract of Autauga country, Alabama, US (for tests against Tiger data)
+ * additional data from `test/testdb/additional_api_test.data.osm`

 
 

-```
-wget https://ftp5.gwdg.de/pub/misc/openstreetmap/planet.openstreetmap.org/pbf/planet-180924.osm.pbf
-osmconvert planet-180924.osm.pbf -B=test/testdb/testdb.polys -o=testdb.pbf
-```
+API tests should only be testing the functionality of the website PHP code.
+Most tests should be formulated as BDD DB creation tests (see below) instead.

 
 

-#### Code Coverage
+#### Code Coverage (PHP engine only)

 
 The API tests also support code coverage tests. You need to install
 [PHP_CodeCoverage](https://github.com/sebastianbergmann/php-code-coverage).
 
 The API tests also support code coverage tests. You need to install
 [PHP_CodeCoverage](https://github.com/sebastianbergmann/php-code-coverage).