X-Git-Url: https://git.openstreetmap.org./nominatim.git/blobdiff_plain/a338ba695b67a65aaa5102a0cfe2bd7da398a0b3..4825a0bda3e2b5d6a9c153b7cd0b8da2959cbc81:/docs/develop/Testing.md?ds=sidebyside diff --git a/docs/develop/Testing.md b/docs/develop/Testing.md index 3c9d437f..12673d40 100644 --- a/docs/develop/Testing.md +++ b/docs/develop/Testing.md @@ -8,9 +8,9 @@ the tests, see the [Development setup chapter](Development-Environment.md). There are two kind of tests in this test suite. There are functional tests which test the API interface using a BDD test framework and there are unit -tests for specific PHP functions. +tests for the Python code. -This test directory is sturctured as follows: +This test directory is structured as follows: ``` -+- bdd Functional API tests @@ -20,26 +20,25 @@ This test directory is sturctured as follows: | +- db Tests for internal data processing on import and update | +- api Tests for API endpoints (search, reverse, etc.) | - +- php PHP unit tests - +- scenes Geometry test data + +- python Python unit tests +- testdb Base data for generating API test database + +- testdata Additional test data used by unit tests ``` -## PHP Unit Tests (`test/php`) +## Python Unit Tests (`test/python`) -Unit tests can be found in the php/ directory. They test selected php functions. -Very low coverage. +Unit tests for Python code can be found in the `python/` directory. The goal is +to have complete coverage of the Python library in `nominatim`. -To execute the test suite run +To execute the tests run - cd test/php - UNIT_TEST_DSN='pgsql:dbname=nominatim_unit_tests' phpunit ../ + py.test-3 test/python -It will read phpunit.xml which points to the library, test path, bootstrap -strip and set other parameters. +or -It will use (and destroy) a local database 'nominatim_unit_tests'. You can set -a different connection string with e.g. UNIT_TEST_DSN='pgsql:dbname=foo_unit_tests'. + pytest test/python + +The name of the pytest binary depends on your installation. ## BDD Functional Tests (`test/bdd`) @@ -62,22 +61,21 @@ To run the functional tests, do The tests can be configured with a set of environment variables (`behave -D key=val`): - * `BUILDDIR` - build directory of Nominatim installation to test * `TEMPLATE_DB` - name of template database used as a skeleton for the test databases (db tests) * `TEST_DB` - name of test database (db tests) * `API_TEST_DB` - name of the database containing the API test data (api tests) + * `API_TEST_FILE` - OSM file to be imported into the API test database (api tests) + * `API_ENGINE` - webframe to use for running search queries, same values as + `nominatim serve --engine` parameter * `DB_HOST` - (optional) hostname of database host * `DB_PORT` - (optional) port of database on host * `DB_USER` - (optional) username of database login * `DB_PASS` - (optional) password for database login - * `SERVER_MODULE_PATH` - (optional) path on the Postgres server to Nominatim - module shared library file - * `TEST_SETTINGS_TEMPLATE` - file to write temporary Nominatim settings to - * `REMOVE_TEMPLATE` - if true, the template database will not be reused during - the next run. Reusing the base templates speeds up tests - considerably but might lead to outdated errors for some - changes in the database layout. + * `REMOVE_TEMPLATE` - if true, the template and API database will not be reused + during the next run. Reusing the base templates speeds + up tests considerably but might lead to outdated errors + for some changes in the database layout. * `KEEP_TEST_DB` - if true, the test database will not be dropped after a test is finished. Should only be used if one single scenario is run, otherwise the result is undefined. @@ -89,40 +87,20 @@ feature of behave which comes in handy when writing new tests. ### API Tests (`test/bdd/api`) These tests are meant to test the different API endpoints and their parameters. -They require to import several datasets into a test database. -See the [Development Setup chapter](Development-Environment.md#preparing-the-test-database) -for instructions on how to set up this database. - -The official test dataset was derived from the 180924 planet (note: such -file no longer exists at https://planet.openstreetmap.org/planet/2018/). -Newer planets are likely to work as well but you may see isolated test -failures where the data has changed. - -The official test dataset can always be downloaded from -[nominatim.org](https://www.nominatim.org/data/test/nominatim-api-testdata.pbf) -To recreate the input data for the test database run: - -``` -wget https://ftp5.gwdg.de/pub/misc/openstreetmap/planet.openstreetmap.org/pbf/planet-180924.osm.pbf -osmconvert planet-180924.osm.pbf -B=test/testdb/testdb.polys -o=testdb.pbf -``` - -#### Code Coverage - -The API tests also support code coverage tests. You need to install -[PHP_CodeCoverage](https://github.com/sebastianbergmann/php-code-coverage). -On Debian/Ubuntu run: - - apt-get install php-codecoverage php-xdebug - -Then run the API tests as follows: +They require to import several datasets into a test database. This is normally +done automatically during setup of the test. The API test database is then +kept around and reused in subsequent runs of behave. Use `behave -DREMOVE_TEMPLATE` +to force a reimport of the database. - behave api -DPHPCOV= +The official test dataset is saved in the file `test/testdb/apidb-test-data.pbf` +and compromises the following data: -The output directory must be an absolute path. To generate reports, you can use -the [phpcov](https://github.com/sebastianbergmann/phpcov) tool: + * Geofabrik extract of Liechtenstein + * extract of Autauga country, Alabama, US (for tests against Tiger data) + * additional data from `test/testdb/additional_api_test.data.osm` - phpcov merge --html= +API tests should only be testing the functionality of the website frontend code. +Most tests should be formulated as BDD DB creation tests (see below) instead. ### DB Creation Tests (`test/bdd/db`)