From: Sarah Hoffmann Date: Sun, 1 Jan 2017 16:36:07 +0000 (+0100) Subject: Merge branch 'moreurl-with-countrycodes' of https://github.com/mtmail/Nominatim into... X-Git-Tag: v3.0.0~84 X-Git-Url: https://git.openstreetmap.org./nominatim.git/commitdiff_plain/65500927c2955860d392f76fa9b2c318ccd716d2?hp=adb6ea546ff420a9b68deed77edd5e900c93833a Merge branch 'moreurl-with-countrycodes' of https://github.com/mtmail/Nominatim into mtmail-moreurl-with-countrycodes --- diff --git a/phpunit.xml b/phpunit.xml index bea876d5..addce5ce 100644 --- a/phpunit.xml +++ b/phpunit.xml @@ -8,13 +8,13 @@ processIsolation="false" stopOnFailure="false" syntaxCheck="true" - bootstrap="tests-php/bootstrap.php" + bootstrap="test/php/bootstrap.php" > - ./tests-php/Nominatim + ./test/php/Nominatim diff --git a/settings/defaults.php b/settings/defaults.php index a953e460..e34f4db0 100644 --- a/settings/defaults.php +++ b/settings/defaults.php @@ -42,6 +42,7 @@ if (isset($_GET['debug']) && $_GET['debug']) @define('CONST_Debug', true); @define('CONST_Osm2pgsql_Binary', CONST_InstallPath.'/osm2pgsql/osm2pgsql'); @define('CONST_Osmosis_Binary', '/usr/bin/osmosis'); @define('CONST_Tiger_Data_Path', CONST_BasePath.'/data/tiger'); +@define('CONST_Wikipedia_Data_Path', CONST_BasePath.'/data'); // osm2pgsql settings @define('CONST_Osm2pgsql_Flatnode_File', null); diff --git a/test/README.md b/test/README.md new file mode 100644 index 00000000..6e8f7f97 --- /dev/null +++ b/test/README.md @@ -0,0 +1,134 @@ +This directory contains functional and unit tests for the Nominatim API. + +Prerequisites +============= + + * Python 3 (https://www.python.org/) + * behave test framework >= 1.2.5 (https://github.com/behave/behave) + * nose (https://nose.readthedocs.org) + * pytidylib (http://countergram.com/open-source/pytidylib) + * psycopg2 (http://initd.org/psycopg/) + +To get the prerequisites on a a fresh Ubuntu LTS 16.04 run: + + [sudo] apt-get install python3-dev python3-pip python3-psycopg2 python3-tidylib phpunit + pip3 install --user behave nose + + +Overall structure +================= + +There are two kind of tests in this test suite. There are functional tests +which test the API interface using a BDD test framework and there are unit +tests for specific PHP functions. + +This test directory is sturctured as follows: + + -+- bdd Functional API tests + | \ + | +- steps Step implementations for test descriptions + | +- osm2pgsql Tests for data import via osm2pgsql + | +- db Tests for internal data processing on import and update + | +- api Tests for API endpoints (search, reverse, etc.) + | + +- php PHP unit tests + +- scenes Geometry test data + +- testdb Base data for generating API test database + + +PHP Unit Tests +============== + +Unit tests can be found in the php/ directory and tests selected php functions. +Very low coverage. + +To execute the test suite run + + cd test/php + phpunit ../ + +It will read phpunit.xml which points to the library, test path, bootstrap +strip and set other parameters. + + +BDD Functional Tests +==================== + +Functional tests are written as BDD instructions. For more information on +the philosophy of BDD testing, see http://pythonhosted.org/behave/philosophy.html + +Usage +----- + +To run the functional tests, do + + cd test/bdd + behave + +The tests can be configured with a set of environment variables: + + * `BUILD_DIR` - build directory of Nominatim installation to test + * `TEMPLATE_DB` - name of template database used as a skeleton for + the test databases (db tests) + * `TEST_DB` - name of test database (db tests) + * `ABI_TEST_DB` - name of the database containing the API test data (api tests) + * `TEST_SETTINGS_TEMPLATE` - file to write temporary Nominatim settings to + * `REMOVE_TEMPLATE` - if true, the template database will not be reused during + the next run. Reusing the base templates speeds up tests + considerably but might lead to outdated errors for some + changes in the database layout. + * `KEEP_TEST_DB` - if true, the test database will not be dropped after a test + is finished. Should only be used if one single scenario is + run, otherwise the result is undefined. + +Logging can be defined through command line parameters of behave itself. Check +out `behave --help` for details. Also keep an eye out for the 'work-in-progress' +feature of behave which comes in handy when writing new tests. + +Writing Tests +------------- + +The following explanation assume that the reader is familiar with the BDD +notations of features, scenarios and steps. + +All possible steps can be found in the `steps` directory and should ideally +be documented. + +### API Tests (`test/bdd/api`) + +These tests are meant to test the different API endpoints and their parameters. +They require a preimported test database, which consists of the import of a +planet extract. The polygons defining the extract can be found in the test/testdb +directory. There is also a reduced set of wikipedia data for this extract, +which you need to import as well. For Tiger tests the data of South Dakota +is required. Get the Tiger files `46*`. + +The official test dataset is derived from the 160725 planet. Newer +planets are likely to work as well but you may see isolated test +failures where the data has changed. To recreate the input data +for the test database run: + + wget http://free.nchc.org.tw/osm.planet/pbf/planet-160725.osm.pbf + osmconvert planet-160725.osm.pbf -B=test/testdb/testdb.polys -o=testdb.pbf + +Before importing make sure to add the following to your local settings: + + @define('CONST_Database_DSN', 'pgsql://@/test_api_nominatim'); + @define('CONST_Wikipedia_Data_Path', CONST_BasePath.'/test/testdb'); + +### Indexing Tests (`test/bdd/db`) + +These tests check the import and update of the Nominatim database. They do not +test the correctness of osm2pgsql. Each test will write some data into the `place` +table (and optionally `the planet_osm_*` tables if required) and then run +Nominatim's processing functions on that. + +These tests need to create their own test databases. By default they will be +called `test_template_nominatim` and `test_nominatim`. Names can be changed with +the environment variables `TEMPLATE_DB` and `TEST_DB`. The user running the tests +needs superuser rights for postgres. + +### Import Tests (`test/bdd/osm2pgsql`) + +These tests check that data is imported correctly into the place table. They +use the same template database as the Indexing tests, so the same remarks apply. diff --git a/test/bdd/api/details/simple.feature b/test/bdd/api/details/simple.feature new file mode 100644 index 00000000..638e89ca --- /dev/null +++ b/test/bdd/api/details/simple.feature @@ -0,0 +1,14 @@ +@APIDB +Feature: Object details + Check details page for correctness + + Scenario Outline: Details via OSM id + When sending details query for + Then the result is valid html + + Examples: + | object | + | 492887 | + | N4267356889 | + | W230804120 | + | R123924 | diff --git a/test/bdd/api/lookup/simple.feature b/test/bdd/api/lookup/simple.feature new file mode 100644 index 00000000..5ec185c5 --- /dev/null +++ b/test/bdd/api/lookup/simple.feature @@ -0,0 +1,17 @@ +@APIDB +Feature: Places by osm_type and osm_id Tests + Simple tests for internal server errors and response format. + + Scenario Outline: address lookup for existing node, way, relation + When sending lookup query for N3284625766,W6065798,,R123924,X99,N0 + Then the result is valid + And exactly 3 results are returned + + Examples: + | format | + | xml | + | json | + + Scenario: address lookup for non-existing or invalid node, way, relation + When sending xml lookup query for X99,,N0,nN158845944,ABC,,W9 + Then exactly 0 results are returned diff --git a/test/bdd/api/reverse/language.feature b/test/bdd/api/reverse/language.feature new file mode 100644 index 00000000..9bde2d4e --- /dev/null +++ b/test/bdd/api/reverse/language.feature @@ -0,0 +1,36 @@ +@APIDB +Feature: Localization of reverse search results + + Scenario: default language + When sending json reverse coordinates 18.1147,-15.95 + Then result addresses contain + | ID | country | + | 0 | Mauritanie موريتانيا | + + Scenario: accept-language parameter + When sending json reverse coordinates 18.1147,-15.95 + | accept-language | + | en,fr | + Then result addresses contain + | ID | country | + | 0 | Mauritania | + + Scenario: HTTP accept language header + Given the HTTP header + | accept-language | + | fr-ca,fr;q=0.8,en-ca;q=0.5,en;q=0.3 | + When sending json reverse coordinates 18.1147,-15.95 + Then result addresses contain + | ID | country | + | 0 | Mauritanie | + + Scenario: accept-language parameter and HTTP header + Given the HTTP header + | accept-language | + | fr-ca,fr;q=0.8,en-ca;q=0.5,en;q=0.3 | + When sending json reverse coordinates 18.1147,-15.95 + | accept-language | + | en | + Then result addresses contain + | ID | country | + | 0 | Mauritania | diff --git a/test/bdd/api/reverse/params.feature b/test/bdd/api/reverse/params.feature new file mode 100644 index 00000000..0d35cdc7 --- /dev/null +++ b/test/bdd/api/reverse/params.feature @@ -0,0 +1,102 @@ +@APIDB +Feature: Parameters for Reverse API + Testing diferent parameter options for reverse API. + + Scenario Outline: Reverse-geocoding without address + When sending reverse coordinates 53.603,10.041 + | addressdetails | + | 0 | + Then exactly 1 result is returned + And result has not attributes address + + Examples: + | format | + | json | + | jsonv2 | + | xml | + + Scenario Outline: Reverse Geocoding with extratags + When sending reverse coordinates 10.776234290950017,106.70425325632095 + | extratags | + | 1 | + Then result 0 has attributes extratags + + Examples: + | format | + | xml | + | json | + | jsonv2 | + + Scenario Outline: Reverse Geocoding with namedetails + When sending reverse coordinates 10.776455623137625,106.70175343751907 + | namedetails | + | 1 | + Then result 0 has attributes namedetails + + Examples: + | format | + | xml | + | json | + | jsonv2 | + + Scenario Outline: Reverse Geocoding contains TEXT geometry + When sending reverse coordinates 47.165989816710066,9.515774846076965 + | polygon_text | + | 1 | + Then result 0 has attributes + + Examples: + | format | response_attribute | + | xml | geotext | + | json | geotext | + | jsonv2 | geotext | + + Scenario Outline: Reverse Geocoding contains polygon-as-points geometry + When sending reverse coordinates 47.165989816710066,9.515774846076965 + | polygon | + | 1 | + Then result 0 has not attributes + + Examples: + | format | response_attribute | + | xml | polygonpoints | + | json | polygonpoints | + | jsonv2 | polygonpoints | + + Scenario Outline: Reverse Geocoding contains SVG geometry + When sending reverse coordinates 47.165989816710066,9.515774846076965 + | polygon_svg | + | 1 | + Then result 0 has attributes + + Examples: + | format | response_attribute | + | xml | geosvg | + | json | svg | + | jsonv2 | svg | + + Scenario Outline: Reverse Geocoding contains KML geometry + When sending reverse coordinates 47.165989816710066,9.515774846076965 + | polygon_kml | + | 1 | + Then result 0 has attributes + + Examples: + | format | response_attribute | + | xml | geokml | + | json | geokml | + | jsonv2 | geokml | + + Scenario Outline: Reverse Geocoding contains GEOJSON geometry + When sending reverse coordinates 47.165989816710066,9.515774846076965 + | polygon_geojson | + | 1 | + Then result 0 has attributes + + Examples: + | format | response_attribute | + | xml | geojson | + | json | geojson | + | jsonv2 | geojson | + + diff --git a/test/bdd/api/reverse/queries.feature b/test/bdd/api/reverse/queries.feature new file mode 100644 index 00000000..e1d089b9 --- /dev/null +++ b/test/bdd/api/reverse/queries.feature @@ -0,0 +1,25 @@ +@APIDB +Feature: Reverse geocoding + Testing the reverse function + + @Tiger + Scenario: TIGER house number + When sending jsonv2 reverse coordinates 45.3345,-97.5214 + Then results contain + | osm_type | category | type | + | way | place | house | + And result addresses contain + | house_number | road | postcode | country_code | + | 906 | West 1st Street | 57274 | us | + + @Tiger + Scenario: No TIGER house number for zoom < 18 + When sending jsonv2 reverse coordinates 45.3345,-97.5214 + | zoom | + | 17 | + Then results contain + | osm_type | category | + | way | highway | + And result addresses contain + | road | postcode | country_code | + | West 1st Street | 57274 | us | diff --git a/test/bdd/api/reverse/simple.feature b/test/bdd/api/reverse/simple.feature new file mode 100644 index 00000000..2b484736 --- /dev/null +++ b/test/bdd/api/reverse/simple.feature @@ -0,0 +1,130 @@ +@APIDB +Feature: Simple Reverse Tests + Simple tests for internal server errors and response format. + + Scenario Outline: Simple reverse-geocoding + When sending reverse coordinates , + Then the result is valid xml + When sending xml reverse coordinates , + Then the result is valid xml + When sending json reverse coordinates , + Then the result is valid json + When sending jsonv2 reverse coordinates , + Then the result is valid json + When sending html reverse coordinates , + Then the result is valid html + + Examples: + | lat | lon | + | 0.0 | 0.0 | + | -34.830 | -56.105 | + | 45.174 | -103.072 | + | 21.156 | -12.2744 | + + Scenario Outline: Testing different parameters + When sending reverse coordinates 53.603,10.041 + | param | value | + | | | + Then the result is valid xml + When sending html reverse coordinates 53.603,10.041 + | param | value | + | | | + Then the result is valid html + When sending xml reverse coordinates 53.603,10.041 + | param | value | + | | | + Then the result is valid xml + When sending json reverse coordinates 53.603,10.041 + | param | value | + | | | + Then the result is valid json + When sending jsonv2 reverse coordinates 53.603,10.041 + | param | value | + | | | + Then the result is valid json + + Examples: + | parameter | value | + | polygon | 1 | + | polygon | 0 | + | polygon_text | 1 | + | polygon_text | 0 | + | polygon_kml | 1 | + | polygon_kml | 0 | + | polygon_geojson | 1 | + | polygon_geojson | 0 | + | polygon_svg | 1 | + | polygon_svg | 0 | + + Scenario Outline: Wrapping of legal jsonp requests + When sending reverse coordinates 67.3245,0.456 + | json_callback | + | foo | + Then the result is valid json + + Examples: + | format | + | json | + | jsonv2 | + + @wip + Scenario Outline: Boundingbox is returned + When sending reverse coordinates 14.62,108.1 + | zoom | + | 4 | + Then result has bounding box in 9,20,102,113 + + Examples: + | format | + | json | + | jsonv2 | + | xml | + + Scenario Outline: Reverse-geocoding with zoom + When sending reverse coordinates 53.603,10.041 + | zoom | + | 10 | + Then exactly 1 result is returned + + Examples: + | format | + | json | + | jsonv2 | + | html | + | xml | + + Scenario: Missing lon parameter + When sending reverse coordinates 52.52, + Then a HTTP 400 is returned + + Scenario: Missing lat parameter + When sending reverse coordinates ,52.52 + Then a HTTP 400 is returned + + Scenario: Missing osm_id parameter + When sending reverse coordinates , + | osm_type | + | N | + Then a HTTP 400 is returned + + Scenario: Missing osm_type parameter + When sending reverse coordinates , + | osm_id | + | 3498564 | + Then a HTTP 400 is returned + + Scenario Outline: Bad format for lat or lon + When sending reverse coordinates , + | lat | lon | + | | | + Then a HTTP 400 is returned + + Examples: + | lat | lon | + | 48.9660 | 8,4482 | + | 48,9660 | 8.4482 | + | 48,9660 | 8,4482 | + | 48.966.0 | 8.4482 | + | 48.966 | 8.448.2 | + | Nan | 8.448 | + | 48.966 | Nan | diff --git a/test/bdd/api/search/language.feature b/test/bdd/api/search/language.feature new file mode 100644 index 00000000..d077e4da --- /dev/null +++ b/test/bdd/api/search/language.feature @@ -0,0 +1,62 @@ +@APIDB +Feature: Localization of search results + + Scenario: default language + When sending json search query "Vietnam" + Then results contain + | ID | display_name | + | 0 | Việt Nam | + + Scenario: accept-language first + When sending json search query "Mauretanien" + | accept-language | + | en,de | + Then results contain + | ID | display_name | + | 0 | Mauritania | + + Scenario: accept-language missing + When sending json search query "Mauretanien" + | accept-language | + | xx,fr,en,de | + Then results contain + | ID | display_name | + | 0 | Mauritanie | + + Scenario: http accept language header first + Given the HTTP header + | accept-language | + | fr-ca,fr;q=0.8,en-ca;q=0.5,en;q=0.3 | + When sending json search query "Mauretanien" + Then results contain + | ID | display_name | + | 0 | Mauritanie | + + Scenario: http accept language header and accept-language + Given the HTTP header + | accept-language | + | fr-ca,fr;q=0.8,en-ca;q=0.5,en;q=0.3 | + When sending json search query "Mauretanien" + | accept-language | + | de,en | + Then results contain + | ID | display_name | + | 0 | Mauretanien | + + Scenario: http accept language header fallback + Given the HTTP header + | accept-language | + | fr-ca,en-ca;q=0.5 | + When sending json search query "Mauretanien" + Then results contain + | ID | display_name | + | 0 | Mauritanie | + + Scenario: http accept language header fallback (upper case) + Given the HTTP header + | accept-language | + | fr-FR;q=0.8,en-ca;q=0.5 | + When sending json search query "Mauretanie" + Then results contain + | ID | display_name | + | 0 | Mauritanie | diff --git a/test/bdd/api/search/params.feature b/test/bdd/api/search/params.feature new file mode 100644 index 00000000..1fa16383 --- /dev/null +++ b/test/bdd/api/search/params.feature @@ -0,0 +1,300 @@ +@APIDB +Feature: Search queries + Testing different queries and parameters + + Scenario: Simple XML search + When sending xml search query "Schaan" + Then result 0 has attributes place_id,osm_type,osm_id + And result 0 has attributes place_rank,boundingbox + And result 0 has attributes lat,lon,display_name + And result 0 has attributes class,type,importance,icon + And result 0 has not attributes address + And result 0 has bounding box in 46.5,47.5,9,10 + + Scenario: Simple JSON search + When sending json search query "Vaduz" + Then result 0 has attributes place_id,licence,icon,class,type + And result 0 has attributes osm_type,osm_id,boundingbox + And result 0 has attributes lat,lon,display_name,importance + And result 0 has not attributes address + And result 0 has bounding box in 46.5,47.5,9,10 + + Scenario: JSON search with addressdetails + When sending json search query "Montevideo" with address + Then address of result 0 is + | type | value | + | city | Montevideo | + | state | Montevideo | + | country | Uruguay | + | country_code | uy | + + Scenario: XML search with addressdetails + When sending xml search query "Aleg" with address + | accept-language | + | en | + Then address of result 0 is + | type | value | + | city | Aleg | + | state | Brakna | + | country | Mauritania | + | country_code | mr | + + Scenario: coordinate search with addressdetails + When sending json search query "14.271104294939,107.69828796387" + | accept-language | + | en | + Then results contain + | display_name | + | Plei Ya Rê, Kon Tum province, Vietnam | + + Scenario: Address details with unknown class types + When sending json search query "Hundeauslauf, Hamburg" with address + Then results contain + | ID | class | type | + | 0 | leisure | dog_park | + And result addresses contain + | ID | address29 | + | 0 | Hundeauslauf | + And address of result 0 has no types leisure,dog_park + + Scenario: Disabling deduplication + When sending json search query "Sievekingsallee, Hamburg" + Then there are no duplicates + When sending json search query "Sievekingsallee, Hamburg" + | dedupe | + | 0 | + Then there are duplicates + + Scenario: Search with bounded viewbox in right area + When sending json search query "restaurant" with address + | bounded | viewbox | + | 1 | 9.93027,53.61634,10.10073,53.54500 | + Then result addresses contain + | state | + | Hamburg | + + Scenario: Search with bounded viewboxlbrt in right area + When sending json search query "restaurant" with address + | bounded | viewboxlbrt | + | 1 | 9.93027,53.54500,10.10073,53.61634 | + Then result addresses contain + | state | + | Hamburg | + + Scenario: No POI search with unbounded viewbox + When sending json search query "restaurant" + | viewbox | + | 9.93027,53.61634,10.10073,53.54500 | + Then results contain + | display_name | + | ^[^,]*[Rr]estaurant.* | + + Scenario: bounded search remains within viewbox, even with no results + When sending json search query "restaurant" + | bounded | viewbox | + | 1 | 43.5403125,-5.6563282,43.54285,-5.662003 | + Then less than 1 result is returned + + Scenario: bounded search remains within viewbox with results + When sending json search query "restaurant" + | bounded | viewbox | + | 1 | 9.93027,53.61634,10.10073,53.54500 | + Then result has bounding box in 53.54500,53.61634,9.93027,10.10073 + + Scenario: Prefer results within viewbox + When sending json search query "25 de Mayo" with address + | accept-language | + | en | + Then result addresses contain + | ID | state | + | 0 | Salto | + When sending json search query "25 de Mayo" with address + | accept-language | viewbox | + | en | -56.35879,-34.18330,-56.31618,-34.20815 | + Then result addresses contain + | ID | state | + | 0 | Florida | + + Scenario: Overly large limit number for search results + When sending json search query "restaurant" + | limit | + | 1000 | + Then at most 50 results are returned + + Scenario: Limit number of search results + When sending json search query "restaurant" + | limit | + | 4 | + Then exactly 4 results are returned + + Scenario: Restrict to feature type country + When sending xml search query "Uruguay" + Then results contain + | ID | place_rank | + | 1 | 16 | + When sending xml search query "Uruguay" + | featureType | + | country | + Then results contain + | place_rank | + | 4 | + + Scenario: Restrict to feature type state + When sending xml search query "Dakota" + Then results contain + | place_rank | + | 12 | + When sending xml search query "Dakota" + | featureType | + | state | + Then results contain + | place_rank | + | 8 | + + Scenario: Restrict to feature type city + When sending xml search query "vaduz" + Then results contain + | ID | place_rank | + | 1 | 30 | + When sending xml search query "vaduz" + | featureType | + | city | + Then results contain + | place_rank | + | 16 | + + Scenario: Restrict to feature type settlement + When sending json search query "Burg" + Then results contain + | ID | class | + | 1 | amenity | + When sending json search query "Burg" + | featureType | + | settlement | + Then results contain + | class | type | + | boundary | administrative | + + Scenario Outline: Search with polygon threshold (json) + When sending json search query "switzerland" + | polygon_geojson | polygon_threshold | + | 1 | | + Then at least 1 result is returned + And result 0 has attributes geojson + + Examples: + | th | + | -1 | + | 0.0 | + | 0.5 | + | 999 | + + Scenario Outline: Search with polygon threshold (xml) + When sending xml search query "switzerland" + | polygon_geojson | polygon_threshold | + | 1 | | + Then at least 1 result is returned + And result 0 has attributes geojson + + Examples: + | th | + | -1 | + | 0.0 | + | 0.5 | + | 999 | + + Scenario Outline: Search with invalid polygon threshold (xml) + When sending xml search query "switzerland" + | polygon_geojson | polygon_threshold | + | 1 | | + Then a HTTP 400 is returned + + Examples: + | th | + | x | + | ;; | + | 1m | + + Scenario Outline: Search with extratags + When sending search query "Hauptstr" + | extratags | + | 1 | + Then result has attributes extratags + + Examples: + | format | + | xml | + | json | + | jsonv2 | + + Scenario Outline: Search with namedetails + When sending search query "Hauptstr" + | namedetails | + | 1 | + Then result has attributes namedetails + + Examples: + | format | + | xml | + | json | + | jsonv2 | + + Scenario Outline: Search result with contains TEXT geometry + When sending search query "Highmore" + | polygon_text | + | 1 | + Then result has attributes + + Examples: + | format | response_attribute | + | xml | geotext | + | json | geotext | + | jsonv2 | geotext | + + Scenario Outline: Search result contains polygon-as-points geometry + When sending search query "Highmore" + | polygon | + | 1 | + Then result has attributes + + Examples: + | format | response_attribute | + | xml | polygonpoints | + | json | polygonpoints | + | jsonv2 | polygonpoints | + + Scenario Outline: Search result contains SVG geometry + When sending search query "Highmore" + | polygon_svg | + | 1 | + Then result has attributes + + Examples: + | format | response_attribute | + | xml | geosvg | + | json | svg | + | jsonv2 | svg | + + Scenario Outline: Search result contains KML geometry + When sending search query "Highmore" + | polygon_kml | + | 1 | + Then result has attributes + + Examples: + | format | response_attribute | + | xml | geokml | + | json | geokml | + | jsonv2 | geokml | + + Scenario Outline: Search result contains GEOJSON geometry + When sending search query "Highmore" + | polygon_geojson | + | 1 | + Then result has attributes + + Examples: + | format | response_attribute | + | xml | geojson | + | json | geojson | + | jsonv2 | geojson | diff --git a/test/bdd/api/search/queries.feature b/test/bdd/api/search/queries.feature new file mode 100644 index 00000000..0074e334 --- /dev/null +++ b/test/bdd/api/search/queries.feature @@ -0,0 +1,67 @@ +@APIDB +Feature: Search queries + Generic search result correctness + + Scenario: House number search for non-street address + When sending json search query "2 Steinwald, Austria" with address + | accept-language | + | en | + Then address of result 0 is + | type | value | + | house_number | 2 | + | hamlet | Steinwald | + | postcode | 6811 | + | country | Austria | + | country_code | at | + + Scenario: House number interpolation even + When sending json search query "Schellingstr 86, Hamburg" with address + | accept-language | + | de | + Then address of result 0 is + | type | value | + | house_number | 86 | + | road | Schellingstraße | + | suburb | Eilbek | + | postcode | 22089 | + | city_district | Wandsbek | + | state | Hamburg | + | country | Deutschland | + | country_code | de | + + Scenario: House number interpolation odd + When sending json search query "Schellingstr 73, Hamburg" with address + | accept-language | + | de | + Then address of result 0 is + | type | value | + | house_number | 73 | + | road | Schellingstraße | + | suburb | Eilbek | + | postcode | 22089 | + | city_district | Wandsbek | + | state | Hamburg | + | country | Deutschland | + | country_code | de | + + @Tiger + Scenario: TIGER house number + When sending json search query "323 22nd Street Southwest, Huron" + Then results contain + | osm_type | + | way | + + Scenario: Search with class-type feature + When sending jsonv2 search query "Hotel California" + Then results contain + | place_rank | + | 30 | + + # https://trac.openstreetmap.org/ticket/5094 + Scenario: housenumbers are ordered by complete match first + When sending json search query "6395 geminis, montevideo" with address + Then result addresses contain + | ID | house_number | + | 0 | 6395 | + | 1 | 6395 BIS | + diff --git a/test/bdd/api/search/simple.feature b/test/bdd/api/search/simple.feature new file mode 100644 index 00000000..4d77eac4 --- /dev/null +++ b/test/bdd/api/search/simple.feature @@ -0,0 +1,221 @@ +@APIDB +Feature: Simple Tests + Simple tests for internal server errors and response format. + + Scenario Outline: Testing different parameters + When sending search query "Hamburg" + | param | value | + | | | + Then at least 1 result is returned + When sending html search query "Hamburg" + | param | value | + | | | + Then at least 1 result is returned + When sending xml search query "Hamburg" + | param | value | + | | | + Then at least 1 result is returned + When sending json search query "Hamburg" + | param | value | + | | | + Then at least 1 result is returned + When sending jsonv2 search query "Hamburg" + | param | value | + | | | + Then at least 1 result is returned + + Examples: + | parameter | value | + | addressdetails | 1 | + | addressdetails | 0 | + | polygon | 1 | + | polygon | 0 | + | polygon_text | 1 | + | polygon_text | 0 | + | polygon_kml | 1 | + | polygon_kml | 0 | + | polygon_geojson | 1 | + | polygon_geojson | 0 | + | polygon_svg | 1 | + | polygon_svg | 0 | + | accept-language | de,en | + | countrycodes | de | + | bounded | 1 | + | bounded | 0 | + | exclude_place_ids| 385252,1234515 | + | limit | 1000 | + | dedupe | 1 | + | dedupe | 0 | + | extratags | 1 | + | extratags | 0 | + | namedetails | 1 | + | namedetails | 0 | + + Scenario: Search with invalid output format + When sending search query "Berlin" + | format | + | fd$# | + Then a HTTP 400 is returned + + Scenario Outline: Simple Searches + When sending search query "" + Then the result is valid html + When sending html search query "" + Then the result is valid html + When sending xml search query "" + Then the result is valid xml + When sending json search query "" + Then the result is valid json + When sending jsonv2 search query "" + Then the result is valid json + + Examples: + | query | + | New York, New York | + | France | + | 12, Main Street, Houston | + | München | + | 東京都 | + | hotels in nantes | + | xywxkrf | + | gh; foo() | + | %#$@*&l;der#$! | + | 234 | + | 47.4,8.3 | + + Scenario: Empty XML search + When sending xml search query "xnznxvcx" + Then result header contains + | attr | value | + | querystring | xnznxvcx | + | polygon | false | + | more_url | .*format=xml.*q=xnznxvcx.* | + + Scenario: Empty XML search with special XML characters + When sending xml search query "xfdghn&zxn"xvbyxcssdex" + Then result header contains + | attr | value | + | querystring | xfdghn&zxn"xvbyxcssdex | + | polygon | false | + | more_url | .*format=xml.*q=xfdghn%26zxn%22xvbyx%3Cvxx%3Ecssdex.* | + + Scenario: Empty XML search with viewbox + When sending xml search query "xnznxvcx" + | viewbox | + | 12,45.13,77,33 | + Then result header contains + | attr | value | + | querystring | xnznxvcx | + | polygon | false | + | viewbox | 12,45.13,77,33 | + + Scenario: Empty XML search with viewboxlbrt + When sending xml search query "xnznxvcx" + | viewboxlbrt | + | 12,34.13,77,45 | + Then result header contains + | attr | value | + | querystring | xnznxvcx | + | polygon | false | + | viewbox | 12,45,77,34.13 | + + Scenario: Empty XML search with viewboxlbrt and viewbox + When sending xml search query "pub" + | viewbox | viewboxblrt | + | 12,45.13,77,33 | 1,2,3,4 | + Then result header contains + | attr | value | + | querystring | pub | + | polygon | false | + | viewbox | 12,45.13,77,33 | + + Scenario Outline: Empty XML search with polygon values + When sending xml search query "xnznxvcx" + | param | value | + | polygon | | + Then result header contains + | attr | value | + | polygon | | + + Examples: + | result | polyval | + | false | 0 | + | true | 1 | + | true | True | + | true | true | + | true | false | + | true | FALSE | + | true | yes | + | true | no | + | true | '; delete from foobar; select ' | + + Scenario: Empty XML search with exluded place ids + When sending xml search query "jghrleoxsbwjer" + | exclude_place_ids | + | 123,76,342565 | + Then result header contains + | attr | value | + | exclude_place_ids | 123,76,342565 | + + Scenario: Empty XML search with bad exluded place ids + When sending xml search query "jghrleoxsbwjer" + | exclude_place_ids | + | , | + Then result header has not attributes exclude_place_ids + + Scenario Outline: Wrapping of legal jsonp search requests + When sending json search query "Tokyo" + | param | value | + |json_callback | | + Then result header contains + | attr | value | + | json_func | | + + Examples: + | data | result | + | foo | foo | + | FOO | FOO | + | __world | __world | + | $me | \$me | + | m1[4] | m1\[4\] | + | d_r[$d] | d_r\[\$d\] | + + Scenario Outline: Wrapping of illegal jsonp search requests + When sending json search query "Tokyo" + | param | value | + |json_callback | | + Then a HTTP 400 is returned + + Examples: + | data | + | 1asd | + | bar(foo) | + | XXX['bad'] | + | foo; evil | + + Scenario: Ignore jsonp parameter for anything but json + When sending json search query "Malibu" + | json_callback | + | 234 | + Then a HTTP 400 is returned + When sending xml search query "Malibu" + | json_callback | + | 234 | + Then the result is valid xml + When sending html search query "Malibu" + | json_callback | + | 234 | + Then the result is valid html + + Scenario: Empty JSON search + When sending json search query "YHlERzzx" + Then exactly 0 results are returned + + Scenario: Empty JSONv2 search + When sending jsonv2 search query "Flubb XdfESSaZx" + Then exactly 0 results are returned + + Scenario: Search for non-existing coordinates + When sending json search query "-21.0,-33.0" + Then exactly 0 results are returned + diff --git a/test/bdd/api/search/structured.feature b/test/bdd/api/search/structured.feature new file mode 100644 index 00000000..c93603d6 --- /dev/null +++ b/test/bdd/api/search/structured.feature @@ -0,0 +1,38 @@ +@APIDB +Feature: Structured search queries + Testing correctness of results with + structured queries + + Scenario: Country only + When sending json search query "" with address + | country | + | Liechtenstein | + Then address of result 0 is + | type | value | + | country | Liechtenstein | + | country_code | li | + + Scenario: Postcode only + When sending json search query "" with address + | postalcode | + | 22547 | + Then results contain + | type | + | postcode | + And result addresses contain + | postcode | + | 22547 | + + Scenario: Street, postcode and country + When sending xml search query "" with address + | street | postalcode | country | + | Old Palace Road | GU2 7UP | United Kingdom | + Then result header contains + | attr | value | + | querystring | Old Palace Road, GU2 7UP, United Kingdom | + + Scenario: gihub #176 + When sending json search query "" with address + | city | + | Mercedes | + Then at least 1 result is returned diff --git a/test/bdd/db/import/interpolation.feature b/test/bdd/db/import/interpolation.feature new file mode 100644 index 00000000..4ed66b91 --- /dev/null +++ b/test/bdd/db/import/interpolation.feature @@ -0,0 +1,303 @@ +@DB +Feature: Import of address interpolations + Tests that interpolated addresses are added correctly + + Scenario: Simple even interpolation line with two points + Given the places + | osm | class | type | housenr | geometry | + | N1 | place | house | 2 | 1 1 | + | N2 | place | house | 6 | 1 1.001 | + | W1 | place | houses | even | 1 1, 1 1.001 | + And the ways + | id | nodes | + | 1 | 1,2 | + When importing + Then W1 expands to interpolation + | start | end | geometry | + | 2 | 6 | 1 1, 1 1.001 | + + Scenario: Backwards even two point interpolation line + Given the places + | osm | class | type | housenr | geometry | + | N1 | place | house | 2 | 1 1 | + | N2 | place | house | 6 | 1 1.001 | + | W1 | place | houses | even | 1 1.001, 1 1 | + And the ways + | id | nodes | + | 1 | 2,1 | + When importing + Then W1 expands to interpolation + | start | end | geometry | + | 2 | 6 | 1 1, 1 1.001 | + + Scenario: Simple odd two point interpolation + Given the places + | osm | class | type | housenr | geometry | + | N1 | place | house | 1 | 1 1 | + | N2 | place | house | 11 | 1 1.001 | + | W1 | place | houses | odd | 1 1, 1 1.001 | + And the ways + | id | nodes | + | 1 | 1,2 | + When importing + Then W1 expands to interpolation + | start | end | geometry | + | 1 | 11 | 1 1, 1 1.001 | + + Scenario: Simple all two point interpolation + Given the places + | osm | class | type | housenr | geometry | + | N1 | place | house | 1 | 1 1 | + | N2 | place | house | 3 | 1 1.001 | + | W1 | place | houses | all | 1 1, 1 1.001 | + And the ways + | id | nodes | + | 1 | 1,2 | + When importing + Then W1 expands to interpolation + | start | end | geometry | + | 1 | 3 | 1 1, 1 1.001 | + + Scenario: Even two point interpolation line with intermediate empty node + Given the places + | osm | class | type | housenr | geometry | + | N1 | place | house | 2 | 1 1 | + | N2 | place | house | 10 | 1.001 1.001 | + | W1 | place | houses | even | 1 1, 1 1.001, 1.001 1.001 | + And the ways + | id | nodes | + | 1 | 1,3,2 | + When importing + Then W1 expands to interpolation + | start | end | geometry | + | 2 | 10 | 1 1, 1 1.001, 1.001 1.001 | + + Scenario: Even two point interpolation line with intermediate duplicated empty node + Given the places + | osm | class | type | housenr | geometry | + | N1 | place | house | 2 | 1 1 | + | N2 | place | house | 10 | 1.001 1.001 | + | W1 | place | houses | even | 1 1, 1 1.001, 1.001 1.001 | + And the ways + | id | nodes | + | 1 | 1,3,3,2 | + When importing + Then W1 expands to interpolation + | start | end | geometry | + | 2 | 10 | 1 1, 1 1.001, 1.001 1.001 | + + Scenario: Simple even three point interpolation line + Given the places + | osm | class | type | housenr | geometry | + | N1 | place | house | 2 | 1 1 | + | N2 | place | house | 14 | 1.001 1.001 | + | N3 | place | house | 10 | 1 1.001 | + | W1 | place | houses | even | 1 1, 1 1.001, 1.001 1.001 | + And the ways + | id | nodes | + | 1 | 1,3,2 | + When importing + Then W1 expands to interpolation + | start | end | geometry | + | 2 | 10 | 1 1, 1 1.001 | + | 10 | 14 | 1 1.001, 1.001 1.001 | + + Scenario: Simple even four point interpolation line + Given the places + | osm | class | type | housenr | geometry | + | N1 | place | house | 2 | 1 1 | + | N2 | place | house | 14 | 1.001 1.001 | + | N3 | place | house | 10 | 1 1.001 | + | N4 | place | house | 18 | 1.001 1.002 | + | W1 | place | houses | even | 1 1, 1 1.001, 1.001 1.001, 1.001 1.002 | + And the ways + | id | nodes | + | 1 | 1,3,2,4 | + When importing + Then W1 expands to interpolation + | start | end | geometry | + | 2 | 10 | 1 1, 1 1.001 | + | 10 | 14 | 1 1.001, 1.001 1.001 | + | 14 | 18 | 1.001 1.001, 1.001 1.002 | + + Scenario: Reverse simple even three point interpolation line + Given the places + | osm | class | type | housenr | geometry | + | N1 | place | house | 2 | 1 1 | + | N2 | place | house | 14 | 1.001 1.001 | + | N3 | place | house | 10 | 1 1.001 | + | W1 | place | houses | even | 1.001 1.001, 1 1.001, 1 1 | + And the ways + | id | nodes | + | 1 | 2,3,1 | + When importing + Then W1 expands to interpolation + | start | end | geometry | + | 2 | 10 | 1 1, 1 1.001 | + | 10 | 14 | 1 1.001, 1.001 1.001 | + + Scenario: Even three point interpolation line with odd center point + Given the places + | osm | class | type | housenr | geometry | + | N1 | place | house | 2 | 1 1 | + | N2 | place | house | 8 | 1.001 1.001 | + | N3 | place | house | 7 | 1 1.001 | + | W1 | place | houses | even | 1 1, 1 1.001, 1.001 1.001 | + And the ways + | id | nodes | + | 1 | 1,3,2 | + When importing + Then W1 expands to interpolation + | start | end | geometry | + | 2 | 7 | 1 1, 1 1.001 | + | 7 | 8 | 1 1.001, 1.001 1.001 | + + Scenario: Interpolation line with self-intersecting way + Given the places + | osm | class | type | housenr | geometry | + | N1 | place | house | 2 | 0 0 | + | N2 | place | house | 6 | 0 0.001 | + | N3 | place | house | 10 | 0 0.002 | + | W1 | place | houses | even | 0 0, 0 0.001, 0 0.002, 0 0.001 | + And the ways + | id | nodes | + | 1 | 1,2,3,2 | + When importing + Then W1 expands to interpolation + | start | end | geometry | + | 2 | 6 | 0 0, 0 0.001 | + | 6 | 10 | 0 0.001, 0 0.002 | + | 6 | 10 | 0 0.001, 0 0.002 | + + Scenario: Interpolation line with self-intersecting way II + Given the places + | osm | class | type | housenr | geometry | + | N1 | place | house | 2 | 0 0 | + | N2 | place | house | 6 | 0 0.001 | + | W1 | place | houses | even | 0 0, 0 0.001, 0 0.002, 0 0.001 | + And the ways + | id | nodes | + | 1 | 1,2,3,2 | + When importing + Then W1 expands to interpolation + | start | end | geometry | + | 2 | 6 | 0 0, 0 0.001 | + + Scenario: addr:street on interpolation way + Given the scene parallel-road + And the places + | osm | class | type | housenr | geometry | + | N1 | place | house | 2 | :n-middle-w | + | N2 | place | house | 6 | :n-middle-e | + | N3 | place | house | 12 | :n-middle-w | + | N4 | place | house | 16 | :n-middle-e | + And the places + | osm | class | type | housenr | street | geometry | + | W10 | place | houses | even | | :w-middle | + | W11 | place | houses | even | Cloud Street | :w-middle | + And the places + | osm | class | type | name | geometry | + | W2 | highway | tertiary | Sun Way | :w-north | + | W3 | highway | tertiary | Cloud Street | :w-south | + And the ways + | id | nodes | + | 10 | 1,100,101,102,2 | + | 11 | 3,200,201,202,4 | + When importing + Then placex contains + | object | parent_place_id | + | N1 | W2 | + | N2 | W2 | + | N3 | W3 | + | N4 | W3 | + Then W10 expands to interpolation + | parent_place_id | start | end | + | W2 | 2 | 6 | + Then W11 expands to interpolation + | parent_place_id | start | end | + | W3 | 12 | 16 | + When searching for "16 Cloud Street" + Then results contain + | ID | osm_type | osm_id | + | 0 | N | 4 | + When searching for "14 Cloud Street" + Then results contain + | ID | osm_type | osm_id | + | 0 | W | 11 | + When searching for "18 Cloud Street" + Then results contain + | ID | osm_type | osm_id | + | 0 | W | 3 | + + Scenario: addr:street on housenumber way + Given the scene parallel-road + And the places + | osm | class | type | housenr | street | geometry | + | N1 | place | house | 2 | | :n-middle-w | + | N2 | place | house | 6 | | :n-middle-e | + | N3 | place | house | 12 | Cloud Street | :n-middle-w | + | N4 | place | house | 16 | Cloud Street | :n-middle-e | + And the places + | osm | class | type | housenr | geometry | + | W10 | place | houses | even | :w-middle | + | W11 | place | houses | even | :w-middle | + And the places + | osm | class | type | name | geometry | + | W2 | highway | tertiary | Sun Way | :w-north | + | W3 | highway | tertiary | Cloud Street | :w-south | + And the ways + | id | nodes | + | 10 | 1,100,101,102,2 | + | 11 | 3,200,201,202,4 | + When importing + Then placex contains + | object | parent_place_id | + | N1 | W2 | + | N2 | W2 | + | N3 | W3 | + | N4 | W3 | + Then W10 expands to interpolation + | parent_place_id | start | end | + | W2 | 2 | 6 | + Then W11 expands to interpolation + | parent_place_id | start | end | + | W3 | 12 | 16 | + When searching for "16 Cloud Street" + Then results contain + | ID | osm_type | osm_id | + | 0 | N | 4 | + When searching for "14 Cloud Street" + Then results contain + | ID | osm_type | osm_id | + | 0 | W | 11 | + + Scenario: Geometry of points and way don't match (github #253) + Given the places + | osm | class | type | housenr | geometry | + | N1 | place | house | 10 | 144.9632341 -37.76163 | + | N2 | place | house | 6 | 144.9630541 -37.7628174 | + | N3 | shop | supermarket | 2 | 144.9629794 -37.7630755 | + | W1 | place | houses | even | 144.9632341 -37.76163,144.9630541 -37.7628172,144.9629794 -37.7630755 | + And the ways + | id | nodes | + | 1 | 1,2,3 | + When importing + Then W1 expands to interpolation + | start | end | geometry | + | 2 | 6 | 144.9629794 -37.7630755, 144.9630541 -37.7628174 | + | 6 | 10 | 144.9630541 -37.7628174, 144.9632341 -37.76163 | + + Scenario: Place with missing address information + Given the places + | osm | class | type | housenr | geometry | + | N1 | place | house | 23 | 0.0001 0.0001 | + | N2 | amenity | school | | 0.0001 0.0002 | + | N3 | place | house | 29 | 0.0001 0.0004 | + | W1 | place | houses | odd | 0.0001 0.0001,0.0001 0.0002,0.0001 0.0004 | + And the ways + | id | nodes | + | 1 | 1,2,3 | + When importing + Then W1 expands to interpolation + | start | end | geometry | + | 23 | 29 | 0.0001 0.0001, 0.0001 0.0002, 0.0001 0.0004 | diff --git a/test/bdd/db/import/linking.feature b/test/bdd/db/import/linking.feature new file mode 100644 index 00000000..0954ed4e --- /dev/null +++ b/test/bdd/db/import/linking.feature @@ -0,0 +1,110 @@ +@DB +Feature: Linking of places + Tests for correctly determining linked places + + Scenario: Only address-describing places can be linked + Given the scene way-area-with-center + And the places + | osm | class | type | name | geometry | + | R13 | landuse | forest | Garbo | :area | + | N256 | natural | peak | Garbo | :inner-C | + When importing + Then placex contains + | object | linked_place_id | + | R13 | - | + | N256 | - | + + Scenario: Waterways are linked when in waterway relations + Given the scene split-road + And the places + | osm | class | type | name | geometry | + | W1 | waterway | river | Rhein | :w-2 | + | W2 | waterway | river | Rhein | :w-3 | + | R13 | waterway | river | Rhein | :w-1 + :w-2 + :w-3 | + | R23 | waterway | river | Limmat| :w-4a | + And the relations + | id | members | tags+type | + | 13 | R23:tributary,W1,W2:main_stream | waterway | + When importing + Then placex contains + | object | linked_place_id | + | W1 | R13 | + | W2 | R13 | + | R13 | - | + | R23 | - | + When searching for "rhein" + Then results contain + | osm_type | + | R | + + Scenario: Relations are not linked when in waterway relations + Given the scene split-road + And the places + | osm | class | type | name | geometry | + | W1 | waterway | river | Rhein | :w-2 | + | W2 | waterway | river | Rhein | :w-3 | + | R1 | waterway | river | Rhein | :w-1 + :w-2 + :w-3 | + | R2 | waterway | river | Limmat| :w-4a | + And the relations + | id | members | tags+type | + | 1 | R2 | waterway | + When importing + Then placex contains + | object | linked_place_id | + | W1 | - | + | W2 | - | + | R1 | - | + | R2 | - | + + Scenario: Empty waterway relations are handled correctly + Given the scene split-road + And the places + | osm | class | type | name | geometry | + | R1 | waterway | river | Rhein | :w-1 + :w-2 + :w-3 | + And the relations + | id | members | tags+type | + | 1 | | waterway | + When importing + Then placex contains + | object | linked_place_id | + | R1 | - | + + Scenario: Waterways are not linked when waterway types don't match + Given the scene split-road + And the places + | osm | class | type | name | geometry | + | W1 | waterway | drain | Rhein | :w-2 | + | R1 | waterway | river | Rhein | :w-1 + :w-2 + :w-3 | + And the relations + | id | members | tags+type | + | 1 | N23,N34,W1,R45 | multipolygon | + When importing + Then placex contains + | object | linked_place_id | + | W1 | - | + | R1 | - | + When searching for "rhein" + Then results contain + | ID | osm_type | + | 0 | R | + | 1 | W | + + Scenario: Side streams are linked only when they have the same name + Given the scene split-road + And the places + | osm | class | type | name | geometry | + | W1 | waterway | river | Rhein2 | :w-2 | + | W2 | waterway | river | Rhein | :w-3 | + | R1 | waterway | river | Rhein | :w-1 + :w-2 + :w-3 | + And the relations + | id | members | tags+type | + | 1 | W1:side_stream,W2:side_stream | waterway | + When importing + Then placex contains + | object | linked_place_id | + | W1 | - | + | W2 | R1 | + When searching for "rhein2" + Then results contain + | osm_type | + | W | diff --git a/test/bdd/db/import/naming.feature b/test/bdd/db/import/naming.feature new file mode 100644 index 00000000..d2339376 --- /dev/null +++ b/test/bdd/db/import/naming.feature @@ -0,0 +1,39 @@ +@DB +Feature: Import and search of names + Tests all naming related import issues + + Scenario: No copying name tag if only one name + Given the places + | osm | class | type | name | geometry | + | N1 | place | locality | german | country:de | + When importing + Then placex contains + | object | calculated_country_code | name+name | + | N1 | de | german | + + Scenario: Copying name tag to default language if it does not exist + Given the places + | osm | class | type | name | name+name:fi | geometry | + | N1 | place | locality | german | finnish | country:de | + When importing + Then placex contains + | object | calculated_country_code | name | name+name:fi | name+name:de | + | N1 | de | german | finnish | german | + + Scenario: Copying default language name tag to name if it does not exist + Given the places + | osm | class | type | name+name:de | name+name:fi | geometry | + | N1 | place | locality | german | finnish | country:de | + When importing + Then placex contains + | object | calculated_country_code | name | name+name:fi | name+name:de | + | N1 | de | german | finnish | german | + + Scenario: Do not overwrite default language with name tag + Given the places + | osm | class | type | name | name+name:fi | name+name:de | geometry | + | N1 | place | locality | german | finnish | local | country:de | + When importing + Then placex contains + | object | calculated_country_code | name | name+name:fi | name+name:de | + | N1 | de | german | finnish | local | diff --git a/test/bdd/db/import/parenting.feature b/test/bdd/db/import/parenting.feature new file mode 100644 index 00000000..b0b76438 --- /dev/null +++ b/test/bdd/db/import/parenting.feature @@ -0,0 +1,443 @@ +@DB +Feature: Parenting of objects + Tests that the correct parent is choosen + + Scenario: Address inherits postcode from its street unless it has a postcode + Given the scene roads-with-pois + And the places + | osm | class | type | housenr | geometry | + | N1 | place | house | 4 | :p-N1 | + And the places + | osm | class | type | housenr | postcode | geometry | + | N2 | place | house | 5 | 99999 | :p-N1 | + And the places + | osm | class | type | name | postcode | geometry | + | W1 | highway | residential | galoo | 12345 | :w-north | + When importing + Then placex contains + | object | parent_place_id | + | N1 | W1 | + | N2 | W1 | + When searching for "4 galoo" + Then results contain + | ID | osm_type | osm_id | langaddress + | 0 | N | 1 | 4, galoo, 12345 + When searching for "5 galoo" + Then results contain + | ID | osm_type | osm_id | langaddress + | 0 | N | 2 | 5, galoo, 99999 + + Scenario: Address without tags, closest street + Given the scene roads-with-pois + And the places + | osm | class | type | geometry | + | N1 | place | house | :p-N1 | + | N2 | place | house | :p-N2 | + | N3 | place | house | :p-S1 | + | N4 | place | house | :p-S2 | + And the named places + | osm | class | type | geometry | + | W1 | highway | residential | :w-north | + | W2 | highway | residential | :w-south | + When importing + Then placex contains + | object | parent_place_id | + | N1 | W1 | + | N2 | W1 | + | N3 | W2 | + | N4 | W2 | + + Scenario: Address without tags avoids unnamed streets + Given the scene roads-with-pois + And the places + | osm | class | type | geometry | + | N1 | place | house | :p-N1 | + | N2 | place | house | :p-N2 | + | N3 | place | house | :p-S1 | + | N4 | place | house | :p-S2 | + | W1 | highway | residential | :w-north | + And the named places + | osm | class | type | geometry | + | W2 | highway | residential | :w-south | + When importing + Then placex contains + | object | parent_place_id | + | N1 | W2 | + | N2 | W2 | + | N3 | W2 | + | N4 | W2 | + + Scenario: addr:street tag parents to appropriately named street + Given the scene roads-with-pois + And the places + | osm | class | type | street| geometry | + | N1 | place | house | south | :p-N1 | + | N2 | place | house | north | :p-N2 | + | N3 | place | house | south | :p-S1 | + | N4 | place | house | north | :p-S2 | + And the places + | osm | class | type | name | geometry | + | W1 | highway | residential | north | :w-north | + | W2 | highway | residential | south | :w-south | + When importing + Then placex contains + | object | parent_place_id | + | N1 | W2 | + | N2 | W1 | + | N3 | W2 | + | N4 | W1 | + + Scenario: addr:street tag parents to next named street + Given the scene roads-with-pois + And the places + | osm | class | type | street | geometry | + | N1 | place | house | abcdef | :p-N1 | + | N2 | place | house | abcdef | :p-N2 | + | N3 | place | house | abcdef | :p-S1 | + | N4 | place | house | abcdef | :p-S2 | + And the places + | osm | class | type | name | geometry | + | W1 | highway | residential | abcdef | :w-north | + | W2 | highway | residential | abcdef | :w-south | + When importing + Then placex contains + | object | parent_place_id | + | N1 | W1 | + | N2 | W1 | + | N3 | W2 | + | N4 | W2 | + + Scenario: addr:street tag without appropriately named street + Given the scene roads-with-pois + And the places + | osm | class | type | street | geometry | + | N1 | place | house | abcdef | :p-N1 | + | N2 | place | house | abcdef | :p-N2 | + | N3 | place | house | abcdef | :p-S1 | + | N4 | place | house | abcdef | :p-S2 | + And the places + | osm | class | type | name | geometry | + | W1 | highway | residential | abcde | :w-north | + | W2 | highway | residential | abcde | :w-south | + When importing + Then placex contains + | object | parent_place_id | + | N1 | W1 | + | N2 | W1 | + | N3 | W2 | + | N4 | W2 | + + Scenario: addr:place address + Given the scene road-with-alley + And the places + | osm | class | type | addr_place | geometry | + | N1 | place | house | myhamlet | :n-alley | + And the places + | osm | class | type | name | geometry | + | N2 | place | hamlet | myhamlet | :n-main-west | + | W1 | highway | residential | myhamlet | :w-main | + When importing + Then placex contains + | object | parent_place_id | + | N1 | N2 | + + Scenario: addr:street is preferred over addr:place + Given the scene road-with-alley + And the places + | osm | class | type | addr_place | street | geometry | + | N1 | place | house | myhamlet | mystreet| :n-alley | + And the places + | osm | class | type | name | geometry | + | N2 | place | hamlet | myhamlet | :n-main-west | + | W1 | highway | residential | mystreet | :w-main | + When importing + Then placex contains + | object | parent_place_id | + | N1 | W1 | + + Scenario: Untagged address in simple associated street relation + Given the scene road-with-alley + And the places + | osm | class | type | geometry | + | N1 | place | house | :n-alley | + | N2 | place | house | :n-corner | + | N3 | place | house | :n-main-west | + And the places + | osm | class | type | name | geometry | + | W1 | highway | residential | foo | :w-main | + | W2 | highway | service | bar | :w-alley | + And the relations + | id | members | tags+type | + | 1 | W1:street,N1,N2,N3 | associatedStreet | + When importing + Then placex contains + | object | parent_place_id | + | N1 | W1 | + | N2 | W1 | + | N3 | W1 | + + Scenario: Avoid unnamed streets in simple associated street relation + Given the scene road-with-alley + And the places + | osm | class | type | geometry | + | N1 | place | house | :n-alley | + | N2 | place | house | :n-corner | + | N3 | place | house | :n-main-west | + | W2 | highway | residential | :w-alley | + And the named places + | osm | class | type | geometry | + | W1 | highway | residential | :w-main | + And the relations + | id | members | tags+type | + | 1 | N1,N2,N3,W2:street,W1:street | associatedStreet | + When importing + Then placex contains + | object | parent_place_id | + | N1 | W1 | + | N2 | W1 | + | N3 | W1 | + + Scenario: Associated street relation overrides addr:street + Given the scene road-with-alley + And the places + | osm | class | type | street | geometry | + | N1 | place | house | bar | :n-alley | + And the places + | osm | class | type | name | geometry | + | W1 | highway | residential | foo | :w-main | + | W2 | highway | residential | bar | :w-alley | + And the relations + | id | members | tags+type | + | 1 | W1:street,N1,N2,N3 | associatedStreet | + When importing + Then placex contains + | object | parent_place_id | + | N1 | W1 | + + Scenario: Building without tags, closest street from center point + Given the scene building-on-street-corner + And the named places + | osm | class | type | geometry | + | W1 | building | yes | :w-building | + | W2 | highway | primary | :w-WE | + | W3 | highway | residential | :w-NS | + When importing + Then placex contains + | object | parent_place_id | + | W1 | W3 | + + Scenario: Building with addr:street tags + Given the scene building-on-street-corner + And the named places + | osm | class | type | street | geometry | + | W1 | building | yes | bar | :w-building | + And the places + | osm | class | type | name | geometry | + | W2 | highway | primary | bar | :w-WE | + | W3 | highway | residential | foo | :w-NS | + When importing + Then placex contains + | object | parent_place_id | + | W1 | W2 | + + Scenario: Building with addr:place tags + Given the scene building-on-street-corner + And the places + | osm | class | type | name | geometry | + | N1 | place | village | bar | :n-outer | + | W2 | highway | primary | bar | :w-WE | + | W3 | highway | residential | foo | :w-NS | + And the named places + | osm | class | type | addr_place | geometry | + | W1 | building | yes | bar | :w-building | + When importing + Then placex contains + | object | parent_place_id | + | W1 | N1 | + + Scenario: Building in associated street relation + Given the scene building-on-street-corner + And the named places + | osm | class | type | geometry | + | W1 | building | yes | :w-building | + And the places + | osm | class | type | name | geometry | + | W2 | highway | primary | bar | :w-WE | + | W3 | highway | residential | foo | :w-NS | + And the relations + | id | members | tags+type | + | 1 | W1:house,W2:street | associatedStreet | + When importing + Then placex contains + | object | parent_place_id | + | W1 | W2 | + + Scenario: Building in associated street relation overrides addr:street + Given the scene building-on-street-corner + And the named places + | osm | class | type | street | geometry | + | W1 | building | yes | foo | :w-building | + And the places + | osm | class | type | name | geometry | + | W2 | highway | primary | bar | :w-WE | + | W3 | highway | residential | foo | :w-NS | + And the relations + | id | members | tags+type | + | 1 | W1:house,W2:street | associatedStreet | + When importing + Then placex contains + | object | parent_place_id | + | W1 | W2 | + + Scenario: Wrong member in associated street relation is ignored + Given the scene building-on-street-corner + And the named places + | osm | class | type | geometry | + | N1 | place | house | :n-outer | + And the named places + | osm | class | type | street | geometry | + | W1 | building | yes | foo | :w-building | + And the places + | osm | class | type | name | geometry | + | W2 | highway | primary | bar | :w-WE | + | W3 | highway | residential | foo | :w-NS | + And the relations + | id | members | tags+type | + | 1 | N1:house,W1:street,W3:street | associatedStreet | + When importing + Then placex contains + | object | parent_place_id | + | N1 | W3 | + + Scenario: POIs in building inherit address + Given the scene building-on-street-corner + And the named places + | osm | class | type | geometry | + | N1 | amenity | bank | :n-inner | + | N2 | shop | bakery | :n-edge-NS | + | N3 | shop | supermarket| :n-edge-WE | + And the places + | osm | class | type | street | addr_place | housenr | geometry | + | W1 | building | yes | foo | nowhere | 3 | :w-building | + And the places + | osm | class | type | name | geometry | + | W2 | highway | primary | bar | :w-WE | + | W3 | highway | residential | foo | :w-NS | + When importing + Then placex contains + | object | parent_place_id | street | addr_place | housenumber | + | W1 | W3 | foo | nowhere | 3 | + | N1 | W3 | foo | nowhere | 3 | + | N2 | W3 | foo | nowhere | 3 | + | N3 | W3 | foo | nowhere | 3 | + + Scenario: POIs don't inherit from streets + Given the scene building-on-street-corner + And the named places + | osm | class | type | geometry | + | N1 | amenity | bank | :n-inner | + And the places + | osm | class | type | street | addr_place | housenr | geometry | + | W1 | highway | path | foo | nowhere | 3 | :w-building | + And the places + | osm | class | type | name | geometry | + | W3 | highway | residential | foo | :w-NS | + When importing + Then placex contains + | object | parent_place_id | street | addr_place | housenumber | + | N1 | W3 | None | None | None | + + Scenario: POIs with own address do not inherit building address + Given the scene building-on-street-corner + And the named places + | osm | class | type | street | geometry | + | N1 | amenity | bank | bar | :n-inner | + And the named places + | osm | class | type | housenr | geometry | + | N2 | shop | bakery | 4 | :n-edge-NS | + And the named places + | osm | class | type | addr_place | geometry | + | N3 | shop | supermarket| nowhere | :n-edge-WE | + And the places + | osm | class | type | name | geometry | + | N4 | place | isolated_dwelling | theplace | :n-outer | + And the places + | osm | class | type | addr_place | housenr | geometry | + | W1 | building | yes | theplace | 3 | :w-building | + And the places + | osm | class | type | name | geometry | + | W2 | highway | primary | bar | :w-WE | + | W3 | highway | residential | foo | :w-NS | + When importing + Then placex contains + | object | parent_place_id | street | addr_place | housenumber | + | W1 | N4 | None | theplace | 3 | + | N1 | W2 | bar | None | None | + | N2 | W3 | None | None | 4 | + | N3 | W2 | None | nowhere | None | + + Scenario: POIs parent a road if they are attached to it + Given the scene points-on-roads + And the named places + | osm | class | type | street | geometry | + | N1 | highway | bus_stop | North St | :n-SE | + | N2 | highway | bus_stop | South St | :n-NW | + | N3 | highway | bus_stop | North St | :n-S-unglued | + | N4 | highway | bus_stop | South St | :n-N-unglued | + And the places + | osm | class | type | name | geometry | + | W1 | highway | secondary | North St | :w-north | + | W2 | highway | unclassified | South St | :w-south | + And the ways + | id | nodes | + | 1 | 100,101,2,103,104 | + | 2 | 200,201,1,202,203 | + When importing + Then placex contains + | object | parent_place_id | + | N1 | W1 | + | N2 | W2 | + | N3 | W1 | + | N4 | W2 | + + Scenario: POIs do not parent non-roads they are attached to + Given the scene points-on-roads + And the named places + | osm | class | type | street | geometry | + | N1 | highway | bus_stop | North St | :n-SE | + | N2 | highway | bus_stop | South St | :n-NW | + And the places + | osm | class | type | name | geometry | + | W1 | landuse | residential | North St | :w-north | + | W2 | waterway| river | South St | :w-south | + And the ways + | id | nodes | + | 1 | 100,101,2,103,104 | + | 2 | 200,201,1,202,203 | + When importing + Then placex contains + | object | parent_place_id | + | N1 | 0 | + | N2 | 0 | + + Scenario: POIs on building outlines inherit associated street relation + Given the scene building-on-street-corner + And the named places + | osm | class | type | geometry | + | N1 | place | house | :n-edge-NS | + | W1 | building | yes | :w-building | + And the places + | osm | class | type | name | geometry | + | W2 | highway | primary | bar | :w-WE | + | W3 | highway | residential | foo | :w-NS | + And the relations + | id | members | tags+type | + | 1 | W1:house,W2:street | associatedStreet | + And the ways + | id | nodes | + | 1 | 100,1,101,102,100 | + When importing + Then placex contains + | object | parent_place_id | + | N1 | W2 | + diff --git a/test/bdd/db/import/placex.feature b/test/bdd/db/import/placex.feature new file mode 100644 index 00000000..7cbedaa3 --- /dev/null +++ b/test/bdd/db/import/placex.feature @@ -0,0 +1,313 @@ +@DB +Feature: Import into placex + Tests that data in placex is completed correctly. + + Scenario: No country code tag is available + Given the named places + | osm | class | type | geometry | + | N1 | highway | primary | country:us | + When importing + Then placex contains + | object | country_code | calculated_country_code | + | N1 | None | us | + + Scenario: Location overwrites country code tag + Given the named places + | osm | class | type | country | geometry | + | N1 | highway | primary | de | country:us | + When importing + Then placex contains + | object | country_code | calculated_country_code | + | N1 | de | us | + + Scenario: Country code tag overwrites location for countries + Given the named places + | osm | class | type | admin | country | geometry | + | R1 | boundary | administrative | 2 | de | (-100 40, -101 40, -101 41, -100 41, -100 40) | + When importing + Then placex contains + | object | country_code | calculated_country_code | + | R1 | de | de | + + Scenario: Illegal country code tag for countries is ignored + Given the named places + | osm | class | type | admin | country | geometry | + | R1 | boundary | administrative | 2 | xx | (-100 40, -101 40, -101 41, -100 41, -100 40) | + When importing + Then placex contains + | object | country_code | calculated_country_code | + | R1 | xx | us | + + Scenario: admin level is copied over + Given the named places + | osm | class | type | admin | + | N1 | place | state | 3 | + When importing + Then placex contains + | object | admin_level | + | N1 | 3 | + + Scenario: admin level is default 15 + Given the named places + | osm | class | type | + | N1 | amenity | prison | + When importing + Then placex contains + | object | admin_level | + | N1 | 15 | + + Scenario: admin level is never larger than 15 + Given the named places + | osm | class | type | admin | + | N1 | amenity | prison | 16 | + When importing + Then placex contains + | object | admin_level | + | N1 | 15 | + + Scenario: postcode node without postcode is dropped + Given the places + | osm | class | type | name+ref | + | N1 | place | postcode | 12334 | + When importing + Then placex has no entry for N1 + + Scenario: postcode boundary without postcode is dropped + Given the places + | osm | class | type | name+ref | geometry | + | R1 | boundary | postal_code | 554476 | poly-area:0.1 | + When importing + Then placex has no entry for R1 + + Scenario: search and address ranks for GB post codes correctly assigned + Given the places + | osm | class | type | postcode | geometry | + | N1 | place | postcode | E45 2CD | country:gb | + | N2 | place | postcode | E45 2 | country:gb | + | N3 | place | postcode | Y45 | country:gb | + When importing + Then placex contains + | object | postcode | calculated_country_code | rank_search | rank_address | + | N1 | E45 2CD | gb | 25 | 5 | + | N2 | E45 2 | gb | 23 | 5 | + | N3 | Y45 | gb | 21 | 5 | + + Scenario: wrongly formatted GB postcodes are down-ranked + Given the places + | osm | class | type | postcode | geometry | + | N1 | place | postcode | EA452CD | country:gb | + | N2 | place | postcode | E45 23 | country:gb | + | N3 | place | postcode | y45 | country:gb | + When importing + Then placex contains + | object | calculated_country_code | rank_search | rank_address | + | N1 | gb | 30 | 30 | + | N2 | gb | 30 | 30 | + | N3 | gb | 30 | 30 | + + Scenario: search and address rank for DE postcodes correctly assigned + Given the places + | osm | class | type | postcode | geometry | + | N1 | place | postcode | 56427 | country:de | + | N2 | place | postcode | 5642 | country:de | + | N3 | place | postcode | 5642A | country:de | + | N4 | place | postcode | 564276 | country:de | + When importing + Then placex contains + | object | calculated_country_code | rank_search | rank_address | + | N1 | de | 21 | 11 | + | N2 | de | 30 | 30 | + | N3 | de | 30 | 30 | + | N4 | de | 30 | 30 | + + Scenario: search and address rank for other postcodes are correctly assigned + Given the places + | osm | class | type | postcode | geometry | + | N1 | place | postcode | 1 | country:ca | + | N2 | place | postcode | X3 | country:ca | + | N3 | place | postcode | 543 | country:ca | + | N4 | place | postcode | 54dc | country:ca | + | N5 | place | postcode | 12345 | country:ca | + | N6 | place | postcode | 55TT667 | country:ca | + | N7 | place | postcode | 123-65 | country:ca | + | N8 | place | postcode | 12 445 4 | country:ca | + | N9 | place | postcode | A1:bc10 | country:ca | + When importing + Then placex contains + | object | calculated_country_code | rank_search | rank_address | + | N1 | ca | 21 | 11 | + | N2 | ca | 21 | 11 | + | N3 | ca | 21 | 11 | + | N4 | ca | 21 | 11 | + | N5 | ca | 21 | 11 | + | N6 | ca | 21 | 11 | + | N7 | ca | 25 | 11 | + | N8 | ca | 25 | 11 | + | N9 | ca | 25 | 11 | + + Scenario: search and address ranks for places are correctly assigned + Given the named places + | osm | class | type | + | N1 | foo | bar | + | N11 | place | Continent | + | N12 | place | continent | + | N13 | place | sea | + | N14 | place | country | + | N15 | place | state | + | N16 | place | region | + | N17 | place | county | + | N18 | place | city | + | N19 | place | island | + | N20 | place | town | + | N21 | place | village | + | N22 | place | hamlet | + | N23 | place | municipality | + | N24 | place | district | + | N25 | place | unincorporated_area | + | N26 | place | borough | + | N27 | place | suburb | + | N28 | place | croft | + | N29 | place | subdivision | + | N30 | place | isolated_dwelling | + | N31 | place | farm | + | N32 | place | locality | + | N33 | place | islet | + | N34 | place | mountain_pass | + | N35 | place | neighbourhood | + | N36 | place | house | + | N37 | place | building | + | N38 | place | houses | + And the named places + | osm | class | type | extra+locality | + | N100 | place | locality | townland | + And the named places + | osm | class | type | extra+capital | + | N101 | place | city | yes | + When importing + Then placex contains + | object | rank_search | rank_address | + | N1 | 30 | 30 | + | N11 | 30 | 30 | + | N12 | 2 | 2 | + | N13 | 2 | 0 | + | N14 | 4 | 4 | + | N15 | 8 | 8 | + | N16 | 18 | 0 | + | N17 | 12 | 12 | + | N18 | 16 | 16 | + | N19 | 17 | 0 | + | N20 | 18 | 16 | + | N21 | 19 | 16 | + | N22 | 19 | 16 | + | N23 | 19 | 16 | + | N24 | 19 | 16 | + | N25 | 19 | 16 | + | N26 | 19 | 16 | + | N27 | 20 | 20 | + | N28 | 20 | 20 | + | N29 | 20 | 20 | + | N30 | 20 | 20 | + | N31 | 20 | 0 | + | N32 | 20 | 0 | + | N33 | 20 | 0 | + | N34 | 20 | 0 | + | N100 | 20 | 20 | + | N101 | 15 | 16 | + | N35 | 22 | 22 | + | N36 | 30 | 30 | + | N37 | 30 | 30 | + | N38 | 28 | 0 | + + Scenario: search and address ranks for boundaries are correctly assigned + Given the named places + | osm | class | type | + | N1 | boundary | administrative | + And the named places + | osm | class | type | geometry | + | W10 | boundary | administrative | 10 10, 11 11 | + And the named places + | osm | class | type | admin | geometry | + | R20 | boundary | administrative | 2 | (1 1, 2 2, 1 2, 1 1) | + | R21 | boundary | administrative | 32 | (3 3, 4 4, 3 4, 3 3) | + | R22 | boundary | nature_park | 6 | (0 0, 1 0, 0 1, 0 0) | + | R23 | boundary | natural_reserve| 10 | (0 0, 1 1, 1 0, 0 0) | + When importing + Then placex has no entry for N1 + And placex has no entry for W10 + And placex contains + | object | rank_search | rank_address | + | R20 | 4 | 4 | + | R21 | 30 | 30 | + | R22 | 12 | 0 | + | R23 | 20 | 0 | + + Scenario: search and address ranks for highways correctly assigned + Given the scene roads-with-pois + And the places + | osm | class | type | + | N1 | highway | bus_stop | + And the places + | osm | class | type | geometry | + | W1 | highway | primary | :w-south | + | W2 | highway | secondary | :w-south | + | W3 | highway | tertiary | :w-south | + | W4 | highway | residential | :w-north | + | W5 | highway | unclassified | :w-north | + | W6 | highway | something | :w-north | + When importing + Then placex contains + | object | rank_search | rank_address | + | N1 | 30 | 30 | + | W1 | 26 | 26 | + | W2 | 26 | 26 | + | W3 | 26 | 26 | + | W4 | 26 | 26 | + | W5 | 26 | 26 | + | W6 | 26 | 26 | + + Scenario: rank and inclusion of landuses + Given the named places + | osm | class | type | + | N2 | landuse | residential | + And the named places + | osm | class | type | geometry | + | W2 | landuse | residential | 1 1, 1 1.1 | + | W4 | landuse | residential | poly-area:0.1 | + | R2 | landuse | residential | poly-area:0.05 | + | R3 | landuse | forrest | poly-area:0.5 | + When importing + Then placex contains + | object | rank_search | rank_address | + | N2 | 30 | 30 | + | W2 | 30 | 30 | + | W4 | 22 | 22 | + | R2 | 22 | 22 | + | R3 | 22 | 0 | + + Scenario: rank and inclusion of naturals + Given the named places + | osm | class | type | + | N2 | natural | peak | + | N4 | natural | volcano | + | N5 | natural | foobar | + And the named places + | osm | class | type | geometry | + | W2 | natural | mountain_range | 12 12,11 11 | + | W3 | natural | foobar | 13 13,13.1 13 | + | R3 | natural | volcano | poly-area:0.1 | + | R4 | natural | foobar | poly-area:0.5 | + | R5 | natural | sea | poly-area:5.0 | + | R6 | natural | sea | poly-area:0.01 | + When importing + Then placex contains + | object | rank_search | rank_address | + | N2 | 18 | 0 | + | N4 | 18 | 0 | + | N5 | 30 | 30 | + | W2 | 18 | 0 | + | R3 | 18 | 0 | + | R4 | 22 | 0 | + | R5 | 4 | 4 | + | R6 | 4 | 4 | + | W3 | 30 | 30 | + diff --git a/test/bdd/db/import/search_name.feature b/test/bdd/db/import/search_name.feature new file mode 100644 index 00000000..98def330 --- /dev/null +++ b/test/bdd/db/import/search_name.feature @@ -0,0 +1,39 @@ +@DB +Feature: Creation of search terms + Tests that search_name table is filled correctly + + Scenario: POIs without a name have no search entry + Given the scene roads-with-pois + And the places + | osm | class | type | geometry | + | N1 | place | house | :p-N1 | + And the named places + | osm | class | type | geometry | + | W1 | highway | residential | :w-north | + When importing + Then search_name has no entry for N1 + + Scenario: Named POIs inherit address from parent + Given the scene roads-with-pois + And the places + | osm | class | type | name | geometry | + | N1 | place | house | foo | :p-N1 | + | W1 | highway | residential | the road | :w-north | + When importing + Then search_name contains + | object | name_vector | nameaddress_vector | + | N1 | foo | the road | + + Scenario: Roads take over the postcode from attached houses + Given the scene roads-with-pois + And the places + | osm | class | type | housenr | postcode | street | geometry | + | N1 | place | house | 1 | 12345 | North St | :p-S1 | + And the places + | osm | class | type | name | geometry | + | W1 | highway | residential | North St | :w-north | + When importing + Then search_name contains + | object | nameaddress_vector | + | W1 | 12345 | + diff --git a/test/bdd/db/query/normalization.feature b/test/bdd/db/query/normalization.feature new file mode 100644 index 00000000..71c69dec --- /dev/null +++ b/test/bdd/db/query/normalization.feature @@ -0,0 +1,139 @@ +@DB +Feature: Import and search of names + Tests all naming related issues: normalisation, + abbreviations, internationalisation, etc. + + Scenario: Case-insensitivity of search + Given the places + | osm | class | type | name | + | N1 | place | locality | FooBar | + When importing + Then placex contains + | object | class | type | name+name | + | N1 | place | locality | FooBar | + When searching for "FooBar" + Then results contain + | ID | osm_type | osm_id | + | 0 | N | 1 | + When searching for "foobar" + Then results contain + | ID | osm_type | osm_id | + | 0 | N | 1 | + When searching for "fOObar" + Then results contain + | ID | osm_type | osm_id | + | 0 | N | 1 | + When searching for "FOOBAR" + Then results contain + | ID | osm_type | osm_id | + | 0 | N | 1 | + + Scenario: Multiple spaces in name + Given the places + | osm | class | type | name | + | N1 | place | locality | one two three | + When importing + When searching for "one two three" + Then results contain + | ID | osm_type | osm_id | + | 0 | N | 1 | + When searching for "one two three" + Then results contain + | ID | osm_type | osm_id | + | 0 | N | 1 | + When searching for "one two three" + Then results contain + | ID | osm_type | osm_id | + | 0 | N | 1 | + When searching for " one two three" + Then results contain + | ID | osm_type | osm_id | + | 0 | N | 1 | + + Scenario: Special characters in name + Given the places + | osm | class | type | name | + | N1 | place | locality | Jim-Knopf-Str | + | N2 | place | locality | Smith/Weston | + | N3 | place | locality | space mountain | + | N4 | place | locality | space | + | N5 | place | locality | mountain | + When importing + When searching for "Jim-Knopf-Str" + Then results contain + | ID | osm_type | osm_id | + | 0 | N | 1 | + When searching for "Jim Knopf-Str" + Then results contain + | ID | osm_type | osm_id | + | 0 | N | 1 | + When searching for "Jim Knopf Str" + Then results contain + | ID | osm_type | osm_id | + | 0 | N | 1 | + When searching for "Jim/Knopf-Str" + Then results contain + | ID | osm_type | osm_id | + | 0 | N | 1 | + When searching for "Jim-Knopfstr" + Then results contain + | ID | osm_type | osm_id | + | 0 | N | 1 | + When searching for "Smith/Weston" + Then results contain + | ID | osm_type | osm_id | + | 0 | N | 2 | + When searching for "Smith Weston" + Then results contain + | ID | osm_type | osm_id | + | 0 | N | 2 | + When searching for "Smith-Weston" + Then results contain + | ID | osm_type | osm_id | + | 0 | N | 2 | + When searching for "space mountain" + Then results contain + | ID | osm_type | osm_id | + | 0 | N | 3 | + When searching for "space-mountain" + Then results contain + | ID | osm_type | osm_id | + | 0 | N | 3 | + When searching for "space/mountain" + Then results contain + | ID | osm_type | osm_id | + | 0 | N | 3 | + When searching for "space\mountain" + Then results contain + | ID | osm_type | osm_id | + | 0 | N | 3 | + When searching for "space(mountain)" + Then results contain + | ID | osm_type | osm_id | + | 0 | N | 3 | + + Scenario: Landuse with name are found + Given the places + | osm | class | type | name | geometry | + | R1 | natural | meadow | landuse1 | (0 0, 1 0, 1 1, 0 1, 0 0) | + | R2 | landuse | industrial | landuse2 | (0 0, -1 0, -1 -1, 0 -1, 0 0) | + When importing + When searching for "landuse1" + Then results contain + | ID | osm_type | osm_id | + | 0 | R | 1 | + When searching for "landuse2" + Then results contain + | ID | osm_type | osm_id | + | 0 | R | 2 | + + @wip + Scenario: Postcode boundaries without ref + Given the places + | osm | class | type | postcode | geometry | + | R1 | boundary | postal_code | 12345 | (0 0, 1 0, 1 1, 0 1, 0 0) | + When importing + When searching for "12345" + Then results contain + | ID | osm_type | osm_id | + | 0 | R | 1 | diff --git a/test/bdd/db/query/search_simple.feature b/test/bdd/db/query/search_simple.feature new file mode 100644 index 00000000..417df769 --- /dev/null +++ b/test/bdd/db/query/search_simple.feature @@ -0,0 +1,32 @@ +@DB +Feature: Searching of simple objects + Testing simple stuff + + Scenario: Search for place node + Given the places + | osm | class | type | name+name | geometry | + | N1 | place | village | Foo | 10.0 -10.0 | + When importing + And searching for "Foo" + Then results contain + | ID | osm | class | type | centroid | + | 0 | N1 | place | village | 10 -10 | + + Scenario: Updating postcode in postcode boundaries without ref + Given the places + | osm | class | type | postcode | geometry | + | R1 | boundary | postal_code | 12345 | poly-area:1.0 | + When importing + And searching for "12345" + Then results contain + | ID | osm_type | osm_id | + | 0 | R | 1 | + When updating places + | osm | class | type | postcode | geometry | + | R1 | boundary | postal_code | 54321 | poly-area:1.0 | + And searching for "12345" + Then exactly 0 results are returned + When searching for "54321" + Then results contain + | ID | osm_type | osm_id | + | 0 | R | 1 | diff --git a/test/bdd/db/update/interpolation.feature b/test/bdd/db/update/interpolation.feature new file mode 100644 index 00000000..7dd5bdc0 --- /dev/null +++ b/test/bdd/db/update/interpolation.feature @@ -0,0 +1,269 @@ +@DB +Feature: Update of address interpolations + Test the interpolated address are updated correctly + + @wip + Scenario: new interpolation added to existing street + Given the scene parallel-road + And the places + | osm | class | type | name | geometry | + | W2 | highway | unclassified | Sun Way | :w-north | + | W3 | highway | unclassified | Cloud Street | :w-south | + And the ways + | id | nodes | + | 10 | 1,100,101,102,2 | + When importing + Then W10 expands to no interpolation + When updating places + | osm | class | type | housenr | geometry | + | N1 | place | house | 2 | :n-middle-w | + | N2 | place | house | 6 | :n-middle-e | + | W10 | place | houses | even | :w-middle | + Then placex contains + | object | parent_place_id | + | N1 | W2 | + | N2 | W2 | + And W10 expands to interpolation + | parent_place_id | start | end | + | W2 | 2 | 6 | + + Scenario: addr:street added to interpolation + Given the scene parallel-road + And the places + | osm | class | type | housenr | geometry | + | N1 | place | house | 2 | :n-middle-w | + | N2 | place | house | 6 | :n-middle-e | + | W10 | place | houses | even | :w-middle | + And the places + | osm | class | type | name | geometry | + | W2 | highway | unclassified | Sun Way | :w-north | + | W3 | highway | unclassified | Cloud Street | :w-south | + And the ways + | id | nodes | + | 10 | 1,100,101,102,2 | + When importing + Then placex contains + | object | parent_place_id | + | N1 | W2 | + | N2 | W2 | + And W10 expands to interpolation + | parent_place_id | start | end | + | W2 | 2 | 6 | + When updating places + | osm | class | type | housenr | street | geometry | + | W10 | place | houses | even | Cloud Street | :w-middle | + Then placex contains + | object | parent_place_id | + | N1 | W3 | + | N2 | W3 | + And W10 expands to interpolation + | parent_place_id | start | end | + | W3 | 2 | 6 | + + Scenario: addr:street added to housenumbers + Given the scene parallel-road + And the places + | osm | class | type | housenr | geometry | + | N1 | place | house | 2 | :n-middle-w | + | N2 | place | house | 6 | :n-middle-e | + | W10 | place | houses | even | :w-middle | + And the places + | osm | class | type | name | geometry | + | W2 | highway | unclassified | Sun Way | :w-north | + | W3 | highway | unclassified | Cloud Street | :w-south | + And the ways + | id | nodes | + | 10 | 1,100,101,102,2 | + When importing + Then placex contains + | object | parent_place_id | + | N1 | W2 | + | N2 | W2 | + And W10 expands to interpolation + | parent_place_id | start | end | + | W2 | 2 | 6 | + When updating places + | osm | class | type | street | housenr | geometry | + | N1 | place | house | Cloud Street| 2 | :n-middle-w | + | N2 | place | house | Cloud Street| 6 | :n-middle-e | + Then placex contains + | object | parent_place_id | + | N1 | W3 | + | N2 | W3 | + And W10 expands to interpolation + | parent_place_id | start | end | + | W3 | 2 | 6 | + + Scenario: interpolation tag removed + Given the scene parallel-road + And the places + | osm | class | type | housenr | geometry | + | N1 | place | house | 2 | :n-middle-w | + | N2 | place | house | 6 | :n-middle-e | + | W10 | place | houses | even | :w-middle | + And the places + | osm | class | type | name | geometry | + | W2 | highway | unclassified | Sun Way | :w-north | + | W3 | highway | unclassified | Cloud Street | :w-south | + And the ways + | id | nodes | + | 10 | 1,100,101,102,2 | + When importing + Then placex contains + | object | parent_place_id | + | N1 | W2 | + | N2 | W2 | + And W10 expands to interpolation + | parent_place_id | start | end | + | W2 | 2 | 6 | + When marking for delete W10 + Then W10 expands to no interpolation + And placex contains + | object | parent_place_id | + | N1 | W2 | + | N2 | W2 | + + Scenario: referenced road added + Given the scene parallel-road + And the places + | osm | class | type | housenr | geometry | + | N1 | place | house | 2 | :n-middle-w | + | N2 | place | house | 6 | :n-middle-e | + And the places + | osm | class | type | housenr | street | geometry | + | W10 | place | houses | even | Cloud Street| :w-middle | + And the places + | osm | class | type | name | geometry | + | W2 | highway | unclassified | Sun Way | :w-north | + And the ways + | id | nodes | + | 10 | 1,100,101,102,2 | + When importing + Then placex contains + | object | parent_place_id | + | N1 | W2 | + | N2 | W2 | + And W10 expands to interpolation + | parent_place_id | start | end | + | W2 | 2 | 6 | + When updating places + | osm | class | type | name | geometry | + | W3 | highway | unclassified | Cloud Street | :w-south | + Then placex contains + | object | parent_place_id | + | N1 | W3 | + | N2 | W3 | + And W10 expands to interpolation + | parent_place_id | start | end | + | W3 | 2 | 6 | + + Scenario: referenced road deleted + Given the scene parallel-road + And the places + | osm | class | type | housenr | geometry | + | N1 | place | house | 2 | :n-middle-w | + | N2 | place | house | 6 | :n-middle-e | + And the places + | osm | class | type | housenr | street | geometry | + | W10 | place | houses | even | Cloud Street| :w-middle | + And the places + | osm | class | type | name | geometry | + | W2 | highway | unclassified | Sun Way | :w-north | + | W3 | highway | unclassified | Cloud Street | :w-south | + And the ways + | id | nodes | + | 10 | 1,100,101,102,2 | + When importing + Then placex contains + | object | parent_place_id | + | N1 | W3 | + | N2 | W3 | + And W10 expands to interpolation + | parent_place_id | start | end | + | W3 | 2 | 6 | + When marking for delete W3 + Then placex contains + | object | parent_place_id | + | N1 | W2 | + | N2 | W2 | + And W10 expands to interpolation + | parent_place_id | start | end | + | W2 | 2 | 6 | + + Scenario: building becomes interpolation + Given the scene building-with-parallel-streets + And the places + | osm | class | type | housenr | geometry | + | W1 | place | house | 3 | :w-building | + And the places + | osm | class | type | name | geometry | + | W2 | highway | unclassified | Cloud Street | :w-south | + When importing + Then placex contains + | object | parent_place_id | + | W1 | W2 | + Given the ways + | id | nodes | + | 1 | 1,100,101,102,2 | + When updating places + | osm | class | type | housenr | geometry | + | N1 | place | house | 2 | :n-north-w | + | N2 | place | house | 6 | :n-north-e | + And updating places + | osm | class | type | housenr | street | geometry | + | W1 | place | houses | even | Cloud Street| :w-north | + Then placex has no entry for W1 + And W1 expands to interpolation + | parent_place_id | start | end | + | W2 | 2 | 6 | + + Scenario: interpolation becomes building + Given the scene building-with-parallel-streets + And the places + | osm | class | type | housenr | geometry | + | N1 | place | house | 2 | :n-north-w | + | N2 | place | house | 6 | :n-north-e | + And the places + | osm | class | type | name | geometry | + | W2 | highway | unclassified | Cloud Street | :w-south | + And the ways + | id | nodes | + | 1 | 1,100,101,102,2 | + And the places + | osm | class | type | housenr | street | geometry | + | W1 | place | houses | even | Cloud Street| :w-north | + When importing + Then placex has no entry for W1 + And W1 expands to interpolation + | parent_place_id | start | end | + | W2 | 2 | 6 | + When updating places + | osm | class | type | housenr | geometry | + | W1 | place | house | 3 | :w-building | + Then placex contains + | object | parent_place_id | + | W1 | W2 | + + Scenario: housenumbers added to interpolation + Given the scene building-with-parallel-streets + And the places + | osm | class | type | name | geometry | + | W2 | highway | unclassified | Cloud Street | :w-south | + And the ways + | id | nodes | + | 1 | 1,100,101,102,2 | + And the places + | osm | class | type | housenr | geometry | + | W1 | place | houses | even | :w-north | + When importing + Then W1 expands to no interpolation + When updating places + | osm | class | type | housenr | geometry | + | N1 | place | house | 2 | :n-north-w | + | N2 | place | house | 6 | :n-north-e | + And updating places + | osm | class | type | housenr | street | geometry | + | W1 | place | houses | even | Cloud Street| :w-north | + Then W1 expands to interpolation + | parent_place_id | start | end | + | W2 | 2 | 6 | diff --git a/test/bdd/db/update/linked_places.feature b/test/bdd/db/update/linked_places.feature new file mode 100644 index 00000000..17ca8003 --- /dev/null +++ b/test/bdd/db/update/linked_places.feature @@ -0,0 +1,91 @@ +@DB +Feature: Updates of linked places + Tests that linked places are correctly added and deleted. + + Scenario: Add linked place when linking relation is renamed + Given the places + | osm | class | type | name | geometry | + | N1 | place | city | foo | 0 0 | + And the places + | osm | class | type | name | admin | geometry | + | R1 | boundary | administrative | foo | 8 | poly-area:0.1 | + When importing + And searching for "foo" with dups + Then results contain + | osm_type | + | R | + When updating places + | osm | class | type | name | admin | geometry | + | R1 | boundary | administrative | foobar | 8 | poly-area:0.1 | + Then placex contains + | object | linked_place_id | + | N1 | - | + When searching for "foo" with dups + Then results contain + | osm_type | + | N | + + Scenario: Add linked place when linking relation is removed + Given the places + | osm | class | type | name | geometry | + | N1 | place | city | foo | 0 0 | + And the places + | osm | class | type | name | admin | geometry | + | R1 | boundary | administrative | foo | 8 | poly-area:0.1 | + When importing + And searching for "foo" with dups + Then results contain + | osm_type | + | R | + When marking for delete R1 + Then placex contains + | object | linked_place_id | + | N1 | - | + When searching for "foo" with dups + Then results contain + | osm_type | + | N | + + Scenario: Remove linked place when linking relation is added + Given the places + | osm | class | type | name | geometry | + | N1 | place | city | foo | 0 0 | + When importing + And searching for "foo" with dups + Then results contain + | osm_type | + | N | + When updating places + | osm | class | type | name | admin | geometry | + | R1 | boundary | administrative | foo | 8 | poly-area:0.1 | + Then placex contains + | object | linked_place_id | + | N1 | R1 | + When searching for "foo" with dups + Then results contain + | osm_type | + | R | + + Scenario: Remove linked place when linking relation is renamed + Given the places + | osm | class | type | name | geometry | + | N1 | place | city | foo | 0 0 | + And the places + | osm | class | type | name | admin | geometry | + | R1 | boundary | administrative | foobar | 8 | poly-area:0.1 | + When importing + And searching for "foo" with dups + Then results contain + | osm_type | + | N | + When updating places + | osm | class | type | name | admin | geometry | + | R1 | boundary | administrative | foo | 8 | poly-area:0.1 | + Then placex contains + | object | linked_place_id | + | N1 | R1 | + When searching for "foo" with dups + Then results contain + | osm_type | + | R | + diff --git a/test/bdd/db/update/naming.feature b/test/bdd/db/update/naming.feature new file mode 100644 index 00000000..4b5222fc --- /dev/null +++ b/test/bdd/db/update/naming.feature @@ -0,0 +1,18 @@ +@DB +Feature: Update of names in place objects + Test all naming related issues in updates + + Scenario: Delete postcode from postcode boundaries without ref + Given the places + | osm | class | type | postcode | geometry | + | R1 | boundary | postal_code | 12345 | poly-area:0.5 | + When importing + And searching for "12345" + Then results contain + | ID | osm_type | osm_id | + | 0 | R | 1 | + When updating places + | osm | class | type | geometry | + | R1 | boundary | postal_code | poly-area:0.5 | + Then placex has no entry for R1 + diff --git a/test/bdd/db/update/poi-inherited-postcode.feature b/test/bdd/db/update/poi-inherited-postcode.feature new file mode 100644 index 00000000..1b2065e6 --- /dev/null +++ b/test/bdd/db/update/poi-inherited-postcode.feature @@ -0,0 +1,57 @@ +@DB +Feature: Update of POI-inherited poscode + Test updates of postcodes on street which was inherited from a related POI + + Background: Street and house with postcode + Given the scene roads-with-pois + And the places + | osm | class | type | housenr | postcode | street | geometry | + | N1 | place | house | 1 | 12345 | North St |:p-S1 | + And the places + | osm | class | type | name | geometry | + | W1 | highway | residential | North St | :w-north | + When importing + Then search_name contains + | object | nameaddress_vector | + | W1 | 12345 | + + Scenario: POI-inherited postcode remains when way type is changed + When updating places + | osm | class | type | name | geometry | + | W1 | highway | unclassified | North St | :w-north | + Then search_name contains + | object | nameaddress_vector | + | W1 | 12345 | + + Scenario: POI-inherited postcode remains when way name is changed + When updating places + | osm | class | type | name | geometry | + | W1 | highway | unclassified | South St | :w-north | + Then search_name contains + | object | nameaddress_vector | + | W1 | 12345 | + + Scenario: POI-inherited postcode remains when way geometry is changed + When updating places + | osm | class | type | name | geometry | + | W1 | highway | unclassified | South St | :w-south | + Then search_name contains + | object | nameaddress_vector | + | W1 | 12345 | + + Scenario: POI-inherited postcode is added when POI postcode changes + When updating places + | osm | class | type | housenr | postcode | street | geometry | + | N1 | place | house | 1 | 54321 | North St |:p-S1 | + Then search_name contains + | object | nameaddress_vector | + | W1 | 54321 | + + Scenario: POI-inherited postcode remains when POI geometry changes + When updating places + | osm | class | type | housenr | postcode | street | geometry | + | N1 | place | house | 1 | 12345 | North St |:p-S2 | + Then search_name contains + | object | nameaddress_vector | + | W1 | 12345 | + diff --git a/test/bdd/db/update/search_terms.feature b/test/bdd/db/update/search_terms.feature new file mode 100644 index 00000000..07dbd451 --- /dev/null +++ b/test/bdd/db/update/search_terms.feature @@ -0,0 +1,21 @@ +@DB +Feature: Update of search terms + Tests that search_name table is updated correctly + + Scenario: POI-inherited postcode remains when another POI is deleted + Given the scene roads-with-pois + And the places + | osm | class | type | housenr | postcode | street | geometry | + | N1 | place | house | 1 | 12345 | North St |:p-S1 | + | N2 | place | house | 2 | | North St |:p-S2 | + And the places + | osm | class | type | name | geometry | + | W1 | highway | residential | North St | :w-north | + When importing + Then search_name contains + | object | nameaddress_vector | + | W1 | 12345 | + When marking for delete N2 + Then search_name contains + | object | nameaddress_vector | + | W1 | 12345 | diff --git a/test/bdd/db/update/simple.feature b/test/bdd/db/update/simple.feature new file mode 100644 index 00000000..0833c90c --- /dev/null +++ b/test/bdd/db/update/simple.feature @@ -0,0 +1,71 @@ +@DB +Feature: Update of simple objects + Testing simple updating functionality + + Scenario: Do delete small boundary features + Given the places + | osm | class | type | admin | geometry | + | R1 | boundary | administrative | 3 | poly-area:1.0 | + When importing + Then placex contains + | object | rank_search | + | R1 | 6 | + When marking for delete R1 + Then placex has no entry for R1 + + Scenario: Do not delete large boundary features + Given the places + | osm | class | type | admin | geometry | + | R1 | boundary | administrative | 3 | poly-area:5.0 | + When importing + Then placex contains + | object | rank_search | + | R1 | 6 | + When marking for delete R1 + Then placex contains + | object | rank_search | + | R1 | 6 | + + Scenario: Do delete large features of low rank + Given the named places + | osm | class | type | geometry | + | W1 | place | house | poly-area:5.0 | + | R1 | boundary | national_park | poly-area:5.0 | + When importing + Then placex contains + | object | rank_address | + | R1 | 0 | + | W1 | 30 | + When marking for delete R1,W1 + Then placex has no entry for W1 + Then placex has no entry for R1 + + Scenario: type mutation + Given the places + | osm | class | type | geometry | + | N3 | shop | toys | 1 -1 | + When importing + Then placex contains + | object | class | type | centroid | + | N3 | shop | toys | 1 -1 | + When updating places + | osm | class | type | geometry | + | N3 | shop | grocery | 1 -1 | + Then placex contains + | object | class | type | centroid | + | N3 | shop | grocery | 1 -1 | + + Scenario: remove postcode place when house number is added + Given the places + | osm | class | type | postcode | geometry | + | N3 | place | postcode | 12345 | 1 -1 | + When importing + Then placex contains + | object | class | type | + | N3 | place | postcode | + When updating places + | osm | class | type | postcode | housenr | geometry | + | N3 | place | house | 12345 | 13 | 1 -1 | + Then placex contains + | object | class | type | + | N3 | place | house | diff --git a/test/bdd/environment.py b/test/bdd/environment.py new file mode 100644 index 00000000..6411d011 --- /dev/null +++ b/test/bdd/environment.py @@ -0,0 +1,225 @@ +from behave import * +import logging +import os +import psycopg2 +import psycopg2.extras +import subprocess +from nose.tools import * # for assert functions +from sys import version_info as python_version + +logger = logging.getLogger(__name__) + +userconfig = { + 'BUILDDIR' : os.path.join(os.path.split(__file__)[0], "../../build"), + 'REMOVE_TEMPLATE' : False, + 'KEEP_TEST_DB' : False, + 'TEMPLATE_DB' : 'test_template_nominatim', + 'TEST_DB' : 'test_nominatim', + 'API_TEST_DB' : 'test_api_nominatim', + 'TEST_SETTINGS_FILE' : '/tmp/nominatim_settings.php' +} + +use_step_matcher("re") + +class NominatimEnvironment(object): + """ Collects all functions for the execution of Nominatim functions. + """ + + def __init__(self, config): + self.build_dir = os.path.abspath(config['BUILDDIR']) + self.template_db = config['TEMPLATE_DB'] + self.test_db = config['TEST_DB'] + self.api_test_db = config['API_TEST_DB'] + self.local_settings_file = config['TEST_SETTINGS_FILE'] + self.reuse_template = not config['REMOVE_TEMPLATE'] + self.keep_scenario_db = config['KEEP_TEST_DB'] + os.environ['NOMINATIM_SETTINGS'] = self.local_settings_file + + self.template_db_done = False + + def write_nominatim_config(self, dbname): + f = open(self.local_settings_file, 'w') + f.write("') + logger.debug("running osm2pgsql for template: %s\n%s\n%s" % (osm2pgsql, outstr, errstr)) + self.run_setup_script('create-functions', 'create-tables', + 'create-partition-tables', 'create-partition-functions', + 'load-data', 'create-search-indices') + + def setup_api_db(self, context): + self.write_nominatim_config(self.api_test_db) + + def setup_db(self, context): + self.setup_template_db() + self.write_nominatim_config(self.test_db) + conn = psycopg2.connect(database=self.template_db) + conn.set_isolation_level(0) + cur = conn.cursor() + cur.execute('DROP DATABASE IF EXISTS %s' % (self.test_db, )) + cur.execute('CREATE DATABASE %s TEMPLATE = %s' % (self.test_db, self.template_db)) + conn.close() + context.db = psycopg2.connect(database=self.test_db) + if python_version[0] < 3: + psycopg2.extras.register_hstore(context.db, globally=False, unicode=True) + else: + psycopg2.extras.register_hstore(context.db, globally=False) + + def teardown_db(self, context): + if 'db' in context: + context.db.close() + + if not self.keep_scenario_db: + self.db_drop_database(self.test_db) + + def run_setup_script(self, *args, **kwargs): + self.run_nominatim_script('setup', *args, **kwargs) + + def run_update_script(self, *args, **kwargs): + self.run_nominatim_script('update', *args, **kwargs) + + def run_nominatim_script(self, script, *args, **kwargs): + cmd = [os.path.join(self.build_dir, 'utils', '%s.php' % script)] + cmd.extend(['--%s' % x for x in args]) + for k, v in kwargs.items(): + cmd.extend(('--' + k.replace('_', '-'), str(v))) + proc = subprocess.Popen(cmd, cwd=self.build_dir, + stdout=subprocess.PIPE, stderr=subprocess.PIPE) + (outp, outerr) = proc.communicate() + logger.debug("run_nominatim_script: %s\n%s\n%s" % (cmd, outp, outerr)) + assert (proc.returncode == 0), "Script '%s' failed:\n%s\n%s\n" % (script, outp, outerr) + + +class OSMDataFactory(object): + + def __init__(self): + scriptpath = os.path.dirname(os.path.abspath(__file__)) + self.scene_path = os.environ.get('SCENE_PATH', + os.path.join(scriptpath, '..', 'scenes', 'data')) + self.scene_cache = {} + + def parse_geometry(self, geom, scene): + if geom.find(':') >= 0: + out = self.get_scene_geometry(scene, geom) + elif geom.find(',') < 0: + out = "'POINT(%s)'::geometry" % geom + elif geom.find('(') < 0: + out = "'LINESTRING(%s)'::geometry" % geom + else: + out = "'POLYGON(%s)'::geometry" % geom + + return "ST_SetSRID(%s, 4326)" % out + + def get_scene_geometry(self, default_scene, name): + geoms = [] + for obj in name.split('+'): + oname = obj.strip() + if oname.startswith(':'): + assert_is_not_none(default_scene, "You need to set a scene") + defscene = self.load_scene(default_scene) + wkt = defscene[oname[1:]] + else: + scene, obj = oname.split(':', 2) + scene_geoms = self.load_scene(scene) + wkt = scene_geoms[obj] + + geoms.append("'%s'::geometry" % wkt) + + if len(geoms) == 1: + return geoms[0] + else: + return 'ST_LineMerge(ST_Collect(ARRAY[%s]))' % ','.join(geoms) + + def load_scene(self, name): + if name in self.scene_cache: + return self.scene_cache[name] + + scene = {} + with open(os.path.join(self.scene_path, "%s.wkt" % name), 'r') as fd: + for line in fd: + if line.strip(): + obj, wkt = line.split('|', 2) + scene[obj.strip()] = wkt.strip() + self.scene_cache[name] = scene + + return scene + + +def before_all(context): + # logging setup + context.config.setup_logging() + # set up -D options + for k,v in userconfig.items(): + context.config.userdata.setdefault(k, v) + logging.debug('User config: %s' %(str(context.config.userdata))) + # Nominatim test setup + context.nominatim = NominatimEnvironment(context.config.userdata) + context.osm = OSMDataFactory() + +def after_all(context): + context.nominatim.cleanup() + + +def before_scenario(context, scenario): + if 'DB' in context.tags: + context.nominatim.setup_db(context) + elif 'APIDB' in context.tags: + context.nominatim.setup_api_db(context) + context.scene = None + +def after_scenario(context, scenario): + if 'DB' in context.tags: + context.nominatim.teardown_db(context) + diff --git a/test/bdd/osm2pgsql/import/broken.feature b/test/bdd/osm2pgsql/import/broken.feature new file mode 100644 index 00000000..aa8d8e35 --- /dev/null +++ b/test/bdd/osm2pgsql/import/broken.feature @@ -0,0 +1,33 @@ +@DB +Feature: Import of objects with broken geometries by osm2pgsql + + Scenario: Import way with double nodes + When loading osm data + """ + n100 x0 y0 + n101 x0 y0.1 + n102 x0.1 y0.2 + w1 Thighway=primary Nn100,n101,n101,n102 + """ + Then place contains + | object | class | type | geometry | + | W1 | highway | primary | 0 0, 0 0.1, 0.1 0.2 | + + @wip + Scenario: Import of ballon areas + When loading osm data + """ + n1 x0 y0 + n2 x0 y0.0001 + n3 x0.00001 y0.0001 + n4 x0.00001 y0 + n5 x-0.00001 y0 + w1 Thighway=unclassified Nn1,n2,n3,n4,n1,n5 + w2 Thighway=unclassified Nn1,n2,n3,n4,n1 + w3 Thighway=unclassified Nn1,n2,n3,n4,n3 + """ + Then place contains + | object | geometrytype | + | W1 | ST_LineString | + | W2 | ST_Polygon | + | W3 | ST_LineString | diff --git a/test/bdd/osm2pgsql/import/relation.feature b/test/bdd/osm2pgsql/import/relation.feature new file mode 100644 index 00000000..7010779e --- /dev/null +++ b/test/bdd/osm2pgsql/import/relation.feature @@ -0,0 +1,11 @@ +@DB +Feature: Import of relations by osm2pgsql + Testing specific relation problems related to members. + + Scenario: Don't import empty waterways + When loading osm data + """ + n1 Tamenity=prison,name=foo + r1 Ttype=waterway,waterway=river,name=XZ Mn1@ + """ + Then place has no entry for R1 diff --git a/test/bdd/osm2pgsql/import/simple.feature b/test/bdd/osm2pgsql/import/simple.feature new file mode 100644 index 00000000..46a18199 --- /dev/null +++ b/test/bdd/osm2pgsql/import/simple.feature @@ -0,0 +1,59 @@ +@DB +Feature: Import of simple objects by osm2pgsql + Testing basic tagging in osm2pgsql imports. + + Scenario: Import simple objects + When loading osm data + """ + n1 Tamenity=prison,name=foo x34.3 y-23 + n100 x0 y0 + n101 x0 y0.1 + n102 x0.1 y0.2 + n200 x0 y0 + n201 x0 y1 + n202 x1 y1 + n203 x1 y0 + w1 Tshop=toys,name=tata Nn100,n101,n102 + w2 Tref=45 Nn200,n201,n202,n203,n200 + r1 Ttype=multipolygon,tourism=hotel,name=XZ Mn1@,w2@ + """ + Then place contains exactly + | object | class | type | name | geometry | + | N1 | amenity | prison | 'name' : 'foo' | 34.3 -23 | + | W1 | shop | toys | 'name' : 'tata' | 0 0, 0 0.1, 0.1 0.2 | + | R1 | tourism | hotel | 'name' : 'XZ' | (0 0, 0 1, 1 1, 1 0, 0 0) | + + Scenario: Import object with two main tags + When loading osm data + """ + n1 Ttourism=hotel,amenity=restaurant,name=foo + """ + Then place contains + | object | type | name | + | N1:tourism | hotel | 'name' : 'foo' | + | N1:amenity | restaurant | 'name' : 'foo' | + + Scenario: Import stand-alone house number with postcode + When loading osm data + """ + n1 Taddr:housenumber=4,addr:postcode=3345 + """ + Then place contains + | object | class | type | + | N1 | place | house | + + Scenario: Landuses are only imported when named + When loading osm data + """ + n100 x0 y0 + n101 x0 y0.1 + n102 x0.1 y0.1 + n200 x0 y0 + n202 x1 y1 + n203 x1 y0 + w1 Tlanduse=residential,name=rainbow Nn100,n101,n102,n100 + w2 Tlanduse=residential Nn200,n202,n203,n200 + """ + Then place contains exactly + | object | class | type | + | W1 | landuse | residential | diff --git a/test/bdd/osm2pgsql/import/tags.feature b/test/bdd/osm2pgsql/import/tags.feature new file mode 100644 index 00000000..d81b6c72 --- /dev/null +++ b/test/bdd/osm2pgsql/import/tags.feature @@ -0,0 +1,554 @@ +@DB +Feature: Tag evaluation + Tests if tags are correctly imported into the place table + + Scenario Outline: Name tags + When loading osm data + """ + n1 Thighway=yes,=Foo + """ + Then place contains + | object | name | + | N1 | '' : 'Foo' | + + Examples: + | nametag | + | ref | + | int_ref | + | nat_ref | + | reg_ref | + | loc_ref | + | old_ref | + | iata | + | icao | + | pcode:1 | + | pcode:2 | + | pcode:3 | + | name | + | name:de | + | name:bt-BR | + | int_name | + | int_name:xxx | + | nat_name | + | nat_name:fr | + | reg_name | + | reg_name:1 | + | loc_name | + | loc_name:DE | + | old_name | + | old_name:v1 | + | alt_name | + | alt_name:dfe | + | alt_name_1 | + | official_name | + | short_name | + | short_name:CH | + | addr:housename | + | brand | + + Scenario: operator only for shops and amenities + When loading osm data + """ + n1 Thighway=yes,operator=Foo,name=null + n2 Tshop=grocery,operator=Foo + n3 Tamenity=hospital,operator=Foo + n4 Ttourism=hotel,operator=Foo + """ + Then place contains + | object | name | + | N1 | 'name' : 'null' | + | N2 | 'operator' : 'Foo' | + | N3 | 'operator' : 'Foo' | + | N4 | 'operator' : 'Foo' | + + Scenario Outline: Ignored name tags + When loading osm data + """ + n1 Thighway=yes,=Foo,name=real + """ + Then place contains + | object | name | + | N1 | 'name' : 'real' | + + Examples: + | nametag | + | name_de | + | Name | + | ref:de | + | ref_de | + | my:ref | + | br:name | + | name:prefix | + | name:source | + + Scenario: Special character in name tag + When loading osm data + """ + n1 Thighway=yes,name:%20%de=Foo,name=real1 + n2 Thighway=yes,name:%a%de=Foo,name=real2 + n3 Thighway=yes,name:%9%de=Foo,name:\\=real3 + n4 Thighway=yes,name:%9%de=Foo,name=rea\l3 + """ + Then place contains + | object | name | + | N1 | 'name: de' : 'Foo', 'name' : 'real1' | + | N2 | 'name: de' : 'Foo', 'name' : 'real2' | + | N3 | 'name: de' : 'Foo', 'name:\\\\' : 'real3' | + | N4 | 'name: de' : 'Foo', 'name' : 'rea\\l3' | + + Scenario Outline: Included places + When loading osm data + """ + n1 T=,name=real + """ + Then place contains + | object | name | + | N1 | 'name' : 'real' | + + Examples: + | key | value | + | emergency | phone | + | tourism | information | + | historic | castle | + | military | barracks | + | natural | water | + | highway | residential | + | aerialway | station | + | aeroway | way | + | boundary | administrative | + | craft | butcher | + | leisure | playground | + | office | bookmaker | + | railway | rail | + | shop | bookshop | + | waterway | stream | + | landuse | cemetry | + | man_made | tower | + | mountain_pass | yes | + + Scenario Outline: Bridges and Tunnels take special name tags + When loading osm data + """ + n1 Thighway=road,=yes,name=Rd,:name=My + n2 Thighway=road,=yes,name=Rd + """ + Then place contains + | object | type | name | + | N1:highway | road | 'name' : 'Rd' | + | N1: | yes | 'name' : 'My' | + | N2:highway | road | 'name' : 'Rd' | + And place has no entry for N2: + + Examples: + | key | + | bridge | + | tunnel | + + Scenario Outline: Excluded places + When loading osm data + """ + n1 T=,name=real + n2 Thighway=motorway,name=To%20%Hell + """ + Then place has no entry for N1 + + Examples: + | key | value | + | emergency | yes | + | emergency | no | + | tourism | yes | + | tourism | no | + | historic | yes | + | historic | no | + | military | yes | + | military | no | + | natural | yes | + | natural | no | + | highway | no | + | highway | turning_circle | + | highway | mini_roundabout | + | highway | noexit | + | highway | crossing | + | aerialway | no | + | aerialway | pylon | + | man_made | survey_point | + | man_made | cutline | + | aeroway | no | + | amenity | no | + | bridge | no | + | craft | no | + | leisure | no | + | office | no | + | railway | no | + | railway | level_crossing | + | shop | no | + | tunnel | no | + | waterway | riverbank | + + Scenario Outline: Some tags only are included when named + When loading osm data + """ + n1 T= + n2 T=,name=To%20%Hell + n3 T=,ref=123 + """ + Then place contains exactly + | object | class | type | + | N2 | | | + + Examples: + | key | value | + | landuse | residential | + | natural | meadow | + | highway | traffic_signals | + | highway | service | + | highway | cycleway | + | highway | path | + | highway | footway | + | highway | steps | + | highway | bridleway | + | highway | track | + | highway | byway | + | highway | motorway_link | + | highway | primary_link | + | highway | trunk_link | + | highway | secondary_link | + | highway | tertiary_link | + | railway | rail | + | boundary | administrative | + | waterway | stream | + + Scenario: Footways are not included if they are sidewalks + When loading osm data + """ + n2 Thighway=footway,name=To%20%Hell,footway=sidewalk + n23 Thighway=footway,name=x + """ + Then place has no entry for N2 + + Scenario: named junctions are included if there is no other tag + When loading osm data + """ + n1 Tjunction=yes + n2 Thighway=secondary,junction=roundabout,name=To-Hell + n3 Tjunction=yes,name=Le%20%Croix + """ + Then place has no entry for N1 + And place has no entry for N2:junction + And place contains + | object | class | type | + | N3 | junction | yes | + + Scenario: Boundary with place tag + When loading osm data + """ + n200 x0 y0 + n201 x0 y1 + n202 x1 y1 + n203 x1 y0 + w2 Tboundary=administrative,place=city,name=Foo Nn200,n201,n202,n203,n200 + w4 Tboundary=administrative,place=island,name=Foo Nn200,n201,n202,n203,n200 + w20 Tplace=city,name=ngng Nn200,n201,n202,n203,n200 + w40 Tplace=city,boundary=statistical,name=BB Nn200,n201,n202,n203,n200 + """ + Then place contains + | object | class | extratags | type | + | W2 | boundary | 'place' : 'city' | administrative | + | W4:boundary | boundary | - | administrative | + | W4:place | place | - | island | + | W20 | place | - | city | + | W40:boundary | boundary | - | statistical | + | W40:place | place | - | city | + And place has no entry for W2:place + + Scenario Outline: Tags that describe a house + When loading osm data + """ + n100 T= + n999 Tamenity=prison,= + """ + Then place contains exactly + | object | class | type | + | N100 | place | house | + | N999 | amenity | prison | + + Examples: + | key | value | + | addr:housename | My%20%Mansion | + | addr:housenumber | 456 | + | addr:conscriptionnumber | 4 | + | addr:streetnumber | 4568765 | + + Scenario: Only named with no other interesting tag + When loading osm data + """ + n1 Tlanduse=meadow + n2 Tlanduse=residential,name=important + n3 Tlanduse=residential,name=important,place=hamlet + """ + Then place contains + | object | class | type | + | N2 | landuse | residential | + | N3 | place | hamlet | + And place has no entry for N1 + And place has no entry for N3:landuse + + Scenario Outline: Import of postal codes + When loading osm data + """ + n10 Thighway=secondary,= + n11 T= + """ + Then place contains + | object | class | type | postcode | + | N10 | highway | secondary | | + | N11 | place | postcode | | + And place has no entry for N10:place + + Examples: + | key | value | + | postal_code | 45736 | + | postcode | xxx | + | addr:postcode | 564 | + | tiger:zip_left | 00011 | + | tiger:zip_right | 09123 | + + Scenario: Import of street and place + When loading osm data + """ + n10 Tamenity=hospital,addr:street=Foo%20%St + n20 Tamenity=hospital,addr:place=Foo%20%Town + """ + Then place contains + | object | class | type | street | addr_place | + | N10 | amenity | hospital | Foo St | None | + | N20 | amenity | hospital | - | Foo Town | + + + Scenario Outline: Import of country + When loading osm data + """ + n10 Tplace=village,= + """ + Then place contains + | object | class | type | country_code | + | N10 | place | village | | + + Examples: + | key | value | + | country_code | us | + | ISO3166-1 | XX | + | is_in:country_code | __ | + | addr:country | .. | + | addr:country_code | cv | + + Scenario Outline: Ignore country codes with wrong length + When loading osm data + """ + n10 Tplace=village,country_code= + """ + Then place contains + | object | class | type | country_code | + | N10 | place | village | - | + + Examples: + | value | + | X | + | x | + | ger | + | dkeufr | + | d%20%e | + + Scenario: Import of house numbers + When loading osm data + """ + n10 Tbuilding=yes,addr:housenumber=4b + n11 Tbuilding=yes,addr:conscriptionnumber=003 + n12 Tbuilding=yes,addr:streetnumber=2345 + n13 Tbuilding=yes,addr:conscriptionnumber=3,addr:streetnumber=111 + """ + Then place contains + | object | class | type | housenumber | + | N10 | building | yes | 4b | + | N11 | building | yes | 003 | + | N12 | building | yes | 2345 | + | N13 | building | yes | 3/111 | + + Scenario: Import of address interpolations + When loading osm data + """ + n10 Taddr:interpolation=odd + n11 Taddr:housenumber=10,addr:interpolation=odd + n12 Taddr:interpolation=odd,addr:housenumber=23 + """ + Then place contains + | object | class | type | housenumber | + | N10 | place | houses | odd | + | N11 | place | houses | odd | + | N12 | place | houses | odd | + + Scenario: Shorten tiger:county tags + When loading osm data + """ + n10 Tplace=village,tiger:county=Feebourgh%2c%%20%AL + n11 Tplace=village,addr:state=Alabama,tiger:county=Feebourgh%2c%%20%AL + n12 Tplace=village,tiger:county=Feebourgh + """ + Then place contains + | object | class | type | isin | + | N10 | place | village | Feebourgh county | + | N11 | place | village | Alabama,Feebourgh county | + | N12 | place | village | Feebourgh county | + + Scenario Outline: Import of address tags + When loading osm data + """ + n10 Tplace=village,= + """ + Then place contains + | object | class | type | isin | + | N10 | place | village | | + + Examples: + | key | value | + | is_in:country | Xanadu | + | addr:suburb | hinein | + | addr:city | Sydney | + | addr:state | Jura | + + Scenario: Import of isin tags with space + When loading osm data + """ + n10 Tplace=village,is_in=Stockholm%2c%%20%Sweden + n11 Tplace=village,addr:county=le%20%havre + """ + Then place contains + | object | class | type | isin | + | N10 | place | village | Stockholm, Sweden | + | N11 | place | village | le havre | + + Scenario: Import of admin level + When loading osm data + """ + n10 Tamenity=hospital,admin_level=3 + n11 Tamenity=hospital,admin_level=b + n12 Tamenity=hospital + n13 Tamenity=hospital,admin_level=3.0 + """ + Then place contains + | object | class | type | admin_level | + | N10 | amenity | hospital | 3 | + | N11 | amenity | hospital | 100 | + | N12 | amenity | hospital | 100 | + | N13 | amenity | hospital | 3 | + + Scenario Outline: Import of extra tags + When loading osm data + """ + n10 Ttourism=hotel,=foo + """ + Then place contains + | object | class | type | extratags | + | N10 | tourism | hotel | '' : 'foo' | + + Examples: + | key | + | tracktype | + | traffic_calming | + | service | + | cuisine | + | capital | + | dispensing | + | religion | + | denomination | + | sport | + | internet_access | + | lanes | + | surface | + | smoothness | + | width | + | est_width | + | incline | + | opening_hours | + | collection_times | + | service_times | + | disused | + | wheelchair | + | sac_scale | + | trail_visibility | + | mtb:scale | + | mtb:description | + | wood | + | drive_in | + | access | + | vehicle | + | bicyle | + | foot | + | goods | + | hgv | + | motor_vehicle | + | motor_car | + | access:foot | + | contact:phone | + | drink:mate | + | oneway | + | date_on | + | date_off | + | day_on | + | day_off | + | hour_on | + | hour_off | + | maxweight | + | maxheight | + | maxspeed | + | disused | + | toll | + | charge | + | population | + | description | + | image | + | attribution | + | fax | + | email | + | url | + | website | + | phone | + | real_ale | + | smoking | + | food | + | camera | + | brewery | + | locality | + | wikipedia | + | wikipedia:de | + | wikidata | + | name:prefix | + | name:botanical | + | name:etymology:wikidata | + + Scenario: buildings + When loading osm data + """ + n10 Ttourism=hotel,building=yes + n11 Tbuilding=house + n12 Tbuilding=shed,addr:housenumber=1 + n13 Tbuilding=yes,name=Das-Haus + n14 Tbuilding=yes,addr:postcode=12345 + """ + Then place contains + | object | class | type | + | N10 | tourism | hotel | + | N12 | building| yes | + | N13 | building| yes | + | N14 | building| yes | + And place has no entry for N10:building + And place has no entry for N11 + + Scenario: complete node entry + When loading osm data + """ + n290393920 Taddr:city=Perpignan,addr:country=FR,addr:housenumber=43\,addr:postcode=66000,addr:street=Rue%20%Pierre%20%Constant%20%d`Ivry,source=cadastre-dgi-fr%20%source%20%:%20%Direction%20%Générale%20%des%20%Impôts%20%-%20%Cadastre%20%;%20%mise%20%à%20%jour%20%:2008 + """ + Then place contains + | object | class | type | housenumber | + | N290393920 | place | house| 43\ | diff --git a/test/bdd/osm2pgsql/update/relation.feature b/test/bdd/osm2pgsql/update/relation.feature new file mode 100644 index 00000000..88889f9a --- /dev/null +++ b/test/bdd/osm2pgsql/update/relation.feature @@ -0,0 +1,126 @@ +@DB +Feature: Update of relations by osm2pgsql + Testing relation update by osm2pgsql. + + Scenario: Remove all members of a relation + When loading osm data + """ + n1 Tamenity=prison,name=foo + n200 x0 y0 + n201 x0 y0.0001 + n202 x0.0001 y0.0001 + n203 x0.0001 y0 + w2 Tref=45' Nn200,n201,n202,n203,n200 + r1 Ttype=multipolygon,tourism=hotel,name=XZ' Mw2@ + """ + Then place contains + | object | class | type | name + | R1 | tourism | hotel | 'name' : 'XZ' + When updating osm data + """ + r1 Ttype=multipolygon,tourism=hotel,name=XZ Mn1@ + """ + Then place has no entry for R1 + + + Scenario: Change type of a relation + When loading osm data + """ + n200 x0 y0 + n201 x0 y0.0001 + n202 x0.0001 y0.0001 + n203 x0.0001 y0 + w2 Tref=45 Nn200,n201,n202,n203,n200 + r1 Ttype=multipolygon,tourism=hotel,name=XZ Mw2@ + """ + Then place contains + | object | class | type | name + | R1 | tourism | hotel | 'name' : 'XZ' + When updating osm data + """ + r1 Ttype=multipolygon,amenity=prison,name=XZ Mw2@ + """ + Then place has no entry for R1:tourism + And place contains + | object | class | type | name + | R1 | amenity | prison | 'name' : 'XZ' + + Scenario: Change name of a relation + When loading osm data + """ + n200 x0 y0 + n201 x0 y0.0001 + n202 x0.0001 y0.0001 + n203 x0.0001 y0 + w2 Tref=45 Nn200,n201,n202,n203,n200 + r1 Ttype=multipolygon,tourism=hotel,name=AB Mw2@ + """ + Then place contains + | object | class | type | name + | R1 | tourism | hotel | 'name' : 'AB' + When updating osm data + """ + r1 Ttype=multipolygon,tourism=hotel,name=XY Mw2@ + """ + Then place contains + | object | class | type | name + | R1 | tourism | hotel | 'name' : 'XZ' + + Scenario: Change type of a relation into something unknown + When loading osm data + """ + n200 x0 y0 + n201 x0 y0.0001 + n202 x0.0001 y0.0001 + n203 x0.0001 y0 + w2 Tref=45 Nn200,n201,n202,n203,n200 + r1 Ttype=multipolygon,tourism=hotel,name=XY Mw2@ + """ + Then place contains + | object | class | type | name + | R1 | tourism | hotel | 'name' : 'XZ' + When updating osm data + """ + r1 Ttype=multipolygon,amenities=prison,name=XY Mw2@ + """ + Then place has no entry for R1 + + Scenario: Type tag is removed + When loading osm data + """ + n200 x0 y0 + n201 x0 y0.0001 + n202 x0.0001 y0.0001 + n203 x0.0001 y0 + w2 Tref=45 Nn200,n201,n202,n203,n200 + r1 Ttype=multipolygon,tourism=hotel,name=XY Mw2@ + """ + Then place contains + | object | class | type | name + | R1 | tourism | hotel | 'name' : 'XZ' + When updating osm data + """ + r1 Ttourism=hotel,name=XY Mw2@ + """ + Then place has no entry for R1 + + @wip + Scenario: Type tag is renamed to something unknown + When loading osm data + """ + n200 x0 y0 + n201 x0 y0.0001 + n202 x0.0001 y0.0001 + n203 x0.0001 y0 + w2 Tref=45 Nn200,n201,n202,n203,n200 + r1 Ttype=multipolygon,tourism=hotel,name=XY Mw2@ + """ + Then place contains + | object | class | type | name + | R1 | tourism | hotel | 'name' : 'XZ' + When updating osm data + """ + r1 Ttype=multipolygonn,tourism=hotel,name=XY Mw2@ + """ + Then place has no entry for R1 + diff --git a/test/bdd/osm2pgsql/update/simple.feature b/test/bdd/osm2pgsql/update/simple.feature new file mode 100644 index 00000000..33c21039 --- /dev/null +++ b/test/bdd/osm2pgsql/update/simple.feature @@ -0,0 +1,26 @@ +@DB +Feature: Update of simple objects by osm2pgsql + Testing basic update functions of osm2pgsql. + + Scenario: Import object with two main tags + When loading osm data + """ + n1 Ttourism=hotel,amenity=restaurant,name=foo + n2 Tplace=locality,name=spotty + """ + Then place contains + | object | type | name + | N1:tourism | hotel | 'name' : 'foo' + | N1:amenity | restaurant | 'name' : 'foo' + | N2:place | locality | 'name' : 'spotty' + When updating osm data + """ + n1 dV Ttourism=hotel,name=foo + n2 dD + """ + Then place has no entry for N1:amenity + And place has no entry for N2 + And place contains + | object | class | type | name + | N1:tourism | tourism | hotel | 'name' : 'foo' + diff --git a/test/bdd/steps/db_ops.py b/test/bdd/steps/db_ops.py new file mode 100644 index 00000000..3c5c5632 --- /dev/null +++ b/test/bdd/steps/db_ops.py @@ -0,0 +1,426 @@ +import base64 +import random +import string +import re +from nose.tools import * # for assert functions +import psycopg2.extras + +class PlaceColumn: + + def __init__(self, context, force_name): + self.columns = { 'admin_level' : 100} + self.force_name = force_name + self.context = context + self.geometry = None + + def add(self, key, value): + if hasattr(self, 'set_key_' + key): + getattr(self, 'set_key_' + key)(value) + elif key.startswith('name+'): + self.add_hstore('name', key[5:], value) + elif key.startswith('extra+'): + self.add_hstore('extratags', key[6:], value) + else: + assert_in(key, ('class', 'type', 'street', 'addr_place', + 'isin', 'postcode')) + self.columns[key] = None if value == '' else value + + def set_key_name(self, value): + self.add_hstore('name', 'name', value) + + def set_key_osm(self, value): + assert_in(value[0], 'NRW') + ok_(value[1:].isdigit()) + + self.columns['osm_type'] = value[0] + self.columns['osm_id'] = int(value[1:]) + + def set_key_admin(self, value): + self.columns['admin_level'] = int(value) + + def set_key_housenr(self, value): + self.columns['housenumber'] = None if value == '' else value + + def set_key_country(self, value): + self.columns['country_code'] = None if value == '' else value + + def set_key_geometry(self, value): + self.geometry = self.context.osm.parse_geometry(value, self.context.scene) + assert_is_not_none(self.geometry) + + def add_hstore(self, column, key, value): + if column in self.columns: + self.columns[column][key] = value + else: + self.columns[column] = { key : value } + + def db_insert(self, cursor): + assert_in('osm_type', self.columns) + if self.force_name and 'name' not in self.columns: + self.add_hstore('name', 'name', ''.join(random.choice(string.printable) + for _ in range(int(random.random()*30)))) + + if self.columns['osm_type'] == 'N' and self.geometry is None: + self.geometry = "ST_SetSRID(ST_Point(%f, %f), 4326)" % ( + random.random()*360 - 180, random.random()*180 - 90) + else: + assert_is_not_none(self.geometry, "Geometry missing") + query = 'INSERT INTO place (%s, geometry) values(%s, %s)' % ( + ','.join(self.columns.keys()), + ','.join(['%s' for x in range(len(self.columns))]), + self.geometry) + cursor.execute(query, list(self.columns.values())) + +class NominatimID: + """ Splits a unique identifier for places into its components. + As place_ids cannot be used for testing, we use a unique + identifier instead that is of the form [:]. + """ + + id_regex = re.compile(r"(?P[NRW])(?P\d+)(:(?P\w+))?") + + def __init__(self, oid): + self.typ = self.oid = self.cls = None + + if oid is not None: + m = self.id_regex.fullmatch(oid) + assert_is_not_none(m, "ID '%s' not of form [:]" % oid) + + self.typ = m.group('tp') + self.oid = m.group('id') + self.cls = m.group('cls') + + def table_select(self): + """ Return where clause and parameter list to select the object + from a Nominatim table. + """ + where = 'osm_type = %s and osm_id = %s' + params = [self.typ, self. oid] + + if self.cls is not None: + where += ' and class = %s' + params.append(self.cls) + + return where, params + + def get_place_id(self, cur): + where, params = self.table_select() + cur.execute("SELECT place_id FROM placex WHERE %s" % where, params) + eq_(1, cur.rowcount, "Expected exactly 1 entry in placex found %s" % cur.rowcount) + + return cur.fetchone()[0] + + +def assert_db_column(row, column, value, context): + if column == 'object': + return + + if column.startswith('centroid'): + fac = float(column[9:]) if column.startswith('centroid*') else 1.0 + x, y = value.split(' ') + assert_almost_equal(float(x) * fac, row['cx'], "Bad x coordinate") + assert_almost_equal(float(y) * fac, row['cy'], "Bad y coordinate") + elif column == 'geometry': + geom = context.osm.parse_geometry(value, context.scene) + cur = context.db.cursor() + query = "SELECT ST_Equals(ST_SnapToGrid(%s, 0.00001, 0.00001), ST_SnapToGrid(ST_SetSRID('%s'::geometry, 4326), 0.00001, 0.00001))" % ( + geom, row['geomtxt'],) + cur.execute(query) + eq_(cur.fetchone()[0], True, "(Row %s failed: %s)" % (column, query)) + elif value == '-': + assert_is_none(row[column], "Row %s" % column) + else: + eq_(value, str(row[column]), + "Row '%s': expected: %s, got: %s" + % (column, value, str(row[column]))) + + +################################ STEPS ################################## + +@given(u'the scene (?P.+)') +def set_default_scene(context, scene): + context.scene = scene + +@given("the (?Pnamed )?places") +def add_data_to_place_table(context, named): + cur = context.db.cursor() + cur.execute('ALTER TABLE place DISABLE TRIGGER place_before_insert') + for r in context.table: + col = PlaceColumn(context, named is not None) + + for h in r.headings: + col.add(h, r[h]) + + col.db_insert(cur) + cur.execute('ALTER TABLE place ENABLE TRIGGER place_before_insert') + cur.close() + context.db.commit() + +@given("the relations") +def add_data_to_planet_relations(context): + cur = context.db.cursor() + for r in context.table: + last_node = 0 + last_way = 0 + parts = [] + if r['members']: + members = [] + for m in r['members'].split(','): + mid = NominatimID(m) + if mid.typ == 'N': + parts.insert(last_node, int(mid.oid)) + last_node += 1 + last_way += 1 + elif mid.typ == 'W': + parts.insert(last_way, int(mid.oid)) + last_way += 1 + else: + parts.append(int(mid.oid)) + + members.extend((mid.typ.lower() + mid.oid, mid.cls or '')) + else: + members = None + + tags = [] + for h in r.headings: + if h.startswith("tags+"): + tags.extend((h[5:], r[h])) + + cur.execute("""INSERT INTO planet_osm_rels (id, way_off, rel_off, parts, members, tags) + VALUES (%s, %s, %s, %s, %s, %s)""", + (r['id'], last_node, last_way, parts, members, tags)) + context.db.commit() + +@given("the ways") +def add_data_to_planet_ways(context): + cur = context.db.cursor() + for r in context.table: + tags = [] + for h in r.headings: + if h.startswith("tags+"): + tags.extend((h[5:], r[h])) + + nodes = [ int(x.strip()) for x in r['nodes'].split(',') ] + + cur.execute("INSERT INTO planet_osm_ways (id, nodes, tags) VALUES (%s, %s, %s)", + (r['id'], nodes, tags)) + context.db.commit() + +@when("importing") +def import_and_index_data_from_place_table(context): + context.nominatim.run_setup_script('create-functions', 'create-partition-functions') + cur = context.db.cursor() + cur.execute( + """insert into placex (osm_type, osm_id, class, type, name, admin_level, + housenumber, street, addr_place, isin, postcode, country_code, extratags, + geometry) + select * from place where not (class='place' and type='houses' and osm_type='W')""") + cur.execute( + """select insert_osmline (osm_id, housenumber, street, addr_place, + postcode, country_code, geometry) + from place where class='place' and type='houses' and osm_type='W'""") + context.db.commit() + context.nominatim.run_setup_script('index', 'index-noanalyse') + +@when("updating places") +def update_place_table(context): + context.nominatim.run_setup_script( + 'create-functions', 'create-partition-functions', 'enable-diff-updates') + cur = context.db.cursor() + for r in context.table: + col = PlaceColumn(context, False) + + for h in r.headings: + col.add(h, r[h]) + + col.db_insert(cur) + + context.db.commit() + context.nominatim.run_update_script('index') + +@when("marking for delete (?P.*)") +def delete_places(context, oids): + context.nominatim.run_setup_script( + 'create-functions', 'create-partition-functions', 'enable-diff-updates') + cur = context.db.cursor() + for oid in oids.split(','): + where, params = NominatimID(oid).table_select() + cur.execute("DELETE FROM place WHERE " + where, params) + context.db.commit() + context.nominatim.run_update_script('index') + +@then("placex contains(?P exactly)?") +def check_placex_contents(context, exact): + cur = context.db.cursor(cursor_factory=psycopg2.extras.DictCursor) + + expected_content = set() + for row in context.table: + nid = NominatimID(row['object']) + where, params = nid.table_select() + cur.execute("""SELECT *, ST_AsText(geometry) as geomtxt, + ST_X(centroid) as cx, ST_Y(centroid) as cy + FROM placex where %s""" % where, + params) + assert_less(0, cur.rowcount, "No rows found for " + row['object']) + + for res in cur: + if exact: + expected_content.add((res['osm_type'], res['osm_id'], res['class'])) + for h in row.headings: + if h.startswith('name'): + name = h[5:] if h.startswith('name+') else 'name' + assert_in(name, res['name']) + eq_(res['name'][name], row[h]) + elif h.startswith('extratags+'): + eq_(res['extratags'][h[10:]], row[h]) + elif h in ('linked_place_id', 'parent_place_id'): + if row[h] == '0': + eq_(0, res[h]) + elif row[h] == '-': + assert_is_none(res[h]) + else: + eq_(NominatimID(row[h]).get_place_id(context.db.cursor()), + res[h]) + else: + assert_db_column(res, h, row[h], context) + + if exact: + cur.execute('SELECT osm_type, osm_id, class from placex') + eq_(expected_content, set([(r[0], r[1], r[2]) for r in cur])) + + context.db.commit() + +@then("place contains(?P exactly)?") +def check_placex_contents(context, exact): + cur = context.db.cursor(cursor_factory=psycopg2.extras.DictCursor) + + expected_content = set() + for row in context.table: + nid = NominatimID(row['object']) + where, params = nid.table_select() + cur.execute("""SELECT *, ST_AsText(geometry) as geomtxt, + ST_GeometryType(geometry) as geometrytype + FROM place where %s""" % where, + params) + assert_less(0, cur.rowcount, "No rows found for " + row['object']) + + for res in cur: + if exact: + expected_content.add((res['osm_type'], res['osm_id'], res['class'])) + for h in row.headings: + msg = "%s: %s" % (row['object'], h) + if h in ('name', 'extratags'): + if row[h] == '-': + assert_is_none(res[h], msg) + else: + vdict = eval('{' + row[h] + '}') + assert_equals(vdict, res[h], msg) + elif h.startswith('name+'): + assert_equals(res['name'][h[5:]], row[h], msg) + elif h.startswith('extratags+'): + assert_equals(res['extratags'][h[10:]], row[h], msg) + elif h in ('linked_place_id', 'parent_place_id'): + if row[h] == '0': + assert_equals(0, res[h], msg) + elif row[h] == '-': + assert_is_none(res[h], msg) + else: + assert_equals(NominatimID(row[h]).get_place_id(context.db.cursor()), + res[h], msg) + else: + assert_db_column(res, h, row[h], context) + + if exact: + cur.execute('SELECT osm_type, osm_id, class from place') + eq_(expected_content, set([(r[0], r[1], r[2]) for r in cur])) + + context.db.commit() + +@then("search_name contains") +def check_search_name_contents(context): + cur = context.db.cursor(cursor_factory=psycopg2.extras.DictCursor) + + for row in context.table: + pid = NominatimID(row['object']).get_place_id(cur) + cur.execute("""SELECT *, ST_X(centroid) as cx, ST_Y(centroid) as cy + FROM search_name WHERE place_id = %s""", (pid, )) + assert_less(0, cur.rowcount, "No rows found for " + row['object']) + + for res in cur: + for h in row.headings: + if h in ('name_vector', 'nameaddress_vector'): + terms = [x.strip().replace('#', ' ') for x in row[h].split(',')] + subcur = context.db.cursor() + subcur.execute("""SELECT word_id, word_token + FROM word, (SELECT unnest(%s) as term) t + WHERE word_token = make_standard_name(t.term)""", + (terms,)) + ok_(subcur.rowcount >= len(terms)) + for wid in subcur: + assert_in(wid[0], res[h], + "Missing term for %s/%s: %s" % (pid, h, wid[1])) + else: + assert_db_column(res, h, row[h], context) + + + context.db.commit() + +@then("(?P\w+) expands to(?P no)? interpolation") +def check_location_property_osmline(context, oid, neg): + cur = context.db.cursor(cursor_factory=psycopg2.extras.DictCursor) + nid = NominatimID(oid) + + eq_('W', nid.typ, "interpolation must be a way") + + cur.execute("""SELECT *, ST_AsText(linegeo) as geomtxt + FROM location_property_osmline WHERE osm_id = %s""", + (nid.oid, )) + + if neg: + eq_(0, cur.rowcount) + return + + todo = list(range(len(list(context.table)))) + for res in cur: + for i in todo: + row = context.table[i] + if (int(row['start']) == res['startnumber'] + and int(row['end']) == res['endnumber']): + todo.remove(i) + break + else: + assert False, "Unexpected row %s" % (str(res)) + + for h in row.headings: + if h in ('start', 'end'): + continue + elif h == 'parent_place_id': + if row[h] == '0': + eq_(0, res[h]) + elif row[h] == '-': + assert_is_none(res[h]) + else: + eq_(NominatimID(row[h]).get_place_id(context.db.cursor()), + res[h]) + else: + assert_db_column(res, h, row[h], context) + + eq_(todo, []) + + +@then("(?Pplacex|place) has no entry for (?P.*)") +def check_placex_has_entry(context, table, oid): + cur = context.db.cursor(cursor_factory=psycopg2.extras.DictCursor) + nid = NominatimID(oid) + where, params = nid.table_select() + cur.execute("SELECT * FROM %s where %s" % (table, where), params) + eq_(0, cur.rowcount) + context.db.commit() + +@then("search_name has no entry for (?P.*)") +def check_search_name_has_entry(context, oid): + cur = context.db.cursor(cursor_factory=psycopg2.extras.DictCursor) + pid = NominatimID(oid).get_place_id(cur) + cur.execute("SELECT * FROM search_name WHERE place_id = %s", (pid, )) + eq_(0, cur.rowcount) + context.db.commit() diff --git a/test/bdd/steps/osm_data.py b/test/bdd/steps/osm_data.py new file mode 100644 index 00000000..926fb9ab --- /dev/null +++ b/test/bdd/steps/osm_data.py @@ -0,0 +1,58 @@ +import subprocess +import tempfile +import random +import os +from nose.tools import * # for assert functions + +@when(u'loading osm data') +def load_osm_file(context): + + # create a OSM file in /tmp and import it + with tempfile.NamedTemporaryFile(dir='/tmp', suffix='.opl', delete=False) as fd: + fname = fd.name + for line in context.text.splitlines(): + if line.startswith('n') and line.find(' x') < 0: + line += " x%d y%d" % (random.random() * 360 - 180, + random.random() * 180 - 90) + fd.write(line.encode('utf-8')) + fd.write(b'\n') + + context.nominatim.run_setup_script('import-data', osm_file=fname, + osm2pgsql_cache=300) + + ### reintroduce the triggers/indexes we've lost by having osm2pgsql set up place again + cur = context.db.cursor() + cur.execute("""CREATE TRIGGER place_before_delete BEFORE DELETE ON place + FOR EACH ROW EXECUTE PROCEDURE place_delete()""") + cur.execute("""CREATE TRIGGER place_before_insert BEFORE INSERT ON place + FOR EACH ROW EXECUTE PROCEDURE place_insert()""") + cur.execute("""CREATE UNIQUE INDEX idx_place_osm_unique on place using btree(osm_id,osm_type,class,type)""") + context.db.commit() + + os.remove(fname) + +@when(u'updating osm data') +def update_from_osm_file(context): + context.nominatim.run_setup_script('create-functions', 'create-partition-functions') + + cur = context.db.cursor() + cur.execute("""insert into placex (osm_type, osm_id, class, type, name, + admin_level, housenumber, street, addr_place, isin, postcode, + country_code, extratags, geometry) select * from place""") + context.db.commit() + context.nominatim.run_setup_script('index', 'index-noanalyse') + context.nominatim.run_setup_script('create-functions', 'create-partition-functions', + 'enable-diff-updates') + + # create a OSM file in /tmp and import it + with tempfile.NamedTemporaryFile(dir='/tmp', suffix='.opl', delete=False) as fd: + fname = fd.name + for line in context.text.splitlines(): + if line.startswith('n') and line.find(' x') < 0: + line += " x%d y%d" % (random.random() * 360 - 180, + random.random() * 180 - 90) + fd.write(line.encode('utf-8')) + fd.write(b'\n') + + context.nominatim.run_update_script(import_diff=fname) + os.remove(fname) diff --git a/test/bdd/steps/queries.py b/test/bdd/steps/queries.py new file mode 100644 index 00000000..c429f082 --- /dev/null +++ b/test/bdd/steps/queries.py @@ -0,0 +1,514 @@ +""" Steps that run search queries. + + Queries may either be run directly via PHP using the query script + or via the HTTP interface. +""" + +import json +import os +import io +import re +from tidylib import tidy_document +import xml.etree.ElementTree as ET +import subprocess +from urllib.parse import urlencode +from collections import OrderedDict +from nose.tools import * # for assert functions + +BASE_SERVER_ENV = { + 'HTTP_HOST' : 'localhost', + 'HTTP_USER_AGENT' : 'Mozilla/5.0 (X11; Linux x86_64; rv:51.0) Gecko/20100101 Firefox/51.0', + 'HTTP_ACCEPT' : 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8', + 'HTTP_ACCEPT_ENCODING' : 'gzip, deflate', + 'HTTP_CONNECTION' : 'keep-alive', + 'SERVER_SIGNATURE' : '
Nominatim BDD Tests
', + 'SERVER_SOFTWARE' : 'Nominatim test', + 'SERVER_NAME' : 'localhost', + 'SERVER_ADDR' : '127.0.1.1', + 'SERVER_PORT' : '80', + 'REMOTE_ADDR' : '127.0.0.1', + 'DOCUMENT_ROOT' : '/var/www', + 'REQUEST_SCHEME' : 'http', + 'CONTEXT_PREFIX' : '/', + 'SERVER_ADMIN' : 'webmaster@localhost', + 'REMOTE_PORT' : '49319', + 'GATEWAY_INTERFACE' : 'CGI/1.1', + 'SERVER_PROTOCOL' : 'HTTP/1.1', + 'REQUEST_METHOD' : 'GET', + 'REDIRECT_STATUS' : 'CGI' +} + + +def compare(operator, op1, op2): + if operator == 'less than': + return op1 < op2 + elif operator == 'more than': + return op1 > op2 + elif operator == 'exactly': + return op1 == op2 + elif operator == 'at least': + return op1 >= op2 + elif operator == 'at most': + return op1 <= op2 + else: + raise Exception("unknown operator '%s'" % operator) + +class GenericResponse(object): + + def match_row(self, row): + if 'ID' in row.headings: + todo = [int(row['ID'])] + else: + todo = range(len(self.result)) + + for i in todo: + res = self.result[i] + for h in row.headings: + if h == 'ID': + pass + elif h == 'osm': + assert_equal(res['osm_type'], row[h][0]) + assert_equal(res['osm_id'], row[h][1:]) + elif h == 'centroid': + x, y = row[h].split(' ') + assert_almost_equal(float(y), float(res['lat'])) + assert_almost_equal(float(x), float(res['lon'])) + elif row[h].startswith("^"): + assert_in(h, res) + assert_is_not_none(re.fullmatch(row[h], res[h]), + "attribute '%s': expected: '%s', got '%s'" + % (h, row[h], res[h])) + else: + assert_in(h, res) + assert_equal(str(res[h]), str(row[h])) + + def property_list(self, prop): + return [ x[prop] for x in self.result ] + + +class SearchResponse(GenericResponse): + + def __init__(self, page, fmt='json', errorcode=200): + self.page = page + self.format = fmt + self.errorcode = errorcode + self.result = [] + self.header = dict() + + if errorcode == 200: + getattr(self, 'parse_' + fmt)() + + def parse_json(self): + m = re.fullmatch(r'([\w$][^(]*)\((.*)\)', self.page) + if m is None: + code = self.page + else: + code = m.group(2) + self.header['json_func'] = m.group(1) + self.result = json.JSONDecoder(object_pairs_hook=OrderedDict).decode(code) + + def parse_html(self): + content, errors = tidy_document(self.page, + options={'char-encoding' : 'utf8'}) + #eq_(len(errors), 0 , "Errors found in HTML document:\n%s" % errors) + + b = content.find('nominatim_results =') + e = content.find('') + content = content[b:e] + b = content.find('[') + e = content.rfind(']') + + self.result = json.JSONDecoder(object_pairs_hook=OrderedDict).decode(content[b:e+1]) + + def parse_xml(self): + et = ET.fromstring(self.page) + + self.header = dict(et.attrib) + + for child in et: + assert_equal(child.tag, "place") + self.result.append(dict(child.attrib)) + + address = {} + for sub in child: + if sub.tag == 'extratags': + self.result[-1]['extratags'] = {} + for tag in sub: + self.result[-1]['extratags'][tag.attrib['key']] = tag.attrib['value'] + elif sub.tag == 'namedetails': + self.result[-1]['namedetails'] = {} + for tag in sub: + self.result[-1]['namedetails'][tag.attrib['desc']] = tag.text + elif sub.tag in ('geokml'): + self.result[-1][sub.tag] = True + else: + address[sub.tag] = sub.text + + if len(address) > 0: + self.result[-1]['address'] = address + + +class ReverseResponse(GenericResponse): + + def __init__(self, page, fmt='json', errorcode=200): + self.page = page + self.format = fmt + self.errorcode = errorcode + self.result = [] + self.header = dict() + + if errorcode == 200: + getattr(self, 'parse_' + fmt)() + + def parse_html(self): + content, errors = tidy_document(self.page, + options={'char-encoding' : 'utf8'}) + #eq_(len(errors), 0 , "Errors found in HTML document:\n%s" % errors) + + b = content.find('nominatim_results =') + e = content.find('') + content = content[b:e] + b = content.find('[') + e = content.rfind(']') + + self.result = json.JSONDecoder(object_pairs_hook=OrderedDict).decode(content[b:e+1]) + + def parse_json(self): + m = re.fullmatch(r'([\w$][^(]*)\((.*)\)', self.page) + if m is None: + code = self.page + else: + code = m.group(2) + self.header['json_func'] = m.group(1) + self.result = [json.JSONDecoder(object_pairs_hook=OrderedDict).decode(code)] + + def parse_xml(self): + et = ET.fromstring(self.page) + + self.header = dict(et.attrib) + self.result = [] + + for child in et: + if child.tag == 'result': + eq_(0, len(self.result), "More than one result in reverse result") + self.result.append(dict(child.attrib)) + elif child.tag == 'addressparts': + address = {} + for sub in child: + address[sub.tag] = sub.text + self.result[0]['address'] = address + elif child.tag == 'extratags': + self.result[0]['extratags'] = {} + for tag in child: + self.result[0]['extratags'][tag.attrib['key']] = tag.attrib['value'] + elif child.tag == 'namedetails': + self.result[0]['namedetails'] = {} + for tag in child: + self.result[0]['namedetails'][tag.attrib['desc']] = tag.text + elif child.tag in ('geokml'): + self.result[0][child.tag] = True + else: + assert child.tag == 'error', \ + "Unknown XML tag %s on page: %s" % (child.tag, self.page) + + +class DetailsResponse(GenericResponse): + + def __init__(self, page, fmt='json', errorcode=200): + self.page = page + self.format = fmt + self.errorcode = errorcode + self.result = [] + self.header = dict() + + if errorcode == 200: + getattr(self, 'parse_' + fmt)() + + def parse_html(self): + content, errors = tidy_document(self.page, + options={'char-encoding' : 'utf8'}) + self.result = {} + +@when(u'searching for "(?P.*)"(?P with dups)?') +def query_cmd(context, query, dups): + """ Query directly via PHP script. + """ + cmd = [os.path.join(context.nominatim.build_dir, 'utils', 'query.php'), + '--search', query] + # add more parameters in table form + if context.table: + for h in context.table.headings: + value = context.table[0][h].strip() + if value: + cmd.extend(('--' + h, value)) + + if dups: + cmd.extend(('--dedupe', '0')) + + proc = subprocess.Popen(cmd, cwd=context.nominatim.build_dir, + stdout=subprocess.PIPE, stderr=subprocess.PIPE) + (outp, err) = proc.communicate() + + assert_equals (0, proc.returncode, "query.php failed with message: %s\noutput: %s" % (err, outp)) + + context.response = SearchResponse(outp.decode('utf-8'), 'json') + +def send_api_query(endpoint, params, fmt, context): + if fmt is not None: + params['format'] = fmt.strip() + if context.table: + if context.table.headings[0] == 'param': + for line in context.table: + params[line['param']] = line['value'] + else: + for h in context.table.headings: + params[h] = context.table[0][h] + + env = dict(BASE_SERVER_ENV) + env['QUERY_STRING'] = urlencode(params) + + env['SCRIPT_NAME'] = '/%s.php' % endpoint + env['REQUEST_URI'] = '%s?%s' % (env['SCRIPT_NAME'], env['QUERY_STRING']) + env['CONTEXT_DOCUMENT_ROOT'] = os.path.join(context.nominatim.build_dir, 'website') + env['SCRIPT_FILENAME'] = os.path.join(env['CONTEXT_DOCUMENT_ROOT'], + '%s.php' % endpoint) + env['NOMINATIM_SETTINGS'] = context.nominatim.local_settings_file + + if hasattr(context, 'http_headers'): + env.update(context.http_headers) + + cmd = ['/usr/bin/php-cgi', env['SCRIPT_FILENAME']] + for k,v in params.items(): + cmd.append("%s=%s" % (k, v)) + + proc = subprocess.Popen(cmd, cwd=context.nominatim.build_dir, env=env, + stdout=subprocess.PIPE, stderr=subprocess.PIPE) + + (outp, err) = proc.communicate() + + assert_equals(0, proc.returncode, + "query.php failed with message: %s\noutput: %s" % (err, outp)) + + assert_equals(0, len(err), "Unexpected PHP error: %s" % (err)) + + outp = outp.decode('utf-8') + + if outp.startswith('Status: '): + status = int(outp[8:11]) + else: + status = 200 + + content_start = outp.find('\r\n\r\n') + + return outp[content_start + 4:], status + +@given(u'the HTTP header') +def add_http_header(context): + if not hasattr(context, 'http_headers'): + context.http_headers = {} + + for h in context.table.headings: + envvar = 'HTTP_' + h.upper().replace('-', '_') + context.http_headers[envvar] = context.table[0][h] + + +@when(u'sending (?P\S+ )?search query "(?P.*)"(?P with address)?') +def website_search_request(context, fmt, query, addr): + params = {} + if query: + params['q'] = query + if addr is not None: + params['addressdetails'] = '1' + + outp, status = send_api_query('search', params, fmt, context) + + if fmt is None: + outfmt = 'html' + elif fmt == 'jsonv2 ': + outfmt = 'json' + else: + outfmt = fmt.strip() + + context.response = SearchResponse(outp, outfmt, status) + +@when(u'sending (?P\S+ )?reverse coordinates (?P[0-9.-]+)?,(?P[0-9.-]+)?') +def website_reverse_request(context, fmt, lat, lon): + params = {} + if lat is not None: + params['lat'] = lat + if lon is not None: + params['lon'] = lon + + outp, status = send_api_query('reverse', params, fmt, context) + + if fmt is None: + outfmt = 'xml' + elif fmt == 'jsonv2 ': + outfmt = 'json' + else: + outfmt = fmt.strip() + + context.response = ReverseResponse(outp, outfmt, status) + +@when(u'sending (?P\S+ )?details query for (?P.*)') +def website_details_request(context, fmt, query): + params = {} + if query[0] in 'NWR': + params['osmtype'] = query[0] + params['osmid'] = query[1:] + else: + params['place_id'] = query + outp, status = send_api_query('details', params, fmt, context) + + context.response = DetailsResponse(outp, 'html', status) + +@when(u'sending (?P\S+ )?lookup query for (?P.*)') +def website_lookup_request(context, fmt, query): + params = { 'osm_ids' : query } + outp, status = send_api_query('lookup', params, fmt, context) + + if fmt == 'json ': + outfmt = 'json' + else: + outfmt = 'xml' + + context.response = SearchResponse(outp, outfmt, status) + + +@step(u'(?Pless than|more than|exactly|at least|at most) (?P\d+) results? (?:is|are) returned') +def validate_result_number(context, operator, number): + eq_(context.response.errorcode, 200) + numres = len(context.response.result) + ok_(compare(operator, numres, int(number)), + "Bad number of results: expected %s %s, got %d." % (operator, number, numres)) + +@then(u'a HTTP (?P\d+) is returned') +def check_http_return_status(context, status): + eq_(context.response.errorcode, int(status)) + +@then(u'the result is valid (?P\w+)') +def step_impl(context, fmt): + context.execute_steps("Then a HTTP 200 is returned") + eq_(context.response.format, fmt) + +@then(u'result header contains') +def check_header_attr(context): + for line in context.table: + assert_is_not_none(re.fullmatch(line['value'], context.response.header[line['attr']]), + "attribute '%s': expected: '%s', got '%s'" + % (line['attr'], line['value'], + context.response.header[line['attr']])) + +@then(u'result header has (?Pnot )?attributes (?P.*)') +def check_header_no_attr(context, neg, attrs): + for attr in attrs.split(','): + if neg: + assert_not_in(attr, context.response.header) + else: + assert_in(attr, context.response.header) + +@then(u'results contain') +def step_impl(context): + context.execute_steps("then at least 1 result is returned") + + for line in context.table: + context.response.match_row(line) + +@then(u'result (?P\d+ )?has (?Pnot )?attributes (?P.*)') +def validate_attributes(context, lid, neg, attrs): + if lid is None: + idx = range(len(context.response.result)) + context.execute_steps("then at least 1 result is returned") + else: + idx = [int(lid.strip())] + context.execute_steps("then more than %sresults are returned" % lid) + + for i in idx: + for attr in attrs.split(','): + if neg: + assert_not_in(attr, context.response.result[i]) + else: + assert_in(attr, context.response.result[i]) + +@then(u'result addresses contain') +def step_impl(context): + context.execute_steps("then at least 1 result is returned") + + if 'ID' not in context.table.headings: + addr_parts = context.response.property_list('address') + + for line in context.table: + if 'ID' in context.table.headings: + addr_parts = [dict(context.response.result[int(line['ID'])]['address'])] + + for h in context.table.headings: + if h != 'ID': + for p in addr_parts: + assert_in(h, p) + assert_equal(p[h], line[h], "Bad address value for %s" % h) + +@then(u'address of result (?P\d+) has(?P no)? types (?P.*)') +def check_address(context, lid, neg, attrs): + context.execute_steps("then more than %s results are returned" % lid) + + addr_parts = context.response.result[int(lid)]['address'] + + for attr in attrs.split(','): + if neg: + assert_not_in(attr, addr_parts) + else: + assert_in(attr, addr_parts) + +@then(u'address of result (?P\d+) is') +def check_address(context, lid): + context.execute_steps("then more than %s results are returned" % lid) + + addr_parts = dict(context.response.result[int(lid)]['address']) + + for line in context.table: + assert_in(line['type'], addr_parts) + assert_equal(addr_parts[line['type']], line['value'], + "Bad address value for %s" % line['type']) + del addr_parts[line['type']] + + eq_(0, len(addr_parts), "Additional address parts found: %s" % str(addr_parts)) + +@then(u'result (?P\d+ )?has bounding box in (?P[\d,.-]+)') +def step_impl(context, lid, coords): + if lid is None: + context.execute_steps("then at least 1 result is returned") + bboxes = context.response.property_list('boundingbox') + else: + context.execute_steps("then more than %sresults are returned" % lid) + bboxes = [ context.response.result[int(lid)]['boundingbox']] + + coord = [ float(x) for x in coords.split(',') ] + + for bbox in bboxes: + if isinstance(bbox, str): + bbox = bbox.split(',') + bbox = [ float(x) for x in bbox ] + + assert_greater_equal(bbox[0], coord[0]) + assert_less_equal(bbox[1], coord[1]) + assert_greater_equal(bbox[2], coord[2]) + assert_less_equal(bbox[3], coord[3]) + +@then(u'there are(?P no)? duplicates') +def check_for_duplicates(context, neg): + context.execute_steps("then at least 1 result is returned") + + resarr = set() + has_dupe = False + + for res in context.response.result: + dup = (res['osm_type'], res['class'], res['type'], res['display_name']) + if dup in resarr: + has_dupe = True + break + resarr.add(dup) + + if neg: + assert not has_dupe, "Found duplicate for %s" % (dup, ) + else: + assert has_dupe, "No duplicates found" diff --git a/tests-php/Nominatim/NominatimTest.php b/test/php/Nominatim/NominatimTest.php similarity index 99% rename from tests-php/Nominatim/NominatimTest.php rename to test/php/Nominatim/NominatimTest.php index 7822c5dc..f8ba14c1 100644 --- a/tests-php/Nominatim/NominatimTest.php +++ b/test/php/Nominatim/NominatimTest.php @@ -2,7 +2,7 @@ namespace Nominatim; -require '../lib/lib.php'; +require '../../lib/lib.php'; class NominatimTest extends \PHPUnit_Framework_TestCase { diff --git a/tests-php/bootstrap.php b/test/php/bootstrap.php similarity index 100% rename from tests-php/bootstrap.php rename to test/php/bootstrap.php diff --git a/tests/scenes/bin/Makefile b/test/scenes/bin/Makefile similarity index 100% rename from tests/scenes/bin/Makefile rename to test/scenes/bin/Makefile diff --git a/tests/scenes/bin/make_scenes.sh b/test/scenes/bin/make_scenes.sh similarity index 100% rename from tests/scenes/bin/make_scenes.sh rename to test/scenes/bin/make_scenes.sh diff --git a/tests/scenes/bin/osm2wkt.cc b/test/scenes/bin/osm2wkt.cc similarity index 100% rename from tests/scenes/bin/osm2wkt.cc rename to test/scenes/bin/osm2wkt.cc diff --git a/tests/scenes/data/building-on-street-corner.wkt b/test/scenes/data/building-on-street-corner.wkt similarity index 100% rename from tests/scenes/data/building-on-street-corner.wkt rename to test/scenes/data/building-on-street-corner.wkt diff --git a/tests/scenes/data/building-with-parallel-streets.wkt b/test/scenes/data/building-with-parallel-streets.wkt similarity index 100% rename from tests/scenes/data/building-with-parallel-streets.wkt rename to test/scenes/data/building-with-parallel-streets.wkt diff --git a/tests/scenes/data/country.sql b/test/scenes/data/country.sql similarity index 100% rename from tests/scenes/data/country.sql rename to test/scenes/data/country.sql diff --git a/tests/scenes/data/country.wkt b/test/scenes/data/country.wkt similarity index 100% rename from tests/scenes/data/country.wkt rename to test/scenes/data/country.wkt diff --git a/tests/scenes/data/parallel-road.wkt b/test/scenes/data/parallel-road.wkt similarity index 100% rename from tests/scenes/data/parallel-road.wkt rename to test/scenes/data/parallel-road.wkt diff --git a/tests/scenes/data/points-on-roads.wkt b/test/scenes/data/points-on-roads.wkt similarity index 100% rename from tests/scenes/data/points-on-roads.wkt rename to test/scenes/data/points-on-roads.wkt diff --git a/tests/scenes/data/poly-area.wkt b/test/scenes/data/poly-area.wkt similarity index 100% rename from tests/scenes/data/poly-area.wkt rename to test/scenes/data/poly-area.wkt diff --git a/tests/scenes/data/poly-areas.osm b/test/scenes/data/poly-areas.osm similarity index 100% rename from tests/scenes/data/poly-areas.osm rename to test/scenes/data/poly-areas.osm diff --git a/tests/scenes/data/road-with-alley.wkt b/test/scenes/data/road-with-alley.wkt similarity index 100% rename from tests/scenes/data/road-with-alley.wkt rename to test/scenes/data/road-with-alley.wkt diff --git a/tests/scenes/data/roads-with-pois.wkt b/test/scenes/data/roads-with-pois.wkt similarity index 100% rename from tests/scenes/data/roads-with-pois.wkt rename to test/scenes/data/roads-with-pois.wkt diff --git a/tests/scenes/data/roads.osm b/test/scenes/data/roads.osm similarity index 100% rename from tests/scenes/data/roads.osm rename to test/scenes/data/roads.osm diff --git a/tests/scenes/data/split-road.wkt b/test/scenes/data/split-road.wkt similarity index 100% rename from tests/scenes/data/split-road.wkt rename to test/scenes/data/split-road.wkt diff --git a/tests/scenes/data/way-area-with-center.wkt b/test/scenes/data/way-area-with-center.wkt similarity index 100% rename from tests/scenes/data/way-area-with-center.wkt rename to test/scenes/data/way-area-with-center.wkt diff --git a/test/testdb/testdb.polys b/test/testdb/testdb.polys new file mode 100644 index 00000000..4298d68e --- /dev/null +++ b/test/testdb/testdb.polys @@ -0,0 +1,188 @@ +hamburg +1 + 9.5842804224817 53.5792118965693 + 10.2155260812517 53.8246176085747 + 10.475796519837 53.4477065812749 + 9.86815657040402 53.3278566584492 + 9.5842804224817 53.5792118965693 +END +END +liechtenstein +1 + 9.20853378041844 47.0559465458986 + 9.29384606832709 47.3507444175206 + 9.49848868129809 47.4492015884201 + 9.89867967626406 47.130228397937 + 9.58252408463202 46.8691824262863 + 9.20853378041844 47.0559465458986 +END +END +mauretania +1 + -17.1644809253606 20.8842205115601 + -16.9724694177095 21.4269590060279 + -13.1021317602129 21.4296172232924 + -13.1921945122782 22.933479308252 + -12.6268357994672 23.3657053906301 + -12.097953263953 23.5240373343518 + -12.0829087151283 26.0504750040867 + -8.76667736314239 26.0902806494621 + -8.70696146546581 27.398150020094 + -4.74082309576697 25.0335832288387 + -4.81791817472112 24.9110447612753 + -6.46188681458007 24.9097418021523 + -5.52019664485951 16.5528424294381 + -5.23944558032214 16.3345963721271 + -5.46430959536222 15.4167603246987 + -10.5098309475637 15.3543804758815 + -10.8990110578091 15.0165664449548 + -11.4524241105327 15.5284484048548 + -11.6506044402694 15.4177129737211 + -11.7217380574693 14.8443740855381 + -12.0680385872662 14.631162290351 + -12.9159650211166 15.1353675285383 + -13.4682080737434 15.990978880421 + -13.9034306609561 16.0335336430301 + -14.4204259536969 16.5367120778098 + -16.2084993845136 16.4236729998869 + -16.4302891135388 15.983629954068 + -16.7862990158802 16.0015581695182 + -16.3398604501701 18.2277294784962 + -16.8886326298423 19.403294102207 + -16.585950103956 20.1812628878206 + -17.0813413461875 20.5018960498623 + -17.1644809253606 20.8842205115601 +END +END +southdakota +1 + -104.297439187793 45.5046231666747 + -104.299553299189 45.9775616107873 + -104.096382287788 46.1214554410027 + -96.4569073278818 46.0989703969216 + -96.3044767039811 45.9115675974812 + -96.5247176872565 45.5862877816162 + -96.1968327207445 45.3001425265336 + -96.1895592635092 43.1140553398185 + -96.3329810429579 42.753709571028 + -96.2088431240585 42.4315154785281 + -96.5131423452368 42.3018249286814 + -97.3698668669773 42.6757095208851 + -98.0672608450503 42.5864899816312 + -98.6364176029847 42.8190548434256 + -104.146030684593 42.8349704964079 + -104.298744415682 43.0007971343175 + -104.297439187793 45.5046231666747 +END +END +uruguay +1 + -58.7094503498495 -33.5893528935793 + -58.5930297220504 -33.0935229194446 + -58.329380679061 -32.9223715673938 + -58.4160616358367 -31.8529190894557 + -58.2162152055053 -31.6271374679322 + -58.2872417410783 -31.4234579824293 + -58.0141967102111 -30.7805399817192 + -58.0801823804181 -30.438369563871 + -57.7465673402929 -30.0366166581386 + -57.3919054971047 -30.0920714480735 + -57.0841245854315 -29.904689506793 + -56.5203007925187 -30.0577138349604 + -55.8148965965951 -30.7486942236281 + -55.4992686810269 -30.6637735134172 + -55.0823825399047 -31.0951827436534 + -54.4609533378373 -31.3096231186724 + -52.8647106347639 -32.7122837473293 + -53.3056885052038 -33.2040687582016 + -53.3095867684494 -33.547639206286 + -52.9652990494926 -33.8719452856167 + -53.3564833683333 -34.6077542513996 + -55.6741509399751 -35.9609110600942 + -58.0955146798429 -34.8078487405856 + -58.1517292851949 -34.5120322638008 + -58.490557396808 -34.2574246976253 + -58.7094503498495 -33.5893528935793 +END +END + + +vietnam +1 + 111.737359200422 8.65966389848196 + 112.121385535871 9.03974154821598 + 112.431435613709 8.95854407291052 + 112.38091038991 8.72869141993135 + 112.09887919106 8.68752900955559 + 111.97764097397 8.47027379584868 + 111.737359200422 8.65966389848196)) + ((101.952539778161 22.3744843276505 + 102.42115782215 22.9512287654112 + 103.011765362757 22.7267436381062 + 103.309708309047 22.9942421815192 + 103.88404879754 22.8346923138744 + 104.62599830253 23.0355978361405 + 104.77147320329 23.3199292571035 + 105.357960240763 23.571602116825 + 105.914016228782 23.1523756402279 + 106.944000642084 22.9589139939517 + 106.898848084023 22.1610135134167 + 107.607567716302 21.7959107906373 + 107.974793702983 21.8006347108588 + 108.387669146529 21.2765369215022 + 108.350851164743 20.968001228975 + 107.807714207219 20.5205405897832 + 108.126799006308 20.25069143158 + 108.024397106408 19.8655870027683 + 107.557799016272 19.7778131848293 + 107.347203511678 19.9746481898397 + 107.334187929597 20.2503746970115 + 107.017188364275 20.2484351047036 + 106.88168214676 19.9452753668929 + 106.293211714996 19.5544614034104 + 106.229050701227 19.084494263666 + 106.339912028886 18.6544696395528 + 107.02787120458 18.0908270105482 + 106.992048357062 17.6610336194013 + 107.61356106657 17.4455208732243 + 109.497501740771 15.5635441996556 + 109.851305717105 12.5942782065681 + 109.388462041025 9.73419261849773 + 106.766311197844 8.27477170182912 + 104.780565299066 8.00956568214471 + 103.236641212428 8.93273491992321 + 102.869224286517 9.37238182036115 + 102.909736286172 9.54505245725422 + 103.717955146996 10.1271353150665 + 103.686354089781 10.5210100495922 + 104.015476398095 10.662190116791 + 104.247688572369 10.5092223585766 + 104.775438554966 10.7014491439673 + 104.980441972174 11.1056591699101 + 105.7024629092 11.2045497780933 + 105.626753980109 11.6300119740348 + 105.748149258633 11.8111016132902 + 107.378741221451 12.5115515071093 + 107.300732476862 12.958173736809 + 107.435951728865 13.4799057185978 + 107.150187488306 14.1107044150892 + 107.411164618718 15.2667546971578 + 107.034817435829 15.6854570535274 + 107.112895173759 15.9819504632758 + 106.538255811019 16.3296204491326 + 106.173473133351 17.0787384431658 + 105.048356842632 18.1992017641427 + 104.927464569104 18.5590394564598 + 103.69790542148 19.2392576346486 + 103.937765477152 19.8144348780526 + 104.649272403028 19.9254452865302 + 104.202109979964 20.3691301778448 + 104.304114148338 20.6519242141597 + 103.979630359435 20.7152945884418 + 103.65735416319 20.4809125958884 + 103.000528825335 20.7510959123255 + 102.627912181829 21.1973495084135 + 102.683177661723 21.4733395191357 + 101.952539778161 22.3744843276505 +END +END diff --git a/test/testdb/wikipedia_article.sql.bin b/test/testdb/wikipedia_article.sql.bin new file mode 100644 index 00000000..628e2af4 Binary files /dev/null and b/test/testdb/wikipedia_article.sql.bin differ diff --git a/test/testdb/wikipedia_redirect.sql.bin b/test/testdb/wikipedia_redirect.sql.bin new file mode 100644 index 00000000..9c4b513d Binary files /dev/null and b/test/testdb/wikipedia_redirect.sql.bin differ diff --git a/tests-php/README.txt b/tests-php/README.txt deleted file mode 100644 index d551c1da..00000000 --- a/tests-php/README.txt +++ /dev/null @@ -1,14 +0,0 @@ -Basic unit tests of PHP code. Very low coverage. Doesn't cover interaction -with the webserver/HTTP or database (yet). - -You need to have -https://phpunit.de/manual/4.2/en/ -installed. - -To execute the test suite run -$ cd tests-php -$ phpunit ./ - -It will read phpunit.xml which points to the library, test path, bootstrap -strip and set other parameters. - diff --git a/tests/README.md b/tests/README.md deleted file mode 100644 index 1b1663c3..00000000 --- a/tests/README.md +++ /dev/null @@ -1,102 +0,0 @@ -This directory contains functional tests for the Nominatim API, -for the import/update from osm files and for indexing. - -The tests use the lettuce framework (http://lettuce.it/) and -nose (https://nose.readthedocs.org). API tests are meant to be run -against a Nominatim installation with a complete planet-wide -setup based on a fairly recent planet. If you only have an -excerpt, some of the API tests may fail. Database tests can be -run without having a database installed. - -Prerequisites -============= - - * lettuce framework (http://lettuce.it/) - * nose (https://nose.readthedocs.org) - * pytidylib (http://countergram.com/open-source/pytidylib) - * haversine (https://github.com/mapado/haversine) - * shapely (https://github.com/Toblerity/Shapely) - -Usage -===== - - * get prerequisites - - # on a fresh Ubuntu LTS 14.04 you'll also need these system-wide packages - [sudo] apt-get install python-dev python-pip python-Levenshtein tidy - - [sudo] pip install lettuce nose pytidylib haversine psycopg2 shapely - - * run the tests - - NOMINATIM_SERVER=http://your.nominatim.instance/ lettuce features - -The tests can be configured with a set of environment variables: - - * `NOMINATIM_SERVER` - URL of the nominatim instance (API tests) - * `NOMINATIM_DIR` - source directory of Nominatim (import tests) - * `TEMPLATE_DB` - name of template database used as a skeleton for - the test databases (db tests) - * `TEST_DB` - name of test database (db tests) - * `NOMINATIM_SETTINGS` - file to write temporary Nominatim settings to (db tests) - * `NOMINATIM_REUSE_TEMPLATE` - if defined, the template database will not be - deleted after the test runs and reused during - the next run. This speeds up tests considerably - but might lead to outdated errors for some - changes in the database layout. - * `NOMINATIM_KEEP_SCENARIO_DB` - if defined, the test database will not be - dropped after a test is finished. Should - only be used if one single scenario is run, - otherwise the result is undefined. - * `LOGLEVEL` - set to 'debug' to get more verbose output (only works properly - when output to a logfile is configured) - * `LOGFILE` - sends debug output to the given file - -Writing Tests -============= - -The following explanation assume that the reader is familiar with the lettuce -notations of features, scenarios and steps. - -All possible steps can be found in the `steps` directory and should ideally -be documented. - - -API Tests (`features/api`) --------------------------- - -These tests are meant to test the different API calls and their parameters. - -There are two kind of steps defined for these tests: -request setup steps (see `steps/api_setup.py`) -and steps for checking results (see `steps/api_result.py`). - -Each scenario follows this simple sequence of steps: - - 1. One or more steps to define parameters and HTTP headers of the request. - These are cumulative, so you can use multiple steps. - 2. A single step to call the API. This sends a HTTP request to the configured - server and collects the answer. The cached parameters will be deleted, - to ensure that the setup works properly with scenario outlines. - 3. As many result checks as necessary. The result remains cached, so that - multiple tests can be added here. - -Indexing Tests (`features/db`) ---------------------------------------------------- - -These tests check the import and update of the Nominatim database. They do not -test the correctness of osm2pgsql. Each test will write some data into the `place` -table (and optionally `the planet_osm_*` tables if required) and then run -Nominatim's processing functions on that. - -These tests need to create their own test databases. By default they will be -called `test_template_nominatim` and `test_nominatim`. Names can be changed with -the environment variables `TEMPLATE_DB` and `TEST_DB`. The user running the tests -needs superuser rights for postgres. - - -Import Tests (`features/osm2pgsql`) ------------------------------------ - -These tests check that data is imported correctly into the place table. They -use the same template database as the Indexing tests, so the same remarks apply. diff --git a/tests/features/api/details.feature b/tests/features/api/details.feature deleted file mode 100644 index e59659c3..00000000 --- a/tests/features/api/details.feature +++ /dev/null @@ -1,13 +0,0 @@ -Feature: Object details - Check details page for correctness - - Scenario Outline: Details via OSM id - When looking up details for - Then the result is valid - - Examples: - | object - | 1758375 - | N158845944 - | W72493656 - | R62422 diff --git a/tests/features/api/language.feature b/tests/features/api/language.feature deleted file mode 100644 index 529dc021..00000000 --- a/tests/features/api/language.feature +++ /dev/null @@ -1,100 +0,0 @@ -Feature: Localization of search results - - Scenario: Search - default language - When sending json search query "Germany" - Then results contain - | ID | display_name - | 0 | Deutschland.* - - Scenario: Search - accept-language first - Given the request parameters - | accept-language - | en,de - When sending json search query "Deutschland" - Then results contain - | ID | display_name - | 0 | Germany.* - - Scenario: Search - accept-language missing - Given the request parameters - | accept-language - | xx,fr,en,de - When sending json search query "Deutschland" - Then results contain - | ID | display_name - | 0 | Allemagne.* - - Scenario: Search - http accept language header first - Given the HTTP header - | accept-language - | fr-ca,fr;q=0.8,en-ca;q=0.5,en;q=0.3 - When sending json search query "Deutschland" - Then results contain - | ID | display_name - | 0 | Allemagne.* - - Scenario: Search - http accept language header and accept-language - Given the request parameters - | accept-language - | de,en - Given the HTTP header - | accept-language - | fr-ca,fr;q=0.8,en-ca;q=0.5,en;q=0.3 - When sending json search query "Deutschland" - Then results contain - | ID | display_name - | 0 | Deutschland.* - - Scenario: Search - http accept language header fallback - Given the HTTP header - | accept-language - | fr-ca,en-ca;q=0.5 - When sending json search query "Deutschland" - Then results contain - | ID | display_name - | 0 | Allemagne.* - - Scenario: Search - http accept language header fallback (upper case) - Given the HTTP header - | accept-language - | fr-FR;q=0.8,en-ca;q=0.5 - When sending json search query "Deutschland" - Then results contain - | ID | display_name - | 0 | Allemagne.* - - Scenario: Reverse - default language - When looking up coordinates 48.13921,11.57328 - Then result addresses contain - | ID | city - | 0 | München - - Scenario: Reverse - accept-language parameter - Given the request parameters - | accept-language - | en,fr - When looking up coordinates 48.13921,11.57328 - Then result addresses contain - | ID | city - | 0 | Munich - - Scenario: Reverse - HTTP accept language header - Given the HTTP header - | accept-language - | fr-ca,fr;q=0.8,en-ca;q=0.5,en;q=0.3 - When looking up coordinates 48.13921,11.57328 - Then result addresses contain - | ID | city - | 0 | Munich - - Scenario: Reverse - accept-language parameter and HTTP header - Given the request parameters - | accept-language - | it - Given the HTTP header - | accept-language - | fr-ca,fr;q=0.8,en-ca;q=0.5,en;q=0.3 - When looking up coordinates 48.13921,11.57328 - Then result addresses contain - | ID | city - | 0 | Monaco di Baviera diff --git a/tests/features/api/lookup.feature b/tests/features/api/lookup.feature deleted file mode 100644 index 7b86fb49..00000000 --- a/tests/features/api/lookup.feature +++ /dev/null @@ -1,15 +0,0 @@ -Feature: Places by osm_type and osm_id Tests - Simple tests for internal server errors and response format. - - Scenario: address lookup for existing node, way, relation - When looking up xml places N158845944,W72493656,,R62422,X99,N0 - Then the result is valid xml - exactly 3 results are returned - When looking up json places N158845944,W72493656,,R62422,X99,N0 - Then the result is valid json - exactly 3 results are returned - - Scenario: address lookup for non-existing or invalid node, way, relation - When looking up xml places X99,,N0,nN158845944,ABC,,W9 - Then the result is valid xml - exactly 0 results are returned \ No newline at end of file diff --git a/tests/features/api/regression.feature b/tests/features/api/regression.feature deleted file mode 100644 index 08156d62..00000000 --- a/tests/features/api/regression.feature +++ /dev/null @@ -1,217 +0,0 @@ -Feature: API regression tests - Tests error cases reported in tickets. - - Scenario: trac #2430 - When sending json search query "89 River Avenue, Hoddesdon, Hertfordshire, EN11 0JT" - Then at least 1 result is returned - - Scenario: trac #2440 - When sending json search query "East Harvard Avenue, Denver" - Then more than 2 results are returned - - Scenario: trac #2456 - When sending xml search query "Borlänge Kommun" - Then results contain - | ID | place_rank - | 0 | 19 - - Scenario: trac #2530 - When sending json search query "Lange Straße, Bamberg" with address - Then result addresses contain - | ID | town - | 0 | Bamberg - - Scenario: trac #2541 - When sending json search query "pad, germany" - Then results contain - | ID | class | display_name - | 0 | aeroway | Paderborn/Lippstadt,.* - - Scenario: trac #2579 - When sending json search query "Johnsons Close, hackbridge" with address - Then result addresses contain - | ID | postcode - | 0 | SM5 2LU - - @Fail - Scenario Outline: trac #2586 - When sending json search query "" with address - Then result addresses contain - | ID | country_code - | 0 | uk - - Examples: - | query - | DL7 0SN - | DL70SN - - Scenario: trac #2628 (1) - When sending json search query "Adam Kraft Str" with address - Then result addresses contain - | ID | road - | 0 | Adam-Kraft-Straße - - Scenario: trac #2628 (2) - When sending json search query "Maxfeldstr. 5, Nürnberg" with address - Then result addresses contain - | ID | house_number | road | city - | 0 | 5 | Maxfeldstraße | Nürnberg - - Scenario: trac #2638 - When sending json search query "Nöthnitzer Str. 40, 01187 Dresden" with address - Then result addresses contain - | ID | house_number | road | city - | 0 | 40 | Nöthnitzer Straße | Dresden - - Scenario Outline: trac #2667 - When sending json search query "" with address - Then result addresses contain - | ID | house_number - | 0 | - - Examples: - | number | query - | 16 | 16 Woodpecker Way, Cambourne - | 14906 | 14906, 114 Street Northwest, Edmonton, Alberta, Canada - | 14904 | 14904, 114 Street Northwest, Edmonton, Alberta, Canada - | 15022 | 15022, 114 Street Northwest, Edmonton, Alberta, Canada - | 15024 | 15024, 114 Street Northwest, Edmonton, Alberta, Canada - - Scenario: trac #2681 - When sending json search query "kirchstraße troisdorf Germany" - Then results contain - | ID | display_name - | 0 | .*, Troisdorf, .* - - Scenario: trac #2758 - When sending json search query "6а, полуботка, чернигов" with address - Then result addresses contain - | ID | house_number - | 0 | 6а - - Scenario: trac #2790 - When looking up coordinates 49.0942079697809,8.27565898861822 - Then result addresses contain - | ID | road | village | country - | 0 | Daimlerstraße | Jockgrim | Deutschland - - Scenario: trac #2794 - When sending json search query "4008" - Then results contain - | ID | class | type - | 0 | place | postcode - - Scenario: trac #2797 - When sending json search query "Philippstr.4, 52349 Düren" with address - Then result addresses contain - | ID | road | town - | 0 | Philippstraße | Düren - - Scenario: trac #2830 - When sending json search query "207, Boardman Street, S0J 1L0, CA" with address - Then result addresses contain - | ID | house_number | road | postcode | country - | 0 | 207 | Boardman Street | S0J 1L0 | Canada - - Scenario: trac #2830 - When sending json search query "S0J 1L0,CA" - Then results contain - | ID | class | type | display_name - | 0 | place | postcode | .*, Canada - - Scenario: trac #2845 - When sending json search query "Leliestraat 31, Zwolle" with address - Then result addresses contain - | ID | city - | 0 | Zwolle - - Scenario: trac #2852 - When sending json search query "berlinerstrasse, leipzig" with address - Then result addresses contain - | ID | road - | 0 | Berliner Straße - - Scenario: trac #2871 - When looking up coordinates -33.906895553,150.99609375 - Then result addresses contain - | ID | city | country - | 0 | [^0-9]*$ | Australia - - Scenario: trac #2981 - When sending json search query "Ohmstraße 7, Berlin" with address - Then at least 2 results are returned - And result addresses contain - | house_number | road | state - | 7 | Ohmstraße | Berlin - - Scenario: trac #3049 - When sending json search query "Soccer City" - Then results contain - | ID | class | type | latlon - | 0 | leisure | stadium | -26.2347261,27.982645 +-50m - - Scenario: trac #3130 - When sending json search query "Old Way, Frinton" - Then results contain - | ID | class | latlon - | 0 | highway | 51.8324206,1.2447352 +-100m - - Scenario Outline: trac #5025 - When sending json search query "Kriegsstr , Karlsruhe" with address - Then result addresses contain - | house_number | road - | | Kriegsstraße - - Examples: - | house_nr - | 5c - | 25 - | 78 - | 80 - | 99 - | 130 - | 153 - | 196 - | 256 - | 294 - - Scenario: trac #5238 - Given the request parameters - | bounded | viewbox - | 1 | -1,0,0,-1 - When sending json search query "sy" - Then exactly 0 results are returned - - Scenario: trac #5274 - When sending json search query "Goedestraat 41-BS, Utrecht" with address - Then result addresses contain - | house_number | road | city - | 41-BS | Goedestraat | Utrecht - - @poldi-only - Scenario Outline: github #36 - When sending json search query "" with address - Then result addresses contain - | ID | road | city - | 0 | Seegasse | .*Wieselburg-Land - - Examples: - | query - | Seegasse, Gemeinde Wieselburg-Land - | Seegasse, Wieselburg-Land - | Seegasse, Wieselburg - - Scenario: github #190 - When looking up place N257363453 - Then the results contain - | osm_type | osm_id | latlon - | node | 257363453 | 35.8404121,128.5586643 +-100m - - Scenario: trac #5427 - Given the request parameters - | countrycodes | - | DE | - When sending json search query "12345" with address - Then result addresses contain - | country_code | - | de | diff --git a/tests/features/api/reverse.feature b/tests/features/api/reverse.feature deleted file mode 100644 index 7bd12913..00000000 --- a/tests/features/api/reverse.feature +++ /dev/null @@ -1,148 +0,0 @@ -Feature: Reverse geocoding - Testing the reverse function - - # Make sure country is not overwritten by the postcode - Scenario: Country is returned - Given the request parameters - | accept-language - | de - When looking up coordinates 53.9788769,13.0830313 - Then result addresses contain - | ID | country - | 0 | Deutschland - - - Scenario: Boundingbox is returned - Given the request parameters - | format | zoom - | xml | 4 - When looking up coordinates 53.9788769,13.0830313 - And results contain valid boundingboxes - - Scenario: Reverse geocoding for odd interpolated housenumber - - Scenario: Reverse geocoding for even interpolated housenumber - - @Tiger - Scenario: TIGER house number - Given the request parameters - | addressdetails - | 1 - When looking up jsonv2 coordinates 40.6863624710666,-112.060005720023 - And exactly 1 result is returned - And result addresses contain - | ID | house_number | road | postcode | country_code - | 0 | 709. | Kings Estate Drive | 84128 | us - And results contain - | osm_type | category | type - | way | place | house - - @Tiger - Scenario: No TIGER house number for zoom < 18 - Given the request parameters - | addressdetails | zoom - | 1 | 17 - When looking up coordinates 40.6863624710666,-112.060005720023 - And exactly 1 result is returned - And result addresses contain - | ID | road | postcode | country_code - | 0 | Kings Estate Drive | 84128 | us - And result 0 has attributes osm_id,osm_type - - Scenario Outline: Reverse Geocoding with extratags - Given the request parameters - | extratags - | 1 - When looking up coordinates 48.86093,2.2978 - Then result 0 has attributes extratags - - Examples: - | format - | xml - | json - | jsonv2 - - Scenario Outline: Reverse Geocoding with namedetails - Given the request parameters - | namedetails - | 1 - When looking up coordinates 48.86093,2.2978 - Then result 0 has attributes namedetails - - Examples: - | format - | xml - | json - | jsonv2 - - - Scenario Outline: Reverse Geocoding contains TEXT geometry - Given the request parameters - | polygon_text - | 1 - When looking up coordinates 48.86093,2.2978 - Then result 0 has attributes - - Examples: - | format | response_attribute - | xml | geotext - | json | geotext - | jsonv2 | geotext - - Scenario Outline: Reverse Geocoding contains polygon-as-points geometry - Given the request parameters - | polygon - | 1 - When looking up coordinates 48.86093,2.2978 - Then result 0 has not attributes - - Examples: - | format | response_attribute - | xml | polygonpoints - | json | polygonpoints - | jsonv2 | polygonpoints - - - - Scenario Outline: Reverse Geocoding contains SVG geometry - Given the request parameters - | polygon_svg - | 1 - When looking up coordinates 48.86093,2.2978 - Then result 0 has attributes - - Examples: - | format | response_attribute - | xml | geosvg - | json | svg - | jsonv2 | svg - - - Scenario Outline: Reverse Geocoding contains KML geometry - Given the request parameters - | polygon_kml - | 1 - When looking up coordinates 48.86093,2.2978 - Then result 0 has attributes - - Examples: - | format | response_attribute - | xml | geokml - | json | geokml - | jsonv2 | geokml - - - Scenario Outline: Reverse Geocoding contains GEOJSON geometry - Given the request parameters - | polygon_geojson - | 1 - When looking up coordinates 48.86093,2.2978 - Then result 0 has attributes - - Examples: - | format | response_attribute - | xml | geojson - | json | geojson - | jsonv2 | geojson - - diff --git a/tests/features/api/reverse_by_id.feature b/tests/features/api/reverse_by_id.feature deleted file mode 100644 index 5f5a8f8d..00000000 --- a/tests/features/api/reverse_by_id.feature +++ /dev/null @@ -1,13 +0,0 @@ -Feature: Reverse lookup by ID - Testing reverse geocoding via OSM ID - - # see github issue #269 - Scenario: Get address of linked places - Given the request parameters - | osm_type | osm_id - | N | 151421301 - When sending an API call reverse - Then exactly 1 result is returned - And result addresses contain - | county | state - | Pratt County | Kansas diff --git a/tests/features/api/reverse_simple.feature b/tests/features/api/reverse_simple.feature deleted file mode 100644 index 6100f54c..00000000 --- a/tests/features/api/reverse_simple.feature +++ /dev/null @@ -1,140 +0,0 @@ -Feature: Simple Reverse Tests - Simple tests for internal server errors and response format. - These tests should pass on any Nominatim installation. - - Scenario Outline: Simple reverse-geocoding - When looking up xml coordinates , - Then the result is valid xml - When looking up json coordinates , - Then the result is valid json - When looking up jsonv2 coordinates , - Then the result is valid json - - Examples: - | lat | lon - | 0.0 | 0.0 - | 45.3 | 3.5 - | -79.34 | 23.5 - | 0.23 | -178.555 - - Scenario Outline: Testing different parameters - Given the request parameters - | - | - When sending search query "Manchester" - Then the result is valid html - Given the request parameters - | - | - When sending html search query "Manchester" - Then the result is valid html - Given the request parameters - | - | - When sending xml search query "Manchester" - Then the result is valid xml - Given the request parameters - | - | - When sending json search query "Manchester" - Then the result is valid json - Given the request parameters - | - | - When sending jsonv2 search query "Manchester" - Then the result is valid json - - Examples: - | parameter | value - | polygon | 1 - | polygon | 0 - | polygon_text | 1 - | polygon_text | 0 - | polygon_kml | 1 - | polygon_kml | 0 - | polygon_geojson | 1 - | polygon_geojson | 0 - | polygon_svg | 1 - | polygon_svg | 0 - - - - - Scenario Outline: Wrapping of legal jsonp requests - Given the request parameters - | json_callback - | foo - When looking up coordinates 67.3245,0.456 - Then the result is valid json - - Examples: - | format - | json - | jsonv2 - - Scenario: Reverse-geocoding without address - Given the request parameters - | addressdetails - | 0 - When looking up xml coordinates 36.791966,127.171726 - Then the result is valid xml - When looking up json coordinates 36.791966,127.171726 - Then the result is valid json - When looking up jsonv2 coordinates 36.791966,127.171726 - Then the result is valid json - - Scenario: Reverse-geocoding with zoom - Given the request parameters - | zoom - | 10 - When looking up xml coordinates 36.791966,127.171726 - Then the result is valid xml - When looking up json coordinates 36.791966,127.171726 - Then the result is valid json - When looking up jsonv2 coordinates 36.791966,127.171726 - Then the result is valid json - - Scenario: Missing lon parameter - Given the request parameters - | lat - | 51.51 - When sending an API call reverse - Then a HTTP 400 is returned - - Scenario: Missing lat parameter - Given the request parameters - | lon - | -79.39114 - When sending an API call reverse - Then a HTTP 400 is returned - - Scenario: Missing osm_id parameter - Given the request parameters - | osm_type - | N - When sending an API call reverse - Then a HTTP 400 is returned - - Scenario: Missing osm_type parameter - Given the request parameters - | osm_id - | 3498564 - When sending an API call reverse - Then a HTTP 400 is returned - - Scenario Outline: Bad format for lat or lon - Given the request parameters - | lat | lon | - | | | - When sending an API call reverse - Then a HTTP 400 is returned - - Examples: - | lat | lon - | 48.9660 | 8,4482 - | 48,9660 | 8.4482 - | 48,9660 | 8,4482 - | 48.966.0 | 8.4482 - | 48.966 | 8.448.2 - | Nan | 8.448 - | 48.966 | Nan diff --git a/tests/features/api/search.feature b/tests/features/api/search.feature deleted file mode 100644 index 91050daf..00000000 --- a/tests/features/api/search.feature +++ /dev/null @@ -1,86 +0,0 @@ -Feature: Search queries - Testing correctness of results - - Scenario: UK House number search - When sending json search query "27 Thoresby Road, Broxtowe" with address - Then address of result 0 contains - | type | value - | house_number | 27 - | road | Thoresby Road - | city | Broxtowe - | state | England - | country | U.*K.* - | country_code | gb - - - Scenario: House number search for non-street address - Given the request parameters - | accept-language - | en - When sending json search query "4 Pomocnia, Pokrzywnica, Poland" with address - Then address of result 0 contains - | type | value - | house_number | 4 - | county | gmina Pokrzywnica - | state | Masovian Voivodeship - | postcode | 06-121 - | country | Poland - | country_code | pl - Then address of result 0 does not contain road - - Scenario: House number interpolation even - Given the request parameters - | accept-language - | en - When sending json search query "140 rue Don Bosco, Saguenay" with address - Then address of result 0 contains - | type | value - | house_number | 140 - | road | [Rr]ue Don Bosco - | city | .*Saguenay - | state | Quebec - | country | Canada - | country_code | ca - - Scenario: House number interpolation odd - Given the request parameters - | accept-language - | en - When sending json search query "141 rue Don Bosco, Saguenay" with address - Then address of result 0 contains - | type | value - | house_number | 141 - | road | [rR]ue Don Bosco - | city | .*Saguenay - | state | Quebec - | country | Canada - | country_code | ca - - @Tiger - Scenario: TIGER house number - When sending json search query "3 West Victory Way, Craig" - Then results contain - | osm_type - | way - - @Tiger - Scenario: TIGER house number (road fallback) - When sending json search query "3030 West Victory Way, Craig" - Then results contain - | osm_type - | way - - Scenario: Expansion of Illinois - Given the request parameters - | accept-language - | en - When sending json search query "il, us" - Then results contain - | ID | display_name - | 0 | Illinois.* - - Scenario: Search with class-type feature - When sending jsonv2 search query "Hotel California" - Then results contain - | place_rank - | 30 diff --git a/tests/features/api/search_order.feature b/tests/features/api/search_order.feature deleted file mode 100644 index fad5e89c..00000000 --- a/tests/features/api/search_order.feature +++ /dev/null @@ -1,36 +0,0 @@ -Feature: Result order for Geocoding - Testing that importance ordering returns sensible results - - Scenario Outline: city order in street search - Given the request parameters - | limit - | 100 - When sending json search query ", " with address - Then address of result 0 contains - | type | value - | | - - Examples: - | type | city | street - | city | Zürich | Rigistr - | city | Karlsruhe | Sophienstr - | city | München | Karlstr - | city | Praha | Dlouhá - - Scenario Outline: use more important city in street search - When sending json search query ", " with address - Then result addresses contain - | ID | country_code - | 0 | - - Examples: - | country | city | street - | gb | London | Main St - | gb | Manchester | Central Street - - # https://trac.openstreetmap.org/ticket/5094 - Scenario: housenumbers are ordered by complete match first - When sending json search query "4 Докукина Москва" with address - Then result addresses contain - | ID | house_number - | 0 | 4 diff --git a/tests/features/api/search_params.feature b/tests/features/api/search_params.feature deleted file mode 100644 index cd0db091..00000000 --- a/tests/features/api/search_params.feature +++ /dev/null @@ -1,315 +0,0 @@ -Feature: Search queries - Testing different queries and parameters - - Scenario: Simple XML search - When sending xml search query "Schaan" - Then result 0 has attributes place_id,osm_type,osm_id - And result 0 has attributes place_rank,boundingbox - And result 0 has attributes lat,lon,display_name - And result 0 has attributes class,type,importance,icon - And result 0 has not attributes address - And results contain valid boundingboxes - - Scenario: Simple JSON search - When sending json search query "Vaduz" - And result 0 has attributes place_id,licence,icon,class,type - And result 0 has attributes osm_type,osm_id,boundingbox - And result 0 has attributes lat,lon,display_name,importance - And result 0 has not attributes address - And results contain valid boundingboxes - - Scenario: JSON search with addressdetails - When sending json search query "Montevideo" with address - Then address of result 0 is - | type | value - | city | Montevideo - | state | Montevideo - | country | Uruguay - | country_code | uy - - Scenario: XML search with addressdetails - When sending xml search query "Inuvik" with address - Then address of result 0 contains - | type | value - | state | Northwest Territories - | country | Canada - | country_code | ca - - Scenario: coordinate search with addressdetails - When sending json search query "51.193058013916,15.5245780944824" with address - Then result addresses contain - | village | country | country_code - | Kraszowice | Polska | pl - - Scenario: Address details with unknown class types - When sending json search query "foobar, Essen" with address - Then results contain - | ID | class | type - | 0 | leisure | hackerspace - And result addresses contain - | ID | address29 - | 0 | Chaospott - And address of result 0 does not contain leisure,hackerspace - - Scenario: Disabling deduplication - When sending json search query "Oxford Street, London" - Then there are no duplicates - Given the request parameters - | dedupe - | 0 - When sending json search query "Oxford Street, London" - Then there are duplicates - - Scenario: Search with bounded viewbox in right area - Given the request parameters - | bounded | viewbox - | 1 | -87.7,41.9,-87.57,41.85 - When sending json search query "restaurant" with address - Then result addresses contain - | ID | city - | 0 | Chicago - - Scenario: Search with bounded viewboxlbrt in right area - Given the request parameters - | bounded | viewboxlbrt - | 1 | -87.7,41.85,-87.57,41.9 - When sending json search query "restaurant" with address - Then result addresses contain - | ID | city - | 0 | Chicago - - Scenario: No POI search with unbounded viewbox - Given the request parameters - | viewbox - | -87.7,41.9,-87.57,41.85 - When sending json search query "restaurant" - Then results contain - | display_name - | [^,]*(?i)restaurant.* - - Scenario: bounded search remains within viewbox, even with no results - Given the request parameters - | bounded | viewbox - | 1 | 43.5403125,-5.6563282,43.54285,-5.662003 - When sending json search query "restaurant" - Then less than 1 result is returned - - Scenario: bounded search remains within viewbox with results - Given the request parameters - | bounded | viewbox - | 1 | -5.662003,43.55,-5.6563282,43.5403125 - When sending json search query "restaurant" - | lon | lat - | >= -5.662003 | >= 43.5403125 - | <= -5.6563282| <= 43.55 - - Scenario: Prefer results within viewbox - Given the request parameters - | accept-language - | en - When sending json search query "royan" with address - Then result addresses contain - | ID | country - | 0 | France - Given the request parameters - | accept-language | viewbox - | en | 51.94,36.59,51.99,36.56 - When sending json search query "royan" with address - Then result addresses contain - | ID | country - | 0 | Iran - - Scenario: Overly large limit number for search results - Given the request parameters - | limit - | 1000 - When sending json search query "Neustadt" - Then at most 50 results are returned - - Scenario: Limit number of search results - Given the request parameters - | limit - | 4 - When sending json search query "Neustadt" - Then exactly 4 results are returned - - Scenario: Restrict to feature type country - Given the request parameters - | featureType - | country - When sending xml search query "Monaco" - Then results contain - | place_rank - | 4 - - Scenario: Restrict to feature type state - When sending xml search query "Berlin" - Then results contain - | ID | place_rank - | 0 | 1[56] - Given the request parameters - | featureType - | state - When sending xml search query "Berlin" - Then results contain - | place_rank - | [78] - - Scenario: Restrict to feature type city - Given the request parameters - | featureType - | city - When sending xml search query "Monaco" - Then results contain - | place_rank - | 1[56789] - - - Scenario: Restrict to feature type settlement - When sending json search query "Everest" - Then results contain - | ID | display_name - | 0 | Mount Everest.* - Given the request parameters - | featureType - | settlement - When sending json search query "Everest" - Then results contain - | ID | display_name - | 0 | Everest.* - - Scenario Outline: Search with polygon threshold (json) - Given the request parameters - | polygon_geojson | polygon_threshold - | 1 |
- When sending json search query "switzerland" - Then at least 1 result is returned - And result 0 has attributes geojson - - Examples: - | th - | -1 - | 0.0 - | 0.5 - | 999 - - Scenario Outline: Search with polygon threshold (xml) - Given the request parameters - | polygon_geojson | polygon_threshold - | 1 | - When sending xml search query "switzerland" - Then at least 1 result is returned - And result 0 has attributes geojson - - Examples: - | th - | -1 - | 0.0 - | 0.5 - | 999 - - Scenario Outline: Search with invalid polygon threshold (xml) - Given the request parameters - | polygon_geojson | polygon_threshold - | 1 | - When sending xml search query "switzerland" - Then a HTTP 400 is returned - - - Scenario Outline: Search with extratags - Given the request parameters - | extratags - | 1 - When sending search query "Hauptstr" - Then result 0 has attributes extratags - And result 1 has attributes extratags - - Examples: - | format - | xml - | json - | jsonv2 - - Scenario Outline: Search with namedetails - Given the request parameters - | namedetails - | 1 - When sending search query "Hauptstr" - Then result 0 has attributes namedetails - And result 1 has attributes namedetails - - Examples: - | format - | xml - | json - | jsonv2 - - - Scenario Outline: Search result with contains TEXT geometry - Given the request parameters - | polygon_text - | 1 - When sending search query "switzerland" - Then result 0 has attributes - - Examples: - | format | response_attribute - | xml | geotext - | json | geotext - | jsonv2 | geotext - - Scenario Outline: Search result contains polygon-as-points geometry - Given the request parameters - | polygon - | 1 - When sending search query "switzerland" - Then result 0 has attributes - - Examples: - | format | response_attribute - | xml | polygonpoints - | json | polygonpoints - | jsonv2 | polygonpoints - - - - Scenario Outline: Search result contains SVG geometry - Given the request parameters - | polygon_svg - | 1 - When sending search query "switzerland" - Then result 0 has attributes - - Examples: - | format | response_attribute - | xml | geosvg - | json | svg - | jsonv2 | svg - - - Scenario Outline: Search result contains KML geometry - Given the request parameters - | polygon_kml - | 1 - When sending search query "switzerland" - Then result 0 has attributes - - Examples: - | format | response_attribute - | xml | geokml - | json | geokml - | jsonv2 | geokml - - - Scenario Outline: Search result contains GEOJSON geometry - Given the request parameters - | polygon_geojson - | 1 - When sending search query "switzerland" - Then result 0 has attributes - - Examples: - | format | response_attribute - | xml | geojson - | json | geojson - | jsonv2 | geojson diff --git a/tests/features/api/search_simple.feature b/tests/features/api/search_simple.feature deleted file mode 100644 index 0020cc2e..00000000 --- a/tests/features/api/search_simple.feature +++ /dev/null @@ -1,238 +0,0 @@ -Feature: Simple Tests - Simple tests for internal server errors and response format. - These tests should pass on any Nominatim installation. - - Scenario Outline: Testing different parameters - Given the request parameters - | - | - When sending search query "Manchester" - Then the result is valid html - Given the request parameters - | - | - When sending html search query "Manchester" - Then the result is valid html - Given the request parameters - | - | - When sending xml search query "Manchester" - Then the result is valid xml - Given the request parameters - | - | - When sending json search query "Manchester" - Then the result is valid json - Given the request parameters - | - | - When sending jsonv2 search query "Manchester" - Then the result is valid json - - Examples: - | parameter | value - | addressdetails | 1 - | addressdetails | 0 - | polygon | 1 - | polygon | 0 - | polygon_text | 1 - | polygon_text | 0 - | polygon_kml | 1 - | polygon_kml | 0 - | polygon_geojson | 1 - | polygon_geojson | 0 - | polygon_svg | 1 - | polygon_svg | 0 - | accept-language | de,en - | countrycodes | uk,ir - | bounded | 1 - | bounded | 0 - | exclude_place_ids| 385252,1234515 - | limit | 1000 - | dedupe | 1 - | dedupe | 0 - | extratags | 1 - | extratags | 0 - | namedetails | 1 - | namedetails | 0 - - Scenario: Search with invalid output format - Given the request parameters - | format - | fd$# - When sending search query "Berlin" - Then a HTTP 400 is returned - - Scenario Outline: Simple Searches - When sending search query "" - Then the result is valid html - When sending html search query "" - Then the result is valid html - When sending xml search query "" - Then the result is valid xml - When sending json search query "" - Then the result is valid json - When sending jsonv2 search query "" - Then the result is valid json - - Examples: - | query - | New York, New York - | France - | 12, Main Street, Houston - | München - | 東京都 - | hotels in nantes - | xywxkrf - | gh; foo() - | %#$@*&l;der#$! - | 234 - | 47.4,8.3 - - Scenario: Empty XML search - When sending xml search query "xnznxvcx" - Then result header contains - | attr | value - | querystring | xnznxvcx - | polygon | false - | more_url | .*format=xml.*q=xnznxvcx.* - - Scenario: Empty XML search with special XML characters - When sending xml search query "xfdghn&zxn"xvbyxcssdex" - Then result header contains - | attr | value - | querystring | xfdghn&zxn"xvbyxcssdex - | polygon | false - | more_url | .*format=xml.*q=xfdghn&zxn"xvbyxcssdex.* - - Scenario: Empty XML search with viewbox - Given the request parameters - | viewbox - | 12,45.13,77,33 - When sending xml search query "xnznxvcx" - Then result header contains - | attr | value - | querystring | xnznxvcx - | polygon | false - | viewbox | 12,45.13,77,33 - - Scenario: Empty XML search with viewboxlbrt - Given the request parameters - | viewboxlbrt - | 12,34.13,77,45 - When sending xml search query "xnznxvcx" - Then result header contains - | attr | value - | querystring | xnznxvcx - | polygon | false - | viewbox | 12,45.13,77,33 - - Scenario: Empty XML search with viewboxlbrt and viewbox - Given the request parameters - | viewbox | viewboxblrt - | 12,45.13,77,33 | 1,2,3,4 - When sending xml search query "pub" - Then result header contains - | attr | value - | querystring | pub - | polygon | false - | viewbox | 12,45.13,77,33 - - - Scenario Outline: Empty XML search with polygon values - Given the request parameters - | polygon - | - When sending xml search query "xnznxvcx" - Then result header contains - | attr | value - | polygon | - - Examples: - | result | polyval - | false | 0 - | true | 1 - | true | True - | true | true - | true | false - | true | FALSE - | true | yes - | true | no - | true | '; delete from foobar; select ' - - - Scenario: Empty XML search with exluded place ids - Given the request parameters - | exclude_place_ids - | 123,76,342565 - When sending xml search query "jghrleoxsbwjer" - Then result header contains - | attr | value - | exclude_place_ids | 123,76,342565 - - Scenario: Empty XML search with bad exluded place ids - Given the request parameters - | exclude_place_ids - | , - When sending xml search query "jghrleoxsbwjer" - Then result header has no attribute exclude_place_ids - - Scenario Outline: Wrapping of legal jsonp search requests - Given the request parameters - | json_callback - | - When sending json search query "Tokyo" - Then there is a json wrapper "" - - Examples: - | data - | foo - | FOO - | __world - | $me - | m1[4] - | d_r[$d] - - Scenario Outline: Wrapping of illegal jsonp search requests - Given the request parameters - | json_callback - | - When sending json search query "Tokyo" - Then a HTTP 400 is returned - - Examples: - | data - | 1asd - | bar(foo) - | XXX['bad'] - | foo; evil - - Scenario Outline: Ignore jsonp parameter for anything but json - Given the request parameters - | json_callback - | 234 - When sending json search query "Malibu" - Then a HTTP 400 is returned - Given the request parameters - | json_callback - | 234 - When sending xml search query "Malibu" - Then the result is valid xml - Given the request parameters - | json_callback - | 234 - When sending html search query "Malibu" - Then the result is valid html - - Scenario: Empty JSON search - When sending json search query "YHlERzzx" - Then exactly 0 results are returned - - Scenario: Empty JSONv2 search - When sending jsonv2 search query "Flubb XdfESSaZx" - Then exactly 0 results are returned - - Scenario: Search for non-existing coordinates - When sending json search query "-21.0,-33.0" - Then exactly 0 results are returned - diff --git a/tests/features/api/search_structured.feature b/tests/features/api/search_structured.feature deleted file mode 100644 index 27e5d344..00000000 --- a/tests/features/api/search_structured.feature +++ /dev/null @@ -1,41 +0,0 @@ -Feature: Structured search queries - Testing correctness of results with - structured queries - - Scenario: Country only - When sending json structured query with address - | country - | Canada - Then address of result 0 is - | type | value - | country | Canada - | country_code | ca - - Scenario: Postcode only - When sending json structured query with address - | postalcode - | 22547 - Then at least 1 result is returned - And results contain - | type - | post(al_)?code - And result addresses contain - | postcode - | 22547 - - - Scenario: Street, postcode and country - When sending xml structured query with address - | street | postalcode | country - | Old Palace Road | GU2 7UP | United Kingdom - Then at least 1 result is returned - Then result header contains - | attr | value - | querystring | Old Palace Road, GU2 7UP, United Kingdom - - - Scenario: gihub #176 - When sending json structured query with address - | city - | Washington - Then at least 1 result is returned diff --git a/tests/features/db/import/interpolation.feature b/tests/features/db/import/interpolation.feature deleted file mode 100644 index 6974e7be..00000000 --- a/tests/features/db/import/interpolation.feature +++ /dev/null @@ -1,327 +0,0 @@ -@DB -Feature: Import of address interpolations - Tests that interpolated addresses are added correctly - - Scenario: Simple even interpolation line with two points - Given the place nodes - | osm_id | osm_type | class | type | housenumber | geometry - | 1 | N | place | house | 2 | 1 1 - | 2 | N | place | house | 6 | 1 1.001 - And the place ways - | osm_id | osm_type | class | type | housenumber | geometry - | 1 | W | place | houses | even | 1 1, 1 1.001 - And the ways - | id | nodes - | 1 | 1,2 - When importing - Then way 1 expands to lines - | startnumber | endnumber | geometry - | 2 | 6 | 1 1, 1 1.001 - - Scenario: Backwards even two point interpolation line - Given the place nodes - | osm_id | class | type | housenumber | geometry - | 1 | place | house | 2 | 1 1 - | 2 | place | house | 6 | 1 1.001 - And the place ways - | osm_id | class | type | housenumber | geometry - | 1 | place | houses | even | 1 1.001, 1 1 - And the ways - | id | nodes - | 1 | 2,1 - When importing - Then way 1 expands to lines - | startnumber | endnumber | geometry - | 2 | 6 | 1 1, 1 1.001 - - Scenario: Simple odd two point interpolation - Given the place nodes - | osm_id | class | type | housenumber | geometry - | 1 | place | house | 1 | 1 1 - | 2 | place | house | 11 | 1 1.001 - And the place ways - | osm_id | class | type | housenumber | geometry - | 1 | place | houses | odd | 1 1, 1 1.001 - And the ways - | id | nodes - | 1 | 1,2 - When importing - Then way 1 expands to lines - | startnumber | endnumber | geometry - | 1 | 11 | 1 1, 1 1.001 - - Scenario: Simple all two point interpolation - Given the place nodes - | osm_id | class | type | housenumber | geometry - | 1 | place | house | 1 | 1 1 - | 2 | place | house | 3 | 1 1.001 - And the place ways - | osm_id | class | type | housenumber | geometry - | 1 | place | houses | all | 1 1, 1 1.001 - And the ways - | id | nodes - | 1 | 1,2 - When importing - Then way 1 expands to lines - | startnumber | endnumber | geometry - | 1 | 3 | 1 1, 1 1.001 - - Scenario: Even two point interpolation line with intermediate empty node - Given the place nodes - | osm_id | class | type | housenumber | geometry - | 1 | place | house | 2 | 1 1 - | 2 | place | house | 10 | 1.001 1.001 - And the place ways - | osm_id | class | type | housenumber | geometry - | 1 | place | houses | even | 1 1, 1 1.001, 1.001 1.001 - And the ways - | id | nodes - | 1 | 1,3,2 - When importing - Then way 1 expands to lines - | startnumber | endnumber | geometry - | 2 | 10 | 1 1, 1 1.001, 1.001 1.001 - - Scenario: Even two point interpolation line with intermediate duplicated empty node - Given the place nodes - | osm_id | class | type | housenumber | geometry - | 1 | place | house | 2 | 1 1 - | 2 | place | house | 10 | 1.001 1.001 - And the place ways - | osm_id | class | type | housenumber | geometry - | 1 | place | houses | even | 1 1, 1 1.001, 1.001 1.001 - And the ways - | id | nodes - | 1 | 1,3,3,2 - When importing - Then way 1 expands to lines - | startnumber | endnumber | geometry - | 2 | 10 | 1 1, 1 1.001, 1.001 1.001 - - Scenario: Simple even three point interpolation line - Given the place nodes - | osm_id | class | type | housenumber | geometry - | 1 | place | house | 2 | 1 1 - | 2 | place | house | 14 | 1.001 1.001 - | 3 | place | house | 10 | 1 1.001 - And the place ways - | osm_id | class | type | housenumber | geometry - | 1 | place | houses | even | 1 1, 1 1.001, 1.001 1.001 - And the ways - | id | nodes - | 1 | 1,3,2 - When importing - Then way 1 expands to lines - | startnumber | endnumber | geometry - | 2 | 10 | 1 1, 1 1.001 - | 10 | 14 | 1 1.001, 1.001 1.001 - - Scenario: Simple even four point interpolation line - Given the place nodes - | osm_id | class | type | housenumber | geometry - | 1 | place | house | 2 | 1 1 - | 2 | place | house | 14 | 1.001 1.001 - | 3 | place | house | 10 | 1 1.001 - | 4 | place | house | 18 | 1.001 1.002 - And the place ways - | osm_id | class | type | housenumber | geometry - | 1 | place | houses | even | 1 1, 1 1.001, 1.001 1.001, 1.001 1.002 - And the ways - | id | nodes - | 1 | 1,3,2,4 - When importing - Then way 1 expands to lines - | startnumber | endnumber | geometry - | 2 | 10 | 1 1, 1 1.001 - | 10 | 14 | 1 1.001, 1.001 1.001 - | 14 | 18 | 1.001 1.001, 1.001 1.002 - - Scenario: Reverse simple even three point interpolation line - Given the place nodes - | osm_id | class | type | housenumber | geometry - | 1 | place | house | 2 | 1 1 - | 2 | place | house | 14 | 1.001 1.001 - | 3 | place | house | 10 | 1 1.001 - And the place ways - | osm_id | class | type | housenumber | geometry - | 1 | place | houses | even | 1.001 1.001, 1 1.001, 1 1 - And the ways - | id | nodes - | 1 | 2,3,1 - When importing - Then way 1 expands to lines - | startnumber | endnumber | geometry - | 2 | 10 | 1 1, 1 1.001 - | 10 | 14 | 1 1.001, 1.001 1.001 - - Scenario: Even three point interpolation line with odd center point - Given the place nodes - | osm_id | class | type | housenumber | geometry - | 1 | place | house | 2 | 1 1 - | 2 | place | house | 8 | 1.001 1.001 - | 3 | place | house | 7 | 1 1.001 - And the place ways - | osm_id | class | type | housenumber | geometry - | 1 | place | houses | even | 1 1, 1 1.001, 1.001 1.001 - And the ways - | id | nodes - | 1 | 1,3,2 - When importing - Then way 1 expands to lines - | startnumber | endnumber | geometry - | 2 | 7 | 1 1, 1 1.001 - | 7 | 8 | 1 1.001, 1.001 1.001 - - Scenario: Interpolation line with self-intersecting way - Given the place nodes - | osm_id | class | type | housenumber | geometry - | 1 | place | house | 2 | 0 0 - | 2 | place | house | 6 | 0 0.001 - | 3 | place | house | 10 | 0 0.002 - And the place ways - | osm_id | class | type | housenumber | geometry - | 1 | place | houses | even | 0 0, 0 0.001, 0 0.002, 0 0.001 - And the ways - | id | nodes - | 1 | 1,2,3,2 - When importing - Then way 1 expands to lines - | startnumber | endnumber | geometry - | 2 | 6 | 0 0, 0 0.001 - | 6 | 10 | 0 0.001, 0 0.002 - | 6 | 10 | 0 0.001, 0 0.002 - - Scenario: Interpolation line with self-intersecting way II - Given the place nodes - | osm_id | class | type | housenumber | geometry - | 1 | place | house | 2 | 0 0 - | 2 | place | house | 6 | 0 0.001 - And the place ways - | osm_id | class | type | housenumber | geometry - | 1 | place | houses | even | 0 0, 0 0.001, 0 0.002, 0 0.001 - And the ways - | id | nodes - | 1 | 1,2,3,2 - When importing - Then way 1 expands to lines - | startnumber | endnumber | geometry - | 2 | 6 | 0 0, 0 0.001 - - Scenario: addr:street on interpolation way - Given the scene parallel-road - And the place nodes - | osm_id | class | type | housenumber | geometry - | 1 | place | house | 2 | :n-middle-w - | 2 | place | house | 6 | :n-middle-e - | 3 | place | house | 12 | :n-middle-w - | 4 | place | house | 16 | :n-middle-e - And the place ways - | osm_id | class | type | housenumber | street | geometry - | 10 | place | houses | even | | :w-middle - | 11 | place | houses | even | Cloud Street | :w-middle - And the place ways - | osm_id | class | type | name | geometry - | 2 | highway | tertiary | 'name' : 'Sun Way' | :w-north - | 3 | highway | tertiary | 'name' : 'Cloud Street' | :w-south - And the ways - | id | nodes - | 10 | 1,100,101,102,2 - | 11 | 3,200,201,202,4 - When importing - Then table placex contains - | object | parent_place_id - | N1 | W2 - | N2 | W2 - | N3 | W3 - | N4 | W3 - Then table location_property_osmline contains - | object | parent_place_id | startnumber | endnumber - | W10 | W2 | 2 | 6 - | W11 | W3 | 12 | 16 - When sending query "16 Cloud Street" - Then results contain - | ID | osm_type | osm_id - | 0 | N | 4 - When sending query "14 Cloud Street" - Then results contain - | ID | osm_type | osm_id - | 0 | W | 11 - When sending query "18 Cloud Street" - Then results contain - | ID | osm_type | osm_id - | 0 | W | 3 - - Scenario: addr:street on housenumber way - Given the scene parallel-road - And the place nodes - | osm_id | class | type | housenumber | street | geometry - | 1 | place | house | 2 | | :n-middle-w - | 2 | place | house | 6 | | :n-middle-e - | 3 | place | house | 12 | Cloud Street | :n-middle-w - | 4 | place | house | 16 | Cloud Street | :n-middle-e - And the place ways - | osm_id | class | type | housenumber | geometry - | 10 | place | houses | even | :w-middle - | 11 | place | houses | even | :w-middle - And the place ways - | osm_id | class | type | name | geometry - | 2 | highway | tertiary | 'name' : 'Sun Way' | :w-north - | 3 | highway | tertiary | 'name' : 'Cloud Street' | :w-south - And the ways - | id | nodes - | 10 | 1,100,101,102,2 - | 11 | 3,200,201,202,4 - When importing - Then table placex contains - | object | parent_place_id - | N1 | W2 - | N2 | W2 - | N3 | W3 - | N4 | W3 - And table location_property_osmline contains - | object | parent_place_id | startnumber | endnumber - | W10 | W2 | 2 | 6 - | W11 | W3 | 12 | 16 - When sending query "16 Cloud Street" - Then results contain - | ID | osm_type | osm_id - | 0 | N | 4 - When sending query "14 Cloud Street" - Then results contain - | ID | osm_type | osm_id - | 0 | W | 11 - - Scenario: Geometry of points and way don't match (github #253) - Given the place nodes - | osm_id | class | type | housenumber | geometry - | 1 | place | house | 10 | 144.9632341 -37.76163 - | 2 | place | house | 6 | 144.9630541 -37.7628174 - | 3 | shop | supermarket | 2 | 144.9629794 -37.7630755 - And the place ways - | osm_id | class | type | housenumber | geometry - | 1 | place | houses | even | 144.9632341 -37.76163,144.9630541 -37.7628172,144.9629794 -37.7630755 - And the ways - | id | nodes - | 1 | 1,2,3 - When importing - Then way 1 expands to lines - | startnumber | endnumber | geometry - | 2 | 6 | 144.9629794 -37.7630755, 144.9630541 -37.7628174 - | 6 | 10 | 144.9630541 -37.7628174, 144.9632341 -37.76163 - - Scenario: Place with missing address information - Given the place nodes - | osm_id | class | type | housenumber | geometry - | 1 | place | house | 23 | 0.0001 0.0001 - | 2 | amenity | school | | 0.0001 0.0002 - | 3 | place | house | 29 | 0.0001 0.0004 - And the place ways - | osm_id | class | type | housenumber | geometry - | 1 | place | houses | odd | 0.0001 0.0001,0.0001 0.0002,0.0001 0.0004 - And the ways - | id | nodes - | 1 | 1,2,3 - When importing - Then way 1 expands to lines - | startnumber | endnumber | geometry - | 23 | 29 | 0.0001 0.0001, 0.0001 0.0002, 0.0001 0.0004 diff --git a/tests/features/db/import/linking.feature b/tests/features/db/import/linking.feature deleted file mode 100644 index 299087ae..00000000 --- a/tests/features/db/import/linking.feature +++ /dev/null @@ -1,112 +0,0 @@ -@DB -Feature: Linking of places - Tests for correctly determining linked places - - Scenario: Only address-describing places can be linked - Given the scene way-area-with-center - And the place areas - | osm_type | osm_id | class | type | name | geometry - | R | 13 | landuse | forest | Garbo | :area - And the place nodes - | osm_id | class | type | name | geometry - | 256 | natural | peak | Garbo | :inner-C - When importing - Then table placex contains - | object | linked_place_id - | R13 | None - | N256 | None - - Scenario: Waterways are linked when in waterway relations - Given the scene split-road - And the place ways - | osm_type | osm_id | class | type | name | geometry - | W | 1 | waterway | river | Rhein | :w-2 - | W | 2 | waterway | river | Rhein | :w-3 - | R | 13 | waterway | river | Rhein | :w-1 + :w-2 + :w-3 - | R | 23 | waterway | river | Limmat| :w-4a - And the relations - | id | members | tags - | 13 | R23:tributary,W1,W2:main_stream | 'type' : 'waterway' - When importing - Then table placex contains - | object | linked_place_id - | W1 | R13 - | W2 | R13 - | R13 | None - | R23 | None - When sending query "rhein" - Then results contain - | osm_type - | R - - Scenario: Relations are not linked when in waterway relations - Given the scene split-road - And the place ways - | osm_type | osm_id | class | type | name | geometry - | W | 1 | waterway | river | Rhein | :w-2 - | W | 2 | waterway | river | Rhein | :w-3 - | R | 1 | waterway | river | Rhein | :w-1 + :w-2 + :w-3 - | R | 2 | waterway | river | Limmat| :w-4a - And the relations - | id | members | tags - | 1 | R2 | 'type' : 'waterway' - When importing - Then table placex contains - | object | linked_place_id - | W1 | None - | W2 | None - | R1 | None - | R2 | None - - Scenario: Empty waterway relations are handled correctly - Given the scene split-road - And the place ways - | osm_type | osm_id | class | type | name | geometry - | R | 1 | waterway | river | Rhein | :w-1 + :w-2 + :w-3 - And the relations - | id | members | tags - | 1 | | 'type' : 'waterway' - When importing - Then table placex contains - | object | linked_place_id - | R1 | None - - Scenario: Waterways are not linked when waterway types don't match - Given the scene split-road - And the place ways - | osm_type | osm_id | class | type | name | geometry - | W | 1 | waterway | drain | Rhein | :w-2 - | R | 1 | waterway | river | Rhein | :w-1 + :w-2 + :w-3 - And the relations - | id | members | tags - | 1 | N23,N34,W1,R45 | 'type' : 'multipolygon' - When importing - Then table placex contains - | object | linked_place_id - | W1 | None - | R1 | None - When sending query "rhein" - Then results contain - | ID | osm_type - | 0 | R - | 1 | W - - Scenario: Side streams are linked only when they have the same name - Given the scene split-road - And the place ways - | osm_type | osm_id | class | type | name | geometry - | W | 1 | waterway | river | Rhein2 | :w-2 - | W | 2 | waterway | river | Rhein | :w-3 - | R | 1 | waterway | river | Rhein | :w-1 + :w-2 + :w-3 - And the relations - | id | members | tags - | 1 | W1:side_stream,W2:side_stream | 'type' : 'waterway' - When importing - Then table placex contains - | object | linked_place_id - | W1 | None - | W2 | R1 - When sending query "rhein2" - Then results contain - | osm_type - | W diff --git a/tests/features/db/import/naming.feature b/tests/features/db/import/naming.feature deleted file mode 100644 index 64a3f8b1..00000000 --- a/tests/features/db/import/naming.feature +++ /dev/null @@ -1,193 +0,0 @@ -@DB -Feature: Import and search of names - Tests all naming related issues: normalisation, - abbreviations, internationalisation, etc. - - - Scenario: Case-insensitivity of search - Given the place nodes - | osm_id | class | type | name - | 1 | place | locality | 'name' : 'FooBar' - When importing - Then table placex contains - | object | class | type | name - | N1 | place | locality | 'name' : 'FooBar' - When sending query "FooBar" - Then results contain - | ID | osm_type | osm_id - | 0 | N | 1 - When sending query "foobar" - Then results contain - | ID | osm_type | osm_id - | 0 | N | 1 - When sending query "fOObar" - Then results contain - | ID | osm_type | osm_id - | 0 | N | 1 - When sending query "FOOBAR" - Then results contain - | ID | osm_type | osm_id - | 0 | N | 1 - - Scenario: Multiple spaces in name - Given the place nodes - | osm_id | class | type | name - | 1 | place | locality | 'name' : 'one two three' - When importing - When sending query "one two three" - Then results contain - | ID | osm_type | osm_id - | 0 | N | 1 - When sending query "one two three" - Then results contain - | ID | osm_type | osm_id - | 0 | N | 1 - When sending query "one two three" - Then results contain - | ID | osm_type | osm_id - | 0 | N | 1 - When sending query " one two three" - Then results contain - | ID | osm_type | osm_id - | 0 | N | 1 - - Scenario: Special characters in name - Given the place nodes - | osm_id | class | type | name - | 1 | place | locality | 'name' : 'Jim-Knopf-Str' - | 2 | place | locality | 'name' : 'Smith/Weston' - | 3 | place | locality | 'name' : 'space mountain' - | 4 | place | locality | 'name' : 'space' - | 5 | place | locality | 'name' : 'mountain' - When importing - When sending query "Jim-Knopf-Str" - Then results contain - | ID | osm_type | osm_id - | 0 | N | 1 - When sending query "Jim Knopf-Str" - Then results contain - | ID | osm_type | osm_id - | 0 | N | 1 - When sending query "Jim Knopf Str" - Then results contain - | ID | osm_type | osm_id - | 0 | N | 1 - When sending query "Jim/Knopf-Str" - Then results contain - | ID | osm_type | osm_id - | 0 | N | 1 - When sending query "Jim-Knopfstr" - Then results contain - | ID | osm_type | osm_id - | 0 | N | 1 - When sending query "Smith/Weston" - Then results contain - | ID | osm_type | osm_id - | 0 | N | 2 - When sending query "Smith Weston" - Then results contain - | ID | osm_type | osm_id - | 0 | N | 2 - When sending query "Smith-Weston" - Then results contain - | ID | osm_type | osm_id - | 0 | N | 2 - When sending query "space mountain" - Then results contain - | ID | osm_type | osm_id - | 0 | N | 3 - When sending query "space-mountain" - Then results contain - | ID | osm_type | osm_id - | 0 | N | 3 - When sending query "space/mountain" - Then results contain - | ID | osm_type | osm_id - | 0 | N | 3 - When sending query "space\mountain" - Then results contain - | ID | osm_type | osm_id - | 0 | N | 3 - When sending query "space(mountain)" - Then results contain - | ID | osm_type | osm_id - | 0 | N | 3 - - Scenario: No copying name tag if only one name - Given the place nodes - | osm_id | class | type | name | geometry - | 1 | place | locality | 'name' : 'german' | country:de - When importing - Then table placex contains - | object | calculated_country_code | - | N1 | de - And table placex contains as names for N1 - | object | k | v - | N1 | name | german - - Scenario: Copying name tag to default language if it does not exist - Given the place nodes - | osm_id | class | type | name | geometry - | 1 | place | locality | 'name' : 'german', 'name:fi' : 'finnish' | country:de - When importing - Then table placex contains - | object | calculated_country_code | - | N1 | de - And table placex contains as names for N1 - | k | v - | name | german - | name:fi | finnish - | name:de | german - - Scenario: Copying default language name tag to name if it does not exist - Given the place nodes - | osm_id | class | type | name | geometry - | 1 | place | locality | 'name:de' : 'german', 'name:fi' : 'finnish' | country:de - When importing - Then table placex contains - | object | calculated_country_code | - | N1 | de - And table placex contains as names for N1 - | k | v - | name | german - | name:fi | finnish - | name:de | german - - Scenario: Do not overwrite default language with name tag - Given the place nodes - | osm_id | class | type | name | geometry - | 1 | place | locality | 'name' : 'german', 'name:fi' : 'finnish', 'name:de' : 'local' | country:de - When importing - Then table placex contains - | object | calculated_country_code | - | N1 | de - And table placex contains as names for N1 - | k | v - | name | german - | name:fi | finnish - | name:de | local - - Scenario: Landuse with name are found - Given the place areas - | osm_type | osm_id | class | type | name | geometry - | R | 1 | natural | meadow | 'name' : 'landuse1' | (0 0, 1 0, 1 1, 0 1, 0 0) - | R | 2 | landuse | industrial | 'name' : 'landuse2' | (0 0, -1 0, -1 -1, 0 -1, 0 0) - When importing - When sending query "landuse1" - Then results contain - | ID | osm_type | osm_id - | 0 | R | 1 - When sending query "landuse2" - Then results contain - | ID | osm_type | osm_id - | 0 | R | 2 - - Scenario: Postcode boundaries without ref - Given the place areas - | osm_type | osm_id | class | type | postcode | geometry - | R | 1 | boundary | postal_code | 12345 | (0 0, 1 0, 1 1, 0 1, 0 0) - When importing - When sending query "12345" - Then results contain - | ID | osm_type | osm_id - | 0 | R | 1 diff --git a/tests/features/db/import/parenting.feature b/tests/features/db/import/parenting.feature deleted file mode 100644 index 36754f84..00000000 --- a/tests/features/db/import/parenting.feature +++ /dev/null @@ -1,458 +0,0 @@ -@DB -Feature: Parenting of objects - Tests that the correct parent is choosen - - Scenario: Address inherits postcode from its street unless it has a postcode - Given the scene roads-with-pois - And the place nodes - | osm_id | class | type | housenumber | geometry - | 1 | place | house | 4 | :p-N1 - And the place nodes - | osm_id | class | type | housenumber | postcode | geometry - | 2 | place | house | 5 | 99999 | :p-N1 - And the place ways - | osm_id | class | type | name | postcode | geometry - | 1 | highway | residential | galoo | 12345 | :w-north - When importing - Then table placex contains - | object | parent_place_id - | N1 | W1 - | N2 | W1 - When sending query "4 galoo" - Then results contain - | ID | osm_type | osm_id | langaddress - | 0 | N | 1 | 4, galoo, 12345 - When sending query "5 galoo" - Then results contain - | ID | osm_type | osm_id | langaddress - | 0 | N | 2 | 5, galoo, 99999 - - - Scenario: Address without tags, closest street - Given the scene roads-with-pois - And the place nodes - | osm_id | class | type | geometry - | 1 | place | house | :p-N1 - | 2 | place | house | :p-N2 - | 3 | place | house | :p-S1 - | 4 | place | house | :p-S2 - And the named place ways - | osm_id | class | type | geometry - | 1 | highway | residential | :w-north - | 2 | highway | residential | :w-south - When importing - Then table placex contains - | object | parent_place_id - | N1 | W1 - | N2 | W1 - | N3 | W2 - | N4 | W2 - - Scenario: Address without tags avoids unnamed streets - Given the scene roads-with-pois - And the place nodes - | osm_id | class | type | geometry - | 1 | place | house | :p-N1 - | 2 | place | house | :p-N2 - | 3 | place | house | :p-S1 - | 4 | place | house | :p-S2 - And the place ways - | osm_id | class | type | geometry - | 1 | highway | residential | :w-north - And the named place ways - | osm_id | class | type | geometry - | 2 | highway | residential | :w-south - When importing - Then table placex contains - | object | parent_place_id - | N1 | W2 - | N2 | W2 - | N3 | W2 - | N4 | W2 - - Scenario: addr:street tag parents to appropriately named street - Given the scene roads-with-pois - And the place nodes - | osm_id | class | type | street| geometry - | 1 | place | house | south | :p-N1 - | 2 | place | house | north | :p-N2 - | 3 | place | house | south | :p-S1 - | 4 | place | house | north | :p-S2 - And the place ways - | osm_id | class | type | name | geometry - | 1 | highway | residential | north | :w-north - | 2 | highway | residential | south | :w-south - When importing - Then table placex contains - | object | parent_place_id - | N1 | W2 - | N2 | W1 - | N3 | W2 - | N4 | W1 - - Scenario: addr:street tag parents to next named street - Given the scene roads-with-pois - And the place nodes - | osm_id | class | type | street | geometry - | 1 | place | house | abcdef | :p-N1 - | 2 | place | house | abcdef | :p-N2 - | 3 | place | house | abcdef | :p-S1 - | 4 | place | house | abcdef | :p-S2 - And the place ways - | osm_id | class | type | name | geometry - | 1 | highway | residential | abcdef | :w-north - | 2 | highway | residential | abcdef | :w-south - When importing - Then table placex contains - | object | parent_place_id - | N1 | W1 - | N2 | W1 - | N3 | W2 - | N4 | W2 - - Scenario: addr:street tag without appropriately named street - Given the scene roads-with-pois - And the place nodes - | osm_id | class | type | street | geometry - | 1 | place | house | abcdef | :p-N1 - | 2 | place | house | abcdef | :p-N2 - | 3 | place | house | abcdef | :p-S1 - | 4 | place | house | abcdef | :p-S2 - And the place ways - | osm_id | class | type | name | geometry - | 1 | highway | residential | abcde | :w-north - | 2 | highway | residential | abcde | :w-south - When importing - Then table placex contains - | object | parent_place_id - | N1 | W1 - | N2 | W1 - | N3 | W2 - | N4 | W2 - - Scenario: addr:place address - Given the scene road-with-alley - And the place nodes - | osm_id | class | type | addr_place | geometry - | 1 | place | house | myhamlet | :n-alley - And the place nodes - | osm_id | class | type | name | geometry - | 2 | place | hamlet | myhamlet | :n-main-west - And the place ways - | osm_id | class | type | name | geometry - | 1 | highway | residential | myhamlet | :w-main - When importing - Then table placex contains - | object | parent_place_id - | N1 | N2 - - Scenario: addr:street is preferred over addr:place - Given the scene road-with-alley - And the place nodes - | osm_id | class | type | addr_place | street | geometry - | 1 | place | house | myhamlet | mystreet| :n-alley - And the place nodes - | osm_id | class | type | name | geometry - | 2 | place | hamlet | myhamlet | :n-main-west - And the place ways - | osm_id | class | type | name | geometry - | 1 | highway | residential | mystreet | :w-main - When importing - Then table placex contains - | object | parent_place_id - | N1 | W1 - - Scenario: Untagged address in simple associated street relation - Given the scene road-with-alley - And the place nodes - | osm_id | class | type | geometry - | 1 | place | house | :n-alley - | 2 | place | house | :n-corner - | 3 | place | house | :n-main-west - And the place ways - | osm_id | class | type | name | geometry - | 1 | highway | residential | foo | :w-main - | 2 | highway | service | bar | :w-alley - And the relations - | id | members | tags - | 1 | W1:street,N1,N2,N3 | 'type' : 'associatedStreet' - When importing - Then table placex contains - | object | parent_place_id - | N1 | W1 - | N2 | W1 - | N3 | W1 - - Scenario: Avoid unnamed streets in simple associated street relation - Given the scene road-with-alley - And the place nodes - | osm_id | class | type | geometry - | 1 | place | house | :n-alley - | 2 | place | house | :n-corner - | 3 | place | house | :n-main-west - And the named place ways - | osm_id | class | type | geometry - | 1 | highway | residential | :w-main - And the place ways - | osm_id | class | type | geometry - | 2 | highway | residential | :w-alley - And the relations - | id | members | tags - | 1 | N1,N2,N3,W2:street,W1:street | 'type' : 'associatedStreet' - When importing - Then table placex contains - | object | parent_place_id - | N1 | W1 - | N2 | W1 - | N3 | W1 - - ### Scenario 10 - Scenario: Associated street relation overrides addr:street - Given the scene road-with-alley - And the place nodes - | osm_id | class | type | street | geometry - | 1 | place | house | bar | :n-alley - And the place ways - | osm_id | class | type | name | geometry - | 1 | highway | residential | foo | :w-main - | 2 | highway | residential | bar | :w-alley - And the relations - | id | members | tags - | 1 | W1:street,N1,N2,N3 | 'type' : 'associatedStreet' - When importing - Then table placex contains - | object | parent_place_id - | N1 | W1 - - Scenario: Building without tags, closest street from center point - Given the scene building-on-street-corner - And the named place ways - | osm_id | class | type | geometry - | 1 | building | yes | :w-building - | 2 | highway | primary | :w-WE - | 3 | highway | residential | :w-NS - When importing - Then table placex contains - | object | parent_place_id - | W1 | W3 - - Scenario: Building with addr:street tags - Given the scene building-on-street-corner - And the named place ways - | osm_id | class | type | street | geometry - | 1 | building | yes | bar | :w-building - And the place ways - | osm_id | class | type | name | geometry - | 2 | highway | primary | bar | :w-WE - | 3 | highway | residential | foo | :w-NS - When importing - Then table placex contains - | object | parent_place_id - | W1 | W2 - - Scenario: Building with addr:place tags - Given the scene building-on-street-corner - And the place nodes - | osm_id | class | type | name | geometry - | 1 | place | village | bar | :n-outer - And the named place ways - | osm_id | class | type | addr_place | geometry - | 1 | building | yes | bar | :w-building - And the place ways - | osm_id | class | type | name | geometry - | 2 | highway | primary | bar | :w-WE - | 3 | highway | residential | foo | :w-NS - When importing - Then table placex contains - | object | parent_place_id - | W1 | N1 - - Scenario: Building in associated street relation - Given the scene building-on-street-corner - And the named place ways - | osm_id | class | type | geometry - | 1 | building | yes | :w-building - And the place ways - | osm_id | class | type | name | geometry - | 2 | highway | primary | bar | :w-WE - | 3 | highway | residential | foo | :w-NS - And the relations - | id | members | tags - | 1 | W1:house,W2:street | 'type' : 'associatedStreet' - When importing - Then table placex contains - | object | parent_place_id - | W1 | W2 - - Scenario: Building in associated street relation overrides addr:street - Given the scene building-on-street-corner - And the named place ways - | osm_id | class | type | street | geometry - | 1 | building | yes | foo | :w-building - And the place ways - | osm_id | class | type | name | geometry - | 2 | highway | primary | bar | :w-WE - | 3 | highway | residential | foo | :w-NS - And the relations - | id | members | tags - | 1 | W1:house,W2:street | 'type' : 'associatedStreet' - When importing - Then table placex contains - | object | parent_place_id - | W1 | W2 - - Scenario: Wrong member in associated street relation is ignored - Given the scene building-on-street-corner - And the named place nodes - | osm_id | class | type | geometry - | 1 | place | house | :n-outer - And the named place ways - | osm_id | class | type | street | geometry - | 1 | building | yes | foo | :w-building - And the place ways - | osm_id | class | type | name | geometry - | 2 | highway | primary | bar | :w-WE - | 3 | highway | residential | foo | :w-NS - And the relations - | id | members | tags - | 1 | N1:house,W1:street,W3:street | 'type' : 'associatedStreet' - When importing - Then table placex contains - | object | parent_place_id - | N1 | W3 - - Scenario: POIs in building inherit address - Given the scene building-on-street-corner - And the named place nodes - | osm_id | class | type | geometry - | 1 | amenity | bank | :n-inner - | 2 | shop | bakery | :n-edge-NS - | 3 | shop | supermarket| :n-edge-WE - And the place ways - | osm_id | class | type | street | addr_place | housenumber | geometry - | 1 | building | yes | foo | nowhere | 3 | :w-building - And the place ways - | osm_id | class | type | name | geometry - | 2 | highway | primary | bar | :w-WE - | 3 | highway | residential | foo | :w-NS - When importing - Then table placex contains - | object | parent_place_id | street | addr_place | housenumber - | W1 | W3 | foo | nowhere | 3 - | N1 | W3 | foo | nowhere | 3 - | N2 | W3 | foo | nowhere | 3 - | N3 | W3 | foo | nowhere | 3 - - Scenario: POIs don't inherit from streets - Given the scene building-on-street-corner - And the named place nodes - | osm_id | class | type | geometry - | 1 | amenity | bank | :n-inner - And the place ways - | osm_id | class | type | street | addr_place | housenumber | geometry - | 1 | highway | path | foo | nowhere | 3 | :w-building - And the place ways - | osm_id | class | type | name | geometry - | 3 | highway | residential | foo | :w-NS - When importing - Then table placex contains - | object | parent_place_id | street | addr_place | housenumber - | N1 | W3 | None | None | None - - Scenario: POIs with own address do not inherit building address - Given the scene building-on-street-corner - And the named place nodes - | osm_id | class | type | street | geometry - | 1 | amenity | bank | bar | :n-inner - And the named place nodes - | osm_id | class | type | housenumber | geometry - | 2 | shop | bakery | 4 | :n-edge-NS - And the named place nodes - | osm_id | class | type | addr_place | geometry - | 3 | shop | supermarket| nowhere | :n-edge-WE - And the place nodes - | osm_id | class | type | name | geometry - | 4 | place | isolated_dwelling | theplace | :n-outer - And the place ways - | osm_id | class | type | addr_place | housenumber | geometry - | 1 | building | yes | theplace | 3 | :w-building - And the place ways - | osm_id | class | type | name | geometry - | 2 | highway | primary | bar | :w-WE - | 3 | highway | residential | foo | :w-NS - When importing - Then table placex contains - | object | parent_place_id | street | addr_place | housenumber - | W1 | N4 | None | theplace | 3 - | N1 | W2 | bar | None | None - | N2 | W3 | None | None | 4 - | N3 | W2 | None | nowhere | None - - ### Scenario 20 - Scenario: POIs parent a road if they are attached to it - Given the scene points-on-roads - And the named place nodes - | osm_id | class | type | street | geometry - | 1 | highway | bus_stop | North St | :n-SE - | 2 | highway | bus_stop | South St | :n-NW - | 3 | highway | bus_stop | North St | :n-S-unglued - | 4 | highway | bus_stop | South St | :n-N-unglued - And the place ways - | osm_id | class | type | name | geometry - | 1 | highway | secondary | North St | :w-north - | 2 | highway | unclassified | South St | :w-south - And the ways - | id | nodes - | 1 | 100,101,2,103,104 - | 2 | 200,201,1,202,203 - When importing - Then table placex contains - | object | parent_place_id - | N1 | W1 - | N2 | W2 - | N3 | W1 - | N4 | W2 - - Scenario: POIs do not parent non-roads they are attached to - Given the scene points-on-roads - And the named place nodes - | osm_id | class | type | street | geometry - | 1 | highway | bus_stop | North St | :n-SE - | 2 | highway | bus_stop | South St | :n-NW - And the place ways - | osm_id | class | type | name | geometry - | 1 | landuse | residential | North St | :w-north - | 2 | waterway| river | South St | :w-south - And the ways - | id | nodes - | 1 | 100,101,2,103,104 - | 2 | 200,201,1,202,203 - When importing - Then table placex contains - | object | parent_place_id - | N1 | 0 - | N2 | 0 - - Scenario: POIs on building outlines inherit associated street relation - Given the scene building-on-street-corner - And the named place nodes - | osm_id | class | type | geometry - | 1 | place | house | :n-edge-NS - And the named place ways - | osm_id | class | type | geometry - | 1 | building | yes | :w-building - And the place ways - | osm_id | class | type | name | geometry - | 2 | highway | primary | bar | :w-WE - | 3 | highway | residential | foo | :w-NS - And the relations - | id | members | tags - | 1 | W1:house,W2:street | 'type' : 'associatedStreet' - And the ways - | id | nodes - | 1 | 100,1,101,102,100 - When importing - Then table placex contains - | object | parent_place_id - | N1 | W2 - diff --git a/tests/features/db/import/placex.feature b/tests/features/db/import/placex.feature deleted file mode 100644 index 95e0bc91..00000000 --- a/tests/features/db/import/placex.feature +++ /dev/null @@ -1,318 +0,0 @@ -@DB -Feature: Import into placex - Tests that data in placex is completed correctly. - - Scenario: No country code tag is available - Given the place nodes - | osm_id | class | type | name | geometry - | 1 | highway | primary | 'name' : 'A1' | country:us - When importing - Then table placex contains - | object | country_code | calculated_country_code | - | N1 | None | us | - - Scenario: Location overwrites country code tag - Given the scene country - And the place nodes - | osm_id | class | type | name | country_code | geometry - | 1 | highway | primary | 'name' : 'A1' | de | :us - When importing - Then table placex contains - | object | country_code | calculated_country_code | - | N1 | de | us | - - Scenario: Country code tag overwrites location for countries - Given the place areas - | osm_type | osm_id | class | type | admin_level | name | country_code | geometry - | R | 1 | boundary | administrative | 2 | 'name' : 'foo' | de | (-100 40, -101 40, -101 41, -100 41, -100 40) - When importing - Then table placex contains - | object | country_code | calculated_country_code | - | R1 | de | de | - - Scenario: Illegal country code tag for countries is ignored - And the place areas - | osm_type | osm_id | class | type | admin_level | name | country_code | geometry - | R | 1 | boundary | administrative | 2 | 'name' : 'foo' | xx | (-100 40, -101 40, -101 41, -100 41, -100 40) - When importing - Then table placex contains - | object | country_code | calculated_country_code | - | R1 | xx | us | - - Scenario: admin level is copied over - Given the place nodes - | osm_id | class | type | admin_level | name - | 1 | place | state | 3 | 'name' : 'foo' - When importing - Then table placex contains - | object | admin_level | - | N1 | 3 | - - Scenario: admin level is default 15 - Given the place nodes - | osm_id | class | type | name - | 1 | amenity | prison | 'name' : 'foo' - When importing - Then table placex contains - | object | admin_level | - | N1 | 15 | - - Scenario: admin level is never larger than 15 - Given the place nodes - | osm_id | class | type | name | admin_level - | 1 | amenity | prison | 'name' : 'foo' | 16 - When importing - Then table placex contains - | object | admin_level | - | N1 | 15 | - - - Scenario: postcode node without postcode is dropped - Given the place nodes - | osm_id | class | type - | 1 | place | postcode - When importing - Then table placex has no entry for N1 - - Scenario: postcode boundary without postcode is dropped - Given the place areas - | osm_type | osm_id | class | type | geometry - | R | 1 | boundary | postal_code | poly-area:0.1 - When importing - Then table placex has no entry for R1 - - Scenario: search and address ranks for GB post codes correctly assigned - Given the place nodes - | osm_id | class | type | postcode | geometry - | 1 | place | postcode | E45 2CD | country:gb - | 2 | place | postcode | E45 2 | country:gb - | 3 | place | postcode | Y45 | country:gb - When importing - Then table placex contains - | object | postcode | calculated_country_code | rank_search | rank_address - | N1 | E45 2CD | gb | 25 | 5 - | N2 | E45 2 | gb | 23 | 5 - | N3 | Y45 | gb | 21 | 5 - - Scenario: wrongly formatted GB postcodes are down-ranked - Given the place nodes - | osm_id | class | type | postcode | geometry - | 1 | place | postcode | EA452CD | country:gb - | 2 | place | postcode | E45 23 | country:gb - | 3 | place | postcode | y45 | country:gb - When importing - Then table placex contains - | object | calculated_country_code | rank_search | rank_address - | N1 | gb | 30 | 30 - | N2 | gb | 30 | 30 - | N3 | gb | 30 | 30 - - Scenario: search and address rank for DE postcodes correctly assigned - Given the place nodes - | osm_id | class | type | postcode | geometry - | 1 | place | postcode | 56427 | country:de - | 2 | place | postcode | 5642 | country:de - | 3 | place | postcode | 5642A | country:de - | 4 | place | postcode | 564276 | country:de - When importing - Then table placex contains - | object | calculated_country_code | rank_search | rank_address - | N1 | de | 21 | 11 - | N2 | de | 30 | 30 - | N3 | de | 30 | 30 - | N4 | de | 30 | 30 - - Scenario: search and address rank for other postcodes are correctly assigned - Given the place nodes - | osm_id | class | type | postcode | geometry - | 1 | place | postcode | 1 | country:ca - | 2 | place | postcode | X3 | country:ca - | 3 | place | postcode | 543 | country:ca - | 4 | place | postcode | 54dc | country:ca - | 5 | place | postcode | 12345 | country:ca - | 6 | place | postcode | 55TT667 | country:ca - | 7 | place | postcode | 123-65 | country:ca - | 8 | place | postcode | 12 445 4 | country:ca - | 9 | place | postcode | A1:bc10 | country:ca - When importing - Then table placex contains - | object | calculated_country_code | rank_search | rank_address - | N1 | ca | 21 | 11 - | N2 | ca | 21 | 11 - | N3 | ca | 21 | 11 - | N4 | ca | 21 | 11 - | N5 | ca | 21 | 11 - | N6 | ca | 21 | 11 - | N7 | ca | 25 | 11 - | N8 | ca | 25 | 11 - | N9 | ca | 25 | 11 - - - Scenario: search and address ranks for places are correctly assigned - Given the named place nodes - | osm_id | class | type | - | 1 | foo | bar | - | 11 | place | Continent | - | 12 | place | continent | - | 13 | place | sea | - | 14 | place | country | - | 15 | place | state | - | 16 | place | region | - | 17 | place | county | - | 18 | place | city | - | 19 | place | island | - | 20 | place | town | - | 21 | place | village | - | 22 | place | hamlet | - | 23 | place | municipality | - | 24 | place | district | - | 25 | place | unincorporated_area | - | 26 | place | borough | - | 27 | place | suburb | - | 28 | place | croft | - | 29 | place | subdivision | - | 30 | place | isolated_dwelling | - | 31 | place | farm | - | 32 | place | locality | - | 33 | place | islet | - | 34 | place | mountain_pass | - | 35 | place | neighbourhood | - | 36 | place | house | - | 37 | place | building | - | 38 | place | houses | - And the named place nodes - | osm_id | class | type | extratags - | 100 | place | locality | 'locality' : 'townland' - | 101 | place | city | 'capital' : 'yes' - When importing - Then table placex contains - | object | rank_search | rank_address | - | N1 | 30 | 30 | - | N11 | 30 | 30 | - | N12 | 2 | 2 | - | N13 | 2 | 0 | - | N14 | 4 | 4 | - | N15 | 8 | 8 | - | N16 | 18 | 0 | - | N17 | 12 | 12 | - | N18 | 16 | 16 | - | N19 | 17 | 0 | - | N20 | 18 | 16 | - | N21 | 19 | 16 | - | N22 | 19 | 16 | - | N23 | 19 | 16 | - | N24 | 19 | 16 | - | N25 | 19 | 16 | - | N26 | 19 | 16 | - | N27 | 20 | 20 | - | N28 | 20 | 20 | - | N29 | 20 | 20 | - | N30 | 20 | 20 | - | N31 | 20 | 0 | - | N32 | 20 | 0 | - | N33 | 20 | 0 | - | N34 | 20 | 0 | - | N100 | 20 | 20 | - | N101 | 15 | 16 | - | N35 | 22 | 22 | - | N36 | 30 | 30 | - | N37 | 30 | 30 | - | N38 | 28 | 0 | - - Scenario: search and address ranks for boundaries are correctly assigned - Given the named place nodes - | osm_id | class | type - | 1 | boundary | administrative - And the named place ways - | osm_id | class | type | geometry - | 10 | boundary | administrative | 10 10, 11 11 - And the named place areas - | osm_type | osm_id | class | type | admin_level | geometry - | R | 20 | boundary | administrative | 2 | (1 1, 2 2, 1 2, 1 1) - | R | 21 | boundary | administrative | 32 | (3 3, 4 4, 3 4, 3 3) - | R | 22 | boundary | nature_park | 6 | (0 0, 1 0, 0 1, 0 0) - | R | 23 | boundary | natural_reserve| 10 | (0 0, 1 1, 1 0, 0 0) - When importing - Then table placex has no entry for N1 - And table placex has no entry for W10 - And table placex contains - | object | rank_search | rank_address - | R20 | 4 | 4 - | R21 | 30 | 30 - | R22 | 12 | 0 - | R23 | 20 | 0 - - Scenario: search and address ranks for highways correctly assigned - Given the scene roads-with-pois - And the place nodes - | osm_id | class | type - | 1 | highway | bus_stop - And the place ways - | osm_id | class | type | geometry - | 1 | highway | primary | :w-south - | 2 | highway | secondary | :w-south - | 3 | highway | tertiary | :w-south - | 4 | highway | residential | :w-north - | 5 | highway | unclassified | :w-north - | 6 | highway | something | :w-north - When importing - Then table placex contains - | object | rank_search | rank_address - | N1 | 30 | 30 - | W1 | 26 | 26 - | W2 | 26 | 26 - | W3 | 26 | 26 - | W4 | 26 | 26 - | W5 | 26 | 26 - | W6 | 26 | 26 - - Scenario: rank and inclusion of landuses - And the named place nodes - | osm_id | class | type - | 2 | landuse | residential - And the named place ways - | osm_id | class | type | geometry - | 2 | landuse | residential | 1 1, 1 1.1 - And the named place areas - | osm_type | osm_id | class | type | geometry - | W | 4 | landuse | residential | poly-area:0.1 - | R | 2 | landuse | residential | poly-area:0.05 - | R | 3 | landuse | forrest | poly-area:0.5 - When importing - And table placex contains - | object | rank_search | rank_address - | N2 | 30 | 30 - | W2 | 30 | 30 - | W4 | 22 | 22 - | R2 | 22 | 22 - | R3 | 22 | 0 - - Scenario: rank and inclusion of naturals - And the named place nodes - | osm_id | class | type - | 2 | natural | peak - | 4 | natural | volcano - | 5 | natural | foobar - And the named place ways - | osm_id | class | type | geometry - | 2 | natural | mountain_range | 12 12,11 11 - | 3 | natural | foobar | 13 13,13.1 13 - And the named place areas - | osm_type | osm_id | class | type | geometry - | R | 3 | natural | volcano | poly-area:0.1 - | R | 4 | natural | foobar | poly-area:0.5 - | R | 5 | natural | sea | poly-area:5.0 - | R | 6 | natural | sea | poly-area:0.01 - When importing - And table placex contains - | object | rank_search | rank_address - | N2 | 18 | 0 - | N4 | 18 | 0 - | N5 | 30 | 30 - | W2 | 18 | 0 - | R3 | 18 | 0 - | R4 | 22 | 0 - | R5 | 4 | 4 - | R6 | 4 | 4 - | W3 | 30 | 30 - diff --git a/tests/features/db/import/search_terms.feature b/tests/features/db/import/search_terms.feature deleted file mode 100644 index f68fe61c..00000000 --- a/tests/features/db/import/search_terms.feature +++ /dev/null @@ -1,42 +0,0 @@ -@DB -Feature: Creation of search terms - Tests that search_name table is filled correctly - - Scenario: POIs without a name have no search entry - Given the scene roads-with-pois - And the place nodes - | osm_id | class | type | geometry - | 1 | place | house | :p-N1 - And the place ways - | osm_id | class | type | geometry - | 1 | highway | residential | :w-north - When importing - Then table search_name has no entry for N1 - - - Scenario: Named POIs inherit address from parent - Given the scene roads-with-pois - And the place nodes - | osm_id | class | type | name | geometry - | 1 | place | house | foo | :p-N1 - And the place ways - | osm_id | class | type | name | geometry - | 1 | highway | residential | the road | :w-north - When importing - Then search_name table contains - | place_id | name_vector | nameaddress_vector - | N1 | foo | the road - - Scenario: Roads take over the postcode from attached houses - Given the scene roads-with-pois - And the place nodes - | osm_id | class | type | housenumber | postcode | street | geometry - | 1 | place | house | 1 | 12345 | North St |:p-S1 - And the place ways - | osm_id | class | type | name | geometry - | 1 | highway | residential | North St | :w-north - When importing - Then search_name table contains - | place_id | nameaddress_vector - | W1 | 12345 - diff --git a/tests/features/db/import/simple.feature b/tests/features/db/import/simple.feature deleted file mode 100644 index 2e2c825a..00000000 --- a/tests/features/db/import/simple.feature +++ /dev/null @@ -1,17 +0,0 @@ -@DB -Feature: Import of simple objects - Testing simple stuff - - Scenario: Import place node - Given the place nodes: - | osm_id | class | type | name | geometry - | 1 | place | village | 'name' : 'Foo' | 10.0 -10.0 - When importing - Then table placex contains - | object | class | type | name | centroid - | N1 | place | village | 'name' : 'Foo' | 10.0,-10.0 +- 1m - When sending query "Foo" - Then results contain - | ID | osm_type | osm_id - | 0 | N | 1 - diff --git a/tests/features/db/update/interpolation.feature b/tests/features/db/update/interpolation.feature deleted file mode 100644 index 66367b10..00000000 --- a/tests/features/db/update/interpolation.feature +++ /dev/null @@ -1,258 +0,0 @@ -@DB -Feature: Update of address interpolations - Test the interpolated address are updated correctly - - Scenario: addr:street added to interpolation - Given the scene parallel-road - And the place nodes - | osm_id | class | type | housenumber | geometry - | 1 | place | house | 2 | :n-middle-w - | 2 | place | house | 6 | :n-middle-e - And the place ways - | osm_id | class | type | housenumber | geometry - | 10 | place | houses | even | :w-middle - And the place ways - | osm_id | class | type | name | geometry - | 2 | highway | unclassified | 'name' : 'Sun Way' | :w-north - | 3 | highway | unclassified | 'name' : 'Cloud Street' | :w-south - And the ways - | id | nodes - | 10 | 1,100,101,102,2 - When importing - Then table placex contains - | object | parent_place_id - | N1 | W2 - | N2 | W2 - And table location_property_osmline contains - | object | parent_place_id | startnumber | endnumber - | W10 | W2 | 2 | 6 - When updating place ways - | osm_id | class | type | housenumber | street | geometry - | 10 | place | houses | even | Cloud Street | :w-middle - Then table placex contains - | object | parent_place_id - | N1 | W3 - | N2 | W3 - And table location_property_osmline contains - | object | parent_place_id | startnumber | endnumber - | W10 | W3 | 2 | 6 - - Scenario: addr:street added to housenumbers - Given the scene parallel-road - And the place nodes - | osm_id | class | type | housenumber | geometry - | 1 | place | house | 2 | :n-middle-w - | 2 | place | house | 6 | :n-middle-e - And the place ways - | osm_id | class | type | housenumber | geometry - | 10 | place | houses | even | :w-middle - And the place ways - | osm_id | class | type | name | geometry - | 2 | highway | unclassified | 'name' : 'Sun Way' | :w-north - | 3 | highway | unclassified | 'name' : 'Cloud Street' | :w-south - And the ways - | id | nodes - | 10 | 1,100,101,102,2 - When importing - Then table placex contains - | object | parent_place_id - | N1 | W2 - | N2 | W2 - And table location_property_osmline contains - | object | parent_place_id | startnumber | endnumber - | W10 | W2 | 2 | 6 - When updating place nodes - | osm_id | class | type | street | housenumber | geometry - | 1 | place | house | Cloud Street| 2 | :n-middle-w - | 2 | place | house | Cloud Street| 6 | :n-middle-e - Then table placex contains - | object | parent_place_id - | N1 | W3 - | N2 | W3 - And table location_property_osmline contains - | object | parent_place_id | startnumber | endnumber - | W10 | W3 | 2 | 6 - - - Scenario: interpolation tag removed - Given the scene parallel-road - And the place nodes - | osm_id | class | type | housenumber | geometry - | 1 | place | house | 2 | :n-middle-w - | 2 | place | house | 6 | :n-middle-e - And the place ways - | osm_id | class | type | housenumber | geometry - | 10 | place | houses | even | :w-middle - And the place ways - | osm_id | class | type | name | geometry - | 2 | highway | unclassified | 'name' : 'Sun Way' | :w-north - | 3 | highway | unclassified | 'name' : 'Cloud Street' | :w-south - And the ways - | id | nodes - | 10 | 1,100,101,102,2 - When importing - Then table placex contains - | object | parent_place_id - | N1 | W2 - | N2 | W2 - And table location_property_osmline contains - | object | parent_place_id | startnumber | endnumber - | W10 | W2 | 2 | 6 - When marking for delete W10 - Then table location_property_osmline has no entry for W10 - And table placex contains - | object | parent_place_id - | N1 | W2 - | N2 | W2 - - - Scenario: referenced road added - Given the scene parallel-road - And the place nodes - | osm_id | class | type | housenumber | geometry - | 1 | place | house | 2 | :n-middle-w - | 2 | place | house | 6 | :n-middle-e - And the place ways - | osm_id | class | type | housenumber | street | geometry - | 10 | place | houses | even | Cloud Street| :w-middle - And the place ways - | osm_id | class | type | name | geometry - | 2 | highway | unclassified | 'name' : 'Sun Way' | :w-north - And the ways - | id | nodes - | 10 | 1,100,101,102,2 - When importing - Then table placex contains - | object | parent_place_id - | N1 | W2 - | N2 | W2 - And table location_property_osmline contains - | object | parent_place_id | startnumber | endnumber - | W10 | W2 | 2 | 6 - When updating place ways - | osm_id | class | type | name | geometry - | 3 | highway | unclassified | 'name' : 'Cloud Street' | :w-south - Then table placex contains - | object | parent_place_id - | N1 | W3 - | N2 | W3 - And table location_property_osmline contains - | object | parent_place_id | startnumber | endnumber - | W10 | W3 | 2 | 6 - - - Scenario: referenced road deleted - Given the scene parallel-road - And the place nodes - | osm_id | class | type | housenumber | geometry - | 1 | place | house | 2 | :n-middle-w - | 2 | place | house | 6 | :n-middle-e - And the place ways - | osm_id | class | type | housenumber | street | geometry - | 10 | place | houses | even | Cloud Street| :w-middle - And the place ways - | osm_id | class | type | name | geometry - | 2 | highway | unclassified | 'name' : 'Sun Way' | :w-north - | 3 | highway | unclassified | 'name' : 'Cloud Street' | :w-south - And the ways - | id | nodes - | 10 | 1,100,101,102,2 - When importing - Then table placex contains - | object | parent_place_id - | N1 | W3 - | N2 | W3 - And table location_property_osmline contains - | object | parent_place_id | startnumber | endnumber - | W10 | W3 | 2 | 6 - When marking for delete W3 - Then table placex contains - | object | parent_place_id - | N1 | W2 - | N2 | W2 - And table location_property_osmline contains - | object | parent_place_id | startnumber | endnumber - | W10 | W2 | 2 | 6 - - Scenario: building becomes interpolation - Given the scene building-with-parallel-streets - And the place ways - | osm_id | class | type | housenumber | geometry - | 1 | place | house | 3 | :w-building - And the place ways - | osm_id | class | type | name | geometry - | 2 | highway | unclassified | 'name' : 'Cloud Street' | :w-south - When importing - Then table placex contains - | object | parent_place_id - | W1 | W2 - When updating place nodes - | osm_id | class | type | housenumber | geometry - | 1 | place | house | 2 | :n-north-w - | 2 | place | house | 6 | :n-north-e - And the ways - | id | nodes - | 1 | 1,100,101,102,2 - And updating place ways - | osm_id | class | type | housenumber | street | geometry - | 1 | place | houses | even | Cloud Street| :w-north - Then table placex has no entry for W1 - And table location_property_osmline contains - | object | parent_place_id | startnumber | endnumber - | W1 | W2 | 2 | 6 - - - - Scenario: interpolation becomes building - Given the scene building-with-parallel-streets - And the place nodes - | osm_id | class | type | housenumber | geometry - | 1 | place | house | 2 | :n-north-w - | 2 | place | house | 6 | :n-north-e - And the place ways - | osm_id | class | type | name | geometry - | 2 | highway | unclassified | 'name' : 'Cloud Street' | :w-south - And the ways - | id | nodes - | 1 | 1,100,101,102,2 - And the place ways - | osm_id | class | type | housenumber | street | geometry - | 1 | place | houses | even | Cloud Street| :w-north - When importing - Then table placex has no entry for W1 - And table location_property_osmline contains - | object | parent_place_id | startnumber | endnumber - | W1 | W2 | 2 | 6 - When updating place ways - | osm_id | class | type | housenumber | geometry - | 1 | place | house | 3 | :w-building - Then table placex contains - | object | parent_place_id - | W1 | W2 - - Scenario: housenumbers added to interpolation - Given the scene building-with-parallel-streets - And the place ways - | osm_id | class | type | name | geometry - | 2 | highway | unclassified | 'name' : 'Cloud Street' | :w-south - And the ways - | id | nodes - | 1 | 1,100,101,102,2 - And the place ways - | osm_id | class | type | housenumber | geometry - | 1 | place | houses | even | :w-north - When importing - Then table location_property_osmline has no entry for W1 - When updating place nodes - | osm_id | class | type | housenumber | geometry - | 1 | place | house | 2 | :n-north-w - | 2 | place | house | 6 | :n-north-e - And updating place ways - | osm_id | class | type | housenumber | street | geometry - | 1 | place | houses | even | Cloud Street| :w-north - Then table location_property_osmline contains - | object | parent_place_id | startnumber | endnumber - | W1 | W2 | 2 | 6 - - - diff --git a/tests/features/db/update/linked_places.feature b/tests/features/db/update/linked_places.feature deleted file mode 100644 index 777a02f0..00000000 --- a/tests/features/db/update/linked_places.feature +++ /dev/null @@ -1,96 +0,0 @@ -@DB -Feature: Updates of linked places - Tests that linked places are correctly added and deleted. - - - Scenario: Add linked place when linking relation is renamed - Given the place nodes - | osm_id | class | type | name | geometry - | 1 | place | city | foo | 0 0 - And the place areas - | osm_type | osm_id | class | type | name | admin_level | geometry - | R | 1 | boundary | administrative | foo | 8 | poly-area:0.1 - When importing - And sending query "foo" with dups - Then results contain - | osm_type - | R - When updating place areas - | osm_type | osm_id | class | type | name | admin_level | geometry - | R | 1 | boundary | administrative | foobar | 8 | poly-area:0.1 - Then table placex contains - | object | linked_place_id - | N1 | None - When updating place areas - | osm_type | osm_id | class | type | name | admin_level | geometry - When sending query "foo" with dups - Then results contain - | osm_type - | N - - Scenario: Add linked place when linking relation is removed - Given the place nodes - | osm_id | class | type | name | geometry - | 1 | place | city | foo | 0 0 - And the place areas - | osm_type | osm_id | class | type | name | admin_level | geometry - | R | 1 | boundary | administrative | foo | 8 | poly-area:0.1 - When importing - And sending query "foo" with dups - Then results contain - | osm_type - | R - When marking for delete R1 - Then table placex contains - | object | linked_place_id - | N1 | None - When updating place areas - | osm_type | osm_id | class | type | name | admin_level | geometry - And sending query "foo" with dups - Then results contain - | osm_type - | N - - Scenario: Remove linked place when linking relation is added - Given the place nodes - | osm_id | class | type | name | geometry - | 1 | place | city | foo | 0 0 - When importing - And sending query "foo" with dups - Then results contain - | osm_type - | N - When updating place areas - | osm_type | osm_id | class | type | name | admin_level | geometry - | R | 1 | boundary | administrative | foo | 8 | poly-area:0.1 - Then table placex contains - | object | linked_place_id - | N1 | R1 - When sending query "foo" with dups - Then results contain - | osm_type - | R - - Scenario: Remove linked place when linking relation is renamed - Given the place nodes - | osm_id | class | type | name | geometry - | 1 | place | city | foo | 0 0 - And the place areas - | osm_type | osm_id | class | type | name | admin_level | geometry - | R | 1 | boundary | administrative | foobar | 8 | poly-area:0.1 - When importing - And sending query "foo" with dups - Then results contain - | osm_type - | N - When updating place areas - | osm_type | osm_id | class | type | name | admin_level | geometry - | R | 1 | boundary | administrative | foo | 8 | poly-area:0.1 - Then table placex contains - | object | linked_place_id - | N1 | R1 - When sending query "foo" with dups - Then results contain - | osm_type - | R - diff --git a/tests/features/db/update/naming.feature b/tests/features/db/update/naming.feature deleted file mode 100644 index 261f02dc..00000000 --- a/tests/features/db/update/naming.feature +++ /dev/null @@ -1,39 +0,0 @@ -@DB -Feature: Update of names in place objects - Test all naming related issues in updates - - - Scenario: Updating postcode in postcode boundaries without ref - Given the place areas - | osm_type | osm_id | class | type | postcode | geometry - | R | 1 | boundary | postal_code | 12345 | (0 0, 1 0, 1 1, 0 1, 0 0) - When importing - And sending query "12345" - Then results contain - | ID | osm_type | osm_id - | 0 | R | 1 - When updating place areas - | osm_type | osm_id | class | type | postcode | geometry - | R | 1 | boundary | postal_code | 54321 | (0 0, 1 0, 1 1, 0 1, 0 0) - And sending query "12345" - Then exactly 0 results are returned - When sending query "54321" - Then results contain - | ID | osm_type | osm_id - | 0 | R | 1 - - - Scenario: Delete postcode from postcode boundaries without ref - Given the place areas - | osm_type | osm_id | class | type | postcode | geometry - | R | 1 | boundary | postal_code | 12345 | (0 0, 1 0, 1 1, 0 1, 0 0) - When importing - And sending query "12345" - Then results contain - | ID | osm_type | osm_id - | 0 | R | 1 - When updating place areas - | osm_type | osm_id | class | type | geometry - | R | 1 | boundary | postal_code | (0 0, 1 0, 1 1, 0 1, 0 0) - Then table placex has no entry for R1 - diff --git a/tests/features/db/update/search_terms.feature b/tests/features/db/update/search_terms.feature deleted file mode 100644 index d8c4440a..00000000 --- a/tests/features/db/update/search_terms.feature +++ /dev/null @@ -1,117 +0,0 @@ -@DB -Feature: Update of search terms - Tests that search_name table is filled correctly - - Scenario: POI-inherited postcode remains when way type is changed - Given the scene roads-with-pois - And the place nodes - | osm_id | class | type | housenumber | postcode | street | geometry - | 1 | place | house | 1 | 12345 | North St |:p-S1 - And the place ways - | osm_id | class | type | name | geometry - | 1 | highway | residential | North St | :w-north - When importing - Then search_name table contains - | place_id | nameaddress_vector - | W1 | 12345 - When updating place ways - | osm_id | class | type | name | geometry - | 1 | highway | unclassified | North St | :w-north - Then search_name table contains - | place_id | nameaddress_vector - | W1 | 12345 - - Scenario: POI-inherited postcode remains when way name is changed - Given the scene roads-with-pois - And the place nodes - | osm_id | class | type | housenumber | postcode | street | geometry - | 1 | place | house | 1 | 12345 | North St |:p-S1 - And the place ways - | osm_id | class | type | name | geometry - | 1 | highway | residential | North St | :w-north - When importing - Then search_name table contains - | place_id | nameaddress_vector - | W1 | 12345 - When updating place ways - | osm_id | class | type | name | geometry - | 1 | highway | unclassified | South St | :w-north - Then search_name table contains - | place_id | nameaddress_vector - | W1 | 12345 - - Scenario: POI-inherited postcode remains when way geometry is changed - Given the scene roads-with-pois - And the place nodes - | osm_id | class | type | housenumber | postcode | street | geometry - | 1 | place | house | 1 | 12345 | North St |:p-S1 - And the place ways - | osm_id | class | type | name | geometry - | 1 | highway | residential | North St | :w-north - When importing - Then search_name table contains - | place_id | nameaddress_vector - | W1 | 12345 - When updating place ways - | osm_id | class | type | name | geometry - | 1 | highway | unclassified | South St | :w-south - Then search_name table contains - | place_id | nameaddress_vector - | W1 | 12345 - - Scenario: POI-inherited postcode is added when POI postcode changes - Given the scene roads-with-pois - And the place nodes - | osm_id | class | type | housenumber | postcode | street | geometry - | 1 | place | house | 1 | 12345 | North St |:p-S1 - And the place ways - | osm_id | class | type | name | geometry - | 1 | highway | residential | North St | :w-north - When importing - Then search_name table contains - | place_id | nameaddress_vector - | W1 | 12345 - When updating place nodes - | osm_id | class | type | housenumber | postcode | street | geometry - | 1 | place | house | 1 | 54321 | North St |:p-S1 - Then search_name table contains - | place_id | nameaddress_vector - | W1 | 54321 - - Scenario: POI-inherited postcode remains when POI geometry changes - Given the scene roads-with-pois - And the place nodes - | osm_id | class | type | housenumber | postcode | street | geometry - | 1 | place | house | 1 | 12345 | North St |:p-S1 - And the place ways - | osm_id | class | type | name | geometry - | 1 | highway | residential | North St | :w-north - When importing - Then search_name table contains - | place_id | nameaddress_vector - | W1 | 12345 - When updating place nodes - | osm_id | class | type | housenumber | postcode | street | geometry - | 1 | place | house | 1 | 12345 | North St |:p-S2 - Then search_name table contains - | place_id | nameaddress_vector - | W1 | 12345 - - - Scenario: POI-inherited postcode remains when another POI is deleted - Given the scene roads-with-pois - And the place nodes - | osm_id | class | type | housenumber | postcode | street | geometry - | 1 | place | house | 1 | 12345 | North St |:p-S1 - | 2 | place | house | 2 | | North St |:p-S2 - And the place ways - | osm_id | class | type | name | geometry - | 1 | highway | residential | North St | :w-north - When importing - Then search_name table contains - | place_id | nameaddress_vector - | W1 | 12345 - When marking for delete N2 - Then search_name table contains - | place_id | nameaddress_vector - | W1 | 12345 diff --git a/tests/features/db/update/simple.feature b/tests/features/db/update/simple.feature deleted file mode 100644 index 517e7bde..00000000 --- a/tests/features/db/update/simple.feature +++ /dev/null @@ -1,73 +0,0 @@ -@DB -Feature: Update of simple objects - Testing simple stuff - - Scenario: Do delete small boundary features - Given the place areas - | osm_type | osm_id | class | type | admin_level | geometry - | R | 1 | boundary | administrative | 3 | (0 0, 1 0, 1 1, 0 1, 0 0) - When importing - Then table placex contains - | object | rank_search - | R1 | 6 - When marking for delete R1 - Then table placex has no entry for R1 - - Scenario: Do not delete large boundary features - Given the place areas - | osm_type | osm_id | class | type | admin_level | geometry - | R | 1 | boundary | administrative | 3 | (0 0, 2 0, 2 2.1, 0 2, 0 0) - When importing - Then table placex contains - | object | rank_search - | R1 | 6 - When marking for delete R1 - Then table placex contains - | object | rank_search - | R1 | 6 - - Scenario: Do delete large features of low rank - Given the named place areas - | osm_type | osm_id | class | type | geometry - | W | 1 | place | house | (0 0, 2 0, 2 2.1, 0 2, 0 0) - | R | 1 | boundary | national_park | (0 0, 2 0, 2 2.1, 0 2, 0 0) - When importing - Then table placex contains - | object | rank_address - | R1 | 0 - | W1 | 30 - When marking for delete R1,W1 - Then table placex has no entry for W1 - Then table placex has no entry for R1 - - - Scenario: type mutation - Given the place nodes - | osm_id | class | type | geometry - | 3 | shop | toys | 1 -1 - When importing - Then table placex contains - | object | class | type - | N3 | shop | toys - When updating place nodes - | osm_id | class | type | geometry - | 3 | shop | grocery | 1 -1 - Then table placex contains - | object | class | type - | N3 | shop | grocery - - - Scenario: remove postcode place when house number is added - Given the place nodes - | osm_id | class | type | postcode | geometry - | 3 | place | postcode | 12345 | 1 -1 - When importing - Then table placex contains - | object | class | type - | N3 | place | postcode - When updating place nodes - | osm_id | class | type | postcode | housenumber | geometry - | 3 | place | house | 12345 | 13 | 1 -1 - Then table placex contains - | object | class | type - | N3 | place | house diff --git a/tests/features/osm2pgsql/import/broken.feature b/tests/features/osm2pgsql/import/broken.feature deleted file mode 100644 index 58a45f91..00000000 --- a/tests/features/osm2pgsql/import/broken.feature +++ /dev/null @@ -1,37 +0,0 @@ -@DB -Feature: Import of objects with broken geometries by osm2pgsql - - @Fail - Scenario: Import way with double nodes - Given the osm nodes: - | id | geometry - | 100 | 0 0 - | 101 | 0 0.1 - | 102 | 0.1 0.2 - And the osm ways: - | id | tags | nodes - | 1 | 'highway' : 'primary' | 100 101 101 102 - When loading osm data - Then table place contains - | object | class | type | geometry - | W1 | highway | primary | (0 0, 0 0.1, 0.1 0.2) - - Scenario: Import of ballon areas - Given the osm nodes: - | id | geometry - | 1 | 0 0 - | 2 | 0 0.0001 - | 3 | 0.00001 0.0001 - | 4 | 0.00001 0 - | 5 | -0.00001 0 - And the osm ways: - | id | tags | nodes - | 1 | 'highway' : 'unclassified' | 1 2 3 4 1 5 - | 2 | 'highway' : 'unclassified' | 1 2 3 4 1 - | 3 | 'highway' : 'unclassified' | 1 2 3 4 3 - When loading osm data - Then table place contains - | object | geometrytype - | W1 | ST_LineString - | W2 | ST_Polygon - | W3 | ST_LineString diff --git a/tests/features/osm2pgsql/import/relation.feature b/tests/features/osm2pgsql/import/relation.feature deleted file mode 100644 index aba99a47..00000000 --- a/tests/features/osm2pgsql/import/relation.feature +++ /dev/null @@ -1,13 +0,0 @@ -@DB -Feature: Import of relations by osm2pgsql - Testing specific relation problems related to members. - - Scenario: Don't import empty waterways - Given the osm nodes: - | id | tags - | 1 | 'amenity' : 'prison', 'name' : 'foo' - And the osm relations: - | id | tags | members - | 1 | 'type' : 'waterway', 'waterway' : 'river', 'name' : 'XZ' | N1 - When loading osm data - Then table place has no entry for R1 diff --git a/tests/features/osm2pgsql/import/simple.feature b/tests/features/osm2pgsql/import/simple.feature deleted file mode 100644 index 447dab0d..00000000 --- a/tests/features/osm2pgsql/import/simple.feature +++ /dev/null @@ -1,68 +0,0 @@ -@DB -Feature: Import of simple objects by osm2pgsql - Testing basic tagging in osm2pgsql imports. - - Scenario: Import simple objects - Given the osm nodes: - | id | tags - | 1 | 'amenity' : 'prison', 'name' : 'foo' - Given the osm nodes: - | id | geometry - | 100 | 0 0 - | 101 | 0 0.1 - | 102 | 0.1 0.2 - | 200 | 0 0 - | 201 | 0 1 - | 202 | 1 1 - | 203 | 1 0 - And the osm ways: - | id | tags | nodes - | 1 | 'shop' : 'toys', 'name' : 'tata' | 100 101 102 - | 2 | 'ref' : '45' | 200 201 202 203 200 - And the osm relations: - | id | tags | members - | 1 | 'type' : 'multipolygon', 'tourism' : 'hotel', 'name' : 'XZ' | N1,W2 - When loading osm data - Then table place contains - | object | class | type | name - | N1 | amenity | prison | 'name' : 'foo' - | W1 | shop | toys | 'name' : 'tata' - | R1 | tourism | hotel | 'name' : 'XZ' - - Scenario: Import object with two main tags - Given the osm nodes: - | id | tags - | 1 | 'tourism' : 'hotel', 'amenity' : 'restaurant', 'name' : 'foo' - When loading osm data - Then table place contains - | object | class | type | name - | N1:tourism | tourism | hotel | 'name' : 'foo' - | N1:amenity | amenity | restaurant | 'name' : 'foo' - - Scenario: Import stand-alone house number with postcode - Given the osm nodes: - | id | tags - | 1 | 'addr:housenumber' : '4', 'addr:postcode' : '3345' - When loading osm data - Then table place contains - | object | class | type - | N1 | place | house - - Scenario: Landuses are only imported when named - Given the osm nodes: - | id | geometry - | 100 | 0 0 - | 101 | 0 0.1 - | 102 | 0.1 0.1 - | 200 | 0 0 - | 202 | 1 1 - | 203 | 1 0 - And the osm ways: - | id | tags | nodes - | 1 | 'landuse' : 'residential', 'name' : 'rainbow' | 100 101 102 100 - | 2 | 'landuse' : 'residential' | 200 202 203 200 - When loading osm data - Then table place contains - | object | class | type - | W1 | landuse | residential - And table place has no entry for W2 diff --git a/tests/features/osm2pgsql/import/tags.feature b/tests/features/osm2pgsql/import/tags.feature deleted file mode 100644 index 3e3d853b..00000000 --- a/tests/features/osm2pgsql/import/tags.feature +++ /dev/null @@ -1,550 +0,0 @@ -@DB -Feature: Tag evaluation - Tests if tags are correctly imported into the place table - - Scenario Outline: Name tags - Given the osm nodes: - | id | tags - | 1 | 'highway' : 'yes', '' : 'Foo' - When loading osm data - Then table place contains - | object | name - | N1 | '' : 'Foo' - - Examples: - | nametag - | ref - | int_ref - | nat_ref - | reg_ref - | loc_ref - | old_ref - | iata - | icao - | pcode:1 - | pcode:2 - | pcode:3 - | name - | name:de - | name:bt-BR - | int_name - | int_name:xxx - | nat_name - | nat_name:fr - | reg_name - | reg_name:1 - | loc_name - | loc_name:DE - | old_name - | old_name:v1 - | alt_name - | alt_name:dfe - | alt_name_1 - | official_name - | short_name - | short_name:CH - | addr:housename - | brand - - Scenario Outline: operator only for shops and amenities - Given the osm nodes: - | id | tags - | 1 | 'highway' : 'yes', 'operator' : 'Foo', 'name' : 'null' - | 2 | 'shop' : 'grocery', 'operator' : 'Foo' - | 3 | 'amenity' : 'hospital', 'operator' : 'Foo' - | 4 | 'tourism' : 'hotel', 'operator' : 'Foo' - When loading osm data - Then table place contains - | object | name - | N1 | 'name' : 'null' - | N2 | 'operator' : 'Foo' - | N3 | 'operator' : 'Foo' - | N4 | 'operator' : 'Foo' - - Scenario Outline: Ignored name tags - Given the osm nodes: - | id | tags - | 1 | 'highway' : 'yes', '' : 'Foo', 'name' : 'real' - When loading osm data - Then table place contains - | object | name - | N1 | 'name' : 'real' - - Examples: - | nametag - | name_de - | Name - | ref:de - | ref_de - | my:ref - | br:name - | name:prefix - | name:source - - Scenario: Special character in name tag - Given the osm nodes: - | id | tags - | 1 | 'highway' : 'yes', 'name: de' : 'Foo', 'name' : 'real1' - | 2 | 'highway' : 'yes', 'name: de' : 'Foo', 'name' : 'real2' - | 3 | 'highway' : 'yes', 'name: de' : 'Foo', 'name:\\' : 'real3' - When loading osm data - Then table place contains - | object | name - | N1 | 'name: de' : 'Foo', 'name' : 'real1' - | N2 | 'name: de' : 'Foo', 'name' : 'real2' - | N3 | 'name: de' : 'Foo', 'name:\\' : 'real3' - - Scenario Outline: Included places - Given the osm nodes: - | id | tags - | 1 | '' : '', 'name' : 'real' - When loading osm data - Then table place contains - | object | name - | N1 | 'name' : 'real' - - Examples: - | key | value - | emergency | phone - | tourism | information - | historic | castle - | military | barracks - | natural | water - | highway | residential - | aerialway | station - | aeroway | way - | boundary | administrative - | craft | butcher - | leisure | playground - | office | bookmaker - | railway | rail - | shop | bookshop - | waterway | stream - | landuse | cemetry - | man_made | tower - | mountain_pass | yes - - Scenario Outline: Bridges and Tunnels take special name tags - Given the osm nodes: - | id | tags - | 1 | 'highway' : 'road', '' : 'yes', 'name' : 'Rd', ':name' : 'My' - | 2 | 'highway' : 'road', '' : 'yes', 'name' : 'Rd' - When loading osm data - Then table place contains - | object | class | type | name - | N1:highway | highway | road | 'name' : 'Rd' - | N1: | | yes | 'name' : 'My' - | N2:highway | highway | road | 'name' : 'Rd' - And table place has no entry for N2: - - Examples: - | key - | bridge - | tunnel - - Scenario Outline: Excluded places - Given the osm nodes: - | id | tags - | 1 | '' : '', 'name' : 'real' - | 2 | 'highway' : 'motorway', 'name' : 'To Hell' - When loading osm data - Then table place has no entry for N1 - - Examples: - | key | value - | emergency | yes - | emergency | no - | tourism | yes - | tourism | no - | historic | yes - | historic | no - | military | yes - | military | no - | natural | yes - | natural | no - | highway | no - | highway | turning_circle - | highway | mini_roundabout - | highway | noexit - | highway | crossing - | aerialway | no - | aerialway | pylon - | man_made | survey_point - | man_made | cutline - | aeroway | no - | amenity | no - | bridge | no - | craft | no - | leisure | no - | office | no - | railway | no - | railway | level_crossing - | shop | no - | tunnel | no - | waterway | riverbank - - Scenario: Some tags only are included when named - Given the osm nodes: - | id | tags - | 1 | '' : '' - | 2 | '' : '', 'name' : 'To Hell' - | 3 | '' : '', 'ref' : '123' - When loading osm data - Then table place has no entry for N1 - And table place has no entry for N3 - And table place contains - | object | class | type - | N2 | | - - Examples: - | key | value - | landuse | residential - | natural | meadow - | highway | traffic_signals - | highway | service - | highway | cycleway - | highway | path - | highway | footway - | highway | steps - | highway | bridleway - | highway | track - | highway | byway - | highway | motorway_link - | highway | primary_link - | highway | trunk_link - | highway | secondary_link - | highway | tertiary_link - | railway | rail - | boundary | administrative - | waterway | stream - - Scenario: Footways are not included if they are sidewalks - Given the osm nodes: - | id | tags - | 2 | 'highway' : 'footway', 'name' : 'To Hell', 'footway' : 'sidewalk' - | 23 | 'highway' : 'footway', 'name' : 'x' - When loading osm data - Then table place has no entry for N2 - - Scenario: named junctions are included if there is no other tag - Given the osm nodes: - | id | tags - | 1 | 'junction' : 'yes' - | 2 | 'highway' : 'secondary', 'junction' : 'roundabout', 'name' : 'To Hell' - | 3 | 'junction' : 'yes', 'name' : 'Le Croix' - When loading osm data - Then table place has no entry for N1 - And table place has no entry for N2:junction - And table place contains - | object | class | type - | N3 | junction | yes - - Scenario: Boundary with place tag - Given the osm nodes: - | id | geometry - | 200 | 0 0 - | 201 | 0 1 - | 202 | 1 1 - | 203 | 1 0 - And the osm ways: - | id | tags | nodes - | 2 | 'boundary' : 'administrative', 'place' : 'city', 'name' : 'Foo' | 200 201 202 203 200 - | 4 | 'boundary' : 'administrative', 'place' : 'island','name' : 'Foo' | 200 201 202 203 200 - | 20 | 'place' : 'city', 'name' : 'ngng' | 200 201 202 203 200 - | 40 | 'place' : 'city', 'boundary' : 'statistical', 'name' : 'BB' | 200 201 202 203 200 - When loading osm data - Then table place contains - | object | class | extratags | type - | W2 | boundary | 'place' : 'city' | administrative - | W4:boundary | boundary | None | administrative - | W4:place | place | None | island - | W20 | place | None | city - | W40:boundary | boundary | None | statistical - | W40:place | place | None | city - And table place has no entry for W2:place - - Scenario Outline: Tags that describe a house - Given the osm nodes: - | id | tags - | 100 | '' : '' - | 999 | 'amenity' : 'prison', '' : '' - When loading osm data - Then table place contains - | object | class | type - | N100 | place | house - | N999 | amenity | prison - And table place has no entry for N100: - And table place has no entry for N999: - And table place has no entry for N999:place - - Examples: - | key | value - | addr:housename | My Mansion - | addr:housenumber | 456 - | addr:conscriptionnumber | 4 - | addr:streetnumber | 4568765 - - Scenario: Only named with no other interesting tag - Given the osm nodes: - | id | tags - | 1 | 'landuse' : 'meadow' - | 2 | 'landuse' : 'residential', 'name' : 'important' - | 3 | 'landuse' : 'residential', 'name' : 'important', 'place' : 'hamlet' - When loading osm data - Then table place contains - | object | class | type - | N2 | landuse | residential - | N3 | place | hamlet - And table place has no entry for N1 - And table place has no entry for N3:landuse - - Scenario Outline: Import of postal codes - Given the osm nodes: - | id | tags - | 10 | 'highway' : 'secondary', '' : '' - | 11 | '' : '' - When loading osm data - Then table place contains - | object | class | type | postcode - | N10 | highway | secondary | - | N11 | place | postcode | - And table place has no entry for N10:place - - Examples: - | key | value - | postal_code | 45736 - | postcode | xxx - | addr:postcode | 564 - | tiger:zip_left | 00011 - | tiger:zip_right | 09123 - - Scenario: Import of street and place - Given the osm nodes: - | id | tags - | 10 | 'amenity' : 'hospital', 'addr:street' : 'Foo St' - | 20 | 'amenity' : 'hospital', 'addr:place' : 'Foo Town' - When loading osm data - Then table place contains - | object | class | type | street | addr_place - | N10 | amenity | hospital | Foo St | None - | N20 | amenity | hospital | None | Foo Town - - - Scenario Outline: Import of country - Given the osm nodes: - | id | tags - | 10 | 'place' : 'village', '' : '' - When loading osm data - Then table place contains - | object | class | type | country_code - | N10 | place | village | - - Examples: - | key | value - | country_code | us - | ISO3166-1 | XX - | is_in:country_code | __ - | addr:country | .. - | addr:country_code | cv - - Scenario Outline: Ignore country codes with wrong length - Given the osm nodes: - | id | tags - | 10 | 'place' : 'village', 'country_code' : '' - When loading osm data - Then table place contains - | object | class | type | country_code - | N10 | place | village | None - - Examples: - | value - | X - | x - | ger - | dkeufr - | d e - - Scenario: Import of house numbers - Given the osm nodes: - | id | tags - | 10 | 'building' : 'yes', 'addr:housenumber' : '4b' - | 11 | 'building' : 'yes', 'addr:conscriptionnumber' : '003' - | 12 | 'building' : 'yes', 'addr:streetnumber' : '2345' - | 13 | 'building' : 'yes', 'addr:conscriptionnumber' : '3', 'addr:streetnumber' : '111' - When loading osm data - Then table place contains - | object | class | type | housenumber - | N10 | building | yes | 4b - | N11 | building | yes | 003 - | N12 | building | yes | 2345 - | N13 | building | yes | 3/111 - - Scenario: Import of address interpolations - Given the osm nodes: - | id | tags - | 10 | 'addr:interpolation' : 'odd' - | 11 | 'addr:housenumber' : '10', 'addr:interpolation' : 'odd' - | 12 | 'addr:interpolation' : 'odd', 'addr:housenumber' : '23' - When loading osm data - Then table place contains - | object | class | type | housenumber - | N10 | place | houses | odd - | N11 | place | houses | odd - | N12 | place | houses | odd - - Scenario: Shorten tiger:county tags - Given the osm nodes: - | id | tags - | 10 | 'place' : 'village', 'tiger:county' : 'Feebourgh, AL' - | 11 | 'place' : 'village', 'addr:state' : 'Alabama', 'tiger:county' : 'Feebourgh, AL' - | 12 | 'place' : 'village', 'tiger:county' : 'Feebourgh' - When loading osm data - Then table place contains - | object | class | type | isin - | N10 | place | village | Feebourgh county - | N11 | place | village | Feebourgh county,Alabama - | N12 | place | village | Feebourgh county - - Scenario Outline: Import of address tags - Given the osm nodes: - | id | tags - | 10 | 'place' : 'village', '' : '' - When loading osm data - Then table place contains - | object | class | type | isin - | N10 | place | village | - - Examples: - | key | value - | is_in | Stockholm, Sweden - | is_in:country | Xanadu - | addr:suburb | hinein - | addr:county | le havre - | addr:city | Sydney - | addr:state | Jura - - Scenario: Import of admin level - Given the osm nodes: - | id | tags - | 10 | 'amenity' : 'hospital', 'admin_level' : '3' - | 11 | 'amenity' : 'hospital', 'admin_level' : 'b' - | 12 | 'amenity' : 'hospital' - | 13 | 'amenity' : 'hospital', 'admin_level' : '3.0' - When loading osm data - Then table place contains - | object | class | type | admin_level - | N10 | amenity | hospital | 3 - | N11 | amenity | hospital | 100 - | N12 | amenity | hospital | 100 - | N13 | amenity | hospital | 3 - - Scenario: Import of extra tags - Given the osm nodes: - | id | tags - | 10 | 'tourism' : 'hotel', '' : 'foo' - When loading osm data - Then table place contains - | object | class | type | extratags - | N10 | tourism | hotel | '' : 'foo' - - Examples: - | key - | tracktype - | traffic_calming - | service - | cuisine - | capital - | dispensing - | religion - | denomination - | sport - | internet_access - | lanes - | surface - | smoothness - | width - | est_width - | incline - | opening_hours - | collection_times - | service_times - | disused - | wheelchair - | sac_scale - | trail_visibility - | mtb:scale - | mtb:description - | wood - | drive_in - | access - | vehicle - | bicyle - | foot - | goods - | hgv - | motor_vehicle - | motor_car - | access:foot - | contact:phone - | drink:mate - | oneway - | date_on - | date_off - | day_on - | day_off - | hour_on - | hour_off - | maxweight - | maxheight - | maxspeed - | disused - | toll - | charge - | population - | description - | image - | attribution - | fax - | email - | url - | website - | phone - | real_ale - | smoking - | food - | camera - | brewery - | locality - | wikipedia - | wikipedia:de - | wikidata - | name:prefix - | name:botanical - | name:etymology:wikidata - - Scenario: buildings - Given the osm nodes: - | id | tags - | 10 | 'tourism' : 'hotel', 'building' : 'yes' - | 11 | 'building' : 'house' - | 12 | 'building' : 'shed', 'addr:housenumber' : '1' - | 13 | 'building' : 'yes', 'name' : 'Das Haus' - | 14 | 'building' : 'yes', 'addr:postcode' : '12345' - When loading osm data - Then table place contains - | object | class | type - | N10 | tourism | hotel - | N12 | building| yes - | N13 | building| yes - | N14 | building| yes - And table place has no entry for N10:building - And table place has no entry for N11 - - Scenario: complete node entry - Given the osm nodes: - | id | tags - | 290393920 | 'addr:city':'Perpignan','addr:country':'FR','addr:housenumber':'43\\','addr:postcode':'66000','addr:street':'Rue Pierre Constant d`Ivry','source':'cadastre-dgi-fr source : Direction Générale des Impôts - Cadastre ; mise à jour :2008' - When loading osm data - Then table place contains - | object | class | type | housenumber - | N290393920 | place | house| 43\ diff --git a/tests/features/osm2pgsql/update/relation.feature b/tests/features/osm2pgsql/update/relation.feature deleted file mode 100644 index f7bf53aa..00000000 --- a/tests/features/osm2pgsql/update/relation.feature +++ /dev/null @@ -1,152 +0,0 @@ -@DB -Feature: Update of relations by osm2pgsql - Testing relation update by osm2pgsql. - -Scenario: Remove all members of a relation - Given the osm nodes: - | id | tags - | 1 | 'amenity' : 'prison', 'name' : 'foo' - Given the osm nodes: - | id | geometry - | 200 | 0 0 - | 201 | 0 0.0001 - | 202 | 0.0001 0.0001 - | 203 | 0.0001 0 - Given the osm ways: - | id | tags | nodes - | 2 | 'ref' : '45' | 200 201 202 203 200 - Given the osm relations: - | id | tags | members - | 1 | 'type' : 'multipolygon', 'tourism' : 'hotel', 'name' : 'XZ' | W2 - When loading osm data - Then table place contains - | object | class | type | name - | R1 | tourism | hotel | 'name' : 'XZ' - Given the osm relations: - | action | id | tags | members - | M | 1 | 'type' : 'multipolygon', 'tourism' : 'hotel', 'name' : 'XZ' | N1 - When updating osm data - Then table place has no entry for R1 - - -Scenario: Change type of a relation - Given the osm nodes: - | id | geometry - | 200 | 0 0 - | 201 | 0 0.0001 - | 202 | 0.0001 0.0001 - | 203 | 0.0001 0 - Given the osm ways: - | id | tags | nodes - | 2 | 'ref' : '45' | 200 201 202 203 200 - Given the osm relations: - | id | tags | members - | 1 | 'type' : 'multipolygon', 'tourism' : 'hotel', 'name' : 'XZ' | W2 - When loading osm data - Then table place contains - | object | class | type | name - | R1 | tourism | hotel | 'name' : 'XZ' - Given the osm relations: - | action | id | tags | members - | M | 1 | 'type' : 'multipolygon', 'amenity' : 'prison', 'name' : 'XZ' | W2 - When updating osm data - Then table place has no entry for R1:tourism - And table place contains - | object | class | type | name - | R1 | amenity | prison | 'name' : 'XZ' - -Scenario: Change name of a relation - Given the osm nodes: - | id | geometry - | 200 | 0 0 - | 201 | 0 0.0001 - | 202 | 0.0001 0.0001 - | 203 | 0.0001 0 - Given the osm ways: - | id | tags | nodes - | 2 | 'ref' : '45' | 200 201 202 203 200 - Given the osm relations: - | id | tags | members - | 1 | 'type' : 'multipolygon', 'tourism' : 'hotel', 'name' : 'AB' | W2 - When loading osm data - Then table place contains - | object | class | type | name - | R1 | tourism | hotel | 'name' : 'AB' - Given the osm relations: - | action | id | tags | members - | M | 1 | 'type' : 'multipolygon', 'tourism' : 'hotel', 'name' : 'XZ' | W2 - When updating osm data - Then table place contains - | object | class | type | name - | R1 | tourism | hotel | 'name' : 'XZ' - - -Scenario: Change type of a relation into something unknown - Given the osm nodes: - | id | geometry - | 200 | 0 0 - | 201 | 0 0.0001 - | 202 | 0.0001 0.0001 - | 203 | 0.0001 0 - Given the osm ways: - | id | tags | nodes - | 2 | 'ref' : '45' | 200 201 202 203 200 - Given the osm relations: - | id | tags | members - | 1 | 'type' : 'multipolygon', 'tourism' : 'hotel', 'name' : 'XZ' | W2 - When loading osm data - Then table place contains - | object | class | type | name - | R1 | tourism | hotel | 'name' : 'XZ' - Given the osm relations: - | action | id | tags | members - | M | 1 | 'type' : 'multipolygon', 'amenities' : 'prison', 'name' : 'XZ' | W2 - When updating osm data - Then table place has no entry for R1 - -Scenario: Type tag is removed - Given the osm nodes: - | id | geometry - | 200 | 0 0 - | 201 | 0 0.0001 - | 202 | 0.0001 0.0001 - | 203 | 0.0001 0 - Given the osm ways: - | id | tags | nodes - | 2 | 'ref' : '45' | 200 201 202 203 200 - Given the osm relations: - | id | tags | members - | 1 | 'type' : 'multipolygon', 'tourism' : 'hotel', 'name' : 'XZ' | W2 - When loading osm data - Then table place contains - | object | class | type | name - | R1 | tourism | hotel | 'name' : 'XZ' - Given the osm relations: - | action | id | tags | members - | M | 1 | 'tourism' : 'hotel', 'name' : 'XZ' | W2 - When updating osm data - Then table place has no entry for R1 - -Scenario: Type tag is renamed to something unknown - Given the osm nodes: - | id | geometry - | 200 | 0 0 - | 201 | 0 0.0001 - | 202 | 0.0001 0.0001 - | 203 | 0.0001 0 - Given the osm ways: - | id | tags | nodes - | 2 | 'ref' : '45' | 200 201 202 203 200 - Given the osm relations: - | id | tags | members - | 1 | 'type' : 'multipolygon', 'tourism' : 'hotel', 'name' : 'XZ' | W2 - When loading osm data - Then table place contains - | object | class | type | name - | R1 | tourism | hotel | 'name' : 'XZ' - Given the osm relations: - | action | id | tags | members - | M | 1 | 'type' : 'multipolygonn', 'tourism' : 'hotel', 'name' : 'XZ' | W2 - When updating osm data - Then table place has no entry for R1 - diff --git a/tests/features/osm2pgsql/update/simple.feature b/tests/features/osm2pgsql/update/simple.feature deleted file mode 100644 index e0c9b005..00000000 --- a/tests/features/osm2pgsql/update/simple.feature +++ /dev/null @@ -1,22 +0,0 @@ -@DB -Feature: Update of simple objects by osm2pgsql - Testing basic update functions of osm2pgsql. - - Scenario: Import object with two main tags - Given the osm nodes: - | id | tags - | 1 | 'tourism' : 'hotel', 'amenity' : 'restaurant', 'name' : 'foo' - When loading osm data - Then table place contains - | object | class | type | name - | N1:tourism | tourism | hotel | 'name' : 'foo' - | N1:amenity | amenity | restaurant | 'name' : 'foo' - Given the osm nodes: - | action | id | tags - | M | 1 | 'tourism' : 'hotel', 'name' : 'foo' - When updating osm data - Then table place has no entry for N1:amenity - And table place contains - | object | class | type | name - | N1:tourism | tourism | hotel | 'name' : 'foo' - diff --git a/tests/steps/api_result.py b/tests/steps/api_result.py deleted file mode 100644 index 2644d4a2..00000000 --- a/tests/steps/api_result.py +++ /dev/null @@ -1,286 +0,0 @@ -""" Steps for checking the results of queries. -""" - -from nose.tools import * -from lettuce import * -from tidylib import tidy_document -from collections import OrderedDict -import json -import logging -import re -from xml.dom.minidom import parseString - -logger = logging.getLogger(__name__) - -def _parse_xml(): - """ Puts the DOM structure into more convenient python - with a similar structure as the json document, so - that the same the semantics can be used. It does not - check if the content is valid (or at least not more than - necessary to transform it into a dict structure). - """ - page = parseString(world.page).documentElement - - # header info - world.result_header = OrderedDict(page.attributes.items()) - logger.debug('Result header: %r' % (world.result_header)) - world.results = [] - - # results - if page.nodeName == 'searchresults' or page.nodeName == 'lookupresults': - for node in page.childNodes: - if node.nodeName != "#text": - assert_equals(node.nodeName, 'place', msg="Unexpected element '%s'" % node.nodeName) - newresult = OrderedDict(node.attributes.items()) - assert_not_in('address', newresult) - assert_not_in('geokml', newresult) - assert_not_in('extratags', newresult) - assert_not_in('namedetails', newresult) - address = OrderedDict() - for sub in node.childNodes: - if sub.nodeName == 'geokml': - newresult['geokml'] = sub.childNodes[0].toxml() - elif sub.nodeName == 'extratags': - newresult['extratags'] = {} - for tag in sub.childNodes: - assert_equals(tag.nodeName, 'tag') - attrs = dict(tag.attributes.items()) - assert_in('key', attrs) - assert_in('value', attrs) - newresult['extratags'][attrs['key']] = attrs['value'] - elif sub.nodeName == 'namedetails': - newresult['namedetails'] = {} - for tag in sub.childNodes: - assert_equals(tag.nodeName, 'name') - attrs = dict(tag.attributes.items()) - assert_in('desc', attrs) - newresult['namedetails'][attrs['desc']] = tag.firstChild.nodeValue.strip() - - elif sub.nodeName == '#text': - pass - else: - address[sub.nodeName] = sub.firstChild.nodeValue.strip() - if address: - newresult['address'] = address - world.results.append(newresult) - elif page.nodeName == 'reversegeocode': - haserror = False - address = {} - for node in page.childNodes: - if node.nodeName == 'result': - assert_equals(len(world.results), 0) - assert (not haserror) - world.results.append(OrderedDict(node.attributes.items())) - assert_not_in('display_name', world.results[0]) - assert_not_in('address', world.results[0]) - world.results[0]['display_name'] = node.firstChild.nodeValue.strip() - elif node.nodeName == 'error': - assert_equals(len(world.results), 0) - haserror = True - elif node.nodeName == 'addressparts': - assert (not haserror) - address = OrderedDict() - for sub in node.childNodes: - address[sub.nodeName] = sub.firstChild.nodeValue.strip() - world.results[0]['address'] = address - elif node.nodeName == 'extratags': - world.results[0]['extratags'] = {} - for tag in node.childNodes: - assert_equals(tag.nodeName, 'tag') - attrs = dict(tag.attributes.items()) - assert_in('key', attrs) - assert_in('value', attrs) - world.results[0]['extratags'][attrs['key']] = attrs['value'] - elif node.nodeName == 'namedetails': - world.results[0]['namedetails'] = {} - for tag in node.childNodes: - assert_equals(tag.nodeName, 'name') - attrs = dict(tag.attributes.items()) - assert_in('desc', attrs) - world.results[0]['namedetails'][attrs['desc']] = tag.firstChild.nodeValue.strip() - elif node.nodeName == "geokml": - world.results[0]['geokml'] = node - elif node.nodeName == "#text": - pass - else: - assert False, "Unknown content '%s' in XML" % node.nodeName - else: - assert False, "Unknown document node name %s in XML" % page.nodeName - - logger.debug("The following was parsed out of XML:") - logger.debug(world.results) - -@step(u'a HTTP (\d+) is returned') -def api_result_http_error(step, error): - assert_equals(world.returncode, int(error)) - -@step(u'the result is valid( \w+)?') -def api_result_is_valid(step, fmt): - assert_equals(world.returncode, 200) - - if world.response_format == 'html': - document, errors = tidy_document(world.page, - options={'char-encoding' : 'utf8'}) - # assert(len(errors) == 0), "Errors found in HTML document:\n%s" % errors - world.results = document - elif world.response_format == 'xml': - _parse_xml() - elif world.response_format == 'json': - world.results = json.JSONDecoder(object_pairs_hook=OrderedDict).decode(world.page) - if world.request_type == 'reverse': - world.results = (world.results,) - else: - assert False, "Unknown page format: %s" % (world.response_format) - - if fmt: - assert_equals (fmt.strip(), world.response_format) - - -def compare(operator, op1, op2): - if operator == 'less than': - return op1 < op2 - elif operator == 'more than': - return op1 > op2 - elif operator == 'exactly': - return op1 == op2 - elif operator == 'at least': - return op1 >= op2 - elif operator == 'at most': - return op1 <= op2 - else: - raise Exception("unknown operator '%s'" % operator) - -@step(u'(less than|more than|exactly|at least|at most) (\d+) results? (?:is|are) returned') -def validate_result_number(step, operator, number): - step.given('the result is valid') - numres = len(world.results) - assert compare(operator, numres, int(number)), \ - "Bad number of results: expected %s %s, got %d." % (operator, number, numres) - -@step(u'result (\d+) has( not)? attributes (\S+)') -def search_check_for_result_attribute(step, num, invalid, attrs): - num = int(num) - step.given('at least %d results are returned' % (num + 1)) - res = world.results[num] - for attr in attrs.split(','): - if invalid: - assert_not_in(attr.strip(), res) - else: - assert_in(attr.strip(),res) - -@step(u'there is a json wrapper "([^"]*)"') -def api_result_check_json_wrapper(step, wrapper): - step.given('the result is valid json') - assert_equals(world.json_callback, wrapper) - -@step(u'result header contains') -def api_result_header_contains(step): - step.given('the result is valid') - for line in step.hashes: - assert_in(line['attr'], world.result_header) - m = re.match("%s$" % (line['value'],), world.result_header[line['attr']]) - -@step(u'result header has no attribute (.*)') -def api_result_header_contains_not(step, attr): - step.given('the result is valid') - assert_not_in(attr, world.result_header) - -@step(u'results contain$') -def api_result_contains(step): - step.given('at least 1 result is returned') - for line in step.hashes: - if 'ID' in line: - reslist = (world.results[int(line['ID'])],) - else: - reslist = world.results - for k,v in line.iteritems(): - if k == 'latlon': - for curres in reslist: - world.match_geometry((float(curres['lat']), float(curres['lon'])), v) - elif k != 'ID': - for curres in reslist: - assert_in(k, curres) - if v[0] in '<>=': - # mathematical operation - evalexp = '%s %s' % (curres[k], v) - res = eval(evalexp) - logger.debug('Evaluating: %s = %s' % (res, evalexp)) - assert_true(res, "Evaluation failed: %s" % (evalexp, )) - else: - # regex match - m = re.match("%s$" % (v,), curres[k]) - assert_is_not_none(m, msg="field %s does not match: %s$ != %s." % (k, v, curres[k])) - -@step(u'results contain valid boundingboxes$') -def api_result_address_contains(step): - step.given('the result is valid') - for curres in world.results: - bb = curres['boundingbox'] - if world.response_format == 'json': - bb = ','.join(bb) - m = re.match('^(-?\d+\.\d+),(-?\d+\.\d+),(-?\d+\.\d+),(-?\d+\.\d+)$', bb) - assert_is_not_none(m, msg="invalid boundingbox: %s." % (curres['boundingbox'])) - -@step(u'result addresses contain$') -def api_result_address_contains(step): - step.given('the result is valid') - for line in step.hashes: - if 'ID' in line: - reslist = (world.results[int(line['ID'])],) - else: - reslist = world.results - for k,v in line.iteritems(): - if k != 'ID': - for res in reslist: - curres = res['address'] - assert_in(k, curres) - m = re.match("%s$" % (v,), curres[k]) - assert_is_not_none(m, msg="field %s does not match: %s$ != %s." % (k, v, curres[k])) - - -@step(u'address of result (\d+) contains') -def api_result_address_exact(step, resid): - resid = int(resid) - step.given('at least %d results are returned' % (resid + 1)) - addr = world.results[resid]['address'] - for line in step.hashes: - assert_in(line['type'], addr) - m = re.match("%s$" % line['value'], addr[line['type']]) - assert_is_not_none(m, msg="field %s does not match: %s$ != %s." % ( - line['type'], line['value'], addr[line['type']])) - #assert_equals(line['value'], addr[line['type']]) - -@step(u'address of result (\d+) does not contain (.*)') -def api_result_address_details_missing(step, resid, types): - resid = int(resid) - step.given('at least %d results are returned' % (resid + 1)) - addr = world.results[resid]['address'] - for t in types.split(','): - assert_not_in(t.strip(), addr) - - -@step(u'address of result (\d+) is') -def api_result_address_exact(step, resid): - resid = int(resid) - step.given('at least %d results are returned' % (resid + 1)) - result = world.results[resid] - linenr = 0 - assert_equals(len(step.hashes), len(result['address'])) - for k,v in result['address'].iteritems(): - assert_equals(step.hashes[linenr]['type'], k) - assert_equals(step.hashes[linenr]['value'], v) - linenr += 1 - - -@step('there are( no)? duplicates') -def api_result_check_for_duplicates(step, nodups=None): - step.given('at least 1 result is returned') - resarr = [] - for res in world.results: - resarr.append((res['osm_type'], res['class'], - res['type'], res['display_name'])) - - if nodups is None: - assert len(resarr) > len(set(resarr)) - else: - assert_equal(len(resarr), len(set(resarr))) diff --git a/tests/steps/api_setup.py b/tests/steps/api_setup.py deleted file mode 100644 index c9a4bac4..00000000 --- a/tests/steps/api_setup.py +++ /dev/null @@ -1,136 +0,0 @@ -""" Steps for setting up and sending API requests. -""" - -from nose.tools import * -from lettuce import * -import urllib -import urllib2 -import logging - -logger = logging.getLogger(__name__) - -def api_call(requesttype): - world.request_type = requesttype - world.json_callback = None - data = urllib.urlencode(world.params) - url = "%s/%s?%s" % (world.config.base_url, requesttype, data) - req = urllib2.Request(url=url, headers=world.header) - try: - fd = urllib2.urlopen(req) - world.page = fd.read() - world.returncode = 200 - except urllib2.HTTPError, ex: - world.returncode = ex.code - world.page = None - return - - pageinfo = fd.info() - assert_equal('utf-8', pageinfo.getparam('charset').lower()) - pagetype = pageinfo.gettype() - - fmt = world.params.get('format') - if fmt == 'html': - assert_equals('text/html', pagetype) - world.response_format = fmt - elif fmt == 'xml': - assert_equals('text/xml', pagetype) - world.response_format = fmt - elif fmt in ('json', 'jsonv2'): - if 'json_callback' in world.params: - world.json_callback = world.params['json_callback'].encode('utf8') - assert world.page.startswith(world.json_callback + '(') - assert world.page.endswith(')') - world.page = world.page[(len(world.json_callback)+1):-1] - assert_equals('application/javascript', pagetype) - else: - assert_equals('application/json', pagetype) - world.response_format = 'json' - else: - if requesttype == 'reverse': - assert_equals('text/xml', pagetype) - world.response_format = 'xml' - else: - assert_equals('text/html', pagetype) - world.response_format = 'html' - logger.debug("Page received (%s):" % world.response_format) - logger.debug(world.page) - - api_setup_prepare_params(None) - -@before.each_scenario -def api_setup_prepare_params(scenario): - world.results = [] - world.params = {} - world.header = {} - -@step(u'the request parameters$') -def api_setup_parameters(step): - """Define the parameters of the request as a hash. - Resets parameter list. - """ - world.params = step.hashes[0] - -@step(u'the HTTP header$') -def api_setup_parameters(step): - """Define additional HTTP header parameters as a hash. - Resets parameter list. - """ - world.header = step.hashes[0] - - -@step(u'sending( \w+)? search query "([^"]*)"( with address)?') -def api_setup_search(step, fmt, query, doaddr): - world.params['q'] = query.encode('utf8') - if doaddr: - world.params['addressdetails'] = 1 - if fmt: - world.params['format'] = fmt.strip() - api_call('search') - -@step(u'sending( \w+)? structured query( with address)?$') -def api_setup_structured_search(step, fmt, doaddr): - world.params.update(step.hashes[0]) - if doaddr: - world.params['addressdetails'] = 1 - if fmt: - world.params['format'] = fmt.strip() - api_call('search') - -@step(u'looking up (\w+ )?coordinates ([-\d.]+),([-\d.]+)') -def api_setup_reverse(step, fmt, lat, lon): - world.params['lat'] = lat - world.params['lon'] = lon - if fmt and fmt.strip(): - world.params['format'] = fmt.strip() - api_call('reverse') - -@step(u'looking up place ([NRW]?\d+)') -def api_setup_details_reverse(step, obj): - if obj[0] in ('N', 'R', 'W'): - # an osm id - world.params['osm_type'] = obj[0] - world.params['osm_id'] = obj[1:] - else: - world.params['place_id'] = obj - api_call('reverse') - -@step(u'looking up details for ([NRW]?\d+)') -def api_setup_details(step, obj): - if obj[0] in ('N', 'R', 'W'): - # an osm id - world.params['osmtype'] = obj[0] - world.params['osmid'] = obj[1:] - else: - world.params['place_id'] = obj - api_call('details') - -@step(u'looking up (\w+) places ((?:[a-z]\d+,*)+)') -def api_setup_lookup(step, fmt, ids): - world.params['osm_ids'] = ids - if fmt and fmt.strip(): - world.params['format'] = fmt.strip() - api_call('lookup') - -@step(u'sending an API call (\w+)') -def api_general_call(step, call): - api_call(call) diff --git a/tests/steps/db_results.py b/tests/steps/db_results.py deleted file mode 100644 index 53374e71..00000000 --- a/tests/steps/db_results.py +++ /dev/null @@ -1,201 +0,0 @@ -""" Steps for checking the DB after import and update tests. - - There are two groups of test here. The first group tests - the contents of db tables directly, the second checks - query results by using the command line query tool. -""" - -from nose.tools import * -from lettuce import * -import psycopg2 -import psycopg2.extensions -import psycopg2.extras -import os -import random -import json -import re -import logging -from collections import OrderedDict - -logger = logging.getLogger(__name__) - -@step(u'table placex contains as names for (N|R|W)(\d+)') -def check_placex_names(step, osmtyp, osmid): - """ Check for the exact content of the name hstore in placex. - """ - cur = world.conn.cursor(cursor_factory=psycopg2.extras.DictCursor) - cur.execute('SELECT name FROM placex where osm_type = %s and osm_id =%s', (osmtyp, int(osmid))) - for line in cur: - names = dict(line['name']) - for name in step.hashes: - assert_in(name['k'], names) - assert_equals(names[name['k']], name['v']) - del names[name['k']] - assert_equals(len(names), 0) - - - - -@step(u'table ([a-z_]+) contains$') -def check_placex_content(step, tablename): - """ check that the given lines are in the given table - Entries are searched by osm_type/osm_id and then all - given columns are tested. If there is more than one - line for an OSM object, they must match in these columns. - """ - try: - cur = world.conn.cursor(cursor_factory=psycopg2.extras.DictCursor) - for line in step.hashes: - osmtype, osmid, cls = world.split_id(line['object']) - q = 'SELECT *' - if tablename == 'placex': - q = q + ", ST_X(centroid) as clat, ST_Y(centroid) as clon" - if tablename == 'location_property_osmline': - q = q + ' FROM %s where osm_id = %%s' % (tablename,) - else: - q = q + ", ST_GeometryType(geometry) as geometrytype" - q = q + ' FROM %s where osm_type = %%s and osm_id = %%s' % (tablename,) - if cls is None: - if tablename == 'location_property_osmline': - params = (osmid,) - else: - params = (osmtype, osmid) - else: - q = q + ' and class = %s' - if tablename == 'location_property_osmline': - params = (osmid, cls) - else: - params = (osmtype, osmid, cls) - cur.execute(q, params) - assert(cur.rowcount > 0) - for res in cur: - for k,v in line.iteritems(): - if not k == 'object': - assert_in(k, res) - if type(res[k]) is dict: - val = world.make_hash(v) - assert_equals(res[k], val) - elif k in ('parent_place_id', 'linked_place_id'): - pid = world.get_placeid(v) - assert_equals(pid, res[k], "Results for '%s'/'%s' differ: '%s' != '%s'" % (line['object'], k, pid, res[k])) - elif k == 'centroid': - world.match_geometry((res['clat'], res['clon']), v) - else: - assert_equals(str(res[k]), v, "Results for '%s'/'%s' differ: '%s' != '%s'" % (line['object'], k, str(res[k]), v)) - finally: - cur.close() - world.conn.commit() - -@step(u'table (placex?) has no entry for (N|R|W)(\d+)(:\w+)?') -def check_placex_missing(step, tablename, osmtyp, osmid, placeclass): - cur = world.conn.cursor() - try: - q = 'SELECT count(*) FROM %s where osm_type = %%s and osm_id = %%s' % (tablename, ) - args = [osmtyp, int(osmid)] - if placeclass is not None: - q = q + ' and class = %s' - args.append(placeclass[1:]) - cur.execute(q, args) - numres = cur.fetchone()[0] - assert_equals (numres, 0) - finally: - cur.close() - world.conn.commit() - -@step(u'table location_property_osmline has no entry for W(\d+)?') -def check_osmline_missing(step, osmid): - cur = world.conn.cursor() - try: - q = 'SELECT count(*) FROM location_property_osmline where osm_id = %s' % (osmid, ) - cur.execute(q) - numres = cur.fetchone()[0] - assert_equals (numres, 0) - finally: - cur.close() - world.conn.commit() - -@step(u'search_name table contains$') -def check_search_name_content(step): - cur = world.conn.cursor(cursor_factory=psycopg2.extras.DictCursor) - for line in step.hashes: - placeid = world.get_placeid(line['place_id']) - cur.execute('SELECT * FROM search_name WHERE place_id = %s', (placeid,)) - assert(cur.rowcount > 0) - for res in cur: - for k,v in line.iteritems(): - if k in ('search_rank', 'address_rank'): - assert_equals(int(v), res[k], "Results for '%s'/'%s' differ: '%s' != '%d'" % (line['place_id'], k, v, res[k])) - elif k in ('importance'): - assert_equals(float(v), res[k], "Results for '%s'/'%s' differ: '%s' != '%d'" % (line['place_id'], k, v, res[k])) - elif k in ('name_vector', 'nameaddress_vector'): - terms = [x.strip().replace('#', ' ') for x in v.split(',')] - cur.execute('SELECT word_id, word_token FROM word, (SELECT unnest(%s) as term) t WHERE word_token = make_standard_name(t.term)', (terms,)) - assert cur.rowcount >= len(terms) - for wid in cur: - assert_in(wid['word_id'], res[k], "Missing term for %s/%s: %s" % (line['place_id'], k, wid['word_token'])) - elif k in ('country_code'): - assert_equals(v, res[k], "Results for '%s'/'%s' differ: '%s' != '%d'" % (line['place_id'], k, v, res[k])) - elif k == 'place_id': - pass - else: - raise Exception("Cannot handle field %s in search_name table" % (k, )) - -@step(u'way (\d+) expands to lines') -def check_interpolation_lines(step, wayid): - """Check that the correct interpolation line has been entered in - location_property_osmline for the given source line/nodes. - Expected are three columns: - startnumber, endnumber and linegeo - """ - lines = [] - for line in step.hashes: - lines.append((line["startnumber"], line["endnumber"], line["geometry"])) - cur = world.conn.cursor(cursor_factory=psycopg2.extras.DictCursor) - cur.execute("""SELECT startnumber::text, endnumber::text, st_astext(linegeo) as geometry - FROM location_property_osmline WHERE osm_id = %s""", - (int(wayid),)) - assert_equals(len(lines), cur.rowcount) - for r in cur: - linegeo = str(str(r["geometry"].split('(')[1]).split(')')[0]).replace(',', ', ') - exp = (r["startnumber"], r["endnumber"], linegeo) - assert_in(exp, lines) - lines.remove(exp) - -@step(u'way (\d+) expands exactly to housenumbers ([0-9,]*)') -def check_interpolated_housenumber_list(step, nodeid, numberlist): - """ Checks that the interpolated house numbers corresponds - to the given list. - """ - expected = numberlist.split(','); - cur = world.conn.cursor() - cur.execute("""SELECT housenumber FROM placex - WHERE osm_type = 'W' and osm_id = %s - and class = 'place' and type = 'address'""", (int(nodeid),)) - for r in cur: - assert_in(r[0], expected, "Unexpected house number %s for node %s." % (r[0], nodeid)) - expected.remove(r[0]) - assert_equals(0, len(expected), "Missing house numbers for way %s: %s" % (nodeid, expected)) - -@step(u'way (\d+) expands to no housenumbers') -def check_no_interpolated_housenumber_list(step, nodeid): - """ Checks that the interpolated house numbers corresponds - to the given list. - """ - cur = world.conn.cursor() - cur.execute("""SELECT housenumber FROM placex - WHERE osm_type = 'W' and osm_id = %s - and class = 'place' and type = 'address'""", (int(nodeid),)) - res = [r[0] for r in cur] - assert_equals(0, len(res), "Unexpected house numbers for way %s: %s" % (nodeid, res)) - -@step(u'table search_name has no entry for (.*)') -def check_placex_missing(step, osmid): - """ Checks if there is an entry in the search index for the - given place object. - """ - cur = world.conn.cursor() - placeid = world.get_placeid(osmid) - cur.execute('SELECT count(*) FROM search_name WHERE place_id =%s', (placeid,)) - numres = cur.fetchone()[0] - assert_equals (numres, 0) - diff --git a/tests/steps/db_setup.py b/tests/steps/db_setup.py deleted file mode 100644 index 727e6105..00000000 --- a/tests/steps/db_setup.py +++ /dev/null @@ -1,278 +0,0 @@ -""" Steps for setting up a test database with imports and updates. - - There are two ways to state geometries for test data: with coordinates - and via scenes. - - Coordinates should be given as a wkt without the enclosing type name. - - Scenes are prepared geometries which can be found in the scenes/data/ - directory. Each scene is saved in a .wkt file with its name, which - contains a list of id/wkt pairs. A scene can be set globally - for a scene by using the step `the scene `. Then each - object should be refered to as `:`. A geometry can also - be referred to without loading the scene by explicitly stating the - scene: `:`. -""" - -from nose.tools import * -from lettuce import * -import psycopg2 -import psycopg2.extensions -import psycopg2.extras -import os -import subprocess -import random -import base64 -import sys - -psycopg2.extensions.register_type(psycopg2.extensions.UNICODE) - -@before.each_scenario -def setup_test_database(scenario): - """ Creates a new test database from the template database - that was set up earlier in terrain.py. Will be done only - for scenarios whose feature is tagged with 'DB'. - """ - if scenario.feature.tags is not None and 'DB' in scenario.feature.tags: - world.db_template_setup() - world.write_nominatim_config(world.config.test_db) - conn = psycopg2.connect(database=world.config.template_db) - conn.set_isolation_level(0) - cur = conn.cursor() - cur.execute('DROP DATABASE IF EXISTS %s' % (world.config.test_db, )) - cur.execute('CREATE DATABASE %s TEMPLATE = %s' % (world.config.test_db, world.config.template_db)) - conn.close() - world.conn = psycopg2.connect(database=world.config.test_db) - psycopg2.extras.register_hstore(world.conn, globally=False, unicode=True) - -@step('a wiped database') -def db_setup_wipe_db(step): - """Explicit DB scenario setup only needed - to work around a bug where scenario outlines don't call - before_each_scenario correctly. - """ - if hasattr(world, 'conn'): - world.conn.close() - conn = psycopg2.connect(database=world.config.template_db) - conn.set_isolation_level(0) - cur = conn.cursor() - cur.execute('DROP DATABASE IF EXISTS %s' % (world.config.test_db, )) - cur.execute('CREATE DATABASE %s TEMPLATE = %s' % (world.config.test_db, world.config.template_db)) - conn.close() - world.conn = psycopg2.connect(database=world.config.test_db) - psycopg2.extras.register_hstore(world.conn, globally=False, unicode=True) - - -@after.each_scenario -def tear_down_test_database(scenario): - """ Drops any previously created test database. - """ - if hasattr(world, 'conn'): - world.conn.close() - if scenario.feature.tags is not None and 'DB' in scenario.feature.tags and not world.config.keep_scenario_db: - conn = psycopg2.connect(database=world.config.template_db) - conn.set_isolation_level(0) - cur = conn.cursor() - cur.execute('DROP DATABASE %s' % (world.config.test_db,)) - conn.close() - - -def _format_placex_cols(cols, geomtype, force_name): - if 'name' in cols: - if cols['name'].startswith("'"): - cols['name'] = world.make_hash(cols['name']) - else: - cols['name'] = { 'name' : cols['name'] } - elif force_name: - cols['name'] = { 'name' : base64.urlsafe_b64encode(os.urandom(int(random.random()*30))) } - if 'extratags' in cols: - cols['extratags'] = world.make_hash(cols['extratags']) - if 'admin_level' not in cols: - cols['admin_level'] = 100 - if 'geometry' in cols: - coords = world.get_scene_geometry(cols['geometry']) - if coords is None: - coords = "'%s(%s)'::geometry" % (geomtype, cols['geometry']) - else: - coords = "'%s'::geometry" % coords.wkt - cols['geometry'] = coords - for k in cols: - if not cols[k]: - cols[k] = None - - -def _insert_place_table_nodes(places, force_name): - cur = world.conn.cursor() - for line in places: - cols = dict(line) - cols['osm_type'] = 'N' - _format_placex_cols(cols, 'POINT', force_name) - if 'geometry' in cols: - coords = cols.pop('geometry') - else: - coords = "ST_Point(%f, %f)" % (random.random()*360 - 180, random.random()*180 - 90) - - query = 'INSERT INTO place (%s,geometry) values(%s, ST_SetSRID(%s, 4326))' % ( - ','.join(cols.iterkeys()), - ','.join(['%s' for x in range(len(cols))]), - coords - ) - cur.execute(query, cols.values()) - world.conn.commit() - - -def _insert_place_table_objects(places, geomtype, force_name): - cur = world.conn.cursor() - for line in places: - cols = dict(line) - if 'osm_type' not in cols: - cols['osm_type'] = 'W' - _format_placex_cols(cols, geomtype, force_name) - coords = cols.pop('geometry') - - query = 'INSERT INTO place (%s, geometry) values(%s, ST_SetSRID(%s, 4326))' % ( - ','.join(cols.iterkeys()), - ','.join(['%s' for x in range(len(cols))]), - coords - ) - cur.execute(query, cols.values()) - world.conn.commit() - -@step(u'the scene (.*)') -def import_set_scene(step, scene): - world.load_scene(scene) - -@step(u'the (named )?place (node|way|area)s') -def import_place_table_nodes(step, named, osmtype): - """Insert a list of nodes into the place table. - Expects a table where columns are named in the same way as place. - """ - cur = world.conn.cursor() - cur.execute('ALTER TABLE place DISABLE TRIGGER place_before_insert') - if osmtype == 'node': - _insert_place_table_nodes(step.hashes, named is not None) - elif osmtype == 'way' : - _insert_place_table_objects(step.hashes, 'LINESTRING', named is not None) - elif osmtype == 'area' : - _insert_place_table_objects(step.hashes, 'POLYGON', named is not None) - cur.execute('ALTER TABLE place ENABLE TRIGGER place_before_insert') - cur.close() - world.conn.commit() - - -@step(u'the relations') -def import_fill_planet_osm_rels(step): - """Adds a raw relation to the osm2pgsql table. - Three columns need to be suplied: id, tags, members. - """ - cur = world.conn.cursor() - for line in step.hashes: - members = [] - parts = { 'n' : [], 'w' : [], 'r' : [] } - if line['members'].strip(): - for mem in line['members'].split(','): - memparts = mem.strip().split(':', 2) - memid = memparts[0].lower() - parts[memid[0]].append(int(memid[1:])) - members.append(memid) - if len(memparts) == 2: - members.append(memparts[1]) - else: - members.append('') - tags = [] - for k,v in world.make_hash(line['tags']).iteritems(): - tags.extend((k,v)) - if not members: - members = None - - cur.execute("""INSERT INTO planet_osm_rels - (id, way_off, rel_off, parts, members, tags) - VALUES (%s, %s, %s, %s, %s, %s)""", - (line['id'], len(parts['n']), len(parts['n']) + len(parts['w']), - parts['n'] + parts['w'] + parts['r'], members, tags)) - world.conn.commit() - - -@step(u'the ways') -def import_fill_planet_osm_ways(step): - cur = world.conn.cursor() - for line in step.hashes: - if 'tags' in line: - tags = world.make_hash(line['tags']) - else: - tags = None - nodes = [int(x.strip()) for x in line['nodes'].split(',')] - - cur.execute("""INSERT INTO planet_osm_ways (id, nodes, tags) - VALUES (%s, %s, %s)""", - (line['id'], nodes, tags)) - world.conn.commit() - -############### import and update steps ####################################### - -@step(u'importing') -def import_database(step): - """ Runs the actual indexing. """ - world.run_nominatim_script('setup', 'create-functions', 'create-partition-functions') - cur = world.conn.cursor() - #world.db_dump_table('place') - cur.execute("""insert into placex (osm_type, osm_id, class, type, name, admin_level, - housenumber, street, addr_place, isin, postcode, country_code, extratags, - geometry) select * from place where not (class='place' and type='houses' and osm_type='W')""") - cur.execute("""select insert_osmline (osm_id, housenumber, street, addr_place, postcode, country_code, geometry) from place where class='place' and type='houses' and osm_type='W'""") - world.conn.commit() - world.run_nominatim_script('setup', 'index', 'index-noanalyse') - #world.db_dump_table('placex') - #world.db_dump_table('location_property_osmline') - -@step(u'updating place (node|way|area)s') -def update_place_table_nodes(step, osmtype): - """ Replace a geometry in place by reinsertion and reindex database.""" - world.run_nominatim_script('setup', 'create-functions', 'create-partition-functions', 'enable-diff-updates') - if osmtype == 'node': - _insert_place_table_nodes(step.hashes, False) - elif osmtype == 'way': - _insert_place_table_objects(step.hashes, 'LINESTRING', False) - elif osmtype == 'area': - _insert_place_table_objects(step.hashes, 'POLYGON', False) - world.run_nominatim_script('update', 'index') - -@step(u'marking for delete (.*)') -def update_delete_places(step, places): - """ Remove an entry from place and reindex database. - """ - world.run_nominatim_script('setup', 'create-functions', 'create-partition-functions', 'enable-diff-updates') - cur = world.conn.cursor() - for place in places.split(','): - osmtype, osmid, cls = world.split_id(place) - if cls is None: - q = "delete from place where osm_type = %s and osm_id = %s" - params = (osmtype, osmid) - else: - q = "delete from place where osm_type = %s and osm_id = %s and class = %s" - params = (osmtype, osmid, cls) - cur.execute(q, params) - world.conn.commit() - #world.db_dump_table('placex') - world.run_nominatim_script('update', 'index') - - - -@step(u'sending query "(.*)"( with dups)?$') -def query_cmd(step, query, with_dups): - """ Results in standard query output. The same tests as for API queries - can be used. - """ - cmd = [os.path.join(world.config.source_dir, 'utils', 'query.php'), - '--search', query] - if with_dups is not None: - cmd.append('--nodedupe') - proc = subprocess.Popen(cmd, cwd=world.config.source_dir, - stdout=subprocess.PIPE, stderr=subprocess.PIPE) - (outp, err) = proc.communicate() - assert (proc.returncode == 0), "query.php failed with message: %s" % err - world.page = outp - world.response_format = 'json' - world.request_type = 'search' - world.returncode = 200 - diff --git a/tests/steps/osm2pgsql_setup.py b/tests/steps/osm2pgsql_setup.py deleted file mode 100644 index 4b03b1ea..00000000 --- a/tests/steps/osm2pgsql_setup.py +++ /dev/null @@ -1,214 +0,0 @@ -""" Steps for setting up a test database for osm2pgsql import. - - Note that osm2pgsql features need a database and therefore need - to be tagged with @DB. -""" - -from nose.tools import * -from lettuce import * - -import logging -import random -import tempfile -import os -import subprocess - -logger = logging.getLogger(__name__) - -@before.each_scenario -def osm2pgsql_setup_test(scenario): - world.osm2pgsql = [] - -@step(u'the osm nodes:') -def osm2pgsql_import_nodes(step): - """ Define a list of OSM nodes to be imported, given as a table. - Each line describes one node with all its attributes. - 'id' is mendatory, all other fields are filled with random values - when not given. If 'tags' is missing an empty tag list is assumed. - For updates, a mandatory 'action' column needs to contain 'A' (add), - 'M' (modify), 'D' (delete). - """ - for line in step.hashes: - node = { 'type' : 'N', 'version' : '1', 'timestamp': "2012-05-01T15:06:20Z", - 'changeset' : "11470653", 'uid' : "122294", 'user' : "foo" - } - node.update(line) - node['id'] = int(node['id']) - if 'geometry' in node: - lat, lon = node['geometry'].split(' ') - node['lat'] = float(lat) - node['lon'] = float(lon) - else: - node['lon'] = random.random()*360 - 180 - node['lat'] = random.random()*180 - 90 - if 'tags' in node: - node['tags'] = world.make_hash(line['tags']) - else: - node['tags'] = {} - - world.osm2pgsql.append(node) - - -@step(u'the osm ways:') -def osm2pgsql_import_ways(step): - """ Define a list of OSM ways to be imported. - """ - for line in step.hashes: - way = { 'type' : 'W', 'version' : '1', 'timestamp': "2012-05-01T15:06:20Z", - 'changeset' : "11470653", 'uid' : "122294", 'user' : "foo" - } - way.update(line) - - way['id'] = int(way['id']) - if 'tags' in way: - way['tags'] = world.make_hash(line['tags']) - else: - way['tags'] = None - way['nodes'] = way['nodes'].strip().split() - - world.osm2pgsql.append(way) - -membertype = { 'N' : 'node', 'W' : 'way', 'R' : 'relation' } - -@step(u'the osm relations:') -def osm2pgsql_import_rels(step): - """ Define a list of OSM relation to be imported. - """ - for line in step.hashes: - rel = { 'type' : 'R', 'version' : '1', 'timestamp': "2012-05-01T15:06:20Z", - 'changeset' : "11470653", 'uid' : "122294", 'user' : "foo" - } - rel.update(line) - - rel['id'] = int(rel['id']) - if 'tags' in rel: - rel['tags'] = world.make_hash(line['tags']) - else: - rel['tags'] = {} - members = [] - if rel['members'].strip(): - for mem in line['members'].split(','): - memparts = mem.strip().split(':', 2) - memid = memparts[0].upper() - members.append((membertype[memid[0]], - memid[1:], - memparts[1] if len(memparts) == 2 else '' - )) - rel['members'] = members - - world.osm2pgsql.append(rel) - - - -def _sort_xml_entries(x, y): - if x['type'] == y['type']: - return cmp(x['id'], y['id']) - else: - return cmp('NWR'.find(x['type']), 'NWR'.find(y['type'])) - -def write_osm_obj(fd, obj): - if obj['type'] == 'N': - fd.write('\n') - else: - fd.write('>\n') - for k,v in obj['tags'].iteritems(): - fd.write(' \n' % (k, v)) - fd.write('\n') - elif obj['type'] == 'W': - fd.write('\n' % obj) - for nd in obj['nodes']: - fd.write('\n' % (nd,)) - for k,v in obj['tags'].iteritems(): - fd.write(' \n' % (k, v)) - fd.write('\n') - elif obj['type'] == 'R': - fd.write('\n' % obj) - for mem in obj['members']: - fd.write(' \n' % mem) - for k,v in obj['tags'].iteritems(): - fd.write(' \n' % (k, v)) - fd.write('\n') - -@step(u'loading osm data') -def osm2pgsql_load_place(step): - """Imports the previously defined OSM data into a fresh copy of a - Nominatim test database. - """ - - world.osm2pgsql.sort(cmp=_sort_xml_entries) - - # create a OSM file in /tmp - with tempfile.NamedTemporaryFile(dir='/tmp', suffix='.osm', delete=False) as fd: - fname = fd.name - fd.write("\n") - fd.write('\n') - fd.write('\t\n') - - for obj in world.osm2pgsql: - write_osm_obj(fd, obj) - - fd.write('\n') - - logger.debug( "Filename: %s" % fname) - - cmd = [os.path.join(world.config.source_dir, 'utils', 'setup.php')] - cmd.extend(['--osm-file', fname, '--import-data','--osm2pgsql-cache', '300']) - proc = subprocess.Popen(cmd, cwd=world.config.source_dir, - stdout=subprocess.PIPE, stderr=subprocess.PIPE) - (outp, outerr) = proc.communicate() - assert (proc.returncode == 0), "OSM data import failed:\n%s\n%s\n" % (outp, outerr) - - ### reintroduce the triggers/indexes we've lost by having osm2pgsql set up place again - cur = world.conn.cursor() - cur.execute("""CREATE TRIGGER place_before_delete BEFORE DELETE ON place - FOR EACH ROW EXECUTE PROCEDURE place_delete()""") - cur.execute("""CREATE TRIGGER place_before_insert BEFORE INSERT ON place - FOR EACH ROW EXECUTE PROCEDURE place_insert()""") - cur.execute("""CREATE UNIQUE INDEX idx_place_osm_unique on place using btree(osm_id,osm_type,class,type)""") - world.conn.commit() - - - os.remove(fname) - world.osm2pgsql = [] - -actiontypes = { 'C' : 'create', 'M' : 'modify', 'D' : 'delete' } - -@step(u'updating osm data') -def osm2pgsql_update_place(step): - """Creates an osc file from the previously defined data and imports it - into the database. - """ - world.run_nominatim_script('setup', 'create-functions', 'create-partition-functions') - cur = world.conn.cursor() - cur.execute("""insert into placex (osm_type, osm_id, class, type, name, admin_level, - housenumber, street, addr_place, isin, postcode, country_code, extratags, - geometry) select * from place""") - world.conn.commit() - world.run_nominatim_script('setup', 'index', 'index-noanalyse') - world.run_nominatim_script('setup', 'create-functions', 'create-partition-functions', 'enable-diff-updates') - - with tempfile.NamedTemporaryFile(dir='/tmp', suffix='.osc', delete=False) as fd: - fname = fd.name - fd.write("\n") - fd.write('\n') - - for obj in world.osm2pgsql: - fd.write('<%s>\n' % (actiontypes[obj['action']], )) - write_osm_obj(fd, obj) - fd.write('\n' % (actiontypes[obj['action']], )) - - fd.write('\n') - - logger.debug( "Filename: %s" % fname) - - cmd = [os.path.join(world.config.source_dir, 'utils', 'update.php')] - cmd.extend(['--import-diff', fname]) - proc = subprocess.Popen(cmd, cwd=world.config.source_dir, - stdout=subprocess.PIPE, stderr=subprocess.PIPE) - (outp, outerr) = proc.communicate() - assert (proc.returncode == 0), "OSM data update failed:\n%s\n%s\n" % (outp, outerr) - - os.remove(fname) - world.osm2pgsql = [] diff --git a/tests/steps/terrain.py b/tests/steps/terrain.py deleted file mode 100644 index 80beebd5..00000000 --- a/tests/steps/terrain.py +++ /dev/null @@ -1,255 +0,0 @@ -from lettuce import * -from nose.tools import * -import logging -import os -import subprocess -import psycopg2 -import re -from haversine import haversine -from shapely.wkt import loads as wkt_load -from shapely.ops import linemerge - -logger = logging.getLogger(__name__) - -class NominatimConfig: - - def __init__(self): - # logging setup - loglevel = getattr(logging, os.environ.get('LOGLEVEL','info').upper()) - if 'LOGFILE' in os.environ: - logging.basicConfig(filename=os.environ.get('LOGFILE','run.log'), - level=loglevel) - else: - logging.basicConfig(level=loglevel) - # Nominatim test setup - self.base_url = os.environ.get('NOMINATIM_SERVER', 'http://localhost/nominatim') - self.source_dir = os.path.abspath(os.environ.get('NOMINATIM_DIR', '../build')) - self.template_db = os.environ.get('TEMPLATE_DB', 'test_template_nominatim') - self.test_db = os.environ.get('TEST_DB', 'test_nominatim') - self.local_settings_file = os.environ.get('NOMINATIM_SETTINGS', '/tmp/nominatim_settings.php') - self.reuse_template = 'NOMINATIM_REMOVE_TEMPLATE' not in os.environ - self.keep_scenario_db = 'NOMINATIM_KEEP_SCENARIO_DB' in os.environ - os.environ['NOMINATIM_SETTINGS'] = '/tmp/nominatim_settings.php' - - scriptpath = os.path.dirname(os.path.abspath(__file__)) - self.scene_path = os.environ.get('SCENE_PATH', - os.path.join(scriptpath, '..', 'scenes', 'data')) - - - def __str__(self): - return 'Server URL: %s\nSource dir: %s\n' % (self.base_url, self.source_dir) - -world.config = NominatimConfig() - -@world.absorb -def write_nominatim_config(dbname): - f = open(world.config.local_settings_file, 'w') - f.write("[:class]. - """ - oid = oid.strip() - if oid == 'None': - return None, None, None - osmtype = oid[0] - assert_in(osmtype, ('R','N','W')) - if ':' in oid: - osmid, cls = oid[1:].split(':') - return (osmtype, int(osmid), cls) - else: - return (osmtype, int(oid[1:]), None) - -@world.absorb -def get_placeid(oid): - """ Tries to retrive the place_id for a unique identifier. """ - if oid[0].isdigit(): - return int(oid) - - osmtype, osmid, cls = world.split_id(oid) - if osmtype is None: - return None - cur = world.conn.cursor() - if cls is None: - q = 'SELECT place_id FROM placex where osm_type = %s and osm_id = %s' - params = (osmtype, osmid) - else: - q = 'SELECT place_id FROM placex where osm_type = %s and osm_id = %s and class = %s' - params = (osmtype, osmid, cls) - cur.execute(q, params) - assert_equals(cur.rowcount, 1, "%d rows found for place %s" % (cur.rowcount, oid)) - return cur.fetchone()[0] - - -@world.absorb -def match_geometry(coord, matchstring): - m = re.match(r'([-0-9.]+),\s*([-0-9.]+)\s*(?:\+-([0-9.]+)([a-z]+)?)?', matchstring) - assert_is_not_none(m, "Invalid match string") - - logger.debug("Distmatch: %s/%s %s %s" % (m.group(1), m.group(2), m.group(3), m.group(4) )) - dist = haversine(coord, (float(m.group(1)), float(m.group(2)))) - - if m.group(3) is not None: - expdist = float(m.group(3)) - if m.group(4) is not None: - if m.group(4) == 'm': - expdist = expdist/1000 - elif m.group(4) == 'km': - pass - else: - raise Exception("Unknown unit '%s' in geometry match" % (m.group(4), )) - else: - expdist = 0 - - logger.debug("Distances expected: %f, got: %f" % (expdist, dist)) - assert dist <= expdist, "Geometry too far away, expected: %f, got: %f" % (expdist, dist) - -@world.absorb -def print_statement(element): - print '\n\n\n'+str(element)+'\n\n\n' - - -@world.absorb -def db_dump_table(table): - cur = world.conn.cursor() - cur.execute('SELECT * FROM %s' % table) - print '\n\n\n<<<<<<< BEGIN OF TABLE DUMP %s' % table - for res in cur: - print res - print '<<<<<<< END OF TABLE DUMP %s\n\n\n' % table - -@world.absorb -def db_drop_database(name): - conn = psycopg2.connect(database='postgres') - conn.set_isolation_level(0) - cur = conn.cursor() - cur.execute('DROP DATABASE IF EXISTS %s' % (name, )) - conn.close() - - -world.is_template_set_up = False - -@world.absorb -def db_template_setup(): - """ Set up a template database, containing all tables - but not yet any functions. - """ - if world.is_template_set_up: - return - - world.is_template_set_up = True - world.write_nominatim_config(world.config.template_db) - if world.config.reuse_template: - # check that the template is there - conn = psycopg2.connect(database='postgres') - cur = conn.cursor() - cur.execute('select count(*) from pg_database where datname = %s', - (world.config.template_db,)) - if cur.fetchone()[0] == 1: - return - else: - # just in case... make sure a previous table has been dropped - world.db_drop_database(world.config.template_db) - # call the first part of database setup - world.run_nominatim_script('setup', 'create-db', 'setup-db') - # remove external data to speed up indexing for tests - conn = psycopg2.connect(database=world.config.template_db) - psycopg2.extras.register_hstore(conn, globally=False, unicode=True) - cur = conn.cursor() - for table in ('gb_postcode', 'us_postcode'): - cur.execute("select * from pg_tables where tablename = '%s'" % (table, )) - if cur.rowcount > 0: - cur.execute('TRUNCATE TABLE %s' % (table,)) - conn.commit() - conn.close() - # execute osm2pgsql on an empty file to get the right tables - osm2pgsql = os.path.join(world.config.source_dir, 'osm2pgsql', 'osm2pgsql') - proc = subprocess.Popen([osm2pgsql, '-lsc', '-r', 'xml', '-O', 'gazetteer', '-d', world.config.template_db, '-'], - cwd=world.config.source_dir, stdin=subprocess.PIPE, - stdout=subprocess.PIPE, stderr=subprocess.PIPE) - [outstr, errstr] = proc.communicate(input='') - logger.debug("running osm2pgsql for template: %s\n%s\n%s" % (osm2pgsql, outstr, errstr)) - world.run_nominatim_script('setup', 'create-functions', 'create-tables', 'create-partition-tables', 'create-partition-functions', 'load-data', 'create-search-indices') - - -# Leave the table around so it can be reused again after a non-reuse test round. -#@after.all -def db_template_teardown(total): - """ Set up a template database, containing all tables - but not yet any functions. - """ - if world.is_template_set_up: - # remove template DB - if not world.config.reuse_template: - world.db_drop_database(world.config.template_db) - try: - os.remove(world.config.local_settings_file) - except OSError: - pass # ignore missing file - - -########################################################################## -# -# Data scene handling -# - -world.scenes = {} -world.current_scene = None - -@world.absorb -def load_scene(name): - if name in world.scenes: - world.current_scene = world.scenes[name] - else: - with open(os.path.join(world.config.scene_path, "%s.wkt" % name), 'r') as fd: - scene = {} - for line in fd: - if line.strip(): - obj, wkt = line.split('|', 2) - wkt = wkt.strip() - scene[obj.strip()] = wkt_load(wkt) - world.scenes[name] = scene - world.current_scene = scene - -@world.absorb -def get_scene_geometry(name): - if not ':' in name: - # Not a scene description - return None - - geoms = [] - for obj in name.split('+'): - oname = obj.strip() - if oname.startswith(':'): - geoms.append(world.current_scene[oname[1:]]) - else: - scene, obj = oname.split(':', 2) - oldscene = world.current_scene - world.load_scene(scene) - wkt = world.current_scene[obj] - world.current_scene = oldscene - geoms.append(wkt) - - if len(geoms) == 1: - return geoms[0] - else: - return linemerge(geoms) diff --git a/utils/setup.php b/utils/setup.php index 06216587..e4e91674 100755 --- a/utils/setup.php +++ b/utils/setup.php @@ -292,8 +292,8 @@ if ($aCMDResult['create-partition-functions'] || $aCMDResult['all']) { if ($aCMDResult['import-wikipedia-articles'] || $aCMDResult['all']) { $bDidSomething = true; - $sWikiArticlesFile = CONST_BasePath.'/data/wikipedia_article.sql.bin'; - $sWikiRedirectsFile = CONST_BasePath.'/data/wikipedia_redirect.sql.bin'; + $sWikiArticlesFile = CONST_Wikipedia_Data_Path.'/wikipedia_article.sql.bin'; + $sWikiRedirectsFile = CONST_Wikipedia_Data_Path.'/wikipedia_redirect.sql.bin'; if (file_exists($sWikiArticlesFile)) { echo "Importing wikipedia articles..."; pgsqlRunDropAndRestore($sWikiArticlesFile);