From: Sarah Hoffmann Date: Mon, 20 Aug 2018 21:28:32 +0000 (+0200) Subject: Merge remote-tracking branch 'upstream/master' X-Git-Tag: deploy~311 X-Git-Url: https://git.openstreetmap.org./nominatim.git/commitdiff_plain/ae2ad38040f8515caa79bf274ddc345c6efeb6ef?hp=6f383d6073b68409e8da243356eccf5e2847f264 Merge remote-tracking branch 'upstream/master' --- diff --git a/docs/api/Overview.md b/docs/api/Overview.md index 8bf14c43..3fb1c242 100644 --- a/docs/api/Overview.md +++ b/docs/api/Overview.md @@ -1,12 +1,12 @@ ### Nominatim API -Nominatim indexes named (or numbered) features with the OSM data set and a subset of other unnamed features (pubs, hotels, churches, etc). +Nominatim indexes named (or numbered) features within the OpenStreetMap (OSM) dataset and a subset of other unnamed features (pubs, hotels, churches, etc). Its API has the following endpoints for querying the data: * __[/search](Search.md)__ - search OSM objects by name or type * __[/reverse](Reverse.md)__ - search OSM object by their location - * __[/lookup](Lookup.md)__ - look up address details for OSM objects by thier ID + * __[/lookup](Lookup.md)__ - look up address details for OSM objects by their ID * __/status__ - query the status of the server * __/deletable__ - list objects that have been deleted in OSM but are held back in Nominatim in case the deletion was accidental diff --git a/docs/api/Reverse.md b/docs/api/Reverse.md index ae368e1c..22e01331 100644 --- a/docs/api/Reverse.md +++ b/docs/api/Reverse.md @@ -34,7 +34,7 @@ See [Place Output Formats](Output.md) for details on each format. (Default: html * `json_callback=` -Wrap json output in a callback function (JSONP) i.e. `()`. +Wrap json output in a callback function ([JSONP](https://en.wikipedia.org/wiki/JSONP)) i.e. `()`. Only has an effect for JSON output formats. ### Output details @@ -69,7 +69,7 @@ comma-separated list of language codes. * `zoom=[0-18]` -Level of detail required for the address. This is a number that corresponds +Level of detail required for the address. Default: 18. This is a number that corresponds roughly to the zoom level used in map frameworks like Leaflet.js, Openlayers etc. In terms of address details the zoom levels are as follows: diff --git a/docs/api/Search.md b/docs/api/Search.md index ab702379..4b9d773c 100644 --- a/docs/api/Search.md +++ b/docs/api/Search.md @@ -5,10 +5,9 @@ Nominatim supports structured as well as free-form search queries. The search query may also contain [special phrases](https://wiki.openstreetmap.org/wiki/Nominatim/Special_Phrases) -which are tranlated into specific OpenStreetMap(OSM) tags (e.g. Pub => amenity=pub). -Note that this only limits the items to be found. It is not suited to return complete -lists of OSM objects of a specific type. -Use [Overpass API](https://overpass-api.de/) for that. +which are translated into specific OpenStreetMap (OSM) tags (e.g. Pub => `amenity=pub`). +Note that this only limits the items to be found, it's not suited to return complete +lists of OSM objects of a specific type. For those use [Overpass API](https://overpass-api.de/). ## Parameters @@ -30,22 +29,20 @@ In this form, the query may be given through two different sets of parameters: * `q=` Free-form query string to search for. - Free-form queries are processed first left to right and then right to - left if that fails. So you may search for + Free-form queries are processed first left-to-right and then right-to-left if that fails. So you may search for [pilkington avenue, birmingham](//nominatim.openstreetmap.org/search?q=pilkington+avenue,birmingham) as well as for [birmingham, pilkington avenue](//nominatim.openstreetmap.org/search?q=birmingham,+pilkington+avenue). - Commas are optional, but improve performance by reducing the complexity - of the search. + Commas are optional, but improve performance by reducing the complexity of the search. * `street= ` - `city=` - `county=` - `state=` - `country=` - `postalcode=` +* `city=` +* `county=` +* `state=` +* `country=` +* `postalcode=` - Alternative query string format for structured requests. + Alternative query string format split into several parameters for structured requests. Structured requests are faster but are less robust against alternative OSM tagging schemas. **Do not combine with** `q=` **parameter**. @@ -59,7 +56,7 @@ See [Place Output Formats](Output.md) for details on each format. (Default: html * `json_callback=` -Wrap json output in a callback function (JSONP) i.e. `()`. +Wrap json output in a callback function ([JSONP](https://en.wikipedia.org/wiki/JSONP)) i.e. `()`. Only has an effect for JSON output formats. ### Output details @@ -86,7 +83,7 @@ language variants, references, operator and brand. (Default: 0) * `accept-language=` Preferred language order for showing search results, overrides the value -specified in the "Accept-Language" HTTP header. +specified in the ["Accept-Language" HTTP header](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Accept-Language). Either use a standard RFC2616 accept-language string or a simple comma-separated list of language codes. @@ -109,7 +106,7 @@ the search to return other, less accurate, matches (if possible). * `limit=` -Limit the number of returned results. (Default: 10) +Limit the number of returned results. (Default: 10, Maximum: 50) * `viewbox=,,,` @@ -121,9 +118,9 @@ are accepted in any order as long as they span a real box. * `bounded=[0|1]` When a viewbox is given, restrict the result to items contained with that -viewbox (see above). When `viewbox` and `bounded` are given, an amenity +viewbox (see above). When `viewbox` and `bounded=1` are given, an amenity only search is allowed. In this case, give the special keyword for the -amenity in squaer brackets, e.g. `[pub]`. (Default: 0) +amenity in square brackets, e.g. `[pub]`. (Default: 0) ### Polygon output diff --git a/lib/DebugHtml.php b/lib/DebugHtml.php index a600fae5..98da8794 100644 --- a/lib/DebugHtml.php +++ b/lib/DebugHtml.php @@ -155,6 +155,8 @@ class Debug } } elseif (is_object($mVar) && method_exists($mVar, 'debugInfo')) { Debug::outputVar($mVar->debugInfo(), $sPreNL); + } elseif (is_a($mVar, 'stdClass')) { + Debug::outputVar(json_decode(json_encode($mVar), true), $sPreNL); } else { Debug::outputSimpleVar($mVar); } diff --git a/lib/ReverseGeocode.php b/lib/ReverseGeocode.php index 681403a1..fc5e04d6 100644 --- a/lib/ReverseGeocode.php +++ b/lib/ReverseGeocode.php @@ -47,13 +47,12 @@ class ReverseGeocode /** * Find the closest interpolation with the given search diameter. * - * @param string $sPointSQL Reverse geocoding point as SQL - * @param float $fSearchDiam Search diameter - * @param integer $iParentPlaceID Id of parent object + * @param string $sPointSQL Reverse geocoding point as SQL + * @param float $fSearchDiam Search diameter * * @return Record of the interpolation or null. */ - protected function lookupInterpolation($sPointSQL, $fSearchDiam, $iParentPlaceID = null) + protected function lookupInterpolation($sPointSQL, $fSearchDiam) { $sSQL = 'SELECT place_id, parent_place_id, 30 as rank_search,'; $sSQL .= ' ST_LineLocatePoint(linegeo,'.$sPointSQL.') as fraction,'; @@ -62,9 +61,6 @@ class ReverseGeocode $sSQL .= ' FROM location_property_osmline'; $sSQL .= ' WHERE ST_DWithin('.$sPointSQL.', linegeo, '.$fSearchDiam.')'; $sSQL .= ' and indexed_status = 0 and startnumber is not NULL '; - if (isset($iParentPlaceID)) { - $sSQL .= ' and parent_place_id = '.$iParentPlaceID; - } $sSQL .= ' ORDER BY distance ASC limit 1'; return chksql( @@ -73,62 +69,59 @@ class ReverseGeocode ); } - protected function polygonFunctions($sPointSQL, $iMaxRank) + protected function lookupLargeArea($sPointSQL, $iMaxRank) { - // starts the nopolygonFound function if no polygon is found with the lookupPolygon function $oResult = null; - $aPlace = $this->lookupPolygon($sPointSQL, $iMaxRank); - if ($aPlace) { - $oResult = new Result($aPlace['place_id']); - // if no polygon which contains the searchpoint is found, - // the noPolygonFound function searches in the country_osm_grid table for a polygon - } elseif (!$aPlace && $iMaxRank > 4) { - $aPlace = $this->noPolygonFound($sPointSQL, $iMaxRank); + if ($iMaxRank > 4) { + $aPlace = $this->lookupPolygon($sPointSQL, $iMaxRank); if ($aPlace) { - $oResult = new Result($aPlace['place_id']); + return new Result($aPlace['place_id']); } } - return $oResult; + + // If no polygon which contains the searchpoint is found, + // searches in the country_osm_grid table for a polygon. + return $this->lookupInCountry($sPointSQL, $iMaxRank); } - protected function noPolygonFound($sPointSQL, $iMaxRank) + protected function lookupInCountry($sPointSQL, $iMaxRank) { // searches for polygon in table country_osm_grid which contains the searchpoint // and searches for the nearest place node to the searchpoint in this polygon $sSQL = 'SELECT country_code FROM country_osm_grid'; - $sSQL .= ' WHERE ST_CONTAINS (geometry, '.$sPointSQL.') limit 1'; + $sSQL .= ' WHERE ST_CONTAINS(geometry, '.$sPointSQL.') LIMIT 1'; - $aPoly = chksql( - $this->oDB->getRow($sSQL), - 'Could not determine polygon containing the point.' + $sCountryCode = chksql( + $this->oDB->getOne($sSQL), + 'Could not determine country polygon containing the point.' ); - if ($aPoly) { - $sCountryCode = $aPoly['country_code']; - - // look for place nodes with the given country code - $sSQL = 'SELECT place_id FROM'; - $sSQL .= ' (SELECT place_id, rank_search,'; - $sSQL .= ' ST_distance('.$sPointSQL.', geometry) as distance'; - $sSQL .= ' FROM placex'; - $sSQL .= ' WHERE osm_type = \'N\''; - $sSQL .= ' AND country_code = \''.$sCountryCode.'\''; - $sSQL .= ' AND rank_search between 5 and ' .min(25, $iMaxRank); - $sSQL .= ' AND class = \'place\' AND type != \'postcode\''; - $sSQL .= ' AND name IS NOT NULL '; - $sSQL .= ' and indexed_status = 0 and linked_place_id is null'; - $sSQL .= ' AND ST_DWithin('.$sPointSQL.', geometry, 1.8)) p '; - $sSQL .= 'WHERE distance <= reverse_place_diameter(rank_search)'; - $sSQL .= ' ORDER BY rank_search DESC, distance ASC'; - $sSQL .= ' LIMIT 1'; + if ($sCountryCode) { + if ($iMaxRank > 4) { + // look for place nodes with the given country code + $sSQL = 'SELECT place_id FROM'; + $sSQL .= ' (SELECT place_id, rank_search,'; + $sSQL .= ' ST_distance('.$sPointSQL.', geometry) as distance'; + $sSQL .= ' FROM placex'; + $sSQL .= ' WHERE osm_type = \'N\''; + $sSQL .= ' AND country_code = \''.$sCountryCode.'\''; + $sSQL .= ' AND rank_search between 5 and ' .min(25, $iMaxRank); + $sSQL .= ' AND class = \'place\' AND type != \'postcode\''; + $sSQL .= ' AND name IS NOT NULL '; + $sSQL .= ' and indexed_status = 0 and linked_place_id is null'; + $sSQL .= ' AND ST_DWithin('.$sPointSQL.', geometry, 1.8)) p '; + $sSQL .= 'WHERE distance <= reverse_place_diameter(rank_search)'; + $sSQL .= ' ORDER BY rank_search DESC, distance ASC'; + $sSQL .= ' LIMIT 1'; - if (CONST_Debug) var_dump($sSQL); - $aPlacNode = chksql( - $this->oDB->getRow($sSQL), - 'Could not determine place node.' - ); - if ($aPlacNode) { - return $aPlacNode; + if (CONST_Debug) var_dump($sSQL); + $aPlace = chksql( + $this->oDB->getRow($sSQL), + 'Could not determine place node.' + ); + if ($aPlace) { + return new Result($aPlace['place_id']); + } } // still nothing, then return the country object @@ -140,21 +133,32 @@ class ReverseGeocode $sSQL .= ' ORDER BY distance ASC'; if (CONST_Debug) var_dump($sSQL); - $aPlacNode = chksql( + $aPlace = chksql( $this->oDB->getRow($sSQL), 'Could not determine place node.' ); - if ($aPlacNode) { - return $aPlacNode; + if ($aPlace) { + return new Result($aPlace['place_id']); } } + + return null; } + /** + * Search for areas or nodes for areas or nodes between state and suburb level. + * + * @param string $sPointSQL Search point as SQL string. + * @param int $iMaxRank Maximum address rank of the feature. + * + * @return Record of the found feature or null. + * + * Searches first for polygon that contains the search point. + * If such a polygon is found, place nodes with a higher rank are + * searched inside the polygon. + */ protected function lookupPolygon($sPointSQL, $iMaxRank) { - // searches for polygon where the searchpoint is within - // if a polygon is found, placenodes with a higher rank are searched inside the polygon - // polygon search begins at suburb-level if ($iMaxRank > 25) $iMaxRank = 25; // no polygon search over country-level @@ -241,8 +245,6 @@ class ReverseGeocode $fSearchDiam = 0.006; $oResult = null; $aPlace = null; - $fMaxAreaDistance = 1; - $bIsTigerStreet = false; // for POI or street level if ($iMaxRank >= 26) { @@ -271,28 +273,40 @@ class ReverseGeocode 'Could not determine closest place.' ); + if (CONST_Debug) var_dump($aPlace); if ($aPlace) { - $iDistance = $aPlace['distance']; $iPlaceID = $aPlace['place_id']; $oResult = new Result($iPlaceID); + $iRankAddress = $aPlace['rank_address']; $iParentPlaceID = $aPlace['parent_place_id']; + } - if ($bDoInterpolation && $iMaxRank >= 30) { - if ($aPlace['rank_address'] <=27) { - $iDistance = 0.001; - } - $aHouse = $this->lookupInterpolation($sPointSQL, $iDistance); + if ($bDoInterpolation && $iMaxRank >= 30) { + $fDistance = $fSearchDiam; + if ($aPlace) { + // We can't reliably go from the closest street to an + // interpolation line because the closest interpolation + // may have a different street segments as a parent. + // Therefore allow an interpolation line to take precendence + // even when the street is closer. + $fDistance = $iRankAddress < 28 ? 0.001 : $aPlace['distance']; + } - if ($aHouse) { - $oResult = new Result($aHouse['place_id'], Result::TABLE_OSMLINE); - $oResult->iHouseNumber = closestHouseNumber($aHouse); - } + $aHouse = $this->lookupInterpolation($sPointSQL, $fDistance); + + if ($aHouse) { + $oResult = new Result($aHouse['place_id'], Result::TABLE_OSMLINE); + $oResult->iHouseNumber = closestHouseNumber($aHouse); + $aPlace = $aHouse; + $iRankAddress = 30; } + } + if ($aPlace) { // if street and maxrank > streetlevel - if (($aPlace['rank_address'] <=27)&& $iMaxRank > 27) { + if ($iRankAddress <= 27 && $iMaxRank > 27) { // find the closest object (up to a certain radius) of which the street is a parent of - $sSQL = ' select place_id,parent_place_id,rank_address,country_code,'; + $sSQL = ' select place_id,'; $sSQL .= ' ST_distance('.$sPointSQL.', geometry) as distance'; $sSQL .= ' FROM '; $sSQL .= ' placex'; @@ -310,31 +324,23 @@ class ReverseGeocode 'Could not determine closest place.' ); if ($aStreet) { - $iDistance = $aStreet['distance']; - $iPlaceID = $aStreet['place_id']; - $oResult = new Result($iPlaceID); - $iParentPlaceID = $aStreet['parent_place_id']; - - if ($bDoInterpolation && $iMaxRank >= 30) { - $aHouse = $this->lookupInterpolation($sPointSQL, $iDistance, $iParentPlaceID); - - if ($aHouse) { - $oResult = new Result($aHouse['place_id'], Result::TABLE_OSMLINE); - $oResult->iHouseNumber = closestHouseNumber($aHouse); - } - } + if (CONST_Debug) var_dump($aStreet); + $oResult = new Result($aStreet['place_id']); } } // In the US we can check TIGER data for nearest housenumber - if (CONST_Use_US_Tiger_Data && $aPlace['country_code'] == 'us' && $this->iMaxRank >= 28) { - $fSearchDiam = $aPlace['rank_address'] > 28 ? $aPlace['distance'] : 0.001; + if (CONST_Use_US_Tiger_Data + && $iRankAddress <= 27 + && $aPlace['country_code'] == 'us' + && $this->iMaxRank >= 28 + ) { $sSQL = 'SELECT place_id,parent_place_id,30 as rank_search,'; $sSQL .= 'ST_LineLocatePoint(linegeo,'.$sPointSQL.') as fraction,'; $sSQL .= 'ST_distance('.$sPointSQL.', linegeo) as distance,'; $sSQL .= 'startnumber,endnumber,interpolationtype'; $sSQL .= ' FROM location_property_tiger WHERE parent_place_id = '.$oResult->iId; - $sSQL .= ' AND ST_DWithin('.$sPointSQL.', linegeo, '.$fSearchDiam.')'; + $sSQL .= ' AND ST_DWithin('.$sPointSQL.', linegeo, 0.001)'; $sSQL .= ' ORDER BY distance ASC limit 1'; if (CONST_Debug) var_dump($sSQL); $aPlaceTiger = chksql( @@ -343,18 +349,17 @@ class ReverseGeocode ); if ($aPlaceTiger) { if (CONST_Debug) var_dump('found Tiger housenumber', $aPlaceTiger); - $aPlace = $aPlaceTiger; - $oResult = new Result($aPlace['place_id'], Result::TABLE_TIGER); + $oResult = new Result($aPlaceTiger['place_id'], Result::TABLE_TIGER); $oResult->iHouseNumber = closestHouseNumber($aPlaceTiger); } } - // if no POI or street is found ... } else { - $oResult = $this->PolygonFunctions($sPointSQL, $iMaxRank); + // if no POI or street is found ... + $oResult = $this->lookupLargeArea($sPointSQL, 25); } - // lower than street level ($iMaxRank < 26 ) } else { - $oResult = $this->PolygonFunctions($sPointSQL, $iMaxRank); + // lower than street level ($iMaxRank < 26 ) + $oResult = $this->lookupLargeArea($sPointSQL, $iMaxRank); } return $oResult; } diff --git a/lib/cmd.php b/lib/cmd.php index 9ec290d1..9efe5653 100644 --- a/lib/cmd.php +++ b/lib/cmd.php @@ -203,7 +203,8 @@ function runWithEnv($sCmd, $aEnv) $aFDs = array( 0 => array('pipe', 'r'), 1 => STDOUT, - 2 => STDERR); + 2 => STDERR + ); $aPipes = null; $hProc = @proc_open($sCmd, $aFDs, $aPipes, null, $aEnv); if (!is_resource($hProc)) { diff --git a/test/bdd/api/reverse/queries.feature b/test/bdd/api/reverse/queries.feature index e06b1775..88f3bccb 100644 --- a/test/bdd/api/reverse/queries.feature +++ b/test/bdd/api/reverse/queries.feature @@ -59,3 +59,35 @@ Feature: Reverse geocoding Then results contain | display_name | | Tacuarembó, Uruguay | + + Scenario Outline: Zoom levels below 5 result in country + When sending jsonv2 reverse coordinates -33.28,-56.29 + | zoom | + | | + Then results contain + | display_name | + | Uruguay | + + Examples: + | zoom | + | 0 | + | 1 | + | 2 | + | 3 | + | 4 | + + Scenario: When on a street, the closest interpolation is shown + When sending jsonv2 reverse coordinates -33.2309430210215,-54.38126470020989 + | zoom | + | 18 | + Then results contain + | display_name | + | 1429, Andrés Areguati, Treinta y Tres, 33000, Uruguay | + + Scenario: When on a street with zoom 18, the closest housenumber is returned + When sending jsonv2 reverse coordinates 53.551826690895226,9.885258475318201 + | zoom | + | 18 | + Then result addresses contain + | house_number | + | 33 | diff --git a/test/bdd/api/search/simple.feature b/test/bdd/api/search/simple.feature index 5cd80a83..ca441258 100644 --- a/test/bdd/api/search/simple.feature +++ b/test/bdd/api/search/simple.feature @@ -233,6 +233,17 @@ Feature: Simple Tests When sending xml search query "Vaduz" | countrycodes | | pl,1,,invalid,undefined,%3Cb%3E,bo,, | - Then result header contains + Then result header contains | attr | value | | more_url | .*&countrycodes=pl%2Cbo&.* | + + Scenario Outline: Search with debug prints valid HTML + When sending html search query "" + | extratags | addressdetails | namedetails | debug | + | 1 | 1 | 1 | 1 | + Then the result is valid html + + Examples: + | query | + | 10, Alvierweg, 9490, Vaduz | + | Hamburg | diff --git a/test/bdd/steps/queries.py b/test/bdd/steps/queries.py index fd13dd13..df34b5cc 100644 --- a/test/bdd/steps/queries.py +++ b/test/bdd/steps/queries.py @@ -122,13 +122,17 @@ class SearchResponse(GenericResponse): options={'char-encoding' : 'utf8'}) #eq_(len(errors), 0 , "Errors found in HTML document:\n%s" % errors) + self.result = [] b = content.find('nominatim_results =') e = content.find('') - content = content[b:e] - b = content.find('[') - e = content.rfind(']') - - self.result = json.JSONDecoder(object_pairs_hook=OrderedDict).decode(content[b:e+1]) + if b >= 0 and e >= 0: + content = content[b:e] + + b = content.find('[') + e = content.rfind(']') + if b >= 0 and e >= 0: + self.result = json.JSONDecoder(object_pairs_hook=OrderedDict)\ + .decode(content[b:e+1]) def parse_xml(self): et = ET.fromstring(self.page)