PHP Warning: file_get_contents(): open_basedir restriction in effect.
-You need to adjust the [open_basedir](https://www.php.net/manual/en/ini.core.php#ini.open-basedir) setting
-in your PHP configuration (`php.ini file`). By default this setting may look like this:
+You need to adjust the
+[open_basedir](https://www.php.net/manual/en/ini.core.php#ini.open-basedir)
+setting in your PHP configuration (`php.ini` file). By default this setting may
+look like this:
open_basedir = /srv/http/:/home/:/tmp/:/usr/share/pear/
-Either add reported directories to the list or disable this setting temporarily by
-dding ";" at the beginning of the line. Don't forget to enable this setting again
-once you are done with the PHP command line operations.
+Either add reported directories to the list or disable this setting temporarily
+by adding ";" at the beginning of the line. Don't forget to enable this setting
+again once you are done with the PHP command line operations.
### PHP timzeone warnings
`could not connect to server: No such file or directory`
-On CentOS v7 the PostgreSQL server is started with `systemd`.
-Check if `/usr/lib/systemd/system/httpd.service` contains a line `PrivateTmp=true`.
-If so then Apache cannot see the `/tmp/.s.PGSQL.5432` file. It's a good security feature,
-so use the [preferred solution](../appendix/Install-on-Centos-7/#adding-selinux-security-settings).
+On CentOS v7 the PostgreSQL server is started with `systemd`. Check if
+`/usr/lib/systemd/system/httpd.service` contains a line `PrivateTmp=true`. If
+so then Apache cannot see the `/tmp/.s.PGSQL.5432` file. It's a good security
+feature, so use the
+[preferred solution](../appendix/Install-on-Centos-7/#adding-selinux-security-settings).
However, you can solve this the quick and dirty way by commenting out that line and then run
sudo systemctl restart httpd
-### "must be an array or an object that implements Countable" warning in /usr/share/pear/DB.php
-
-The warning started with PHP 7.2. Make sure you have at least [version 1.9.3 of PEAR DB](https://github.com/pear/DB/releases)
-installed.
-
### Website reports "DB Error: insufficient permissions"
-The user the webserver, e.g. Apache, runs under needs to have access to the Nominatim database. You can find the user like [this](https://serverfault.com/questions/125865/finding-out-what-user-apache-is-running-as), for default Ubuntu operating system for example it's `www-data`.
+The user the webserver, e.g. Apache, runs under needs to have access to the
+Nominatim database. You can find the user like
+[this](https://serverfault.com/questions/125865/finding-out-what-user-apache-is-running-as),
+for default Ubuntu operating system for example it's `www-data`.
1. Repeat the `createuser` step of the installation instructions.
### Setup.php fails with "DB Error: extension not found"
Make sure you have the PostgreSQL extensions "hstore" and "postgis" installed.
-See the installation instruction for a full list of required packages.
-
-
-### Setup.php reports "Cannot redeclare getDB()"
-
-`Cannot redeclare getDB() (previously declared in /your/path/Nominatim/lib/db.php:4)`
-
-The message is a bit misleading as PHP needs to load the file `DB.php` and
-instead re-loads Nominatim's `db.php`. To solve this make sure you
-have the [Pear module 'DB'](https://pear.php.net/package/DB/) installed.
+See the installation instructions for a full list of required packages.
- sudo pear install DB
### I forgot to delete the flatnodes file before starting an import.
## Downloading additional data
-### Wikipedia rankings
+### Wikipedia/Wikidata rankings
Wikipedia can be used as an optional auxiliary data source to help indicate
the importance of OSM features. Nominatim will work without this information
This data is available as a binary download:
cd $NOMINATIM_SOURCE_DIR/data
- wget https://www.nominatim.org/data/wikipedia_article.sql.bin
- wget https://www.nominatim.org/data/wikipedia_redirect.sql.bin
+ wget https://www.nominatim.org/data/wikimedia-importance.sql.gz
-Combined the 2 files are around 1.5GB and add around 30GB to the install
-size of Nominatim. They also increase the install time by an hour or so.
+The file is about 400MB and adds around 4GB to Nominatim database.
-*NOTE:* you'll need to download the Wikipedia rankings before performing
-the initial import of the data if you want the rankings applied to the
-loaded data.
+*NOTE:* if you forgot to download the wikipedia rankings, you can also add
+them after the import by running `./utils/setup.php --import-wikipedia-articles`
+and then `./utils/update.php --recompute-importance`.
### Great Britain, USA postcodes
SQL statements should be executed from the PostgreSQL commandline. Execute
`psql nominatim` to enter command line mode.
+## 3.4.0 -> master
+
+### New Wikipedia/Wikidata importance tables
+
+The `wikipedia_*` tables have a new format that also includes references to
+Wikidata. You need to update the computation functions and the tables as
+follows:
+
+ * download the new Wikipedia tables as described in the import section
+ * reimport the tables: `./utils/setup.php --import-wikipedia-articles`
+ * update the functions: `./utils/setup.php --create-functions --enable-diff-updates`
+ * compute importance: `./utils/update.php --recompute-importance`
+
+The last step takes about 10 hours on the full planet.
+
## 3.3.0 -> 3.4.0
### Reorganisation of location_area_country table
The possible fields are:
- * `place_id` - reference to the Nominatim internal database ID (see notes below)
+ * `place_id` - reference to the Nominatim internal database ID ([see notes](#place_id-is-not-a-persistent-id))
* `osm_type`, `osm_id` - reference to the OSM object
- * `boundingbox` - area of corner coordinates
+ * `boundingbox` - area of corner coordinates ([see notes](#boundingbox))
* `lat`, `lon` - latitude and longitude of the centroid of the object
* `display_name` - full comma-separated address
* `class`, `type` - key and value of the main OSM tag
The feature list has the following fields:
- * `place_id` - reference to the Nominatim internal database ID (see notes below)
+ * `place_id` - reference to the Nominatim internal database ID ([see notes](#place_id-is-not-a-persistent-id))
* `osm_type`, `osm_id` - reference to the OSM object
* `category`, `type` - key and value of the main OSM tag
* `display_name` - full comma-separated address
The place information can be found in the `result` element. The attributes of that element contain:
- * `place_id` - reference to the Nominatim internal database ID (see notes below)
+ * `place_id` - reference to the Nominatim internal database ID ([see notes](#place_id-is-not-a-persistent-id))
* `osm_type`, `osm_id` - reference to the OSM object
* `ref` - content of `ref` tag if it exists
* `lat`, `lon` - latitude and longitude of the centroid of the object
- * `boundingbox` - comma-separated list of corner coordinates
+ * `boundingbox` - comma-separated list of corner coordinates ([see notes](#boundingbox))
The full address of the result can be found in the content of the
`result` element as a comma-separated list.
The place information can be found in the `place` elements, of which there may
be more than one. The attributes of that element contain:
- * `place_id` - reference to the Nominatim internal database ID (see notes below)
+ * `place_id` - reference to the Nominatim internal database ID ([see notes](#place_id-is-not-a-persistent-id))
* `osm_type`, `osm_id` - reference to the OSM object
* `ref` - content of `ref` tag if it exists
* `lat`, `lon` - latitude and longitude of the centroid of the object
- * `boundingbox` - comma-separated list of corner coordinates
+ * `boundingbox` - comma-separated list of corner coordinates ([see notes](#boundingbox))
* `place_rank` - class search rank
* `display_name` - full comma-separated address
* `class`, `type` - key and value of the main OSM tag
Comma separated list of min latitude, max latitude, min longitude, max longitude.
The whole planet would be `-90,90,-180,180`.
+
+Can we used to pan and center the map on the result, for example with leafletjs
+mapping library
+`map.fitBounds([[bbox[0],bbox[2]],[bbox[1],bbox[3]]], {padding: [20, 20], maxzoom: 16});`
+
+Bounds crossing the antimeridian have a min latitude -180 and max latitude 180,
+essentially covering the planet (See [issue 184](https://github.com/openstreetmap/Nominatim/issues/184)).
# Documentation Pages
-The [Nominatim documentation](https://nominatim.org/release-docs/develop/) is built using the [MkDocs](https://www.mkdocs.org/) static site generation framework. The master branch is automatically deployed every night on under [https://nominatim.org/release-docs/develop/]()
+The [Nominatim documentation](https://nominatim.org/release-docs/develop/) is built using the [MkDocs](https://www.mkdocs.org/) static site generation framework. The master branch is automatically deployed every night on under [https://nominatim.org/release-docs/develop/](https://nominatim.org/release-docs/develop/)
-To preview local changes:
+To preview local changes, first install MkDocs
-1. Install MkDocs
+```
+pip3 install --user mkdocs
+```
- ```
- pip3 install --user mkdocs
- ```
+Then go to the build directory and run
-2. In build directory run
+```
+make doc
+INFO - Cleaning site directory
+INFO - Building documentation to directory: /home/vagrant/build/site-html
+```
- ```
- make doc
- INFO - Cleaning site directory
- INFO - Building documentation to directory: /home/vagrant/build/site-html
- ```
+This runs `mkdocs build` plus extra transformation of some files and adds
+symlinks (see `CMakeLists.txt` for the exact steps).
- This runs `mkdocs build` plus extra transformion of some files and adds symlinks (see `CMakeLists.txt` for the exact steps).
+Now you can start webserver for local testing
+```
+build> mkdocs serve
+[server:296] Serving on http://127.0.0.1:8000
+[handlers:62] Start watching changes
+```
-3. Start webserver for local testing
+If you develop inside a Vagrant virtual machine:
- ```
- mkdocs serve
- [server:296] Serving on http://127.0.0.1:8000
- [handlers:62] Start watching changes
- ```
-
- If you develop inside a Vagrant virtual machine:
- * add port forwarding to your Vagrantfile, e.g. `config.vm.network "forwarded_port", guest: 8000, host: 8000`
- * use `mkdocs serve --dev-addr 0.0.0.0:8000` because the default localhost
- IP does not get forwarded.
+ * add port forwarding to your Vagrantfile,
+ e.g. `config.vm.network "forwarded_port", guest: 8000, host: 8000`
+ * use `mkdocs serve --dev-addr 0.0.0.0:8000` because the default localhost
+ IP does not get forwarded.
--- /dev/null
+# Postcodes in Nominatim
+
+The blog post
+[Nominatim and Postcodes](https://www.openstreetmap.org/user/lonvia/diary/43143)
+describes the handling implemented since Nominatim 3.1.
+
+Postcode centroids (aka 'calculated postcodes') are generated by looking at all
+postcodes of a country, grouping them and calculating the geometric centroid.
+There is currently no logic to deal with extreme outliers (typos or other
+mistakes in OSM data). There is also no check if a postcodes adheres to a
+country's format, e.g. if Swiss postcodes are 4 digits.
+
+
+## Regular updating calculated postcodes
+
+The script to rerun the calculation is
+`build/utils/update.php --calculate-postcodes`
+and runs once per night on nominatim.openstreetmap.org.
+
+
+## Finding places that share a specific postcode
+
+In the Nominatim database run
+
+```sql
+SELECT address->'postcode' as pc,
+ osm_type, osm_id, class, type,
+ st_x(centroid) as lon, st_y(centroid) as lat
+FROM placex
+WHERE country_code='fr'
+ AND upper(trim (both ' ' from address->'postcode')) = '33210';
+```
+
+Alternatively on [Overpass](https://overpass-turbo.eu/) run the following query
+
+```
+[out:json][timeout:250];
+area["name"="France"]->.boundaryarea;
+(
+nwr(area.boundaryarea)["addr:postcode"="33210"];
+);
+out body;
+>;
+out skel qt;
+```
- 'Overview' : 'develop/overview.md'
- 'OSM Data Import' : 'develop/Import.md'
- 'Place Ranking' : 'develop/Ranking.md'
+ - 'Postcodes' : 'develop/Postcodes.md'
- 'Documentation' : 'develop/Documentation.md'
- 'External Data Sources':
- 'Overview' : 'data-sources/overview.md'
{
// https://secure.php.net/manual/en/ref.pdo-pgsql.connection.php
$aInfo = array();
- if (preg_match('/^pgsql:(.+)/', $sDSN, $aMatches)) {
+ if (preg_match('/^pgsql:(.+)$/', $sDSN, $aMatches)) {
foreach (explode(';', $aMatches[1]) as $sKeyVal) {
list($sKey, $sVal) = explode('=', $sKeyVal, 2);
if ($sKey == 'host') $sKey = 'hostspec';
// Convert database DSN to psql parameters
$aDSNInfo = \Nominatim\DB::parseDSN(CONST_Database_DSN);
if (!isset($aDSNInfo['port']) || !$aDSNInfo['port']) $aDSNInfo['port'] = 5432;
- $sCMD = 'psql -p '.$aDSNInfo['port'].' -d '.$aDSNInfo['database'];
+ $sCMD = 'psql'
+ .' -p '.escapeshellarg($aDSNInfo['port'])
+ .' -d '.escapeshellarg($aDSNInfo['database']);
if (isset($aDSNInfo['hostspec']) && $aDSNInfo['hostspec']) {
- $sCMD .= ' -h ' . $aDSNInfo['hostspec'];
+ $sCMD .= ' -h ' . escapeshellarg($aDSNInfo['hostspec']);
}
if (isset($aDSNInfo['username']) && $aDSNInfo['username']) {
- $sCMD .= ' -U ' . $aDSNInfo['username'];
+ $sCMD .= ' -U ' . escapeshellarg($aDSNInfo['username']);
}
$aProcEnv = null;
if (isset($aDSNInfo['password']) && $aDSNInfo['password']) {
fail('database already exists ('.CONST_Database_DSN.')');
}
- $sCreateDBCmd = 'createdb -E UTF-8 -p '.$this->aDSNInfo['port'].' '.$this->aDSNInfo['database'];
+ $sCreateDBCmd = 'createdb -E UTF-8'
+ .' -p '.escapeshellarg($this->aDSNInfo['port'])
+ .' '.escapeshellarg($this->aDSNInfo['database']);
if (isset($this->aDSNInfo['username'])) {
- $sCreateDBCmd .= ' -U '.$this->aDSNInfo['username'];
+ $sCreateDBCmd .= ' -U '.escapeshellarg($this->aDSNInfo['username']);
}
if (isset($this->aDSNInfo['hostspec'])) {
- $sCreateDBCmd .= ' -h '.$this->aDSNInfo['hostspec'];
+ $sCreateDBCmd .= ' -h '.escapeshellarg($this->aDSNInfo['hostspec']);
}
$result = $this->runWithPgEnv($sCreateDBCmd);
if ($this->bNoPartitions) {
$this->pgsqlRunScript('update country_name set partition = 0');
}
-
- // the following will be needed by createFunctions later but
- // is only defined in the subsequently called createTables
- // Create dummies here that will be overwritten by the proper
- // versions in create-tables.
- $this->pgsqlRunScript('CREATE TABLE IF NOT EXISTS place_boundingbox ()');
- $this->pgsqlRunScript('CREATE TYPE wikipedia_article_match AS ()', false);
}
public function importData($sOSMFile)
fail("osm2pgsql not found in '$osm2pgsql'");
}
- $osm2pgsql .= ' -S '.CONST_Import_Style;
+ $osm2pgsql .= ' -S '.escapeshellarg(CONST_Import_Style);
if (!is_null(CONST_Osm2pgsql_Flatnode_File) && CONST_Osm2pgsql_Flatnode_File) {
- $osm2pgsql .= ' --flat-nodes '.CONST_Osm2pgsql_Flatnode_File;
+ $osm2pgsql .= ' --flat-nodes '.escapeshellarg(CONST_Osm2pgsql_Flatnode_File);
}
if (CONST_Tablespace_Osm2pgsql_Data)
- $osm2pgsql .= ' --tablespace-slim-data '.CONST_Tablespace_Osm2pgsql_Data;
+ $osm2pgsql .= ' --tablespace-slim-data '.escapeshellarg(CONST_Tablespace_Osm2pgsql_Data);
if (CONST_Tablespace_Osm2pgsql_Index)
- $osm2pgsql .= ' --tablespace-slim-index '.CONST_Tablespace_Osm2pgsql_Index;
+ $osm2pgsql .= ' --tablespace-slim-index '.escapeshellarg(CONST_Tablespace_Osm2pgsql_Index);
if (CONST_Tablespace_Place_Data)
- $osm2pgsql .= ' --tablespace-main-data '.CONST_Tablespace_Place_Data;
+ $osm2pgsql .= ' --tablespace-main-data '.escapeshellarg(CONST_Tablespace_Place_Data);
if (CONST_Tablespace_Place_Index)
- $osm2pgsql .= ' --tablespace-main-index '.CONST_Tablespace_Place_Index;
+ $osm2pgsql .= ' --tablespace-main-index '.escapeshellarg(CONST_Tablespace_Place_Index);
$osm2pgsql .= ' -lsc -O gazetteer --hstore --number-processes 1';
- $osm2pgsql .= ' -C '.$this->iCacheMemory;
- $osm2pgsql .= ' -P '.$this->aDSNInfo['port'];
+ $osm2pgsql .= ' -C '.escapeshellarg($this->iCacheMemory);
+ $osm2pgsql .= ' -P '.escapeshellarg($this->aDSNInfo['port']);
if (isset($this->aDSNInfo['username'])) {
- $osm2pgsql .= ' -U '.$this->aDSNInfo['username'];
+ $osm2pgsql .= ' -U '.escapeshellarg($this->aDSNInfo['username']);
}
if (isset($this->aDSNInfo['hostspec'])) {
- $osm2pgsql .= ' -H '.$this->aDSNInfo['hostspec'];
+ $osm2pgsql .= ' -H '.escapeshellarg($this->aDSNInfo['hostspec']);
}
- $osm2pgsql .= ' -d '.$this->aDSNInfo['database'].' '.$sOSMFile;
+ $osm2pgsql .= ' -d '.escapeshellarg($this->aDSNInfo['database']).' '.escapeshellarg($sOSMFile);
$this->runWithPgEnv($osm2pgsql);
public function importWikipediaArticles()
{
- $sWikiArticlesFile = CONST_Wikipedia_Data_Path.'/wikipedia_article.sql.bin';
- $sWikiRedirectsFile = CONST_Wikipedia_Data_Path.'/wikipedia_redirect.sql.bin';
+ $sWikiArticlesFile = CONST_Wikipedia_Data_Path.'/wikimedia-importance.sql.gz';
if (file_exists($sWikiArticlesFile)) {
- info('Importing wikipedia articles');
- $this->pgsqlRunDropAndRestore($sWikiArticlesFile);
- } else {
- warn('wikipedia article dump file not found - places will have default importance');
- }
- if (file_exists($sWikiRedirectsFile)) {
- info('Importing wikipedia redirects');
- $this->pgsqlRunDropAndRestore($sWikiRedirectsFile);
+ info('Importing wikipedia articles and redirects');
+ $this->pgExec('DROP TABLE IF EXISTS wikipedia_article');
+ $this->pgExec('DROP TABLE IF EXISTS wikipedia_redirect');
+ $this->pgsqlRunScriptFile($sWikiArticlesFile);
} else {
- warn('wikipedia redirect dump file not found - some place importance values may be missing');
+ warn('wikipedia importance dump file not found - places will have default importance');
}
}
echo '.';
$this->pgExec('TRUNCATE place_addressline');
echo '.';
- $this->pgExec('TRUNCATE place_boundingbox');
- echo '.';
$this->pgExec('TRUNCATE location_area');
echo '.';
if (!$this->dbReverseOnly()) {
public function index($bIndexNoanalyse)
{
$sOutputFile = '';
- $sBaseCmd = CONST_InstallPath.'/nominatim/nominatim -i -d '.$this->aDSNInfo['database'].' -P '
- .$this->aDSNInfo['port'].' -t '.$this->iInstances.$sOutputFile;
+ $sBaseCmd = CONST_InstallPath.'/nominatim/nominatim -i'
+ .' -d '.escapeshellarg($this->aDSNInfo['database'])
+ .' -P '.escapeshellarg($this->aDSNInfo['port'])
+ .' -t '.escapeshellarg($this->iInstances.$sOutputFile);
if (isset($this->aDSNInfo['hostspec'])) {
- $sBaseCmd .= ' -H '.$this->aDSNInfo['hostspec'];
+ $sBaseCmd .= ' -H '.escapeshellarg($this->aDSNInfo['hostspec']);
}
if (isset($this->aDSNInfo['username'])) {
- $sBaseCmd .= ' -U '.$this->aDSNInfo['username'];
+ $sBaseCmd .= ' -U '.escapeshellarg($this->aDSNInfo['username']);
}
info('Index ranks 0 - 4');
}
}
- private function pgsqlRunDropAndRestore($sDumpFile)
- {
- $sCMD = 'pg_restore -p '.$this->aDSNInfo['port'].' -d '.$this->aDSNInfo['database'].' --no-owner -Fc --clean '.$sDumpFile;
- if ($this->oDB->getPostgresVersion() >= 9.04) {
- $sCMD .= ' --if-exists';
- }
- if (isset($this->aDSNInfo['hostspec'])) {
- $sCMD .= ' -h '.$this->aDSNInfo['hostspec'];
- }
- if (isset($this->aDSNInfo['username'])) {
- $sCMD .= ' -U '.$this->aDSNInfo['username'];
- }
-
- $this->runWithPgEnv($sCMD);
- }
-
private function pgsqlRunScript($sScript, $bfatal = true)
{
runSQLScript(
{
if (!file_exists($sFilename)) fail('unable to find '.$sFilename);
- $sCMD = 'psql -p '.$this->aDSNInfo['port'].' -d '.$this->aDSNInfo['database'];
+ $sCMD = 'psql'
+ .' -p '.escapeshellarg($this->aDSNInfo['port'])
+ .' -d '.escapeshellarg($this->aDSNInfo['database']);
if (!$this->bVerbose) {
$sCMD .= ' -q';
}
if (isset($this->aDSNInfo['hostspec'])) {
- $sCMD .= ' -h '.$this->aDSNInfo['hostspec'];
+ $sCMD .= ' -h '.escapeshellarg($this->aDSNInfo['hostspec']);
}
if (isset($this->aDSNInfo['username'])) {
- $sCMD .= ' -U '.$this->aDSNInfo['username'];
+ $sCMD .= ' -U '.escapeshellarg($this->aDSNInfo['username']);
}
$aProcEnv = null;
if (isset($this->aDSNInfo['password'])) {
1 => array('pipe', 'w'),
2 => array('file', '/dev/null', 'a')
);
- $hGzipProcess = proc_open('zcat '.$sFilename, $aDescriptors, $ahGzipPipes);
+ $hGzipProcess = proc_open('zcat '.escapeshellarg($sFilename), $aDescriptors, $ahGzipPipes);
if (!is_resource($hGzipProcess)) fail('unable to start zcat');
$aReadPipe = $ahGzipPipes[1];
fclose($ahGzipPipes[0]);
} else {
- $sCMD .= ' -f '.$sFilename;
+ $sCMD .= ' -f '.escapeshellarg($sFilename);
$aReadPipe = array('pipe', 'r');
}
$aDescriptors = array(
<meta name="viewport" content="width=device-width, initial-scale=1">
<base href="<?php echo CONST_Website_BaseURL;?>" />
- <link href="nominatim.xml" rel="search" title="Nominatim Search" type="application/opensearchdescription+xml" />
<link href="css/leaflet.css" rel="stylesheet" />
<link href="css/Control.Minimap.min.css" rel="stylesheet" />
<link href="css/bootstrap-theme.min.css" rel="stylesheet" />
END LOOP;
NEW.importance := null;
- select language||':'||title,importance from get_wikipedia_match(NEW.extratags, NEW.country_code) INTO NEW.wikipedia,NEW.importance;
- IF NEW.importance IS NULL THEN
- select language||':'||title,importance from wikipedia_article where osm_type = NEW.osm_type and osm_id = NEW.osm_id order by importance desc limit 1 INTO NEW.wikipedia,NEW.importance;
- END IF;
+ SELECT wikipedia, importance
+ FROM compute_importance(NEW.extratags, NEW.country_code, NEW.osm_type, NEW.osm_id)
+ INTO NEW.wikipedia,NEW.importance;
--DEBUG: RAISE WARNING 'Importance computed from wikipedia: %', NEW.importance;
-- mark the linked place (excludes from search results)
UPDATE placex set linked_place_id = NEW.place_id where place_id = linkedPlacex.place_id;
- -- keep a note of the node id in case we need it for wikipedia in a bit
- linked_node_id := linkedPlacex.osm_id;
- select language||':'||title,importance from get_wikipedia_match(linkedPlacex.extratags, NEW.country_code) INTO linked_wikipedia,linked_importance;
+ select wikipedia, importance
+ FROM compute_importance(linkedPlacex.extratags, NEW.country_code,
+ 'N', linkedPlacex.osm_id)
+ INTO linked_wikipedia,linked_importance;
--DEBUG: RAISE WARNING 'Linked label member';
END LOOP;
-- mark the linked place (excludes from search results)
UPDATE placex set linked_place_id = NEW.place_id where place_id = linkedPlacex.place_id;
- -- keep a note of the node id in case we need it for wikipedia in a bit
- linked_node_id := linkedPlacex.osm_id;
- select language||':'||title,importance from get_wikipedia_match(linkedPlacex.extratags, NEW.country_code) INTO linked_wikipedia,linked_importance;
+ select wikipedia, importance
+ FROM compute_importance(linkedPlacex.extratags, NEW.country_code,
+ 'N', linkedPlacex.osm_id)
+ INTO linked_wikipedia,linked_importance;
--DEBUG: RAISE WARNING 'Linked admin_center';
END IF;
-- mark the linked place (excludes from search results)
UPDATE placex set linked_place_id = NEW.place_id where place_id = linkedPlacex.place_id;
- -- keep a note of the node id in case we need it for wikipedia in a bit
- linked_node_id := linkedPlacex.osm_id;
- select language||':'||title,importance from get_wikipedia_match(linkedPlacex.extratags, NEW.country_code) INTO linked_wikipedia,linked_importance;
+ select wikipedia, importance
+ FROM compute_importance(linkedPlacex.extratags, NEW.country_code,
+ 'N', linkedPlacex.osm_id)
+ INTO linked_wikipedia,linked_importance;
--DEBUG: RAISE WARNING 'Linked named place';
END LOOP;
END IF;
(NEW.importance is null or NEW.importance < linked_importance) THEN
NEW.importance = linked_importance;
END IF;
-
- -- Still null? how about looking it up by the node id
- IF NEW.importance IS NULL THEN
- --DEBUG: RAISE WARNING 'Looking up importance by linked node id';
- select language||':'||title,importance from wikipedia_article where osm_type = 'N'::char(1) and osm_id = linked_node_id order by importance desc limit 1 INTO NEW.wikipedia,NEW.importance;
- END IF;
-
END IF;
-- make sure all names are in the word table
$$
LANGUAGE plpgsql IMMUTABLE;
-DROP TYPE wikipedia_article_match CASCADE;
+DROP TYPE IF EXISTS wikipedia_article_match CASCADE;
create type wikipedia_article_match as (
language TEXT,
title TEXT,
$$
LANGUAGE plpgsql;
+DROP TYPE IF EXISTS place_importance CASCADE;
+create type place_importance as (
+ importance FLOAT,
+ wikipedia TEXT
+);
+
+CREATE OR REPLACE FUNCTION compute_importance(extratags HSTORE, country_code varchar(2), osm_type varchar(1), osm_id BIGINT)
+ RETURNS place_importance
+ AS $$
+DECLARE
+ match RECORD;
+ result place_importance;
+BEGIN
+ FOR match IN SELECT * FROM get_wikipedia_match(extratags, country_code)
+ WHERE language is not NULL
+ LOOP
+ result.importance := match.importance;
+ result.wikipedia := match.language || ':' || match.title;
+ RETURN result;
+ END LOOP;
+
+ IF extratags ? 'wikidata' THEN
+ FOR match IN SELECT * FROM wikipedia_article
+ WHERE wd_page_title = extratags->'wikidata'
+ ORDER BY language = 'en' DESC, langcount DESC LIMIT 1 LOOP
+ result.importance := match.importance;
+ result.wikipedia := match.language || ':' || match.title;
+ RETURN result;
+ END LOOP;
+ END IF;
+
+ RETURN null;
+END;
+$$
+LANGUAGE plpgsql;
+
CREATE OR REPLACE FUNCTION quad_split_geometry(geometry GEOMETRY, maxarea FLOAT, maxdepth INTEGER)
RETURNS SETOF GEOMETRY
AS $$
lon double precision,
importance double precision,
osm_type character(1),
- osm_id bigint
+ osm_id bigint,
+ wd_page_title text,
+ instance_of text
);
ALTER TABLE ONLY wikipedia_article ADD CONSTRAINT wikipedia_article_pkey PRIMARY KEY (language, title);
CREATE INDEX idx_wikipedia_article_osm_id ON wikipedia_article USING btree (osm_type, osm_id);
| en |
Then result addresses contain
| ID | state |
- | 0 | Salto |
+ | 0 | Florida |
When sending json search query "25 de Mayo" with address
| accept-language | viewbox |
- | en | -56.35879,-34.18330,-56.31618,-34.20815 |
+ | en | -57.95468,-31.39261,-57.94741,-31.39490 |
Then result addresses contain
| ID | state |
- | 0 | Florida |
+ | 0 | Salto |
Scenario: viewboxes cannot be points
When sending json search query "foo"
array('deduplicate', '', 0, 1, 0, 0, 'bool', 'Deduplicate tokens'),
array('recompute-word-counts', '', 0, 1, 0, 0, 'bool', 'Compute frequency of full-word search terms'),
array('update-address-levels', '', 0, 1, 0, 0, 'bool', 'Reimport address level configuration (EXPERT)'),
+ array('recompute-importance', '', 0, 1, 0, 0, 'bool', 'Recompute place importances'),
array('no-npi', '', 0, 1, 0, 0, 'bool', '(obsolete)'),
);
$oAlParser->createTable($oDB, 'address_levels');
}
+if ($aResult['recompute-importance']) {
+ echo "Updating importance values for database.\n";
+ $oDB = new Nominatim\DB();
+ $oDB->connect();
+
+ $sSQL = 'ALTER TABLE placex DISABLE TRIGGER ALL;';
+ $sSQL .= 'UPDATE placex SET (wikipedia, importance) =';
+ $sSQL .= ' (SELECT wikipedia, importance';
+ $sSQL .= ' FROM compute_importance(extratags, country_code, osm_type, osm_id));';
+ $sSQL .= 'UPDATE placex s SET wikipedia = d.wikipedia, importance = d.importance';
+ $sSQL .= ' FROM placex d';
+ $sSQL .= ' WHERE s.place_id = d.linked_place_id and d.wikipedia is not null';
+ $sSQL .= ' and (s.wikipedia is null or s.importance < d.importance);';
+ $sSQL .= 'ALTER TABLE placex ENABLE TRIGGER ALL;';
+ $oDB->exec($sSQL);
+}
+
if ($aResult['import-osmosis'] || $aResult['import-osmosis-all']) {
//
if (strpos(CONST_Replication_Url, 'download.geofabrik.de') !== false && CONST_Replication_Update_Interval < 86400) {
sudo yum install -y epel-release
+# More repositories for postgresql 11 (CentOS default 'postgresql' is 9.2), postgis
+# and llvm-toolset (https://github.com/theory/pg-semver/issues/35)
+
+ sudo yum install -y https://download.postgresql.org/pub/repos/yum/11/redhat/rhel-7-x86_64/pgdg-redhat-repo-latest.noarch.rpm
+ sudo yum install -y centos-release-scl-rh
+
# Now you can install all packages needed for Nominatim:
#DOCS: :::sh
- sudo yum install -y postgresql-server postgresql-contrib postgresql-devel \
- postgis postgis-utils \
+
+ sudo yum install -y postgresql11-server postgresql11-contrib postgresql11-devel \
+ postgis25_11 postgis25_11-utils \
wget git cmake make gcc gcc-c++ libtool policycoreutils-python \
+ devtoolset-7 llvm-toolset-7 \
php-pgsql php php-intl libpqxx-devel \
proj-epsg bzip2-devel proj-devel libxml2-devel boost-devel \
expat-devel zlib-devel
# CentOS does not automatically create a database cluster. Therefore, start
# with initializing the database, then enable the server to start at boot:
- sudo postgresql-setup initdb
- sudo systemctl enable postgresql
+ echo 'PATH=$PATH:/usr/pgsql-11/bin' > .bash_profile
+ source .bash_profile
+
+ sudo /usr/pgsql-11/bin/postgresql-11-setup initdb
+ sudo systemctl enable postgresql-11
#
# Next tune the postgresql configuration, which is located in
#
# Now start the postgresql service after updating this config file.
- sudo systemctl restart postgresql
+ sudo systemctl restart postgresql-11
#
# Finally, we need to add two postgres users: one for the user that does
# download the country grid:
if [ ! -f data/country_osm_grid.sql.gz ]; then #DOCS: :::sh
- wget -O data/country_osm_grid.sql.gz https://www.nominatim.org/data/country_grid.sql.gz
+ wget --no-verbose -O data/country_osm_grid.sql.gz https://www.nominatim.org/data/country_grid.sql.gz
fi #DOCS:
# The code must be built in a separate directory. Create this directory,
projects / nominatim.git / commitdiff