PHP Warning: file_get_contents(): open_basedir restriction in effect.
-You need to adjust the [open_basedir](https://www.php.net/manual/en/ini.core.php#ini.open-basedir) setting
-in your PHP configuration (`php.ini file`). By default this setting may look like this:
+You need to adjust the
+[open_basedir](https://www.php.net/manual/en/ini.core.php#ini.open-basedir)
+setting in your PHP configuration (`php.ini` file). By default this setting may
+look like this:
open_basedir = /srv/http/:/home/:/tmp/:/usr/share/pear/
-Either add reported directories to the list or disable this setting temporarily by
-dding ";" at the beginning of the line. Don't forget to enable this setting again
-once you are done with the PHP command line operations.
+Either add reported directories to the list or disable this setting temporarily
+by adding ";" at the beginning of the line. Don't forget to enable this setting
+again once you are done with the PHP command line operations.
### PHP timzeone warnings
`could not connect to server: No such file or directory`
-On CentOS v7 the PostgreSQL server is started with `systemd`.
-Check if `/usr/lib/systemd/system/httpd.service` contains a line `PrivateTmp=true`.
-If so then Apache cannot see the `/tmp/.s.PGSQL.5432` file. It's a good security feature,
-so use the [preferred solution](../appendix/Install-on-Centos-7/#adding-selinux-security-settings).
+On CentOS v7 the PostgreSQL server is started with `systemd`. Check if
+`/usr/lib/systemd/system/httpd.service` contains a line `PrivateTmp=true`. If
+so then Apache cannot see the `/tmp/.s.PGSQL.5432` file. It's a good security
+feature, so use the
+[preferred solution](../appendix/Install-on-Centos-7/#adding-selinux-security-settings).
However, you can solve this the quick and dirty way by commenting out that line and then run
sudo systemctl restart httpd
-### "must be an array or an object that implements Countable" warning in /usr/share/pear/DB.php
-
-The warning started with PHP 7.2. Make sure you have at least [version 1.9.3 of PEAR DB](https://github.com/pear/DB/releases)
-installed.
-
### Website reports "DB Error: insufficient permissions"
-The user the webserver, e.g. Apache, runs under needs to have access to the Nominatim database. You can find the user like [this](https://serverfault.com/questions/125865/finding-out-what-user-apache-is-running-as), for default Ubuntu operating system for example it's `www-data`.
+The user the webserver, e.g. Apache, runs under needs to have access to the
+Nominatim database. You can find the user like
+[this](https://serverfault.com/questions/125865/finding-out-what-user-apache-is-running-as),
+for default Ubuntu operating system for example it's `www-data`.
1. Repeat the `createuser` step of the installation instructions.
### Setup.php fails with "DB Error: extension not found"
Make sure you have the PostgreSQL extensions "hstore" and "postgis" installed.
-See the installation instruction for a full list of required packages.
-
-
-### Setup.php reports "Cannot redeclare getDB()"
-
-`Cannot redeclare getDB() (previously declared in /your/path/Nominatim/lib/db.php:4)`
-
-The message is a bit misleading as PHP needs to load the file `DB.php` and
-instead re-loads Nominatim's `db.php`. To solve this make sure you
-have the [Pear module 'DB'](https://pear.php.net/package/DB/) installed.
+See the installation instructions for a full list of required packages.
- sudo pear install DB
### I forgot to delete the flatnodes file before starting an import.
# Documentation Pages
-The [Nominatim documentation](https://nominatim.org/release-docs/develop/) is built using the [MkDocs](https://www.mkdocs.org/) static site generation framework. The master branch is automatically deployed every night on under [https://nominatim.org/release-docs/develop/]()
+The [Nominatim documentation](https://nominatim.org/release-docs/develop/) is built using the [MkDocs](https://www.mkdocs.org/) static site generation framework. The master branch is automatically deployed every night on under [https://nominatim.org/release-docs/develop/](https://nominatim.org/release-docs/develop/)
-To preview local changes:
+To preview local changes, first install MkDocs
-1. Install MkDocs
+```
+pip3 install --user mkdocs
+```
- ```
- pip3 install --user mkdocs
- ```
+Then go to the build directory and run
-2. In build directory run
+```
+make doc
+INFO - Cleaning site directory
+INFO - Building documentation to directory: /home/vagrant/build/site-html
+```
- ```
- make doc
- INFO - Cleaning site directory
- INFO - Building documentation to directory: /home/vagrant/build/site-html
- ```
+This runs `mkdocs build` plus extra transformation of some files and adds
+symlinks (see `CMakeLists.txt` for the exact steps).
- This runs `mkdocs build` plus extra transformion of some files and adds symlinks (see `CMakeLists.txt` for the exact steps).
+Now you can start webserver for local testing
+```
+build> mkdocs serve
+[server:296] Serving on http://127.0.0.1:8000
+[handlers:62] Start watching changes
+```
-3. Start webserver for local testing
+If you develop inside a Vagrant virtual machine:
- ```
- mkdocs serve
- [server:296] Serving on http://127.0.0.1:8000
- [handlers:62] Start watching changes
- ```
-
- If you develop inside a Vagrant virtual machine:
- * add port forwarding to your Vagrantfile, e.g. `config.vm.network "forwarded_port", guest: 8000, host: 8000`
- * use `mkdocs serve --dev-addr 0.0.0.0:8000` because the default localhost
- IP does not get forwarded.
+ * add port forwarding to your Vagrantfile,
+ e.g. `config.vm.network "forwarded_port", guest: 8000, host: 8000`
+ * use `mkdocs serve --dev-addr 0.0.0.0:8000` because the default localhost
+ IP does not get forwarded.
--- /dev/null
+# Postcodes in Nominatim
+
+The blog post
+[Nominatim and Postcodes](https://www.openstreetmap.org/user/lonvia/diary/43143)
+describes the handling implemented since Nominatim 3.1.
+
+Postcode centroids (aka 'calculated postcodes') are generated by looking at all
+postcodes of a country, grouping them and calculating the geometric centroid.
+There is currently no logic to deal with extreme outliers (typos or other
+mistakes in OSM data). There is also no check if a postcodes adheres to a
+country's format, e.g. if Swiss postcodes are 4 digits.
+
+
+## Regular updating calculated postcodes
+
+The script to rerun the calculation is
+`build/utils/update.php --calculate-postcodes`
+and runs once per night on nominatim.openstreetmap.org.
+
+
+## Finding places that share a specific postcode
+
+In the Nominatim database run
+
+```sql
+SELECT address->'postcode' as pc,
+ osm_type, osm_id, class, type,
+ st_x(centroid) as lon, st_y(centroid) as lat
+FROM placex
+WHERE country_code='fr'
+ AND upper(trim (both ' ' from address->'postcode')) = '33210';
+```
+
+Alternatively on [Overpass](https://overpass-turbo.eu/) run the following query
+
+```
+[out:json][timeout:250];
+area["name"="France"]->.boundaryarea;
+(
+nwr(area.boundaryarea)["addr:postcode"="33210"];
+);
+out body;
+>;
+out skel qt;
+```
- 'Overview' : 'develop/overview.md'
- 'OSM Data Import' : 'develop/Import.md'
- 'Place Ranking' : 'develop/Ranking.md'
+ - 'Postcodes' : 'develop/Postcodes.md'
- 'Documentation' : 'develop/Documentation.md'
- 'External Data Sources':
- 'Overview' : 'data-sources/overview.md'
{
// https://secure.php.net/manual/en/ref.pdo-pgsql.connection.php
$aInfo = array();
- if (preg_match('/^pgsql:(.+)/', $sDSN, $aMatches)) {
+ if (preg_match('/^pgsql:(.+)$/', $sDSN, $aMatches)) {
foreach (explode(';', $aMatches[1]) as $sKeyVal) {
list($sKey, $sVal) = explode('=', $sKeyVal, 2);
if ($sKey == 'host') $sKey = 'hostspec';
// Convert database DSN to psql parameters
$aDSNInfo = \Nominatim\DB::parseDSN(CONST_Database_DSN);
if (!isset($aDSNInfo['port']) || !$aDSNInfo['port']) $aDSNInfo['port'] = 5432;
- $sCMD = 'psql -p '.$aDSNInfo['port'].' -d '.$aDSNInfo['database'];
+ $sCMD = 'psql'
+ .' -p '.escapeshellarg($aDSNInfo['port'])
+ .' -d '.escapeshellarg($aDSNInfo['database']);
if (isset($aDSNInfo['hostspec']) && $aDSNInfo['hostspec']) {
- $sCMD .= ' -h ' . $aDSNInfo['hostspec'];
+ $sCMD .= ' -h ' . escapeshellarg($aDSNInfo['hostspec']);
}
if (isset($aDSNInfo['username']) && $aDSNInfo['username']) {
- $sCMD .= ' -U ' . $aDSNInfo['username'];
+ $sCMD .= ' -U ' . escapeshellarg($aDSNInfo['username']);
}
$aProcEnv = null;
if (isset($aDSNInfo['password']) && $aDSNInfo['password']) {
fail('database already exists ('.CONST_Database_DSN.')');
}
- $sCreateDBCmd = 'createdb -E UTF-8 -p '.$this->aDSNInfo['port'].' '.$this->aDSNInfo['database'];
+ $sCreateDBCmd = 'createdb -E UTF-8'
+ .' -p '.escapeshellarg($this->aDSNInfo['port'])
+ .' '.escapeshellarg($this->aDSNInfo['database']);
if (isset($this->aDSNInfo['username'])) {
- $sCreateDBCmd .= ' -U '.$this->aDSNInfo['username'];
+ $sCreateDBCmd .= ' -U '.escapeshellarg($this->aDSNInfo['username']);
}
if (isset($this->aDSNInfo['hostspec'])) {
- $sCreateDBCmd .= ' -h '.$this->aDSNInfo['hostspec'];
+ $sCreateDBCmd .= ' -h '.escapeshellarg($this->aDSNInfo['hostspec']);
}
$result = $this->runWithPgEnv($sCreateDBCmd);
fail("osm2pgsql not found in '$osm2pgsql'");
}
- $osm2pgsql .= ' -S '.CONST_Import_Style;
+ $osm2pgsql .= ' -S '.escapeshellarg(CONST_Import_Style);
if (!is_null(CONST_Osm2pgsql_Flatnode_File) && CONST_Osm2pgsql_Flatnode_File) {
- $osm2pgsql .= ' --flat-nodes '.CONST_Osm2pgsql_Flatnode_File;
+ $osm2pgsql .= ' --flat-nodes '.escapeshellarg(CONST_Osm2pgsql_Flatnode_File);
}
if (CONST_Tablespace_Osm2pgsql_Data)
- $osm2pgsql .= ' --tablespace-slim-data '.CONST_Tablespace_Osm2pgsql_Data;
+ $osm2pgsql .= ' --tablespace-slim-data '.escapeshellarg(CONST_Tablespace_Osm2pgsql_Data);
if (CONST_Tablespace_Osm2pgsql_Index)
- $osm2pgsql .= ' --tablespace-slim-index '.CONST_Tablespace_Osm2pgsql_Index;
+ $osm2pgsql .= ' --tablespace-slim-index '.escapeshellarg(CONST_Tablespace_Osm2pgsql_Index);
if (CONST_Tablespace_Place_Data)
- $osm2pgsql .= ' --tablespace-main-data '.CONST_Tablespace_Place_Data;
+ $osm2pgsql .= ' --tablespace-main-data '.escapeshellarg(CONST_Tablespace_Place_Data);
if (CONST_Tablespace_Place_Index)
- $osm2pgsql .= ' --tablespace-main-index '.CONST_Tablespace_Place_Index;
+ $osm2pgsql .= ' --tablespace-main-index '.escapeshellarg(CONST_Tablespace_Place_Index);
$osm2pgsql .= ' -lsc -O gazetteer --hstore --number-processes 1';
- $osm2pgsql .= ' -C '.$this->iCacheMemory;
- $osm2pgsql .= ' -P '.$this->aDSNInfo['port'];
+ $osm2pgsql .= ' -C '.escapeshellarg($this->iCacheMemory);
+ $osm2pgsql .= ' -P '.escapeshellarg($this->aDSNInfo['port']);
if (isset($this->aDSNInfo['username'])) {
- $osm2pgsql .= ' -U '.$this->aDSNInfo['username'];
+ $osm2pgsql .= ' -U '.escapeshellarg($this->aDSNInfo['username']);
}
if (isset($this->aDSNInfo['hostspec'])) {
- $osm2pgsql .= ' -H '.$this->aDSNInfo['hostspec'];
+ $osm2pgsql .= ' -H '.escapeshellarg($this->aDSNInfo['hostspec']);
}
- $osm2pgsql .= ' -d '.$this->aDSNInfo['database'].' '.$sOSMFile;
+ $osm2pgsql .= ' -d '.escapeshellarg($this->aDSNInfo['database']).' '.escapeshellarg($sOSMFile);
$this->runWithPgEnv($osm2pgsql);
public function index($bIndexNoanalyse)
{
$sOutputFile = '';
- $sBaseCmd = CONST_InstallPath.'/nominatim/nominatim -i -d '.$this->aDSNInfo['database'].' -P '
- .$this->aDSNInfo['port'].' -t '.$this->iInstances.$sOutputFile;
+ $sBaseCmd = CONST_InstallPath.'/nominatim/nominatim -i'
+ .' -d '.escapeshellarg($this->aDSNInfo['database'])
+ .' -P '.escapeshellarg($this->aDSNInfo['port'])
+ .' -t '.escapeshellarg($this->iInstances.$sOutputFile);
if (isset($this->aDSNInfo['hostspec'])) {
- $sBaseCmd .= ' -H '.$this->aDSNInfo['hostspec'];
+ $sBaseCmd .= ' -H '.escapeshellarg($this->aDSNInfo['hostspec']);
}
if (isset($this->aDSNInfo['username'])) {
- $sBaseCmd .= ' -U '.$this->aDSNInfo['username'];
+ $sBaseCmd .= ' -U '.escapeshellarg($this->aDSNInfo['username']);
}
info('Index ranks 0 - 4');
private function pgsqlRunDropAndRestore($sDumpFile)
{
- $sCMD = 'pg_restore -p '.$this->aDSNInfo['port'].' -d '.$this->aDSNInfo['database'].' --no-owner -Fc --clean '.$sDumpFile;
+ $sCMD = 'pg_restore'
+ .' -p '.escapeshellarg($this->aDSNInfo['port'])
+ .' -d '.escapeshellarg($this->aDSNInfo['database'])
+ .' --no-owner -Fc --clean '.escapeshellarg($sDumpFile);
if ($this->oDB->getPostgresVersion() >= 9.04) {
$sCMD .= ' --if-exists';
}
if (isset($this->aDSNInfo['hostspec'])) {
- $sCMD .= ' -h '.$this->aDSNInfo['hostspec'];
+ $sCMD .= ' -h '.escapeshellarg($this->aDSNInfo['hostspec']);
}
if (isset($this->aDSNInfo['username'])) {
- $sCMD .= ' -U '.$this->aDSNInfo['username'];
+ $sCMD .= ' -U '.escapeshellarg($this->aDSNInfo['username']);
}
$this->runWithPgEnv($sCMD);
{
if (!file_exists($sFilename)) fail('unable to find '.$sFilename);
- $sCMD = 'psql -p '.$this->aDSNInfo['port'].' -d '.$this->aDSNInfo['database'];
+ $sCMD = 'psql'
+ .' -p '.escapeshellarg($this->aDSNInfo['port'])
+ .' -d '.escapeshellarg($this->aDSNInfo['database']);
if (!$this->bVerbose) {
$sCMD .= ' -q';
}
if (isset($this->aDSNInfo['hostspec'])) {
- $sCMD .= ' -h '.$this->aDSNInfo['hostspec'];
+ $sCMD .= ' -h '.escapeshellarg($this->aDSNInfo['hostspec']);
}
if (isset($this->aDSNInfo['username'])) {
- $sCMD .= ' -U '.$this->aDSNInfo['username'];
+ $sCMD .= ' -U '.escapeshellarg($this->aDSNInfo['username']);
}
$aProcEnv = null;
if (isset($this->aDSNInfo['password'])) {
1 => array('pipe', 'w'),
2 => array('file', '/dev/null', 'a')
);
- $hGzipProcess = proc_open('zcat '.$sFilename, $aDescriptors, $ahGzipPipes);
+ $hGzipProcess = proc_open('zcat '.escapeshellarg($sFilename), $aDescriptors, $ahGzipPipes);
if (!is_resource($hGzipProcess)) fail('unable to start zcat');
$aReadPipe = $ahGzipPipes[1];
fclose($ahGzipPipes[0]);
} else {
- $sCMD .= ' -f '.$sFilename;
+ $sCMD .= ' -f '.escapeshellarg($sFilename);
$aReadPipe = array('pipe', 'r');
}
$aDescriptors = array(
sudo yum install -y epel-release
+# More repositories for postgresql 11 (CentOS default 'postgresql' is 9.2), postgis
+# and llvm-toolset (https://github.com/theory/pg-semver/issues/35)
+
+ sudo yum install -y https://download.postgresql.org/pub/repos/yum/11/redhat/rhel-7-x86_64/pgdg-redhat-repo-latest.noarch.rpm
+ sudo yum install -y centos-release-scl-rh
+
# Now you can install all packages needed for Nominatim:
#DOCS: :::sh
- sudo yum install -y postgresql-server postgresql-contrib postgresql-devel \
- postgis postgis-utils \
+
+ sudo yum install -y postgresql11-server postgresql11-contrib postgresql11-devel \
+ postgis25_11 postgis25_11-utils \
wget git cmake make gcc gcc-c++ libtool policycoreutils-python \
+ devtoolset-7 llvm-toolset-7 \
php-pgsql php php-intl libpqxx-devel \
proj-epsg bzip2-devel proj-devel libxml2-devel boost-devel \
expat-devel zlib-devel
# CentOS does not automatically create a database cluster. Therefore, start
# with initializing the database, then enable the server to start at boot:
- sudo postgresql-setup initdb
- sudo systemctl enable postgresql
+ echo 'PATH=$PATH:/usr/pgsql-11/bin' > .bash_profile
+ source .bash_profile
+
+ sudo /usr/pgsql-11/bin/postgresql-11-setup initdb
+ sudo systemctl enable postgresql-11
#
# Next tune the postgresql configuration, which is located in
#
# Now start the postgresql service after updating this config file.
- sudo systemctl restart postgresql
+ sudo systemctl restart postgresql-11
#
# Finally, we need to add two postgres users: one for the user that does
# download the country grid:
if [ ! -f data/country_osm_grid.sql.gz ]; then #DOCS: :::sh
- wget -O data/country_osm_grid.sql.gz https://www.nominatim.org/data/country_grid.sql.gz
+ wget --no-verbose -O data/country_osm_grid.sql.gz https://www.nominatim.org/data/country_grid.sql.gz
fi #DOCS:
# The code must be built in a separate directory. Create this directory,
projects / nominatim.git / commitdiff