]> git.openstreetmap.org Git - nominatim.git/log
nominatim.git
12 years agoDB Scheme changes: alter table placex add column calculated_country_code varchar(2);
Brian Quinion [Tue, 22 May 2012 14:27:42 +0000 (16:27 +0200)]
DB Scheme changes: alter table placex add column calculated_country_code varchar(2);

function changes:
-----------------
Move to ST_PointOnSurface from ST_Centroid in various places to avoid looking up a point outside the polygon
Move to ST_Covers from ST_Contains to include points on admin boundaries
Re-order preference for get_country_code now our data is better. country_osm_grid is now the preffered source.
Fix code to calculate country code in placex_insert, rank_search test was too early
Add extra field to placex 'calculated_country_code' to improve structure of code
Move split_geometery function out of add_location into its own function
Rewrite split_geometery to be more efficient.
Change place_insert to do more updates and less delete/inserts (delete is slow)
Include wikipedia links in details.php ouput
Cleanup no longer used geometry validation (adding overhead)
Include debug statements in function.sql (--DEBUG: ) and add flag to setup.php to turn them on

setup.php:
----------
add flag --disable-token-precalc to speed up debuging
add flag --index-noanalyse to disable analysising DB at rank 4 and 26 (previously removed, but on my local DB it seems to be required)
add flag --enable-diff-updates (modifier to --create-functions) to turn on the code required for diff updates without having to modify functions.sql
add flag --enable-debug-statements (modifier to --create-functions) to turn on debug warning statements

update.php:
-----------
added flag --no-index to import osmosis changes without indexing them
extend the hack to allow import of JOSM generated osm files

country_grid.sql - reference copy of the sql used to generate the country_osm_grid table, needs cleanup

12 years agoSuport postgis 2.0 for search
Andree Klattenhoff [Tue, 22 May 2012 12:04:37 +0000 (14:04 +0200)]
Suport postgis 2.0 for search

12 years agoMerge branch 'master' of github.com:twain47/Nominatim
Brian Quinion [Tue, 15 May 2012 01:06:12 +0000 (03:06 +0200)]
Merge branch 'master' of github.com:twain47/Nominatim

12 years agomove index over nameaddress_vector back to delayed index creation
Sarah Hoffmann [Sun, 13 May 2012 09:38:50 +0000 (11:38 +0200)]
move index over nameaddress_vector back to delayed index creation

No longer needed for linking places.

12 years agorevert timezone setting
Sarah Hoffmann [Sun, 13 May 2012 09:04:04 +0000 (11:04 +0200)]
revert timezone setting

Does not work with poldi's php version.

12 years agobe less verbose
Sarah Hoffmann [Sun, 13 May 2012 09:03:05 +0000 (11:03 +0200)]
be less verbose

12 years agoremove intermediate ANALYSE while indexing
Sarah Hoffmann [Sun, 13 May 2012 08:50:08 +0000 (10:50 +0200)]
remove intermediate ANALYSE while indexing

Autovacuum does a much better job.

12 years agomake sure nominatim.so is accessible for postgresql
Sarah Hoffmann [Sun, 13 May 2012 08:05:27 +0000 (10:05 +0200)]
make sure nominatim.so is accessible for postgresql

Even if the user has a very restrictive umask in place.

12 years agopre-create word list to avoid duplicates due to parallel indexing
Sarah Hoffmann [Sun, 13 May 2012 07:15:56 +0000 (09:15 +0200)]
pre-create word list to avoid duplicates due to parallel indexing

12 years agoautomated banning of excessive bulk users
Sarah Hoffmann [Sat, 12 May 2012 15:20:27 +0000 (17:20 +0200)]
automated banning of excessive bulk users

12 years agoCan't filter on rank_search before it has been assigned
Brian Quinion [Fri, 11 May 2012 17:23:44 +0000 (19:23 +0200)]
Can't filter on rank_search before it has been assigned

12 years agoupdated wikipedia article code to allow for mutliple languages and remove code duplic...
Brian Quinion [Sun, 6 May 2012 23:18:31 +0000 (00:18 +0100)]
updated wikipedia article code to allow for mutliple languages and remove code duplication

12 years agomodifiers to encourage use of indices
Sarah Hoffmann [Sat, 5 May 2012 09:38:26 +0000 (11:38 +0200)]
modifiers to encourage use of indices

12 years agodon't have an address property in the return if we have not got address data
Brian Quinion [Thu, 3 May 2012 23:59:26 +0000 (00:59 +0100)]
don't have an address property in the return if we have not got address data

12 years agocatch utf8 encoding errors and at least stop them aborting processing
Brian Quinion [Thu, 3 May 2012 23:58:26 +0000 (00:58 +0100)]
catch utf8 encoding errors and at least stop them aborting processing

12 years agoMerge branch 'master' of github.com:twain47/Nominatim
Brian Quinion [Mon, 30 Apr 2012 00:05:13 +0000 (01:05 +0100)]
Merge branch 'master' of github.com:twain47/Nominatim

12 years agocalculate importance from wikipedia
Brian Quinion [Mon, 30 Apr 2012 00:05:06 +0000 (01:05 +0100)]
calculate importance from wikipedia

12 years agomore name checks in point and poly mergeing, and put the index creation earlier in...
Brian Quinion [Sun, 29 Apr 2012 15:40:44 +0000 (16:40 +0100)]
more name checks in point and poly mergeing, and put the index creation earlier in the process

12 years agoadd licence file
Sarah Hoffmann [Fri, 27 Apr 2012 20:19:55 +0000 (22:19 +0200)]
add licence file

12 years agofix typos in mapicons
Sarah Hoffmann [Thu, 26 Apr 2012 21:23:27 +0000 (23:23 +0200)]
fix typos in mapicons

12 years agoallow slashes in house numbers
Sarah Hoffmann [Mon, 16 Apr 2012 13:33:13 +0000 (15:33 +0200)]
allow slashes in house numbers

12 years agotimezone trouble
Sarah Hoffmann [Thu, 26 Apr 2012 22:05:55 +0000 (00:05 +0200)]
timezone trouble

12 years agodisable vacuuming of planet_osm_ways, too expensive
Sarah Hoffmann [Thu, 26 Apr 2012 22:04:11 +0000 (00:04 +0200)]
disable vacuuming of planet_osm_ways, too expensive

12 years agoimmediately exit on log rotation errors
Sarah Hoffmann [Thu, 26 Apr 2012 22:03:15 +0000 (00:03 +0200)]
immediately exit on log rotation errors

12 years agofix typos in mapicons
Sarah Hoffmann [Thu, 26 Apr 2012 21:23:27 +0000 (23:23 +0200)]
fix typos in mapicons

12 years agomake it run with 64bit and 32bit osm2pgsql
Sarah Hoffmann [Mon, 16 Apr 2012 22:17:05 +0000 (00:17 +0200)]
make it run with 64bit and 32bit osm2pgsql

Needs tweaking on the osm2pgsql side, too: the osmid column
needs to have the same format as in osm2pgsql. That means
that the changes are still not compatible with existing installations.

12 years agochange code to work with a 64-bit node size osm2pgsql
Sarah Hoffmann [Mon, 16 Apr 2012 20:26:44 +0000 (22:26 +0200)]
change code to work with a 64-bit node size osm2pgsql

This breaks the 32-bit node size version.

12 years agoMerge pull request #5 from mfn/improve_errmsg
Brian Quinion [Thu, 26 Apr 2012 13:50:58 +0000 (06:50 -0700)]
Merge pull request #5 from mfn/improve_errmsg

Do not swallow PEAR error during setup in import-data.

12 years agodepreciate place=region due to common miss-use
Brian Quinion [Thu, 26 Apr 2012 12:11:56 +0000 (13:11 +0100)]
depreciate place=region due to common miss-use

12 years agolink nodes and relations using name->'name' index lookup instead of search_name
Brian Quinion [Thu, 26 Apr 2012 12:01:26 +0000 (13:01 +0100)]
link nodes and relations using name->'name' index lookup instead of search_name

12 years agoDo not swallow PEAR error.
mfn [Thu, 26 Apr 2012 11:30:49 +0000 (13:30 +0200)]
Do not swallow PEAR error.

12 years agoset osm2pgsql cache memory to a default based on memory in the machine
Brian Quinion [Thu, 26 Apr 2012 00:35:06 +0000 (01:35 +0100)]
set osm2pgsql cache memory to a default based on memory in the machine

12 years agoMerge branch 'continent' of https://github.com/datendelphin/Nominatim
Sarah Hoffmann [Fri, 20 Apr 2012 21:19:52 +0000 (23:19 +0200)]
Merge branch 'continent' of https://github.com/datendelphin/Nominatim

12 years agoadd support for Irish townlands
Sarah Hoffmann [Fri, 20 Apr 2012 20:13:03 +0000 (22:13 +0200)]
add support for Irish townlands

To work properly, osm2pgsql needs to be updated to the latest version.

12 years agoonly calculate country for search ranks country and below
Datendelphin [Sun, 15 Apr 2012 21:05:31 +0000 (23:05 +0200)]
only calculate country for search ranks country and below

12 years agoallow slashes in house numbers
Sarah Hoffmann [Mon, 16 Apr 2012 13:33:13 +0000 (15:33 +0200)]
allow slashes in house numbers

12 years agoCollapse two log tables into one.
Sarah Hoffmann [Sun, 15 Apr 2012 17:15:33 +0000 (19:15 +0200)]
Collapse two log tables into one.

This requires to add a searchterm column to new_query_log.

12 years agoproper silence
Sarah Hoffmann [Sun, 15 Apr 2012 17:13:57 +0000 (19:13 +0200)]
proper silence

12 years agocron script for log rotation
Sarah Hoffmann [Sun, 15 Apr 2012 17:11:35 +0000 (19:11 +0200)]
cron script for log rotation

12 years agoFormat of replicate-sequence URL has changed
Sarah Hoffmann [Sun, 15 Apr 2012 15:45:01 +0000 (17:45 +0200)]
Format of replicate-sequence URL has changed

Additionally update/init import_status in the DB. If this is not
initialized here, subsequent updates of the status won't work.

12 years agocron job for vacuuming intarray indices
Sarah Hoffmann [Sun, 15 Apr 2012 13:57:50 +0000 (15:57 +0200)]
cron job for vacuuming intarray indices

12 years agoignore postcode boundaries without postcode
Sarah Hoffmann [Sat, 14 Apr 2012 07:27:10 +0000 (09:27 +0200)]
ignore postcode boundaries without postcode

12 years agomissing brackets in default name assignment
Sarah Hoffmann [Sat, 14 Apr 2012 06:56:07 +0000 (08:56 +0200)]
missing brackets in default name assignment

12 years agoadd switch for osm2pgsql cache size
Sarah Hoffmann [Wed, 11 Apr 2012 21:39:07 +0000 (23:39 +0200)]
add switch for osm2pgsql cache size

also increased the default to 15GB to accaommodate growing planet

12 years agoMerge branch 'mountain_pass' of https://github.com/datendelphin/Nominatim
Sarah Hoffmann [Wed, 11 Apr 2012 20:21:06 +0000 (22:21 +0200)]
Merge branch 'mountain_pass' of https://github.com/datendelphin/Nominatim

12 years agoforce UTF-8 database
Sarah Hoffmann [Wed, 11 Apr 2012 20:16:35 +0000 (22:16 +0200)]
force UTF-8 database

12 years agoranking for mountain_pass
Datendelphin [Fri, 6 Apr 2012 16:34:45 +0000 (18:34 +0200)]
ranking for mountain_pass

12 years agoOnly link polygons on admin_centre is not already found a label AND if exactl search_...
Brian Quinion [Mon, 2 Apr 2012 11:48:51 +0000 (12:48 +0100)]
Only link polygons on admin_centre is not already found a label AND if exactl search_rank match

12 years agosearch_name.name_vector index is now required during initial index due to code to...
Brian Quinion [Sun, 1 Apr 2012 12:52:58 +0000 (13:52 +0100)]
search_name.name_vector index is now required during initial index due to code to merge relations and nodes

12 years agoUpdate details.php to show linked places
Brian Quinion [Sun, 1 Apr 2012 01:38:40 +0000 (02:38 +0100)]
Update details.php to show linked places

12 years agoMerge branch 'master' of github.com:twain47/Nominatim
Brian Quinion [Sun, 1 Apr 2012 00:48:45 +0000 (01:48 +0100)]
Merge branch 'master' of github.com:twain47/Nominatim

12 years agoAdds code to merge place polygon and points using:
Brian Quinion [Sun, 1 Apr 2012 00:40:50 +0000 (01:40 +0100)]
Adds code to merge place polygon and points using:
  label relation member
  admin_center, admin_centre relation member (with same name)
  exact name, search_rank and location match

Adding this requires a new column and index:
  SELECT AddGeometryColumn('placex', 'centroid', 4326, 'GEOMETRY', 2);
  CREATE INDEX idx_placex_linked_place_id ON placex USING BTREE (linked_place_id);

12 years agopostgresq 8.4 backward compatibility: count hash size with akeys instead of hash_to_array
Sarah Hoffmann [Thu, 29 Mar 2012 22:52:08 +0000 (00:52 +0200)]
postgresq 8.4 backward compatibility: count hash size with akeys instead of hash_to_array

12 years agodon't use column names for variable names, 8.4 fails on them
Sarah Hoffmann [Thu, 29 Mar 2012 21:48:29 +0000 (23:48 +0200)]
don't use column names for variable names, 8.4 fails on them

12 years agoMerge branch 'master' of github.com:twain47/Nominatim
Brian Quinion [Mon, 26 Mar 2012 22:57:24 +0000 (23:57 +0100)]
Merge branch 'master' of github.com:twain47/Nominatim

12 years agoUse GB postcode table as definitive source. resort by pressence of search word in...
Brian Quinion [Thu, 22 Mar 2012 00:33:28 +0000 (00:33 +0000)]
Use GB postcode table as definitive source. resort by pressence of search word in output address and other misc very minor changes not commited from MQ servers

12 years agocountries need language list as well
Sarah Hoffmann [Wed, 21 Mar 2012 19:59:35 +0000 (20:59 +0100)]
countries need language list as well

12 years agoBlack/whitelisting for special phrases
Sarah Hoffmann [Tue, 20 Mar 2012 23:40:35 +0000 (00:40 +0100)]
Black/whitelisting for special phrases

Allows to restrict the special phrases imported from the wiki.
Blacklist allows to exclude certain class/type combinations.
Whitelist allows to define an allowed subset of types for a class.
Adjust to your liking in settings/phrase_settings.php before running
the specialphrases script.

12 years agoMerge branch 'master' of http://github.com/twain47/Nominatim
Sarah Hoffmann [Sun, 18 Mar 2012 17:23:11 +0000 (18:23 +0100)]
Merge branch 'master' of http://github.com/twain47/Nominatim

12 years agouse X/Y max/min functions instead of box making
Sarah Hoffmann [Sun, 18 Mar 2012 17:18:33 +0000 (18:18 +0100)]
use X/Y max/min functions instead of box making

There is a strange bug in postgis where for some coordinates
it does not make a proper box around a point.

12 years agoreturn a 500 on database error
Sarah Hoffmann [Sun, 18 Mar 2012 16:58:19 +0000 (17:58 +0100)]
return a 500 on database error

12 years agoonly re-index addressable places
Sarah Hoffmann [Sun, 18 Mar 2012 15:15:45 +0000 (16:15 +0100)]
only re-index addressable places

exclude places from hierarchical updates when they are only addressed
through their parent and do not have a name (thus no entry in search
name)

12 years agoremove array_agg
Sarah Hoffmann [Sun, 18 Mar 2012 15:12:41 +0000 (16:12 +0100)]
remove array_agg

shipped with postgresql in all supported versions

12 years agoMerge branch 'master' of http://github.com/twain47/Nominatim
Sarah Hoffmann [Sat, 10 Mar 2012 21:54:29 +0000 (22:54 +0100)]
Merge branch 'master' of http://github.com/twain47/Nominatim

12 years agomunin graph for request counts
Sarah Hoffmann [Sat, 10 Mar 2012 21:53:38 +0000 (22:53 +0100)]
munin graph for request counts

12 years agoput indexing in update mode
Sarah Hoffmann [Sat, 10 Mar 2012 13:14:33 +0000 (14:14 +0100)]
put indexing in update mode

12 years agoadapt settings for poldi
Sarah Hoffmann [Sat, 10 Mar 2012 13:13:14 +0000 (14:13 +0100)]
adapt settings for poldi

12 years agouser-agent based blocking
Sarah Hoffmann [Sat, 10 Mar 2012 11:57:19 +0000 (12:57 +0100)]
user-agent based blocking

12 years agosend 403 and HTML page on block
Sarah Hoffmann [Sat, 10 Mar 2012 11:21:08 +0000 (12:21 +0100)]
send 403 and HTML page on block

12 years agofrequent words hack
Sarah Hoffmann [Sat, 10 Mar 2012 10:48:02 +0000 (11:48 +0100)]
frequent words hack

Adds logging of queries on search_name that take too long
because of word tokens that are too frequent.
Words can be excluded from the token search by setting the
search_name_count column to something greater than 0.

12 years agocustomized tablespace settings for poldi
Sarah Hoffmann [Sat, 10 Mar 2012 10:31:17 +0000 (11:31 +0100)]
customized tablespace settings for poldi

12 years agosanity check for address interpolation
Sarah Hoffmann [Fri, 9 Mar 2012 23:08:06 +0000 (00:08 +0100)]
sanity check for address interpolation

12 years agoavoid deletes for new places
Sarah Hoffmann [Fri, 9 Mar 2012 23:06:28 +0000 (00:06 +0100)]
avoid deletes for new places

12 years agorestrict deferred delete to concerned place
Sarah Hoffmann [Fri, 9 Mar 2012 22:47:25 +0000 (23:47 +0100)]
restrict deferred delete to concerned place

Deleting by osm_type/osm_id goes wrong if the place was reimported
because of a change in geometry, admin_level etc. The newly created
place is deleted as well in this case.

12 years agosend fail messages to error log
Sarah Hoffmann [Sun, 4 Mar 2012 18:06:20 +0000 (19:06 +0100)]
send fail messages to error log

12 years agorespect addressdetails parameter in json output
Sarah Hoffmann [Sun, 4 Mar 2012 16:45:53 +0000 (17:45 +0100)]
respect addressdetails parameter in json output

12 years agoconvert CRLF to make git happy
Sarah Hoffmann [Sun, 4 Mar 2012 15:41:43 +0000 (16:41 +0100)]
convert CRLF to make git happy

12 years agorestrict detection of lat/lon pairs
Sarah Hoffmann [Sun, 4 Mar 2012 13:50:26 +0000 (14:50 +0100)]
restrict detection of lat/lon pairs

Only interpret numbers as geo coordinates if they are in the
actual range for latitude and longitude.

12 years agocreate proper token array for unknown housenumbers
Sarah Hoffmann [Sun, 4 Mar 2012 13:40:13 +0000 (14:40 +0100)]
create proper token array for unknown housenumbers

12 years agoconfigurable file logging
Sarah Hoffmann [Sun, 4 Mar 2012 11:50:45 +0000 (12:50 +0100)]
configurable file logging

Respect CONST_Log_File setting and add two new configuration
variables CONST_Log_File_SearchLog and CONST_Log_File_ReverseLog
for the log file names.

12 years agoadd script to list broken polygons
Sarah Hoffmann [Sun, 4 Mar 2012 09:54:07 +0000 (10:54 +0100)]
add script to list broken polygons

12 years agocreation of special phrases for country codes (forwardported from gazeteer)
Sarah Hoffmann [Sun, 26 Feb 2012 20:32:40 +0000 (20:32 +0000)]
creation of special phrases for country codes (forwardported from gazeteer)

12 years agoupdate classtype tables for special phrases along with the places
Sarah Hoffmann [Sun, 19 Feb 2012 10:33:21 +0000 (10:33 +0000)]
update classtype tables for special phrases along with the places

12 years agoextend scripts for Tiger 2011 data
Sarah Hoffmann [Fri, 17 Feb 2012 20:34:28 +0000 (20:34 +0000)]
extend scripts for Tiger 2011 data

12 years agohack around yes quotes in wiki and add a simple sanity check against wiki accidents
Sarah Hoffmann [Mon, 13 Feb 2012 22:37:03 +0000 (22:37 +0000)]
hack around yes quotes in wiki and add a simple sanity check against wiki accidents

12 years agoreturn a marginally better error message when query string is not in UTF-8,
Sarah Hoffmann [Sun, 12 Feb 2012 12:01:24 +0000 (12:01 +0000)]
return a marginally better error message when query string is not in UTF-8,
force UTF-8 in search formular

12 years agomove place=isolated_dwelling up to the same rank as place=locality
Sarah Hoffmann [Sun, 12 Feb 2012 10:12:51 +0000 (10:12 +0000)]
move place=isolated_dwelling up to the same rank as place=locality

12 years agoenable exact search of GB postcodes in aux postcode data,
Sarah Hoffmann [Sat, 11 Feb 2012 09:19:42 +0000 (09:19 +0000)]
enable exact search of GB postcodes in aux postcode data,
partial search still disabled due to performance problems

12 years agoduring initial setup defer creation of non-vital until after nominatim indexing,
Sarah Hoffmann [Fri, 10 Feb 2012 21:01:35 +0000 (21:01 +0000)]
during initial setup defer creation of non-vital until after nominatim indexing,
analyse DB during nominatim indexing so the query planer uses indexes correctly

12 years agoavoid warnings when geometry of object cannot be determined
Sarah Hoffmann [Mon, 6 Feb 2012 23:14:21 +0000 (23:14 +0000)]
avoid warnings when geometry of object cannot be determined

12 years agodon't print address details if they are not requested
Sarah Hoffmann [Mon, 6 Feb 2012 22:26:08 +0000 (22:26 +0000)]
don't print address details if they are not requested

12 years agoremove unused protolib
Sarah Hoffmann [Sat, 4 Feb 2012 08:20:51 +0000 (08:20 +0000)]
remove unused protolib

12 years agoremove trailing commas in HTML to make IE happy (thanks to TomH)
Sarah Hoffmann [Sat, 4 Feb 2012 07:53:02 +0000 (07:53 +0000)]
remove trailing commas in HTML to make IE happy (thanks to TomH)

12 years agomunin graph for the import lag
Sarah Hoffmann [Wed, 1 Feb 2012 21:13:05 +0000 (21:13 +0000)]
munin graph for the import lag

12 years agodon't classify normal SQL errors as deadlocks
Sarah Hoffmann [Sat, 28 Jan 2012 23:33:28 +0000 (23:33 +0000)]
don't classify normal SQL errors as deadlocks

12 years agoimprove import performance for special phrases
Sarah Hoffmann [Sat, 28 Jan 2012 23:27:25 +0000 (23:27 +0000)]
improve import performance for special phrases

- create a temporary index on placex
- create indices on place_classtype_* tables after insert
- grant www-data permission to tables

12 years agoextensions where introduced in postgresql in 9.1, not in 9.0
Sarah Hoffmann [Sat, 21 Jan 2012 12:09:43 +0000 (12:09 +0000)]
extensions where introduced in postgresql in 9.1, not in 9.0

12 years ago- remove dependency on intarray extension
Sarah Hoffmann [Sat, 21 Jan 2012 10:21:42 +0000 (10:21 +0000)]
- remove dependency on intarray extension
- new config options for postgresql version and
  location of osm2pgsql and osmosis binaries
- new option --no-npi for update.php that supresses writing of NPIs in
  osmosis update mode
- add more GRANTs for www-data

13 years agofix typo mentioned by mathieu monney on mapquest's forum two weeks ago. this typo...
Frederik Ramm [Fri, 21 Oct 2011 06:48:04 +0000 (06:48 +0000)]
fix typo mentioned by mathieu monney on mapquest's forum two weeks ago. this typo is responsible for 'location_property_tiger does not exist' errors.

13 years agocreate 'file' sequence required by update.php
Frederik Ramm [Fri, 9 Sep 2011 16:02:31 +0000 (16:02 +0000)]
create 'file' sequence required by update.php