]> git.openstreetmap.org Git - nominatim.git/log
nominatim.git
2 years agoRemove legacy Postgres options
otbutz [Tue, 12 Jul 2022 07:49:10 +0000 (09:49 +0200)]
Remove legacy Postgres options

2 years agoMerge pull request #2691 from mtmail/ubuntu-22
Sarah Hoffmann [Mon, 11 Jul 2022 13:37:51 +0000 (15:37 +0200)]
Merge pull request #2691 from mtmail/ubuntu-22

Vagrant and CI tests for Ubuntu 22.04

2 years agoIn tests for PHP 8 disable Just-in-time, it conflicts with tools that determine coverage
marc tobias [Mon, 4 Jul 2022 21:52:36 +0000 (23:52 +0200)]
In tests for PHP 8 disable Just-in-time, it conflicts with tools that determine coverage

2 years agoVagrant and CI tests for Ubuntu 22.04
Marc Tobias [Mon, 2 May 2022 16:16:08 +0000 (18:16 +0200)]
Vagrant and CI tests for Ubuntu 22.04

2 years agodecode_json() always create arrays instead of objects
Sarah Hoffmann [Sat, 9 Jul 2022 07:10:21 +0000 (09:10 +0200)]
decode_json() always create arrays instead of objects

2 years agoconvert admin --analyse-indexing to new indexing method
Sarah Hoffmann [Thu, 7 Jul 2022 09:23:14 +0000 (11:23 +0200)]
convert admin --analyse-indexing to new indexing method

A proper run of indexing requires the place information from the
analyzer. Add the pre-processing of place data, so the right
information is handed into the update function.

2 years agoMerge pull request #2760 from lonvia/reorganize-data-classes
Sarah Hoffmann [Thu, 7 Jul 2022 14:12:11 +0000 (16:12 +0200)]
Merge pull request #2760 from lonvia/reorganize-data-classes

Code cleanup: move some common code into the data submodule

2 years agoremove analyze() from PlaceInfo class
Sarah Hoffmann [Wed, 6 Jul 2022 09:33:07 +0000 (11:33 +0200)]
remove analyze() from PlaceInfo class

The function creates circular dependencies.

2 years agomove country_info into data submodule
Sarah Hoffmann [Wed, 6 Jul 2022 09:08:36 +0000 (11:08 +0200)]
move country_info into data submodule

2 years agomove PlaceInfo into data submodule
Sarah Hoffmann [Wed, 6 Jul 2022 08:54:47 +0000 (10:54 +0200)]
move PlaceInfo into data submodule

This data structure is shared between indexer and tokenizer.

2 years agotest: avoid column names with upper-case letters
Sarah Hoffmann [Tue, 5 Jul 2022 07:12:55 +0000 (09:12 +0200)]
test: avoid column names with upper-case letters

This may cause problems when the column names get quoted.

2 years agoCI: remove unneed stuff to make space for DB
Sarah Hoffmann [Sun, 3 Jul 2022 12:52:16 +0000 (14:52 +0200)]
CI: remove unneed stuff to make space for DB

2 years agoMerge pull request #2706 from mtmail/php-fixes-php7-vs-php8
Sarah Hoffmann [Sun, 3 Jul 2022 09:28:52 +0000 (11:28 +0200)]
Merge pull request #2706 from mtmail/php-fixes-php7-vs-php8

PHP 8 behaves slightly different with in_array and usort

2 years agoPHP 8 behaves slightly different with in_array and usort
Marc Tobias [Tue, 10 May 2022 16:30:49 +0000 (18:30 +0200)]
PHP 8 behaves slightly different with in_array and usort

2 years agofix syntax error with tablespaces
Sarah Hoffmann [Thu, 30 Jun 2022 07:19:16 +0000 (09:19 +0200)]
fix syntax error with tablespaces

2 years agodocs: replace deprecated pages option
Sarah Hoffmann [Sat, 25 Jun 2022 19:29:00 +0000 (21:29 +0200)]
docs: replace deprecated pages option

Fixes #2661.

2 years agofix handling of zero importance
Sarah Hoffmann [Wed, 29 Jun 2022 15:54:30 +0000 (17:54 +0200)]
fix handling of zero importance

To avoid importance becoming zero and cancelling out other weights,
df008d99f549d850d07580b4592435388e44387c introduced a minimum value
for importance. That broke importances for interpolated addresses,
which are less than zero.

Instead of setting a minimum, set zero importances to a very small
value.

Fixes #2753.

2 years agoMerge pull request #2757 from lonvia/filter-postcodes
Sarah Hoffmann [Fri, 24 Jun 2022 19:09:41 +0000 (21:09 +0200)]
Merge pull request #2757 from lonvia/filter-postcodes

Add filtering, normalisation and variants for postcodes

2 years agoignore 5+ postcodes in the US for now
Sarah Hoffmann [Thu, 23 Jun 2022 14:17:47 +0000 (16:17 +0200)]
ignore 5+ postcodes in the US for now

Hierarchical postcodes need a different treatment.

2 years agobdd: correctly skip postcode tests for legacy
Sarah Hoffmann [Wed, 22 Jun 2022 09:38:23 +0000 (11:38 +0200)]
bdd: correctly skip postcode tests for legacy

2 years agobdd: do not expect legacy word table to be without empty tokens
Sarah Hoffmann [Wed, 22 Jun 2022 08:47:08 +0000 (10:47 +0200)]
bdd: do not expect legacy word table to be without empty tokens

It can happen for bogus names and this will not get fixed anymore.

2 years agoadapt search algorithm to new postcode format in word
Sarah Hoffmann [Wed, 22 Jun 2022 07:54:47 +0000 (09:54 +0200)]
adapt search algorithm to new postcode format in word

2 years agohandle postcodes properly on word table updates
Sarah Hoffmann [Tue, 21 Jun 2022 20:05:35 +0000 (22:05 +0200)]
handle postcodes properly on word table updates

update_postcodes_from_db() needs to do the full postcode treatment
in order to derive the correct word table entries.

2 years agoadd documentation for postcode customization
Sarah Hoffmann [Mon, 20 Jun 2022 15:42:12 +0000 (17:42 +0200)]
add documentation for postcode customization

2 years agofix linting issue
Sarah Hoffmann [Fri, 17 Jun 2022 16:14:23 +0000 (18:14 +0200)]
fix linting issue

2 years agofix up BDD tests for postcode changes
Sarah Hoffmann [Fri, 17 Jun 2022 15:28:51 +0000 (17:28 +0200)]
fix up BDD tests for postcode changes

Includes smaller code fixes found by the tests.

2 years agoport legacy tokenizer to new postcode handling
Sarah Hoffmann [Wed, 8 Jun 2022 06:19:55 +0000 (08:19 +0200)]
port legacy tokenizer to new postcode handling

Also documents the changes to the SQL functions of the tokenizer.

2 years agofix postcode pattern for Mozambique
Sarah Hoffmann [Wed, 8 Jun 2022 05:42:35 +0000 (07:42 +0200)]
fix postcode pattern for Mozambique

Optional groups are not implemented yet.

2 years agoadd tests for discarding bad postcodes
Sarah Hoffmann [Wed, 8 Jun 2022 05:24:53 +0000 (07:24 +0200)]
add tests for discarding bad postcodes

2 years agofix liniting problems
Sarah Hoffmann [Wed, 8 Jun 2022 04:33:11 +0000 (06:33 +0200)]
fix liniting problems

2 years agocache postcode normalization
Sarah Hoffmann [Tue, 7 Jun 2022 10:08:22 +0000 (12:08 +0200)]
cache postcode normalization

2 years agoonly add well-formatted postcodes to location table
Sarah Hoffmann [Mon, 6 Jun 2022 21:44:51 +0000 (23:44 +0200)]
only add well-formatted postcodes to location table

2 years agomove postcode matcher in a separate file
Sarah Hoffmann [Mon, 6 Jun 2022 21:37:04 +0000 (23:37 +0200)]
move postcode matcher in a separate file

2 years agomove postcode centroid computation to Python
Sarah Hoffmann [Mon, 6 Jun 2022 08:46:48 +0000 (10:46 +0200)]
move postcode centroid computation to Python

2 years agoadd class for online centroid computation
Sarah Hoffmann [Mon, 6 Jun 2022 07:49:00 +0000 (09:49 +0200)]
add class for online centroid computation

2 years agoicu: switch postcodes to using the pre-formatted one
Sarah Hoffmann [Fri, 3 Jun 2022 15:12:01 +0000 (17:12 +0200)]
icu: switch postcodes to using the pre-formatted one

2 years agointroduce and use analyzer for postcodes
Sarah Hoffmann [Tue, 24 May 2022 19:45:06 +0000 (21:45 +0200)]
introduce and use analyzer for postcodes

2 years agopostcodes: introduce a default pattern for countries without postcodes
Sarah Hoffmann [Tue, 24 May 2022 16:25:37 +0000 (18:25 +0200)]
postcodes: introduce a default pattern for countries without postcodes

2 years agopostcode: generate a generic form
Sarah Hoffmann [Tue, 24 May 2022 15:11:40 +0000 (17:11 +0200)]
postcode: generate a generic form

2 years agoadd postcodes for most of the remaining countries
Sarah Hoffmann [Mon, 23 May 2022 14:11:16 +0000 (16:11 +0200)]
add postcodes for most of the remaining countries

Now includes all postcodes that have optional parts.

2 years agopostcodes: add support for optional spaces
Sarah Hoffmann [Mon, 23 May 2022 12:04:22 +0000 (14:04 +0200)]
postcodes: add support for optional spaces

2 years agoadd postcode formats with optional country code
Sarah Hoffmann [Mon, 23 May 2022 09:10:35 +0000 (11:10 +0200)]
add postcode formats with optional country code

If the country code is not part of the mandatory output, the
country code filter will do the correct handling.

2 years agopostcodes: strip leading country codes
Sarah Hoffmann [Mon, 23 May 2022 09:01:57 +0000 (11:01 +0200)]
postcodes: strip leading country codes

2 years agoadd postcodes patterns without optional spaces
Sarah Hoffmann [Thu, 19 May 2022 14:26:51 +0000 (16:26 +0200)]
add postcodes patterns without optional spaces

2 years agoinitial postcode cleaner for simple patterns
Sarah Hoffmann [Thu, 19 May 2022 13:49:36 +0000 (15:49 +0200)]
initial postcode cleaner for simple patterns

Moves postcodes that are either in countries without a postcode
system or don't correspond to the local pattern for postcodes into
a field for a normal address part. Makes them searchable but not as
a special address. This has two consequences: they are no longer a
skippable part of the address and the postcodes cannot be searched
on their own.

2 years agoadd postcode patterns for numeric postcodes
Sarah Hoffmann [Thu, 19 May 2022 10:03:26 +0000 (12:03 +0200)]
add postcode patterns for numeric postcodes

Adds patterns for countries that have simple numeric-only postcodes.

2 years agoremove postcodes from countries that don't have them
Sarah Hoffmann [Thu, 12 May 2022 09:43:47 +0000 (11:43 +0200)]
remove postcodes from countries that don't have them

The postcodes will only be removed as a 'computed postcode' they
are still searchable for the given object.

2 years agoadd info about countries without a postcode
Sarah Hoffmann [Wed, 11 May 2022 15:23:28 +0000 (17:23 +0200)]
add info about countries without a postcode

2 years agoavoid near searches in very large areas
Sarah Hoffmann [Thu, 23 Jun 2022 20:06:48 +0000 (22:06 +0200)]
avoid near searches in very large areas

At some point the contains call becomes too expensive.

2 years agoMerge pull request #2755 from Luflosi/fix-typo
Sarah Hoffmann [Mon, 20 Jun 2022 20:23:36 +0000 (22:23 +0200)]
Merge pull request #2755 from Luflosi/fix-typo

Fix typo

2 years agoFix typo
Luflosi [Mon, 20 Jun 2022 18:41:00 +0000 (20:41 +0200)]
Fix typo

2 years agoMerge pull request #2751 from mtmail/issue-2750
Sarah Hoffmann [Mon, 20 Jun 2022 08:21:06 +0000 (10:21 +0200)]
Merge pull request #2751 from mtmail/issue-2750

Documentation fix: should be "nominatim refresh"

2 years agoDocumentation fix: should be "nominatim refresh"
marc tobias [Mon, 20 Jun 2022 00:32:23 +0000 (02:32 +0200)]
Documentation fix: should be "nominatim refresh"

2 years agoMerge pull request #2746 from bgo-eiu/patch-2
Sarah Hoffmann [Sat, 18 Jun 2022 07:40:47 +0000 (09:40 +0200)]
Merge pull request #2746 from bgo-eiu/patch-2

Added additional languages for Pakistan in country settings

2 years agoMerge pull request #2749 from stefkiourk/patch-1
Sarah Hoffmann [Fri, 17 Jun 2022 20:11:55 +0000 (22:11 +0200)]
Merge pull request #2749 from stefkiourk/patch-1

Typos and syntax on Reverse.md

2 years agoTypos and syntax on Reverse.md
Stef Ki [Fri, 17 Jun 2022 19:01:38 +0000 (21:01 +0200)]
Typos and syntax on Reverse.md

2 years agoMerge pull request #2748 from lonvia/bdd-grid-origin
Sarah Hoffmann [Fri, 17 Jun 2022 13:25:29 +0000 (15:25 +0200)]
Merge pull request #2748 from lonvia/bdd-grid-origin

BDD tests: remove support for scenes

2 years agobdd: remove support for scenes
Sarah Hoffmann [Fri, 17 Jun 2022 08:06:56 +0000 (10:06 +0200)]
bdd: remove support for scenes

Only keep support for the special point geometry 'country:xx'.

2 years agobdd: move query tests from scene to grid description
Sarah Hoffmann [Thu, 16 Jun 2022 20:08:30 +0000 (22:08 +0200)]
bdd: move query tests from scene to grid description

2 years agobdd: remove step for scene setup
Sarah Hoffmann [Thu, 16 Jun 2022 19:59:35 +0000 (21:59 +0200)]
bdd: remove step for scene setup

2 years agobdd: move update tests from scenes to grid descriptions
Sarah Hoffmann [Thu, 16 Jun 2022 19:56:39 +0000 (21:56 +0200)]
bdd: move update tests from scenes to grid descriptions

2 years agobdd: move import tests from scenes to grid descriptions
Sarah Hoffmann [Wed, 15 Jun 2022 20:22:13 +0000 (22:22 +0200)]
bdd: move import tests from scenes to grid descriptions

2 years agobdd: clear lof buffer after a successful import run
Sarah Hoffmann [Mon, 13 Jun 2022 09:49:38 +0000 (11:49 +0200)]
bdd: clear lof buffer after a successful import run

2 years agobdd: allow to set an origin of the grid
Sarah Hoffmann [Mon, 13 Jun 2022 07:44:37 +0000 (09:44 +0200)]
bdd: allow to set an origin of the grid

2 years agoMerge pull request #2745 from lonvia/city-in-city-fix
Sarah Hoffmann [Thu, 16 Jun 2022 13:36:39 +0000 (15:36 +0200)]
Merge pull request #2745 from lonvia/city-in-city-fix

Improve hierarchy computation for place areas

2 years agoadded additional languages for pakistan in country settings
bgo-eiu [Thu, 16 Jun 2022 10:26:44 +0000 (06:26 -0400)]
added additional languages for pakistan in country settings

2 years agouse default ranks when reorganising rank_address
Sarah Hoffmann [Thu, 16 Jun 2022 09:20:23 +0000 (11:20 +0200)]
use default ranks when reorganising rank_address

When shifting address ranks, the evaluation is always done against
unshifted address ranks on import because the objects we compare against
have not been indexed yet. This changes for updates when the object have
been touched in the meantime. To ensure consistent behaviour across
imports and updates, always use the  unshifted address ranks.

2 years agoimprove address ordering with mixes of place and admin areas
Sarah Hoffmann [Thu, 16 Jun 2022 08:44:16 +0000 (10:44 +0200)]
improve address ordering with mixes of place and admin areas

Resolves a couple of situations where a mixed use of places areas and
administrative boundaries would result in a hierarchy that did not
properly respect the contains relation.

2 years agoMerge pull request #2737 from lonvia/reset-linking-ranks
Sarah Hoffmann [Mon, 6 Jun 2022 07:29:32 +0000 (09:29 +0200)]
Merge pull request #2737 from lonvia/reset-linking-ranks

Fix rank inheritance from linked places

2 years agoadd testcase for #2551
Sarah Hoffmann [Sun, 5 Jun 2022 19:39:17 +0000 (21:39 +0200)]
add testcase for #2551

This test proves that places that are linked need to be reindexed.

2 years agofix rank inheritance from linked places
Sarah Hoffmann [Sun, 5 Jun 2022 17:38:14 +0000 (19:38 +0200)]
fix rank inheritance from linked places

When taking over the address rank from a linked place, it needs
to be the originally computed rank, not the one that might have
been adjusted in the meantime. The adjustment was made under the
assumption that the node is not linked.

2 years agoMerge pull request #2736 from lonvia/reverse-interpolation-index-order
Sarah Hoffmann [Fri, 3 Jun 2022 08:42:54 +0000 (10:42 +0200)]
Merge pull request #2736 from lonvia/reverse-interpolation-index-order

Change indexing order for interpolations and non-addressable objects

2 years agochange indexing order for interpolations
Sarah Hoffmann [Tue, 31 May 2022 12:16:06 +0000 (14:16 +0200)]
change indexing order for interpolations

Interpolations are now indexed after rank 30 objects. The housenumber
nodes no longer need information from the interpolations while the
interpolations can make use of precomputed postcodes.

2 years agoMerge pull request #2735 from lonvia/geocodejson-type-reverse
Sarah Hoffmann [Wed, 1 Jun 2022 20:14:06 +0000 (22:14 +0200)]
Merge pull request #2735 from lonvia/geocodejson-type-reverse

Also fix type output in geocodejson for reverse

2 years agouse getattr() instead of __getattr__
Sarah Hoffmann [Wed, 1 Jun 2022 19:26:13 +0000 (21:26 +0200)]
use getattr() instead of __getattr__

Makes the linter happy.

2 years agoalso fix type output in geocodejson for reverse
Sarah Hoffmann [Tue, 24 May 2022 08:57:45 +0000 (10:57 +0200)]
also fix type output in geocodejson for reverse

2 years agoMerge pull request #2732 from lonvia/fix-ordering-address-parts
Sarah Hoffmann [Tue, 31 May 2022 18:26:05 +0000 (20:26 +0200)]
Merge pull request #2732 from lonvia/fix-ordering-address-parts

Fix order when searching for addr:* components

2 years agoMerge pull request #2731 from lonvia/cleanup-special-phrases
Sarah Hoffmann [Tue, 31 May 2022 15:13:56 +0000 (17:13 +0200)]
Merge pull request #2731 from lonvia/cleanup-special-phrases

Minor code reorganisation around special phrase parsing

2 years agoMerge pull request #2730 from lonvia/exclude-inclusion-tag
Sarah Hoffmann [Tue, 31 May 2022 15:13:37 +0000 (17:13 +0200)]
Merge pull request #2730 from lonvia/exclude-inclusion-tag

Exclude addr:inclusion from search

2 years agofix order when searching for addr:* components
Sarah Hoffmann [Tue, 31 May 2022 14:57:37 +0000 (16:57 +0200)]
fix order when searching for addr:* components

When matching addr:* components the preference was given to
matches that do not intersect with the place.

2 years agoexclude addr:inclusion from search
Sarah Hoffmann [Tue, 31 May 2022 09:23:42 +0000 (11:23 +0200)]
exclude addr:inclusion from search

2 years agoCI: always use the latest version of pylint
Sarah Hoffmann [Tue, 31 May 2022 07:12:26 +0000 (09:12 +0200)]
CI: always use the latest version of pylint

This makes it easier to reproduce issues locally.

2 years agocustom comparison for SpecialPhrase
Sarah Hoffmann [Mon, 30 May 2022 14:30:41 +0000 (16:30 +0200)]
custom comparison for SpecialPhrase

Duplicate elemination only works when a custom hash/equal function
is implemented that is based on the members.

2 years agomove quoting hack to wiki loader
Sarah Hoffmann [Mon, 30 May 2022 12:32:36 +0000 (14:32 +0200)]
move quoting hack to wiki loader

The bad quotes around the type for special phrases
specifically occure in the Wiki pages, so it should be
removed by the loader and not in the generic SpecialPhrase
object.

2 years agoconvert special phrase loaders to generators
Sarah Hoffmann [Mon, 30 May 2022 12:12:46 +0000 (14:12 +0200)]
convert special phrase loaders to generators

Generators simplify the code quite a bit compared to the previous
Iterator approach.

2 years agoremove the language parameter in the SPWikiLoader
Sarah Hoffmann [Sun, 29 May 2022 12:06:05 +0000 (14:06 +0200)]
remove the language parameter in the SPWikiLoader

Languages must always be configured through config or environment.
Also use monkeypatched environment in tests.

2 years agoadd get_str_list() for config
Sarah Hoffmann [Sun, 29 May 2022 11:53:50 +0000 (13:53 +0200)]
add get_str_list() for config

Converts a config value written as a comma-sparated list into
a Python list of strings.

2 years agoMerge pull request #2728 from lonvia/allow-more-partials
Sarah Hoffmann [Fri, 27 May 2022 16:09:11 +0000 (18:09 +0200)]
Merge pull request #2728 from lonvia/allow-more-partials

Allow search for partials consisting of 3 or more words

2 years agoallow search for partials consisting of 3 or more words
Sarah Hoffmann [Fri, 27 May 2022 14:49:14 +0000 (16:49 +0200)]
allow search for partials consisting of 3 or more words

The search query builder currently rejects searches for partial
names only, when the partial terms are all very frequent to avoid
queries that return too many results.

This change slightly relaxes the condition to allow the search when
there are 3 or more partial terms. With so many terms the number
of matches should be managable.

2 years agoMerge pull request #2715 from otbutz/patch-2
Sarah Hoffmann [Tue, 24 May 2022 12:40:28 +0000 (14:40 +0200)]
Merge pull request #2715 from otbutz/patch-2

Simplify apache rewrite rules

2 years agoChange to regular regex group
otbutz [Tue, 24 May 2022 09:32:59 +0000 (11:32 +0200)]
Change to regular regex group

2 years agoMerge pull request #2718 from nslxndr/fix-log-endtime
Sarah Hoffmann [Mon, 23 May 2022 14:25:41 +0000 (16:25 +0200)]
Merge pull request #2718 from nslxndr/fix-log-endtime

Undefined offset in error log

2 years agoMerge pull request #2722 from lonvia/fix-relinking-on-updates
Sarah Hoffmann [Mon, 23 May 2022 09:36:20 +0000 (11:36 +0200)]
Merge pull request #2722 from lonvia/fix-relinking-on-updates

Fix bug with keeping linking on updates

2 years agofix bug with keeping linking on updates
Sarah Hoffmann [Mon, 23 May 2022 08:11:28 +0000 (10:11 +0200)]
fix bug with keeping linking on updates

When moving the finding of linked places to the precomputation stage,
it was also moved before the statement where the linked_place_id was
removed from the linkee. The result was that the current linkee was
excluded when looking for a linked place on updates because it was
still linked to the boundary to be updated.

Fixed by allowing to either keep the linkage or change to an unlinked
place.

2 years agocorrect end time computation
Sandor Nagy [Mon, 9 May 2022 20:00:36 +0000 (22:00 +0200)]
correct end time computation

2 years agoSimplify apache rewrite rules
otbutz [Fri, 20 May 2022 08:15:28 +0000 (10:15 +0200)]
Simplify apache rewrite rules

2 years agoMerge pull request #2713 from lonvia/remove-county-nodes-in-canada
Sarah Hoffmann [Thu, 19 May 2022 08:21:09 +0000 (10:21 +0200)]
Merge pull request #2713 from lonvia/remove-county-nodes-in-canada

Remove county nodes in Canada from addresses

2 years agoremove county nodes in Canada from addresses
Sarah Hoffmann [Wed, 18 May 2022 08:19:05 +0000 (10:19 +0200)]
remove county nodes in Canada from addresses

Canada has complete coverage for administrative boundaries on
county level. Removing the county nodes from the addresses avoids error
due to a wide-spread doubling of place nodes for city counties.

2 years agoMerge pull request #2710 from lonvia/offline-import-mode
Sarah Hoffmann [Thu, 12 May 2022 09:08:29 +0000 (11:08 +0200)]
Merge pull request #2710 from lonvia/offline-import-mode

Assorted performance improvements for BDD tests

2 years agobdd: restrict imports to four languages
Sarah Hoffmann [Wed, 11 May 2022 14:40:53 +0000 (16:40 +0200)]
bdd: restrict imports to four languages

This mainly restricts the number of country names that are loaded.