]> git.openstreetmap.org Git - nominatim.git/log
nominatim.git
10 months agoalways add a penalty for name + address search fallback
Sarah Hoffmann [Sun, 28 Jan 2024 10:48:03 +0000 (11:48 +0100)]
always add a penalty for name + address search fallback

If there already was a search by full names, the search is likely
a repeatition that yields the same results, only running slower.

10 months agodisallow category tokens in the middle of a query string
Sarah Hoffmann [Sun, 28 Jan 2024 10:35:30 +0000 (11:35 +0100)]
disallow category tokens in the middle of a query string

This already worked for left-to-right readings and now is also
implemented for right-to-left reading. A qualifier must always be
before or after the name.

10 months agoprotect against very frequent bad partials
Sarah Hoffmann [Sat, 27 Jan 2024 20:26:03 +0000 (21:26 +0100)]
protect against very frequent bad partials

10 months agoavoid duplicates in the list of partial tokens for a query
Sarah Hoffmann [Sat, 27 Jan 2024 19:48:40 +0000 (20:48 +0100)]
avoid duplicates in the list of partial tokens for a query

This messes with the estimates for expected results.

10 months agoMerge pull request #3320 from lonvia/fix-timeout-return-code
Sarah Hoffmann [Sun, 28 Jan 2024 09:37:37 +0000 (10:37 +0100)]
Merge pull request #3320 from lonvia/fix-timeout-return-code

Fix returned HTTP error when query runs too long

10 months agocatch special async timeout error in servers
Sarah Hoffmann [Sat, 27 Jan 2024 18:53:13 +0000 (19:53 +0100)]
catch special async timeout error in servers

In Python <= 3.10 this is not yet the same as TimeoutError.

Fixes #3303.

10 months agoMerge pull request #3317 from lonvia/postcodes-for-highway-areas
Sarah Hoffmann [Fri, 26 Jan 2024 18:51:02 +0000 (19:51 +0100)]
Merge pull request #3317 from lonvia/postcodes-for-highway-areas

Search postcodes for highway areas around the area

10 months agosearch postcodes for highway areas around the area
Sarah Hoffmann [Fri, 26 Jan 2024 16:26:29 +0000 (17:26 +0100)]
search postcodes for highway areas around the area

So far the code would only accept postcodes that are inside the area.

Fixes #3304.

11 months agoMerge pull request #3301 from lonvia/fix-class-search-regression
Sarah Hoffmann [Wed, 17 Jan 2024 09:47:35 +0000 (10:47 +0100)]
Merge pull request #3301 from lonvia/fix-class-search-regression

Interpret stand-alone special terms always as near term

11 months agointerpret stand-alone special terms always as near term
Sarah Hoffmann [Tue, 16 Jan 2024 16:19:21 +0000 (17:19 +0100)]
interpret stand-alone special terms always as near term

Fixes #3298.

11 months agoMerge pull request #3293 from lonvia/rematch-against-country-code
Sarah Hoffmann [Mon, 8 Jan 2024 17:48:32 +0000 (18:48 +0100)]
Merge pull request #3293 from lonvia/rematch-against-country-code

Add country code to words to be rematched

11 months agoadd country code to words to be rematched
Sarah Hoffmann [Mon, 8 Jan 2024 11:23:23 +0000 (12:23 +0100)]
add country code to words to be rematched

11 months agoMerge pull request #3292 from lonvia/faster-country-search
Sarah Hoffmann [Sun, 7 Jan 2024 19:42:20 +0000 (20:42 +0100)]
Merge pull request #3292 from lonvia/faster-country-search

Speed up country search

11 months agowhen a country is in the results, restrict further searches to places
Sarah Hoffmann [Sun, 7 Jan 2024 16:29:12 +0000 (17:29 +0100)]
when a country is in the results, restrict further searches to places

A country search result usually comes with a very high importance.
As a result only other very well known places will show up together
with country results and that means only places with lower address
ranks. Name searches for country names tend to yield a lot of POI
results because the country name is part of the name
(think "embassy of Sweden"). By excluding POIs from further searches,
the search is sped up quite a bit.

11 months agodump params in log view
Sarah Hoffmann [Sun, 7 Jan 2024 14:37:53 +0000 (15:37 +0100)]
dump params in log view

11 months agoprioritize country searches when penaly is equal
Sarah Hoffmann [Sun, 7 Jan 2024 10:11:11 +0000 (11:11 +0100)]
prioritize country searches when penaly is equal

11 months agoMerge pull request #3238 from mtmail/check-database-for-version-match
Sarah Hoffmann [Sun, 7 Jan 2024 14:24:00 +0000 (15:24 +0100)]
Merge pull request #3238 from mtmail/check-database-for-version-match

admin --check-database also checks database vs nominatim version

11 months agoMerge pull request #3289 from lonvia/viewbox-and-housenumbers
Sarah Hoffmann [Sun, 7 Jan 2024 14:23:14 +0000 (15:23 +0100)]
Merge pull request #3289 from lonvia/viewbox-and-housenumbers

Do not restrict by viewbox when housenumber or postcode is available

11 months agoMerge pull request #3290 from lonvia/near-vs-quaifier-words
Sarah Hoffmann [Sun, 7 Jan 2024 14:23:00 +0000 (15:23 +0100)]
Merge pull request #3290 from lonvia/near-vs-quaifier-words

Do not run near queries on qualifier words

11 months agoMerge pull request #3291 from lonvia/fix-timezone-handling
Sarah Hoffmann [Sun, 7 Jan 2024 14:22:42 +0000 (15:22 +0100)]
Merge pull request #3291 from lonvia/fix-timezone-handling

Fix timezone handling for timestamps from the database

11 months agofix timezone handling for timestamps from the database
Sarah Hoffmann [Sat, 6 Jan 2024 21:31:38 +0000 (22:31 +0100)]
fix timezone handling for timestamps from the database

SQLite is not timezone-aware, so make sure to convert to UTC
before inserting any data.

11 months agodo not run near queries on qualifier words
Sarah Hoffmann [Sat, 6 Jan 2024 16:49:58 +0000 (17:49 +0100)]
do not run near queries on qualifier words

There is too much potential for confusion (e.g. 'Rio Grande' read
as 'river near Grande') fir too little gain. Use near phrases
instead.

11 months agodo not restrict by viewbox when housenumber or postcode is available
Sarah Hoffmann [Sat, 6 Jan 2024 15:53:48 +0000 (16:53 +0100)]
do not restrict by viewbox when housenumber or postcode is available

Fixes #3274.

11 months agocheck-database also checks database vs nominatim version
Marc Tobias [Thu, 2 Nov 2023 15:19:47 +0000 (16:19 +0100)]
check-database also checks database vs nominatim version

11 months agoMerge pull request #3286 from lonvia/avoid-bind-parameters-in-lambdas
Sarah Hoffmann [Fri, 5 Jan 2024 20:24:48 +0000 (21:24 +0100)]
Merge pull request #3286 from lonvia/avoid-bind-parameters-in-lambdas

Avoid closure variables in lambda statements

11 months agoavoid closure variables in lambda statements
Sarah Hoffmann [Fri, 5 Jan 2024 16:49:28 +0000 (17:49 +0100)]
avoid closure variables in lambda statements

There is a bug in SQLAlchemy that assigns the wrong value to bind
parameters from closure variables when reusing lambda statements
that are later extended with other non-lambda expressions.

Thus either avoid lambda statements with closure variables or extending
them with non-lambda expressions.

11 months agouse correct SQLAlchemy pool for asynchronous connections
Sarah Hoffmann [Tue, 2 Jan 2024 15:15:44 +0000 (16:15 +0100)]
use correct SQLAlchemy pool for asynchronous connections

See https://github.com/sqlalchemy/sqlalchemy/issues/8771

11 months agoAPI: avoid engine disposal on startup
Sarah Hoffmann [Tue, 2 Jan 2024 15:10:30 +0000 (16:10 +0100)]
API: avoid engine disposal on startup

11 months agoupdate typing for latest changes in SQLAlchemy
Sarah Hoffmann [Fri, 29 Dec 2023 19:55:33 +0000 (20:55 +0100)]
update typing for latest changes in SQLAlchemy

12 months agoMerge pull request #3273 from lonvia/search-with-sqlite
Sarah Hoffmann [Tue, 12 Dec 2023 11:15:22 +0000 (12:15 +0100)]
Merge pull request #3273 from lonvia/search-with-sqlite

Add forward search capability for SQLite databases

12 months agoadd minimal documentation for the SQLite usage
Sarah Hoffmann [Sat, 9 Dec 2023 15:30:31 +0000 (16:30 +0100)]
add minimal documentation for the SQLite usage

12 months agoerror out when a SQLite database does not exist
Sarah Hoffmann [Thu, 7 Dec 2023 09:24:53 +0000 (10:24 +0100)]
error out when a SQLite database does not exist

Requires to mark the databse r/w when it is newly created in the
convert function.

12 months agoenable search for sqlite conversion by default
Sarah Hoffmann [Thu, 7 Dec 2023 08:33:42 +0000 (09:33 +0100)]
enable search for sqlite conversion by default

12 months agoenable all BDD API tests for sqlite
Sarah Hoffmann [Wed, 6 Dec 2023 19:57:09 +0000 (20:57 +0100)]
enable all BDD API tests for sqlite

12 months agoenable all API tests for sqlite and port missing features
Sarah Hoffmann [Wed, 6 Dec 2023 19:56:21 +0000 (20:56 +0100)]
enable all API tests for sqlite and port missing features

12 months agoextend sqlite converter for search tables
Sarah Hoffmann [Wed, 6 Dec 2023 12:42:58 +0000 (13:42 +0100)]
extend sqlite converter for search tables

12 months agoremove unused function
Sarah Hoffmann [Wed, 6 Dec 2023 10:14:36 +0000 (11:14 +0100)]
remove unused function

12 months agoimprove typing for @compiles constructs
Sarah Hoffmann [Wed, 6 Dec 2023 10:13:12 +0000 (11:13 +0100)]
improve typing for @compiles constructs

The first parameter is in fact the self parameter referring to
the function class.

12 months agovoid unnecessary aliases
Sarah Hoffmann [Wed, 6 Dec 2023 10:03:12 +0000 (11:03 +0100)]
void unnecessary aliases

12 months agofactor out SQL for filtering by location
Sarah Hoffmann [Wed, 6 Dec 2023 09:55:21 +0000 (10:55 +0100)]
factor out SQL for filtering by location

Also improves on the decision if an indexed is used or not.

12 months agosimplify handling of SQL lookup code for search_name
Sarah Hoffmann [Wed, 6 Dec 2023 09:37:06 +0000 (10:37 +0100)]
simplify handling of SQL lookup code for search_name

Use function classes which can be instantiated directly.

12 months agocorrectly close API objects during testing
Sarah Hoffmann [Tue, 5 Dec 2023 20:20:57 +0000 (21:20 +0100)]
correctly close API objects during testing

12 months agoclean up ST_DWithin and intersects() functions
Sarah Hoffmann [Tue, 5 Dec 2023 17:02:40 +0000 (18:02 +0100)]
clean up ST_DWithin and intersects() functions

A non-index version of ST_DWithin is not necessary. ST_Distance
can be used for that purpose. Index use for intersects can be
covered with a simple parameter.

12 months agosimplify weigh_search() function
Sarah Hoffmann [Tue, 5 Dec 2023 15:07:56 +0000 (16:07 +0100)]
simplify weigh_search() function

Use JSON arrays which can have mixed types and therefore have
a more logical structure than separate arrays. Avoid JSON dicts
because of their verboseness.

12 months agofix parameter formatting in sqlite debug output
Sarah Hoffmann [Tue, 5 Dec 2023 14:23:16 +0000 (15:23 +0100)]
fix parameter formatting in sqlite debug output

12 months agoenable connection pools for sqlite
Sarah Hoffmann [Tue, 5 Dec 2023 11:22:00 +0000 (12:22 +0100)]
enable connection pools for sqlite

Connecting is reasonably expensive because the spatialite extension
needs to be loaded. Disable pooling for tests because there is some
memory leak when quickly opening and closing QueuePools with sqlite
connections.

12 months agohide type differences between Postgres and Sqlite in custom types
Sarah Hoffmann [Tue, 5 Dec 2023 10:29:16 +0000 (11:29 +0100)]
hide type differences between Postgres and Sqlite in custom types

Also define a custom set of operators in preparation of differences
in implementation.

12 months agoMerge pull request #3108 from mtmail/remove-legacy-wikipedia-tag-syntax
Sarah Hoffmann [Thu, 7 Dec 2023 08:24:32 +0000 (09:24 +0100)]
Merge pull request #3108 from mtmail/remove-legacy-wikipedia-tag-syntax

These days the OSM wikipedia tab no longer contains URLs

12 months agoMerge pull request #3268 from mtmail/wikipedia-file-path-warning
Sarah Hoffmann [Thu, 7 Dec 2023 08:21:59 +0000 (09:21 +0100)]
Merge pull request #3268 from mtmail/wikipedia-file-path-warning

Improve error message when Wikipedia importance file is not found

12 months agoRemoved unnecessary check for --prepare-database flag
Robbe Haesendonck [Thu, 23 Nov 2023 14:59:01 +0000 (15:59 +0100)]
Removed unnecessary check for --prepare-database flag

12 months agoConnect using localhost instead of socket
Robbe Haesendonck [Thu, 23 Nov 2023 14:51:35 +0000 (15:51 +0100)]
Connect using localhost instead of socket

12 months agoAdded changing permissions of nominatim-project dir
Robbe Haesendonck [Thu, 23 Nov 2023 12:41:13 +0000 (13:41 +0100)]
Added changing permissions of nominatim-project dir

12 months agoAdded osm-import to sudoers file for tokenizer setup
Robbe Haesendonck [Thu, 23 Nov 2023 12:17:33 +0000 (13:17 +0100)]
Added osm-import to sudoers file for tokenizer setup

12 months agoFixing CI tests for install-no-superuser
Robbe Haesendonck [Thu, 23 Nov 2023 11:03:36 +0000 (12:03 +0100)]
Fixing CI tests for install-no-superuser

12 months agoFixed typechecking error
Robbe Haesendonck [Thu, 23 Nov 2023 10:25:49 +0000 (11:25 +0100)]
Fixed typechecking error

12 months agoAdded missing return types to functions
Robbe Haesendonck [Fri, 29 Sep 2023 08:23:35 +0000 (10:23 +0200)]
Added missing return types to functions

12 months ago CI tests: Removed creation of user www-data.
Robbe Haesendonck [Fri, 29 Sep 2023 08:18:44 +0000 (10:18 +0200)]
 CI tests: Removed creation of user www-data.
Removed creation of user www-data as it already exists

12 months agoFixed legacy import command
Robbe Haesendonck [Thu, 28 Sep 2023 07:59:42 +0000 (09:59 +0200)]
Fixed legacy import command
Check whether prepare-database is true, if so exit early

12 months agoDisabled pylint too-many-branches
Robbe Haesendonck [Wed, 27 Sep 2023 14:40:08 +0000 (16:40 +0200)]
Disabled pylint too-many-branches

12 months agoFixed setting tokenizer property
Robbe Haesendonck [Wed, 27 Sep 2023 14:32:28 +0000 (16:32 +0200)]
Fixed setting tokenizer property

12 months agoUpdated no-superuser install ci test
Robbe Haesendonck [Wed, 27 Sep 2023 13:28:21 +0000 (15:28 +0200)]
Updated no-superuser install ci test

Added nominatim_database_webuser (www-data).
Set non-superuser password for importing

12 months agoUpdated check to see if osm_file is set
Robbe Haesendonck [Wed, 27 Sep 2023 08:50:40 +0000 (10:50 +0200)]
Updated check to see if osm_file is set

12 months agoFixed formatting
Robbe Haesendonck [Wed, 27 Sep 2023 08:37:57 +0000 (10:37 +0200)]
Fixed formatting

12 months agoUpdated ci tests to reflect changes in Nominatim CLI
Robbe Haesendonck [Wed, 27 Sep 2023 08:28:20 +0000 (10:28 +0200)]
Updated ci tests to reflect changes in Nominatim CLI

12 months agoMade sure legacy import command still works
Robbe Haesendonck [Wed, 27 Sep 2023 07:29:54 +0000 (09:29 +0200)]
Made sure legacy import command still works

12 months agoRemoved unused variable, fixed connection
Robbe Haesendonck [Wed, 27 Sep 2023 07:24:04 +0000 (09:24 +0200)]
Removed unused variable, fixed connection

12 months agoRemoved _is_complete_import check
Robbe Haesendonck [Wed, 27 Sep 2023 07:21:56 +0000 (09:21 +0200)]
Removed _is_complete_import check

12 months agoImproved logic.
Robbe Haesendonck [Tue, 26 Sep 2023 12:46:28 +0000 (14:46 +0200)]
Improved logic.

Fixed small oversight in mutually exclusiveness of arguments

12 months agoAdded new psql user for importing the data
Robbe Haesendonck [Tue, 26 Sep 2023 10:59:14 +0000 (12:59 +0200)]
Added new psql user for importing the data

12 months agoChanged naming of flags.
Robbe Haesendonck [Tue, 26 Sep 2023 09:59:32 +0000 (11:59 +0200)]
Changed naming of flags.

Made sure legacy import is also still ok

12 months agoAdded check to see if hstore is loaded
Robbe Haesendonck [Mon, 25 Sep 2023 10:39:38 +0000 (12:39 +0200)]
Added check to see if hstore is loaded

12 months agoFixed ci-tests, osm-file flag
Robbe Haesendonck [Mon, 25 Sep 2023 09:14:14 +0000 (11:14 +0200)]
Fixed ci-tests, osm-file flag

Preparing database should work without osm-file

12 months agoRenamed flags
Robbe Haesendonck [Mon, 25 Sep 2023 09:09:20 +0000 (11:09 +0200)]
Renamed flags

12 months agoFixed pylint warnings
Robbe Haesendonck [Thu, 21 Sep 2023 10:28:45 +0000 (12:28 +0200)]
Fixed pylint warnings

12 months agoAdded check for hstore extension
Robbe Haesendonck [Thu, 21 Sep 2023 10:20:46 +0000 (12:20 +0200)]
Added check for hstore extension

12 months agoFixed typo
Robbe Haesendonck [Thu, 21 Sep 2023 09:20:24 +0000 (11:20 +0200)]
Fixed typo

12 months agoFixed linting errors
Robbe Haesendonck [Thu, 21 Sep 2023 07:58:38 +0000 (09:58 +0200)]
Fixed linting errors

12 months agoAdded version check for PostGis and Postgres
Robbe Haesendonck [Wed, 20 Sep 2023 14:05:12 +0000 (16:05 +0200)]
Added version check for PostGis and Postgres

12 months agoUpdated CI tests to check new import flags
Robbe Haesendonck [Wed, 20 Sep 2023 11:42:06 +0000 (13:42 +0200)]
Updated CI tests to check new import flags

12 months agoAdded --prepare-database flag
Robbe Haesendonck [Wed, 20 Sep 2023 11:00:39 +0000 (13:00 +0200)]
Added --prepare-database flag

12 months agoAdded --no-superuser flag
Robbe Haesendonck [Tue, 19 Sep 2023 14:19:11 +0000 (16:19 +0200)]
Added --no-superuser flag

To allow import into already existing databases.
Refs: #2719

12 months agoImprove error message when Wikipedia importance file is not found
marc tobias [Wed, 6 Dec 2023 13:31:07 +0000 (14:31 +0100)]
Improve error message when Wikipedia importance file is not found

12 months agoactions: pin pytest-asyncio version for Ubuntu 20
Sarah Hoffmann [Sun, 3 Dec 2023 14:51:42 +0000 (15:51 +0100)]
actions: pin pytest-asyncio version for Ubuntu 20

Newest versions are no longer compatible with older pytest.

12 months agoskip lookup with full names when there are none
Sarah Hoffmann [Fri, 1 Dec 2023 11:11:58 +0000 (12:11 +0100)]
skip lookup with full names when there are none

12 months agoMerge pull request #3263 from lonvia/near-search-penalty-inheritance
Sarah Hoffmann [Wed, 29 Nov 2023 13:02:24 +0000 (14:02 +0100)]
Merge pull request #3263 from lonvia/near-search-penalty-inheritance

More tweaks for Python frontend search

12 months agofurther restrict stop search criterion
Sarah Hoffmann [Wed, 29 Nov 2023 10:19:06 +0000 (11:19 +0100)]
further restrict stop search criterion

12 months agouse restrict for housenumber lookups with few numbers
Sarah Hoffmann [Sat, 25 Nov 2023 17:39:28 +0000 (18:39 +0100)]
use restrict for housenumber lookups with few numbers

12 months agoNearSearch needs to inherit penalty from inner search
Sarah Hoffmann [Tue, 28 Nov 2023 19:12:12 +0000 (20:12 +0100)]
NearSearch needs to inherit penalty from inner search

12 months agoMerge pull request #3262 from lonvia/fix-category-search
Sarah Hoffmann [Tue, 28 Nov 2023 19:02:36 +0000 (20:02 +0100)]
Merge pull request #3262 from lonvia/fix-category-search

Fix use of category parameter for search() function

12 months agocorrectly exclude streets with housenumber searches
Sarah Hoffmann [Tue, 28 Nov 2023 16:53:37 +0000 (17:53 +0100)]
correctly exclude streets with housenumber searches

Street result are not subject to the full filtering in the SQL
query, so recheck.

12 months agoadd tests for interaction of category parameter with category terms
Sarah Hoffmann [Tue, 28 Nov 2023 15:56:08 +0000 (16:56 +0100)]
add tests for interaction of category parameter with category terms

12 months agorename use of category as POI search to near_item
Sarah Hoffmann [Tue, 28 Nov 2023 15:27:05 +0000 (16:27 +0100)]
rename use of category as POI search to near_item

Use the term category only as a short-cut for "tuple of key and value".

12 months agothe category parameter in search should result in a qualifier
Sarah Hoffmann [Tue, 28 Nov 2023 11:01:49 +0000 (12:01 +0100)]
the category parameter in search should result in a qualifier

12 months agoMerge pull request #3260 from lonvia/improve-catgeory-search
Sarah Hoffmann [Mon, 27 Nov 2023 11:02:11 +0000 (12:02 +0100)]
Merge pull request #3260 from lonvia/improve-catgeory-search

Various improvements to search with special phrases for Python frontend

12 months agodrop category tokens when they make up a full phrase
Sarah Hoffmann [Sun, 26 Nov 2023 19:58:50 +0000 (20:58 +0100)]
drop category tokens when they make up a full phrase

12 months agorestrict base results in near search by rank
Sarah Hoffmann [Sun, 26 Nov 2023 16:41:29 +0000 (17:41 +0100)]
restrict base results in near search by rank

This avoids in particular that roads or POIs are used as base
for the near search when a place result is present.

12 months agodeduplicate categories/qualifiers
Sarah Hoffmann [Sun, 26 Nov 2023 16:11:15 +0000 (17:11 +0100)]
deduplicate categories/qualifiers

12 months agoorder near searches by distance instead of importance
Sarah Hoffmann [Sun, 26 Nov 2023 15:48:04 +0000 (16:48 +0100)]
order near searches by distance instead of importance

12 months agoavoid duplicate lines during category search
Sarah Hoffmann [Sun, 26 Nov 2023 08:50:59 +0000 (09:50 +0100)]
avoid duplicate lines during category search

13 months agofix polygon selection for classtable lookups
Sarah Hoffmann [Sat, 25 Nov 2023 20:01:27 +0000 (21:01 +0100)]
fix polygon selection for classtable lookups

Polygons should be used preferably with higher address ranks
where the areas are smaller.