]>
git.openstreetmap.org Git - nominatim.git/log
Darkshredder [Mon, 19 Apr 2021 10:23:01 +0000 (15:53 +0530)]
Fix: tiger-data tarfile test
Sarah Hoffmann [Mon, 19 Apr 2021 10:14:25 +0000 (12:14 +0200)]
Merge pull request #2282 from lonvia/add-paths-to-config
Include software paths in Python config object
Sarah Hoffmann [Mon, 19 Apr 2021 08:01:17 +0000 (10:01 +0200)]
simplify sql and website creation functions
Sarah Hoffmann [Mon, 19 Apr 2021 07:38:17 +0000 (09:38 +0200)]
simplify constructor for SQL preprocessor
Use sql path from config.
Sarah Hoffmann [Mon, 19 Apr 2021 07:23:37 +0000 (09:23 +0200)]
simplify interface for adding tiger data
Also simplifies tests using existing fixtures.
Sarah Hoffmann [Mon, 19 Apr 2021 07:06:42 +0000 (09:06 +0200)]
add library directories to config
Allows to reduce the number of parameters in functions that take
the config anyway.
Sarah Hoffmann [Mon, 19 Apr 2021 06:42:59 +0000 (08:42 +0200)]
Merge pull request #2281 from changpingc/changping/fix-tiger-index
fix index on location_property_tiger (parent_place_id)
Channgping Chen [Mon, 19 Apr 2021 00:01:01 +0000 (00:01 +0000)]
fix index on location_property_tiger (parent_place_id)
Looks like
2af82975cd968ec09683ae5b16a9aa157a7f2176
accidentally renamed an index. Because of the added "if not
exists" clause, the index doesn't get created. This
significantly slows down reverse queries because they now
require full scans on location_property_tiger.
Without this fix, reverse queries can take 8s on a full
planet install on an r5.8xlarge instance in EC2.
Sarah Hoffmann [Sun, 18 Apr 2021 09:57:19 +0000 (11:57 +0200)]
Merge pull request #2280 from AntoJvlt/Fix-special-phrases-import-and-tests-cleaning
Fix regex and sanity check for the import of special phrases and tests cleaning.
AntoJvlt [Sat, 17 Apr 2021 17:45:24 +0000 (19:45 +0200)]
Only log a warning if a wrong input is detected on the wiki while importing special phrases
AntoJvlt [Sat, 17 Apr 2021 17:24:13 +0000 (19:24 +0200)]
Fix occurence regex
AntoJvlt [Sat, 17 Apr 2021 17:23:33 +0000 (19:23 +0200)]
Cleaned tests and add database cleaning tests on test_import_from_wiki
Sarah Hoffmann [Sat, 17 Apr 2021 09:51:21 +0000 (11:51 +0200)]
Merge pull request #2279 from lonvia/add-index-for-continued-indexing
Add index for continued indexing
Sarah Hoffmann [Sat, 17 Apr 2021 09:10:36 +0000 (11:10 +0200)]
add tests for continuing import
Sarah Hoffmann [Sat, 17 Apr 2021 09:07:04 +0000 (11:07 +0200)]
add support index when continuing import at index phase
Indexing scans the placex table sequentially during indexing
on the initial import. That is okay because we know that all
rows need to be processed anywhere. When continuing the import,
however, a large part might already be indexed, so that the
process spends a lot of time going through rows that are no
longer of interest. Create a supporting index for all unindexed
rows to speed up the scan. This is the same index as used later
for updates.
Sarah Hoffmann [Sat, 17 Apr 2021 08:13:33 +0000 (10:13 +0200)]
Merge pull request #2278 from lonvia/remove-transistion-functions
Remove transition functions
Sarah Hoffmann [Fri, 16 Apr 2021 16:41:14 +0000 (18:41 +0200)]
remove transition functions from Python
Sarah Hoffmann [Fri, 16 Apr 2021 15:40:43 +0000 (17:40 +0200)]
Merge pull request #2277 from lonvia/update-osm2pgsql
Update osm2pgsql to current master
Sarah Hoffmann [Fri, 16 Apr 2021 15:28:51 +0000 (17:28 +0200)]
remove PHP code for transition functions
Sarah Hoffmann [Fri, 16 Apr 2021 15:09:40 +0000 (17:09 +0200)]
remove installation of PHP util scripts
Sarah Hoffmann [Fri, 16 Apr 2021 14:57:04 +0000 (16:57 +0200)]
Merge pull request #2276 from lonvia/port-country-code-creation-to-python
Port country code creation to python
Sarah Hoffmann [Fri, 16 Apr 2021 13:37:53 +0000 (15:37 +0200)]
add test for new postcode import function
Sarah Hoffmann [Fri, 16 Apr 2021 13:05:40 +0000 (15:05 +0200)]
port function to compute initial postcodes to Python
Sarah Hoffmann [Fri, 16 Apr 2021 13:04:10 +0000 (15:04 +0200)]
Merge pull request #2275 from lonvia/switch-to-absolute-imports
Use absolute imports in Python code
Sarah Hoffmann [Fri, 16 Apr 2021 12:20:09 +0000 (14:20 +0200)]
use absolute imports in Python code
Relative imports are no longer officially recommended.
Sarah Hoffmann [Thu, 15 Apr 2021 08:24:01 +0000 (10:24 +0200)]
update osm2pgsql to current master (fixes version output)
Sarah Hoffmann [Thu, 15 Apr 2021 08:13:25 +0000 (10:13 +0200)]
Merge pull request #2263 from AntoJvlt/special-phrases-autoupdate
Implemented auto update of special phrases while importing them
Sarah Hoffmann [Thu, 15 Apr 2021 08:12:53 +0000 (10:12 +0200)]
Merge pull request #2270 from lonvia/simplify-place-boundary-merge
Simplify matching between place and boundary names
Sarah Hoffmann [Wed, 14 Apr 2021 07:58:14 +0000 (09:58 +0200)]
adapt database check to new index layout
Sarah Hoffmann [Fri, 9 Apr 2021 19:24:35 +0000 (21:24 +0200)]
add migration for new placenode geometry index
Sarah Hoffmann [Fri, 9 Apr 2021 19:10:00 +0000 (21:10 +0200)]
simplify name matching between boundary and place node
Instead of normalising the names simply compare them in lower
case. This removes the dependency on the tokenizer for
linking boundaries and nodes. When looking up the linked places
by place type also allow that one name is simply contained in the
other. This catches the frequent case where one of the names has
an addendum (e.g. Newport vs. City of Newport).
Drops the special index for the name lookup and insted relies
on a slightly extended version of the geometry index used for
reverse lookup. Saves around 100MB on a planet.
Sarah Hoffmann [Wed, 14 Apr 2021 15:50:02 +0000 (17:50 +0200)]
Merge pull request #2269 from lonvia/fix-actions
github actions: reintroduce postgresql repo
Sarah Hoffmann [Wed, 14 Apr 2021 14:19:49 +0000 (16:19 +0200)]
github actions: reintroduce postgresql repo
Sarah Hoffmann [Wed, 14 Apr 2021 08:56:12 +0000 (10:56 +0200)]
Merge pull request #2264 from darkshredder/tiger-data-tests
Fix: Error if last statements is wrong and improved tests in tiger data import
Darkshredder [Tue, 13 Apr 2021 09:36:02 +0000 (15:06 +0530)]
Fix: Removed error if endstatement is wrong and improved tests
AntoJvlt [Mon, 12 Apr 2021 12:10:30 +0000 (14:10 +0200)]
Tests added for the auto update of special phrases during import
AntoJvlt [Mon, 12 Apr 2021 09:55:18 +0000 (11:55 +0200)]
Implemented auto update of special phrases while importing them
Sarah Hoffmann [Sun, 11 Apr 2021 21:09:45 +0000 (23:09 +0200)]
Merge pull request #2260 from AntoJvlt/fix-load-languages-special-phrases
Fix default languages loading for special phrases import
AntoJvlt [Sun, 11 Apr 2021 20:26:31 +0000 (22:26 +0200)]
Fix default languages loading
Sarah Hoffmann [Sat, 10 Apr 2021 19:19:55 +0000 (21:19 +0200)]
Merge pull request #2258 from darkshredder/code-coverage
Disabled Code coverage status checks
Darkshredder [Sat, 10 Apr 2021 16:58:29 +0000 (22:28 +0530)]
CodeCov comment only when codecoverage changes
Darkshredder [Sat, 10 Apr 2021 15:14:52 +0000 (20:44 +0530)]
Disabled Coverage status checks
Sarah Hoffmann [Sat, 10 Apr 2021 14:57:39 +0000 (16:57 +0200)]
add badge for codecov
Sarah Hoffmann [Sat, 10 Apr 2021 14:37:12 +0000 (16:37 +0200)]
Merge pull request #2252 from darkshredder/code-coverage
Added Code coverage support using Codecov
Sarah Hoffmann [Fri, 9 Apr 2021 15:48:28 +0000 (17:48 +0200)]
split LANGUAGES parameter before use
The user supplies the languages as a comma-separated list.
Sarah Hoffmann [Thu, 8 Apr 2021 09:01:19 +0000 (11:01 +0200)]
add migration information for new configuration format
Sarah Hoffmann [Thu, 8 Apr 2021 08:54:16 +0000 (10:54 +0200)]
Merge pull request #2256 from lonvia/remove-reverseinplan-option
Remove ReverseInPlan option
Sarah Hoffmann [Thu, 8 Apr 2021 08:35:14 +0000 (10:35 +0200)]
remove special handling for reversed queries in getGroupedSearches
getGroupedSearches is guaranteed not to be called with reversed
structured queries, so there is no need to have special exclusion
code.
Sarah Hoffmann [Thu, 8 Apr 2021 08:19:27 +0000 (10:19 +0200)]
remove reverseInPlan option from Geocode
Disabling query reversal is no longer possible in the configuration,
so there is no need to keep this as an option. Reversal is
automatically disabled for structured search only.
Sarah Hoffmann [Tue, 6 Apr 2021 19:23:29 +0000 (21:23 +0200)]
prepare 3.7.0 release
Sarah Hoffmann [Tue, 6 Apr 2021 14:09:53 +0000 (16:09 +0200)]
docs: minor spelling corrections
Sarah Hoffmann [Tue, 6 Apr 2021 13:56:08 +0000 (15:56 +0200)]
docs: unpacking tiger data is no longer necessary
Sarah Hoffmann [Mon, 5 Apr 2021 13:48:22 +0000 (15:48 +0200)]
Merge pull request #2250 from lonvia/save-transliterated-housenumbers
Switch to saving transliterated housenumbers in placex
Darkshredder [Mon, 5 Apr 2021 01:00:31 +0000 (06:30 +0530)]
Fixed BDD tests coverage reports
Sarah Hoffmann [Fri, 2 Apr 2021 15:28:52 +0000 (17:28 +0200)]
add migration for transliterated housenumbers
Sarah Hoffmann [Thu, 1 Apr 2021 16:52:06 +0000 (18:52 +0200)]
use new transliteration in initial housenumber word computation
The new create_housenumber_id() function splits housenumber
lists correctly. Otherwise there is no difference.
Sarah Hoffmann [Mon, 29 Mar 2021 15:36:44 +0000 (17:36 +0200)]
correctly handle housenumber lists
Lists are now standardised to use a semicolon separator.
Sarah Hoffmann [Mon, 29 Mar 2021 14:45:09 +0000 (16:45 +0200)]
move transliteration of housenumbers into indexing
Housenumbers are now saved in transliterated form in the housenumber
column. This saves the transliteration step during lookup.
Sarah Hoffmann [Mon, 29 Mar 2021 14:36:45 +0000 (16:36 +0200)]
tests for finding non-ascii housenumbers
Sarah Hoffmann [Mon, 29 Mar 2021 07:36:56 +0000 (09:36 +0200)]
return housenumbers always from address field
This means that we can use normalized versions of the
housenumber in the housenumber field as it is no longer
a user visible field.
Sarah Hoffmann [Sat, 3 Apr 2021 16:31:01 +0000 (18:31 +0200)]
Merge pull request #2248 from darkshredder/special-term-test
Added Test for TokenSpecialTerm
Darkshredder [Thu, 1 Apr 2021 23:19:05 +0000 (04:49 +0530)]
Added Test for TokenSpecialTerm
Sarah Hoffmann [Thu, 1 Apr 2021 16:35:00 +0000 (18:35 +0200)]
Merge pull request #2247 from lonvia/index-for-housenumber-lookup
Index for housenumber lookup
Sarah Hoffmann [Thu, 1 Apr 2021 12:37:20 +0000 (14:37 +0200)]
add recommendation for Postgresql 11+
Sarah Hoffmann [Thu, 1 Apr 2021 12:29:34 +0000 (14:29 +0200)]
use non-key index to speed up housenumber search
On Postgresql versions 11+ add an index to speed up the lookup
of housenumbers for terms found in search_name. This is really
just a band-aid around the query planer's interpretation of the
query.
Darkshredder [Tue, 30 Mar 2021 23:30:03 +0000 (05:00 +0530)]
Added code-cov Support for Code Coverage
Sarah Hoffmann [Tue, 30 Mar 2021 07:19:01 +0000 (09:19 +0200)]
Merge pull request #2243 from darkshredder/XML-format-fix
Fixed: XML format: more_url points to localhost, not base URL
Sarah Hoffmann [Tue, 30 Mar 2021 07:17:27 +0000 (09:17 +0200)]
Merge pull request #2244 from AntoJvlt/import-special-phrases-tests-cleaning
Cleaned tests for special phrases.
Darkshredder [Mon, 29 Mar 2021 21:32:55 +0000 (03:02 +0530)]
Added HTTP_HOST to if statement
AntoJvlt [Mon, 29 Mar 2021 20:07:29 +0000 (22:07 +0200)]
Cleaned tests of exceptions and fix phrase_settings.json test file name.
Darkshredder [Mon, 29 Mar 2021 19:32:43 +0000 (01:02 +0530)]
fixed: XML format: more_url points to localhost, not base URL
Sarah Hoffmann [Mon, 29 Mar 2021 12:25:10 +0000 (14:25 +0200)]
Merge pull request #2234 from darkshredder/add-man-page
Added Manual page for Nominatim tool
Sarah Hoffmann [Mon, 29 Mar 2021 10:06:51 +0000 (12:06 +0200)]
allow sorting by housenumbers for rare street names
Usually we don't narrow down search results by house number when
only a street name is given because there may be a lot of rows
to cross check when the street name is very frequent. However,
when it is known to be rare, the housenumber check may be done
anyway.
Fixes #2238.
Darkshredder [Mon, 29 Mar 2021 08:57:06 +0000 (14:27 +0530)]
Update manpage after rebase
Darkshredder [Mon, 29 Mar 2021 08:46:41 +0000 (14:16 +0530)]
Nominatim/cli.py rebase fixes
Darkshredder [Mon, 29 Mar 2021 08:30:45 +0000 (14:00 +0530)]
Rebase with master
Darkshredder [Wed, 24 Mar 2021 15:08:14 +0000 (20:38 +0530)]
fixed indentation and used sed to remove AUTHORS section
Darkshredder [Tue, 23 Mar 2021 02:40:53 +0000 (08:10 +0530)]
Added Manual page and fixed documentation
Sarah Hoffmann [Mon, 29 Mar 2021 07:49:35 +0000 (09:49 +0200)]
Merge pull request #2228 from AntoJvlt/import-special-phrases-porting-python
Import special phrases porting python
AntoJvlt [Fri, 26 Mar 2021 00:57:51 +0000 (01:57 +0100)]
Change command 'import-special-phrases --from-wiki' to 'special-phrases --import-from-wiki'.
AntoJvlt [Fri, 26 Mar 2021 00:53:33 +0000 (01:53 +0100)]
Errors fixes, Cleaning code, Improvement and addition of tests
AntoJvlt [Wed, 24 Mar 2021 13:26:52 +0000 (14:26 +0100)]
Encapsulation of tools/special_phrases.py into SpecialPhrasesImporter class and add new tests.
AntoJvlt [Mon, 22 Mar 2021 22:56:24 +0000 (23:56 +0100)]
Code cleaning, tests simplification and use of python3-icu package
AntoJvlt [Mon, 22 Mar 2021 00:46:46 +0000 (01:46 +0100)]
Updated documentation for PyICU support
AntoJvlt [Sun, 21 Mar 2021 23:58:49 +0000 (00:58 +0100)]
Introduction of PyICU for transliteration in python. Reversed changes in normalization.sql.
AntoJvlt [Sun, 21 Mar 2021 23:16:39 +0000 (00:16 +0100)]
Added wrapper in specialphrases.php to call corresponding nominatim command.
AntoJvlt [Sun, 21 Mar 2021 23:07:55 +0000 (00:07 +0100)]
Changed phrase_settings.py to phrase-settings.json and added migration function for old php settings file.
AntoJvlt [Sun, 21 Mar 2021 23:07:55 +0000 (00:07 +0100)]
Changed phrase_settings.py to phrase-settings.json and added migration function for old php settings file.
Sarah Hoffmann [Tue, 23 Mar 2021 08:18:10 +0000 (09:18 +0100)]
Merge pull request #2233 from lonvia/index-for-postcode-ids
Create postcode id index earlier
Sarah Hoffmann [Mon, 22 Mar 2021 21:24:56 +0000 (22:24 +0100)]
create postcode id index earlier
Now that the indexer takes care of indexing the postcode tables,
the id index is needed to find the rows to index.
Sarah Hoffmann [Sun, 21 Mar 2021 15:52:20 +0000 (16:52 +0100)]
Merge pull request #2231 from mtmail/correct-cli-help-page
nominatim -h was printing wrong text for lookup and details
Sarah Hoffmann [Sun, 21 Mar 2021 15:47:22 +0000 (16:47 +0100)]
avoid division by zero in progress meter
On Windows systems the timer may not be accurate enough to measure
the time between init() and done(). Avoid computing statistics with
a diff time of 0 in such cases.
Fixes #2230.
marc tobias [Sun, 21 Mar 2021 15:06:41 +0000 (16:06 +0100)]
nominatim -h was priting wrong text for lookup and details
AntoJvlt [Sat, 20 Mar 2021 18:48:05 +0000 (19:48 +0100)]
Deleted specialphrases.php and phrase_settings.php
AntoJvlt [Sat, 20 Mar 2021 18:17:03 +0000 (19:17 +0100)]
Updated specialphrases_testdb.sql
AntoJvlt [Sat, 20 Mar 2021 17:55:08 +0000 (18:55 +0100)]
Ported functions for the import of special phrases from php to python.
- the command is now --import-special-phrases
- the output is not an sql file anymore, data are directly imported to the database.
- the little part on the documentation (section data import) has been modified.
Sarah Hoffmann [Sat, 20 Mar 2021 09:39:29 +0000 (10:39 +0100)]
bdd tests: make indexing less verbose
Do not print progress info for indexing when there is an error
in the BDD tests.
Sarah Hoffmann [Wed, 17 Mar 2021 10:40:34 +0000 (11:40 +0100)]
Merge pull request #2219 from lonvia/bdd-test-remove-php
BDD tests: run all setup via nominatim Python library
Sarah Hoffmann [Tue, 16 Mar 2021 21:13:33 +0000 (22:13 +0100)]
bdd: run all setup via nominatim Python library
Drops all calls to PHP utility functions. nominatim cli functions
are used where possible, to stay as close to the final code as
possible with the tests.
By removing the PHP calls, the test code now only uses osm2pgsql and
the database module from the build directory.
Sarah Hoffmann [Mon, 15 Mar 2021 13:08:54 +0000 (14:08 +0100)]
Merge pull request #2216 from lonvia/fix-reverse-interpolation
Reverse: do not prefer interpolations over closer housenumbers