]>
git.openstreetmap.org Git - nominatim.git/log
Sarah Hoffmann [Thu, 4 Feb 2021 16:05:40 +0000 (17:05 +0100)]
Merge remote-tracking branch 'upstream/master'
Sarah Hoffmann [Thu, 4 Feb 2021 16:04:04 +0000 (17:04 +0100)]
fix off-by-one error in replication download
Sarah Hoffmann [Thu, 4 Feb 2021 15:52:24 +0000 (16:52 +0100)]
Merge pull request #2161 from lonvia/timeout-for-replication
Reintroduce timeout for replication file download
Sarah Hoffmann [Thu, 4 Feb 2021 10:47:11 +0000 (11:47 +0100)]
reintroduce timeout for replication file download
This ports the --socket-timeout parameter from
pyosmium-get-changes which ensures that the update
process eventually times out on hanging network connections.
Sarah Hoffmann [Thu, 4 Feb 2021 08:52:59 +0000 (09:52 +0100)]
Merge pull request #2160 from lonvia/introduce-project-dir
Officially introduce and recommend use of a project directory
Sarah Hoffmann [Wed, 3 Feb 2021 15:17:46 +0000 (16:17 +0100)]
replace make serve with nominatim serve command
With the website directory now tied to the project directory instead
of the build directory, it is no longer possible to use make for
running the web server.
Sarah Hoffmann [Wed, 3 Feb 2021 13:13:09 +0000 (14:13 +0100)]
add migration info for status table
Sarah Hoffmann [Wed, 3 Feb 2021 09:40:13 +0000 (10:40 +0100)]
Merge remote-tracking branch 'upstream/master'
Sarah Hoffmann [Wed, 3 Feb 2021 09:39:41 +0000 (10:39 +0100)]
reenable code for copying the final tiger data
Sarah Hoffmann [Wed, 3 Feb 2021 09:23:38 +0000 (10:23 +0100)]
Revert "increase splitting for large geometries"
This reverts commit
559fe513fa8ffa3a5bc3a7d07c252a872614c073 .
Increasing the splitting results in geometries where with rounding
issues at the split points, so that contain operations do not
work as expected anymore.
Fixes #2137.
Sarah Hoffmann [Wed, 3 Feb 2021 09:17:22 +0000 (10:17 +0100)]
adapt quick start instructions in README to project dir
Sarah Hoffmann [Tue, 2 Feb 2021 16:33:41 +0000 (17:33 +0100)]
do not overwrite custom set module paths
Given that the module is now copied to the project directory
when no module path is set, we need the information that the
module path is empty. Therefore hand in the default module path
in a separate variable.
Sarah Hoffmann [Tue, 2 Feb 2021 10:19:31 +0000 (11:19 +0100)]
print project directory before running any command
Sarah Hoffmann [Tue, 2 Feb 2021 10:15:56 +0000 (11:15 +0100)]
make phpcs happy
Sarah Hoffmann [Tue, 2 Feb 2021 10:03:09 +0000 (11:03 +0100)]
actions: switch to using separate project dir
Also fixes reverse-only import which not run at all.
Sarah Hoffmann [Tue, 2 Feb 2021 09:51:03 +0000 (10:51 +0100)]
docs: add hint about putting the nominatim tool into the PATH
Sarah Hoffmann [Tue, 2 Feb 2021 09:20:05 +0000 (10:20 +0100)]
copy database module on install
When no explicity database module is configured, then the
module is now copied into the project directory and used from
there. This means that Nominatim can be updated to a new
version of the module while existing installation keep their
version of normalisation.
Sarah Hoffmann [Mon, 1 Feb 2021 16:44:13 +0000 (17:44 +0100)]
hand in absolute path to nominatim tool to php scripts
Sarah Hoffmann [Mon, 1 Feb 2021 16:27:29 +0000 (17:27 +0100)]
refer to new nominatim tool in configuration comments
Sarah Hoffmann [Mon, 1 Feb 2021 16:27:01 +0000 (17:27 +0100)]
change the default location for external data to project dir
Sarah Hoffmann [Tue, 2 Feb 2021 09:37:18 +0000 (10:37 +0100)]
actions: revert to reletive paths for caching
Sarah Hoffmann [Tue, 2 Feb 2021 09:35:30 +0000 (10:35 +0100)]
actions: remove setting custom .env
It only set the pyosmium-get-changes binary which is no longer
needed.
Sarah Hoffmann [Tue, 2 Feb 2021 09:33:15 +0000 (10:33 +0100)]
remove pyosmium-get-changes detection from cmake
pyosmium-get-changes is not longer used.
Sarah Hoffmann [Mon, 1 Feb 2021 10:50:05 +0000 (11:50 +0100)]
Merge pull request #2155 from lonvia/port-regresh-to-python
Port replication and part of the refrsh function to native Python
Sarah Hoffmann [Sat, 30 Jan 2021 19:32:46 +0000 (20:32 +0100)]
print nice summary after updates
Sarah Hoffmann [Sat, 30 Jan 2021 15:42:25 +0000 (16:42 +0100)]
fix linting issues
Sarah Hoffmann [Sat, 30 Jan 2021 15:27:02 +0000 (16:27 +0100)]
disable warning about non-toplevel import
They are needed here so nominatim can be run when osmium
is not installed. Everything except replication will work fine.
Sarah Hoffmann [Sat, 30 Jan 2021 15:20:10 +0000 (16:20 +0100)]
introduce custom UsageError
This is a exception to be thrown when the error occures because
of bad user data. We don't want to print a full stack trace in
these cases but just tell the user what went wrong.
Sarah Hoffmann [Sat, 30 Jan 2021 14:55:04 +0000 (15:55 +0100)]
remove setting for PYOSMIUM_BINARY
pyosmium is now called as a library from the python code,
so that pyosmium-get-changes is no longer needed.
Sarah Hoffmann [Sat, 30 Jan 2021 14:52:49 +0000 (15:52 +0100)]
whitelist pyosmium for pylint
Sarah Hoffmann [Sat, 30 Jan 2021 14:50:34 +0000 (15:50 +0100)]
port replication update function to python
Sarah Hoffmann [Thu, 28 Jan 2021 13:34:17 +0000 (14:34 +0100)]
port check-for-update function to python
Sarah Hoffmann [Thu, 28 Jan 2021 08:06:33 +0000 (09:06 +0100)]
Merge pull request #2153 from rizkyarlin/patch-1
fix indentation
Muh. Rizky Eka Arlin [Thu, 28 Jan 2021 06:21:54 +0000 (14:21 +0800)]
fix indentation
Sarah Hoffmann [Tue, 26 Jan 2021 21:45:24 +0000 (22:45 +0100)]
port replication initialisation to Python
Sarah Hoffmann [Sun, 24 Jan 2021 13:35:35 +0000 (14:35 +0100)]
convert functon creation to python
The new functions always creates normal and partitioned functions.
Also adds specialised connection and cursor classes for adding
frequently used helper functions.
Sarah Hoffmann [Sat, 23 Jan 2021 16:25:14 +0000 (17:25 +0100)]
port address level computation to Python
Also adds simple tests for correct table creation.
Sarah Hoffmann [Fri, 22 Jan 2021 22:25:37 +0000 (23:25 +0100)]
move update code for postcode and word count to Python
Adds also tests for the new function to execute a SQL script.
Sarah Hoffmann [Thu, 21 Jan 2021 09:19:38 +0000 (10:19 +0100)]
cli: import python modules for commands on demand
Given that only one command will be executed in the end, it is
not necessary to import what amounts to the whole library. This
becomes in particular important for update functions that have
a dependency on pyosmium. The dependency can remain optional for
people not using updates.
Sarah Hoffmann [Tue, 26 Jan 2021 21:46:35 +0000 (22:46 +0100)]
remove API comparison util
This is outdated and unmaintained. There are tools out there
which can do this better. Try, for example
https://github.com/radarlabs/api-diff
Sarah Hoffmann [Thu, 21 Jan 2021 09:07:50 +0000 (10:07 +0100)]
Merge pull request #2147 from lonvia/tests-for-python-code
Add basic set of tests for Python code
Sarah Hoffmann [Wed, 20 Jan 2021 19:24:39 +0000 (20:24 +0100)]
cli indexer tests need a fake database
The Indexer constructor opens a connection to the given database.
Sarah Hoffmann [Wed, 20 Jan 2021 15:43:06 +0000 (16:43 +0100)]
add more tests for CLI parameter parser
Sarah Hoffmann [Wed, 20 Jan 2021 14:42:58 +0000 (15:42 +0100)]
add tests for indexer
Sarah Hoffmann [Wed, 20 Jan 2021 08:53:47 +0000 (09:53 +0100)]
add simple tests for CLI parsing
Sarah Hoffmann [Wed, 20 Jan 2021 08:04:04 +0000 (09:04 +0100)]
optionally hand in command line arguments to CLI functions
Allows easier testing.
Sarah Hoffmann [Wed, 20 Jan 2021 08:03:09 +0000 (09:03 +0100)]
use pytest mocking functions for manipulating os.environ
Sarah Hoffmann [Wed, 20 Jan 2021 08:02:45 +0000 (09:02 +0100)]
add tests for Python exec_utils
Sarah Hoffmann [Wed, 20 Jan 2021 20:29:49 +0000 (21:29 +0100)]
Merge pull request #2146 from mtmail/two-typos
correct parameter name in query CLI
marc tobias [Wed, 20 Jan 2021 20:09:41 +0000 (21:09 +0100)]
correct parameter name in query CLI
Sarah Hoffmann [Wed, 20 Jan 2021 08:00:45 +0000 (09:00 +0100)]
Merge pull request #2145 from lonvia/cli-query-functions
Add interface to search via command line tool
Sarah Hoffmann [Tue, 19 Jan 2021 10:05:02 +0000 (11:05 +0100)]
add API functions to nominatim tool
Sarah Hoffmann [Mon, 18 Jan 2021 14:57:32 +0000 (15:57 +0100)]
rename nominatim.admin to nominatim.tools
Sarah Hoffmann [Sun, 17 Jan 2021 21:03:54 +0000 (22:03 +0100)]
probe for php_cgi in cmake to be used for querying
Sarah Hoffmann [Tue, 19 Jan 2021 07:42:22 +0000 (08:42 +0100)]
Merge pull request #2143 from lonvia/integrate-indexer-into-nominatim-tool
Integrate indexer into nominatim tool
Sarah Hoffmann [Sun, 17 Jan 2021 20:02:50 +0000 (21:02 +0100)]
remove old nominatim.py in favour of 'nominatim index'
The PHP scripts need to know the position of the nominatim
tool in order to call it. This is handed in as environment
variable, so it can be set by the Python script.
Sarah Hoffmann [Sun, 17 Jan 2021 19:05:41 +0000 (20:05 +0100)]
implementaion of 'nominatim index'
Sarah Hoffmann [Sun, 17 Jan 2021 16:19:17 +0000 (17:19 +0100)]
move indexing function into its own Python module
This makes it mow a standard function of our new Python
library instead of a stand-alone program.
Sarah Hoffmann [Sun, 17 Jan 2021 16:06:18 +0000 (17:06 +0100)]
add function to get a DSN for psycopg
Converts the PHP DSN syntax into psycopg syntax when necessary.
Sarah Hoffmann [Mon, 18 Jan 2021 14:40:50 +0000 (15:40 +0100)]
Merge pull request #2142 from lonvia/update-bdd-api-tests
Update BDD API tests
Sarah Hoffmann [Sat, 16 Jan 2021 21:20:23 +0000 (22:20 +0100)]
bdd: complete coverage for API tests
Also removes some functions that are no longer used and
fixes debug output where the tests found an issue.
Sarah Hoffmann [Sat, 16 Jan 2021 16:39:49 +0000 (17:39 +0100)]
remove unused output formatting functions
Sarah Hoffmann [Sat, 16 Jan 2021 15:57:28 +0000 (16:57 +0100)]
bdd: remove duplicated tests
Sarah Hoffmann [Sat, 16 Jan 2021 11:04:13 +0000 (12:04 +0100)]
bdd: clean up and extend API tests for details
- remove duplicates created by replacing HTML tests
with JSON tests
- add tests for newer functions for returning geometries
and hierarchies
Sarah Hoffmann [Fri, 15 Jan 2021 19:27:14 +0000 (20:27 +0100)]
collect coverage for /website directory as well
Sarah Hoffmann [Fri, 15 Jan 2021 16:37:36 +0000 (17:37 +0100)]
Merge pull request #2139 from lonvia/add-pytest
Introduce unit testing for Python code
Sarah Hoffmann [Fri, 15 Jan 2021 15:45:37 +0000 (16:45 +0100)]
pytest may also be installed as py-test[-3]
Sarah Hoffmann [Fri, 15 Jan 2021 14:22:21 +0000 (15:22 +0100)]
enable pytest testing in CI
Sarah Hoffmann [Fri, 15 Jan 2021 14:18:45 +0000 (15:18 +0100)]
add documentation for new pytest tests
Sarah Hoffmann [Fri, 15 Jan 2021 14:09:36 +0000 (15:09 +0100)]
add pytest as a test goal in cmake
Sarah Hoffmann [Fri, 15 Jan 2021 13:42:03 +0000 (14:42 +0100)]
add initial pytest test for Configuration
Sarah Hoffmann [Fri, 15 Jan 2021 13:39:26 +0000 (14:39 +0100)]
Merge pull request #2136 from lonvia/introduce-pylint
Introduce pylint for code style checking for Python.
Sarah Hoffmann [Fri, 15 Jan 2021 09:52:25 +0000 (10:52 +0100)]
pylint may be available as pylint3 or pylint
Sarah Hoffmann [Fri, 15 Jan 2021 09:38:28 +0000 (10:38 +0100)]
use check parameter of subprocess.run
...instead of checking on our own.
Also increase required version of Python to 3.5 because of
subprocess.run().
Sarah Hoffmann [Fri, 15 Jan 2021 08:04:04 +0000 (09:04 +0100)]
add pylint to list of required linting tools
With pylint being run in the CI, passing it is required now.
Sarah Hoffmann [Fri, 15 Jan 2021 08:10:01 +0000 (09:10 +0100)]
Merge remote-tracking branch 'upstream/master'
Sarah Hoffmann [Thu, 14 Jan 2021 20:36:31 +0000 (21:36 +0100)]
fix errors reported by pylint
Sarah Hoffmann [Fri, 15 Jan 2021 07:56:07 +0000 (08:56 +0100)]
Merge pull request #2135 from lonvia/python-frontend
Introduce new 'nominatim' all-in-one command-line tool
Sarah Hoffmann [Thu, 14 Jan 2021 19:30:27 +0000 (20:30 +0100)]
fix typo
Sarah Hoffmann [Thu, 14 Jan 2021 17:31:18 +0000 (18:31 +0100)]
replace Symfony dotenv dependency with Python dotenv
Sarah Hoffmann [Thu, 14 Jan 2021 17:19:22 +0000 (18:19 +0100)]
bdd: directly call utility scripts in lib
This removes the dependency on php-symfony-dotenv for the tests.
Sarah Hoffmann [Thu, 14 Jan 2021 16:09:22 +0000 (17:09 +0100)]
move dotenv parsing to installed PHP scripts
This means that the php-symfony-dotenv library is now only needed
when using the legacy scripts. This includes the BDD tests which
currently still rely on the PHP utils.
Sarah Hoffmann [Thu, 14 Jan 2021 13:37:52 +0000 (14:37 +0100)]
use cli tool for github CI
Sarah Hoffmann [Thu, 14 Jan 2021 13:36:41 +0000 (14:36 +0100)]
forward cli tool return value as exit code
Sarah Hoffmann [Thu, 14 Jan 2021 11:04:08 +0000 (12:04 +0100)]
adapt documentation to new nominatim cli tool
Sarah Hoffmann [Thu, 14 Jan 2021 11:03:49 +0000 (12:03 +0100)]
maller command execution fixes
Sarah Hoffmann [Thu, 14 Jan 2021 10:12:45 +0000 (11:12 +0100)]
make sure that environment variables have highest prio
Sarah Hoffmann [Thu, 14 Jan 2021 09:19:21 +0000 (10:19 +0100)]
liniting of new python code
Sarah Hoffmann [Thu, 14 Jan 2021 08:37:47 +0000 (09:37 +0100)]
add wrapper calls for all nominatim tool functions
Sarah Hoffmann [Thu, 14 Jan 2021 08:30:23 +0000 (09:30 +0100)]
avoid accessing constants when getting data from env
When a setting can be read from the environment variable, avoid
accessing the internal defaults. This way the scripts can be
accessed directly in \lib as long as the environment is set up
correctly with full defaults.
Sarah Hoffmann [Thu, 14 Jan 2021 08:29:43 +0000 (09:29 +0100)]
fix access to environment variable
Sarah Hoffmann [Wed, 13 Jan 2021 17:25:15 +0000 (18:25 +0100)]
implement warming in new cli tool
Adds infrastructure for calling the legacy PHP scripts. As the
CONST_* values cannot be set from the python script, hand the values
in via secret environment variables instead. These are all
temporary hacks for the transition phase to python code.
Sarah Hoffmann [Wed, 13 Jan 2021 17:12:53 +0000 (18:12 +0100)]
warm.php needs constant setup for queries
Warming is done using the query classes and therefore the same
copy-over from dotenv settings to CONST_* parameters is needed
as for query.php.
Sarah Hoffmann [Wed, 13 Jan 2021 16:00:38 +0000 (17:00 +0100)]
set CONST_LibDir directly from the source scripts
Now that the source scripts have been moved to \lib, they
can determine the position of the PHP library relative to
themselves.
Sarah Hoffmann [Wed, 13 Jan 2021 09:51:44 +0000 (10:51 +0100)]
move PHP utilities into the lib directory
These are not called directly as programs but used in a library
fashion by the installed utilities. So the library directory
is a better place.
Sarah Hoffmann [Wed, 13 Jan 2021 09:11:58 +0000 (10:11 +0100)]
consolidate cli interface to single tool
Sarah Hoffmann [Tue, 12 Jan 2021 21:21:20 +0000 (22:21 +0100)]
create skeleton for new CLI tools
Sarah Hoffmann [Tue, 12 Jan 2021 09:17:28 +0000 (10:17 +0100)]
add skeleton for new Nominatim executables
Sarah Hoffmann [Mon, 11 Jan 2021 09:42:22 +0000 (10:42 +0100)]
Merge pull request #2132 from lonvia/reduce-api-testdb
Reduce BDD API test database to Liechtenstein
Sarah Hoffmann [Sat, 9 Jan 2021 16:58:06 +0000 (17:58 +0100)]
also run BDD API tests in CI