]> git.openstreetmap.org Git - nominatim.git/commit
move word table and normalisation SQL into tokenizer
authorSarah Hoffmann <lonvia@denofr.de>
Thu, 22 Apr 2021 20:47:34 +0000 (22:47 +0200)
committerSarah Hoffmann <lonvia@denofr.de>
Fri, 30 Apr 2021 09:30:51 +0000 (11:30 +0200)
commitfbbdd31399da42b94188d9d4aa4f084efd4876a4
tree0f8eb8578b482fe41d8244482b48ccdd6de2411e
parentb5540dc35c35c7fa8f01979e972ca429b0b521fb
move word table and normalisation SQL into tokenizer

Creating and populating the word table is now the responsibility
of the tokenizer.

The get_maxwordfreq() function has been replaced with a
simple template parameter to the SQL during function installation.
The number is taken from the parameter list in the database to
ensure that it is not changed after installation.
15 files changed:
lib-sql/functions.sql
lib-sql/tables.sql
lib-sql/tokenizer/legacy_tokenizer.sql [moved from lib-sql/functions/normalization.sql with 99% similarity]
lib-sql/tokenizer/legacy_tokenizer_tables.sql [new file with mode: 0644]
nominatim/clicmd/refresh.py
nominatim/clicmd/setup.py
nominatim/db/sql_preprocessor.py
nominatim/tokenizer/legacy_tokenizer.py
nominatim/tools/database_import.py
nominatim/tools/migration.py
test/python/conftest.py
test/python/test_cli.py
test/python/test_db_sql_preprocessor.py
test/python/test_tokenizer_legacy.py
test/python/test_tools_database_import.py