]> git.openstreetmap.org Git - nominatim.git/commit
projects / nominatim.git / commit
? search:
summary | shortlog | log | commit | commitdiff | tree
(parent: 5c7b9ef) | patch
introduce tokenizer modules
authorSarah Hoffmann <lonvia@denofr.de>
Wed, 21 Apr 2021 07:57:17 +0000 (09:57 +0200)
committerSarah Hoffmann <lonvia@denofr.de>
Fri, 30 Apr 2021 09:29:57 +0000 (11:29 +0200)
commitaf968d49039f5cd8f0ef5839558a57867d7ef111
treea144d1d4b70f1ed212f0c3adfdeefd5771792cbatree | snapshot
parent5c7b9ef909b3c30bbe640f4373a1127e15a7e018commit | diff
introduce tokenizer modules

This adds the boilerplate for selecting configurable tokenizers.
A tokenizer can be chosen at import time and will then install
itself such that it is fixed for the given database import even
when the software itself is updated.

The legacy tokenizer implements Nominatim's traditional algorithms.
nominatim/clicmd/setup.py diff | blob | history
nominatim/tokenizer/__init__.py [new file with mode: 0644] blob
nominatim/tokenizer/factory.py [new file with mode: 0644] blob
nominatim/tokenizer/legacy_tokenizer.py [new file with mode: 0644] blob
settings/env.defaults diff | blob | history
test/python/conftest.py diff | blob | history
test/python/dummy_tokenizer.py [new file with mode: 0644] blob
test/python/test_cli.py diff | blob | history
test/python/test_tokenizer_factory.py [new file with mode: 0644] blob
test/python/test_tokenizer_legacy.py [new file with mode: 0644] blob
Open Source search based on OpenStreetMap data