]> git.openstreetmap.org Git - nominatim.git/commit
projects / nominatim.git / commit
? search:
summary | shortlog | log | commit | commitdiff | tree
(parent: 52847b6) | patch
make token analyzers configurable modules
authorSarah Hoffmann <lonvia@denofr.de>
Mon, 4 Oct 2021 15:34:30 +0000 (17:34 +0200)
committerSarah Hoffmann <lonvia@denofr.de>
Mon, 4 Oct 2021 15:37:34 +0000 (17:37 +0200)
commit7cfcbacfc75ab2e39ee7eab6a5cf40e8cbd152f5
treec3c5523fb98bd2d11b1259aa3d905dcd4221db9atree | snapshot
parent52847b61a3e1bc0791dd23809dc3c50fe6810df2commit | diff
make token analyzers configurable modules

Adds a mandatory section 'analyzer' to the token-analysis entries
which define, which analyser to use. Currently there is exactly
one, generic, which implements the former ICUNameProcessor.
nominatim/tokenizer/icu_rule_loader.py diff | blob | history
nominatim/tokenizer/token_analysis/__init__.py [new file with mode: 0644] blob
nominatim/tokenizer/token_analysis/generic.py [moved from nominatim/tokenizer/icu_name_processor.py with 92% similarity] diff | blob | history
settings/icu_tokenizer.yaml diff | blob | history
test/python/test_tokenizer_icu.py diff | blob | history
test/python/test_tokenizer_icu_rule_loader.py diff | blob | history
test/python/tokenizer/token_analysis/test_generic.py [moved from test/python/test_tokenizer_icu_name_processor.py with 97% similarity] diff | blob | history
Open Source search based on OpenStreetMap data