git.openstreetmap.org Git - nominatim.git/commit

]> git.openstreetmap.org Git - nominatim.git/commit

author	Sarah Hoffmann <lonvia@denofr.de>
	Tue, 15 Feb 2022 11:15:18 +0000 (12:15 +0100)
committer	Sarah Hoffmann <lonvia@denofr.de>
	Tue, 1 Mar 2022 08:34:32 +0000 (09:34 +0100)
commit	837d44391c00348c4bc2a39d155d1d1473676285
tree	ce19ebb1451a8b89c902e720a75ca0459d1e5da2	tree \| snapshot
parent	691ec0858601f91adb942d2cb5d3a9b844005780	commit \| diff

move generation of normalized token form to analyzer

This gives the analyzer more flexibility in choosing the normalized
form. In particular, an analyzer creating different variants can choose
the variant that will be used as the canonical form.

Open Source search based on OpenStreetMap data

RSS Atom

nominatim/tokenizer/icu_token_analysis.py		diff \| blob \| history
nominatim/tokenizer/icu_tokenizer.py		diff \| blob \| history
nominatim/tokenizer/token_analysis/generic.py		diff \| blob \| history
test/python/tokenizer/token_analysis/test_generic.py		diff \| blob \| history
test/python/tokenizer/token_analysis/test_generic_mutation.py		diff \| blob \| history