7

Does anyone here have experience with writing custom FTS3 (the full-text-search extension) tokenizers? I'm looking for a tokenizer that will ignore HTML tags.

Thanks.

noamtm
  • 12,435
  • 15
  • 71
  • 107

1 Answers1

4

I have no direct experience, but by doing a web search with "sqlite3 registerTokenizer" I found two tokenizers that can be used as a basis: a snowball tokenizer and a MeCab tokenizer.

hmuelner
  • 8,093
  • 1
  • 28
  • 39