Zend_Search_Lucene_Analysis_Analyzer::tokenize
Tokenize text to a terms Returns array of Zend_Search_Lucene_Analysis_Token objects
Tokens are returned in UTF-8 (internal Zend_Search_Lucene encoding)
Signature
public function tokenize(string
$data,
[string
$encoding = ''] )
Parameters
$data
— string$encoding
— string
Returns
- array