|

String tokenizer

A generic byte string tokenizer. More...

Classes
struct	Ionflux::Tools::TokenType
	Token type. More...
struct	Ionflux::Tools::Token
	Token. More...
class	Ionflux::Tools::Tokenizer
	Generic byte string tokenizer. More...
class	Ionflux::Tools::TokenTypeMap
	Token type lookup map. More...
class	Ionflux::Tools::CharTypeMap
	Character type lookup map. More...
struct	Ionflux::Tools::Utf8Token
	A token for the UTF-8 tokenizer. More...
class	Ionflux::Tools::Utf8Tokenizer
	Tokenizer with UTF-8 support. More...
struct	Ionflux::Tools::Utf8TokenType
	A token type with UTF-8 support. More...
class	Ionflux::Tools::Utf8TokenTypeMap
	Token type map with UTF-8 support. More...

---

Detailed Description

A generic byte string tokenizer.

---