|

Ionflux::Tools::Utf8Token Struct Reference
[String tokenizer]

A token for the UTF-8 tokenizer. More...

#include <Utf8Tokenizer.hpp>

List of all members.

Public Attributes
int	typeID
	Numeric token type ID.
std::string	value
	UTF-8 encoded value of the token.

---

Detailed Description

A token for the UTF-8 tokenizer.

---

Member Data Documentation

int Ionflux::Tools::Utf8Token::typeID

Numeric token type ID.

std::string Ionflux::Tools::Utf8Token::value

UTF-8 encoded value of the token.

---

The documentation for this struct was generated from the following file:

• Utf8Tokenizer.hpp

---