UTF8StringTokenizer Class Reference

UTF8 string tokenizer class. More...

#include <minitscript/utilities/UTF8StringTokenizer.h>

Collaboration diagram for UTF8StringTokenizer:

Public Member Functions
	UTF8StringTokenizer ()
	Public constructor. More...
void	tokenize (const string &str, const string &delimiters, bool emptyTokens=false)
	Tokenize. More...
int32_t	countTokens ()
bool	hasMoreTokens ()
const string &	nextToken ()
const vector< string > &	getTokens ()

Private Attributes
vector< string >	tokens
int	idx { 0 }

Detailed Description

UTF8 string tokenizer class.

Author

Andreas Drewke

Definition at line 16 of file UTF8StringTokenizer.h.

Constructor & Destructor Documentation

UTF8StringTokenizer()

UTF8StringTokenizer ( )

inline

Public constructor.

Definition at line 27 of file UTF8StringTokenizer.h.

Member Function Documentation

tokenize()

void tokenize	(	const string &	str,
		const string &	delimiters,
		bool	emptyTokens = false
	)

Tokenize.

Parameters

str	string to tokenize
delimiters	delimiters
emptyTokens	include empty tokens

Definition at line 15 of file UTF8StringTokenizer.cpp.

countTokens()

int32_t countTokens ( )

inline

Returns

number of tokens

Definition at line 40 of file UTF8StringTokenizer.h.

hasMoreTokens()

bool hasMoreTokens ( )

inline

Returns

has more tokens

Definition at line 47 of file UTF8StringTokenizer.h.

nextToken()

const string& nextToken ( )

inline

Returns

next token

Definition at line 54 of file UTF8StringTokenizer.h.

getTokens()

const vector<string>& getTokens ( )

inline

Returns

tokens

Definition at line 61 of file UTF8StringTokenizer.h.

Member Data Documentation

tokens

vector<string> tokens

private

Definition at line 20 of file UTF8StringTokenizer.h.

idx

int idx { 0 }

private

Definition at line 21 of file UTF8StringTokenizer.h.

---

The documentation for this class was generated from the following files:

• /home/andreas/Development/drewke.net/minitscript/src/minitscript/utilities/UTF8StringTokenizer.h
• /home/andreas/Development/drewke.net/minitscript/src/minitscript/utilities/UTF8StringTokenizer.cpp