## wxStringTokenizer

wxStringTokenizer helps you to break a string up into a number of tokens. It replaces the standard C function `strtok()` and also extends it in a number of ways.

To use this class, you should create a wxStringTokenizer object, give it the string to tokenize and also the delimiters which separate tokens in the string (by default, white space characters will be used).

Then GetNextToken may be called repeatedly until it HasMoreTokens returns `false`.

For example:

    wxStringTokenizer tkz(wxT("first:second:third:fourth"), wxT(":"));
    while ( tkz.HasMoreTokens() )
    {
        wxString token = tkz.GetNextToken();

        // process token here
    }

By default, wxStringTokenizer will behave in the same way as `strtok()` if the delimiters string only contains white space characters but, unlike the standard function, it will return empty tokens if this is not the case. This is helpful for parsing strictly formatted data where the number of fields is fixed but some of them may be empty (i.e. `TAB` or comma delimited text files).

The behaviour is governed by the last constructor/SetString parameter `mode` which may be one of the following:

`wxTOKEN_DEFAULT` | Default behaviour (as described above): same as `wxTOKEN_STRTOK` if the delimiter string contains only whitespaces, same as `wxTOKEN_RET_EMPTY` otherwise
`wxTOKEN_RET_EMPTY` | In this mode, the empty tokens in the middle of the string will be returned, i.e. `"a::b:"` will be tokenized in three tokens 'a', '' and 'b'. Notice that all trailing delimiters are ignored in this mode, not just the last one, i.e. a string `"a::b::"` would still result in the same set of tokens.
`wxTOKEN_RET_EMPTY_ALL` | In this mode, empty trailing tokens (including the one after the last delimiter character) will be returned as well. The string `"a::b:"` will be tokenized in four tokens: the already mentioned ones and another empty one as the last one and a string `"a::b::"` will have five tokens.
`wxTOKEN_RET_DELIMS` | In this mode, the delimiter character after the end of the current token (there may be none if this is the last token) is returned appended to the token. Otherwise, it is the same mode as `wxTOKEN_RET_EMPTY`. Notice that there is no mode like this one but behaving like `wxTOKEN_RET_EMPTY_ALL` instead of `wxTOKEN_RET_EMPTY`, use `wxTOKEN_RET_EMPTY_ALL` and GetLastDelimiter() to emulate it.
`wxTOKEN_STRTOK` | In this mode the class behaves exactly like the standard `strtok()` function: the empty tokens are never returned.

Derived from

wxObject

See also

wxStringTokenize

Include files

<wx/tokenzr.h>

Members

wxStringTokenizer::wxStringTokenizer
wxStringTokenizer::CountTokens
wxStringTokenizer::HasMoreTokens
wxStringTokenizer::GetLastDelimiter
wxStringTokenizer::GetNextToken
wxStringTokenizer::GetPosition
wxStringTokenizer::GetString
wxStringTokenizer::SetString

### wxStringTokenizer::wxStringTokenizer

wxStringTokenizer()

Default constructor. You must call SetString before calling any other methods.

wxStringTokenizer(const wxString& str, const wxString& delims = " \t\r\n", wxStringTokenizerMode mode = wxTOKEN_DEFAULT)

Constructor. Pass the string to tokenize, a string containing delimiters and the mode specifying how the string should be tokenized.

### wxStringTokenizer::CountTokens

int CountTokens() const

Returns the number of tokens remaining in the input string. The number of tokens returned by this function is decremented each time GetNextToken is called and when it reaches 0 HasMoreTokens returns `false`.

### wxStringTokenizer::HasMoreTokens

bool HasMoreTokens() const

Returns `true` if the tokenizer has further tokens, `false` if none are left.

### wxStringTokenizer::GetLastDelimiter

wxChar GetLastDelimiter()

Returns the delimiter which ended scan for the last token returned by GetNextToken() or `NUL` if there had been no calls to this function yet or if it returned the trailing empty token in `wxTOKEN_RET_EMPTY_ALL` mode.

This function is new since wxWidgets version 2.7.0

### wxStringTokenizer::GetNextToken

wxString GetNextToken() const

Returns the next token or empty string if the end of string was reached.

### wxStringTokenizer::GetPosition

size_t GetPosition() const

Returns the current position (i.e. one index after the last returned token or 0 if GetNextToken() has never been called) in the original string.

### wxStringTokenizer::GetString

wxString GetString() const

Returns the part of the starting string without all token already extracted.

### wxStringTokenizer::SetString

void SetString(const wxString& to_tokenize, const wxString& delims = " \t\r\n", wxStringTokenizerMode mode = wxTOKEN_DEFAULT)

Initializes the tokenizer.

Pass the string to tokenize, a string containing delimiters, and the mode specifying how the string should be tokenized.