# Class Tokenizer

This class is used to convert CSS strings into an array of CSS tokens

Namespace: WPGMZA\Selector
Located at class.selector-to-xpath.php

Methods summary
`protected` | # `pushToken( string $type, string $char )` Pushes a new token to the token array Pushes a new token to the token array #### Parameters $type The token type, @see Token $char The character(s) to initialize the new token with
`protected` | # `pushCharToString( string $char )` Either pushes the specified character to the current token if the current token is a string, or initializes and pushes a new token to the token array. Either pushes the specified character to the current token if the current token is a string, or initializes and pushes a new token to the token array. #### Parameters $char The character(s) to push
`protected` | # `pushCharToIdentifier( string $char )` Either pushes the specified character to the current token if the current token is an identifier, or initializes and pushes a new token to the token array. Either pushes the specified character to the current token if the current token is an identifier, or initializes and pushes a new token to the token array. #### Parameters $char The character(s) to push
`protected` | # `pushCharToExpression( string $char )` Either pushes the specified character to the current token if the current token is an expression, or initializes and pushes a new token to the token array. Either pushes the specified character to the current token if the current token is an expression, or initializes and pushes a new token to the token array. #### Parameters $char The character(s) to push
`protected `WPGMZA\Selector\Token`` | # `popToken( )` Pops a token from the end of the token array Pops a token from the end of the token array #### Returns `WPGMZA\Selector\Token` the popped token
`protected `WPGMZA\Selector\Token`|null` | # `getCurrToken( )` Gets the current token (the last token in the token array) Gets the current token (the last token in the token array) #### Returns `WPGMZA\Selector\Token`|null The current token, or null if no tokens are in the array
`public `WPGMZA\Selector\Token[]`` | # `tokenize( string $str )` Attempts to tokenize the specified string Attempts to tokenize the specified string #### Parameters $str The input string #### Returns `WPGMZA\Selector\Token[]` An array of tokens parsed from the string #### Throws `WPGMZA\Selector\ParseException` When parsing the string fails due to invalid CSS

Properties summary
`protected` | $tokens | # ``
`protected` | $prevToken | # ``