Returns an array of tokens given the GLSL source string src
You can specify opt.version string to use different keywords/builtins, such as '300 es' for WebGL2. Otherwise, will assume GLSL 100 (WebGL1).
var tokens = tokenizer(src, { version: '300 es'})
stream = require('glsl-tokenizer/stream')([opt])
Emits 'data' events whenever a token is parsed with a token object as output.
As above, you can specify opt.version.
Tokens
{ 'type': TOKEN_TYPE, 'data': "string of constituent data", 'position': integer position within the GLSL source, 'line': line number within the GLSL source, 'column': column number within the GLSL source }
The available token types are:
block-comment: /* ... */
line-comment: // ... \n
preprocessor: # ... \n
operator: Any operator. If it looks like punctuation, it's an operator.