org.lateralnz.c3d.util
Class Tokenizer

java.lang.Object
  org.lateralnz.c3d.util.Tokenizer

public class Tokenizer
extends java.lang.Object

a custom string tokenizer (based on the standard Java StringTokenizer) that supports marking a position and returning to it. This is most useful in lexical analysis/parsing/etc.

Constructor Summary

Tokenizer(java.lang.String text, java.lang.String delimiters, java.lang.String returnDelimiters)

Method Summary

void	destroy() clean up and shutdown
boolean	hasMoreTokens() return true if there are more tokens
void	mark() mark the current position
java.lang.String	nextToken() get the next token
java.lang.String	peekNextToken() look at the next token without moving the position indicator.
void	release() return to the last marked position
void	reset() reset the position counter to start tokenizing at the beginning again
void	setString(java.lang.String text) set the string to tokenize (a helper to save having to create this object every time)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Tokenizer

public Tokenizer(java.lang.String text,
                 java.lang.String delimiters,
                 java.lang.String returnDelimiters)

Method Detail

setString

public final void setString(java.lang.String text)

set the string to tokenize (a helper to save having to create this object every time)

destroy

public final void destroy()

clean up and shutdown

mark

public final void mark()

mark the current position

release

public final void release()

return to the last marked position

reset

public final void reset()

reset the position counter to start tokenizing at the beginning again

hasMoreTokens

public final boolean hasMoreTokens()

return true if there are more tokens

nextToken

public final java.lang.String nextToken()

get the next token

peekNextToken

public final java.lang.String peekNextToken()

look at the next token without moving the position indicator. (basically the works out to mark, next token, release)