de.unkrig.commons.text.scanner

Class StatefulScanner<TT extends Enum<TT>,S extends Enum<S>>

java.lang.Object

de.unkrig.commons.text.scanner.AbstractScanner<TT>

de.unkrig.commons.text.scanner.StatefulScanner<TT,S>

Type Parameters:

TT - Enumerates the scanner-specific token types

S - Enumerates the scanner-specific non-default states

All Implemented Interfaces:

ProducerWhichThrows<AbstractScanner.Token<TT>,ScanException>, StringScanner<TT>

public class StatefulScanner<TT extends Enum<TT>,S extends Enum<S>>
extends AbstractScanner<TT>

A scanner that produces Tokens from a character stream.

Before produce() is called, the scanner must be configured by invoking its addRule(String, Enum) methods. These define how character sequences are converted into AbstractScanner.Tokens, and also how the scanner changes state. Initially the scanner is in the default state.

The non-default states are defined by the S type parameter.

For an example usage, see JavaScanner.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
class	StatefulScanner.Rule

Nested classes/interfaces inherited from class de.unkrig.commons.text.scanner.AbstractScanner

AbstractScanner.Token<TT extends Enum<TT>>

Field Summary

Fields

Modifier and Type	Field and Description
EnumSet<S>	ANY_STATE Special value for some method parameters.
S	REMAIN Special value for some method parameters; indicates that the current state should remain when the rule applies.

Fields inherited from class de.unkrig.commons.text.scanner.AbstractScanner

cs, offset, previousTokenOffset

Constructor Summary

Constructors

Constructor and Description
StatefulScanner(Class<S> states)
StatefulScanner(StatefulScanner<TT,S> that) Clones the configuration of the other scanner, but has a separate state.

Method Summary

Modifier and Type	Method and Description
StatefulScanner.Rule	addRule(EnumSet<S> states, String regex, TT tokenType) Adds a rule that applies iff states== ANY_STATE, or the scanner is in one of the given non-default states, or the states contain null and the scanner is in the default state.
StatefulScanner.Rule	addRule(EnumSet<S> states, String regex, TT tokenType, S nextState) Deprecated. Use ss.addRule(states, regex, tokenType).goTo(nextState) instead
StatefulScanner.Rule	addRule(S state, String regex, TT tokenType) Adds a rule that applies iff the scanner is in the given non-default state.
StatefulScanner.Rule	addRule(S state, String regex, TT tokenType, S nextState) Deprecated. Use addRule(state, regex, tokenType).goTo(nextState)
StatefulScanner.Rule	addRule(String regex, TT tokenType) Adds a rule that applies iff the scanner is in the "default state".
StatefulScanner.Rule	addRule(String regex, TT tokenType, S nextState) Deprecated. Use ss.addRule(regex, tokenType).goTo(nextState) instead.
S	getCurrentState()
AbstractScanner.Token<TT>	produce() Before StringScanner.setInput(CharSequence) is called, this method returns null.
void	setCurrentState(S newState)

Methods inherited from class de.unkrig.commons.text.scanner.AbstractScanner

getOffset, getPreviousTokenOffset, setInput, suppress, suppress, toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

ANY_STATE

@Nullable
public final EnumSet<S extends Enum<S>> ANY_STATE

Special value for some method parameters.

REMAIN

@Nullable
public final S extends Enum<S> REMAIN

Special value for some method parameters; indicates that the current state should remain when the rule applies.

See Also:

addRule(EnumSet, String, Enum, Enum)

Constructor Detail

StatefulScanner

public StatefulScanner(Class<S> states)

StatefulScanner

public StatefulScanner(StatefulScanner<TT,S> that)

Clones the configuration of the other scanner, but has a separate state.

Method Detail

addRule

public StatefulScanner.Rule addRule(String regex,
                                    TT tokenType)

Adds a rule that applies iff the scanner is in the "default state". After the rule has matched, the scanner remains in the default state.

See Also:

Pattern

addRule

public StatefulScanner.Rule addRule(S state,
                                    String regex,
                                    TT tokenType)

Adds a rule that applies iff the scanner is in the given non-default state. The scanner returns to the default state after the rule has matched.

Parameters:

state - null means "any state"

See Also:

Pattern

addRule

public StatefulScanner.Rule addRule(@Nullable
                                    EnumSet<S> states,
                                    String regex,
                                    TT tokenType)

Adds a rule that applies iff states== ANY_STATE, or the scanner is in one of the given non-default states, or the states contain null and the scanner is in the default state. The scanner returns to the default state after the rule has matched.

See Also:

Pattern

getCurrentState

@Nullable
public S getCurrentState()

Returns:

The current state of this scanner; null for the default state

setCurrentState

public void setCurrentState(@Nullable
                            S newState)

Parameters:

newState - The new state of this scanner; null for the default state

produce

@Nullable
public AbstractScanner.Token<TT> produce()
                                             throws ScanException

Description copied from interface: StringScanner

Before StringScanner.setInput(CharSequence) is called, this method returns null. After StringScanner.setInput(CharSequence) was called, this method breaks the input char character sequence up into tokens and returns them one by one. When the input char sequence is exhausted, null is returned until StringScanner.setInput(CharSequence) is called again.

Returns:

null iff the input string is exhausted

Throws:

ScanException

addRule

@Deprecated
public StatefulScanner.Rule addRule(String regex,
                                                 TT tokenType,
                                                 S nextState)

Deprecated. Use ss.addRule(regex, tokenType).goTo(nextState) instead.

addRule

@Deprecated
public StatefulScanner.Rule addRule(S state,
                                                 String regex,
                                                 TT tokenType,
                                                 @Nullable
                                                 S nextState)

Deprecated. Use addRule(state, regex, tokenType).goTo(nextState)

addRule

@Deprecated
public StatefulScanner.Rule addRule(@Nullable
                                                 EnumSet<S> states,
                                                 String regex,
                                                 TT tokenType,
                                                 @Nullable
                                                 S nextState)

Deprecated. Use ss.addRule(states, regex, tokenType).goTo(nextState) instead