Class Lexer<T extends TokenType>

Method Details

• next

  public Token next() throws LexerException

  Returns:

  the next Token or Lexer.Tokens.eofToken if parsing as reached the end.

  Throws:

  LexerException

• remaining

  public String remaining()

  Returns:

  the remaining source String that still needs to be parsed. This can be useful for error reporting, i.e. show the user the unadulterated string that we were trying to parse next. Note that you may want to truncate this string in error messages, when parsing a large file.

• peek

  public Token peek() throws LexerException

  Returns:

  the Token that will be returned from the Lexer when you call next(), without consuming it.

  Throws:

  LexerException

• isEOF

  public boolean isEOF() throws LexerException

  Returns:

  true if the Lexer is at the end of the source

  Throws:

  LexerException

• peekType

  public T peekType()

  Returns:

  shortcut for peek().getTokenType()

• rewind

  public void rewind(Token backTo)

  Reposition the lexer so that next() returns this Token again.

  Parameters:

  backTo -

• expect

  @SafeVarargs public final Token expect(TokenType... expected)

• expect

  public final Token expect(Set<? extends TokenType> expected)

• tryThese

  @SafeVarargs public final <U> U tryThese(ConditionalParse<U>... attemptsArray)

• consumeIf

  public Optional<Token> consumeIf(TokenType conditionalToken)

• consumeIf

  public Optional<Token> consumeIf(Set<? extends TokenType> oneOf)