In Files

  • rdoc/token_stream.rb

Class/Module Index [+]

Quicksearch

RDoc::TokenStream

A TokenStream is a list of tokens, gathered during the parse of some entity (say a method). Entities populate these streams by being registered with the lexer. Any class can collect tokens by including TokenStream. From the outside, you use such an object by calling the #start_collecting_tokens method, followed by calls to #add_token and pop_token.

Public Instance Methods

add_token(*tokens) click to toggle source
Alias for: add_tokens
add_tokens(*tokens) click to toggle source

Adds tokens to the collected tokens

 
               # File rdoc/token_stream.rb, line 13
def add_tokens(*tokens)
  tokens.flatten.each { |token| @token_stream << token }
end
            
Also aliased as: add_token
collect_tokens() click to toggle source

Starts collecting tokens

 
               # File rdoc/token_stream.rb, line 22
def collect_tokens
  @token_stream = []
end
            
Also aliased as: start_collecting_tokens
pop_token() click to toggle source

Remove the last token from the collected tokens

 
               # File rdoc/token_stream.rb, line 31
def pop_token
  @token_stream.pop
end
            
start_collecting_tokens() click to toggle source
Alias for: collect_tokens
token_stream() click to toggle source

Current token stream

 
               # File rdoc/token_stream.rb, line 38
def token_stream
  @token_stream
end
            
tokens_to_s() click to toggle source

Returns a string representation of the token stream

 
               # File rdoc/token_stream.rb, line 45
def tokens_to_s
  token_stream.map { |token| token.text }.join ''
end