Extended maintenance of Ruby 1.9.3 ended on February 23, 2015. Read more

In Files

  • rdoc/token_stream.rb

Class/Module Index [+]



A TokenStream is a list of tokens, gathered during the parse of some entity (say a method). Entities populate these streams by being registered with the lexer. Any class can collect tokens by including TokenStream. From the outside, you use such an object by calling the #start_collecting_tokens method, followed by calls to #add_token and pop_token.

Public Instance Methods

add_token(*tokens) click to toggle source
Alias for: add_tokens
add_tokens(*tokens) click to toggle source

Adds tokens to the collected tokens

               # File rdoc/token_stream.rb, line 13
def add_tokens(*tokens)
  tokens.flatten.each { |token| @token_stream << token }
Also aliased as: add_token
collect_tokens() click to toggle source

Starts collecting tokens

               # File rdoc/token_stream.rb, line 22
def collect_tokens
  @token_stream = []
Also aliased as: start_collecting_tokens
pop_token() click to toggle source

Remove the last token from the collected tokens

               # File rdoc/token_stream.rb, line 31
def pop_token
start_collecting_tokens() click to toggle source
Alias for: collect_tokens
token_stream() click to toggle source

Current token stream

               # File rdoc/token_stream.rb, line 38
def token_stream
tokens_to_s() click to toggle source

Returns a string representation of the token stream

               # File rdoc/token_stream.rb, line 45
def tokens_to_s
  token_stream.map { |token| token.text }.join ''

Commenting is here to help enhance the documentation. For example, code samples, or clarification of the documentation.

If you have questions about Ruby or the documentation, please post to one of the Ruby mailing lists. You will get better, faster, help that way.

If you wish to post a correction of the docs, please do so, but also file bug report so that it can be corrected for the next release. Thank you.

If you want to help improve the Ruby documentation, please visit Documenting-ruby.org.