In Files

  • rdoc/parser/ruby.rb

Files

Class/Module Index [+]

Quicksearch

RDoc::RubyLex::RDoc::RubyLex::RDoc::Parser::Ruby

Extracts code elements from a source file returning a TopLevel object containing the constituent file elements.

This file is based on rtags

RubyParser understands how to document:

  • classes

  • modules

  • methods

  • constants

  • aliases

  • private, public, protected

  • private_class_function, public_class_function

  • module_function

  • attr, attr_reader, attr_writer, attr_accessor

  • extra accessors given on the command line

  • metaprogrammed methods

  • require

  • include

Method Arguments

The parser extracts the arguments from the method definition. You can override this with a custom argument definition using the :call-seq: directive:

##
# This method can be called with a range or an offset and length
#
# :call-seq:
#   my_method(Range)
#   my_method(offset, length)

def my_method(*args)
end

The parser extracts yield expressions from method bodies to gather the yielded argument names. If your method manually calls a block instead of yielding or you want to override the discovered argument names use the :yields: directive:

##
# My method is awesome

def my_method(&block) # :yields: happy, times
  block.call 1, 2
end

Metaprogrammed Methods

To pick up a metaprogrammed method, the parser looks for a comment starting with ‘##’ before an identifier:

##
# This is a meta-programmed method!

add_my_method :meta_method, :arg1, :arg2

The parser looks at the token after the identifier to determine the name, in this example, :meta_method. If a name cannot be found, a warning is printed and ‘unknown is used.

You can force the name of a method using the :method: directive:

##
# :method: woo_hoo!

By default, meta-methods are instance methods. To indicate that a method is a singleton method instead use the :singleton-method: directive:

##
# :singleton-method:

You can also use the :singleton-method: directive with a name:

##
# :singleton-method: woo_hoo!

Hidden methods

You can provide documentation for methods that don’t appear using the :method: and :singleton-method: directives:

##
# :method: ghost_method
# There is a method here, but you can't see it!

##
# this is a comment for a regular method

def regular_method() end

Note that by default, the :method: directive will be ignored if there is a standard rdocable item following it.

Constants

NORMAL
SINGLE

Public Class Methods

new(top_level, file_name, content, options, stats) click to toggle source
 
               # File rdoc/parser/ruby.rb, line 1469
def initialize(top_level, file_name, content, options, stats)
  super

  @size = 0
  @token_listeners = nil
  @scanner = RDoc::RubyLex.new content, @options
  @scanner.exception_on_syntax_error = false

  reset
end
            

Public Instance Methods

add_token_listener(obj) click to toggle source
 
               # File rdoc/parser/ruby.rb, line 1480
def add_token_listener(obj)
  @token_listeners ||= []
  @token_listeners << obj
end
            
collect_first_comment() click to toggle source

Look for the first comment in a file that isn’t a shebang line.

 
               # File rdoc/parser/ruby.rb, line 1488
def collect_first_comment
  skip_tkspace
  res = ''
  first_line = true

  tk = get_tk

  while TkCOMMENT === tk
    if first_line and tk.text =~ /\A#!/ then
      skip_tkspace
      tk = get_tk
    elsif first_line and tk.text =~ /\A#\s*-\*-/ then
      first_line = false
      skip_tkspace
      tk = get_tk
    else
      first_line = false
      res << tk.text << "\n"
      tk = get_tk

      if TkNL === tk then
        skip_tkspace false
        tk = get_tk
      end
    end
  end

  unget_tk tk

  res
end
            
error(msg) click to toggle source
 
               # File rdoc/parser/ruby.rb, line 1520
def error(msg)
  msg = make_message msg
  $stderr.puts msg
  exit(1)
end
            
extract_call_seq(comment, meth) click to toggle source

Look for a ‘call-seq’ in the comment, and override the normal parameter stuff

 
               # File rdoc/parser/ruby.rb, line 1530
def extract_call_seq(comment, meth)
  if comment.sub!(/:?call-seq:(.*?)^\s*\#?\s*$/m, '') then
    seq = $1
    seq.gsub!(/^\s*\#\s*/, '')
    meth.call_seq = seq
  end

  meth
end
            
get_bool() click to toggle source
 
               # File rdoc/parser/ruby.rb, line 1540
def get_bool
  skip_tkspace
  tk = get_tk
  case tk
  when TkTRUE
    true
  when TkFALSE, TkNIL
    false
  else
    unget_tk tk
    true
  end
end
            
get_class_or_module(container) click to toggle source
Look for the name of a class of module (optionally with a leading

or

with

separated named) and return the ultimate name and container

 
               # File rdoc/parser/ruby.rb, line 1558
  def get_class_or_module(container)
    skip_tkspace
    name_t = get_tk

    # class ::A -> A is in the top level
    if TkCOLON2 === name_t then
      name_t = get_tk
      container = @top_level
    end

    skip_tkspace(false)

    while TkCOLON2 === peek_tk do
      prev_container = container
      container = container.find_module_named(name_t.name)
      if !container
#          warn("Couldn't find module #{name_t.name}")
        container = prev_container.add_module RDoc::NormalModule, name_t.name
      end
      get_tk
      name_t = get_tk
    end
    skip_tkspace(false)
    return [container, name_t]
  end
            
get_class_specification() click to toggle source

Return a superclass, which can be either a constant of an expression

 
               # File rdoc/parser/ruby.rb, line 1587
def get_class_specification
  tk = get_tk
  return "self" if TkSELF === tk

  res = ""
  while TkCOLON2 === tk or TkCOLON3 === tk or TkCONSTANT === tk do
    res += tk.text
    tk = get_tk
  end

  unget_tk(tk)
  skip_tkspace(false)

  get_tkread # empty out read buffer

  tk = get_tk

  case tk
  when TkNL, TkCOMMENT, TkSEMICOLON then
    unget_tk(tk)
    return res
  end

  res += parse_call_parameters(tk)
  res
end
            
get_constant() click to toggle source

Parse a constant, which might be qualified by one or more class or module names

 
               # File rdoc/parser/ruby.rb, line 1618
  def get_constant
    res = ""
    skip_tkspace(false)
    tk = get_tk

    while TkCOLON2 === tk or TkCOLON3 === tk or TkCONSTANT === tk do
      res += tk.text
      tk = get_tk
    end

#      if res.empty?
#        warn("Unexpected token #{tk} in constant")
#      end
    unget_tk(tk)
    res
  end
            
get_constant_with_optional_parens() click to toggle source

Get a constant that may be surrounded by parens

 
               # File rdoc/parser/ruby.rb, line 1638
def get_constant_with_optional_parens
  skip_tkspace(false)
  nest = 0
  while TkLPAREN === (tk = peek_tk) or TkfLPAREN === tk do
    get_tk
    skip_tkspace(true)
    nest += 1
  end

  name = get_constant

  while nest > 0
    skip_tkspace(true)
    tk = get_tk
    nest -= 1 if TkRPAREN === tk
  end
  name
end
            
get_symbol_or_name() click to toggle source
 
               # File rdoc/parser/ruby.rb, line 1657
def get_symbol_or_name
  tk = get_tk
  case tk
  when  TkSYMBOL
    tk.text.sub(/^:/, '')
  when TkId, TkOp
    tk.name
  when TkSTRING
    tk.text
  else
    raise "Name or symbol expected (got #{tk})"
  end
end
            
get_tk() click to toggle source
 
               # File rdoc/parser/ruby.rb, line 1671
def get_tk
  tk = nil
  if @tokens.empty?
    tk = @scanner.token
    @read.push @scanner.get_read
    puts "get_tk1 => #{tk.inspect}" if $TOKEN_DEBUG
  else
    @read.push @unget_read.shift
    tk = @tokens.shift
    puts "get_tk2 => #{tk.inspect}" if $TOKEN_DEBUG
  end

  if TkSYMBEG === tk then
    set_token_position(tk.line_no, tk.char_no)
    tk1 = get_tk
    if TkId === tk1 or TkOp === tk1 or TkSTRING === tk1 then
      if tk1.respond_to?(:name)
        tk = Token(TkSYMBOL).set_text(":" + tk1.name)
      else
        tk = Token(TkSYMBOL).set_text(":" + tk1.text)
      end
      # remove the identifier we just read (we're about to
      # replace it with a symbol)
      @token_listeners.each do |obj|
        obj.pop_token
      end if @token_listeners
    else
      warn("':' not followed by identifier or operator")
      tk = tk1
    end
  end

  # inform any listeners of our shiny new token
  @token_listeners.each do |obj|
    obj.add_token(tk)
  end if @token_listeners

  tk
end
            
get_tkread() click to toggle source
 
               # File rdoc/parser/ruby.rb, line 1711
def get_tkread
  read = @read.join("")
  @read = []
  read
end
            
look_for_directives_in(context, comment) click to toggle source

Look for directives in a normal comment block:

#-- - don't display comment from this point forward

This routine modifies it’s parameter

 
               # File rdoc/parser/ruby.rb, line 1724
def look_for_directives_in(context, comment)
  preprocess = RDoc::Markup::PreProcess.new(@file_name,
                                            @options.rdoc_include)

  preprocess.handle(comment) do |directive, param|
    case directive
    when 'enddoc' then
      throw :enddoc
    when 'main' then
      @options.main_page = param
      ''
    when 'method', 'singleton-method' then
      false # ignore
    when 'section' then
      context.set_current_section(param, comment)
      comment.replace ''
      break
    when 'startdoc' then
      context.start_doc
      context.force_documentation = true
      ''
    when 'stopdoc' then
      context.stop_doc
      ''
    when 'title' then
      @options.title = param
      ''
    else
      warn "Unrecognized directive '#{directive}'"
      false
    end
  end

  remove_private_comments(comment)
end
            
make_message(msg) click to toggle source
 
               # File rdoc/parser/ruby.rb, line 1760
def make_message(msg)
  prefix = "\n" + @file_name + ":"
  if @scanner
    prefix << "#{@scanner.line_no}:#{@scanner.char_no}: "
  end
  return prefix + msg
end
            
parse_alias(context, single, tk, comment) click to toggle source
 
               # File rdoc/parser/ruby.rb, line 1816
def parse_alias(context, single, tk, comment)
  skip_tkspace
  if TkLPAREN === peek_tk then
    get_tk
    skip_tkspace
  end
  new_name = get_symbol_or_name
  @scanner.instance_eval{@lex_state = EXPR_FNAME}
  skip_tkspace
  if TkCOMMA === peek_tk then
    get_tk
    skip_tkspace
  end
  old_name = get_symbol_or_name

  al = RDoc::Alias.new get_tkread, old_name, new_name, comment
  read_documentation_modifiers al, RDoc::ATTR_MODIFIERS
  if al.document_self
    context.add_alias(al)
  end
end
            
parse_attr(context, single, tk, comment) click to toggle source
 
               # File rdoc/parser/ruby.rb, line 1768
def parse_attr(context, single, tk, comment)
  args = parse_symbol_arg(1)
  if args.size > 0
    name = args[0]
    rw = "R"
    skip_tkspace(false)
    tk = get_tk
    if TkCOMMA === tk then
      rw = "RW" if get_bool
    else
      unget_tk tk
    end
    att = RDoc::Attr.new get_tkread, name, rw, comment
    read_documentation_modifiers att, RDoc::ATTR_MODIFIERS
    if att.document_self
      context.add_attribute(att)
    end
  else
    warn("'attr' ignored - looks like a variable")
  end
end
            
parse_attr_accessor(context, single, tk, comment) click to toggle source
 
               # File rdoc/parser/ruby.rb, line 1790
def parse_attr_accessor(context, single, tk, comment)
  args = parse_symbol_arg
  read = get_tkread
  rw = "?"

  # If nodoc is given, don't document any of them

  tmp = RDoc::CodeObject.new
  read_documentation_modifiers tmp, RDoc::ATTR_MODIFIERS
  return unless tmp.document_self

  case tk.name
  when "attr_reader"   then rw = "R"
  when "attr_writer"   then rw = "W"
  when "attr_accessor" then rw = "RW"
  else
    rw = @options.extra_accessor_flags[tk.name]
    rw = '?' if rw.nil?
  end

  for name in args
    att = RDoc::Attr.new get_tkread, name, rw, comment
    context.add_attribute att
  end
end
            
parse_call_parameters(tk) click to toggle source
 
               # File rdoc/parser/ruby.rb, line 1838
def parse_call_parameters(tk)
  end_token = case tk
              when TkLPAREN, TkfLPAREN
                TkRPAREN
              when TkRPAREN
                return ""
              else
                TkNL
              end
  nest = 0

  loop do
      case tk
      when TkSEMICOLON
        break
      when TkLPAREN, TkfLPAREN
        nest += 1
      when end_token
        if end_token == TkRPAREN
          nest -= 1
          break if @scanner.lex_state == EXPR_END and nest <= 0
        else
          break unless @scanner.continue
        end
      when TkCOMMENT
        unget_tk(tk)
        break
      end
      tk = get_tk
  end
  res = get_tkread.tr("\n", " ").strip
  res = "" if res == ";"
  res
end
            
parse_class(container, single, tk, comment) click to toggle source
 
               # File rdoc/parser/ruby.rb, line 1873
def parse_class(container, single, tk, comment)
  container, name_t = get_class_or_module(container)

  case name_t
  when TkCONSTANT
    name = name_t.name
    superclass = "Object"

    if TkLT === peek_tk then
      get_tk
      skip_tkspace(true)
      superclass = get_class_specification
      superclass = "<unknown>" if superclass.empty?
    end

    cls_type = single == SINGLE ? RDoc::SingleClass : RDoc::NormalClass
    cls = container.add_class cls_type, name, superclass

    @stats.add_class cls

    read_documentation_modifiers cls, RDoc::CLASS_MODIFIERS
    cls.record_location @top_level

    parse_statements cls
    cls.comment = comment

  when TkLSHFT
    case name = get_class_specification
    when "self", container.name
      parse_statements(container, SINGLE)
    else
      other = RDoc::TopLevel.find_class_named(name)
      unless other
        #            other = @top_level.add_class(NormalClass, name, nil)
        #            other.record_location(@top_level)
        #            other.comment = comment
        other = RDoc::NormalClass.new "Dummy", nil
      end

      @stats.add_class other

      read_documentation_modifiers other, RDoc::CLASS_MODIFIERS
      parse_statements(other, SINGLE)
    end

  else
    warn("Expected class name or '<<'. Got #{name_t.class}: #{name_t.text.inspect}")
  end
end
            
parse_comment(container, tk, comment) click to toggle source
 
               # File rdoc/parser/ruby.rb, line 1977
def parse_comment(container, tk, comment)
  line_no = tk.line_no
  column  = tk.char_no

  singleton = !!comment.sub!(/(^# +:?)(singleton-)(method:)/, '\1\3')

  if comment.sub!(/^# +:?method: *(\S*).*?\n/i, '') then
    name = $1 unless $1.empty?
  else
    return nil
  end

  meth = RDoc::GhostMethod.new get_tkread, name
  meth.singleton = singleton

  @stats.add_method meth

  meth.start_collecting_tokens
  indent = TkSPACE.new 1, 1
  indent.set_text " " * column

  position_comment = TkCOMMENT.new(line_no, 1, "# File #{@top_level.file_absolute_name}, line #{line_no}")
  meth.add_tokens [position_comment, NEWLINE_TOKEN, indent]

  meth.params = ''

  extract_call_seq comment, meth

  container.add_method meth if meth.document_self

  meth.comment = comment
end
            
parse_constant(container, single, tk, comment) click to toggle source
 
               # File rdoc/parser/ruby.rb, line 1923
def parse_constant(container, single, tk, comment)
  name = tk.name
  skip_tkspace(false)
  eq_tk = get_tk

  unless TkASSIGN === eq_tk then
    unget_tk(eq_tk)
    return
  end


  nest = 0
  get_tkread

  tk = get_tk
  if TkGT === tk then
    unget_tk(tk)
    unget_tk(eq_tk)
    return
  end

  loop do
      case tk
      when TkSEMICOLON
        break
      when TkLPAREN, TkfLPAREN, TkLBRACE, TkLBRACK, TkDO
        nest += 1
      when TkRPAREN, TkRBRACE, TkRBRACK, TkEND
        nest -= 1
      when TkCOMMENT
        if nest <= 0 && @scanner.lex_state == EXPR_END
          unget_tk(tk)
          break
        end
      when TkNL
        if (nest <= 0) && ((@scanner.lex_state == EXPR_END) || (!@scanner.continue))
          unget_tk(tk)
          break
        end
      end
      tk = get_tk
  end

  res = get_tkread.tr("\n", " ").strip
  res = "" if res == ";"

  con = RDoc::Constant.new name, res, comment
  read_documentation_modifiers con, RDoc::CONSTANT_MODIFIERS

  if con.document_self
    container.add_constant(con)
  end
end
            
parse_include(context, comment) click to toggle source
 
               # File rdoc/parser/ruby.rb, line 2010
def parse_include(context, comment)
  loop do
    skip_tkspace_comment

    name = get_constant_with_optional_parens
    context.add_include RDoc::Include.new(name, comment) unless name.empty?

    return unless TkCOMMA === peek_tk
    get_tk
  end
end
            
parse_meta_method(container, single, tk, comment) click to toggle source

Parses a meta-programmed method

 
               # File rdoc/parser/ruby.rb, line 2025
def parse_meta_method(container, single, tk, comment)
  line_no = tk.line_no
  column  = tk.char_no

  start_collecting_tokens
  add_token tk
  add_token_listener self

  skip_tkspace false

  singleton = !!comment.sub!(/(^# +:?)(singleton-)(method:)/, '\1\3')

  if comment.sub!(/^# +:?method: *(\S*).*?\n/i, '') then
    name = $1 unless $1.empty?
  end

  if name.nil? then
    name_t = get_tk
    case name_t
    when TkSYMBOL then
      name = name_t.text[1..-1]
    when TkSTRING then
      name = name_t.text[1..-2]
    else
      warn "#{container.top_level.file_relative_name}:#{name_t.line_no} unknown name token #{name_t.inspect} for meta-method"
      name = 'unknown'
    end
  end

  meth = RDoc::MetaMethod.new get_tkread, name
  meth.singleton = singleton

  @stats.add_method meth

  remove_token_listener self

  meth.start_collecting_tokens
  indent = TkSPACE.new 1, 1
  indent.set_text " " * column

  position_comment = TkCOMMENT.new(line_no, 1, "# File #{@top_level.file_absolute_name}, line #{line_no}")
  meth.add_tokens [position_comment, NEWLINE_TOKEN, indent]
  meth.add_tokens @token_stream

  add_token_listener meth

  meth.params = ''

  extract_call_seq comment, meth

  container.add_method meth if meth.document_self

  last_tk = tk

  while tk = get_tk do
    case tk
    when TkSEMICOLON then
      break
    when TkNL then
      break unless last_tk and TkCOMMA === last_tk
    when TkSPACE then
      # expression continues
    else
      last_tk = tk
    end
  end

  remove_token_listener meth

  meth.comment = comment
end
            
parse_method(container, single, tk, comment) click to toggle source

Parses a method

 
               # File rdoc/parser/ruby.rb, line 2100
def parse_method(container, single, tk, comment)
  line_no = tk.line_no
  column  = tk.char_no

  start_collecting_tokens
  add_token(tk)
  add_token_listener(self)

  @scanner.instance_eval do @lex_state = EXPR_FNAME end

  skip_tkspace(false)
  name_t = get_tk
  back_tk = skip_tkspace
  meth = nil
  added_container = false

  dot = get_tk
  if TkDOT === dot or TkCOLON2 === dot then
    @scanner.instance_eval do @lex_state = EXPR_FNAME end
    skip_tkspace
    name_t2 = get_tk

    case name_t
    when TkSELF then
      name = name_t2.name
    when TkCONSTANT then
      name = name_t2.name
      prev_container = container
      container = container.find_module_named(name_t.name)
      unless container then
        added_container = true
        obj = name_t.name.split("::").inject(Object) do |state, item|
          state.const_get(item)
        end rescue nil

        type = obj.class == Class ? RDoc::NormalClass : RDoc::NormalModule

        unless [Class, Module].include?(obj.class) then
          warn("Couldn't find #{name_t.name}. Assuming it's a module")
        end

        if type == RDoc::NormalClass then
          container = prev_container.add_class(type, name_t.name, obj.superclass.name)
        else
          container = prev_container.add_module(type, name_t.name)
        end

        container.record_location @top_level
      end
    else
      # warn("Unexpected token '#{name_t2.inspect}'")
      # break
      skip_method(container)
      return
    end

    meth = RDoc::AnyMethod.new(get_tkread, name)
    meth.singleton = true
  else
    unget_tk dot
    back_tk.reverse_each do |token|
      unget_tk token
    end
    name = name_t.name

    meth = RDoc::AnyMethod.new get_tkread, name
    meth.singleton = (single == SINGLE)
  end

  @stats.add_method meth

  remove_token_listener self

  meth.start_collecting_tokens
  indent = TkSPACE.new 1, 1
  indent.set_text " " * column

  token = TkCOMMENT.new(line_no, 1, "# File #{@top_level.file_absolute_name}, line #{line_no}")
  meth.add_tokens [token, NEWLINE_TOKEN, indent]
  meth.add_tokens @token_stream

  add_token_listener meth

  @scanner.instance_eval do @continue = false end
  parse_method_parameters meth

  if meth.document_self then
    container.add_method meth
  elsif added_container then
    container.document_self = false
  end

  # Having now read the method parameters and documentation modifiers, we
  # now know whether we have to rename #initialize to ::new

  if name == "initialize" && !meth.singleton then
    if meth.dont_rename_initialize then
      meth.visibility = :protected
    else
      meth.singleton = true
      meth.name = "new"
      meth.visibility = :public
    end
  end

  parse_statements(container, single, meth)

  remove_token_listener(meth)

  extract_call_seq comment, meth

  meth.comment = comment
end
            
parse_method_or_yield_parameters(method = nil, modifiers = RDoc::METHOD_MODIFIERS) click to toggle source
 
               # File rdoc/parser/ruby.rb, line 2214
def parse_method_or_yield_parameters(method = nil,
                                     modifiers = RDoc::METHOD_MODIFIERS)
  skip_tkspace(false)
  tk = get_tk

  # Little hack going on here. In the statement
  #  f = 2*(1+yield)
  # We see the RPAREN as the next token, so we need
  # to exit early. This still won't catch all cases
  # (such as "a = yield + 1"
  end_token = case tk
              when TkLPAREN, TkfLPAREN
                TkRPAREN
              when TkRPAREN
                return ""
              else
                TkNL
              end
  nest = 0

  loop do
      case tk
      when TkSEMICOLON
        break
      when TkLBRACE
        nest += 1
      when TkRBRACE
        # we might have a.each {|i| yield i }
        unget_tk(tk) if nest.zero?
        nest -= 1
        break if nest <= 0
      when TkLPAREN, TkfLPAREN
        nest += 1
      when end_token
        if end_token == TkRPAREN
          nest -= 1
          break if @scanner.lex_state == EXPR_END and nest <= 0
        else
          break unless @scanner.continue
        end
      when method && method.block_params.nil? && TkCOMMENT
        unget_tk(tk)
        read_documentation_modifiers(method, modifiers)
      end
    tk = get_tk
  end
  res = get_tkread.tr("\n", " ").strip
  res = "" if res == ";"
  res
end
            
parse_method_parameters(method) click to toggle source

Capture the method’s parameters. Along the way, look for a comment containing:

# yields: ....

and add this as the block_params for the method

 
               # File rdoc/parser/ruby.rb, line 2273
def parse_method_parameters(method)
  res = parse_method_or_yield_parameters(method)
  res = "(" + res + ")" unless res[0] == ?(
  method.params = res unless method.params
  if method.block_params.nil?
    skip_tkspace(false)
    read_documentation_modifiers method, RDoc::METHOD_MODIFIERS
  end
end
            
parse_module(container, single, tk, comment) click to toggle source
 
               # File rdoc/parser/ruby.rb, line 2283
def parse_module(container, single, tk, comment)
  container, name_t = get_class_or_module(container)

  name = name_t.name

  mod = container.add_module RDoc::NormalModule, name
  mod.record_location @top_level

  @stats.add_module mod

  read_documentation_modifiers mod, RDoc::CLASS_MODIFIERS
  parse_statements(mod)
  mod.comment = comment
end
            
parse_require(context, comment) click to toggle source
 
               # File rdoc/parser/ruby.rb, line 2298
def parse_require(context, comment)
  skip_tkspace_comment
  tk = get_tk
  if TkLPAREN === tk then
    skip_tkspace_comment
    tk = get_tk
  end

  name = nil
  case tk
  when TkSTRING
    name = tk.text
    #    when TkCONSTANT, TkIDENTIFIER, TkIVAR, TkGVAR
    #      name = tk.name
  when TkDSTRING
    warn "Skipping require of dynamic string: #{tk.text}"
    #   else
    #     warn "'require' used as variable"
  end
  if name
    context.add_require RDoc::Require.new(name, comment)
  else
    unget_tk(tk)
  end
end
            
parse_statements(container, single = NORMAL, current_method = nil, comment = '') click to toggle source
 
               # File rdoc/parser/ruby.rb, line 2324
def parse_statements(container, single = NORMAL, current_method = nil,
                     comment = '')
  nest = 1
  save_visibility = container.visibility

  non_comment_seen = true

  while tk = get_tk do
    keep_comment = false

    non_comment_seen = true unless TkCOMMENT === tk

    case tk
    when TkNL then
      skip_tkspace true # Skip blanks and newlines
      tk = get_tk

      if TkCOMMENT === tk then
        if non_comment_seen then
          # Look for RDoc in a comment about to be thrown away
          parse_comment container, tk, comment unless comment.empty?

          comment = ''
          non_comment_seen = false
        end

        while TkCOMMENT === tk do
          comment << tk.text << "\n"
          tk = get_tk          # this is the newline
          skip_tkspace(false)  # leading spaces
          tk = get_tk
        end

        unless comment.empty? then
          look_for_directives_in container, comment

          if container.done_documenting then
            container.ongoing_visibility = save_visibility
          end
        end

        keep_comment = true
      else
        non_comment_seen = true
      end

      unget_tk tk
      keep_comment = true

    when TkCLASS then
      if container.document_children then
        parse_class container, single, tk, comment
      else
        nest += 1
      end

    when TkMODULE then
      if container.document_children then
        parse_module container, single, tk, comment
      else
        nest += 1
      end

    when TkDEF then
      if container.document_self then
        parse_method container, single, tk, comment
      else
        nest += 1
      end

    when TkCONSTANT then
      if container.document_self then
        parse_constant container, single, tk, comment
      end

    when TkALIAS then
      if container.document_self then
        parse_alias container, single, tk, comment
      end

    when TkYIELD then
      if current_method.nil? then
        warn "Warning: yield outside of method" if container.document_self
      else
        parse_yield container, single, tk, current_method
      end

    # Until and While can have a 'do', which shouldn't increase the nesting.
    # We can't solve the general case, but we can handle most occurrences by
    # ignoring a do at the end of a line.

    when  TkUNTIL, TkWHILE then
      nest += 1
      skip_optional_do_after_expression

    # 'for' is trickier
    when TkFOR then
      nest += 1
      skip_for_variable
      skip_optional_do_after_expression

    when TkCASE, TkDO, TkIF, TkUNLESS, TkBEGIN then
      nest += 1

    when TkIDENTIFIER then
      if nest == 1 and current_method.nil? then
        case tk.name
        when 'private', 'protected', 'public', 'private_class_method',
             'public_class_method', 'module_function' then
          parse_visibility container, single, tk
          keep_comment = true
        when 'attr' then
          parse_attr container, single, tk, comment
        when /^attr_(reader|writer|accessor)$/, @options.extra_accessors then
          parse_attr_accessor container, single, tk, comment
        when 'alias_method' then
          if container.document_self then
            parse_alias container, single, tk, comment
          end
        else
          if container.document_self and comment =~ /\A#\#$/ then
            parse_meta_method container, single, tk, comment
          end
        end
      end

      case tk.name
      when "require" then
        parse_require container, comment
      when "include" then
        parse_include container, comment
      end

    when TkEND then
      nest -= 1
      if nest == 0 then
        read_documentation_modifiers container, RDoc::CLASS_MODIFIERS
        container.ongoing_visibility = save_visibility
        return
      end

    end

    comment = '' unless keep_comment

    begin
      get_tkread
      skip_tkspace(false)
    end while peek_tk == TkNL
  end
end
            
parse_symbol_arg(no = nil) click to toggle source
 
               # File rdoc/parser/ruby.rb, line 2476
def parse_symbol_arg(no = nil)
  args = []
  skip_tkspace_comment
  case tk = get_tk
  when TkLPAREN
    loop do
      skip_tkspace_comment
      if tk1 = parse_symbol_in_arg
        args.push tk1
        break if no and args.size >= no
      end

      skip_tkspace_comment
      case tk2 = get_tk
      when TkRPAREN
        break
      when TkCOMMA
      else
        warn("unexpected token: '#{tk2.inspect}'") if $DEBUG_RDOC
        break
      end
    end
  else
    unget_tk tk
    if tk = parse_symbol_in_arg
      args.push tk
      return args if no and args.size >= no
    end

    loop do
      skip_tkspace(false)

      tk1 = get_tk
      unless TkCOMMA === tk1 then
        unget_tk tk1
        break
      end

      skip_tkspace_comment
      if tk = parse_symbol_in_arg
        args.push tk
        break if no and args.size >= no
      end
    end
  end
  args
end
            
parse_symbol_in_arg() click to toggle source
 
               # File rdoc/parser/ruby.rb, line 2524
def parse_symbol_in_arg
  case tk = get_tk
  when TkSYMBOL
    tk.text.sub(/^:/, '')
  when TkSTRING
    eval @read[-1]
  else
    warn("Expected symbol or string, got #{tk.inspect}") if $DEBUG_RDOC
    nil
  end
end
            
parse_toplevel_statements(container) click to toggle source
 
               # File rdoc/parser/ruby.rb, line 2536
def parse_toplevel_statements(container)
  comment = collect_first_comment
  look_for_directives_in(container, comment)
  container.comment = comment unless comment.empty?
  parse_statements container, NORMAL, nil, comment
end
            
parse_visibility(container, single, tk) click to toggle source
 
               # File rdoc/parser/ruby.rb, line 2543
def parse_visibility(container, single, tk)
  singleton = (single == SINGLE)

  vis_type = tk.name

  vis = case vis_type
        when 'private'   then :private
        when 'protected' then :protected
        when 'public'    then :public
        when 'private_class_method' then
          singleton = true
          :private
        when 'public_class_method' then
          singleton = true
          :public
        when 'module_function' then
          singleton = true
          :public
        else
          raise "Invalid visibility: #{tk.name}"
        end

  skip_tkspace_comment false

  case peek_tk
    # Ryan Davis suggested the extension to ignore modifiers, because he
    # often writes
    #
    #   protected unless $TESTING
    #
  when TkNL, TkUNLESS_MOD, TkIF_MOD, TkSEMICOLON then
    container.ongoing_visibility = vis
  else
    if vis_type == 'module_function' then
      args = parse_symbol_arg
      container.set_visibility_for args, :private, false

      module_functions = []

      container.methods_matching args do |m|
        s_m = m.dup
        s_m.singleton = true if RDoc::AnyMethod === s_m
        s_m.visibility = :public
        module_functions << s_m
      end

      module_functions.each do |s_m|
        case s_m
        when RDoc::AnyMethod then
          container.add_method s_m
        when RDoc::Attr then
          container.add_attribute s_m
        end
      end
    else
      args = parse_symbol_arg
      container.set_visibility_for args, vis, singleton
    end
  end
end
            
parse_yield(context, single, tk, method) click to toggle source
 
               # File rdoc/parser/ruby.rb, line 2608
def parse_yield(context, single, tk, method)
  if method.block_params.nil?
    get_tkread
    @scanner.instance_eval{@continue = false}
    method.block_params = parse_yield_parameters
  end
end
            
parse_yield_parameters() click to toggle source
 
               # File rdoc/parser/ruby.rb, line 2604
def parse_yield_parameters
  parse_method_or_yield_parameters
end
            
peek_read() click to toggle source
 
               # File rdoc/parser/ruby.rb, line 2616
def peek_read
  @read.join('')
end
            
peek_tk() click to toggle source

Peek at the next token, but don’t remove it from the stream

 
               # File rdoc/parser/ruby.rb, line 2623
def peek_tk
  unget_tk(tk = get_tk)
  tk
end
            
read_directive(allowed) click to toggle source

Directives are modifier comments that can appear after class, module, or method names. For example:

def fred # :yields: a, b

or:

class MyClass # :nodoc:

We return the directive name and any parameters as a two element array

 
               # File rdoc/parser/ruby.rb, line 2640
def read_directive(allowed)
  tk = get_tk
  result = nil
  if TkCOMMENT === tk
    if tk.text =~ /\s*:?(\w+):\s*(.*)/
      directive = $1.downcase
      if allowed.include?(directive)
        result = [directive, $2]
      end
    end
  else
    unget_tk(tk)
  end
  result
end
            
read_documentation_modifiers(context, allow) click to toggle source
 
               # File rdoc/parser/ruby.rb, line 2656
def read_documentation_modifiers(context, allow)
  dir = read_directive(allow)

  case dir[0]
  when "notnew", "not_new", "not-new" then
    context.dont_rename_initialize = true

  when "nodoc" then
    context.document_self = false
    if dir[1].downcase == "all"
      context.document_children = false
    end

  when "doc" then
    context.document_self = true
    context.force_documentation = true

  when "yield", "yields" then
    unless context.params.nil?
      context.params.sub!(/(,|)\s*&\w+/,'') # remove parameter &proc
    end

    context.block_params = dir[1]

  when "arg", "args" then
    context.params = dir[1]
  end if dir
end
            
remove_private_comments(comment) click to toggle source
 
               # File rdoc/parser/ruby.rb, line 2685
def remove_private_comments(comment)
  comment.gsub!(/^#--\n.*?^#\+\+/m, '')
  comment.sub!(/^#--\n.*/m, '')
end
            
remove_token_listener(obj) click to toggle source
 
               # File rdoc/parser/ruby.rb, line 2690
def remove_token_listener(obj)
  @token_listeners.delete(obj)
end
            
reset() click to toggle source
 
               # File rdoc/parser/ruby.rb, line 2694
def reset
  @tokens = []
  @unget_read = []
  @read = []
end
            
scan() click to toggle source
 
               # File rdoc/parser/ruby.rb, line 2700
  def scan
    reset

    catch(:eof) do
      catch(:enddoc) do
        begin
          parse_toplevel_statements(@top_level)
        rescue Exception => e
          $stderr.puts <<-EOF


RDoc failure in #{@file_name} at or around line #{@scanner.line_no} column
#{@scanner.char_no}

Before reporting this, could you check that the file you're documenting
compiles cleanly--RDoc is not a full Ruby parser, and gets confused easily if
fed invalid programs.

The internal error was:

          EOF

          e.set_backtrace(e.backtrace[0,4])
          raise
        end
      end
    end

    @top_level
  end
            
skip_for_variable() click to toggle source

skip the var [in] part of a ‘for’ statement

 
               # File rdoc/parser/ruby.rb, line 2773
def skip_for_variable
  skip_tkspace(false)
  tk = get_tk
  skip_tkspace(false)
  tk = get_tk
  unget_tk(tk) unless TkIN === tk
end
            
skip_method(container) click to toggle source
 
               # File rdoc/parser/ruby.rb, line 2781
def skip_method(container)
  meth = RDoc::AnyMethod.new "", "anon"
  parse_method_parameters(meth)
  parse_statements(container, false, meth)
end
            
skip_optional_do_after_expression() click to toggle source

while, until, and for have an optional do

 
               # File rdoc/parser/ruby.rb, line 2734
def skip_optional_do_after_expression
  skip_tkspace(false)
  tk = get_tk
  case tk
  when TkLPAREN, TkfLPAREN
    end_token = TkRPAREN
  else
    end_token = TkNL
  end

  nest = 0
  @scanner.instance_eval{@continue = false}

  loop do
    case tk
    when TkSEMICOLON
      break
    when TkLPAREN, TkfLPAREN
      nest += 1
    when TkDO
      break if nest.zero?
    when end_token
      if end_token == TkRPAREN
        nest -= 1
        break if @scanner.lex_state == EXPR_END and nest.zero?
      else
        break unless @scanner.continue
      end
    end
    tk = get_tk
  end
  skip_tkspace(false)

  get_tk if TkDO === peek_tk
end
            
skip_tkspace(skip_nl = true) click to toggle source

Skip spaces

 
               # File rdoc/parser/ruby.rb, line 2790
def skip_tkspace(skip_nl = true)
  tokens = []

  while TkSPACE === (tk = get_tk) or (skip_nl and TkNL === tk) do
    tokens.push tk
  end

  unget_tk(tk)
  tokens
end
            
skip_tkspace_comment(skip_nl = true) click to toggle source

Skip spaces until a comment is found

 
               # File rdoc/parser/ruby.rb, line 2804
def skip_tkspace_comment(skip_nl = true)
  loop do
    skip_tkspace(skip_nl)
    return unless TkCOMMENT === peek_tk
    get_tk
  end
end
            
unget_tk(tk) click to toggle source
 
               # File rdoc/parser/ruby.rb, line 2812
def unget_tk(tk)
  @tokens.unshift tk
  @unget_read.unshift @read.pop

  # Remove this token from any listeners
  @token_listeners.each do |obj|
    obj.pop_token
  end if @token_listeners
end
            
warn(msg) click to toggle source
 
               # File rdoc/parser/ruby.rb, line 2822
def warn(msg)
  return if @options.quiet
  msg = make_message msg
  $stderr.puts msg
end