elf_symbolizer.ELFSymbolizer.Addr2Line Class Reference

Inheritance diagram for elf_symbolizer.ELFSymbolizer.Addr2Line:

Public Member Functions
	__init__ (self, symbolizer)
	EnqueueRequest (self, addr, callback_arg)
	WaitForIdle (self)
	WaitForNextSymbolInQueue (self)
	ProcessAllResolvedSymbolsInQueue (self)
	RecycleIfNecessary (self)
	Terminate (self)
	first_request_id (self)

Static Public Member Functions
	StdoutReaderThread (process_pipe, queue, inlines)

Public Attributes
	queue_size

Static Public Attributes
	SYM_ADDR_RE = re.compile(r'([^:]+):(\?|\d+).*')

Protected Member Functions
	_WriteToA2lStdin (self, addr)
	_ProcessSymbolOutput (self, lines)
	_RestartAddr2LineProcess (self)

Protected Attributes
	_symbolizer
	_lib_file_name
	_request_queue
	_processed_symbols_count
	_proc
	_thread
	_out_queue

Detailed Description

A python wrapper around an addr2line instance.

The communication with the addr2line process looks as follows:
[STDIN]         [STDOUT]  (from addr2line's viewpoint)
> f001111
> f002222
            < Symbol::Name(foo, bar) for f001111
            < /path/to/source/file.c:line_number
> f003333
            < Symbol::Name2() for f002222
            < /path/to/source/file.c:line_number
            < Symbol::Name3() for f003333
            < /path/to/source/file.c:line_number

Definition at line 207 of file elf_symbolizer.py.

Constructor & Destructor Documentation

__init__()

elf_symbolizer.ELFSymbolizer.Addr2Line.__init__	(		self,
			symbolizer
	)

Definition at line 225 of file elf_symbolizer.py.

        def __init__(self, symbolizer):
            self._symbolizer = symbolizer
            self._lib_file_name = posixpath.basename(symbolizer.elf_file_path)
 
            # The request queue (i.e. addresses pushed to addr2line's stdin and not
            # yet retrieved on stdout)
            self._request_queue = collections.deque()
 
            # This is essentially len(self._request_queue). It has been optimized to a
            # separate field because turned out to be a perf hot-spot.
            self.queue_size = 0
 
            # Keep track of the number of symbols a process has processed to
            # avoid a single process growing too big and using all the memory.
            self._processed_symbols_count = 0
 
            # Objects required to handle the addr2line subprocess.
            self._proc = None  # Subprocess.Popen(...) instance.
            self._thread = None  # Threading.thread instance.
            self._out_queue = None  # queue.Queue instance (for buffering a2l stdout).
            self._RestartAddr2LineProcess()
 

Member Function Documentation

_ProcessSymbolOutput()

elf_symbolizer.ELFSymbolizer.Addr2Line._ProcessSymbolOutput (   self,   lines  )

protected

Parses an addr2line symbol output and triggers the client callback.

Definition at line 338 of file elf_symbolizer.py.

        def _ProcessSymbolOutput(self, lines):
            """Parses an addr2line symbol output and triggers the client callback."""
            (_, callback_arg, _) = self._request_queue.popleft()
            self.queue_size -= 1
 
            innermost_sym_info = None
            sym_info = None
            for (line1, line2) in lines:
                prev_sym_info = sym_info
                name = line1 if not line1.startswith('?') else None
                source_path = None
                source_line = None
                m = ELFSymbolizer.Addr2Line.SYM_ADDR_RE.match(line2)
                if m:
                    if not m.group(1).startswith('?'):
                        source_path = m.group(1)
                        if not m.group(2).startswith('?'):
                            source_line = int(m.group(2))
                else:
                    logging.warning(
                        'Got invalid symbol path from addr2line: %s' % line2)
 
                # In case disambiguation is on, and needed
                was_ambiguous = False
                disambiguated = False
                if self._symbolizer.disambiguate:
                    if source_path and not posixpath.isabs(source_path):
                        path = self._symbolizer.disambiguation_table.get(
                            source_path)
                        was_ambiguous = True
                        disambiguated = path is not None
                        source_path = path if disambiguated else source_path
 
                    # Use absolute paths (so that paths are consistent, as disambiguation
                    # uses absolute paths)
                    if source_path and not was_ambiguous:
                        source_path = os.path.abspath(source_path)
 
                if source_path and self._symbolizer.strip_base_path:
                    # Strip the base path
                    source_path = re.sub(
                        '^' + self._symbolizer.strip_base_path,
                        self._symbolizer.source_root_path or '', source_path)
 
                sym_info = ELFSymbolInfo(name, source_path, source_line,
                                         was_ambiguous, disambiguated)
                if prev_sym_info:
                    prev_sym_info.inlined_by = sym_info
                if not innermost_sym_info:
                    innermost_sym_info = sym_info
 
            self._processed_symbols_count += 1
            self._symbolizer.callback(innermost_sym_info, callback_arg)
 

_RestartAddr2LineProcess()

elf_symbolizer.ELFSymbolizer.Addr2Line._RestartAddr2LineProcess (   self )

protected

Definition at line 392 of file elf_symbolizer.py.

        def _RestartAddr2LineProcess(self):
            if self._proc:
                self.Terminate()
 
            # The only reason of existence of this Queue (and the corresponding
            # Thread below) is the lack of a subprocess.stdout.poll_avail_lines().
            # Essentially this is a pipe able to extract a couple of lines atomically.
            self._out_queue = queue.Queue()
 
            # Start the underlying addr2line process in line buffered mode.
 
            cmd = [
                self._symbolizer.addr2line_path, '--functions', '--demangle',
                '--exe=' + self._symbolizer.elf_file_path
            ]
            if self._symbolizer.inlines:
                cmd += ['--inlines']
            self._proc = subprocess.Popen(
                cmd,
                stdout=subprocess.PIPE,
                stdin=subprocess.PIPE,
                stderr=sys.stderr,
                close_fds=True)
 
            # Start the poller thread, which simply moves atomically the lines read
            # from the addr2line's stdout to the |_out_queue|.
            self._thread = threading.Thread(
                target=ELFSymbolizer.Addr2Line.StdoutReaderThread,
                args=(self._proc.stdout, self._out_queue,
                      self._symbolizer.inlines))
            self._thread.daemon = True  # Don't prevent early process exit.
            self._thread.start()
 
            self._processed_symbols_count = 0
 
            # Replay the pending requests on the new process (only for the case
            # of a hung addr2line timing out during the game).
            for (addr, _, _) in self._request_queue:
                self._WriteToA2lStdin(addr)
 

_WriteToA2lStdin()

elf_symbolizer.ELFSymbolizer.Addr2Line._WriteToA2lStdin (   self,   addr  )

protected

Definition at line 330 of file elf_symbolizer.py.

        def _WriteToA2lStdin(self, addr):
            self._proc.stdin.write(('%s\n' % hex(addr)).encode())
            if self._symbolizer.inlines:
                # In the case of inlines we output an extra blank line, which causes
                # addr2line to emit a (??,??:0) tuple that we use as a boundary marker.
                self._proc.stdin.write('\n')
            self._proc.stdin.flush()
 

EnqueueRequest()

elf_symbolizer.ELFSymbolizer.Addr2Line.EnqueueRequest	(		self,
			addr,
			callback_arg
	)

Pushes an address to addr2line's stdin (and keeps track of it).

Definition at line 247 of file elf_symbolizer.py.

        def EnqueueRequest(self, addr, callback_arg):
            """Pushes an address to addr2line's stdin (and keeps track of it)."""
            self._symbolizer.requests_counter += 1  # For global "age" of requests.
            req_idx = self._symbolizer.requests_counter
            self._request_queue.append((addr, callback_arg, req_idx))
            self.queue_size += 1
            self._WriteToA2lStdin(addr)
 

first_request_id()

elf_symbolizer.ELFSymbolizer.Addr2Line.first_request_id

(

self

)

Returns the request_id of the oldest pending request in the queue.

Definition at line 464 of file elf_symbolizer.py.

        def first_request_id(self):
            """Returns the request_id of the oldest pending request in the queue."""
            return self._request_queue[0][2] if self._request_queue else 0
 
 

ProcessAllResolvedSymbolsInQueue()

elf_symbolizer.ELFSymbolizer.Addr2Line.ProcessAllResolvedSymbolsInQueue

(

self

)

Consumes all the addr2line output lines produced (without blocking).

Definition at line 299 of file elf_symbolizer.py.

        def ProcessAllResolvedSymbolsInQueue(self):
            """Consumes all the addr2line output lines produced (without blocking)."""
            if not self.queue_size:
                return
            while True:
                try:
                    lines = self._out_queue.get_nowait()
                except queue.Empty:
                    break
                self._ProcessSymbolOutput(lines)
 

RecycleIfNecessary()

elf_symbolizer.ELFSymbolizer.Addr2Line.RecycleIfNecessary

(

self

)

Restarts the process if it has been used for too long.

A long running addr2line process will consume excessive amounts
of memory without any gain in performance.

Definition at line 310 of file elf_symbolizer.py.

        def RecycleIfNecessary(self):
            """Restarts the process if it has been used for too long.
 
      A long running addr2line process will consume excessive amounts
      of memory without any gain in performance."""
            if self._processed_symbols_count >= ADDR2LINE_RECYCLE_LIMIT:
                self._RestartAddr2LineProcess()
 

StdoutReaderThread()

elf_symbolizer.ELFSymbolizer.Addr2Line.StdoutReaderThread (   process_pipe,   queue,   inlines  )

static

The poller thread fn, which moves the addr2line stdout to the |queue|.

This is the only piece of code not running on the main thread. It merely
writes to a Queue, which is thread-safe. In the case of inlines, it
detects the ??,??:0 marker and sends the lines atomically, such that the
main thread always receives all the lines corresponding to one symbol in
one shot.

Definition at line 433 of file elf_symbolizer.py.

        def StdoutReaderThread(process_pipe, queue, inlines):
            """The poller thread fn, which moves the addr2line stdout to the |queue|.
 
      This is the only piece of code not running on the main thread. It merely
      writes to a Queue, which is thread-safe. In the case of inlines, it
      detects the ??,??:0 marker and sends the lines atomically, such that the
      main thread always receives all the lines corresponding to one symbol in
      one shot."""
            try:
                lines_for_one_symbol = []
                while True:
                    line1 = process_pipe.readline().decode().rstrip('\r\n')
                    line2 = process_pipe.readline().decode().rstrip('\r\n')
                    if not line1 or not line2:
                        break
                    inline_has_more_lines = inlines and (
                        len(lines_for_one_symbol) == 0 or
                        (line1 != '??' and line2 != '??:0'))
                    if not inlines or inline_has_more_lines:
                        lines_for_one_symbol += [(line1, line2)]
                    if inline_has_more_lines:
                        continue
                    queue.put(lines_for_one_symbol)
                    lines_for_one_symbol = []
                process_pipe.close()
 
            # Every addr2line processes will die at some point, please die silently.
            except (IOError, OSError):
                pass
 

Terminate()

elf_symbolizer.ELFSymbolizer.Addr2Line.Terminate

(

self

)

Kills the underlying addr2line process.

The poller |_thread| will terminate as well due to the broken pipe.

Definition at line 318 of file elf_symbolizer.py.

        def Terminate(self):
            """Kills the underlying addr2line process.
 
      The poller |_thread| will terminate as well due to the broken pipe."""
            try:
                self._proc.kill()
                self._proc.communicate(
                )  # Essentially wait() without risking deadlock.
            except Exception:  # An exception while terminating? How interesting.
                pass
            self._proc = None
 

WaitForIdle()

elf_symbolizer.ELFSymbolizer.Addr2Line.WaitForIdle

(

self

)

Waits until all the pending requests have been symbolized.

Definition at line 255 of file elf_symbolizer.py.

        def WaitForIdle(self):
            """Waits until all the pending requests have been symbolized."""
            while self.queue_size > 0:
                self.WaitForNextSymbolInQueue()
 

WaitForNextSymbolInQueue()

elf_symbolizer.ELFSymbolizer.Addr2Line.WaitForNextSymbolInQueue

(

self

)

Waits for the next pending request to be symbolized.

Definition at line 260 of file elf_symbolizer.py.

        def WaitForNextSymbolInQueue(self):
            """Waits for the next pending request to be symbolized."""
            if not self.queue_size:
                return
 
            # This outer loop guards against a2l hanging (detecting stdout timeout).
            while True:
                start_time = datetime.datetime.now()
                timeout = datetime.timedelta(
                    seconds=self._symbolizer.addr2line_timeout)
 
                # The inner loop guards against a2l crashing (checking if it exited).
                while (datetime.datetime.now() - start_time < timeout):
                    # poll() returns !None if the process exited. a2l should never exit.
                    if self._proc.poll():
                        logging.warning(
                            'addr2line crashed, respawning (lib: %s).' %
                            self._lib_file_name)
                        self._RestartAddr2LineProcess()
                        # TODO(primiano): the best thing to do in this case would be
                        # shrinking the pool size as, very likely, addr2line is crashed
                        # due to low memory (and the respawned one will die again soon).
 
                    try:
                        lines = self._out_queue.get(block=True, timeout=0.25)
                    except queue.Empty:
                        # On timeout (1/4 s.) repeat the inner loop and check if either the
                        # addr2line process did crash or we waited its output for too long.
                        continue
 
                    # In nominal conditions, we get straight to this point.
                    self._ProcessSymbolOutput(lines)
                    return
 
                # If this point is reached, we waited more than |addr2line_timeout|.
                logging.warning('Hung addr2line process, respawning (lib: %s).'
                                % self._lib_file_name)
                self._RestartAddr2LineProcess()
 

Member Data Documentation

_lib_file_name

elf_symbolizer.ELFSymbolizer.Addr2Line._lib_file_name

protected

Definition at line 227 of file elf_symbolizer.py.

_out_queue

elf_symbolizer.ELFSymbolizer.Addr2Line._out_queue

protected

Definition at line 244 of file elf_symbolizer.py.

_proc

elf_symbolizer.ELFSymbolizer.Addr2Line._proc

protected

Definition at line 242 of file elf_symbolizer.py.

_processed_symbols_count

elf_symbolizer.ELFSymbolizer.Addr2Line._processed_symbols_count

protected

Definition at line 239 of file elf_symbolizer.py.

_request_queue

elf_symbolizer.ELFSymbolizer.Addr2Line._request_queue

protected

Definition at line 231 of file elf_symbolizer.py.

_symbolizer

elf_symbolizer.ELFSymbolizer.Addr2Line._symbolizer

protected

Definition at line 226 of file elf_symbolizer.py.

_thread

elf_symbolizer.ELFSymbolizer.Addr2Line._thread

protected

Definition at line 243 of file elf_symbolizer.py.

queue_size

elf_symbolizer.ELFSymbolizer.Addr2Line.queue_size

Definition at line 235 of file elf_symbolizer.py.

SYM_ADDR_RE

elf_symbolizer.ELFSymbolizer.Addr2Line.SYM_ADDR_RE = re.compile(r'([^:]+):(\?|\d+).*')

static

Definition at line 223 of file elf_symbolizer.py.

---

The documentation for this class was generated from the following file:

• third_party/dart-lang/sdk/third_party/binary_size/src/elf_symbolizer.py