upstream/mercurial-mirror Files · contrib/showstack.py

worker: Use buffered input from the pickle stream...

worker: Use buffered input from the pickle stream On Python 3, "pickle.load" will raise an exception ("_pickle.UnpicklingError: pickle data was truncated") when it gets a short read, i.e. it receives fewer bytes than it requested. On our build machine, Mercurial seems to frequently hit this problem while updating a mozilla-central clone iff it gets scheduled in batch mode. It is easy to trigger with: #wipe the workdir rm -rf * hg update null chrt -b 0 hg update default I've also written the following program, which demonstrates the core problem: from __future__ import print_function import io import os import pickle import time obj = {"a": 1, "b": 2} obj_data = pickle.dumps(obj) assert len(obj_data) > 10 rfd, wfd = os.pipe() pid = os.fork() if pid == 0: os.close(rfd) for _ in range(4): time.sleep(0.5) print("First write") os.write(wfd, obj_data[:10]) time.sleep(0.5) print("Second write") os.write(wfd, obj_data[10:]) os._exit(0) try: os.close(wfd) rfile = os.fdopen(rfd, "rb", 0) print("Reading") while True: try: obj_copy = pickle.load(rfile) assert obj == obj_copy except EOFError: break print("Success") finally: os.kill(pid, 15) The program reliably fails with Python 3.8 and succeeds with Python 2.7. Providing the unpickler with a buffered reader fixes the issue, so let "os.fdopen" create one. https://bugzilla.mozilla.org/show_bug.cgi?id=1604486 Differential Revision: https://phab.mercurial-scm.org/D8051

Augie Fackler - - Load All Authors

File last commit:

r43346:2372284d default


                r44718:cb52e619

stable

Download file

             showstack.py
        
                    31 lines
            
             | 734 B
            
                | text/x-python
            
             |
                PythonLexer
            
             / contrib / showstack.py
          
                    History
                
                 |
                  Source
                 | Raw
                 |Copy content
                 |Copy permalink

        Matt Mackall
    
contrib: add showstack extension...

              r26123
            
      # showstack.py - extension to dump a Python stack trace on signal

      #

      # binds to both SIGQUIT (Ctrl-\) and SIGINFO (Ctrl-T on BSDs)

        Gregory Szorc
    
showstack: use raw docstring...

              r41689
            
      r"""dump stack trace when receiving SIGQUIT (Ctrl-\) or SIGINFO (Ctrl-T on BSDs)

        Boris Feld
    
showstack: add an extension docstring...

              r35674
            
      """

        Matt Mackall
    
contrib: add showstack extension...

              r26123
            
        Augie Fackler
    
showstack: also handle SIGALRM...

              r40072
            
      from __future__ import absolute_import, print_function

        Pulkit Goyal
    
showstack: use absolute_import

              r28522
            
      import signal

      import sys

      import traceback

        Matt Mackall
    
contrib: add showstack extension...

              r26123
            
        Augie Fackler
    
formatting: blacken the codebase...

              r43346
            
        Matt Mackall
    
contrib: add showstack extension...

              r26123
            
      def sigshow(*args):

          sys.stderr.write("\n")

          traceback.print_stack(args[1], limit=10, file=sys.stderr)

          sys.stderr.write("----\n")

        Augie Fackler
    
formatting: blacken the codebase...

              r43346
            
        Augie Fackler
    
showstack: also handle SIGALRM...

              r40072
            
      def sigexit(*args):

          sigshow(*args)

          print('alarm!')

          sys.exit(1)

        Augie Fackler
    
formatting: blacken the codebase...

              r43346
            
        Matt Mackall
    
contrib: add showstack extension...

              r26123
            
      def extsetup(ui):

          signal.signal(signal.SIGQUIT, sigshow)

        Augie Fackler
    
showstack: also handle SIGALRM...

              r40072
            
          signal.signal(signal.SIGALRM, sigexit)

        Matt Mackall
    
contrib: add showstack extension...

              r26123
            
          try:

              signal.signal(signal.SIGINFO, sigshow)

          except AttributeError:

              pass

	Site-wide shortcuts
/	Use quick search box
g h	Goto home page
g g	Goto my private gists page
g G	Goto my public gists page
g 0-9	Goto bookmarked items from 0-9
n r	New repository page
n g	New gist page

	Repositories
g s	Goto summary page
g c	Goto changelog page
g f	Goto files page
g F	Goto files page with file search activated
g p	Goto pull requests page
g o	Goto repository settings
g O	Goto repository access permissions settings
t s	Toggle sidebar on some pages

Matt Mackall contrib: add showstack extension...	r26123	# showstack.py - extension to dump a Python stack trace on signal
		#
		# binds to both SIGQUIT (Ctrl-\) and SIGINFO (Ctrl-T on BSDs)
Gregory Szorc showstack: use raw docstring...	r41689	r"""dump stack trace when receiving SIGQUIT (Ctrl-\) or SIGINFO (Ctrl-T on BSDs)
Boris Feld showstack: add an extension docstring...	r35674	"""
Matt Mackall contrib: add showstack extension...	r26123
Augie Fackler showstack: also handle SIGALRM...	r40072	from __future__ import absolute_import, print_function
Pulkit Goyal showstack: use absolute_import	r28522	import signal
		import sys
		import traceback
Matt Mackall contrib: add showstack extension...	r26123
Augie Fackler formatting: blacken the codebase...	r43346
Matt Mackall contrib: add showstack extension...	r26123	def sigshow(*args):
		sys.stderr.write("\n")
		traceback.print_stack(args[1], limit=10, file=sys.stderr)
		sys.stderr.write("----\n")

Augie Fackler formatting: blacken the codebase...	r43346
Augie Fackler showstack: also handle SIGALRM...	r40072	def sigexit(*args):
		sigshow(*args)
		print('alarm!')
		sys.exit(1)

Augie Fackler formatting: blacken the codebase...	r43346
Matt Mackall contrib: add showstack extension...	r26123	def extsetup(ui):
		signal.signal(signal.SIGQUIT, sigshow)
Augie Fackler showstack: also handle SIGALRM...	r40072	signal.signal(signal.SIGALRM, sigexit)
Matt Mackall contrib: add showstack extension...	r26123	try:
		signal.signal(signal.SIGINFO, sigshow)
		except AttributeError:
		pass