##// END OF EJS Templates
Backport PR #2738: Unicode content crashes the pager (console)...
Backport PR #2738: Unicode content crashes the pager (console) We've run into an interesting bug in the astropy project. https://github.com/astropy/astropy/issues/600 When displaying a docstring that contains Unicode and is also long enough that it gets sent to the pager it fails since the docstring can't be sent to the pager as ascii. This crashes in the middle of sending content to the pager, so the shell ends up in an inconsistent state and stops echoing the keyboard etc. The fix (attached) is merely to encode the content sent to the pager in the same encoding as the terminal (`sys.stdout.encoding`). Strictly speaking, this isn't always the right thing to do, since the pager may be configured to expect a different encoding than the terminal, but that is sort of an irrational way to configure a machine... ;) For example, `less`, in the absence of any special environment variables to tell it otherwise, uses the standard `LC*` environment variables to determine what to do, which should be the same mechanism the terminal also uses by default. If anyone can suggest a better fix, I'm all for it. Perhaps it should be configurable, defaulting to `sys.stdout.encoding`?

File last commit:

r6208:5902725b
r9853:7f9a133e
Show More
nbjson.py
72 lines | 2.0 KiB | text/x-python | PythonLexer
Brian E. Granger
More review changes....
r4609 """Read and write notebooks in JSON format.
Authors:
* Brian Granger
"""
#-----------------------------------------------------------------------------
# Copyright (C) 2008-2011 The IPython Development Team
#
# Distributed under the terms of the BSD License. The full license is in
# the file COPYING, distributed as part of this software.
#-----------------------------------------------------------------------------
#-----------------------------------------------------------------------------
# Imports
#-----------------------------------------------------------------------------
Brian E. Granger
Adding nbformat subpackage.
r4392
MinRK
split likely multiline strings when writing to/from JSON
r5278 import copy
Brian E. Granger
Adding nbformat subpackage.
r4392 import json
MinRK
split likely multiline strings when writing to/from JSON
r5278 from .nbbase import from_dict
from .rwbase import (
NotebookReader, NotebookWriter, restore_bytes, rejoin_lines, split_lines
)
MinRK
nbjson.writes always returns unicode
r6208 from IPython.utils import py3compat
Brian E. Granger
More review changes....
r4609 #-----------------------------------------------------------------------------
# Code
#-----------------------------------------------------------------------------
Brian E. Granger
Adding nbformat subpackage.
r4392
class BytesEncoder(json.JSONEncoder):
MinRK
fix base64 code in nbformat.v2...
r5175 """A JSON encoder that accepts b64 (and other *ascii*) bytestrings."""
Brian E. Granger
Adding nbformat subpackage.
r4392 def default(self, obj):
if isinstance(obj, bytes):
MinRK
fix base64 code in nbformat.v2...
r5175 return obj.decode('ascii')
Brian E. Granger
Adding nbformat subpackage.
r4392 return json.JSONEncoder.default(self, obj)
class JSONReader(NotebookReader):
Brian E. Granger
Initial draft of more formal notebook format....
r4401 def reads(self, s, **kwargs):
Brian E. Granger
Adding nbformat subpackage.
r4392 nb = json.loads(s, **kwargs)
Brian E. Granger
Full versioning added to nbformat.
r4406 nb = self.to_notebook(nb, **kwargs)
Brian E. Granger
Adding nbformat subpackage.
r4392 return nb
Brian E. Granger
Full versioning added to nbformat.
r4406 def to_notebook(self, d, **kwargs):
MinRK
split likely multiline strings when writing to/from JSON
r5278 return restore_bytes(rejoin_lines(from_dict(d)))
Brian E. Granger
Full versioning added to nbformat.
r4406
Brian E. Granger
Adding nbformat subpackage.
r4392
class JSONWriter(NotebookWriter):
Brian E. Granger
Initial draft of more formal notebook format....
r4401 def writes(self, nb, **kwargs):
Brian E. Granger
Adding nbformat subpackage.
r4392 kwargs['cls'] = BytesEncoder
MinRK
shorten json indent to 1sp
r5293 kwargs['indent'] = 1
MinRK
sort keys when writing json notebooks
r5271 kwargs['sort_keys'] = True
Brian Granger
Fixing minor bugs in nbformat and saving....
r6032 kwargs['separators'] = (',',': ')
MinRK
split likely multiline strings when writing to/from JSON
r5278 if kwargs.pop('split_lines', True):
nb = split_lines(copy.deepcopy(nb))
MinRK
nbjson.writes always returns unicode
r6208 return py3compat.str_to_unicode(json.dumps(nb, **kwargs), 'utf-8')
MinRK
split likely multiline strings when writing to/from JSON
r5278
Brian E. Granger
Adding nbformat subpackage.
r4392
_reader = JSONReader()
_writer = JSONWriter()
reads = _reader.reads
read = _reader.read
Brian E. Granger
Full versioning added to nbformat.
r4406 to_notebook = _reader.to_notebook
Brian E. Granger
Adding nbformat subpackage.
r4392 write = _writer.write
writes = _writer.writes