In Python testsuite, setting stdout to unbuffered causes us to switch to ascii output locale

I had a failing test case which was producing UTF-8 output, and Python was choking on it trying to print it. I traced the problem to these lines of code in testsuite/driver/runtests.py:

sys.stdout.flush()
if PYTHON3:
    # in Python 3, we output text, which cannot be unbuffered
    sys.stdout = os.fdopen(sys.__stdout__.fileno(), "w")
else:
    # set stdout to unbuffered (is this the best way to do it?)
    sys.stdout = os.fdopen(sys.__stdout__.fileno(), "w", 0)

On Python 2.7.11 on Linux, with LANG=en_US.UTF-8 in the environment, if you place a print(u"\u2018") before these lines of code, Python will successfully print it; however, if you place it after, Python will crash:

Traceback (most recent call last):
  File "../../driver/runtests.py", line 279, in <module>
    print(u"\u2018")
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2018' in position 0: ordinal not in range(128)

So, I guess fdopen resets the codec to ascii, rather than whatever the locale detected. Drat!

I did some brief googling but I couldn't tell what the right way to rewrite this line of code is.

Trac metadata

Trac field	Value
Version	8.0.1
Type	Bug
TypeOfFailure	OtherFailure
Priority	normal
Resolution	Unresolved
Component	Test Suite
Test case
Differential revisions
BlockedBy
Related
Blocking
CC
Operating system
Architecture