interactive() readline history chokes on UTF-8 characters

[rofl0r]

test.py (doesn't work from inside REPL):

from pwn import *
r = process('cat', shell=True)
r.interactive()

• enter a string containing UTF-8 characters such löl <ENTER>.
• hit cursor-up to access readline history
• encounter exception:

  File "/lib/python2.7/site-packages/pwnlib/term/readline.py", line 131, in set_buffer
    buffer_left = unicode(left)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 0: ordinal not in range(128)

[zachriggle]

Yup. Welcome to Python2! If you are sending unicode you need to perform the appropriate encode / decode magic to turn it into a bytes object.

Alternately, I recommend setting a default encoding for your system, and setting an appropriate header.

https://stackoverflow.com/questions/2276200/changing-default-encoding-of-python
https://stackoverflow.com/questions/3170211/why-declare-unicode-by-string-in-python

[rofl0r]

so you don't think that pwntools should catch this error ?

[zachriggle]

After some sleep, I see that the issue you're reporting is different than I was expecting.

[rofl0r]

great, ty !

[TethysSvensson]

I haven't tried replicating this, but from the commit log, it seems to have only been fixed for dev?

[TethysSvensson]

Hmm, nope. It just didn't go through a PR when pushed into stable.