Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/serialtube #90

Merged
merged 41 commits into from
Aug 25, 2018
Merged
Show file tree
Hide file tree
Changes from 24 commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
b3fe465
add serialtube, up version 1.0.2
JonathanBeverley Jan 17, 2018
ab1d1ee
auto require SocketTube as part of pwn
JonathanBeverley Jan 17, 2018
5cd3a53
BUGFIX: can't return from within a timer loop, must next
JonathanBeverley Jan 17, 2018
e1a674a
Rubyserial doesn't handle timeouts. Working around.
JonathanBeverley Jan 29, 2018
20e6a38
style fix: no arguments to `next`
JonathanBeverley Jan 29, 2018
31b57b3
revert version to 1.0.1
JonathanBeverley Feb 2, 2018
7dfa4f4
add serialtube_test; some cleanup of serialtube
JonathanBeverley Feb 2, 2018
7cfc0fc
Merge branch 'master' into feature/serialtube
JonathanBeverley May 15, 2018
ad30289
serialtube function comments
JonathanBeverley May 15, 2018
f1930b7
Handle timeouts well enough
JonathanBeverley May 15, 2018
55cc048
style fixes
JonathanBeverley May 15, 2018
ba86f2c
old updates to sock_test
JonathanBeverley May 15, 2018
cbffd90
Merge remote-tracking branch 'refs/remotes/origin/feature/serialtube'…
JonathanBeverley May 15, 2018
39a3fbc
make code uglier to make rubocop happy
JonathanBeverley May 15, 2018
1da19e8
rename SerialTube internal timer to avoid interfering with generic Tu…
JonathanBeverley Jun 7, 2018
3a3661b
revert accidental change to sock_test.rb
JonathanBeverley Jun 7, 2018
7b84223
Bugfix: in case of timeout, we need to deactivate the timer
JonathanBeverley Jun 8, 2018
94c3b08
Bugfix: in recv_pred, we only want to unrecv if there was an error
JonathanBeverley Jun 8, 2018
4772e6a
Bugfix: recv_raw should return early if there is ANY data
JonathanBeverley Jun 8, 2018
78a21a4
Fixing issues in testcode
JonathanBeverley Jun 8, 2018
867b892
reduce timeouts for faster testing; eliminate a rubocop disable
JonathanBeverley Jun 8, 2018
b6077ae
sort requires
JonathanBeverley Jun 9, 2018
ffdee6f
multiplatform testing: install socat for mac, skip for windows
JonathanBeverley Jun 10, 2018
625e103
add macOS style serial PTYs to regex filter
JonathanBeverley Jun 10, 2018
6b61cb1
make requested changes
JonathanBeverley Jun 12, 2018
ebfa490
revert ensure->rescue; clarify some comments
JonathanBeverley Jun 12, 2018
4fc138e
Merge branch 'feature/serialtube' of https://github.com/JonathanBever…
david942j Jun 13, 2018
b04dbff
fix doc
david942j Jun 13, 2018
4ac3f86
Improve test coverage slightly
JonathanBeverley Jun 13, 2018
79a8b33
break encapsulation to test recv_raw raise
JonathanBeverley Jun 13, 2018
33b2d21
bugfix: random_string(length) would sometimes return a shorter string
JonathanBeverley Jul 13, 2018
f6a724d
Merge branch 'master' into feature/serialtube
JonathanBeverley Jul 13, 2018
cf5f22e
explicitly reference ::Process to about name conflict with Pwnlib::Tu…
JonathanBeverley Jul 13, 2018
3add469
Merge branch 'feature/serialtube' of https://github.com/JonathanBever…
david942j Aug 22, 2018
910c0dc
verbose
david942j Aug 22, 2018
55a87c9
Don't build keystone..
david942j Aug 22, 2018
589dd09
recv is suspicious
david942j Aug 22, 2018
764ada5
guess I resolved the bug
david942j Aug 23, 2018
bda6f8b
Add wait after kill
david942j Aug 23, 2018
0f68f83
Clean debug messages
david942j Aug 23, 2018
8257d80
remove debug code
JonathanBeverley Aug 25, 2018
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions Gemfile.lock
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ PATH
method_source (~> 0.9)
rainbow (>= 2.2, < 4.0)
ruby2ruby (~> 2.4)
rubyserial (~> 0.5)

GEM
remote: https://www.rubygems.org/
Expand Down Expand Up @@ -50,6 +51,8 @@ GEM
sexp_processor (~> 4.6)
ruby_parser (3.11.0)
sexp_processor (~> 4.9)
rubyserial (0.5.0)
ffi (~> 1.9, >= 1.9.3)
sexp_processor (4.10.1)
simplecov (0.16.1)
docile (~> 1.1)
Expand Down
1 change: 1 addition & 0 deletions lib/pwnlib/pwn.rb
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
require 'pwnlib/logger'
require 'pwnlib/reg_sort'
require 'pwnlib/shellcraft/shellcraft'
require 'pwnlib/tubes/serialtube'
require 'pwnlib/tubes/sock'

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Put this line before 'pwnlib/tubes/sock'

require 'pwnlib/util/cyclic'
Expand Down
3 changes: 2 additions & 1 deletion lib/pwnlib/timer.rb
Original file line number Diff line number Diff line change
Expand Up @@ -54,8 +54,9 @@ def countdown(timeout = nil)
begin
yield
ensure
raise ::Pwnlib::Errors::TimeoutError unless active?
was_active = active?
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Copy link
Collaborator

@david942j david942j Jun 9, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for pointing out this,
it's fixed by #116

@deadline = nil
raise ::Pwnlib::Errors::TimeoutError unless was_active
end
end
end
Expand Down
107 changes: 107 additions & 0 deletions lib/pwnlib/tubes/serialtube.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
# encoding: ASCII-8BIT

require 'rubyserial'

require 'pwnlib/tubes/tube'

module Pwnlib
module Tubes
# Serial Connections
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

change here to

# @!macro [new] raise_eof
#   @raise [Pwnlib::Errors::EndOfTubeError]
#     If the request is not satisfied when all data is received.

# Serial Connections
class SerialTube < Tube

otherwise YARD will have wrong result

class SerialTube < Tube
# Instantiate a {Pwnlib::Tubes::SerialTube} object.
#
# @param [String] port
# A device name for rubyserial to open, e.g. /dev/ttypUSB0
# @param [Integer] baudrate
# Baud rate.
# @param [Boolean] convert_newlines
# If +true+, will convert local +context.newline+ to +"\\r\\n"+ for remote
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess no escape needed here?
+"\r\n"+ should be enough

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And this sentence seems not complete

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What I try it without the double escape, in the .html files generated by YARD, I see
If true, will MARKER convert local context.newline to "rn" for remote
I copied the syntax from tube.rb, which does the same thing. Note that in @example blocks the rules are different.

# @param [Integer] bytesize
# Serial character byte size. The '8' in '8N1'.
# @param [Symbol] parity
# Serial character parity. The 'N' in '8N1'.
def initialize(port = nil, baudrate: 115_200,
convert_newlines: true,
bytesize: 8, parity: :none)
super()

# go hunting for a port
port ||= Dir.glob('/dev/tty.usbserial*').first
port ||= '/dev/ttyUSB0'

@convert_newlines = convert_newlines
@conn = Serial.new(port, baudrate, bytesize, parity)
@serial_timer = Timer.new
end

# Closes the active connection
def close
@conn.close if @conn
@conn = nil
end

# Implementation of the methods required for tube
private

# Gets bytes over the serial connection until some bytes are received, or
# +@timeout+ has passed. It is an error for it to return no data in less
# than +@timeout+ seconds. It is ok for it to return some data in less
# time.
#
# @param [Integer] numbytes
# An upper limit on the number of bytes to get.
#
# @return [String]
# A string containing read bytes.
#
# @!macro raise_eof
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

def recv_raw(numbytes)
raise EOFError if @conn.nil?
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

raise ::Pwnlib::Errors::EndOfTubeError


@serial_timer.countdown do
data = ''
begin
while @serial_timer.active?
data += @conn.read(numbytes - data.length)
break unless data.empty?
sleep 0.1
end
# TODO: should we reverse @convert_newlines here?
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO(username)

return data
rescue RubySerial::Error
close
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Current tests don't cover this rescue
add tests and ensure 100% coverage

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't see any way to do this. open(2) checks for all the same errors as read(2). Nothing I've done will cause an error in the right place, without writing bad code.

If you know a way the test code can break Ruby's encapsulation model and directly invoke SerialTube.conn.close(), that could do it. But I really don't want to write a method to allow that, and honestly, if I did, I would then tighten up the error checking around @conn and close off that avenue as well.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found a way.

raise EOFError
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here

end
end
end

# Sends bytes over the serial connection. This call will block until all the bytes are sent or an error occurs.
#
# @param [String] data
# A string of the bytes to send.
#
# @return [Integer]
# The number of bytes successfully written.
#
# @!macro raise_eof
def send_raw(data)
raise EOFError if @conn.nil?
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same


data.gsub!(context.newline, "\r\n") if @convert_newlines
begin
return @conn.write(data)
rescue RubySerial::Error
close
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done.

raise EOFError
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here

end
end

# Sets the +timeout+ to use for subsequent +recv_raw+ calls.
#
# @param [Integer] timeout
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The type is Float?
And it's ok to not document private methods if there's nothing worth to be mentioned

def timeout_raw=(timeout)
@serial_timer.timeout = timeout
end
end
end
end
13 changes: 10 additions & 3 deletions lib/pwnlib/tubes/tube.rb
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,9 @@ def initialize(timeout: nil)
# @return [String]
# A string contains bytes received from the tube, or +''+ if a timeout occurred while
# waiting.
#
# @!macro raise_eof
# @!macro raise_timeout
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one isn't cross-file.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

my fault

def recv(num_bytes = nil, timeout: nil)
return '' if @buffer.empty? && !fillbuffer(timeout: timeout)
@buffer.get(num_bytes)
Expand Down Expand Up @@ -101,13 +104,17 @@ def recvpred(timeout: nil)

c = recv(1)

return '' if c.empty?
next if c.empty?
data << c
end
data.slice!(0..-1)
ensure
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why this?
Original code seems correct

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might be a disagreement on what the function is supposed to do. In my opinion, it should not return until the timeout expires unless it has a match with the predicate. Just because a particular recv(1) didn't return anything doesn't mean that one .5s later won't.

Swap from rescue->raise is different. That's because we must not unrecv(data) if we match the predicate. With ensure, we always unrecv.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But recv_raw is a block call
The situation you mentioned should never happen

# rubocop:disable Style/RescueStandardError
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And this must be changed back to ensure

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I finally get what I misunderstood about this code, but I'll raise it as an issue so it can be discussed properly.

# Justification: go home rubocop, you're drunk.
rescue
unrecv(data)
raise
end
# rubocop:enable Style/RescueStandardError
end
end

Expand Down Expand Up @@ -159,7 +166,7 @@ def recvuntil(delims, drop: false, timeout: nil)
while @timer.active?
s = recv(1)

return '' if s.empty?
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this change needed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No. I raised it in issue #91, and it was classified as "notbug".

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So please rollback it

next if s.empty?
matching << s

sidx = matching.size
Expand Down
1 change: 1 addition & 0 deletions pwntools.gemspec
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ Gem::Specification.new do |s|
s.add_runtime_dependency 'method_source', '~> 0.9'
s.add_runtime_dependency 'rainbow', '>= 2.2', '< 4.0'
s.add_runtime_dependency 'ruby2ruby', '~> 2.4'
s.add_runtime_dependency 'rubyserial', '~> 0.5'

# TODO(david942j): check why ruby crash during testing if upgrade minitest to 5.10.2/3
s.add_development_dependency 'minitest', '= 5.10.1'
Expand Down
139 changes: 139 additions & 0 deletions test/tubes/serialtube_test.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,139 @@
# encoding: ASCII-8BIT

require 'open3'

require 'test_helper'

require 'pwnlib/tubes/serialtube'

class SerialTest < MiniTest::Test
include ::Pwnlib::Tubes

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the serial tube intend to work on macOS?
If yes, let the tests pass on macOS (install socat)
If no, skip the test unless the environment is on Linux

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And skip on Windows as well

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't have access to a mac to test it on. It seems likely that it will work off-the-shelf.
How do I "install socat"?

rubyserial is supported on Windows, and so it's possible that serialtube will work there. I don't have a Windows/ruby setup in-place, nor do I have any idea how to test it there.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To install socat, simply add brew install socat in travis/install.sh#setup_osx

Is that possible to test serialtube on Windows? I said skip on Windows because the tests you added (socat ...) can't be supported on Windows.

If serialtube is supposed to support on Windows, maybe you should try another testing method to make it work on Windows as well.
Otherwise, just forget Windows supports and skip the tests.

def skip_windows
skip 'Not test tube/serialtube on Windows' if TTY::Platform.new.windows?
end

def open_pair
Open3.popen3('socat -d -d pty,raw,echo=0 pty,raw,echo=0') do |_i, _o, stderr, thread|
devs = []
2.times do
devs << stderr.readline.chomp.split.last
# First pattern matches Linux, second is macOS
raise IOError, 'Could not create serial crosslink' if devs.last !~ %r{^(/dev/pts/[0-9]+|/dev/ttys[0-9]+)$}
end

serial = SerialTube.new devs[1], convert_newlines: false

begin
File.open devs[0], 'r+' do |file|
file.set_encoding 'default'.encoding
yield file, serial
end
ensure
Process.kill('SIGTERM', thread.pid)
end
end
end

def random_string(length)
Random.rand(36**length).to_s(36).encode('default'.encoding)
end

def test_recv
skip_windows
open_pair do |file, serial|
# recv, recvline
rs = random_string 24
file.puts rs
result = serial.recv 8, timeout: 1

assert_equal(rs[0...8], result)
result = serial.recv 8
assert_equal(rs[8...16], result)
result = serial.recvline.chomp
assert_equal(rs[16..-1], result)

assert_raises(Pwnlib::Errors::TimeoutError) { serial.recv(1, timeout: 0.2) }

# recvpred
rs = random_string 12
file.print rs
result = serial.recvpred do |data|
data[-6..-1] == rs[-6..-1]
end
assert_equal rs, result

assert_raises(Pwnlib::Errors::TimeoutError) { serial.recv(1, timeout: 0.2) }

# recvn
rs = random_string 6
file.print rs
result = ''
assert_raises(Pwnlib::Errors::TimeoutError) do
result = serial.recvn 120, timeout: 1
end
assert_empty result
file.print rs
result = serial.recvn 12
assert_equal rs * 2, result

assert_raises(Pwnlib::Errors::TimeoutError) { serial.recv(1, timeout: 0.2) }

# recvuntil
rs = random_string 12
file.print rs + '|'
result = serial.recvuntil('|').chomp('|')
assert_equal rs, result

assert_raises(Pwnlib::Errors::TimeoutError) { serial.recv(1, timeout: 0.2) }

# gets
rs = random_string 24
file.puts rs
result = serial.gets 12
assert_equal rs[0...12], result
result = serial.gets.chomp
assert_equal rs[12..-1], result

assert_raises(Pwnlib::Errors::TimeoutError) { serial.recv(1, timeout: 0.2) }
end
end

def test_send
skip_windows
open_pair do |file, serial|
# send, sendline
rs = random_string 24
# rubocop:disable Style/Send
# Justification: This isn't Object#send, false positive.
serial.send rs[0...12]
# rubocop:enable Style/Send
serial.sendline rs[12...24]
result = file.readline.chomp
assert_equal rs, result

# puts
r1 = random_string 4
r2 = random_string 4
r3 = random_string 4
serial.puts r1, r2, r3
result = ''
3.times do
result += file.readline.chomp
end
assert_equal r1 + r2 + r3, result
end
end

def test_close
skip_windows
open_pair do |_file, serial|
serial.close
assert_raises(EOFError) { serial.puts(514) }
assert_raises(EOFError) { serial.puts(514) }
assert_raises(EOFError) { serial.recv }
assert_raises(EOFError) { serial.recv }
assert_raises(ArgumentError) { serial.close(:hh) }
end
end
end
3 changes: 3 additions & 0 deletions travis/install.sh
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,9 @@ setup_osx()
install_keystone_from_source
ln -s keystone/build/llvm/lib/libkeystone.dylib libkeystone.dylib # hack, don't know why next line has no effect
# export DYLD_LIBRARY_PATH=$TRAVIS_BUILD_DIR/keystone/build/llvm/lib:$DYLD_LIBRARY_PATH

# install socat
brew install socat
}

if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then
Expand Down