Parsing rspec descriptions results in, invalid byte sequence in UTF-8 (ArgumentError)

[pawelduda]

Having certain unescaped string sequences in either spec description or as an argument for shared example, like so will produce an error:

describe "\x96" do
end

it_behaves_like 'fetching json value', "A\x96Z", 'A-Z'

/home/kogut/.rvm/gems/ruby-2.2.4/gems/rspec-core-3.4.1/lib/rspec/core/metadata.rb:166:in `description_separator': invalid byte sequence in UTF-8 (ArgumentError)
    from /home/kogut/.rvm/gems/ruby-2.2.4/gems/rspec-core-3.4.1/lib/rspec/core/metadata.rb:310:in `full_description'
    from /home/kogut/.rvm/gems/ruby-2.2.4/gems/rspec-core-3.4.1/lib/rspec/core/metadata.rb:128:in `populate'
    from /home/kogut/.rvm/gems/ruby-2.2.4/gems/rspec-core-3.4.1/lib/rspec/core/metadata.rb:250:in `create'
    from /home/kogut/.rvm/gems/ruby-2.2.4/gems/rspec-core-3.4.1/lib/rspec/core/example_group.rb:410:in `set_it_up'
    from /home/kogut/.rvm/gems/ruby-2.2.4/gems/rspec-core-3.4.1/lib/rspec/core/example_group.rb:384:in `subclass'
    from /home/kogut/.rvm/gems/ruby-2.2.4/gems/rspec-core-3.4.1/lib/rspec/core/example_group.rb:255:in `block in define_example_group_method'
    from /home/kogut/mutation_tests_example/spec/rectangle_spec.rb:4:in `block in <top (required)>'
    from /home/kogut/.rvm/gems/ruby-2.2.4/gems/rspec-core-3.4.1/lib/rspec/core/example_group.rb:385:in `module_exec'
    from /home/kogut/.rvm/gems/ruby-2.2.4/gems/rspec-core-3.4.1/lib/rspec/core/example_group.rb:385:in `subclass'
    from /home/kogut/.rvm/gems/ruby-2.2.4/gems/rspec-core-3.4.1/lib/rspec/core/example_group.rb:255:in `block in define_example_group_method'
    from /home/kogut/.rvm/gems/ruby-2.2.4/gems/rspec-core-3.4.1/lib/rspec/core/dsl.rb:43:in `block in expose_example_group_alias'
    from /home/kogut/.rvm/gems/ruby-2.2.4/gems/rspec-core-3.4.1/lib/rspec/core/dsl.rb:82:in `block (2 levels) in expose_example_group_alias_globally'
    from /home/kogut/mutation_tests_example/spec/rectangle_spec.rb:3:in `<top (required)>'
    from /home/kogut/.rvm/gems/ruby-2.2.4/gems/rspec-core-3.4.1/lib/rspec/core/configuration.rb:1361:in `load'
    from /home/kogut/.rvm/gems/ruby-2.2.4/gems/rspec-core-3.4.1/lib/rspec/core/configuration.rb:1361:in `block in load_spec_files'
    from /home/kogut/.rvm/gems/ruby-2.2.4/gems/rspec-core-3.4.1/lib/rspec/core/configuration.rb:1359:in `each'
    from /home/kogut/.rvm/gems/ruby-2.2.4/gems/rspec-core-3.4.1/lib/rspec/core/configuration.rb:1359:in `load_spec_files'
    from /home/kogut/.rvm/gems/ruby-2.2.4/gems/rspec-core-3.4.1/lib/rspec/core/runner.rb:102:in `setup'
    from /home/kogut/.rvm/gems/ruby-2.2.4/gems/mutant-rspec-0.8.8/lib/mutant/integration/rspec.rb:50:in `setup'
    from /home/kogut/.rvm/gems/ruby-2.2.4/gems/memoizable-0.4.2/lib/memoizable/method_builder.rb:117:in `call'
    from /home/kogut/.rvm/gems/ruby-2.2.4/gems/memoizable-0.4.2/lib/memoizable/method_builder.rb:117:in `block (3 levels) in create_memoized_method'
    from /home/kogut/.rvm/gems/ruby-2.2.4/gems/memoizable-0.4.2/lib/memoizable/memory.rb:63:in `block (3 levels) in fetch'
    from /home/kogut/.rvm/gems/ruby-2.2.4/gems/thread_safe-0.3.5/lib/thread_safe/cache.rb:58:in `fetch'
    from /home/kogut/.rvm/gems/ruby-2.2.4/gems/memoizable-0.4.2/lib/memoizable/memory.rb:62:in `block (2 levels) in fetch'
    from /home/kogut/.rvm/rubies/ruby-2.2.4/lib/ruby/2.2.0/monitor.rb:211:in `mon_synchronize'
    from /home/kogut/.rvm/gems/ruby-2.2.4/gems/memoizable-0.4.2/lib/memoizable/memory.rb:61:in `block in fetch'
    from /home/kogut/.rvm/gems/ruby-2.2.4/gems/thread_safe-0.3.5/lib/thread_safe/cache.rb:58:in `fetch'
    from /home/kogut/.rvm/gems/ruby-2.2.4/gems/memoizable-0.4.2/lib/memoizable/memory.rb:60:in `fetch'
    from /home/kogut/.rvm/gems/ruby-2.2.4/gems/memoizable-0.4.2/lib/memoizable/method_builder.rb:116:in `block (2 levels) in create_memoized_method'
    from /home/kogut/.rvm/gems/ruby-2.2.4/gems/mutant-0.8.10/lib/mutant/env/bootstrap.rb:97:in `infect'
    from /home/kogut/.rvm/gems/ruby-2.2.4/gems/mutant-0.8.10/lib/mutant/env/bootstrap.rb:35:in `initialize'
    from /home/kogut/.rvm/gems/ruby-2.2.4/gems/adamantium-0.2.0/lib/adamantium/class_methods.rb:17:in `new'
    from /home/kogut/.rvm/gems/ruby-2.2.4/gems/adamantium-0.2.0/lib/adamantium/class_methods.rb:17:in `new'
    from /home/kogut/.rvm/gems/ruby-2.2.4/gems/procto-0.0.3/lib/procto.rb:62:in `block in initialize'
    from /home/kogut/.rvm/gems/ruby-2.2.4/gems/mutant-0.8.10/lib/mutant/cli.rb:18:in `run'
    from /home/kogut/.rvm/gems/ruby-2.2.4/gems/mutant-0.8.10/bin/mutant:39:in `<top (required)>'
    from /home/kogut/.rvm/gems/ruby-2.2.4/bin/mutant:23:in `load'
    from /home/kogut/.rvm/gems/ruby-2.2.4/bin/mutant:23:in `<main>'
    from /home/kogut/.rvm/gems/ruby-2.2.4/bin/ruby_executable_hooks:15:in `eval'
    from /home/kogut/.rvm/gems/ruby-2.2.4/bin/ruby_executable_hooks:15:in `<main>'

Apparently after some debugging it turns out that Mutant seems to read all spec descriptions for the entire project during initialization, even though I am giving it an explicit class that I want to test. That's why the error was hard to find at first, because I was getting it on a project with hundreds of different classes and 2500~ specs. Only after digging some through the stack trace I managed to print out all spec descriptions that were coming in and find the proper file by checking which file was the last one to provide some descriptions.

So this leads me to a side question: is reading all specs in the project necessary when I only want to run Mutant on a certain class? Not doing this could perhaps lead to some performance improvements. Please correct me if I am wrong, I am not yet that familiar with the Mutant codebase and these are only my assumptions :)

Code to reproduce the error:

rectangle.rb

Point = Struct.new(:x, :y);

class Rectangle
  def initialize(a, b)
    @a = a
    @b = b
  end

  def perimeter
    2 * (length + breadth)
  end

  private

  def length
    (@b.x - @a.x).abs
  end

  def breadth
    (@b.y - @a.y).abs
  end
end

spec/rectangle_spec.rb

require_relative '../rectangle'

describe Rectangle do
  describe "\x96" do
    let(:a) { Point.new(50, 50) }
    let(:b) { Point.new(100, 100) }

    subject { described_class.new(a, b).perimeter }

    it { is_expected.to eq 200 }
  end
end

Temporary workarounds:

1. Exclude the problematic spec file from the test suite, by renaming or commenting it out
2. Fix the encoding within problematic string like so:

describe "\x96".encode('UTF-8', invalid: :replace) do

[mbj]

I do not have time to reply in depth now, a similar issue is discussed here: #357

Also: Mutant needs to load all rspec examples for various reasons I do not have the time to explain right now. At least I can tell you its intended ;)

[mbj]

Related parser issue (actually a question): whitequark/parser#213

[mbj]

@pawelduda I've recently spend a longer time thinking about your case. In conclusion, its actually not mutants bug / problem.

Reason being: Its an RSpec API that raises the exception. Yes, mutants interaction with RSpec surface the problem. Mutant calls ExampleGroup#full_description and this raises in rspec on your examples.

IMO having non printable examples (they could be printed via rspec arguments like --format description) is nothing mutant should workaround in any way.

My stance is: Mutant should work with "correct" rspec tests, not add extra logic to workaround "invalid strings in rspec descriptions".

I'd be willing to re-visit this. But for the moment I'll close this issue. Feel free to discuss my arguments.