Fix tabs having a fixed width internally

[karyon]

Ref #4968. Tabs were always counted to have a width of config.tab_spaces.
Since they actually can have less than that depending on the line's
previous contents, rustfmt would sometimes complain about the line
length even if the line did actually fit.

[MitMaro]

Looks like this still fails if there is more than one tab character in a line.

[calebcartwright]

@MitMaro - thanks for sharing! sounds straightforward enough, but do you have a snippet you'd be willing to share? that makes adding a corresponding test case a bit easier

[MitMaro]

Here's a test case (based on the one above) that still causes a panic:

// rustfmt-error_on_unformatted: true
// rustfmt-error_on_line_overflow: true

// Part of #4968. This line has a width of 100, so it should be fine, but rustfmt panicked.
fn panic_with_tabs() {
    let a = "tab here:		Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonu est \
             est, consetetur sadipscing";
}

Here's the stack:

thread '<unnamed>' panicked at 'SourceAnnotation range `(100, 104)` is bigger than source length `100`', /home/mitmaro/.cargo/registry/src/gitpro.ttaallkk.top-1ecc6299db9ec823/annotate-snippets-0.8.0/src/display_list/from_snippet.rs:273:9
stack backtrace:
   0: rust_begin_unwind
             at /rustc/1af55d19c7a9189374d89472f97dc119659bb67e/library/std/src/panicking.rs:517:5
   1: std::panicking::begin_panic_fmt
             at /rustc/1af55d19c7a9189374d89472f97dc119659bb67e/library/std/src/panicking.rs:460:5
   2: annotate_snippets::display_list::from_snippet::format_body
             at /home/mitmaro/.cargo/registry/src/gitpro.ttaallkk.top-1ecc6299db9ec823/annotate-snippets-0.8.0/src/display_list/from_snippet.rs:273:9
   3: annotate_snippets::display_list::from_snippet::format_slice
             at /home/mitmaro/.cargo/registry/src/gitpro.ttaallkk.top-1ecc6299db9ec823/annotate-snippets-0.8.0/src/display_list/from_snippet.rs:114:20
   4: annotate_snippets::display_list::from_snippet::<impl core::convert::From<annotate_snippets::snippet::Snippet> for annotate_snippets::display_list::structs::DisplayList>::from
             at /home/mitmaro/.cargo/registry/src/gitpro.ttaallkk.top-1ecc6299db9ec823/annotate-snippets-0.8.0/src/display_list/from_snippet.rs:487:30
   5: <rustfmt_nightly::format_report_formatter::FormatReportFormatter as core::fmt::Display>::fmt
             at ./src/format_report_formatter.rs:95:37
   6: core::fmt::write
             at /rustc/1af55d19c7a9189374d89472f97dc119659bb67e/library/core/src/fmt/mod.rs:1163:17
   7: std::io::Write::write_fmt
             at /rustc/1af55d19c7a9189374d89472f97dc119659bb67e/library/std/src/io/mod.rs:1696:15
   8: std::io::stdio::print_to::{{closure}}::{{closure}}
             at /rustc/1af55d19c7a9189374d89472f97dc119659bb67e/library/std/src/io/stdio.rs:1187:25
   9: core::option::Option<T>::map
             at /rustc/1af55d19c7a9189374d89472f97dc119659bb67e/library/core/src/option.rs:848:29
  10: std::io::stdio::print_to::{{closure}}
             at /rustc/1af55d19c7a9189374d89472f97dc119659bb67e/library/std/src/io/stdio.rs:1186:13
  11: std::thread::local::LocalKey<T>::try_with
             at /rustc/1af55d19c7a9189374d89472f97dc119659bb67e/library/std/src/thread/local.rs:399:16
  12: std::io::stdio::print_to
             at /rustc/1af55d19c7a9189374d89472f97dc119659bb67e/library/std/src/io/stdio.rs:1182:12
  13: std::io::stdio::_print
             at /rustc/1af55d19c7a9189374d89472f97dc119659bb67e/library/std/src/io/stdio.rs:1209:5
  14: rustfmt_nightly::test::check_files
             at ./src/test/mod.rs:540:17
  15: rustfmt_nightly::test::idempotence_tests::{{closure}}
             at ./src/test/mod.rs:320:40

[MitMaro]

Another test case, and one I reduced from one of my projects that still causes a crash in rustfmt is leading tabs with hard_tabs: true.

// rustfmt-error_on_unformatted: true
// rustfmt-error_on_line_overflow: true
// rustfmt-hard_tabs: true

fn panic_with_tabs() {
	let a = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx";
}

thread '<unnamed>' panicked at 'SourceAnnotation range `(100, 105)` is bigger than source length `102`', /home/mitmaro/.cargo/registry/src/gitpro.ttaallkk.top-1ecc6299db9ec823/annotate-snippets-0.8.0/src/display_list/from_snippet.rs:273:9

[karyon]

Right. I didn't fix the entire bug, only those (few) cases where the line would actually fit into the width limit. See my comment here: #4968 (comment)

[camsteffen]

This 1 can also cause issues with non-ascii idents. Could be c.len_utf8().

[camsteffen]

Another possible fix for tabs is to use self.line_buffer.push_str(&" ".repeat(self.config.tab_spaces())). Not sure which is correct.

[drahnr]

@karyon @camsteffen gentle ping

[camsteffen]

@drahnr neither of us can do anything to advance this PR AFAICT. The maintainers have to action on it.