Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement LocatedSpan::get_line(). #66

Merged
merged 10 commits into from
Oct 18, 2020
Merged
56 changes: 52 additions & 4 deletions src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -257,17 +257,28 @@ impl<T: AsBytes, X> LocatedSpan<T, X> {
&self.fragment
}

fn get_columns_and_bytes_before(&self) -> (usize, &[u8]) {
// Attempt to get the "original" data slice back, by extending
// self.fragment backwards by self.offset.
// Note that any bytes truncated from after self.fragment will not
// be recovered.
fn get_unoffsetted_slice(&self) -> &[u8] {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, the name doesn't make it clear what this does, but I can't think of a better one. :/
Also some comment to explain what it does would be good

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I considered get_original_size, as that is pretty much the intent. But I didn't, since it has no way of reconstructing the original length. So unoffsetted is what it actually does, it undos the offset.

As for the larger question about a possibly missing trailer of get_line(); Yes, I was a bit worried about that myself, but decided that it's not a big problem in my main use-case of reporting parse errors. In cases I can think of that involves marking an interval (and not just a position) of a line, I think I would have two LocatedSpans to combine, where each of them would (probably) be "the rest of input from a starting point".

But I agree that this should be explained somehow, both in i a comment at get_unoffsetted_slice and in the docstring of get_line. I'll try to write something.

let self_bytes = self.fragment.as_bytes();
let self_ptr = self_bytes.as_ptr();
let before_self = unsafe {
unsafe {
assert!(
self.offset <= isize::max_value() as usize,
"offset is too big"
);
let orig_input_ptr = self_ptr.offset(-(self.offset as isize));
slice::from_raw_parts(orig_input_ptr, self.offset)
};
slice::from_raw_parts(
orig_input_ptr,
self.offset + self_bytes.len(),
)
}
}

fn get_columns_and_bytes_before(&self) -> (usize, &[u8]) {
let before_self = &self.get_unoffsetted_slice()[..self.offset];

let column = match memchr::memrchr(b'\n', before_self) {
None => self.offset + 1,
Expand All @@ -277,6 +288,43 @@ impl<T: AsBytes, X> LocatedSpan<T, X> {
(column, &before_self[self.offset - (column - 1)..])
}

/// Return the line that contains this LocatedSpan.
///
/// The `get_column` and `get_utf8_column` functions returns
/// indexes that corresponds to the line returned by this function.
///
/// Note that if this LocatedSpan ends before the end of the
/// original data, the result of calling `get_line_beginning()`
/// will not include any data from after the LocatedSpan.
///
/// ```
/// # extern crate nom_locate;
/// # extern crate nom;
/// # use nom_locate::LocatedSpan;
/// # use nom::{Slice, FindSubstring};
/// #
/// # fn main() {
/// let program = LocatedSpan::new(
/// "Hello World!\
/// \nThis is a multi-line input\
/// \nthat ends after this line.\n");
/// let multi = program.find_substring("multi").unwrap();
///
/// assert_eq!(
/// program.slice(multi..).get_line_beginning(),
/// "This is a multi-line input".as_bytes(),
/// );
/// # }
/// ```
pub fn get_line_beginning(&self) -> &[u8] {
let column0 = self.get_column() - 1;
let the_line = &self.get_unoffsetted_slice()[self.offset - column0..];
match memchr::memchr(b'\n', &the_line[column0..]) {
None => the_line,
Some(pos) => &the_line[..column0 + pos],
}
}

/// Return the column index, assuming 1 byte = 1 column.
///
/// Use it for ascii text, or use get_utf8_column for UTF8.
Expand Down
106 changes: 106 additions & 0 deletions src/tests.rs
Original file line number Diff line number Diff line change
Expand Up @@ -414,3 +414,109 @@ fn it_should_display_hex() {
"00000000\t61 62 63 \tabc\n".to_owned()
);
}

#[test]
fn line_of_empty_span_is_empty() {
assert_eq!(StrSpan::new("").get_line_beginning(), "".as_bytes());
}

#[test]
fn line_of_single_line_start_is_whole() {
assert_eq!(
StrSpan::new("A single line").get_line_beginning(),
"A single line".as_bytes(),
);
}
#[test]
fn line_of_single_line_end_is_whole() {
let data = "A single line";
assert_eq!(
StrSpan::new(data).slice(data.len()..).get_line_beginning(),
"A single line".as_bytes(),
);
}

#[test]
fn line_of_start_is_first() {
assert_eq!(
StrSpan::new(
"One line of text\
\nFollowed by a second\
\nand a third\n"
)
.get_line_beginning(),
"One line of text".as_bytes(),
);
}

#[test]
fn line_of_nl_is_before() {
let data = "One line of text\
\nFollowed by a second\
\nand a third\n";
assert_eq!(
StrSpan::new(data)
.slice(data.find('\n').unwrap()..)
.get_line_beginning(),
"One line of text".as_bytes(),
);
}

#[test]
fn line_of_end_after_nl_is_empty() {
let data = "One line of text\
\nFollowed by a second\
\nand a third\n";
assert_eq!(
StrSpan::new(data).slice(data.len()..).get_line_beginning(),
"".as_bytes(),
);
}

#[test]
fn line_of_end_no_nl_is_last() {
let data = "One line of text\
\nFollowed by a second\
\nand a third";
assert_eq!(
StrSpan::new(data).slice(data.len()..).get_line_beginning(),
"and a third".as_bytes(),
);
}

/// This test documents how `get_line_beginning()` differs from
/// a hypotetical `get_line()` method.
#[test]
fn line_begining_may_ot_be_entire_len() {
let data = "One line of text\
\nFollowed by a second\
\nand a third";
let by = "by";
let pos = data.find_substring(by).unwrap();
assert_eq!(
StrSpan::new(data).slice(pos..pos+by.len()).get_line_beginning(),
"Followed by".as_bytes(),
);
}

#[cfg(feature = "std")]
#[test]
fn line_for_non_ascii_chars() {
let data = StrSpan::new(
"Några rader text på Svenska.\
\nFörra raden var först, den här är i mitten\
\noch här är sista raden.\n",
);
let s = data.slice(data.find_substring("först").unwrap()..);
assert_eq!(
format!(
"{line_no:3}: {line_text}\n {0:>lpos$}^- The match\n",
"",
line_no = s.location_line(),
line_text = core::str::from_utf8(s.get_line_beginning()).unwrap(),
lpos = s.get_utf8_column(),
),
" 2: Förra raden var först, den här är i mitten\
\n ^- The match\n",
);
}