librustc: Apply null pointer optimization to slices, closures & trait objects.

[luqmana]

Extend the null ptr optimization to work with slices, closures, procs, & trait objects by using the internal pointers as the discriminant.

This decreases the size of Option<&[int]> (and similar) by one word.

[lilyball]

My stalled-out PR #14436 has a commit that relied on changing this to always return self.ptr. The reason given is that otherwise Vec::from_raw_parts(v.len(), v.cap(), v.as_mut_ptr()) will be incorrect. If at all possible, it would be nice to make that work while you're changing things.

My preference would be to make as_ptr() and as_mut_ptr() return null whenever appropriate, and instead do the null-test when constructing the slice.

[thestinger]

The Vec<T> pointer is intended to always be non-null when the length is 0 and the only necessary change should be in the constructor functions. It will never pass the pointer to deallocate with a zero length.

[lilyball]

@thestinger That can't be true, because a brand new Vec::new() has a null pointer. It's only non-null once it's capacity is nonzero. And preserving the behavior of Vec::new() starting with zero capacity is important.

[thestinger]

As I said, the only required change is to the constructor functions. It can use an arbitrary non-null value because it will never pass it to deallocate. Passing a null pointer to deallocate has undefined behaviour and will likely cause a null pointer dereference at the moment.

[lilyball]

@thestinger What benefit is there to having Vec use a non-null value for its pointer field? Right now, to the best of my knowledge, its pointer field is always valid (or null).

Slice should just deal with non-null itself.

That said, having Vec always have a non-null pointer and therefore always returning its pointer field from as_ptr() and as_mut_ptr() would resolve my concern, and so is better than what we're doing today and what this PR is doing, I just think it would be great if the need for a non-null Slice pointer could avoid leaking outside of Slice.

[thestinger]

The non-null enum optimization can and should be extended to library types like Rc<T> and Vec<T>. It doesn't benefit in any way from the pointer being null and the code will be simpler if it's guaranteed not to be.

[lilyball]

This comment is misleading. It suggests the non-null pointer is purely because of 0-sized types (not 0-sized vectors!), but one would expect a size_of() test in that case. It should be rewritten to explain that Slice wants to never have a null pointer so it can benefit from null pointer optimization, and the easiest way to do that is to make Vec never have a null pointer.