When can function pointers be transmuted?

[RalfJung]

On Zulip, @ubsan was asking about function pointer transmutes:

Can you transmute<fn(*const T), fn(*const ())>(x)(p)?

(I assume we have T: Sized here.)

I would say we can modularize the problem (but this goes deep into validity invariant territory):

1. The actual cast is always okay, because the validity invariant for function pointers does not talk about the signature. Maybe it just says "non-NULL", maybe it also says "points to executable memory", but I think it should not take the signature into account.
2. The call, then, is when things may become UB if the signatures are wrong. We probably need some definition of "ABI compatible" here: Which "mismatches" we allow between the caller and callee signature will depend not just on layout but also on ABI concerns. For example, CTFE/miri currently say that either the type must be exactly the same, or else both types must have Scalar ABI that may differ only in its valid_range. In your example, the second condition is satisfied. If the mismatch is not allowed, we have UB.
3. The behavior of allowed argument mismatches, then, is the same as that of a transmute, of union-based type punning, and of type punning through changing pointer types: In your case, I would say your code is equivalent to x(mem::transmute<*const T, *const ()>(p)).

(Taken from #45 (comment) )

[rodrimati1992]

It would be very valuable if this were defined,because I need to cast extern fn(&T) to extern fn(&SomeZSTStruct) to implement ffi-safe trait objects.

I may be able to get around this problem by just taking a struct that always erases the type:

/**
This takes a *const R/*mut R/&R/&mut R as a type parameter.

Here are examples of function types taking it:
    -extern fn(Erase<&T>)
    -extern fn(Erase<&mut T>)
    -extern fn(Erase<*const T>)
    -extern fn(Erase<*mut T>)
*/
#[repr(transparent)]
pub struct Erase<T> {
    pointer: *const c_void,
    type_: PhantomData<T>,
}

[rosekunkel]

I would also like to know this. More specifically, when can unsafe function pointers be transmuted into safe ones? That is, which (if any) of these are valid:

mem::transmute::<unsafe extern "Rust" fn(A) -> B, extern "Rust" fn(A) -> B>(foo)
mem::transmute::<unsafe extern "C" fn(A) -> B, extern "C" fn(A) -> B>(foo)
mem::transmute::<unsafe extern "C" fn(A, ...) -> B, extern "C" fn(A, ...) -> B>(foo)

I would expect at least the extern "C" casts to be valid.

[bjorn3]

I would expect it to be valid, but only sound when you now that the function is always safe to call.

[RalfJung]

Yeah, unsafe does not affect the call ABI in any way.

[rosekunkel]

It would be nice if there was a less error-prone way to do this than just mem::transmute, like a rule that allows you to cast unsafe function pointers to safe ones inside an unsafe block, much like you can call unsafe functions in unsafe blocks.

[JakobDegen]

For triage: Clarifying that we are interpreting this as a question about validity of function pointers, not about ABI.

[RalfJung]

This was closed as a duplicate of #72 and #280