Behavior of typed arrays

[vnen]

The addition of typed arrays seems to be welcomed by many people, but unfortunately we (mostly I) did not have the proper time to polish before 4.0 was out (and now 4.1). There are some issues that depends a lot on how we expect typed arrays to behave, as they have some quirks compared to regular untyped ones, and the we didn't reach a consensus on how they should work. There are multiple conditions that aren't clear which would be the most intuitive and thus the best option.

So I'm listing here a few things that are problematic with typed arrays, especially in relation to what's expected from untyped ones, and the options to solve the problems (or, rather, the trade-offs of each option).

This has become quite long, sorry. I wanted to be thorough in the examples and conditions so it's in everyone's mind when potentially making a final decision.

Equality

Arrays are compared by value:

var a = [1, 2, 3]
var b = [1, 2, 3]
print(a == b) # true
print(is_same(a, b)) # false

They aren't the same reference but have the same contents, so they are considered equal. In this case, both are Variant arrays, so they have the same type, the question arises for typed arrays when they don't share the type:

var a: Array[int] = [1, 2, 3]
var b: Array[Variant] = [1, 2, 3]
print(a == b) # ??

Assuming they are still compared by value, there are two options:

1. Ignore the type. If the contents are the same, they are said to be equal.
2. Include the type. If the types are different, they are never equal.

Consider also this for the empty array case:

var a: Array[int] = []
var b: Array[String] = []
print(a == b) # ??

If ignoring the type, they are equal, otherwise they are different. Also, consider that even if a == b is true, a = b is an invalid assignment given the incompatible types.

This can be used in the of checking if the array is empty, or if it contains specific elements:

var a: Array[int] = []
print(a == []) # ??
var b: Array[int] = [1, 2, 3]
print(a == [1, 2, 3]) # ??

The weirdest case, while still technically possible, is when using arrays as dictionary keys:

var arr1: Array[int] = []
var arr2: Array[int] = [1, 2, 3]
var dict := {arr1: 0, arr2: 3}

# Are the elements found or not?
print(dict[[]])
print(dict[[1, 2, 3]])

Note that there is a proposal for typed array constructors, which can mitigate some of this (by requiring you to use them), but will also incur in an extra array copy just for the comparison which is quite an overhead.

Regardless of the choice, we can always provide a function to do the other type of comparison. While inconvenient it works for most cases (not for dictionary keys for instance). It should be the least used option to become a function though.

Assignment

This is probably the most difficult to grasp and solve. Let me try to show it with an example:

var sprite := Sprite2D.new()
var a: Array[Node] = [sprite]
var b: Array[Sprite2D]

b = a # What should happen here?

Consider that arrays are passed by reference. Given the underlying reference (a) carries elements of type Node, not all of them can fit into the type of b. In this example they do, but there's no future guarantee that it will stay so. Further restricting the underlying reference is not feasible because it is still of type Array[Node] and you wouldn't expect it to fail to add Node of some other type.

Also, consider the case a = b. All elements of b can be included in a given the types. However, since the underlying reference is still Array[Sprite2D], you cannot add other types of Node, even when using a as the base. This can become confusing since the compiler would be happy type-wise but it would fail at runtime.

For one more example, take this:

var a: Array[Node] = [Node.new()]
var b: Array

b = a

In this case, b is untyped. This can be treated the same way as the previous example if you think an untyped array is the same as Array[Variant]. But given Variant itself is not an actual type, just a catch all for any type, it is a bit special, so it could be treated differently. Even so, if assigning a non-Node to b in this example would also compile fine type-wise but fail at runtime.

Given this, we have a few options:

1. Allow the reference. This option includes adding runtime checks to ensure the type are always correct when it matter (like auto-casting when accessing elements). Probably the worst in terms of performance and does add complexity.
2. Forbid the reference. The easiest to do. Might be inconvenient in some cases though, but also gives obvious error messages. This will force the user to make an explicit copy on assignment, as I explain at the end of this section.
3. Create a copy. This is quite controversial, but it would allow assignment between different (but compatible) array types, without creating the clash of references. However, it would be confusing if you expect it still to be a reference.

Note that the examples are very specific which looks obvious but there other nuances especially when considering the boundaries between typed and untyped code (where GDScript cannot see the original type of the value. For instance:

extends Node
class_name CustomNode

var a: Array[Node] = []

# Somewhere else.
extends Node
func _ready():
    var node = $CustomNode # Assume it's in the tree.
    var tmp: Array = node.a # What happens here? The compiler does not know what `node.a` is.

One can assume that since node.a is ultimately an Array, the assignment should be allowed. Or you can think that tmp is an Array[Variant] and given the type mismatch the assignment should fail (all of this at runtime given the lack of information at compile time).

This also applies to the reverse case, when assigning to the property, the type of the array might be conflicting, and this might happen in completely untyped code when interfacing with the engine API or typed addons:

# Somewhere else.
extends Node
func _ready():
	var node = $CustomNode
	node.a = [self] # Untyped (or Variant) array assigned to Array[Node]

Given GDScript is still dynamically typed, this can become quite inconvenient if forbidden.

Note again that typed array constructors will eventually be included, so "forbidden" can also mean "make an explicit copy":

# Somewhere else.
extends Node
func _ready():
	var node = $CustomNode
	node.a = Array[Node]([self])

This forces the user of untyped code to think a bit about types, which is not ideal, but it gives a compromise. There could be another way to create an explicit copy without actually spelling out the types.

Arguments

Similar to assignment, there's the question of how to treat array types when passing them as function arguments.

The options are the same as the ones for assignment and arguably should use the same solution for consistency. But these cases can happen more often in untyped code than with properties, so it has the potential of becoming more annoying in general.

To also keep this section with an example, consider this:

extends Node
class_name CustomNode

func foo(arr: Array[Node]) -> void:
	print(arr)

extends Node
func _ready():
	var custom = $CustomNode
	custom.foo([self]) # Is this allowed, create an implicit copy, or requires explicit copy?

While this is pretty much the same as the assignment, I wanted to make it clear that this case needs to be considered as well when making the solution.

---

Conclusion

While sometimes it might seem "obvious" what the proper behavior should be, there are a lot of nuances and special cases hard to notice in a cursory look. I think it's important we consider the potential cases before jumping into a solution.

Also keep in that this lays some groundwork for potential future features like typed dictionaries and, perhaps, full-fledged generics (not promising anything).

I want to hear the thoughts of the community about this so please do comment with your preferred solutions and reasons.

[AThousandShips]

I would lean toward making comparison of typed arrays require type equality, regardless of empty, but untyped not:

var arr1: Array[int] = []
var arr2: Array[String] = []
print(arr1 == arr2) # False
print(arr1 == []) # True
print(arr2 == []) # True

Treating typedness as a "weak trait"

[vnen]

Note that this case:

print(arr1 == []) # True

The [] is of type Array[Variant] technically (or just Array, but it's mostly the same). So it's not entirely "untyped". For instance, consider this:

var arr1: Array[int] = []
var arr2: Array[Variant] = []
print(arr1 == arr2) # ??

There's really no difference between arr1 == arr2 and arr1 == [] in this case.

[AThousandShips]

Technically but it is untyped in the sense that is_typed is false, and for Array[Variant] that returns false

That's what I mean by "typed" here, it'd be quite confusing if is_typed() == false and "untyped" aren't equivalent

[vnen]

I do not think that much about the underlying implementation. I'm mostly thinking about the user-facing side. For the user Array[Variant] is a type like any other, so why should it behave differently than, say, Array[String]? Having differing behaviors means you have to perform this extra cognitive load to understand the difference. Most of the time the user won't really care what is_typed() returns.

[produno]

In this case I guess we need to decide if we are treating Variant as a type or as a 'catch all'.
If we are treating Variant as a type, then Array[Int] == Array[Variant] would be False, otherwise it would be True. Both concepts seem fairly easy to understand but which would be better for actual programming and developing? I'm not really sure of the answer but just throwing out my thoughts on it.

[nlupugla]

Great write up vnen! I don't have any specific comments yet, but I do have a general note.

I think it's important to consider how each of these decisions impacts whether == is reflexive, symmetric, and transitive. Or put another way, to what extent it is a genuine equivalence relation https://en.wikipedia.org/wiki/Equivalence_relation.

For example, if we allow my_empty_string_array == my_empty_variant_array and my_empty_int_array == my_empty_variant_array then we either have to give up transitivity or make my_empty_string_array == my_empty_int_array. To be clear, I don't think ensuring == is transitive is important beyond all else (there may be valid reasons to want to break it in certain cases), but it is what users will naturally expect and more intuitive.

[timothyqiu]

I think typed array as a way to say "I swear I won't put anything other than TYPE into that array". So for equality, I like ignoring the type.

Assignment: If the type is different, create a copy. But it can still be reference under the hood and copy-on-write. I don't think we should forbid modifying either array after the assignment, so they have to be separate eventually. And this plays nicely for arguments I think.

[Mickeon]

For the following case here:

var sprite := Sprite2D.new()
var a: Array[Node] = [sprite]
var b: Array[Sprite2D]

b = a # What should happen here?

Could it be possible to "double-reference" them?

Say, through the assignment, b stores a reference to the original array, and its own typing rules.
Then, a and b do point to the same Array, but a limits insertions only Node types, while b does the same but for Sprite2D. Would such a... reference of references system be a hassle to develop and maintain?

Regarding equality, I think the array's type should be ignored outright. It is beyond the common user's expectation, and can be checked with other methods if necessary. On the surface and at first glance, there's no difference between [1, 2, 3] and... [1, 2, 3] but only integers.

var children = get_children() # This is a Array[Node]
var my_array = [$Child1, $Child2, $Child3] # This is a standard, untyped Array

# Should this seriously print "false" because the array's types are not equal?
print(my_array == children) 

Furthermore, the operator in Godot 4 checks by equality, not by reference. Being that the expected behavior, requiring such precision from users that may not even make intentional use of the feature (through built-in methods returning typed arrays) may be excessive.

[vnen]

For the following case here:

var sprite := Sprite2D.new()
var a: Array[Node] = [sprite]
var b: Array[Sprite2D]

b = a # What should happen here?

Could it be possible to "double-reference" them?

Say, through the assignment, b stores a reference to the original array, and its own typing rules. Then, a and b do point to the same Array, but a limits insertions only Node types, while b does the same but for Sprite2D. Would such a... reference of references system be a hassle to develop and maintain?

That's not really possible because insertion isn't the only operation you do. If with a you insert something that isn't a Sprite2D and retrieve this element with b, the assumptions about typing are broken. The only solution for this would be a runtime check also for retrieval, which would give an extra performance penalty for reading elements.

[nlupugla]

Regarding equality, I think the array's type should be ignored outright.

This may actually be the right approach. I think we should stick to the philosophy that
a) types should always be optional and
b) adding type hints to your code should not change its behavior (unless it makes the code not run at all because of a static error)

Imagine I have an untyped script that looks like

@onready var sprites = [$DeadSprite, $AliveSprite]

func _ready():
    if not get_children() == sprites:
        reset_sprites()

If I later edit this to be a typed script

@onready var sprites : Array[Sprite2D] = [$DeadSprite as Sprite2D, $AliveSprite as Sprite2D]

func _ready() -> void:
    if not get_children() == sprites:
        reset_sprites()

I should not see any change in behavior.

[ttencate]

If the path points to a Label, foo[0] should be null. I disagree with this part. Type errors should be loud, not silent.

[timothyqiu]

@ttencate An error could be printed in this case. But either way, the behavior has to change. The array will either be empty or contains a null element. Getting an Label from Array[Sprite2D] is unexpected.

[Bromeon]

adding type hints to your code should not change its behavior (unless it makes the code not run at all because of a static error)

Note that this isn't a principle GDScript has generally followed, even leaving edge cases aside.

See #7364 (reply in thread).

[vnen]

b) adding type hints to your code should not change its behavior (unless it makes the code not run at all because of a static error)

That's literally impossible to hold. Adding types mean adding restrictions, if you violate the restrictions you will get errors (either static or runtime). If suddenly you say variable a must be int, but you were assigning it with a String before, the error is unavoidable and even expected (you wouldn't set the type if you didn't want the check).

That's why I even avoiding calling that "type hints". They're not mere hints, they are specification of your variables. When you put types you are creating a contract and the language must respect that (otherwise they become pointless, or even a burden). So it can't really be expected that all works the same after establishing a stricter contract than the one before (unless, of course, you were already respecting the stricter contract).

The places where behavior isn't subject to change are the exception rather than the rule.

The only problem is with

a) types should always be optional

Which I do agree. However, the engine API is typed and that in itself makes it tricky to handle users of untyped code, because there are still validation in the way (which is a good thing in the end). So in this sense it is expected that types are ignored when comparing arrays.

[nlupugla]

Adding types mean adding restrictions, if you violate the restrictions you will get errors (either static or runtime). If suddenly you say variable a must be int, but you were assigning it with a String before, the error is unavoidable and even expected (you wouldn't set the type if you didn't want the check).

I don't think this is incompatible with the principle I'm arguing for. Ideally, adding types to your code will cause it to error in places where you made mistakes. That is essentially the entire point of types after all. That's why I said adding types shouldn't change behavior except for introducing errors. Originally I said these errors should be static, but then I realized there are situations where the errors would have to be at runtime.

Having thought about this principle some more, It does seem overly restrictive. What about something like "any change in behavior due to the introduction of type specification should be accompanied by (in order of preference) a static error, a runtime error, or a warning."

[ttencate]

adding type hints to your code should not change its behavior (unless it makes the code not run at all because of a static error)

I think this is an excellent design principle. See also e.g. #37312 where this principle was violated.

This principle also rules out any solutions that require copying the array if types don't match, because changes to the copy aren't visible in the original and vice versa.

Edited to add: how do we feel about adding type hints changing the behaviour of the code by causing runtime errors? Such errors would only be informational, the code should keep running as if they never happened. E.g. the current Array.push implementation violates this because if there's a type mismatch, the element is never added to the array.

[Bromeon]

adding type hints to your code should not change its behavior (unless it makes the code not run at all because of a static error)

I think this is an excellent design principle. See also e.g. #37312 where this principle was violated.

This isn't a principle GDScript has generally followed, even leaving edge cases aside.
Implicit conversions and literal conversions don't coexist very well with it.

var s = "hello"
print("type: ", typeof(s))  # output: 4

var s: StringName = "hello"
print("type: ", typeof(s))  # output: 21

or:

var s = [1, 2]
print("type: ", typeof(s))  # output: 28

var s: PackedInt32Array = [1, 2]
print("type: ", typeof(s))  # output: 30

[nlupugla]

These are good example, thanks for pointing them out!

Nonetheless, I do think it's a good principle to strive for, even if it doesn't always hold.

The other thing I'll point out is that these example explicitly invoke type reflection. I think type reflection is inherently compatible with the principle that adding types doesn't change behavior. The more important cases, and the cases that are more likely to trip up users, are when they aren't explicitly asking for the typeof their data, but they still see behavioral changes when they add types.

[Bromeon]

The other thing I'll point out is that these example explicitly invoke type reflection. I think type reflection is inherently compatible with the principle that adding types doesn't change behavior. The more important cases, and the cases that are more likely to trip up users, are when they aren't explicitly asking for the typeof their data, but they still see behavioral changes when they add types.

It's not just reflection -- PackedInt32Array and Array have fundamentally different semantics, one being copy-on-write, the other reference-counted. They have different APIs and are exposed differently in both C# and GDExtension. Same about String/StringName.

Nonetheless, I do think it's a good principle to strive for, even if it doesn't always hold.

Where it makes sense, yes. Personally, I'd value reliability (guarantees) of type annotations higher -- meaning Array[Sprite2D] cannot suddenly contain non-Sprite2D instances, for example.

In many cases, it should be possible to combine both reliability and not changing behavior.

[nlupugla]

I definitely agree that reliability is more important than adding types not changing behavior.

[tcoxon]

Adding type hints to your code should not change its behavior except for increasing the amount of runtime or static errors

I don't completely agree with this. One reason users like myself add type hints to their code is to avoid runtime errors by turning them into static errors.

See the example I posted here: #7364 (reply in thread) Before adding the type hint to the parameter, the code was correct and had no runtime errors. After adding the type hint, I get runtime errors. This is worse than useless - it's harmful. In a large project it can go unnoticed for a long time. I'm better off not using type hints in that scenario.

No doubt there are many cases where runtime errors are useful (for instance, on code that would eventually hit an error anyway). I just think the blanket statement that adding runtime errors is always ok is lacking an understanding of why developers use type hints.

[Bromeon]

Equality

Assuming they are still compared by value, there are two options:

1. Ignore the type. If the contents are the same, they are said to be equal.

2. Include the type. If the types are different, they are never equal.

Are there real use cases for approach 2? Java does this and it's tripping over even experienced developers.

Short x = 7;
System.out.println(x.equals(7)); // false, because Short != Integer

I'd heavily advocate structual equality, checking whether contents are equal. This is almost always what users would expect.

Regarding dictionary keys, we should make sure that hashing and equality behave in a consistent manner.
That is: a == b implies a.hash() == b.hash(). And therefore:

var a: Array      = [1, 2, 3]
var b: Array[int] = [1, 2, 3]

#  So these two dictionary accesses should be equivalent:
dict[a]
dict[b]

---

Assignment

Given this, we have a few options:

1. Allow the reference. This option includes adding runtime checks to ensure the type are always correct when it matter (like auto-casting when accessing elements). Probably the worst in terms of performance and does add complexity.

2. Forbid the reference. The easiest to do. Might be inconvenient in some cases though, but also gives obvious error messages. This will force the user to make an explicit copy on assignment, as I explain at the end of this section.

3. Create a copy. This is quite controversial, but it would allow assignment between different (but compatible) array types, without creating the clash of references. However, it would be confusing if you expect it still to be a reference.

Approach 3 seems outright unexpected (introducing copy semantics for a pass-by-reference type in a very specific case). Apart from correctness, it can also introduce hidden perf issues.

Approach 1 also doesn't seem ideal, because you move the problem from one boundary check (when converting) to an infinite number of follow-up checks (when accessing elements).

I would recommend approach 2 because of correctness. As a user, I expect the declaration

var b: Array[Sprite2D]

to mean "b is an array of Sprite2D" and not "it may or may not have Sprite2D, but I find out on access". I think the main benefit of types in GDScript is the added guarantees (e.g. when writing APIs), I wouldn't want to water that down -- otherwise people will need to do manual runtime checks and may as well use untyped GDScript.

As you mention correctly, it does mean there are cases where an explicit copy will be needed -- but I think that's better because people understand what's happening, and it's obvious to readers of the code that reference semantics no longer uphold. An error message could also give clear directions on what to do if that behavior is desired.

---

Arguments

Similar to assignment, there's the question of how to treat array types when passing them as function arguments.

The options are the same as the ones for assignment and arguably should use the same solution for consistency. But these cases can happen more often in untyped code than with properties, so it has the potential of becoming more annoying in general.

To also keep this section with an example, consider this:

extends Node
class_name CustomNode

func foo(arr: Array[Node]) -> void:
	print(arr)

extends Node
func _ready():
	var custom = $CustomNode
	custom.foo([self]) # Is this allowed, create an implicit copy, or requires explicit copy?

While this is pretty much the same as the assignment, I wanted to make it clear that this case needs to be considered as well when making the solution.

Fully agree on making argument passing behave consistently with assignment.

Regarding this code in particular, I'd like to point to my comment godotengine/godot#73005 (comment):

The solution to this problem is a more lenient/flexible interpretation of literals:

• ["string1", "string2"] should be assignable to both Array[String] and Array.

• ["string", 42] should only be assignable to Array.

This would mean that [self] on its own has no defined type yet, that type is determined based on who it is assigned to. I'm not sure how compatible that is with GDScript's implementation, but it's a concept that's used a lot for literals, in many different programming languages (e.g. 7 can be used to initialize a short variable or parameter, despite being an int literal).

[Bromeon]

Separate response because it's not directly answering the questions, but may need discussion on its own.

I think most of the current issues with typed arrays can be reduced to one principle: use of covariance where contravariance is appropriate. To elaborate:

Reading arrays is covariant:

Array[int] can be safely read as Array, because each int is a Variant.
All operations that do not modify the array are perfectly fine on the untyped array.

var array: Array[int] = [1, 2, 3]
var read_only: Array = array

var elem = read_only[0] # this is always safe, result is Variant

Writing arrays is contravariant:

Array (untyped array) can be safely written as Array[int], because each element being written (int) is also a Variant.

var array: Array = [1, "string", false]
var write_only: Array[write int] = array # hypothetical

write_only[0] = 7 # this is always safe!

GDScript however treats both reading and writing as covariant, while the latter is not.

Kotlin approached this problem by having explicit variance specifiers in and out. Even if this may be too complex for GDScript's current approach, the page is absolutely worth a read to fully grasp the underlying concepts 🙂

[vnen]

That could be an interesting solution, allowing references to still be passed while respecting the proper typing.

It would be wonky to use from an untyped context, but I think at this point it's inevitable that some typing will be need for these cases.

[dalexeev]

I agree that many problems are caused by the ambiguity of the [...] syntax, which can mean either an untyped array or a typed one (similar to the {...} syntax in C++, which is suitable for any structures depending on the context). What we could do:

1. Add a typed array constructor (Array[Type](base_array)), see godotengine/godot#71336.
2. Add warning(s) when the [...] syntax is implicitly treated as a typed array.

Equality

I'm not sure about this, as the == operator's semantics are quite confusing. We can have values of different types equal (1 == 1.0, "string" == &"string_name"). That is, "content" is compared, not strict equivalence. However, the == operator can cause a compilation or runtime error. The following options seem to me the most logical:

1. Comparing only the elements, regardless of the type of the array. [] == Array[int]([]), [1, 2, 3] == Array[int]([1, 2, 3]).
2. Array type comparison first, then elements. [] != Array[int]([]), [1, 2, 3] != Array[int]([1, 2, 3]).
3. Error when comparing arrays with different types, including typed and untyped.

For options 1 and 2, we can also add a warning. However, it should probably also be added at runtime (in debug builds), for untyped code.

Assignment

I think for the most part it's ambiguity problem of the [...] syntax. Arrays are passed by reference, so we shouldn't "implicitly convert" them when assigning another array, as this creates a new copy and breaks the reference.

As for the exception we made for Array (aka Array[Variant]), I think it's OK. It's a compromise between reliability and convenience. We have a typed API and users of the untyped GDScript.

Arguments

I think it's the ambiguity problem of the [...] syntax depending on the context. When the function argument is not an array literal, but for example a variable, I think that we should not implicitly cast an array, since arrays are passed by reference.

[vnen]

1. Add a typed array constructor (Array[Type](base_array)), see godotengine/godot#71336.
2. Add warning(s) when the [...] syntax is implicitly treated as a typed array.

The problem with both of these is that GDScript is still expected to work without any types. If we put a warning every time someone uses an untyped construct, it goes against this idea. Also, type inference is at work a good amount of the time when you use types, so if you write var x: Array[int] = [1, 2, 3] it shouldn't be expected that you repeat the typing for the array literal, because it is redundant, for the same reason you can also do var y: float = 1 without having to use 1.0 or doing a cast.

It is also my mistake for only mentioning the typed array constructor only en passant, I should have highlighted it more prominently. I did have this in mind but, as also said, it should work with an untyped context as well. At least as best as possible.

There are some situations where values of wrong type are passed around and then fail at runtime. It wouldn't be the end of the world if that also happens with arrays. The issue is that, historically, doing [1, 2, 3] was pretty much always fine, but now it fails in many cases. My point here is mostly about minimizing where this problem occurs.

[dalexeev]

Also, type inference is at work a good amount of the time when you use types, so if you write var x: Array[int] = [1, 2, 3] it shouldn't be expected that you repeat the typing for the array literal, because it is redundant, for the same reason you can also do var y: float = 1 without having to use 1.0 or doing a cast.

Yes, I thought about it, but did not write. We could make an exception for this case, or use a separate warning disabled by default. Note that in other cases (for example, passing an argument when calling a typed function), there is no boilerplate and there is ambiguity.

It also makes sense to use type inference with the typed array constructor. You can use

var x := Array[int]([1, 2, 3])

like

var label := Label.new()

instead of

var label: Label = Label.new()

See GDScript style guide - Inferred types.

[nlupugla]

To the extent possible, it would be great if the following always either returned true, or failed to compile:

var x : MyType = y
x == y

In addition, it might be worth considering how important it is for the semantics of typed arrays to be consistent with other typed containers such as PackedArrays and Vectors.

For example, how confused might users be if

var my_packed_int_32_array : PackedInt32Array = [1, 2, 3]
var my_packed_float_32_array : PackedFloat32Array = my_packed_int_32_array

behaved differently from

var my_int_array : Array[int] = [1, 2, 3]
var my_float_array : Array[float] = my_int_array

To clarify, I'm not saying that the above blocks need to behave identically. I'm just pointing out that if they do need to behave differently, it could be a point of confusion and the differences should be clearly explained and documented.

[AThousandShips]

That is true

One specific case where this fails though is:

var foo = 1.5
var bar : int = foo

Whereas using for example Vector2 and Vector2i fails

I don't think this principle here should be applied, unless we forbid direct assignment of typed arrays from untyped ones, which I think is overly complicating things:

var foo = [1.5, 2.5, 3.5]
var bar : Array[int] = foo

Further the question is:
Why should bar = foo be different from bar.assign(foo), unless we explicitly demand that you explicitly cast to convert an untyped array to a typed one, which I feel is overly complicating things

I think that having the above, assigning to a typed array from an untyped one, fail for cases like this is a bad idea

Regardless of the rules about comparison between two differently typed arrays, I think comparing a typed and an untyped array should always only compare the values

[AThousandShips]

Note also:
While Vector2() == Vector2i() throws a runtime error, [Vector2(), Vector2i()] == [Vector2i(), Vector2()] does not (though it returns false)

So therefore I think that either typed arrays should always throw an error when comparing with untyped, or never

[AThousandShips]

I'd argue that for conversions:
What you can do with plain assignment is valid for array assignment

So:

var foo = [Vector2i()]
var bar : Array[Vector2] = foo

Is legal

But:

var foo = [Node2D.new()]
var bar : Array[Node3D]
bar = foo # Is not
bar = foo as Array[Node3D] # Is legal, results in [null]
bar = Array[Node3D](foo) # Same?

[nlupugla]

Regardless of the rules about comparison between two differently typed arrays, I think comparing a typed and an untyped array should always only compare the values

Definitely agree on that.

One specific case where this fails though is:

These are two great examples. In the scalar case, it's worth noting that if foo is typed, I get a narrowing warning:

var foo = 1.5
var bar : int = foo
foo == bar # false, no warning

var foo : float = 1.5
var bar : int = foo # narrowing warning
foo == bar # false

In the vector example:

var foo = Vector2(1.5, 1.5)
var bar : Vector2i  = foo
foo == bar # runtime exception

It is interesting that the behavior of these two are so different.

One thing to keep in mind though about int and float examples is that they don't enjoy a strict subtype relation.

If I look at PackedInt32Array and PackedInt64Array, I see yet a third kind of behavior:

var foo : PackedInt32Array = [1, 2, 3]
var bar : PackedInt64Array = foo # static error because of type mismatch
foo == bar

I guess I don't have a particular point that I'm trying to argue with these examples, but I do think it's worth considering the conventions that have been established for similar cases in the language.

[AThousandShips]

Packed arrays aren't entirely comparable here as their behaviour differ in a lot of other ways, but generally agreed

[nlupugla]

Here is a thought for handling typed array assignment: what if assigning one array to another caused the array of supertypes to be marked as "read-only".

For example, if an array of subtypes is assigned to an array of supertypes:

var node_2d : Node2D = Node2D.new()
var array_node_2d : Array[Node2D] = [node_2d]
var array_node : Array[Node] = array_node_2d # array_node is now read-only

array_node.append(Node3D.new()) # error: array_node is read-only
array_node_2d[1] # this problematic line doesn't run because of the error above

If an array of supertypes is assigned to an array of subtypes:

var node_2d : Node2D = Node2D.new()
var array_node : Array[Node] = [node_2d]
var array_node_2d : Array[Node2D] = array_node # array_node is now read-only

array_node.append(Node3D.new()) # error: array_node is read-only
array_node_2d[1] # this problematic line doesn't run because of the error above

In this approach, declaring a variable as an Array would be different from declaring it as an Array[Variant]. A variable marked as Array would not become read-only upon assignment of a typed variable because it is "untyped" and thus exempt from this proposed rule. On the other hand, Array[Variant] would become read-only whenever it is assigned an array of a Variant subtype.

I see a few advantages of this approach.

1. This would bring no changes to untyped code, because Array is exempt from the proposed rule
2. The only way the proposed rule can change the behavior of typed code is by causing it to fail
3. There is minimal performance cost compared to copying the array upon assignment
4. If the user needs to write to the array, they can always explicitly make a duplicate

I'm sure there are issues with this approach, and I welcome others trying to point them out. One issue I can already see is with function arguments. Consider the following example.

func append_new_node_2D(array : Array[Node]) -> void:
    var node_2D : Node2D = Node2D.new()
    array.append(node_2D)


func _ready() -> void:
    var array_of_nodes : Array[Node] = [Node.new()]
    append_new_node_2D(array_of_nodes) # this is fine, no array is marked as read-only

    var array_of_node_2Ds : Array[Node2D] = [Node2D.new()]
    append_new_node_2D(array_of_node_2Ds) # this causes an error, because the proposal forces the passed array to become read-only

I'm not sure how fatal the above example is to the idea, or whether there are clever ways to work around it.

[Bromeon]

This is basically what I suggested with covariance above, but at runtime. I think this is better expressed as part of the type system, because all the information is known at compile/parse time, and this avoids surprises when running a game.

var array_node_2d: Array[Node2D] = [Node2D.new()]
var array_node: Array[Node] = array_node_2d # array_node is now read-only

should rather be:

var array_node_2d: Array[Node2D] = [Node2D.new()]
var array_node: Array[read Node] = array_node_2d

(syntax to be bikeshedded). It's what Java does with ? extends Node and Kotlin with out Node.

[nlupugla]

I see! I liked your explanation about array variance above, I just wasn't sure what the actual proposal would be for putting those ideas into GDScript. This example makes it more clear to me.

Overall I would say I like this approach better than the one I suggested above, but I do have a few questions. What happens with all the engine functions that return typed arrays or accept typed arrays as arguments? Would all these functions need to be updated to make their variance explicit? Is it possible to cast an array with one type of variance to a different type?

One situation I run into all the time that I would love our solution to address is the case where you get_children or get_nodes_in_group and you know that all the nodes are going to be a particular type, eg Node2D. Currently, the only way to mark the returned array as Array[Node2D] is to copy it. It would be great if our solution made this kind of situation natural to deal with.

[dalexeev]

List when an array literal is treated as typed

1. Declaring a variable, constant, or function parameter:

var a: Array[int] = []
const A: Array[int] = []
func f(a: Array[int] = []): pass

2. Returning an array literal from a function, if return type is specified:

func f() -> Array[int]: return []

3. Assignment, if the assignee type is known at compile time:

var a: Array[int]
a = []
# Works with complex expressions like `obj_arr[0].a = []`,
# but the type must be known at compile time too.

4. Passing a parameter when calling a function, if the type is known at compile time:

func f(a: Array[int]): pass
f([])
# Works with complex expressions like `obj_arr[0].f([])`,
# but the type must be known at compile time too.
# Doesn't work with `Callable`s obviously.

5. Type casting operator:

[] as Array[int]
# Works only with literals. `a as Array[int]` will not work.
# In a sense, this is an alternative to the typed constructor (`Array[int]()`),
# but I don't recommend it, since type casting has different semantics
# and doesn't work with arbitrary expressions, only with literals.

6. The for loop list if the variable type is specified:

for x: int in []:
    pass
# You can't access the array, but this limits the type of elements in the array.

Notes:

• There is a bug with bad constant sharing (Const expressions of equal value and type are shared  godot#56306), see GDScript: Fix typed array hash godot#72514 (comment).
• There is a bug (I think) that this works in the context of a weak type (GDScript: Don't make array literal typed in weak type context godot#81332).

[LeandroHPerez]

I believe I found a problem related to arrays:
godotengine/godot#86851

[nlupugla]

Hi all! This discussion came up again recently on the rocket chat and I've learned a lot about typing since I first saw this post and commented on it. For anyone interested, I made some notes that summarize what I've learned from some Type Theory self study: https://gist.github.com/nlupugla/252a23e3c605c4c92c29eeed5d00c6bc

One of my main takeaways related to typed arrays is that the issues that come up are basically all related to the fact that they allow both reading and writing. @Bromeon touched on this a bit when talking about covariance and contravariance. What I'd like to highlight is that the issues that come up with typed arrays also come up when dealing with a language that supports typed references. This is a simpler setting to work in as it removes the complexities of dealing with a container type and focuses the issue on just reading/writing.

Let me explain with an example written in C++. Note: I'm just using the syntax of C++ for clarity. The actual C++ typing rules are overly complicated for our purposes.

// Assume Dog inherits Animal and adds a "bark" property, which Animal does not have.

// References can be read from, so they are not contravariant
Dog dog_1;
Animal animal_1;
Animal *animal_ref_1 = &animal_1 // Animal reference to Animal value, totally fine.
Dog *dog_ref_1 = &animal_1 // Dog reference to Animal value, not good because not all animals are dogs.
(*dog_ref_1).bark; // uh oh, animal_1 can't bark: reference is not contravariant

// References can be written to, so they are not covariant either
Dog dog_2;
Animal animal_2;
Dog *dog_ref_2 = &dog_2; // Dog reference to Dog value, totally fine.
Animal *animal_ref_2 = &dog_2; // Animal reference to Dog value, what could possibly go wrong?
*animal_ref_2 = animal_2; // the referenced value (dog_2) has been replaced with an animal
dog_2.bark; // uh oh, animal_2 can't bark: reference is not covariant.

The reason these kind of typing issues are typically discussed in the context of arrays/collections in scripting languages like GDScript is that they typically don't let the programmer be explicit about how single reference values are read to and written (hence why I switched to C++ syntax for the example above).

One "solution" to the invariance of reference types is to have a read-only reference (sometimes called "const" or "source") and a write-only reference type (sometimes called "sink"). As @Bromeon points out, Kotlin applies this ideas to arrays by including the notion of "in" (sink) and "out" (source) arrays. This level of specificity is probably a bit much for GDScript, but I think the ideas can still be incorporated via e.g. smart defaults (such as making TypedArrays read-only by default unless explicitly marked as writeable or write only).

TL;DR: The "problems" with typed arrays have nothing to do with them being a collection and everything to do with being a readable and writable reference. Thinking about how to handle references can simplify the problem and help pinpoint exactly where the issues come from and how they could be fixed.

[nlupugla]

Suppose the solution we end up going with is to provide a way to mark arrays as being read only in some way. Here are a few points to consider

1. What is the syntax for specifying whether an array is read-only or not?
2. Should the read-only modifier be limited to arrays or should it be possible to declare any variable as read only (e.g. a read only Node)?
3. Should Godot's API and documentation specify when parameters or return types are read-only?

Here's my personal thoughts, in reverse order:
3.
It should. Since the core is written in C++, which already has a const modifier, I suspect it shouldn't be too difficult to make the API and documentation reflect constness.
2.
Perhaps. This would call for a much bigger change to GDScript than just dealing with typed arrays. There is a lot of demand to make typed arrays "just work" and not much demand for GDScript to support some kind of const modifier, so it wouldn't feel right to delay fixing typed array just to have a more "elegant" solution. That said, it would be great if read-only arrays can be added to the language in such a way that it makes room for other kinds of read-only properties in the future.
1.
Beyond just the word choice of read, read_only, const, out or what have you, there is also the more interesting question of where this modifier gets placed in a statement/assignment. Let's go with const just for now. You could have:

a) var a : Array[const Node]
b) var a : const Array[Node]
c) var const a : Array[Node]
d) const var a : Array[Node]
You could also imagine a special symbol for const assignment like let or :: and have something like
e) let a : Array[Node]
f) a :: Array[Node]
although this makes it hard to express constness in docs or function signatures.
Personally, I would favor option b) with maybe :: as syntax sugar so that reading typed GDScript doesn't devolve into just reading a bunch of consts everywhere.

[Bromeon]

Syntax-wise,

var a: Array[read Node]
# or:
var a: Array[out Node]

might be the most intuitive, since expresses that the element type Node may in fact point to something derived from it and can thus only be read, not written.

The array itself is not constant/read-only/immutable. You should always be able to call clear(), for example.

This is also the syntax that Kotlin uses.

---

But more importantly, there are semantic questions to be answered:

1. Would this be a runtime or compile-time property?

   • With Godot's reflection system, you'd likely need to also store the "read-only-ness" at runtime for the type system to remain consistent.

2. How can you convert this to other arrays? The following conversions are safe:

   • Array[T] -> Array[read T]
   • Array -> Array[read Variant] (special-case of above)
   • Array[read Derived] -> Array[read Base]
   • Array[read T] -> Array[read Variant] (considering Variant is technically a "supertype" of everything else)

3. What does read Node even mean?

   • It does not mean you cannot modify the Node -- you can still call set_name() or whatever. Which is why I think const is a bad keyword; read or out is better.
   • But it means you cannot insert any elements through that array. This is what we want to avoid: that the Array[read Node] is in fact pointing to a Array[Node3D] and the user tries to append a Node2D.

---

For the type system to take "not able to insert" into account, certain methods would need to have a "write" or "in" flag:
append, append_array, push_back, push_front, insert, fill, resize.

Such "write" methods would not be allowed to call on a Array[read T].

[nlupugla]

As luck would have it, arrays already have a flag for being read-only :)

https://github.com/godotengine/godot/blob/0bcc0e92b3f0ac57d4c4650722f347593a258572/core/variant/array.h#L129-L130
from array.h:

        void make_read_only();
        bool is_read_only() const;

and methods already take that in to account, e.g. https://github.com/godotengine/godot/blob/0bcc0e92b3f0ac57d4c4650722f347593a258572/core/variant/array.cpp#L108-L111

void Array::clear() {
        ERR_FAIL_COND_MSG(_p->read_only, "Array is in read-only state.");
        _p->array.clear();
}

[nlupugla]

By the way, I'm confused by why you say

The array itself is not constant/read-only/immutable. You should always be able to call clear(), for example.

First of all, if you are not allowed to insert new elements, what is the point of calling clear()? You would be stuck with an empty array. It sounds like what you are describing is less of a read-only array and more of a shrink-only array.

I realize there is potentially confusing point that is worth clarifying. For type safety, we need the array to be read-only/const, but we don't need the variable referencing the array to be read-only/const. In other words, we want to make my_read_only_array[0] = 5 illegal, but permit assignments like my_read_only_array = my_other_read_only_array. In that light, I can see why using const might be confusing as it suggests that my_const_array = my_other_const_array is illegal.

That said, I still think Array[read Node] is confusing because to me, it suggests that the elements are read-only, not the array. In principle, my_node: read Node = Node.new() could be a thing, and likewise represent a reference for which changing the referenced value (e.g. my_node.set_name("bob")) is illegal but changing the reference (e.g. my_node = my_other_node) is legal.

That said, how many actual situations are there where you want an array to be read-only AND be able to assign another array to it? If anything, I would say that semantic like this

var a: read Array[int] = [1, 2]
a[0] = 3 // not okay
a = [3, 4] // okay

would "feel" confusing an inconsistent to the user, even if there's a good reason for it.

The more I think about this, the more I feel like TypedArray constructors are the way to go...

[Bromeon]

First of all, if you are not allowed to insert new elements, what is the point of calling clear()? You would be stuck with an empty array. It sounds like what you are describing is less of a read-only array and more of a shrink-only array.

We need to be careful to not mix const-correctness with variance. I recommend reading this page, Kotlin explains it better than I do, and the exact same thing applies here: https://kotlinlang.org/docs/generics.html#variance

Variance and const-correctness are orthogonal.

• Const-correctness says whether an element or the entire array can be modified or not.
• Variance determines the subtyping relation: whether Array[T] is a subtype Array[U].
  This is the case if T is a subtype of U and the elements can only be "read" (out direction), not "written" (in direction).
  Read/write are not good terms because they make people think of const.

So let's stop using the term const, it's very misleading in this context.

---

That said, I still think Array[read Node] is confusing because to me, it suggests that the elements are read-only, not the array. In principle, my_node: read Node = Node.new() could be a thing, and likewise represent a reference for which changing the referenced value (e.g. my_node.set_name("bob")) is illegal but changing the reference (e.g. my_node = my_other_node) is legal.

Let's go with Array[out Node] then. Kotlin has done their fair share of research, so there is prior art to this.

No, my_node: out Node could not be a thing. It's important to understand that variance is not a property of the type T. It is a property of the generic parameter (here T in Array[T]), and expresses how the generic (Array) behaves when substituting T with its subtype or supertype. The out is thus not meaningful outside Array.

[nlupugla]

I see what you are saying. Yes, variance and mutability are orthogonal concepts in a sense. It is possible for a type constructor to be covariant, but have the resulting type still be mutable (like you are saying with the .clear() method) and it is also possible for a constructed type to be immutable and not be covariant (the language semantics can just decide that the type is invariant). Thank you for the clarification :)

Although there is a technical distinction between variance and readability, I wonder how much it matters to GDScript users. Yes, technically, requiring an array to be read-only to gain covariance is overly restrictive. However, "read-only" is a very simple concept to understand. I'm not sure that the added complexity required to distinguish mutability and variance adds enough value to GDScript for it to be worth it. Although the Kotlin docs on variance are an interesting read for me, I wouldn't want a new user to feel like they have to understand a document like that just to get a line like var objects: Array[Object] = get_children() to "just work".

[betalars]

Edit: it got explained to me.
I propose adding a better error-message, because this is so counter-intuitive. But I agree the current implementation is the best one.

Original Comment:
Sorry for not really understanding a lot of this discussion, but as I was forwarded here for opening an Issue on the behavior of typed Arrays and inheritance

I find it really confusing, why it is not possible to assign a typed array to an array of a parent type.

var node_array: Array[Node]
var node_3d_array: Array[Node3D] = [Node3D.new()]

node_array = node_3d_array

I mean it is not too bad, as this is a possible workaround

node_3d_array: Array[Node] = [Node3D.new()]
Edit: this is the correct workaround: child_array as Array[parent]
Edit: parent_array.assign(child_array) works. The compiler simply shows no error when trying to use the as keyword ...

but this really seemed like a bug and not expected behavior to me.

[nlupugla]

I think you need .assign in this case. Just using as will give you similar trouble I believe (but I could be wrong, haven't tested it recently).

In any case, I think giving a more helpful error message would be great for the time being. Feel free to open an issue about it.

[betalars]

I think you need .assign in this case. Just using as will give you similar trouble I believe (but I could be wrong, haven't tested it recently).

Oh no, it does not raise errors, but it does not seem to work either.

Okay, this is really annoying when using a typed array in a function-call:

var node3d_Array = [Node3d.new()]
var tmp: Array[Node] = []
tmp.assign(node3d_array)
call( tmp )

I think we need some kind of one-liner to do that.

[betalars]

I mean maybe there is a way to do it as a one-liner within a function call, but I cannot think of one.

[nlupugla]

Agreed, the current system is quite cumbersome.

I think typed array constructors are a good solution for this, e.g. Array[Node](Array[Node3D]([Node3D.new()])), but they are a work in progress.

[betalars]

Yeah, that was also the thing I tried first when encountering this issue.

[za3k]

[Bromeon]

This has been discussed further above, and there are two points to be made:

• GDScript's types are not annotations or type hints. Nor are they dynamic information -- types are available and verified at the time the script is parsed, i.e. during startup and not only execution.
• GDScript has never followed the principle that types shouldn't change semantics. It starts with var f = 3 and var f: float = 3, and see link for more examples.

[za3k]

Shoot, I thought I had read all the existing comments. Thanks for the link!

Edit: Oh man, "Delete comment" did not do what I expect. Sorry for turning this comment into a bit of a mess. I was trying to clean up some noise by deleting the whole sub thread.

[AkiYama-Ryou]

First, we should determine whether array is reference type or value type.
My intuition is that array is reference type.

For Equality：

Array A:Array[int] = [1,2,3]
Array B:Array[float] = [1.0, 2.0, 3.0]
print A == B #what happened?

As reference, I just care memory address. Obviously, A and B are not the same object. So I want here print false.
But, how can we Compare their values？
As you said, in different projects, we will expect different comparison methods. So we should provide users with operator overloading so that users can customize the calculation logic of "==".

note that the syntax design of gdscript is based on Python. Many people choose Godot because they already know how to use Python. So consistent is better.

For Assignment

var sprite := Sprite2D.new()
var a: Array[Node] = [sprite]
var b: Array[Sprite2D]

b = a # What should happen here?

Why do we need typed arrays?Because we need the compiler to tell us what is wrong.
So，This should be prevented by the compiler.

Because OOP principles are consensus, we don't need to think too much.
If user really need b = a. He should define B Wider, like var b:Array.

case a = b:
First, the compiler needs to check whether the type is extend.
Sprite2D extend from Node, so here should be pass.
Now Assignment: Here we should distinguish between two case

if a.type == b.type:
    just copy b reference to a.
    print a==b 

First should print true. because a and b use same memory address.

else if b.child_type extends from a.child_type:
    for child in b.children:
        a.append(child as father type)
    print a==b 

Second should print false. because a and b are different array.

Variant

Variant , just comprehend like every class is extend from Variant.

Question about []

var a: Array[int] = []
print(a == []) # ??

In this case, I think "==[]" is not mean a == Array.new().
==[] should be a Syntactic Sugar means that:

a is not null and a.size() == 0

[tiagojdf]

I did not see this mentioned above, but I found a bug with typed-arrays, regarding Types that inherit from other base types. As an example:

@tool
extends EditorScript

class Aux extends Resource:
	pass

var resource_array: Array[Resource]
var aux_array: Array[Aux]

func array(resources : Array[Resource]) -> void:
	pass
func single_element(resources: Resource) -> void:
	pass

func _run() -> void:
	array(resource_array)
	array(aux_array) # Invalid argument for "array()" function: argument 1 should be "Array[Resource]" but is "Array[Aux]".
	single_element(resource_array[0])
	single_element(aux_array[0])

Since the custom type Aux extends Resource, functions using Resource arguments should accept Aux. It's what happens with the function single_element. When we do the same with arrays, we get an error. Calling array(aux_array) gives Invalid argument for "array()" function: argument 1 should be "Array[Resource]" but is "Array[Aux]".

[dalexeev]

@tiagojdf See:

• Behavior of typed arrays #7364 (comment)
• GDScript reference - Typed arrays
• Wikipedia - Covariance and contravariance (computer science)