possible changes to inner constructors

[mlhetland]

I'm trying to create a type whose inner constructor does some adjustment/normalization, and at the same time I want to deduce type parameters from the constructor arguments. And I'm having trouble making that work…

immutable A{N}
    x::NTuple{N, Int}
    y::Int
    A(x, y) = new(x, y + 1)
end

#a = A((1, 2), 3)       # Doesn't work
a = A{2}((1, 2), 3)     # Works (w/cumbersome param)
println(a)

immutable B{N} 
    x::NTuple{N, Int}
    y::Int
end

#B(x, y) = B(x, y + 1)  # Stack overflow...
b = B((1, 2), 3)        # Works (w/wrong answer)
println(b)

The following works, and it solves my problem, but I don't quite understand why it should be necessary, and I wonder if it might be a bug that it is?

immutable A{N}
    x::NTuple{N, Int}
    y::Int
    A(x, y) = new(x, y + 1)
end

A{N}(x::NTuple{N, Int}, y::Int) = A{N}(x, y)

As @tknopp pointed out, it seems the default constructor is removed.

Could I somehow override the more specific default constructor (which doesn't have Any-arguments), and supply the proper type parameter to new or something? (Haven't been able to make that work.)

(See also discussion on Julia-Users.)

[timholy]

This behavior is documented here. Consider this case:

immutable C{N}
    x::NTuple{N,Int}
    y::Int
    C(x, y) = new(tuple(x...,5), y+1)
end

How should it guess the value of N in that case? That's the job of the outer constructor.

[mlhetland]

Ah, right—I guess you're referring to:

If any inner constructor method is defined, no default constructor method is provided: it is presumed that you have supplied yourself with all the inner constructors you need.

Yeah, I remember reading that. It also says that not supplying a constructor is equivalent to

writing your own inner constructor method that takes all of the object’s fields as parameters (constrained to be of the correct type, if the corresponding field has a type), and passes them to new, returning the resulting object […]

The example is Foo(bar, baz) = new(bar, baz). So let's apply this to my case:

type T{N}
    t::NTuple{N,Int}
end

Now you can easily use T((1, 2, 3)) to get an instance of T{3}, for example. According to the docs, this is equivalent to having the inner constructor T(t) = new(t). And adding an inner constructor removes the default constructor, so rewriting the type to the following would seem like it should work according to the docs:

type T{N}
    t::NTuple{N,Int}
    T(t) = new(t)
end

But it doesn't. And I still don't understand why.

I see that what you point to is an important point here. Should the inference be done based on the parameters to the inner constructor or to the parameters to new. Of course, there is really no room for debate on that issue, as the only way to get a valid object is to do the inference based on the arguments to new—unless you want to do it on the inner constructor and then throw an error if the arguments to new don't match. Less flexible, but maybe safer.

In either case, I don't quite see the argument for requiring this to be done in an outer as opposed to an inner constructor. As I understand it, the main point of inner constructors is (i) to enforce invariants, and (ii) create self-referential objects, no? Why does this negate inference of type parameters?

[mlhetland]

I guess one issue might be that by default you have both a constructor T(t::NTuple{N,Int}) and a constructor T(t::Any) in the first version above, and adding my constructor with signature T(t::Any) removes both, jettisoning any inference ability. Which makes sense in a “what's going on” way, but it still seems a bit problematic to me, and adding a proper signature to the inner constructor doesn't seem to do anything to alleviate the problem. Only works with an outer constructor.

[mlhetland]

In fact, the docs explicitly say that the following two types are equivalent…

type T1
  x::Int64
end

type T2
  x::Int64
  T2(x) = new(x)
end

[timholy]

You're right, those statements do muddle the waters. My understanding is that both an outer and an inner constructor are created by default, and defining your own inner constructor nixes the construction of the default outer constructor. See the Rational example at the bottom of that page.

Pull requests improving the documentation would be welcome! As someone who tripped over the current docs, you're ideally suited to make it clearer---no one could do a better job than you 😄.

[mlhetland]

Thanks for the vote of confidence ;-) I see that there are parts of the docs I ignored at the moment (though I have read them before). I guess I should re-read that material first, at the very least :-}

[mlhetland]

BTW: As far as I can see, the inner constructor is also of the form T{N}, where {N} is implicit. I'm still a bit confused about which properties of the external constructor makes it more able to infer the value of N than the internal one. Should I be able to grok that from the docs, there, or could you perhaps explain it? (Sorry for being slow, here…)

[mlhetland]

Ah, no I think I understand. The inner constructor is defined for any concrete instantiation of the type, whereas the outer constructor gan be generic wrt. the type parameters—right?

[timholy]

The way I think about it is that the inner constructor runs with the parameters already fixed.

[kmsquire]

Not sure if this helps, but the type parameters have somewhat different meanings in types and outer constructors.

For types, the type parameter is part of the type name. So T{Int,5} is the type, and T is an abstract supertype.

On the other hand, outer constructors are just functions, so the type parameter indicates that the Type should be extracted from the passed in parameters, and then is itself usable in the function, e.g., when creating a parametrized type. So in

T{N}(a::N) = T{N,5}()

the first N is set to the type of a (As in any function) and is then used to construct an object of type T{N,5}.

[JeffBezanson]

An observation I take away from this is that there's a "missing combination" of sorts: you can't have a type with only an outer constructor (you can have both, or only an inner constructor). In theory it is perfectly coherent to define this outer constructor:

A{N}(x::NTuple{N, Int}, y::Int) = magic_allocate_object(A{N}, x, y+1)

with no inner constructor. Then you could write A((1,2), 3), but not A{2}((1,2), 3). magic_allocate_object here just forcibly allocates an object with a specific type and fields.

For this type, this arrangement is desirable since it removes redundancy: there is no need to be able to write A{3}((1,), 2), which is just an error.

With the planned changes to constructors, defining such a thing will actually be possible, with syntax backwards-compatibility being the only pain point. An outer constructor will just be a method of Type{A}, and an inner constructor will be a method of Type{A{N}}.

[mlhetland]

Thanks for the clarifications, @timholy and @kmsquire! @JeffBezanson: The redundancy you speak of would be an issue for any type without an external constructor, wouldn't it? (You'd never want to specify a type parameter that contradicted the parameters? You you might want to use an abstract supertype, I guess.) And if this type had only an external constructor, that wouldn't be usable for the “invariant enforcement,” I guess—as one could just add another constructor…?

The planned changes sound interesting; is there an issue or the like somewhere? (And: Any reason why magic_allocate_object could be new with an extra argument?)

[JeffBezanson]

#1470 is one related issue.

Invariants are enforced by the restricted availability of new. However I could imagine defining an outer constructor inside a type block:

type Foo{N}
  x::NTuple{N,Int}

  call{N}(::Type{Foo}, x::NTuple{N,Int}) = new{N}(x)
end

That would ensure that Foo((1,2)) is the only way to actually construct a Foo. new{N} would be new special syntax that lets you specify the type parameters to construct with. We don't want to allow new(T, ...) for arbitrary types T.

This issue is actually very helpful. I had not thought through all of this before.

[mlhetland]

Huh—cool!

Read through the discussion on #1470; quite a bit to digest. I guess the reason for using call overload here is that it's a way of solving the problem that falls out of the call overloading mechanism? In principle, you could have defined an inner generic function, that behaved like an outer constructor except that it had access to new{N} (if the language was set up to permit it)?

That is, one could have had something like…

type Foo{N}
  x::NTuple{N,Int}

  Foo{N}(x::NTuple{N,Int}) = new{N}(x)
end

…? Or are the semantics different, in that defining call{N} with a given N (from the concrete type) defines a specific method on call, for ::Type{Foo} whereas defining a method on Type{N} won't work, because it's not a function? It does strike me as confusing that the meaning of this is so different depending on whether it's inside or outside the type block … but at least I'm glad my confusion has lead to a helpful issue xD

[mlhetland]

By the way, something along the lines of the example above (just with a call to plain new) was one of my confused attempts at remedying the issue in my original code, before I gave up and added an external constructor. (Just a data point on uninformed intuition ;-)

[JeffBezanson]

I like adding constructors by explicitly adding methods to call, since it is so clear. But it would seem pretty strange if this were the only way to define constructors.

The idea was that inside the type block the parameters are already set, so you are forced to add constructors only for e.g. Foo{2}. Even defining Foo(x)=... will actually add a method for Foo{N} for a specific N.

One possibility is for Foo{N}(...) = ... inside a type block to add constructors for Foo{N}, and Foo(...) = ... to add constructors for Foo. The Foo{N} case would expand to

call{N}(::Type{Foo{N}}, ...) = ...

so the syntax is not entirely lying --- the first {N} is still a "method parameter" as normal. However we need to make all three possibilities accessible:

call{N}(::Type{Foo{N}}, ...) = ...
call{N}(::Type{Foo}, ..., N, ...) = ...
call(::Type{Foo}, ...) = ...

I can think of some extreme hacks. We could pick between the 2nd and 3rd forms automatically based on whether the names of the type parameters appear in the method signature. For example

Foo(x::NTuple{N}) = new{N}(x)
# becomes
call{N}(::Type{Foo}, x::NTuple{N}) = new{N}(x)

and

Foo(x) = new{2}(x)
# becomes
call(::Type{Foo}, x) = new{2}(x)

@StefanKarpinski might want to chime in here.

[JeffBezanson]

I'm going to reopen this since it's a good discussion of constructor design in light of call overloading.

[JeffBezanson]

The new{N}(...) syntax mentioned in #8135 (comment) is now available.