Method will not match with nested type restrictions

Question

I have this simple method which calculates the weighted average of a collection of vectors

function meanw{T <: Number}(x::AbstractArray{AbstractVector{T}, 1}, w::AbstractVector{T})
  x̄ = sum(x .* w)
  x̃ = map(z -> z - x̄, x)
  x̄, x̃
end

but the dispatch cannot match my method when I try to use it.

ERROR: `meanw` has no method matching meanw(::Array{Array{Float64,1},1}, ::Array{Float64,1})

I suspect that I have misunderstood how to use type restrictions when there is nesting involved. How should I rewrite this function to match my collections?

P.S.

I know vectors and arrays are the same thing but the differentiation makes it clearer how the function is used.

Answer 1

So if you rewrite your code to use the concrete Types it works

function meanw{T <: Number}(x::Array{Vector{T}, 1}, w::Vector{T})
  x̄ = sum(x .* w)
  x̃ = map(z -> z - x̄, x)
  x̄, x̃
end

this also works

function meanw{T <: Number}(x::Array{Vector{T}, 1}, w::AbstractVector{T})
  x̄ = sum(x .* w)
  x̃ = map(z -> z - x̄, x)
  x̄, x̃
end

this doesn't work

function meanw{T <: Number}(x::Array{AbstractVector{T}, 1}, w::AbstractVector{T})
  x̄ = sum(x .* w)
  x̃ = map(z -> z - x̄, x)
  x̄, x̃
end

and this does work again

function meanw{T <: Number}(x::AbstractArray{Vector{T}, 1}, w::AbstractVector{T})
  x̄ = sum(x .* w)
  x̃ = map(z -> z - x̄, x)
  x̄, x̃
end

the problem we have here is described in the Julia manual under parametric types

This last point is very important:

Even though Float64 <: Real we DO NOT have Point{Float64} <: Point{Real}.

In other words, in the parlance of type theory, Julia’s type parameters are invariant, rather than being covariant (or even contravariant). This is for practical reasons: while any instance of Point{Float64} may conceptually be like an instance of Point{Real} as well, the two types have different representations in memory:

What actually does work is this

function meanw{T <: Number, V <: AbstractVector}(x::AbstractArray{V, 1}, w::AbstractVector{T})
  x̄ = sum(x .* w)
  x̃ = map(z -> z - x̄, x)
  x̄, x̃
end

but the real question is why do you want to do this? The Julia Compiler is clever enough even without type annotations to produce efficient code as long as the inputs are well-typed and the function itself is type-stable. Type annotations are only really needed for multiple-dispatch and as contract to specify what the function can actually handle, but this is notoriously difficult in Julia due to the extensive type system.

Answer 2

The root problem is

julia> Vector{Vector} <: Vector{AbstractArray}
false

and you are passing in a vector of vectors. I'd rewrite this as

function meanw{T<:Number,S<:Number}(x::Vector{Array{T}}, w::Vector{S})
    x̄ = sum(x .* w)
    x̃ = map(z -> z - x̄, x)
    x̄, x̃
end

to allow for, say, Float64 x and Int w. Alternatively don't restrict it all - its probably unnecessary anyway as it won't make it faster and is only a defensive coding thing.

Answer 3

Julia is a fast-growing language and with each generation, new capabilities unveiled and some Q&As need review.

IMPORTANT

julia> VERSION
v"0.6.0-dev.2259"

One of the advanced features of Julia is its dispatching system. (excellent reading about multiple dispatch). Now, because of this merge (v0.6.0 2017-1-16) users could benefit from new capability of triangular dispatching (read why is it useful?).
Here we need to address the problem of writing generic methods for two arguments:

1. A one dimensional array (general vector) of numbers w
2. A general array of general vectors of numbers x

@wallnuss's answer involves 4 methods that all work. Here is the first one:

function meanw{T <: Number}(x::Array{Vector{T}, 1}, w::Vector{T}) info("both are Vector for the same element-type: $T") end

julia> meanw([[1],[2]], [1])
INFO: both are Vector for the same element-type: Int32

The above is very specific mainly because it doesn't cover different sub types of AbstractVector e.g.:

julia> (typeof(spzeros(3)) |> supertype |> supertype) <: AbstractVector #=> true

So the second one is defined:

function meanw{T <: Number}(x::Array{Vector{T}, 1}, w::AbstractVector{T}) info("an Array of Vectors and an AbstractVector sub-type both with the same element-type: $T") # end

With the above:

julia> meanw([[1.],[2]], spzeros(3))
INFO: an Array of Vectors and an AbstractVector sub-type both with the same element-type: Float64

And a more general method:

function meanw{T <: Number}(x::AbstractArray{Vector{T}, 1}, w::AbstractVector{T}) info("an AbstractVector subtype of Vectors and an AbstractVector sub-type both with the same element-type $T") end

Even after the latest method, some scenarios isn't fulfilled:

julia> meanw([spzeros(3) for i=1:3], spzeros(3)) #=> Error
julia> meanw([[1],[2]],[1.]) #=> Error

So all the above are more specific than may need.

function meanw{T <: Number, V <: AbstractVector}(x::AbstractArray{V, 1}, w::AbstractVector{T}) info("an AbstractVector of AbstractVector sub-type with Any element and an AbstractVector sub-type of Number") end

Seems that the last method covers all possible scenarios, but it is more general than may need e.g.:

julia> meanw([[""],[2]],[1.]) #=> Not enough restriction for element type of first AbstractVector
INFO: an AbstractVector of AbstractVector sub-type with Any element and an AbstractVector sub-type of Number

Now with the new triangular feature we could write:

function meanw{T <: Number, V <: AbstractVector{T}}(x::AbstractArray{V, 1}, w::AbstractVector{T}) info("an AbstractVector of AbstractVector sub-type and an AbstractVector subtype both for same sub-type of Number (triangular)") end

Check methods precedence:

julia> methods(meanw)
# 5 methods for generic function "meanw":
meanw{T<:Number}(x::Array{Array{T,1},1}, w::Array{T,1}) in Main at REPL[1]:2
meanw{T<:Number}(x::Array{Array{T,1},1}, w::AbstractArray{T,1}) in Main at REPL[22]:2
meanw{T<:Number}(x::AbstractArray{Array{T,1},1}, w::AbstractArray{T,1}) in Main at REPL[24]:2
meanw{T<:Number,V<:AbstractArray{T<:Number,1}}(x::AbstractArray{V,1}, w::AbstractArray{T,1}) in Main at REPL[41]:2 
meanw{T<:Number,V<:(AbstractArray{T,1} where T)}(x::AbstractArray{V,1}, w::AbstractArray{T,1}) in Main at REPL[39]:2

The latest method have taken the 4th seat because it is more specific that its former.

julia> meanw([spzeros(3) for i=1:3], spzeros(3))
INFO: an AbstractVector of AbstractVector sub-type and an AbstractVector subtype both for a same sub-type of Number (triangular)

But meanw([[2],[2]],[1.]) is still directed to the more general method (5th). Let define another method:

function meanw{T <: Number, W <: Number, V <: AbstractVector{W}}(x::AbstractArray{V, 1}, w::AbstractVector{T}) info("an AbstractVector of AbstractVector sub-type and an AbstractVector sub-type both for Number (triangular)") end

julia> meanw([[2],[2]],[1.])
INFO: an AbstractVector of AbstractVector sub-type and an AbstractVector sub-type both for Number (triangular)