Expert f# script compile oddity

Question

On page 209 - 210 there is an extended example (see below) (I'm using F# 4.5).

In summary what I don't understand is: if I type each statement individually, there is a declaration that raises an error. If I submit the whole script at once, with the functions that come after the declaratation that raises the error, everything is OK. So what happens in interactive when I submit all the statements as a batch? Does the function that follows the error get used to create a specific version of the potential generic? The ionide hints do not show a generic for the whole script.

In detail (sorry it's so long) If you execute each of the statements in order upto and including the declaration of formatP , (or paste all of them upto and including formatP at once - e.g. select and alt-enter with ionide in vs code) in interactive, it produces an error:

    binary.fsx(67,5): error FS0030: Value restriction. The value 'formatP' has been inferred to have generic type
        val formatP : ((int * bool) list -> '_a -> unit) when '_a :> OutState
    Either make the arguments to 'formatP' explicit or, if you do not intend for it to be generic, add a type annotation.

• if you paste all of the script including the functions which follow formatP

let writeData file data =
    use outStream = BinaryWriter(File.OpenWrite(file))
    formatP data outStream

let readData file =
    use inStream = BinaryReader(File.OpenRead(file))
    formatU inStream

it works without error.

Given that compilation order is important - what is happening here?

i.e. why do the functions that are declared after formatP allow the function to compile.

Whole script - which functions without error (It's a bit long - but because I really don't understand the issue, I'm not sure what I can remove to create an accurate example illustrating what is happening. I guess I could drop formatU and readData? (which have the same relationship) but they are needed to make the last 2 stamenets work which sahows that the code does complie and execute as expected):

open System.IO

type OutState = BinaryWriter
type InState = BinaryReader
type Pickler<'T> = 'T -> OutState -> unit
type Unpickler<'T> = InState -> 'T

// P is the suffix for pickling and U is the suffix for unpickling
let byteP (b : byte) (st : OutState) = st.Write(b)
let byteU (st : InState) = st.ReadByte()

let boolP b st = byteP (if b then 1uy else 0uy) st
let boolU st = let b = byteU st in (b = 1uy)

let int32P i st =
    byteP (byte (i &&& 0xFF)) st
    byteP (byte ((i >>> 8) &&& 0xFF)) st
    byteP (byte ((i >>> 16) &&& 0xFF)) st
    byteP (byte ((i >>> 24) &&& 0xFF)) st

let int32U st =
    let b0 = int (byteU st)
    let b1 = int (byteU st)
    let b2 = int (byteU st)
    let b3 = int (byteU st)
    b0 ||| (b1 <<< 8) ||| (b2 <<< 16) ||| (b3 <<< 24)

let tup2P p1 p2 (a, b) (st : OutState) =
    (p1 a st : unit)
    (p2 b st : unit)
let tup3P p1 p2 p3 (a, b, c) (st : OutState) =
    (p1 a st : unit)
    (p2 b st : unit)
    (p3 c st : unit)
let tup2U p1 p2 (st : InState) =
    let a = p1 st
    let b = p2 st
    (a, b)
let tup3U p1 p2 p3 (st : InState) =
    let a = p1 st
    let b = p2 st
    let c = p3 st
    (a, b, c)

/// Outputs a list into the given output stream by pickling each element via f.
/// A zero indicates the end of a list, a 1 indicates another element of a list.
let rec listP f lst st =
    match lst with
    | [] -> byteP 0uy st
    | h :: t -> byteP 1uy st; f h st; listP f t st

// Reads a list from a given input stream by unpickling each element via f.
let listU f st =
    let rec loop acc =
        let tag = byteU st
        match tag with
        | 0uy -> List.rev acc
        | 1uy -> let a = f st in loop (a :: acc)
        | n -> failwithf "listU: found number %d" n
    loop []

type format = list<int32 * bool>

//the types in these two lines only get fully inferred if the lines after are part of the batch
//eh?
let formatP = listP (tup2P int32P boolP)
let formatU = listU (tup2U int32U boolU)
//IE if you only run to here it is an error

let writeData file data =
    use outStream = BinaryWriter(File.OpenWrite(file))
    formatP data outStream

let readData file =
    use inStream = BinaryReader(File.OpenRead(file))
    formatU inStream
//If you run to here it does not error

writeData "out.bin" [(102, true); (108, false)] ;;
readData "out.bin"

Answer 1

There are two things going on here, only one of which I can fully explain.

The first is the F# value restriction. In this case, the compiler infers that listP is a generic function, which also makes the value formatP generic. But F# doesn't allow a generic value in this situation, so a compiler error occurs. However, when writeData is present, formatP is no longer generic, so the compiler error disappears. You can see a very similar situation explained in detail here.

But why does the compiler think listP is generic in the first place? I'm not sure about this. It infers that st must be compatible with OutState, which is correct from an object-oriented point of view. But it expresses this as a generic constraint, using st : #OutState instead of just st : OutState. (F# calls this a "flexible type".) This generic constraint then cascades to formatP, causing the problem you reported. Someone with more knowledge of the F# compiler might be able to explain this.

---

Here's a very small example that exhibits the same behavior:

type Example() = class end

let alpha (_ : Example) =
    ()

let beta f x =
    alpha (f x)

let gamma = beta id

// let delta = gamma (Example())

The compiler infers a generic type for beta, which makes gamma illegal when delta is commented out.

To make matters stranger, the same problem occurs even if I explicitly annotate beta so it's not generic:

let beta (f : Example -> Example) (x : Example) =
    alpha (f x)

So the compiler thinks gamma is generic even when beta definitely isn't generic. I don't know why that is.