F# Nested Computation Expression with Custom Operator

Question

I am creating a DSL for modeling, and I would like to be able to create a Settings builder with two Custom Operations: Buffer and Constraint, which themselves are Computation Expressions. The reason for this is that the domain has terms that are heavily overloaded, and Computation Expressions allow you to provide context through the use of Custom Operations.

I cannot figure out how to get this nesting to work as intended. I have provided an example of what my desired outcome is at the bottom of the code example.

type Buffer =
    {
        Name : string
        Capacity : float
    }

type Constraint =
    {
        Name : string
        Limit : float
    }

[<RequireQualifiedAccess>]
type Setting =
    | Buffer of Buffer
    | Constraint of Constraint


type BufferBuilder (name: string) =
    member _.Yield _ : Buffer = { Name = name; Capacity = 0.0 }
    member _.Run x : Buffer = x

    [<CustomOperation("Capacity")>]
    member _.Capacity (b: Buffer, newCapacity) =
        { b with Capacity = newCapacity }

let Buffer = BufferBuilder

type ConstraintBuilder (name: string) =
    member _.Yield _ : Constraint = { Name = name; Limit = 0.0 }
    member _.Run x : Constraint = x

    [<CustomOperation("Limit")>]
    member _.Limit (b: Constraint, newLimit) =
        { b with Limit = newLimit }

let Constraint = ConstraintBuilder

type SettingsBuilder () =

    member _.Yield _ : Setting list = []
    member _.Run x : Setting list = x

    [<CustomOperation("Buffer")>]
    member _.Buffer (settings, name: string, expr) =
        // This does not work
        let newSetting = BufferBuilder name expr
        newSetting :: settings

    [<CustomOperation("Constraint")>]
    member _.Constraint (settings, name: string, expr) =
        // This does not work
        let newSetting = ConstraintBuilder name expr
        newSetting :: settings


// The Computation Expression does not work
let mySettings =
    SettingsBuilder {
        Buffer "b1" {
            Capacity 100.0
        }
        Constraint "c1" {
            Limit 10.0
        }
    }

// Below shows that the desired outcome of `mySettings` would be
let b1 = { Name = "b1"; Capacity = 100.0 }
let c1 = { Name = "c1"; Limit = 10.0 }

let desiredSettings = [
    Setting.Buffer b1
    Setting.Constraint c1
]

Fyodor Soikin · Accepted Answer · 2022-08-03T19:30:37.453

CEs don't work like that. When you write foo { ... }, the foo bit in that expression is not a function. In particular, it means that you cannot do this:

let x = { ... }
let y = foo x

Or this:

let f x = foo x
let y = f { ... }

Doesn't work like that. It's special syntax, not a function call. The thing in front of the curly braces has to be a CE object, with all the CE methods defined on it.

So in particular, it means that your SettingsBuilder.Buffer function cannot accept expr and then pass it to BufferBuilder. BufferBuilder has to come immediately in front of the curly brace.

This means that the SettingsBuilder.Buffer function should accept whatever the result of BufferBuilder is, and then, inside the CE, you should build that result using BufferBuilder and only after that pass it to the Buffer custom operation:

    [<CustomOperation("Buffer")>]
    member _.Buffer (settings, b) =
        (Setting.Buffer b) :: settings

    [<CustomOperation("Constraint")>]
    member _.Constraint (settings, c) =
        (Setting.Constraint c) :: settings

    member _.Zero () = []

...

let mySettings =
    SettingsBuilder () {
        Buffer (BufferBuilder "b1" {
            Capacity 100.0
        })
        Constraint (ConstraintBuilder "c1" {
            Limit 10.0
        })
    }

(note that you also have to define Zero to provide the "initial" value of your expression)

(also note the unit () after SettingsBuilder. Without it, SettingsBuilder is a class, but you need the thing to the left of the curly brace to be an object)

I understand you wanted the nice syntax like Buffer "foo" { ... } instead of the extra BufferBuilder and ugly parentheses in there, but I don't think that can be done. In general, there is no way to have a custom operation behave like a "nested" expression.

Consider an alternative approach: ditch the outer CE, and instead define the settings as a list, each element built with its corresponding CE.

You'll need those inner CEs to each produce a Setting so that their results can be elements of the same list. This can be achieved by modifying their Run methods to wrap the resulting value in the relevant Setting constructor:

type BufferBuilder (name: string) =
    ...
    member _.Run x = Setting.Buffer x

type ConstraintBuilder (name: string) =
    ...
    member _.Run x = Setting.Constraint x

...

let mySettings = [
    BufferBuilder "b1" {
        Capacity 100.0
    }
    ConstraintBuilder "c1" {
        Limit 10.0
    }
]

I believe I have fixed the misspellings. – Matthew Crews Aug 03 '22 at 18:51 — Matthew Crews, Aug 03 '22 at 18:51

Patrick Simpson · Answer 2 · 2022-08-29T09:18:11.877

The above answer is correct in the specifics, but you can create an API shape that looks like what you want.

Sorry for the long answer, but there is SO LITTLE information out there on the capabilities of CE's and their Builders. You should also check out Bolero's HTML builders for some really neat ideas as well.

You can use nested Computations if you provide the appropriate _.Yield, _.Combine and _.Delay implementations. Also, thanks to F# 6.0 [<InlineIfLambda>] attribute you can generate really nice and efficient in-line code for these types of expressions.

The following code provides the API you were trying to achieve using a slightly different computation builder for SettingsBuilder.

...
let mySettings =
    Settings {
        Buffer "b1" {
            Capacity 100.0
        }
        Constraint "c1" {
            Limit 10.0
        }
    }
...
val mySettings: Setting list =
  [Buffer { Name = "b1"
            Capacity = 100.0 }; Constraint { Name = "c1"
                                             Limit = 10.0 }]

The key to getting the API to look how you want is to not create Custom Operations for Buffer and Constraint on the SettingsBuilder, but to instead create functions that create BufferBuilder and ConstraintBuilder instances by name and then including _.Yield methods that take the result of those builder's _.Run(...) methods.

...
let inline Buffer name = BufferBuilder name
...
let inline Constraint name = ConstraintBuilder name
...
type SettingsBuilder () =
...
    member inline _.Yield (b:Buffer) = Setting.Buffer b
    member inline _.Yield (c:Constraint) = Setting.Constraint c
...

You would also have to setup the correct _.Delay and _.Combine methods as well and it is critical to use _.Delay rather than _.Zero. Zero will force you to have to use yield before the Buffer "b1" { ... } lines as such: yield Buffer "b1" { ... }. otherwise the Buffer "b1" { ... } values will be ignored since these get compiled as F# code Sequences rather than _.Delay calls.

Note* If you get something odd in you CE, try examining the CE as a Quotation. <@ Settings { Buffer "b1" { Capacity 100.0 } } @>. This will generate a lot of Syntax Tree nodes, but it can explain why something is being ignored!

I also added a Custom Operation empty to the SettingsBuilder so that you can specify an empty Settings results (ie []).

...
let emptySettings =
    Settings {
        empty
    }
...
val emptySettings: Setting list = []

Since we are allocating Builders dynamically for the Buffer and Constraint settings. We will want then to be structs so that we don't pay for heap allocation when using them. Make sure to put the [<Struct>] attribute on a Builder type when your builders will be dynamically created, otherwise use a normal class since it will be accessing the builder instance via a static property reference.

Also the _.Yield (u:unit), _.Delay(a:unit->unit) and _.For(s:unit,f:unit->unit) methods together make the empty operation only able to exist by itself within the Settings CE.

you can find the Gist of the code here Sharplab.IO Gist

type Buffer =
    {
        Name : string
        Capacity : float
    }

type Constraint =
    {
        Name : string
        Limit : float
    }

[<RequireQualifiedAccess>]
type Setting =
    | Buffer of Buffer
    | Constraint of Constraint


// we make this type a Struct so that we can allocate on the stack for very low cost
type [<Struct; NoComparison; NoEquality>] BufferBuilder (name:string) =
    member this.Yield _ : Buffer = { Name = name; Capacity = 0.0 }
    member inline _.Run x : Buffer = x

    [<CustomOperation("Capacity")>]
    member inline _.Capacity (b: Buffer, newCapacity) =
        { b with Capacity = newCapacity }

let inline Buffer name = BufferBuilder name

type [<Struct; NoComparison; NoEquality>] ConstraintBuilder (name:string) =
    member this.Yield _ : Constraint = { Name = name; Limit = 0.0 }
    member inline _.Run x : Constraint = x

    [<CustomOperation("Limit")>]
    member inline _.Limit (b: Constraint, newLimit) =
        { b with Limit = newLimit }

let inline Constraint name = ConstraintBuilder name

type SettingsBuilder () =
    
    member inline _.Yield (u:unit) = () // used to indicate we are at the front of the computation expression with nothing defined yet
    member inline _.Yield (b:Buffer) = Setting.Buffer b
    member inline _.Yield (c:Constraint) = Setting.Constraint c
    // we use Delay and InlineIfLambda so that the aggressive F# compiler inlining will remove all the "function" calls
    // this produces straight line code after compilation
    member inline _.Delay([<InlineIfLambda>] a:unit -> Setting list) = a() // normal delay method for
    
    member inline _.Delay([<InlineIfLambda>] a:unit -> Setting) = [a()] // used to convert the last setting in the computation
                                                                        // expression into Setting list
                                                                        
    member inline _.Delay([<InlineIfLambda>] a:unit -> unit) = []       // used to allow empty to be used by itself
    
    member inline _.Combine(x1 : Setting, x2 : Setting list) =          // this is working backwards from the end of the computation
                                                                        // to the front
        x1 :: x2
    member inline _.For(s:unit, [<InlineIfLambda>] f:unit -> unit) =    // this makes empty only allowed in an empty Settings expression
        f()
    [<CustomOperation("empty")>]
    member inline _.Empty(s:unit) = () // this only can be called if the computation expression prior to here is the value unit which
                                       // can only happen if _.Yield (u:unit) = () was called prior to this and by returning unit we force
                                       // the For and Delay operations to have to use unit->unit which mean we can restrict this operation to
                                       // be the only operation allowed
    member inline _.Run(s:Setting) = [s] // allow a single Setting to be returned as a list
    member inline _.Run (x : Setting list) = x

let Settings = SettingsBuilder()

// This Computation Expression does work, in with the same API shape you would like to have
let mySettings =
    Settings {
        Buffer "b1" {
            Capacity 100.0
        }
        Constraint "c1" {
            Limit 10.0
        }
    }
(* THIS IS THE RESULTING DISASSEMBLED IL THAT mySettings becomes IN C#:
            Settings@67 = new @_.SettingsBuilder();
            builder@72 = new @_.BufferBuilder("b1");
            builder@72-1 = @_.builder@72;
            b@25 = new @_.Buffer(builder@72-1.name, 0.0);
            @_.Setting head = @_.Setting.NewBuffer(new @_.Buffer(@_.b@25.Name@, 100.0));
            builder@75-2 = new @_.ConstraintBuilder("c1");
            builder@75-3 = @_.builder@75-2;
            b@35-1 = new @_.Constraint(builder@75-3.name, 0.0);
            mySettings@70 = FSharpList<@_.Setting>.Cons(head, FSharpList<@_.Setting>.Cons(@_.Setting.NewConstraint(new @_.Constraint(@_.b@35-1.Name@, 10.0)), FSharpList<@_.Setting>.Empty));
*)   
let emptySettings =
    Settings {
        empty
    }
(* THIS IS THE RESULTING DISASSEMBLED IL THAT emptySettings becomes IN C#:
            emptySettings@90 = FSharpList<@_.Setting>.Empty;
*)

// Below shows that the desired outcome of `mySettings` would be
let b1 = { Name = "b1"; Capacity = 100.0 }
let c1 = { Name = "c1"; Limit = 10.0 }

let desiredSettings = [
    Setting.Buffer b1
    Setting.Constraint c1
]

(* THIS IS THE RESULTING DISASSEMBLED IL THAT desiredSettings becomes IN C#:
            b1@99 = new @_.Buffer("b1", 100.0);
            c1@100 = new @_.Constraint("c1", 10.0);
            desiredSettings@102 = FSharpList<@_.Setting>.Cons(@_.Setting.NewBuffer(@_.b1), FSharpList<@_.Setting>.Cons(@_.Setting.NewConstraint(@_.c1), FSharpList<@_.Setting>.Empty));
*)

mySettings = desiredSettings

In addition if you declare the BufferBuilder and ConstraintBuilder to be Struct types then the allocations are basically free since they won't be on the heap anymore. — Patrick Simpson, Aug 29 '22 at 05:28

F# Nested Computation Expression with Custom Operator

2 Answers2