Fluent API and builders: language-level support

[amal-stack]

Motivation

The fluent API and builder patterns are one of the most commonly used design patterns in C#. It is elegant, expressive and makes the code read like prose. Being used extensively and ubiquitously in libraries and public APIs, it serves as a fundamental building block (pun intended) of the language.

However, creating and extending fluent APIs may involve a lot of ceremony like:

• Creating a separate type
• Creating private fields to store state
• Creating a series of methods with the return type of the enclosing type to support chaining
• Assigning the method parameter to the private field
• Returning this

Therefore, there have been several discussions to make life a bit simple by suggesting to introduce language-level support for creating fluent APIs and builders, just like how the foreach construct offers language-level syntactical support for the iterator design pattern. Here are a few of these discussions:
#902 #311 #252 #7231 #169

The issues linked above suggest constructs like an implicit this return type to eliminate boilerplate.

Objectives

The objectives of this feature include:

1. Reduce the amount of ceremony to create fluent interfaces and implement the builder pattern. Some of the recurring steps when creating a fluent interface are:
   • Specifying the type of the builder as the method's return type.
   • Assigning the method's parameter to private fields.
   • Returning this.
2. Simplify the creation of a stepwise builder where the order of the method calls matter. Currently, this is implemented by creating a set of interfaces, one for each step.
3. Improve the experience of extending builders and fluent APIs via inheritance.

Feature Description

The fluent modifier

The first step of creating a builder or a fluent interface is to create the type. A marker keyword fluent can be used to declare that the type backs a fluent interface:

public fluent class Builder 
{
}

Behaviour of fluent types

Decaring a type with the fluent keyword permits enhanced syntax for creating chainable methods:

public fluent class Builder
{
    public WithName(string this.name);
}

1. Note how the return type is omitted in the WithName() method which implicitly causes the method to return the enclosing type Builder.
2. Secondly, the parameter declared string this.name causes it to implicitly assign the method's parameter name to a private field of the class name which would be available throughout the type's scope. Calling this method will set the autogenerated private field to the argument supplied during the call. Since this is what most methods do in builders, this can be implicit and the body can be omitted in this case.
3. The method does not have a body but will implicitly return this because the return type was omitted and the class is marked fluent.

Optional Body

The method can also have an optional body (block or expression-bodied), but the return this would be implicit, see the WithHeader() method below (expression-bodied). Since the parameters string name and string value below do not have a this. prefix, no implicit private field would be created for them:

public fluent class HttpRequestBuilder
{
     private Dictionary<string, string> _headers = new();

     WithHeader(string name, string value) => headers[name] = value;  // expression-bodied, but method implicitly returns "this"
     
     WithHeaders(params string[] headers)
     {
        // Optional body (block)
         if (headers.Length % 2 != 0)
         {
             throw new ArgumentException("There must be a value for each header", nameof(headers));
         }

         headers.Chunk(2)
             .ToList()
             .ForEach(pair => WithHeader(pair[0], pair[1]));

        // No need of "return this"
     }
}

Apart from the chainable fluent methods, a fluent type can have regular methods, properties, constructors and other members but if any of the methods omit the return type, it would implicitly return this.

Example: Before and after

Builders with methods that merely set a private field and return this, or as I like to call them, "anemic builders", would be greatly simplified. Here's a simple HttpRequestBuilder before and after applying the enhancements discussed.

Before

public class HttpRequestBuilder
{
    private HttpMethod _method = HttpMethod.Get;
    private string _requestUri = string.Empty;
    private HttpContent? _content;
    private List<KeyValuePair<string, string>> _headers = new();

    public HttpRequestBuilder Method(HttpMethod method)
    {
        _method = method;
        return this;
    }

    public HttpRequestBuilder Url(string requestUri)
    {
        _requestUri = requestUri;
        return this;
    }

    public HttpRequestBuilder Content(HttpContent content)
    {
        _content = content;
        return this;
    }
    
    public HttpRequestBuilder JsonContent<T>(T body, Action<JsonSerializerOptions> action = null)
    {
        JsonSerializerOptions options = new (JsonSerializerDefaults.Web);
        action?.Invoke(options);
        _content = new StringContent(JsonSerializer.Serialize(body));
        return this;
    }

    public HttpRequestBuilder Header(string key, string value)
    {
        _headers.Add(new KeyValuePair<string, string>(key, value));
        return this;
    }

     public HttpRequestBuilder Headers(params string[] headers)
     {
         if (headers.Length % 2 != 0)
         {
             throw new ArgumentException("There must be a value for each header", nameof(headers));
         }

         headers.Chunk(2)
             .ToList()
             .ForEach(pair => Header(pair[0], pair[1]));

        return this;
     }

    public HttpRequestMessage Build()
    {
        var request = new HttpRequestMessage(_method, _requestUri);
        request.Content = _content;
        _headers.ForEach(h => request.Headers.Add(h.Key, h.Value));
        return request;
    }

    private HttpRequestBuilder() {}
    
    public static HttpRequestBuilder Create() 
        => new HttpRequestBuilder();   
}

After

public fluent class HttpRequestBuilder   // fluent modifier enables simplified methods in the type without return types
{
    private List<KeyValuePair<string, string>> _headers = new();     // only one private field required
    
    public Method(HttpMethod this.method);    // the "this." prefix would create a private field named "method" 
                                              // accessible throughout the class scope, calling this method will set the private field as the supplied parameter
    public Url(string this.requestUri);                         

    public Content(HttpContent this.content);
    
    public JsonContent<T>(T body, Action<JsonSerializerOptions> action = null)  // simplified methods can also have an optional body without "return this"
    {
        JsonSerializerOptions options = new (JsonSerializerDefaults.Web);
        action?.Invoke(options);
       // use auto-generated field "this.content"
        this.content = new StringContent(JsonSerializer.Serialize(body));
    }

    public Header(string key, string value)            // expression bodies can be supported, implicitly returns "this"
        => _headers.Add(new KeyValuePair<string, string>(key, value));
    
     public Headers(params string[] headers)
     {
         if (headers.Length % 2 != 0)
         {
             throw new ArgumentException("There must be a value for each header", nameof(headers));
         }

         headers.Chunk(2)
             .ToList()
             .ForEach(pair => Header(pair[0], pair[1]));
     }

    public HttpRequestMessage Build()         // regular method with return type
    {
        var request = new HttpRequestMessage(method, requestUri);
        request.Content = content;
        _headers.ForEach(h => request.Headers.Add(h.Key, h.Value));
        return request;
    }
   
    private HttpRequestBuilder() {}            // constructor
    
    public static HttpRequestBuilder Create()  // static method
        => new HttpRequestBuilder();
}

Usage

The client would be able to use the HttpRequestBuilder the same way before and after the enhancements:

HttpRequestBuilder.Create()
    .Url("https://example.com")
    .Method(HttpMethod.Post)
    .Headers(
        "Accept", "application/json",
        "Authorization", "Bearer x"
    )
    .JsonContent(
        new { Name = "X", Quantity = 10 },
        opt => opt.PropertyNameCaseInsensitive = true
    )
    .Build();

Improving the Stepwise Builder

A stepwise builder is where you want control over the order in which the methods are chained. This is especially useful when one step is a prerequisite for the next step.

Traditional approach

The traditional approach to creating a stepwise builder is to create an interface for each step. For instance:

interface HeaderStep
{
    BodyStep HasHeader(string header);
}
interface BodyStep
{
    BodyStep HasBodyTitle(string title);
    FooterStep HasBodyDescription(string description);
}
interface FooterStep
{
    BuildStep HasFooter(string footer); 
}
interface BuildStep
{
    Document Build();
}

class DocumentBuilder : HeaderStep, BodyStep, FooterStep, BuildStep
{
    BodyStep HasHeader(string header) { ...; return this; }
    BodyStep HasBodyTitle(string title) { ...; return this; }
    FooterStep HasBodyDescription(string description) { ...; return this; }
    BuildStep HasFooter(string footer) { ...; return this; }
    Document Build() { return new Document(..); }
}

Enhancement: the on clause

Using the enhanced syntax, the interfaces can be completely eliminated using an on clause which will be permitted in types marked with the fluent modifier which specifies the methods after which the current method can be called:

fluent class DocumentBuilder
{
    HasHeader(string this.header);  

    HasBodyTitle(string this.title) on HasHeader;     // HasBodyTitle() can only be chained to HasHeader()

    private string _description;

    HasBodyDescription(string description) 
        on HasHeader, HasBodyTitle                    // HasBodyDescription() can be chained to either HasHeader() or HasBodyTitle()
    {
         _description =  description.Trim();
        // no return this
        // optional body, can also be expression bodied: (=> _description =  description.Trim();)
    }

    HasFooter(string this.footer) on HasBodyTitle, HasBodyDescription;      // HasFooter() can be chained to HasBodyTitle() or HasBodyDescription()
    Document Build() on HasFooter => new Document(..);                      // Build() can be chained to HasFooter() 
}

Using the contextual on keyword, the on clause specifies the methods after which the current method can be called. In the above example, the method HasBodyTitle() can only be called on the type that HasHeader() returns.
It can be read as HasBodyTitle can be called on HasHeader.
Similarly, the HasBodyDescription() method can be called either after calling HasBodyTitle() or directly after calling HasHeader() allowing the caller to skip the HasBodyTitle() step:

new DocumentBuilder()                              // Available methods: HasHeader()
      .HasHeader("C#")                             // Available methods: HasBodyTitle(), HasBodyDescription()
      .HasBodyTitle("Syntax sugar")                // Available methods: HasBodyDescription(), HasFooter()
      .HasBodyDescription("Enchanced fluent API")  // Available methods: HasFooter()
      .HasFooter("July 6th, 2023")                 // Available methods: Build()
      .Build();

Special case of the on clause

Here the HasHeader() method does not have an on clause. This can also be implemented in a way where the HasHeader() method is always available to be called either first or after every other method. To make the HasHeader() method only available to be called after creating the object and not after calling any other methods (in other words, it always has to be the first method to be called), a special case of the on clause can be introduced using one of the variations below:

HasHeader(string this.header) on class;     
HasHeader(string this.header) on _;
HasHeader(string this.header) on object;   

This would make HasHeader() unavailable after other methods are called. Methods with this special on class clause must be called on the object before other methods are called, even before methods without an on clause. There can be several methods with this constraint, for example, to provide multiple chain paths.

Alternative: the with clause

If specifying after which other methods a particular method can be called using the on clause seems counter-intuitive, a possible alternative is to specify on a method the methods that can be further chained to it. Using the same example above:

fluent class DocumentBuilder
{
    HasHeader(string this.header) with HasBodyTitle, HasBodyDescription; 

    HasBodyTitle(string this.title) with HasBodyDescription, HasFooter;

    private string _description;

    HasBodyDescription(string description) 
        with HasBodyTitle, HasFooter
    {
         _description =  description.Trim();
        // no return this
       // optional body, can also be expression bodied: (=> _description =  description.Trim();)
    }

    HasFooter(string this.footer) with Build;

    Document Build() => new Document(..); 
}

This can be read as HasHeader() can be continued with HasBodyTitle() and HasBodyDescription().

Here, it is unclear what method can be called first after creating the builder. So, it can be specified using the from keyword:

from HasHeader(string this.header) with HasBodyTitle, HasBodyDescription; 

This is equivalent to the special case of the on clause.

A method without an on or with clause will always be available to be called after any method or first. But if there are methods with the special case of the on clause (from clause in the second alternative), only they can be called first.

Fluent Interface with Inheritance

A common issue when creating a fluent API is extension via inheritance. When a derived builder class inherits from a base builder class, and when the consumer of the builder uses an object of the derived builder and calls a method of the base class, the methods of the derived class are masked out due to the base class method returning the type of the base class. A common but not so clean solution is to use recursive generics which come with their own limitations and also introduce unchecked casts with this.

Traditional approach: recursive generics

Extending fluent interfaces with inheritance often requires the use of a recursive generic type parameter on the base class. This allows propagate derived class information up the base class because the base class returns the self-generic parameter for its fluent methods which are later closed by a derived class via inheritance. Closing the self-referencing generic disallows further extension. However, if the derived class leaves the self-type parameter for further extension, it cannot be instantiated.
Consider the following example:

abstract class NamedObjectBuilder<TObject, TBuilder> 
    where  TBuilder : NamedObjectBuilder<TObject, TBuilder> // TBuilder is a self-referencing generic parameter
{
      protected string? _name;
      
      protected TBuilder This => (TBuilder) this;                      // unchecked "this" cast
      
      public TBuilder WithName(string Name)   // return open generic parameter TBuilder
      {
          _name = Name;
          return This;
      }
      
      public abstract TObject Build();
}

record Circle(string? Name, int Radius); 

class CircleBuilder : NamedObjectBuilder<Circle, CircleBuilder> 
// close the self-referencing generic with CircleBuilder
{
   private int _radius;

   public CircleBuilder WithRadius(int radius) // returns the closed type which disallows further extension via inheritance
   {
       _radius = radius;
       return This;
   }

   public override Circle Build() 
   {
       return new Circle(_name, _radius);
   }
}

Now CircleBuilder cannot be extended because the self-referencing generic parameter TBuilder is closed. However, if it is left opened as so:

class CircleBuilder<TBuilder> : NamedObjectBuilder<Circle, TBuilder> 
where TBuilder 
  : CircleBuilder<TBuilder>
{
   private int _radius;

   public TBuilder WithRadius(int radius)
   {
       _radius = radius;
       return This;
   }

   public override Circle Build() 
   {
       return new Circle(_name, _radius);
   }
}

In this case, it is open for extension but I cannot create an instance using new CircleBuilder<TBuilder>() because it is not possible to close the TBuilder argument.

Solution

When the types in the hierarchy are marked with a fluent modifier:

abstract fluent class NamedObjectBuilder<TObject>
{
      public WithName(string this.name);
      
      public abstract TObject Build();
}

fluent class CircleBuilder : NamedObjectBuilder<Circle>
{
   public WithRadius(int this.radius);

   public override Circle Build() 
   {
       return new Circle(_name, _radius);
   }
}

The compiler can add self-referencing constraints implicitly or generate intermediate types so that the WithName() method is callable after WithRadius(). Since these methods do not explicitly specify a return type, the compiler may substitute the derived types on the base methods when the derived object is created or a source generator could be used to pull the base class methods down to the derived class.

Benefits

Benefit	Description
Reduced boilerplate	As seen in the HttpRequestBuilder example above, the code has less boilerplate. Exactly 20 lines of code less after the enhancements, to be precise! Even though this example only has 7-8 methods. Considering the extent and ubiquity of builders and fluent types, this would have a huge impact. No explicit private field declarations, no return types in method signatures, no return this and allows the method body to be optional.
Concise code	The fluent modifier well communicates the intent of the type that its methods are chainable and even with fewer lines of code, it is easy to comprehend.
Encouragement to developers	Introduction of language-level support encourages developers to author and use fluent APIs more often, allowing them to write readable, and expressive code and DSLs with more ease and little ceremony.
Usage of expression-bodies	Allows more methods to be expression-bodied because the return this is implicit.
No interfaces for each step	The on/with clause helps to completely do away with interfaces for each step. Also, it is easier to visualize the order of the steps from the code than with interfaces.
Safer and extensible code with inheritance	When using inheritance, the recursive generic parameter can be omitted and unsafe generic casts with this can be avoided. And calling base class methods does not hide derived class methods.

Drawbacks

• Complexity: Incorporating the fluent interface into the language and compiler introduces additional complexity. While it offers the benefit of reducing code volume significantly, it comes at the expense of increased intricacy within the language itself.
• Reliance on Compiler Magic: This perspective is subjective, as personal preferences vary. Some individuals prefer things to be explicit and would argue too much abstraction and magic make things harder to grasp, especially for beginners.

[CyrusNajmabadi]

Looking at your "Person" example, it seems like the language already provides a very clean and easy to use system here with "records". You already get a terse way of declaring things, and you get the with operator for free. You can both have simple state properties, or computed properties. You can also add whatever other members you want.

[amal-stack]

While records can be a simple and terse way to model objects like a Person while also offering non-destructive mutation and deconstruction, fluent interfaces and builders have use cases that go beyond the scope and purpose of records. They are heavily used all over the place, like EF Core's fluent API for model configuration, .NET's generic host API and in libraries like Fluent Validation. The Person example is a minimal, watered-down example intended to demonstrate how the new syntax can have an optional method body.

[CyrusNajmabadi]

is a minimal, watered-down example

Unfortunately then, it makes for a non-compelling case since it looks like a place where a record would be preferred. Do you have an example where a record would not be a good choice?

[amal-stack]

I have updated the question with a more relevant HttpRequestBuilder example that would be better suited for a fluent API. Thank you for your feedback!

[rheesbeen]

Hi, I like the idea to clean this up. The only thing that I would change is the return type for a method, it’s now a bit unclear that a method explicitly returns this. What if you do it like this: public this WithHeader(string this.header); it would be more clear that the method returns an explicit this?

[amal-stack]

Yes, it did come to my mind once. We could write it either as you mentioned public this WithHeader(string this.header); or one other way would be just like the how the abstract modifier is required both on the class and the method, we could also use the fluent modifier on the method: public fluent WithHeader(string this.header);.

[AlexRadch]

@amal-stack Today I also came up with an idea on how to add convenient and performance-effective support for the Fluent API. It is a slightly different approach that came to my mind. Read my idea, it is somewhat similar to yours. Performance-effective Fluent API support

[m31coding]

Hey @amal-stack and @AlexRadch,

Thank you very much for your insightful design work on fluent builders. I particularly like the 'with' and 'on' syntax. I agree that supporting fluent builders at the language level would be highly beneficial. You might be interested in my library M31.FluentAPI. It leverages source generation to create stepwise fluent builders for classes based on attributes.

For example, the HttpRequest you discussed can be realized as follows:

[FluentApi]
public class HttpRequest
{
    [FluentMember(0)]
    public HttpMethod Method { get; private set; }

    [FluentMember(1)]
    public string Url { get; private set; }

    [FluentCollection(2, "Header")]
    public List<(string, string)> Headers { get; private set; }

    [FluentMember(3)]
    public HttpContent Content { get; private set; }

    [FluentMethod(3)]
    public void WithJsonContent<T>(T body, Action<JsonSerializerOptions>? configureSerializer = null)
    {
        JsonSerializerOptions options = new JsonSerializerOptions(JsonSerializerDefaults.Web);
        configureSerializer?.Invoke(options);
        Content = new StringContent(JsonSerializer.Serialize(body));
    }

    [FluentMethod(4)]
    [FluentReturn]
    public HttpRequestMessage GetMessage()
    {
        HttpRequestMessage request = new HttpRequestMessage(Method, Url);
        request.Content = Content;
        Headers.ForEach(h => request.Headers.Add(h.Item1, h.Item2));
        return request;
    }
}

Usage:

HttpRequestMessage message = CreateHttpRequest
    .WithMethod(HttpMethod.Post)
    .WithUrl("https://example.com")
    .WithHeaders(("Accept", "application/json"), ("Authorization", "Bearer x"))
    .WithJsonContent(
        new { Name = "X", Quantity = 10 },
        opt => opt.PropertyNameCaseInsensitive = true)
    .GetMessage();

Console.WriteLine(JsonSerializer.Serialize(message));

Looking forward to any feedback you might have.

Happy Coding!,
Kevin

[AlexRadch]

Very cool! Thank you, I will use it.