Kotlin DSL Mastery: Techniques, Patterns & Implementation
Picture yourself developing a complex software system that involves intricate configurations, rule definitions, or data transformations. The mainstream programming language you’re using might adequately solve the problem, but the resulting code could become verbose, convoluted, and challenging to maintain. This is where a DSL shines.
Kotlin’s succinct syntax, robust type system, and extension function support render it an optimal choice for constructing internal DSLs.
Before we delve into the nitty-gritty, let’s closely examine DSLs in general.
What is a DSL, Anyway?
Let us first introduce another term, the “General Purpose Language” (GPL). A GPL is what we more often just call “programming language”: a language which can be used to create virtually everything. Kotlin is a GPL. You can create any program you want with Kotlin, from simple calculators to entire business applications.
So what is a “Domain Specific Language” then? Martin Fowler defines it as:
a computer programming language of limited expressiveness focused on a particular domain. (see the reference section below)
So first of all, it is still a programming language, no surprise. But unlike a GPL, a DSL has some fluency in it. Code in a DSL is often closer to a real language than the same construct in a GPL is.
Next, it has “limited expressiveness”. Put simply, you cannot use a calculator DSL to create a word processor. A DSL supports only what is needed for the specific domain and not more. It usually does not contain any loops, conditions and there like.
Lastly, it is focused on and specially made for a specific domain. Domain focus and limited expressiveness go hand in hand and influence each other.
Furthermore, a distinction can be made between external and internal DSLs.
While an internal DSL is written in the same language as the host language (or the program in which the DSL is used), an external DSL is not. Examples for external DSLs are XML (e.g. Spring Bean Configurations) or SQL.
We will focus on an internal DSL, as we write and execute it in Kotlin. This way, we do not have to think much about grammar, parsing and so on as we would have to in an external language.
Benefits of DSLs
Certainly, creating DSLs is enjoyable, and we all love tackling interesting tasks. But what exactly are the benefits — and potential drawbacks — of using a DSL?
In my opinion, the main benefit of a DSL is that it makes the life of the user easier. Imagine that you need to write imperative Kotlin code to query a database instead of using a domain-specific language like SQL. You would need much more knowledge about the underlying system — and certainly need to write more code!
The fewest write such a powerful DSL as SQL. A more frequent use of small, custom DSLs is to configure another object or hierarchy of objects. A good example of this is Ktor’s DSL to configure HttpClient:
val client = HttpClient(CIO) {
install(Logging) {
logger = Logger.DEFAULT
level = LogLevel.HEADERS
filter { request ->
request.url.host.contains("ktor.io")
}
sanitizeHeader { header -> header == HttpHeaders.Authorization }Without the DSL, you would have to create a bunch of objects on your own and wire them together. For example, you might need to create a Logger instance and a Filter instance and then add them to the HttpClient.
Another example is Anko, a DSL for creating Android layouts:
verticalLayout {
editText {
hint = "Enter your name"
}
button("Submit")
}You can see both DSLs as thin layers over an existing API or model. They make the use of the API easier since the DSLs are much more limited. It is harder to do the wrong thing.
However, before one can use a DSL, one must create it. You have to ask yourself if a DSL really provides a benefit. Is the API so complex that you can easily make errors, or are there only three to four objects that you can wire together easily? Is there already a fluent builder (which is — to be honest — also a form of a DSL)?
Plus, you also have to maintain it. If the underlying model changes, you may have to adapt the DSL. So think twice if the extra work brings enough benefits in the end.
Crafting a DSL in Kotlin
Now, let’s roll up our sleeves and delve into creating a DSL in Kotlin. Along the way, we’ll explore more theoretical concepts and discover some patterns. Essentially, there are two primary approaches to embarking on a DSL journey.
First, you might have an existing model or API to which you wish to add a DSL layer. Alternatively, you could take the opposite route: initially forge the DSL, possibly collaborating with a domain expert, and then proceed to shape the underlying model accordingly.
For this endeavor, we’ll be adopting the first approach. Consider a scenario in which you need to integrate the following Java API into your Kotlin project:
public class Speaker {
private String name;
private String bio;
private List<String> expertise;
public Speaker(String name, String bio, List<String> expertise) {
this.name = name;
this.bio = bio;
this.expertise = expertise;
}
// .. getter and setter
}
private String title;
private final List<Speaker> speakers = new ArrayList<>();
private String startTime;
private List<String> audience;
public Talk(String title, String startTime, List<String> audience) {
this.title = title;
this.startTime = startTime;
this.audience = audience;
}
// .. getter and setter
public class Conference {
private List<Speaker> speakers;
private List<Talk> talks;
private String name;
private String organizerContact;
public Conference(String name, String organizerContact) {
this.name = name;
this.organizerContact = organizerContact;
speakers = new ArrayList<>();
talks = new ArrayList<>();
}
public void addSpeaker(Speaker speaker) {
speakers.add(speaker);
}
public void addTalk(Talk talk) {
talks.add(talk);
}
// .. getter and setter
}
You can find the complete code on GitHub. We can use the data model to create a conference schedule like this:
Conference conference = new Conference("TechConf 2023", "info@techconf.com");
// Create speakers
Speaker speaker1 = new Speaker("John Doe", "Software engineer and tech enthusiast.",
List.of("Java", "Python"));
Speaker speaker2 = new Speaker("Jane Smith", "AI researcher and data science enthusiast.",
List.of("Machine Learning", "Natural Language Processing"));
// Create talks
Talk talk1 = new Talk("Introduction to Java", "9:00 AM",
List.of("Developers", "Java enthusiasts"));
talk1.addSpeaker(speaker1);
talk1.addSpeaker(speaker2);
Talk talk2 = new Talk("AI in Everyday Life", "11:30 AM",
List.of("General audience"));
talk2.addSpeaker(speaker2);
// Add speakers, rooms, and talks to the conference
conference.addSpeaker(speaker1);
conference.addSpeaker(speaker2);
conference.addTalk(talk1);
conference.addTalk(talk2);Writing the DSL
We can address multiple concerns with our DSL. First, the model’s usage can lead to code verbosity. Second, there’s a risk of introducing inconsistent data: a Talkcould be associated with a Speakernot added to the Conferenceobject. There is also the chance that one is adding a Speaker without adding her to a Talk . The use of Java for the model, instead of Kotlin, adds an extra layer of awkwardness to the situation.
Let us create the starting point for our DSL.
// DSL Builder
class ConferenceBuilder1 {
var name: String = ""
var organizerContact: String = ""
fun build(): Conference {
return Conference(name, organizerContact)
}
}
// Entry Point for the DSL
fun conference1(init: ConferenceBuilder1.() -> Unit): Conference {
val dsl = ConferenceBuilder1()
dsl.init()
return dsl.build()
}
One can easily see that ConferenceBuilder1 will hold the conference data and assembles them to a Conferenceobject once we are done. The conference1function is the entry point for our DSL. We use it like this:
conference1 {
name = "TechConf 2023"
organizerContact = "organizer"
}But what exactly is ConferenceBuilder1.() -> Unit? Granted, it might seem a bit unusual at first glance. This construct is known as a function literal with receiver. To put it simply, you can see it as an extension function designed for our ConferenceBuilder1, but with a restricted scope. Its visibility is limited solely to the boundaries of the conferencefunction. As we continue to the next step, we’ll dive into why this concept turns out to be very useful.
Next, we aim to enable the addition of speakers to our conference. Nevertheless, is “add” the appropriate term? In the context of the “Conference” domain, a speaker is actually booked rather than simply added. The DSL will look like this after we finished our changes:
conference2 {
name = "TechConf 2023"
organizerContact = "organizer"
speaker {
book {
name = "John Doe"
bio = "Software engineer and tech enthusiast."
expertise = mutableListOf("Java", "Python")
}
book { ... }
}
}This appears more like natural language rather than a series of get and set instructions. To implement the new functionality, we will once again utilize builders and literal functions with receivers. Let’s begin by focusing on the speaker :
// The new builder
class SpeakerListBuilder {
private val speaker = mutableListOf<Speaker>()
fun build(): List<Speaker> {
return speaker
}
}
class ConferenceBuilderWithSpeaker {
//...
fun speaker(init: SpeakerListBuilder.() -> Unit) {
val dsl = SpeakerListBuilder()
dsl.init()
speaker.addAll(dsl.build())
}
fun build(): Conference {
val conference = Conference(name, organizerContact)
conference.speakers.addAll(speaker)
return conference
}
}This allows us to include an empty speakerblock within our conferenceblock. To make it practical, we will now integrate the capability to effectively book speakers, using the same approach! Employing nested builders is a widely used method for implementing a DSL in Kotlin. Therefore, let’s incorporate the SpeakerBuilder to add (no, book!) speakers.
class SpeakerListBuilder {
//...
// this enables us to add speakers
class SpeakerBuilder {
var name: String = ""
var bio: String = ""
var expertise = mutableListOf<String>()
fun build(): Speaker = Speaker(name, bio, expertise)
}
// this enables the book { ... }
fun book(init: SpeakerBuilder.() -> Unit) {
val dsl = SpeakerBuilder()
dsl.init()
speaker.add(dsl.build())
}
}There is one aspect in the current version that requires improvement. Employing a list in the DSL blends the underlying Kotlin syntax with our DSL :
// current version
speaker {
//...
expertise = mutableListOf("Java", "Python")
}
// desired version
speaker {
// ...
expertise {
+"Java"
+"Python"
}
}This +Value syntax is commonly utilized in Kotlin DSLs when a list needs to be constructed. This is possible thanks to Kotlins ability to overload operators. It could be argued that this doesn’t resemble domain language — and that’s a valid point. We will explore an alternative approach later. But for now, let’s implement this method. We utilize the same approach once again.
class SpeakerListBuilder2 {
// ...
class SpeakerBuilder2 {
// ...
private var expertise = mutableListOf<String>()
class ExpertiseBuilder {
val expertise = mutableListOf<String>()
operator fun String.unaryPlus() {
expertise.add(this)
}
}
fun expertise(init: ExpertiseBuilder.() -> Unit) {
val dsl = ExpertiseBuilder()
dsl.init()
expertise.addAll(dsl.expertise)
}
}
}Great! Now, let’s apply these techniques to include the Talk section as well. When we examine the data model, we notice that Talkreferences Speaker. This prompts the question of how we can add Speakers to a Talk in the DSL without duplicating all the information. Ideally, we would only need to input the speaker’s name into the talksection of the DSL. To map from a speaker’s name to the Speaker object, we employ a concept referred to as a Symbol Table in DSL terminology. While it may sound sophisticated, in reality, it’s simply a map.
First, we add the Symbol Table to our ConferenceBuilder :
class ConferenceBuilderComplete {
// ...
private val speakerSymbolTable = mutableMapOf<String, Speaker>()
private val talks = mutableListOf<Talk>()
fun speaker(init: SpeakerListBuilder2.() -> Unit) {
val dsl = SpeakerListBuilder2()
dsl.init()
val bookedSpeaker = dsl.build()
speakerSymbolTable.putAll(bookedSpeaker.map { it.name to it })
speaker.addAll(bookedSpeaker)
}
// ...Then, we add the code to configure talks. This is similar to the previous code. See how the Symbol Table is used to find the correct Speaker objects!
class TaskListBuilder1(private val speakerSymbolTable: Map<String, Speaker>) {
private val talks = mutableListOf<Talk>()
fun schedule(init: TalkBuilder.() -> Unit) {
val dsl = TalkBuilder(speakerSymbolTable)
dsl.init()
talks.add(dsl.build())
}
class TalkBuilder(private val speakerSymbolTable: Map<String, Speaker>) {
// ... (see the complete code on GitHub!)
// holds the speaker names which the user enters in the dsl
private val speaker = mutableSetOf<String>()
// same as the expertise above
class SpeakerNameListBuilder {
val speaker = mutableListOf<String>()
operator fun String.unaryPlus() {
speaker.add(this)
}
}
// enables us to use speaker { ... }
fun speaker(init: SpeakerNameListBuilder.() -> Unit) {
val dsl = SpeakerNameListBuilder()
dsl.init()
speaker.addAll(dsl.speaker)
}
fun build(): Talk {
// ...
require(speakerSymbolTable.keys.containsAll(speaker)) { "Speaker must be booked before scheduling talk" }
talk.speakers.addAll(speakerSymbolTable.filterKeys { speaker.contains(it) }.values)
// ...
}The only remaining element is the audiencefor a Talk. Since this also involves a list of String, we could employ the same approach as we did with expertiseor speakers— and I would suggest doing so. However, as previously discussed, this doesn’t align well with natural language. We can implement it in a manner that allows one to add the audience like this:
audience {
may be "Developers"
may be "Java enthusiasts"
}Now, this resembles an actual sentence. We can achieve this by adding the following to the TalkBuilder:
// in TalkBuilder
object may
class AudienceListBuilder {
val audienceCollection = mutableListOf<String>()
infix fun may.be(name: String) {
audienceCollection.add(name)
}
}
fun audience(init: AudienceListBuilder.() -> Unit) {
val dsl = AudienceListBuilder()
dsl.init()
audienceCollection.addAll(dsl.audienceCollection)
}As you can observe, it appears coherent within the DSL, but it might appear unusual in the code. This becomes even more confusing if you try to add longer sentences. Another approach to achieve a similar outcome is:
// in TaskBuilder
val audience = this
infix fun mayBe(audience:String){
audienceCollection.add(audience)
}
// in the DSL
audience mayBe "Developers"While this is a bit less sentence-like, it’s still closer to natural language compared to using +"Developers". This serves as a strong example illustrating the importance of maintaining the proper equilibrium between a “language-like” approach and maintainability when developing a DSL.
Reviewing the DSL
We’ve made significant progress. You can take a look at the complete DSL on GitHub. But there is one issue. Currently, it is allowed to create a talk block inside a speaker block and vice versa. Even worse, those blocks may be nested. But this should not be the case!
// this may not be allowed!
conferenceNoMarker {
speakers {
talks { }
}
talks {
speakers {
speakers {
talks { }
}
}
}
}Kotlin provides a straightforward solution for this using the @DslMarkerannotation. We need to create our custom annotation and use it to annotate our builder classes:
@DslMarker
annotation class CompleteConferenceDsl
// ...
@CompleteConferenceDsl
class TalkListBuilder
@CompleteConferenceDsl
class SpeakerListBuilderFinal
This approach prevents nesting. You’ll receive a compile error:
fun speaker(init: SpeakerListBuilderFinal.() -> Unit):
Unit' can't be called in this context by implicit receiver.
Use the explicit one if necessaryAnd that’s it — our DSL is complete! Remember that neither the model nor the DSL is flawless. I’ve chosen this setup for illustrative purposes.
References — Delve deeper into DSLs
Although slightly dated, Domain Specific Languages by Martin Fowler contains a lot of valuable information. The book is divided into two parts. The first part contains general information about DSLs. While the code examples may be dated, the primary focus of the book is to establish a theoretical foundation. The second part comprises a pattern catalog. In fact, I used this book as the primary source. I recommend reading chapters one and two, along with specific sections of chapter three and chapter four. Afterward, you can skim through the patterns and read them as needed.
The official Kotlin documentation has a great chapter about the type-safe builders we use: https://kotlinlang.org/docs/type-safe-builders.html
Conclusion
In this article, we saw that Kotlin is great for creating internal DSLs.It provides useful features like:
Infix Functions: https://kotlinlang.org/docs/functions.html#infix-notation
Operator Overloading: https://kotlinlang.org/docs/operator-overloading.html
Type-Safe Builders: https://kotlinlang.org/docs/type-safe-builders.html
Thank you for reading my article! You can find the complete source code on GitHub!