Mastering API Visibility in Kotlin

This article is also available as a talk, check out the slides and recordings here.

Introduction

While the focus of this article is libraries, everything here applies to any project that has multiple modules. Each of those modules presents a public API to the outside world, exactly the same way a library does.

When designing a library, minimizing your API surface – the types, methods, properties, and functions you expose to the outside world – is a great idea.

It might be a cliché at this point to reference Effective Java, but like many great ideas, it has an item covering this: Item 15: Minimize the accessibility of classes and members. This is the very first item of Chapter 4: Classes and Interfaces, which further highlights just how important it is.

There’s a huge amount of great advice in that item, and most of it won’t be covered here, as the focus will be on the Kotlin implementations of the ideas – so do read it after this article.

Some reasons for minimal APIs

Before we get to the practical part, let’s quickly see some simple reasons for minimizing the API surface of a module.

Easier to maintain and change

The fewer public declarations you have, the fewer things you’re stuck with supporting in the future. While APIs are not forever, when you make something public, the expectation is that it’ll be there for a long time, and importantly, that it will behave the same way for a long time.

Anything you make public, whether at the source or binary level, will break client code if you make changes to it. Breaking public API is a bother for your clients – who almost certainly have better things to do than track the changes of your library –, and deprecation processes can be long and complicated, even though Kotlin helps a lot with this. Read Maintaining Compatibility in Kotlin Libraries for more about compatibility and deprecation.

On the other hand, anything that’s hidden from your clients can change freely, without having to worry about the impact on them (as long as public API’s behaviour stays the same, that is).

Easier to learn

If you have fewer classes and fewer members in those classes, your API is also easier to use. Trial and error, as well as random discovery through IDE autocompletion results becomes way more feasible if clients only see the things that they’re supposed to reference in their code.

Make your API focused on the task that it’s supposed to perform, and make it easy to use by providing some entry points and obvious actions from those entry points.

Harder to misuse

Being mindful about what clients of your code will see also prevents them from misusing your library. If they can’t call things that they’re not supposed to, you’ll avoid many complaints about your code misbehaving due to improper use.

So, a minimal API is something to strive for, but how do you get there?

One good approach is to start coding by designing your public API first, and only adding the implementation code after that’s established and reviewed (sometimes called coding against an interface). This helps clearly separate the public API from the private implementation details.

Now, let’s get this into practice, and see how Kotlin manages API visibility.

Internal visibility

Your best friend for removing implementation details from public API in Kotlin is the internal keyword. This visibility is somewhat like package-private visibility in Java, but it makes declarations available to an entire module instead of just a single package. This lets you more freely organize your files into packages, as that won’t affect accessibility.

This is the perfect level of visibility for many library constructs, for example:

• Data classes that you use in many places within your library, but never in public API
• Implementation classes that are exposed publicly through interfaces
• Setters of mutable properties that clients should read, but not modify
• Extensions on common types that you don’t want to pollute client projects with

For all of these, marking them private would prevent you from referencing them outside of the file they’re in, making them very difficult to use. Marking them public to make them broadly accessible would also add them to the public API for clients.

internal is a great intermediate step between private and public that conveniently makes them available only within your library.

Testing (attention, please)

This visibility level is also useful for declarations that you want to expose only for testing, as internal declarations are made available to test source sets within the same module.

If you do expose a declaration this way, and you’re on Android, you can complement the internal modifier with a @VisibleForTesting annotation, and indicate what its visibility would be, if it weren’t for tests:

@VisibleForTesting(otherwise = PRIVATE)
internal var state: State

Java interop considerations

Java clients will still technically be able to reference declarations with internal visibility if they really want to, as this visibility level doesn’t exist on the JVM – internal declarations are public in the bytecode.

If you’re using IntelliJ IDEA, it will attempt to warn you when you do something like this, but it still won’t break compilation.

However, the names of these references will be mangled by the compiler. This will likely discourage anyone from using them. For example, an internal method called createEntity will become something like createEntity$moduleName (on Android, it’s also suffixed by the build variant in addition to the module name).

To override this mangling, you can add the @JvmName annotation, and make the declaration visible from Java by a specified name. This can be used to make the name nicer, but also to make it even scarier for Java clients.

class Repository {
    @JvmName("pleaseDoNotCallThisMethod")
    internal fun createEntity() { ... }
}

void repositoryExample() {
    new Repository().pleaseDoNotCallThisMethod();
}

However, if you want to make a declaration completely inaccessible from Java, the best way to go is the @JvmSynthetic annotation: this prevents Java source code from referencing it.

class Repository {
    @JvmSynthetic
    internal fun createEntity() { ... }
}

void repositoryExample() {
    new Repository().createEntity();
                  // ^ e: Cannot resolve method 'createEntity' in 'Repository'
}

Explicit API mode

Since all Kotlin declarations are public by default, it’s easy to create a class for internal use and simply forget to put an appropriate visibility modifier on it. A great way to force yourself to think more about visibility is the Explicit API mode introduced recently in Kotlin 1.4. Enabling this will force you to add explicit visibility modifiers for all declarations.

After turning on this mode, for each declaration, you’ll have to either:

• Verify that it’s API that you meant to make publicly available, by marking it with the public keyword explicitly, or
• Fix its visibility by restricting it, for example, making it internal (at least for starters – stricter visibility is even better).

To enable explicit API mode, you can add one of the following options to your module’s build script:

kotlin {
    explicitApi() // or explicitApiWarning()
}

Or, alternatively, add one of the items below to your Kotlin compiler options, which I recommend doing, as you’ll likely have other options there as well:

tasks.withType(org.jetbrains.kotlin.gradle.tasks.KotlinCompile).all {
    kotlinOptions {
        freeCompilerArgs += [
            '-Xexplicit-api=strict', // or '-Xexplicit-api=warning'
        ]
    }
}

The default (strict) mode will give you errors for any declarations that don’t comply with explicit API mode, while the alternative configuration shown in comments will only emit warnings.

Biting the bullet and reviewing your API in one go can be great, because it will immediately force the requirements on any new code added to the codebase. You can open a single PR to enable strict explicit mode, and have your team review all newly added visibility modifiers.

Remember that explicit API mode is enabled per-module. This means that you can opt into it gradually within your project. You also don’t need to enable it for modules that nobody will depend on. For example, Android application modules likely don’t need explicit API mode.

If you have large modules, enable just warnings at first to get a nice incremental path to an explicit API within a module. Then, when you’re done with the migration, you should enable errors to enable strict checks on new code in the module from that point forward.

We’re focusing on visibility in this article, but explicit API mode also forces you to add explicit types for all public API. This prevents entire classes of bugs, for example accidentally changing public API due to type inference.

Published API

An interesting edge case of visibility comes up in Kotlin when you provide inline functions to your clients. In these functions, you can’t reference anything with a visibility sticter than that of the inline function. For example, you can’t call an internal API in a public inline function.

internal fun secretFunction() {
    println("through the mountains")
}

public inline fun song() {
    secretFunction()
}   // ^ e: Public-API inline function cannot access non-public-API

When code calling an inline function is compiled, the body of the function is copied to the call site. This would lead to problems if something referenced inside an inline function was private:

• The resulting client bytecode would contain direct references to non-public API, which would break access restrictions.
• Since there’d be a reference to an internal function in compiled client code, making changes to it would break client binaries, even though it’s not public API. This is highly unexpected, and produces errors only at runtime (yikes!).

In some cases, you can consider simply making problematic functions non-inline. With a higher order function that takes functions as parameters, this will mean a performance penalty in exchange for hiding your implementation details better. However, there are cases where using inline is simply unavoidable: when using reified generics.

To solve this problem in a neater way, you can use the @PublishedApi annotation. Marking something that’s otherwise internal with @PublishedApi will let you use it in public inline functions. However, clients still won’t be able to reference your function in source code.

@PublishedApi
internal fun secretFunction() {
    println("through the mountains")
}

public inline fun song() {
    secretFunction()
}

fun clientCode() {
    song() // ok
    secretFunction() // e: Cannot access 'secretFunction': it is internal
}

The annotation in your code will serve as a reminder that you now have to treat that declaration as effectively public API, because making changes to it will break binary compatibility (but not source compatibility, since client sources won’t contain references to it). If you want to learn more about source and binary compatibility, read Maintaining Compatibility in Kotlin Libraries.

Opt-in APIs

One problem you might run into if you have multiple modules within a library is that you might want to share some constructs between your own modules, but not with the outside world.

For example, you might have a core module that contains the essential parts of your library, and an addon module that provides optional extra functionality built on top of the core module. The addon module’s implementation might have to access declarations that clients using the core module shouldn’t be able to access.

You have two available options:

• Mark these declarations public, and hope that your clients don’t use them. You can try scaring them away with comments or deprecation annotations (which you’ll then have to suppress warnings about in your own code).
• Mark these declarations internal. This will unfortunately prevent you from referencing them in the addon module, unless you hack around it.

What’s needed here is some sort of “library group internal” visibility, between the internal and public visibilities.

While that doesn’t exist in Kotlin, and will likely never be added to the language, there is a neat first-party solution to achieve something very similar, called opt-in requirements. This builds on the first idea above – it will make the declarations public, and attempt to scare clients away from using them.

You’ve already encountered this feature if you used annotations like @InternalCoroutinesApi or @ExperimentalTime before, which are some first-party examples of opt-in APIs.

To create opt-in requirements for your own APIs, you have to create a new annotation class, and annotate it with @RequiresOptIn:

package com.example.lib.core

@RequiresOptIn(
        level = RequiresOptIn.Level.ERROR, // can be relaxed to a WARNING
        message = "This is internal API for my library, please don't rely on it."
)
public annotation class InternalMyLibraryApi

You get to specify a severity level – similarly to deprecations – and add a message explaining the purpose of the annotation. The @RequiresOptIn annotation itself requires opt-in, as this is still an experimental feature. To opt-in to using that annotation for a module, a compiler option has to be added:

tasks.withType(org.jetbrains.kotlin.gradle.tasks.KotlinCompile).all {
    kotlinOptions {
        freeCompilerArgs += [
            '-Xopt-in=kotlin.RequiresOptIn',
        ]
    }
}

If you already have explicit API mode configured, just add this option to the existing list of compiler options.

With this done, you can mark your “library group internal” constructs in the core module with the custom annotation:

@InternalMyLibraryApi
public fun coreApi()

Then, when you reference something annotated with an annotation that requires opt-in, you can pick from the following:

• Opt-in to using that API in a module, by adding a compiler option, the same way as the initial opt-in to use @RequiresOptIn, but specifying your own annotation:

  tasks.withType(org.jetbrains.kotlin.gradle.tasks.KotlinCompile).all {
      kotlinOptions {
          freeCompilerArgs += [
              '-Xopt-in=com.example.lib.core.InternalMyLibraryApi',
          ]
      }
  }
  

  With this enabled, anything annotated with @InternalMyLibraryApi becomes available as public API within the module.

• Opt-in to using that API locally, for a statement, method, class, or file (using annotation use-site targets):

  public fun addonFunction() {
      @OptIn(InternalMyLibraryApi::class)
      coreApi()
  }
  

  Using OptIn is experimental and it requires you to opt-in to using the RequiresOptIn annotation (with -Xopt-in=kotlin.RequiresOptIn) in your build configuration.

• Propagate the opt-in requirement to anyone calling your code:

  @InternalMyLibraryApi
  public fun addonFunction() {
      coreApi()
  }
  

  With this option, anyone calling addonFunction will have to either propagate the opt-in requirement further, or opt-in to using @InternalMyLibraryApi APIs as shown above.

Just like the internal visibility, this opt-in feature is not respected in Java. The compiler produces no warnings or errors for Java source code referencing opt-in APIs. Again, using @JvmSynthetic to hide these APIs from Java might be a good idea here.

Conclusion

That’s a wrap! Hope you find all of these practices helpful, and will apply it to your own projects. If you have any questions, feel free to reach out. If you want to learn more, check out the references listed below.

Thanks to everyone who reviewed this article before its release, including @stewemetal and a handful of awesome Android GDEs.

Also thanks to the great people on the Kotlin team who pointed me towards the opt-in solution for sharing library-internal stuff in their recent Reddit AMA.

References

• Effective Java, 3rd edition (Amazon)
• Maintaining Compatibility in Kotlin Libraries
• Official Kotlin documentation
  • Visibility Modifiers
  • KEEP: Explicit API mode
  • What’s New in Kotlin 1.4: Explicit API mode for library authors
  • @PublishedApi
  • Opt-in Requirements