Setting Up a Spanner Emulator in Spring Boot 🌐🔧

Efficiently Bridging Local Development and Cloud Services: A Guide to Setting Up the Spanner Emulator with Spring Boot

Choosing between testing directly in the cloud or using a local emulator is a key decision for developers. Cloud testing offers a real-world scenario, providing a thorough environment for testing applications. However, emulators present a convenient offline option, allowing for flexibility in testing, even though they may not support all the features of cloud services.

This guide focuses on setting up a local Spanner emulator. Utilizing Testcontainers and Spring Boot, we aim to create a testing environment that closely simulates Google Cloud Spanner, enhancing your local testing. This is particularly helpful for those familiar with Spring Boot, though the principles can be applied across various frameworks. Let's get started! 🚀

Spanner Emulator and Spring

First, let's create a basic test application featuring a single entity and repository. The complete code is available on GitHub for reference.

@Table(name = TABLE_NAME)
data class UserEntity(
    @PrimaryKey
    val id: String = UUID.randomUUID().toString(),

    @Column(name = "name")
    val name: String,

    @Column(name = "email")
    val email: String
) {
    companion object {
        const val TABLE_NAME = "users"
    }
}


@Repository
interface UserRepository : SpannerRepository<UserEntity, String>

The DDL for UserEntity is located under resources/db/user-ddl.sql:

CREATE TABLE users (
                       id STRING(36) NOT NULL,
                       name STRING(MAX) NOT NULL,
                       email STRING(MAX) NOT NULL,
) PRIMARY KEY (id)

Failing Test 🛑

Initially, our basic repository test is designed to fail.

@Test
fun `should fail without emulator`() {
    // given
    val entity = UserEntity("id", "name", "email")

    // when
    val actual = userRepository.save(entity)
    val read = userRepository.findById(entity.id)

    // then
    assert(actual == entity)
    assert(actual == read.get())
}

For this test, I created a completely new Google Cloud project. Upon execution, Spring attempts to connect to a Spanner database within that project. Since the Spanner API had not been activated, the test fails as anticipated, throwing a PERMISSION_DENIED error.

com.google.cloud.spanner.SpannerException: PERMISSION_DENIED: com.google.api.gax.rpc.PermissionDeniedException: io.grpc.StatusRuntimeException: PERMISSION_DENIED: Cloud Spanner API has not been used in project ... before or it is disabled.

Working Test with the Emulator ✅

To use the emulator, we must undertake two steps: enable it within the Spring properties and initiate the emulator itself. There are various methods to start the emulator, such as through the gcloud command line tool or docker-compose.

We opt for Testcontainers, which essentially mirrors the docker-compose solution (as it initiates docker containers) but integrates seamlessly with JUnit and the test lifecycle, eliminating the need for manual start or stop actions.

Enabling the emulator in your application properties is straightforward:

spring.cloud.gcp.spanner.emulator.enabled=true

Configuring the emulator requires a bit more effort:

@SpringBootTest
@ActiveProfiles("emulator")
@Testcontainers
internal class WithEmulatorTest {

   ....

    companion object {


        @JvmStatic
        @Container
        val spannerEmulator: SpannerEmulatorContainer =
            SpannerEmulatorContainer(
                DockerImageName.parse("gcr.io/cloud-spanner-emulator/emulator:latest"),
            )

        // Since we do not know the port of the emulator, we need to set it dynamically.
        @JvmStatic
        @DynamicPropertySource
        @Suppress("unused")
        fun emulatorProperties(registry: DynamicPropertyRegistry) {
            registry.add("spring.cloud.gcp.spanner.emulator-host") {
                "http://${spannerEmulator.emulatorGrpcEndpoint}"
            }
        }
    }
}

By activating Testcontainers with the @Testcontainers annotation and configuring the Spanner emulator, we encounter the challenge of connecting Spring to the emulator (as we do not know the container port). This is resolved by dynamically injecting (@DynamicPropertySource) the emulator's host URL into Spring's application properties before the tests begin.

Running the test now yields a different error, indicating progress:

com.google.cloud.spanner.InstanceNotFoundException: NOT_FOUND: com.google.api.gax.rpc.NotFoundException: io.grpc.StatusRuntimeException: NOT_FOUND: Instance not found: projects/test-project/instances/test-instance

Spring now recognizes the emulator but cannot find a configured Spanner instance. To address this, we directly use the Spanner API to add an instance and database to the emulator setup and execute the DDL before testing:

...
internal class WithEmulatorTest {


    @Autowired
    private lateinit var spanner: Spanner
...

    @Test
    fun `should work with emulator`() {
        // given
        val database = createDatabase()
        createUserTable(database)
        ...
    }

    private fun createInstance(): Instance {
        val instanceId = InstanceId.of(projectId, instanceId)
        val instanceInfo = InstanceInfo.newBuilder(instanceId)
            .setDisplayName("Test Instance")
            .setInstanceConfigId(
                InstanceConfigId.of(
                    projectId,
                    "config",
                ),
            )
            .build()
        return spanner.instanceAdminClient.createInstance(instanceInfo).get()


    }

    private fun createDatabase(): Database {
        createInstance()
        val databaseAdminClient = spanner.databaseAdminClient
        val databaseInfo = databaseAdminClient.createDatabase(
            instanceId,
            databaseId,
            emptyList()
        ).get()
        return databaseInfo

    }

    private fun createUserTable(database: Database) {
        val ddl = javaClass.getResource("/db/user-ddl.sql").readText()
        database.updateDdl(listOf(ddl), "createusertable").get()
    }

    ...
}

With these adjustments, the tests run as expected, allowing for application testing without a real Spanner setup.

I have put everything in one class to make it clearer. In the real world, you should make a base class for your tests that you can then reuse.

After successfully integrating the Spanner emulator, let's pause to consider its pros and cons, helping you make an informed choice for your project.

Pros and Cons 🚨✅

When integrating the Spanner emulator into your development process, it's important to weigh its advantages against potential limitations.

Pros:

• Cost-Effectiveness: 🤑 Utilizing the emulator can reduce expenses. Google Cloud Spanner's billing, based on service duration rather than request volume, means continuous operation of a Spanner instance can be costly.

• Offline Testing: 🚫🌐 Enables testing without an internet connection (I am looking at you, Deutsche Bahn!)

• Rapid Changes: ⏩ Allows quick updates to schemas and features without impacting colleagues or test environments.

Cons:

• Feature Limitations: 🚧 The emulator may not support every Cloud Spanner feature, potentially affecting testing accuracy for specific functionalities. See more on GitHub.

• Isolated Environment: Might not accurately replicate cloud environment behaviors like network latency, possibly leading to performance disparities.

• Versioning Challenges: 🏷️ The hard-coded version of the Spanner emulator docker container (e.g., using the latest tag) can lead to unpredictability. It's advisable to specify a particular version to ensure consistency with your development environment and avoid unexpected updates.

Conclusion 🎉

Setting up a local test environment with the Spanner Emulator and Testcontainers is straightforward and enhances efficiency in writing tests for your application. However, be mindful of the emulator's limitations. Ensure to complement this with end-to-end tests for specific Spanner behaviors.

Thank you for your time, and happy testing! 🚀

If this was helpful, consider showing your support with a coffee on Ko-Fi. It truly makes a difference. ☕️ Support here. Thank you!