Writing Unit Tests efficiently in Android with Spek
Ever thought organizing test cases in Android are cumbersome? For instance, you have a function and there are 3 scenarios you want to test, for example, for 3 different inputs, it's supposed to return 3 different outputs, and you want to test this behavior, now what we mostly do in Android and JUnit 4 is create 3 different functions and name the functions like
fun `test functionName should return output1 for input1`()
If the target class has 10 more functions then, assuming you'll want to test each function in multiple scenarios, this Test class may end up with about 25-30 more such test cases, but there's no way to group them.
Let me be more specific about what I want, I still want all those tests within a single file, because they cover a single class that I'm testing, but I want to group all the test cases for a single function together, this way it's easy to find issues if something is broken.
If you're from web / JavaScript background, then you probably already know what I mean, and probably experienced it first hand. What I'm talking about is grouping of test cases like it's done in Jasmine or RSpec or even Cucumber's Gherkin for the matter. If you're not familiar with the above-mentioned test frameworks, you can refer to the cover image of this post to get an idea about how it'll look.
Well, it's now possible in Android, thanks to JUnit5 and JetBrains' Spek.
I was first introduced to Spek about two years ago by my dear friend Adit Lal and using it for personal projects ever since. Let me be honest with you, writing test cases with Spek is pure joy, it provides you two different styles of DSL (more on it later), and you can use your preferred one to write and better organize your test cases. The catch, however, is setup. As you might already know, JUnit5 doesn't support Android (or vice-versa) out-of-the-box, and Spek being the JUnit5 test framework (for Kotlin/JVM), you first need to set up JUnit 5 in your project, following which you need to setup Spek.
Since Spek setup isn't a straightforward process and getting frustrated is pretty easy here, I'll first show here how to setup Spek and JUnit 5, before discussing on writing test cases with Spek. If you want to skip the setup part, you can directly jump here
Setting up Spek and JUnit5 in your Android Project
We'll be using Spek 2.x, and we'll need to install the official Spek Framework plugin for that. You can get it here: https://plugins.jetbrains.com/plugin/10915-spek-framework. Install it and then restart your IDE (Android Studio or IntelliJ IDEA). Below is a screenshot from Android Studio -> Preferences -> Plugins (for Mac) after installing the plugin.
Once you've installed and restarted the IDE, we can proceed towards configuring your project for Spek and JUnit5.
First, we'll need to add JUnit5 and JaCoCo BuildScript dependencies and Spek Maven repo to your project level build.gradle file as shown below.
// Top-level build file where you can add configuration options common to all sub-projects/modules.
buildscript {
ext.kotlin_version = "1.3.72"
repositories {
google()
jcenter()
}
dependencies {
classpath "com.android.tools.build:gradle:4.0.0-beta04"
classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:$kotlin_version"
classpath "de.mannodermaus.gradle.plugins:android-junit5:$android_junit5_version"
classpath "org.jacoco:org.jacoco.core:$jacoco_version"
// NOTE: Do not place your application dependencies here; they belong
// in the individual module build.gradle files
}
}
allprojects {
repositories {
google()
jcenter()
maven { url "https://dl.bintray.com/spekframework/spek-dev" }
}
}
task clean(type: Delete) {
delete rootProject.buildDir
}
I'm using Kotlin version 1.3.72, in Android Studio 4 - Beta 4. The versions of Junit5, JaCoCo, and Spek are specified in gradle.properties file as follows.
spek_version=2.1.0-alpha.0.14+a763c30
android_junit5_version=1.5.2.0
junit5_runner=0.2.2
jacoco_version=0.8.1
Next, we'll move to app level build.gradle file.
Apply the JaCoCo and JUnit5 plugin as shown below.
apply plugin: 'jacoco'
apply plugin: 'de.mannodermaus.android-junit5'
After that, we need to set up InstrumentationRunner, paste the following inside defaultConfig, inside your android block.
testInstrumentationRunner "androidx.test.runner.AndroidJUnitRunner"
testInstrumentationRunnerArgument "runnerBuilder", "de.mannodermaus.junit5.AndroidJUnit5Builder"
Then to set up Spek2 Test Engine and event logging, paste the following code inside android block.
testOptions {
junitPlatform {
filters {
engines {
include 'spek2'
}
}
jacocoOptions {
// here goes all jacoco config, for example
html.enabled = true
xml.enabled = false
csv.enabled = false
}
}
unitTests.all {
testLogging.events = ["passed", "skipped", "failed"]
}
}
Almost done, we just need to add the dependencies now, copy and paste the following inside dependencies block.
//kotlin-stdlib-jdk8 required
implementation "org.jetbrains.kotlin:kotlin-stdlib-jdk8:$kotlin_version"
// assertion
testImplementation "org.jetbrains.kotlin:kotlin-test:$kotlin_version"
// spek2
testImplementation "org.spekframework.spek2:spek-dsl-jvm:$spek_version"
testImplementation "org.spekframework.spek2:spek-runner-junit5:$spek_version"
testImplementation "org.jetbrains.kotlin:kotlin-reflect:$kotlin_version"
androidTestImplementation 'androidx.test.espresso:espresso-core:3.2.0'
androidTestImplementation 'com.android.support.test:runner:1.0.2'
androidTestImplementation "de.mannodermaus.junit5:android-test-core:1.0.0"
androidTestRuntimeOnly "de.mannodermaus.junit5:android-test-runner:1.0.0"
And we're all done.
Don't forget to add implementation "org.jetbrains.kotlin:kotlin-stdlib-jdk8:$kotlin_version", even though you may have implementation "org.jetbrains.kotlin:kotlin-stdlib:$kotlin_version", if you skip the jdk8 dependency, you'll encounter the following issue
Also, you'll need to set target and source compatibility to jdk8, you can do that by pasting the following inside android block in your app level build.gradle.
compileOptions {
sourceCompatibility JavaVersion.VERSION_1_8
targetCompatibility JavaVersion.VERSION_1_8
}
kotlinOptions {
jvmTarget = '1.8'
}
Writing Organised Test Cases with Spek
So, we're done setting up Spek, now it's time to write Tests, but what to test? I'm not following TDD here, as that's an entire topic on its own. What I want to say is that I already have the code I want to write tests on, and it's the following
package dev.rivu.mvijetpackcomposedemo.moviesearch.data.remote
//imports
class RemoteMovieDataStore(private val movieApi: MovieApi) : MovieDataStore {
override fun getMoviesStream(searchQuery: String): Flowable<List<Movie>> {
return Flowable.error(UnsupportedOperationException("Can't get stream from Remote"))
}
override fun getMovies(searchQuery: String): Single<List<Movie>> {
return movieApi.searchMovies(searchQuery)
.map { movieResponse ->
movieResponse.movies
.map { movie ->
Movie(
imdbID = movie.imdbID,
poster = movie.poster,
title = movie.title,
type = movie.type,
year = movie.year
)
}
}
}
override fun addMovies(movieList: List<Movie>): Completable {
return Completable.error(UnsupportedOperationException("Can't add to Remote"))
}
override fun getMovieDetail(imdbId: String): Single<MovieDetail> {
return movieApi.getMovieDetail(imdbId)
.map {
MovieDetail(
actors = it.actors.split(
", ",
ignoreCase = true
), // Jesse Eisenberg, Rooney Mara, Bryan Barter, Dustin Fitzsimons
awards = it.awards, // Won 3 Oscars. Another 171 wins & 183 nominations.
boxOffice = it.boxOffice, // $96,400,000
country = it.country, // USA
dVD = it.dVD, // 11 Jan 2011
director = it.director, // David Fincher
genre = it.genre, // Biography, Drama
imdbID = it.imdbID, // tt1285016
imdbRating = it.imdbRating, // 7.7
imdbVotes = it.imdbVotes, // 590,040
language = it.language, // English, French
metascore = it.metascore, // 95
plot = it.plot, // As Harvard student Mark Zuckerberg creates the social networking site that would become known as Facebook, he is sued by the twins who claimed he stole their idea, and by the co-founder who was later squeezed out of the business.
poster = it.poster, // https://m.media-amazon.com/images/M/MV5BOGUyZDUxZjEtMmIzMC00MzlmLTg4MGItZWJmMzBhZjE0Mjc1XkEyXkFqcGdeQXVyMTMxODk2OTU@._V1_SX300.jpg
production = it.production, // Columbia Pictures
rated = it.rated, // PG-13
ratings = it.ratings.map {
MovieDetail.Rating(
source = it.source,
value = it.value
)
},
released = it.released, // 01 Oct 2010
response = it.response, // True
runtime = it.runtime, // 120 min
title = it.title, // The Social Network
type = it.type, // movie
website = it.website, // N/A
writer = it.writer, // Aaron Sorkin (screenplay), Ben Mezrich (book)
year = it.year // 2010
)
}
}
override fun addMovieDetail(movie: MovieDetail): Completable {
return Completable.error(UnsupportedOperationException("Can't add to Remote"))
}
}
It's basically the remote implementation of DataStore interface of a Movie Search app, it has a dependency, a MovieApi instance, `MovieApi here is a Retrofit API interface as follows (if you're unfamiliar with Retrofit then consider this as a simple interface).
interface MovieApi {
@GET("/")
fun searchMovies(
@Query("s") query: String,
@Query("type") type: String = "movie",
@Query("apikey") apikey: String = BuildConfig.ApiKey
): Single<MovieSearchResponse>
@GET("/")
fun getMovieDetail(
@Query("i") imdbId: String,
@Query("apikey") apikey: String = BuildConfig.ApiKey
): Single<MovieDetailResponse>
}
And, here's the DataStore interface, that our RemoteMovieDataStore implements
interface MovieDataStore {
fun getMoviesStream(searchQuery: String): Flowable<List<Movie>>
fun getMovies(searchQuery: String): Single<List<Movie>>
fun addMovies(movieList: List<Movie>): Completable
fun getMovieDetail(imdbId: String): Single<MovieDetail>
fun addMovieDetail(movie: MovieDetail): Completable
}
So now as we have the code ready, let's get on with it and start writing test cases. As we already know, RemoteMovieDataStore depends on a `MovieApi` instance, so to write test cases for RemoteMovieDataStore, we either need to have a fake implementation of MovieApi or we need to mock it, here we are going to mock. A lot of developers are against mock, and to be honest, they're quite right, if you use mocking incorrectly it can become a pain, but it's a very useful tool as well. I completely agree with what Donn Felker said in this tweet chain below.
I'll be using mockito and mockito-kotlin, you can use mockK as well if you like that.
Just like usual JUnit tests, we will be writing our test cases inside the test directory of your module. So, go inside test directory, go on and create the same package structure as your target class (here dev.rivu.mvijetpackcomposedemo.moviesearch.data.remote), and then create a new Kotlin file, let's name it RemoteMovieDataStoreSpecTest.
We will now declare a class of the same name inside the file and extend Spek, just like the below.
class RemoteMovieDataStoreSpecTest : Spek({
//test cases go here
})
Once you paste the above code in the file, you should be able to see a green arrow in your IDE, just like the ones you see in regular JUnit test cases, below is the screenshot for your reference.
If you try and run the test cases now, it should fail with a message like test framework quit unexpectedly.
Now, before writing the tests, we need to create the mock as well as an instance of the target class, right? Creating mock is same as before, just paste the following inside the curly braces.
val movieApi: MovieApi = mock()
I kept it as val and outside of any initialization block because I want to create the mock instance once, and I believe, it's not really necessary to create a new mock instance for every single test, rather we can use whenever(..).thenReturn(..) for stubbing differently for each scenario or test case as we deem right.
Now since the mock dependency is ready, let's create the target class (RemoteMovieDataStore) instance. Generally, in JUnit4, we create the target class instance inside a function annotated with @Before which is called before each of the functions with @Test annotation is called. Most of the time, we use that function solely for variable initialization. With Spek, if you don't need a setup block solely for initializing variables, Spek provides you with a delegation called memoized, it works pretty much like lazy delegate in Kotlin, only that instead of initializing the variable first time it's used like lazy, a memoized variable will be initialized before running each of the test-cases in the group where the variable is declared. However, you might still want a setup block, for example, when you want to setup RxJava with your TestScheduler or more precisely when you want to set up something using RxJavaPlugins, or to start observing to LiveData, etc., for those scenarios, you can use beforeEachTest in Spek, it works all the same like @Before, within the block/group you defined it.
Now, for writing the test cases, we need to choose which DSL style to use. As I said earlier, Spek offers two styles of DSL for writing test cases, namely Specification which is also used in Jasmine and RSpec (also known as describe -> context -> it and Gherkin. You can use either/both of them, however, I strongly suggest don't mix them in a single project, rather use only one style throughout a project, for different projects you can use different ones. I'll be discussing both of them here, basically, I'll write the same tests using both the styles and you can decide for yourselves which one works for you better. First, we'll see the Specification style, I personally like this style better.
Using Specification style in Spek to write test cases
The first test I'll be writing is for the function getMovies as you can see in the implementation above, it expects a query, will call searchMovies method from MovieApi, map the response object to data layer model class, and will pass on downstream. If there's a network error or if the Single returned from searchMovies method emits an error, it should also emit the same error.
So, there are 2 scenarios here as below
movieApi.searchMoviesemits errormovieApi.searchMoviesemits data
Below is the group of test cases I wrote respecting both the scenarios.
describe("#${RemoteMovieDataStore::getMovies.name}") {
context("movieApi emits error") {
val error = Exception("some error")
beforeEachGroup {
whenever(
movieApi.searchMovies(
query = anyString(),
type = anyString(),
apikey = anyString()
)
)
.thenReturn(
Single.error(error)
)
}
it("should emit the same error, wrapped in Single") {
val testObserver = remoteDataStore.getMovies("").test()
testObserver.assertError(error)
}
}
context("movieApi emits data") {
val dummyResponse = dummyMovieSearchResponse
val dummyMovieList = dummyResponse.movies.map {
Movie(
imdbID = it.imdbID,
poster = it.poster,
title = it.title,
type = it.type,
year = it.year
)
}
beforeEachGroup {
whenever(
movieApi.searchMovies(
query = anyString(),
type = anyString(),
apikey = anyString()
)
)
.thenReturn(
Single.just(dummyResponse)
)
}
it("should return the same error, wrapped in Single") {
val testObserver = remoteDataStore.getMovies("").test()
testObserver.assertValue(dummyMovieList)
}
}
}
describe is the group that contains all the test cases for the method getMovies, inside describe there are two sub-groups I defined with context, in the first subgroup, we tested the error scenario, and in the second subgroup, we tested the success scenario.
So, to break it down, describe and context is for grouping and/or subgrouping of test cases, and we can perform assertions inside it. However, I'd like to note here that it's not mandatory to have context inside describe, you can directly have it block inside your describe block and perform assertions, if you feel you don't need a context blog, I've done it for some methods as well, where there are no multiple scenarios to test.
Below is the whole RemoteMovieDataStoreSpecTest (with test cases for all the methods in RemoteMovieDataStore) for your reference.
package dev.rivu.mvijetpackcomposedemo.moviesearch.data.remote
import com.nhaarman.mockitokotlin2.mock
import com.nhaarman.mockitokotlin2.whenever
import dev.rivu.mvijetpackcomposedemo.moviesearch.data.datafactory.dummyMovieDetail
import dev.rivu.mvijetpackcomposedemo.moviesearch.data.datafactory.dummyMovieDetailResponse
import dev.rivu.mvijetpackcomposedemo.moviesearch.data.datafactory.dummyMovieSearchResponse
import dev.rivu.mvijetpackcomposedemo.moviesearch.data.model.Movie
import io.reactivex.rxjava3.core.Single
import org.mockito.ArgumentMatchers.anyString
import org.spekframework.spek2.Spek
import org.spekframework.spek2.style.specification.describe
class RemoteMovieDataStoreSpecTest : Spek({
val movieApi: MovieApi = mock()
val remoteDataStore by memoized { RemoteMovieDataStore(movieApi) }
describe("#${RemoteMovieDataStore::getMoviesStream.name}") {
it("should return Flowable.error with UnsupportedOperationException") {
val testObserver = remoteDataStore.getMoviesStream("").test()
testObserver.assertError(UnsupportedOperationException::class.java)
}
}
describe("#${RemoteMovieDataStore::addMovies.name}") {
it("should return Completable.error with UnsupportedOperationException") {
val testObserver = remoteDataStore.addMovies(listOf()).test()
testObserver.assertError(UnsupportedOperationException::class.java)
}
}
describe("#${RemoteMovieDataStore::addMovieDetail.name}") {
it("should return Completable.error with UnsupportedOperationException") {
val testObserver = remoteDataStore.addMovies(listOf()).test()
testObserver.assertError(UnsupportedOperationException::class.java)
}
}
describe("#${RemoteMovieDataStore::getMovies.name}") {
context("movieApi emits error") {
val error = Exception("some error")
beforeEachGroup {
whenever(
movieApi.searchMovies(
query = anyString(),
type = anyString(),
apikey = anyString()
)
)
.thenReturn(
Single.error(error)
)
}
it("should emit the same error, wrapped in Single") {
val testObserver = remoteDataStore.getMovies("").test()
testObserver.assertError(error)
}
}
context("movieApi emits data") {
val dummyResponse = dummyMovieSearchResponse
val dummyMovieList = dummyResponse.movies.map {
Movie(
imdbID = it.imdbID,
poster = it.poster,
title = it.title,
type = it.type,
year = it.year
)
}
beforeEachGroup {
whenever(
movieApi.searchMovies(
query = anyString(),
type = anyString(),
apikey = anyString()
)
)
.thenReturn(
Single.just(dummyResponse)
)
}
it("should return the same error, wrapped in Single") {
val testObserver = remoteDataStore.getMovies("").test()
testObserver.assertValue(dummyMovieList)
}
}
}
describe("#${RemoteMovieDataStore::getMovieDetail.name}") {
context("movieApi returns error") {
val error = Exception("some error")
beforeEachGroup {
whenever(movieApi.getMovieDetail(imdbId = anyString(), apikey = anyString()))
.thenReturn(
Single.error(error)
)
}
it("should return the same error, wrapped in Single") {
val testObserver = remoteDataStore.getMovieDetail("").test()
testObserver.assertError(error)
}
}
context("movieApi returns data") {
val dummyResponse = dummyMovieDetailResponse
val dummyMovieDetail = dummyMovieDetail
beforeEachGroup {
whenever(movieApi.getMovieDetail(imdbId = anyString(), apikey = anyString()))
.thenReturn(
Single.just(dummyResponse)
)
}
it("should return the same error, wrapped in Single") {
val testObserver = remoteDataStore.getMovieDetail("").test()
testObserver.assertValue(dummyMovieDetail)
}
}
}
})
Using Gherkin style in Spek to write test cases
So, we already learned Specification style, now let's see the Gherkin style. We'll be refactoring the first test case we wrote into Gherkin style as follows.
Feature("#${RemoteMovieDataStore::getMovies.name}") {
Scenario("movieApi emits error") {
val error = Exception("some error")
beforeEachGroup {
whenever(
movieApi.searchMovies(
query = anyString(),
type = anyString(),
apikey = anyString()
)
)
.thenReturn(
Single.error(error)
)
}
lateinit var testObserver: TestObserver<List<Movie>>
When("remoteDataStore.getMovies is called") {
testObserver = remoteDataStore.getMovies("").test()
}
Then("should emit the same error, wrapped in Single") {
testObserver.assertError(error)
}
}
Scenario("movieApi emits data") {
val dummyResponse = dummyMovieSearchResponse
val dummyMovieList = dummyResponse.movies.map {
Movie(
imdbID = it.imdbID,
poster = it.poster,
title = it.title,
type = it.type,
year = it.year
)
}
beforeEachGroup {
whenever(
movieApi.searchMovies(
query = anyString(),
type = anyString(),
apikey = anyString()
)
)
.thenReturn(
Single.just(dummyResponse)
)
}
lateinit var testObserver: TestObserver<List<Movie>>
When("remoteDataStore.getMovies is called") {
testObserver = remoteDataStore.getMovies("").test()
}
Then("should emit the same data, mapped") {
testObserver.assertValue(dummyMovieList)
}
}
}
So, now we've Feature instead of describe, Scenario instead of context and then we have an additional block When, inside where you need to perform the method call you're testing, then finally do the assertions in Then, if you have multiple assertions to perform for a given method call in a particular scenario, while using the Specification style we can have all of them inside it, but in Gherkin style it's recommended to have each of the assertions separately in Then blocks.
Wrapping Up
So, that's all about Spek, we have covered setup, initializations, and then we learned about two different DSL styles Spek offers for writing test cases. Hope you liked it. You can find them all inside this demo project I'm building with Jetpack-Compose: https://github.com/RivuChk/Jetpack-Compose-MVI-Demo, that project is still WIP.
Please share your feedback/suggestions in the comments section