Todd Way

Using Feature Flags To Fake The State Of External Systems

2022-05-20T00:00:00+00:00

Most applications include code details to interact with external systems. Testing these interactions, especially in non-production systems, can be unpredictable. Feature flags provide a way to swap the real integration details with code that fakes different states of an external system. This avoids interruptions when those systems are offline, not responding as expected, or just hard to change.

Let’s imagine a Thermostat application that, among other things, determines if the outside temperature is between two values:

class Thermostat(private val temperatureService: TemperatureService) {
    fun isBetween(minTemp: Int, maxTemp: Int): Boolean {
        val currentTemp = temperatureService.get()
        return currentTemp > minTemp && currentTemp < maxTemp
    }
}

interface TemperatureService {
    fun get(): Int
}

We could implement the TemperatureService with a version that calls a weather.gov URL and parses the response data:

class WeatherGovTemperatureService: TemperatureService {
    override fun get() =
        HourlyForecast.from("https://api.weather.gov/gridpoints/TOP/31,80/forecast/hourly")
            .currentTemperature()
}

fun createBasicThermostatApp() = Thermostat(WeatherGovTemperatureService())

Arrows indicate creational dependencies. They point from an object to the objects used to create it

Or we could create a version that fakes a TemperatureService and always returns 70 degrees:

class Always70TemperatureService: TemperatureService {
    override fun get() = 70
}

fun createFakeThermostatAppWithFakes() = Thermostat(Always70TemperatureService())

In either case, the Thermostat only depends on the TemperatureService interface and does not need to be changed to use different implementations. This concept is called dependency inversion and helps loosen the coupling between related objects in an application.

With two versions of TemperatureService, we can use a feature flag to control which version the app will use:

interface FakeTemperatureFeature {
    val isEnabled : Boolean
}

fun createThermostatAppWithFlags(): Thermostat {
    val feature =
        object: FakeTemperatureFeature { override val isEnabled = true }
    return Thermostat(
        if (feature.isEnabled) Always70TemperatureService()
        else WeatherGovTemperatureService()
    )
}

Switching the state of FakeTemperatureFeature.isEnabled is now the only change needed to create a Thermostat with either version of the TemperatureService.

So far we have used simple factory functions to isolate the dependencies needed to create a Thermostat. But as the application grows in complexity, so can the the effort to manage these relationships. A dependency injection framework, like Dagger, can greatly reduce factory boilerplate and validate the object graph each time it is compiled:

@Module
class ThermostatModule {
    @Provides fun thermostat(service: TemperatureService) = Thermostat(service)
    @Provides fun nationalWeatherTempService() = WeatherGovTemperatureService()
    @Provides fun always70TempService() = Always70TemperatureService()
    @Provides @Singleton fun fakeTemperatureFeature() =
        object: FakeTemperatureFeature { override val isEnabled = true }
    @Provides fun temperatureService(
        feature: FakeTemperatureFeature,
        real: Provider<WeatherGovTemperatureService>,
        fake: Provider<Always70TemperatureService>
    ) = if (feature.isEnabled) fake.get() else real.get()
}

@Component(modules = [ThermostatModule::class]) @Singleton
interface ThermostatApp {
    fun thermostat(): Thermostat
}

fun createThermostatApp() = DaggerThermostatApp.create().thermostat()

The @Singleton annotation binds created objects to the scope of the @Component so the same instance can be reused

A future version of the dependency graph, with additional feature and flags, might look like this:

Finally, making features easy to switch can be useful in development, but we don’t want those changes to be accidentally released. We can prevent this with basic automated tests:

fun testThatFakeTemperatureFeatureIsFalse() {
    val feature = ThermostatModule().fakeTemperatureFeature()
    assert(!feature.isEnabled) {
        "$feature.isEnabled should be false but was ${feature.isEnabled}"
    }
}

To keep this example simple, the actual value of the flag is hardcoded object: FakeTemperatureFeature { override val isEnabled = true }. In certain situations it may be more useful to control the value from user interface so it can be switched without code changes. Still it’s unlikely you would want a feature that fakes external systems to be released to real users. Even if the flag is mutable in test builds, it should still be possible to write deterministic tests that prevent it’s release.

Summary

Invert dependencies on external details
Write a fake version (or use an existing test double) of the external details
Use a feature flag to declare which version to use
Isolate feature decisions in factories
Use injection to minimize creational boilerplate
Add automated tests to prevent releasing features accidentally

Additional Notes

This example focuses on how to make feature flag implementations safe and easy to maintain, but it leaves the actual value of the flag hardcoded in a factory function object: FakeTemperatureFeature { override val isEnabled = true }. In reality it may be more useful to control this from a user interface and store the value

Creating fake versions may seem like extra work, but if you’re writing unit tests, you may be able to feed two birds with one seed. In the example below, always60 or always70 could be repurposed in a fake temperature feature flag.

fun testThatIsBetweenReturnsExpectedValues() {
    val always60 = object: TemperatureService { override fun get() = 60 }
    val always70 = object: TemperatureService { override fun get() = 70 }
    val minTemp = 68
    val maxTemp = 72
    assert(Thermostat(always60).isBetween(minTemp, maxTemp) == false)
    assert(Thermostat(always70).isBetween(minTemp, maxTemp) == true)
}

You don’t need to make all integrations flaggable from the start. The next time you’re blocked by problems with an outside system, try applying a flag for just the calls needed to unblock your current task. It should be possible to get part of an application working with object fakes even if the rest is not.

FakeTemperatureFlag.isEnabled has a boolean type because only two value states were necessary: true & false (feature flags with only two states are often called feature toggles). But there are situations where flags with more than two states may be suitable. We might want to test with a set of fakes that all throw errors and a set of fakes that all return stubbed values. A flag for this might have three states: real, stubs, and errors.

Share Artifacts With An Orphan Branch

2020-04-18T00:00:00+00:00

Over the last several years I’ve worked with a variety of great hosted services for team code integration:

For build automation - Jenkins, Bitrise, CircleCi, BuddyBuild, TeamCity.
For code analysis - SonarQube, Codacy, Code Climate, Veracode.
For peer review - Github, Bitbucket, GitLab, Gerrit.

When everything works in harmony (checking out files, compiling, running tests, analyzing code structure, assembling artifacts, posting clear results) the whole team sees steady feedback, early in the development cycle, that helps prevent defects, educate new team members, and nudge us all to better design habits.

But for various reasons we can’t always set up the perfect recipe of hosted services for every project. If information that we’ve counted on in the past is missing, unreliable, or hard to access, the impact quickly fades. In the worst cases, the lack of feedback makes us falsely confident.

There are fancy ways to connect build automation services and code analysis services to deployment services and code review services. If these options are available, if you can quickly troubleshoot interruptions, and if your whole team can access the results, then do this. If not, here is a relatively independent alternative:

Most software platforms have standalone tools for code analysis (test execution, test coverage, linting, docgen, etc.) that can generate standalone reports (probably HTML). If you get familiar with how to generate these, you can port them to future projects without relying on an external hosted service or special IDE features. Make a one-step script that generates all of these artifacts.

Make another one-step script that does this: check out an orphan branch from your code repository into a temporary folder. Remove existing files. Copy in the generated files from your previous script. Commit and push. Print a download link for the commit.

Finally copy/paste the download link into your pull request description. Your team can use this link to easily download and browse results. One additional file I include is a root index.html with short descriptions and relative links for each report. This helps everyone know why each report is there and gives a single entry point for everything in the download.

For reference, I’ve set this up on one of my small, public Github projects. You can see an example pull request that includes a direct link to download reports. You can also see the shell script for pushing artifacts and the shell script for initializing the branch with an index.html. The goal for these scripts is to be relatively generic and portable so they can quickly be applied to future projects. The same goal applies to scripts that generate report artifacts, but this is inherently more platform-dependent. In this example, the script to generate artifacts is Gradle-based and the code analysis tools include JaCoCo, CPD, Detekt, BuildChecks, and JUnit.

Checklist Yourself

2018-08-15T00:00:00+00:00

In 1935, Boeing introduced a heavy bomber that outperformed any other aircraft of it’s kind. It crashed tragically in exhibition because a routine step was missed by the well-trained and experienced flight crew. As a result, pilots everywhere began to adopt preflight checklists and failures decreased significantly.

Today, checklists for complex responsibilities are used in many professions. A surgeon named Adul Gawande wrote an entire book called The Checklist Manifesto. I think one reason the idea has spread so successfully is because a checklist is an incredibly easy tool to make and to use. [[MORE]]

I’ve written previously about how we do continuous integration, which is essentially an automated checklist verified by a machine. CI provides a level of safety, consistency, and efficiency that’s hard to match in any other way. The problem is some details are prohibitively hard to automate and too important to ignore. Our current solution for this is to manually review all code before it “takes flight”. We use pull requests for this. What we’ve been missing, tho, is a preflight checklist.

Gawande sees ineptitude (not making use of what we already know) as a greater problem than ignorance (what we don’t know). So our development group did a retrospective on code reviews. We made a list of what we already know we’re looking for when reviewing code. The things we don’t want to forget about in the future. Based on input from that discussion and this handy checklist for creating checklists, I started one:

A checklist for reviewing code

Integration  
- [ ] Will merging this code create source conflicts?  
- [ ] Is there a clear and concise description of the changes?
- [ ] Did all automated checks (build, test, lint) run and pass?  
- [ ] Are there supporting metrics or reports (e.g. test coverage, fitness functions) that measure the impact?
- [ ] Are there obvious logic errors or incorrect behaviors that might break the software?

Readability
- [ ] Is the code self-documenting? Do we need secondary sources to understand it?  
- [ ] Do the names of folders, objects, functions, and variables intuitively represent their responsibilities?  
- [ ] Could comments be replaced by descriptive functions?  
- [ ] Is there an excessively long object, method, function, pull request, parameter list, or property list? Would decomposing make it better? .  

Anti-patterns
- [ ] Does the code introduce any of the following anti-patterns?
- [ ] Sequential coupling - a class that requires its methods to be called in order  
- [ ] Circular dependency - mutual dependencies between objects or software modules  
- [ ] Shotgun surgery - a change needs to be applied to multiple classes at the same time  
- [ ] Magic numbers - unexplained numbers in algorithms  
- [ ] Hard code - embedding assumptions about the environment in the implementation  
- [ ] Error hiding - catching an error and doing nothing or showing a meaningless message  
- [ ] Feature envy - a class that uses methods of another class excessively  
- [ ] Duplicate code - identical or very similar code exists in more than one location.  
- [ ] Boat anchor - retaining a part of a system that no longer has any use  
- [ ] Cyclomatic complexity - a function contains too many branches or loops 
- [ ] Famous volatility - a class or module that many others depend on and is likely to change 
  
Design principles
- [ ] Does the code align with the following principles?
- [ ] Single Responsibility - an object should have only one reason to change  
- [ ] Open/Closed - objects should be open for extension, closed for modification  
- [ ] Liskov Substitution - subtypes should not alter the correctness of code that depends on a supertype  
- [ ] Interface Segregation - many client specific interfaces are better than one general purpose interface  
- [ ] Dependency Inversion - dependencies should run in the direction of abstraction; high level policy should be immune to low level details

Last updated: 8/15/2018

Note: This is not a checklist for *approving* or *merging* code, it is a checklist for *reviewing* code.  It's a list of questions a reviewer should ask themselves as they review.

Put A Motor On Your Code Cycle

2018-07-02T00:00:00+00:00

There’s an old programmer joke:

Show me a line of code and I’ll tell you what’s wrong with it, Show me five hundred lines of code and I’ll say “looks ok to me”

Where I work, we use pair programming and pull requests in our development process. No code is committed to the main branch without peer review.

I’m convinced the investment in extra eyes can pay off for a project of any size. Human brains are uniquely capable of solving hard problems even when objectives are vaguely defined. When we do this together it increases ownership, cohesion, velocity and resilience across the team. This is magical.

But even the collective brainpower of a long-lived, high-functioning team isn’t always reliable. The complexities, pressures, and context-switching of a typical day can wear us down. When we add the eye-glazing drudgery of a long pull request at 4pm in the afternoon, we’re in trouble.

Here’s how we’re using automation to optimize our review process:

Build checks

Before anyone reviews source code, a machine should do it first. It won’t catch everything a human could see, but it’s more consistent and much faster. Source control systems (e.g. Github, Bitbucket) usually have an API so build results can be tracked for each code commit. These are commonly called “status checks” or “build checks”. As shown below, a green icon indicates a check posted from a build server was successful.

Reviewers see a green or red icon immediately and can avoid wasting time reading code with known problems. Even better, we can make these checks required for merging any code into our main code branch. Now the entire team, whether they review code or not, has confidence that the main branch is always protected.

We currently use 3 build checks: build, test, and lint. These can work for almost any project.

To process and post the checks, we use a Gradle plugin called BuildChecks. We use Gradle because it’s free, fast, configurable, testable, well-supported, and portable. From one project to the next, we don’t aways get to use the same development languages, source control system, or build servers, but we want to preserve key processes. BuildChecks can work across multiple languages and source control systems. It can run anywhere Java 7+ is installed. In situations where we aren’t able to use a dedicated build server, it can be run from a developer’s workstation. We can have the same automated integration protection even on shoestring budgets and timelines.

Build

The “build” check tells us if a build finished successfully and how long it took. The process may be different for each project but is typically some variation of: compile source code, assemble artifacts, run tests, run lint, and deploy artifacts. Having this feedback alone for a pull request review will save considerable time and effort.

Test

The “test” check show us the percentage of code that is covered by tests. BuildChecks parses output from coverage tools like JaCoCo, Cobertura, Istanbul, Slather, and OpenCover. A minimum threshold for coverage can be set that will cause the check to fail. This is optional. Even without a threshold, the check clarifies that tests are running and if they’ve changed between commits.

Lint

The “lint” check tells us if the code violates any predefined standards. BuildChecks parses output from linters like ESLint, TSLint, Detekt, Checkstyle, PMD, SwiftLint, and Android Lint. Each linter has different rule sets that span categories like correctness, security, performance, accessibility, formatting style, internationalization, etc. The lists can be overwhelming at first. Many are language or platform-specific, but one category that has some pretty universal rules is maintainability. If you don’t know where to start, this is a good place.

Here’s list of maintainability rules from a multi-language code analysis platform called CodeClimate:

Argument count - Methods or functions defined with a high number of arguments
Complex logic - Boolean logic that may be hard to understand
Method complexity - Functions or methods that may be hard to understand
File length - Excessive lines of code within a single file
Method count - Classes defined with a high number of functions or methods
Method length - Excessive lines of code within a single function or method
Nested control flow - Deeply nested control structures like if or case
Return statements - Functions or methods with a high number of return statements
Similar blocks of code - Duplicate code which is not identical but shares the same structure

Enable any of these that your lint tool supports. Then use pull request conversations to discuss and identify additional patterns you want to add. If the pattern isn’t already available, you can write your own custom rule.

Details

In the image above, each check has a hyperlink labelled “Details”. This links back to detail on the build server. It’s a great way to give reviewers access to the full generated reports from lint and coverage tools as well as build logs and other artifacts. The more context we can provide, the easier it will be for others to give real feedback.

Final thoughts

Automated checks help code reviews scale with consistency. Getting started is as easy as enabling the requirement in your source control system and using a tool like BuildChecks to report it. If you’re not already, this puts you on a path to a several important development practices: automated builds, unit tests, frequent integration, maintainability standards, and protected branches. Don’t worry if you start with low test coverage and high lint violations. You will immediately have a better understanding of your current situation and a way track your progress.

What automation tricks have you found for improving code review and integration?

Continuous Integration For Firebase Cloud Code

2017-09-25T00:00:00+00:00

In a previous post I showed how to add type-checking and unit tests to Firebase cloud code (cloud functions and database rules). Those tests are independent from any Firebase environment and independent from each other. We should be able to run them all quickly and consistently in a clean Node.js environment with a single command. We should also be able to chain that command with others so that we can build, test, and deploy each code commit directly into a live Firebase environment.

Here are the steps we want to automate:

Download project source
Download project dependencies
Compile project
Run all tests
Stop if any test fails, otherwise continue
Deploy (cloud functions and database rules) to a Firebase environment
Write a deployment summary to our Firebase environment (Git info, date, test results)

We’ll assume we have a machine with Node.js and Git installed. This could be a developer machine or a dedicated continuous integration server (I try to make the execution identical for either if I can). The first three steps are pretty straightforward from the command line:

git clone git@github.com:whatever folder-name
npm install
tsc

Now that we have a compiled project environment, we can use Typescript/Javascript to handle the rest of our steps. From the command line, node can execute a function from a local file like this:

node -e 'require("./build.js").runAllTests()'

Now let’s implement a function to run all our tests:

export async function runAllTests() : Promise<testresultsentity> {
    const Mocha = require('mocha');
    const mocha = new Mocha();
    mocha.addFile('./test/tests.functions.js');
    mocha.addFile('./test/tests.database.js');
    const results = new TestResultsEntity();
    return await new Promise<testresultsentity>((resolve, reject) => {
        mocha.run()
            .on('pass', (test) => { results.passed++; })
            .on('fail', (test, err) => { results.failed++; })
            .on('end',  () => { resolve(results) });
    });
}

Here I’m use the Mocha API to point to our test files, run them, and keep track of how many pass and fail. Let’s write a deploy function that grabs the test results and handles our last three steps:

export async function deploy()  {
    const testResults = await runAllTests();
    const gitSummary = await getGitSummary();
    const summary = `${testResults.getSummary()} on ${getDateSummary()} from ${gitSummary}`;

    if (testResults.hasFailures()) {
        console.log('Deploy failed');
    } else {
        await asyncCommand(`firebase deploy --only functions,database`);
        await asyncCommand(`firebase database:set /lastDeploy -d '"${summary}"' -y`);
        console.log('Deploy succeeded');
    }

    console.log(summary);
}

In the else clause above we run two Firebase CLI commands. The first command deploys our code to the currently configured Firebase environment. The second command writes a record to the database of that environment with a summary of our deployment. This makes it easy for anyone on the team to see:

what code is deployed,
when it happened,
and the results of the tests.

To pull all of this together into a single command, we’ll use the package.json file to set up an npm script:

"scripts": {
   "deploy": "npm install && tsc && node -e 'require(\"./build.js\").deploy()'"
 }

Now we can run all steps with these two commands:

git clone git@github.com:whatever folder-name
npm run deploy

Finally, here’s the ancillary code referenced by the deploy() and runAllTests() functions above:

export async function getGitSummary() : Promise<string> {
    const gitSha = await asyncCommand("git rev-parse --short HEAD");
    const gitBranch = await asyncCommand("git rev-parse --abbrev-ref HEAD");
    return Promise.resolve(`${gitSha.trim()}-${gitBranch.trim()}`);
}

function getDateSummary() : string {
    return new Date().toLocaleString("en-US", { timeZone: 'America/Chicago' }).trim();
}


const exec = require('child_process').exec;
function asyncCommand(command : string) : Promise<string> {
    return new Promise<string>((resolve, reject) => {
        exec(command, function(error, stdout, stderr){ resolve(stdout); });
    })
}

export class TestResultsEntity {
    passed : number = 0;
    failed : number = 0;

    getSummary() : string {
        return `${this.passed}/${this.failed+this.passed} tests passed`
    }

    hasFailures() : boolean { return this.failed != 0 }
}

</string></string></string></testresultsentity></testresultsentity>

Types And Tests For Firebase Cloud Code

2017-09-22T00:00:00+00:00

Firebase makes it cheap and easy to code custom functions and database access rules that run in the cloud along with their standard services. This is great because you can try things out very quickly in a real, shared environment that performs at scale. This simplicity make it tempting just to verify each change manually in the cloud environment. I think static type-checking and independent unit tests are an even more appealing way to verify our code with confidence. Here’s a recipe for eliminating dependencies on Firebase so we can run fast, automated tests before deploying code to the cloud.

Typescript + IDE

Typescript is a superset of Javascript that compiles to plain Javascript. It adds many useful language features like type annotations, interfaces, classes, and generics to Javascript. All features are optional, so you can always ignore types and interop with plain Javascript when you want.

Image we want have an app where we want to archive posts if they’ve been flagged too many times. Let’s use a Typescript interface to decouple the Firebase dependency from as much of our code as we can.

Say we have a PostEntity class like this:

class PostEntity {
    id : string;
    flags : number;
    maxFlags = 5;

    hasTooManyFlags() : boolean {
        return this.flags >= this.maxFlags
    }
}

…and a function that fetches a post by id, determines if it has too many flags, and archives it if so:

function archiveIfTooManyFlags(postId : string, datasource : PostDatasource) : Promise<void> {
    return datasource.getPost(postId)
        .then(post => {
            if (post.hasTooManyFlags())
                return datasource.archivePost(post.id);
            else
                return Promise.resolve();
        })
}

interface PostDatasource {
    getPost(postId : string) : Promise<postentity>
    archivePost(postId : string) : Promise<void>
}

Notice that we defined a PostDatasource interface with methods getPost and archivePost. We can write an implementation of that interface that uses Firebase as our datasource or we could write an implementation that uses something completely different. All of the code we’ve written so far is independent of that implementation.

Here’s what that implementation might look like using Firebase:

class PostDatasourceFb implements PostDatasource {
	db = require('firebase-admin').database();

    getPost(postId: string) : Promise<postentity> {
        return this.db.ref(`/posts/${postId}`).once('value')
	        .then(snap => snap.val());
    }

    archivePost(postId: string): Promise<void> {
        return this.db.ref(`/posts/${postId}/isArchived`).set(true);
    }
}

To execute this as a cloud function when a post is flagged in our Firebase Database we do this :

exports.onFlaged = functions.database.ref('/posts/{postId}/flags')
	.onWrite(event => {
	    return archiveIfTooManyFlags(
		    event.params.postId, 
		    new PostDatasourceFb()
		);
	});

In addition you can use a Typescript-aware IDE to make discovering the properties of your objects, language features and other javascript dependencies automatic as you write code. If you’ve used any other OO language with a good IDE you’ll know what this means. WebStorm is my choice because I’m already familiar with the shortcuts and organization of JetBrains tools, but there are many others. They all support features like: instant type checking, code assist, inline refactoring, breakpoint debugging, object navigation, source control tracking, etc.

Mocha + Chai + Sinon

Now that we have the language tools to decouple our application code from dependencies, we can set up unit tests that run locally outside of the Firebase cloud environment. Our test libraries are:

Mocha - lets you describe and execute a set of unit tests.
Chai - lets you make assertions within each of those tests.
Sinon - lets you create spies, stubs, and mocks for your test dependencies.

Here’s what a test for our archiveIfTooManyFlags function might look like:

describe('archiveIfTooManyFlags', () => {
    it('should call archivePost when there are too many flags', async () => {
        const sinon = require('sinon');
        const post = new PostEntity();
        post.flags = 7;
        const postDatasource = <postdatasource>{};
        postDatasource.getPost = sinon.stub().resolves(post);
        postDatasource.archivePost = sinon.stub().resolves(null);

        await archiveIfTooManyFlags("123", postDatasource);

        sinon.assert.calledWith(<sinonstub>postDatasource.archivePost, post.id);
        sinon.assert.calledWith(<sinonstub>postDatasource.getPost, "123");
    });
});

First we set up Sinon, and declare a PostDatasource and a PostEntity. Since PostDatasource is an interface, we have to implement at least the methods we plan to use in our test. We don’t want to use the PostDatasourceFb class from earlier, because it requires a Firebase environment. We could write a second implementation from scratch that returns some mock values and keeps track of method calls, or we could let Sinon do most of that work for us. Creating our PostDatasource as {} means it exists, but none of the methods have been implemented yet. We use sinon.stub() to stub the behavior of the methods we plan to use: getPost and archivePost. Finally we call archiveIfTooManyFlags and verify that getPost and archivePost were called with the expected arguments.

Now we can run this test and others like it locally without connecting to Firebase. This covers our cloud functions, but what about our database rules?

Targaryen

Targaryen lets us write the same kind of Mocha-based unit tests for our Firebase database rules.

Here’s a rule for reading & writing to a post:

{
  "rules": {
    ".read": "false",
    ".write": "false",
    "posts" :  {
      ".read": "true",
      "$post" : {
        ".read" : "true",
        ".write": "auth != null && (data.child('userID').val() == auth.uid || root.child('users/' + auth.uid + '/isAdmin').val() == true)",
        "views":{
          ".write":"true"
        }
      }
    }
}

Only admins and authors should be able to write to (edit) a post. Targaryen lets us test this rule locally like this:

describe('posts/aPostId', () => {
    it(`can write if admin or author`, () => {
        targaryen.setFirebaseData({
            users: {
                adminUser: {
                    isAdmin:true
                }
            },
            posts: {
                aPostId: {
                    userID:"authorUser"
                }
            }
        });

        expect({uid: 'adminUser'}).can.write.path('posts/aPostId');
        expect({uid: 'authorUser'}).can.write.path('posts/aPostId');
        expect({uid: 'randomUser'}).cannot.write.path('posts/aPostId');
        expect(null).cannot.write.path('posts/aPostId');
    });
});

Now we can cover all our cloud functions and all our database rules with fast, automated tests! Wouldn’t it be great if we had a single command that could confirm all our tests are passing, then deploy the code and results to our Firebase environment? I’ll write up my solution to this in a future post.

Api Acceptance Tests With Cucumber And Rest Assured

2017-04-11T00:00:00+00:00

On projects where multiple systems undergo development at the same time, it’s crucial to maintain a clear picture of how they should interact. We commonly have a backend system providing a REST API to multiple frontends (browsers, mobile apps, chatbots, IoT, etc.). Because it will likely change over time, keeping the API picture clear and up to date can be a significant challenge. How can we efficiently describe the currently expected behavior and know if it’s working as expected so teams don’t spin their wheels due to miscommunication?

Cucumber

Cucumber helps us write readable requirements upfront that can be tied directly to executable tests. Here’s an example for a guestbook REST API:

Scenario: Read a list of guestbook entries

  Given I'm using the staging API environment
  And the guestbook has at least "2" entries
  When I make a GET request to "/guestbook/entries"
  Then I get a response code of "200"
  And I get a response with at least "2" entries
  And each entry has a "name"
  And each entry has a "date" formatted as a Unix timestamp

Each line in the scenario above represents a discrete Cucumber step. A developer can now write a short block of code to fulfill each step.

There are options in various languages for fulfilling Cucumber step definitions (e.g. Ruby, Javascript, Python, .NET, Java). I chose the Java implementation, Cucumber-JVM, for these reasons:

Works with many out-of-the-box reporting and automation tools - because it’s JUnit-based
Intuitive IDE support for code assist, breakpoints, debugging, output formatting, etc. (Intellij and Eclipse)
Easy-to-build HTTP request and response assertions using the Rest-assured library

Cucumber on Java

Using Cucumber-JVM and the Intellij IDE, I get automatically generated step definitions like this:

public GuestbookStepDefinitions() {
	Given("^I'm using the staging API environment$", () -> {
		//short block of code goes here
	});
	
	When("^I make a GET request to \"([^\"]*)\"$", (String path) -> {
		//another block here
	});
	
	Then("^I get a response code of \"([^\"]*)\"$", (Integer code) -> {
		//and another
    });
}

Next we fill in the implementations using Rest-assured…

Rest-assured

public GuestbookStepDefinitions() {

    private RequestSpecification request;
    private ValidatableResponse response;
    
    @Before
    public void before(Scenario scenario) {
        request = RestAssured.with();
    }

	Given("^I'm using the staging API environment$", () -> {
		request.given()
	        .contentType(ContentType.JSON)
	        .baseUri("https://staging.mycompany.com");
	});
	
	When("^I make a GET request to \"([^\"]*)\"$", (String path) -> {
		response = request.get(path + ".json").then();
	});
	
	Then("^I get a response code of \"([^\"]*)\"$", (Integer code) -> {
		response.statusCode(code);
    });
}

The Given and When steps are building a request with details for our REST API. The Then step calls response.statusCode(...) which is an assertion of the status code returned by the REST API. If any step fails we get targeted feedback like this:

java.lang.AssertionError: 1 expectation failed.
Expected status code  but was .


...
io.restassured.internal.ValidatableResponseOptionsImpl.statusCode(ValidatableResponseOptionsImpl.java:117)
	at GuestbookStepDefinitions.lambda$new$8(GuestbookStepDefinitions.java:66)
	at ✽.Then I get a response code of "100"(guestbook-entries-read.feature:12)

Failed scenarios:
guestbook-entries-read.feature:9 # Scenario: Read a list of guestbook entries

1 Scenarios (1 failed)
3 Steps (1 failed, 2 passed)
0m1.370s

This output is a bit verbose (we’ll worry about report formatting later), but contains important information about the failure.

The step: we see that the “Then I get a response code of 100” step of our “Read a list of guestbook entries” scenario is where we’re failing. That means the previous two steps passed successfully.

The expectation: we see that we got a response code but expected a response code. If we change the expected status code back to 200, we should get a passing test:

Scenario: Read all guestbook entries          # guestbook-entries-read.feature:9
  Given I'm using the staging API environment # GuestbookStepDefinitions.java:89
  When I make a GET request to "/guestbook"   # GuestbookStepDefinitions.java:59
  Then I get a response code of "200"         # GuestbookStepDefinitions.java:65

1 Scenarios (1 passed)
3 Steps (3 passed)
0m1.937s

Since we’re making HTTP calls it’d be nice to see the request and response details too. We can tell Rest-assured to print those along with our test results:

Request method:	GET
Request URI:	https://staging.mycompany.com/guestbook/entries.json
Proxy:			<none>
Request params:	<none>
Query params:	<none>
Form params:	<none>
Path params:	<none>
Multiparts:		<none>
Headers:		Accept=*/*
				Content-Type=application/json; charset=UTF-8
Cookies:		<none>
Body:			<none>
HTTP/1.1 200 OK
Server: nginx
Date: Mon, 10 Apr 2017 19:09:16 GMT
Content-Type: application/json; charset=utf-8
Content-Length: 4830
Connection: keep-alive
Access-Control-Allow-Origin: *
Cache-Control: no-cache
Strict-Transport-Security: max-age=31556926; includeSubDomains; preload
{
    "-KgBbHUcv2NWn2M6tzGp": {
        "comment": "Hello Guestbook",
        "name": "Test User",
        "timestamp": 1490565277672
    },
    "-KgBbzZE2WtRD9wz1t-D": {
        "comment": "Hello Guestbook",
        "name": "Test User",
        "timestamp": 1490565462287
    }
}

Reporting

Now when we run this test we immediately know three things:

What we expect to happen (the Given-When-Then statement)
How to make it happen (the printed request and response)
Is it currently working as expected (Pass or Fail)

Having this feedback continuously throughout development mitigates communication issues early before teams waste time heading in different directions. The easy-to-read Cucumber steps that everyone can read tie directly to the gritty HTTP definitions that developers need and we can drop it on a CI server to generate formatted reports visible to the whole team.

Here’s an example of formatted results from the Intellij IDE:

On the left we have a collapsible, colored outline of our features, scenarios, and steps. We can select anything in the tree and see corresponding details on the right.

And here’s a standalone HTML report:

Again we have a collapsible, colored outline that documents the expected behavior and HTTP details.

Process

This approach is designed drive collaboration early in the process so it’s a great chance to work in pairs. Pairing a frontend developer and a backend developer can help start the conversation about how systems should interact. Getting other roles like analysts, designers, and testers involved can level-set everyone’s understanding of how the product is supposed to work. As soon as we have requirements for our first feature, we can start writing tests. The code required to fulfill step definitions should be easy enough for any developer to pick up quickly regardless of language choice. I prefer to put API acceptance tests in a separate repository apart from any other production code. This limits external dependencies from affecting our ability to write and run the tests.

Serverless Apps With Firebase

2017-03-17T00:00:00+00:00

Firebase is a set of backend platform services (owned by Google and closely integrated with the Google Cloud Platform) for building web and mobile apps. They have SDKs for Android, iOS, web, C++, Unity, Node.js, and Java. Their generous free tier makes it easy to launch fully functional apps to a modest user base without cost.

Free and unlimited features include: Authentication, Analytics, App Indexing, Cloud Messaging, Crash Reporting, Dynamic Links, Notifications, Remote Config

Free features with usage limits: Realtime Database, Cloud Functions, Hosting, Storage, Test Lab

There is way too much to cover here. If you want all the details, their website docs are some of the best I’ve encountered. You can also demo many features right from the web console. Here are a few of the highlights that I think can considerably reduce effort and improve quality for app development.

Sign in/up simply

Firebase Authentication provides email, social, anonymous, and custom sign in methods out of the box. Accounts can be managed and each method enabled or disabled from the Firebase web console. Access tokens are based on the JWT feature of OpenID Connect which encrypts portable authorization data in each token. This means multi-system architectures can share tokens and without the expense of server-to-server callbacks on client requests.

Realtime apps are no longer a luxury

The Realtime Database is a NoSQL cloud database with REST and SDK (websocket) support. Data is synced across all clients in realtime, and remains available even when offline. Network calls, cache updates, device resources, and intermittent connectivity are managed automatically by the SDK. Clients simply listen for data changes and react with UI updates.

Free backend - if needed

For the most part Firebase requires no server-side code, but if you want something to be handled in a trusted backend environment (e.g. push notification logic), there are two relatively simple and free-tier options: Cloud Functions and App Engine. Cloud Functions is a hosted, private, and scalable Node.js environment where you can run JavaScript code and interact with Firebase. App Engine is a hosted, private, and scalable environment that supports Java, Python, PHP, and Go. If you’re building an Android app, App Engine is a convenient option because your Java code and IDE tools can be shared between the two.

Deep links that survive installs

A Dynamic Link is a deep link that can survive the optional app installation step (on Android and iOS) or fall back to a web link if the user is on a desktop machine. Firebase will generate short links that contain all the details required. This works great for letting users invite their friends to an app or tracking referral codes.

Final thoughts

Serverless isn’t the right approach for every situation, but the potential for reduced effort, early feedback, and tighter operational management is compelling - especially in the early life stages of an app. Firebase is one of the most complete Backend-as-a-Service platforms, but the field is still fairly new. The competition will evolve and so will your app. Design principles like dependency inversion can help minimize these future risks.

Reuse Android Code Without Remote Dependencies

2016-11-11T00:00:00+00:00

You’re starting to write similar code project after project and think it would be useful to establish some base components that can be reused across multiple projects. First you create a library module to isolate your code. Then you need to figure out how you’re going to include it in each project.

If it’s open source, hosting it in a public maven repository like JCenter/Maven Central is a good solution. If the code can’t be shared publicly, the simplest way I’ve found is this:

[[MORE]]

Step 1

Copy the packaged library file:

MyReusableLibrary/build/outputs/aar/MyReusableLibrary-release.aar

and paste it into the app/libs directory of each project you want to use it in.

Step 2

Add the following to the app/build.gradle of each project

repositories {
    ...
    flatDir {
        dirs 'libs'
    }
}

dependencies {
	...
	compile ':MyReusableLibrary-release.aar
}

What I like about it

It’s self-contained for library consumers. If a developer tries to download and build a project that uses this library, there are no additional commands to run and no extra credentials to track down.

It’s low maintenance for library providers. You can host the aar files directly in your source repository or wherever else makes sense. You don’t need to worry about maintaining a private maven server.

What I don’t like about it

It doesn’t include transitive dependencies automatically. Since there is no associated pom file, you need to describe them along with your library. If the example library above depended on the Android Support Library, the project dependencies would need to include both:

dependencies {
	...
	compile ':MyReusableLibrary-release.aar
	compile 'com.android.support:appcompat-v7:24.2.0'
}

Additional tips

Add the following to the build.gradle of your library module to store versioned aar files:

android {
	defaultConfig {
		versionName "1.0"
	}
    buildTypes {
        release {
            archivesBaseName = "${project.name}-${android.defaultConfig.versionName}"
        }
    }
}

task copyAar(type: Copy) {
    from('build/outputs/aar')
    into('../app/libs')
    include(archivesBaseName + '-release.aar')
}
copyAar.dependsOn assemble

Running ./gradlew copyAar assembles the file MyResuableLibrary-1.0-release.aar and copies it to the libs folder.

Have a sample app module in your library project so you can test your library before using it in other projects. See the Picasso library from Square (or many of the other Android library projecst on Github) as an example. The /picasso directory is the library and /picasso-sample directory is the sample app. You can also use this sample app module to test the flatDir dependency approach described above.

Todd Way

Using Feature Flags To Fake The State Of External Systems

Summary

Additional Notes

Share Artifacts With An Orphan Branch

Checklist Yourself

A checklist for reviewing code

Put A Motor On Your Code Cycle

Build checks

Build

Test

Lint

Details

Final thoughts

Continuous Integration For Firebase Cloud Code

Here are the steps we want to automate:

Types And Tests For Firebase Cloud Code

Typescript + IDE

Mocha + Chai + Sinon

Targaryen

Api Acceptance Tests With Cucumber And Rest Assured

Cucumber

Cucumber on Java

Rest-assured

Reporting

Process

Serverless Apps With Firebase

Sign in/up simply

Realtime apps are no longer a luxury

Free backend - if needed

Deep links that survive installs

Final thoughts

Further Reading

Reuse Android Code Without Remote Dependencies

Step 1

Step 2

What I like about it

What I don’t like about it

Additional tips