Feature Flags

Feature flags/toggles/controls are a software development best practice of gating functionality, used by companies like Google and Facebook.  Functionality can be deployed “off”, then turned on via the feature flag, separate from deployment.

If you are looking for an open source solution, check out some great resources here.  If you are looking for a managed SAAS solution, check out LaunchDarkly for feature flags.

Feature flags/toggles/controls are basically ways to control the full lifecycle of your features.  They allow you to manage components and compartmentalize risk.  You can do pretty cool things like roll out features to certain users, exclude groups from seeing a feature, A/B test, and much more.

When a user loads a page, your application will use that user’s attributes to determine what features to show.  For example, if I am a BETA user and I log in to myexamplesite.com, I will see the brand new BETA feature.  However, all non-BETA users will just see the old feature.   The reason I see the BETA feature is that my user is grouped as BETA.

While there are many open source solutions, we can dig deeper into LaunchDarkly’s SAAS platform for feature flags.

In this example below, you will see an explanation of multivariate feature flags.  Multivariate means that you can serve multiple variations of a feature to different user segments.  For example, let’s say I have a purple, orange, and green landing page.  I can select which individual users or groups of users I would like to see each variation.

On the LaunchDarkly side of things, you would do the following:

1. Create a feature flag called “Landing Page”
2. Name three variations “Purple” “Orange” “Green”
3. Select which users you want receive each variation (you can also serve to percentages: '60% of all users get the purple variation')

On your side of things, you would do the following:

1. Add the LaunchDarkly SDK to your platform
2. Wrap your feature in a feature flag
3. Call the SDK method to receive the value for that flag

It’s as simple as that.   You can check out the full documentation here.

Above, we briefly covered how to target individual users and groups.  Let’s take a deeper look into why this is important.  Targeting gives you the power to personalize a user’s experience.

Imagine the ability to create a customized and rewarding experience for every user.  Here are a few notable use cases:

• Plan Management (normal vs. premium) –  You can launch targeted features to users on different plans.  Want to add a new feature for a premium user?  Sure!  Just wrap the new feature in a flag and turn it on for premium users.  Want to extend that feature to normal users eventually?  No problem!  Just add normal users when you’re ready.
• Early Access – Allow only opt-in or power users to experience the latest features.
• Block Users – Exclude users or groups who you do not want to see a new feature.

If you’re deploying a brand new set of features, launching them to 100% of your users at once is a risky business.  In fact, testing things by giving all of your users access isn’t really a test.  A test should be the process of receiving incremental feedback from your users, making improvements, and gradually expanding your release to everyone.  This is where LaunchDarkly’s rollouts come in.  If you want to launch a new feature, you can start by rolling it out to 1% of your users.. then 5%.. then 20%.. then 50%.. then 100%.  If something goes wrong at the 1% rollout, you can instantly roll it back, make the improvements, and test it again.

This is the process of canary launching, whereby you test the efficacy of a new set of features before releasing it to everyone.  It also allows you to test how your features behave at different levels of traffic and incrementally refine your infrastructure to support the deployment.

Feature flags/toggles/controls harness the power of test-driven development (TDD).   This is the process of releasing and iterating features quickly, testing those features, and making improvements.  Think of it as Lean UX methodology.  You release light features to receive market feedback.  You iterate on that feedback, make improvements, and redeploy.

Think of feature flag-driven development as a way to receive iterative market feedback on your product, rather than solely depend on isolated customer feedback.  It’s a way to test how your features perform in the real world and not just in an artificial test environment.

In the world of waterfall development, you will typically see one continuous build that culminates in a single deploy.  After this deploy, you’ll receive feedback and fix some bugs, but you will likely need to restart the process for any major feature releases.

Agile is a bit more forgiving.  You can iteratively test small releases to your users, but this is best performed in a staging environment.  You typically will not release your product to market throughout the agile development process, as most of your testing will be internal and controlled.

Finally, lean UX codifies the process of releasing features to market throughout the development process.  These releases will likely be smaller in scale, but you’ll receive immediate market feedback. When you introduce feature flags into the equation, the process becomes even more efficient.

Feature flags allow you to substantially mitigate the risk of releasing immature functionality.  If a release is having a negative impact, roll it back.  If it’s doing well, keep rolling it out.   This is like having a persistent undo button and a means to recalibrate and improve functionality.

More importantly, you can institutionalize this process within your development cycle.  Your team will develop a cadence for lean releases, where all new components and functionality are wrapped in feature flags.  You can easily test features, cultivate creativity, and reward bold advances – all without compromising the integrity of your platform.

This new development methodology also allows your marketing, design, and engineering teams to collaborate more frequently and more effectively.  With an agile approach, you will typically have one large planning cycle that will launch you into development.  You will then test your iterations on local groups or in a local environment that tries to simulate production.  However, you cannot substitute real market feedback.

Feature flag driven development allows you to quickly release iterations of your features to market, receive feedback, improve, and redeploy.  It allows you to roll out features to small segments of your users in order to mitigate risk all while receiving valuable feedback.  More importantly, your team will converge and collaborate based on real market feedback and make the necessary improvements to drive the product forward.

A/B testing is the practice of comparing different versions of a page to see which one performs better.  In the traditional sense, A/B testing has been used for mainly cosmetic changes.  These include layouts, element position, colors, and copy.   Typically, A/B testing is tied to a goal.  For example, you want to increase sign up conversions, so you use tools like Optimizely, Apptimize, etc to test different layouts, buttons, and call to action language.  These tools work great, but what if you wanted to test backend-level functionality, completely new features, and sign-up flows?

This is where feature flag driven A/B testing comes into play.

Because feature flags are implemented at the code level, you can control deep functional features and then target user segments.   For example, let’s say I want to test a new sign-up flow and welcome tutorial (see above). I can flag the new functional components so that only certain users will receive the new flow.

With a suite like LaunchDarkly, you can then analyze these feature tests using your Optimizely or New Relic goals.  This will allow you to see, for example, which sign up flow is generating better conversions or which check out flow is generating more revenue.

All in all, feature flag driven A/B testing enables companies to test robust functionality instead of just cosmetic changes.

If you are on a budget or want to develop your own feature flagging platform, then I recommend you start with some of the open sourced libraries that you see on the site.  However, if you want to actually adopt a self-supported platform, I would recommend using a system like LaunchDarkly which provides an SDK the installs into your code base and can be controlled from a central dashboard.

Here are a list of some great resources that can help you incorporate feature flag driven development into your dev life cycle.

Stack Overflow – Discussion regarding feature flags

• “A ‘feature flag’ (or Feature Toggle) is the ability to turn features (sub-sections) of your application on/off at a moments notice. I guess the example there was that it’s handy to have the control to reduce the feature-set somewhat if you need to, say, reduce db queries if the load is too high. There are heaps of other reasons you would want to use this tho – one of the main being enabling Continuous Delivery: pushing things into production/live yet having the feature disabled/toggled until it’s completed. We often use what we call a ‘dev cookie’ to show uncompleted features to just the dev team. This way we can test partially completed work in production (oh yeh! is there better integration?) over multiple releases/deployments before we ‘untoggle’ (completed) it and it becomes visible to the public.”

Wiki – Introduction to Feature Toggles & Continuous Delivery

• “A feature toggle, (also feature switch, feature flag, feature flipper, conditional feature, etc.) is a technique in software development that attempts to provide an alternative to maintaining multiple source-code branches (known as feature branches).  Continuous release and continuous deployment provide developers with rapid feedback about their coding. This requires the integration of their code changes as early as possible. Feature branches introduce a bypass to this process. Feature toggles bring developers back to the track, but the execution paths of their features are still “dead” if a toggle is “off”. But the effort is low to enable the new execution paths just by setting a toggle to “on”.”

Martin Fowler – Feature Toggle

• “One of the most common arguments in favor of FeatureBranch is that it provides a mechanism for pending features that take longer than a single release cycle. Imagine you are releasing into production every two weeks, but need to build a feature that’s going to take three months to complete. How do you use Continuous Integration to keep everyone working on the mainline without revealing a half-implemented feature on your releases? We run into this issue quite a lot and feature toggles are a handy tool to deal with it.”

Mathias Meyer – Using Feature Flags to Ship Changes with Confidence

• “A feature flag is a branching point that your code can utilize to determine whether or not a feature should be made available to one or more customers. The original approach was made popular by the fine folks at Flickr. They needed a means to disable features when something is broken in production, to reduce the load on the database or on other parts of the system.”

Beautiful Builds

• “Also known as Feature Bits and Feature Flags, here is a list of some of the frameworks you can find in the wild for your favorite language.”

Leonard Garvey –Better development with feature flags

• “Feature flags are a tool which can be used by software developers to separate a feature from the rest of the code surrounding it. It’s a very simple concept with some very powerful consequences. With a feature flag you can deploy a feature which might not be quite ready for prime-time yet. Deploying early, and often is a good way to keep your team’s progress moving forward. There’s nothing that kills productivity faster than working on the same feature for a really long time without deploying it.”

Jim Bird – Feature toggles and technical debt

• “Feature flags or config flags aka feature toggles aka flippers are an important part of Devops practices like dark launching (releasing features immediately and incrementally), A/B testing, and branching in code or branching by abstraction (so that development teams can all work together directly on the code mainline instead of creating separate feature branches).  Feature toggles can be simple Boolean switches or complex decision trees with multiple different paths. Martin Fowler differentiates between release toggles (which are used by development and ops to temporarily hide incomplete or risky features from all or part of the user base) and business toggles to control what features are available to different users (which may have a longer – even permanent – life). He suggests that these different kinds of flags should be managed separately, in different configuration files for example. But the basic idea is the same, to build conditional branches into mainline code in order to make logic available only to some users or to skip or hide logic at run-time, including code that isn’t complete (the case for branching by abstraction).

Ross Harmes @ Flickr – Flipping out

• “Flickr is somewhat unique in that it uses a code repository with no branches; everything is checked into head, and head is pushed to production several times a day. This works well for bug fixes that we want to go out immediately, but presents a problem when we’re working on a new feature that takes several months to complete. How do we solve that problem? With flags and flippers! Flags allow us to restrict features to certain environments, while still using the same code base on all servers.”

João Miranda – How Etsy Deploys More Than 50 Times a Day

• “Etsy’s development approach revolves around making many small, continuous changes. A direct consequence is the need to do many deployments a day. In the words of Daniel Schauenberg, at any given time every Etsy developer needs to know the answer to the question “how comfortable am I with deploying a change right now?”. To be comfortable at all times, Etsy adopted a range of tools and practices: mandatory IRC-based communication; developer virtual machines; continuous integration; one-click deployments; thorough application and system monitoring; no blame post-mortems and on-call policies for both dev and ops teams.”