oneanddone.mozilla.org is an application that is a vital ingredient to Mozilla QA’s
community strategy. At it's start, the project was originally initiated and designed
by the community to address the problem of discovering meaningful ways to contribute
to Mozilla QA. In its simplest form, oneanddone.mozilla.org allows community
members to browse project tasks and upon finding one that resonates with them,
begin working on it.
The development of the project as well as continued shaping of its features are
handled both by the community as well as by the QA team itself. Project ideation
and community engagement is handled by Rebecca Billings, Karl Thiessen, and
Bob Silverberg. Code management is accomplished on get
github.com/mozilla/oneanddone. The deployment pipeline (stage, production,
etc) and hosting are handled by heroku.com. Feature triage and ideation,
in bugzilla.
For a project of One and Done’s complexity and development requirements, Heroku
was a natural fit. The creation and management of deployment workflows as well
as one-click installation of add-ons are cleanly implemented.
Going into 2016 we have seen many projects at Mozilla standardizing around AWS
as it's web services delivery mechanism. Given this and that QA owns
oneanddone.mozilla.org we felt it would be advantageous to consider a move to
this platform.
The problem space we wanted to explore
Continue to use our application as a platform to understand CD and the deployment environments used by our developers
Build on our knowledge base so we can offer timely suggestions of best practices to projects that are in the process of moving to AWS or may have already made the switch.
Approaching the problem(s)
Investigate the current workflow cases oneanddone.mozilla.org employs and map them to potential AWS equivalents.
Create a working document that summarizes these findings, includes a pros and cons list.
Identify teams that have domain knowledge and form relationships that should we want to move off of Heroku in the future, we can leverage their experience.
What we discovered
Heroku’s platform is quite slick, particularly for projects — like One and
Done — that need flexibility and are self-supported by small teams. Heroku also
scales quite readily if needed, that said our usage of it follows simple
patterns and a low amount of traffic.
Amazon's large array of services often times matched 1:1 the offerings presented
by Heroku. However there is a bit of additional mastery that is required to use,
configure, and deploy using AWS.
Heroku
Creation of deployment workflow via a GUI is straightforward and quick.
For pull requests that contain complex or experimental ideas, Review Apps are easy to deploy. One and Done employs the use of review apps instead of a dev instance.
Dataclips - the ability to create short or long-lived queries that return parseable results saves developer time. In some instances this can stand in for a read-only API.
On the negative, very few of the projects we collaborate on use Heroku and it's workflows.
Both a pro and a con - there is one right way to configure a build pipeline.
Self supported by the Web QA team.
AWS
Transitioning to a solution that is powered by AWS would initially be a fairly high barrier to overcome.
The ecosystem of extensions and external services match what is offered in Heroku.
Leveraging AWS in our deployment and hosting workflows would bring us in line with what the majority of our other projects are using.
We can follow patterns that other teams have used when combining Jenkins and other bits into their pipelines.
AWS has CodePipeline - their version of a continuous delivery service. This isn’t used by our teams at Mozilla, thus I don’t think we should invest in it at this time.
Puppet works well with AWS - our Operations team uses Puppet.
Configuration to use third-party CI pipelines is near infinite.
Mozilla is standardizing around AWS centric solutions. On this merit alone, as a team we should discuss if there is value for us to do so as well.
Support from the Ops team as well as reproducible deployment patterns already exist at Mozilla.
Next
The next steps need to be carefully considered, in the long term I believe it would be
beneficial to move to AWS. This would help us gain direct knowledge of
more complex deployment scenarios and frameworks, mechanisms which a majority of
the projects we contribute on are already using. There would also be the option
of garnering assistance from the Ops team in the form of support, support
which we currently do not have while using Heroku.
The greatest challenge that needs to be weighed is time. Currently we have a
solution that works for One and Done. To move to a new PaaS would cost us time.
Several intermediary steps became clear and would allow us to make the transition
smoother. In particular, we can leverage the work that other teams ar
completed and reuse portions of it to build a workflow that meets our needs.
Moving to AWS seems to add a lot of complexity for little immediate return. We should wait to see what practices other teams standardize on.
Consider moving One and Done to Docker. This will make setting up and maintaining a development environments simpler. This also has the potential to help with portability of between PaaS environments. Ops has familiarity with Docker.
Work with Ops to build a deployment workflow with Puppet, Jenkins, and Travis-CI.
Utilize artifacts created by other teams. The pipeline tiger team, a cross-team group, is working to create reusable templates that teams can use with AWS.