We created a tool that integrates Docker and Maven to help hundreds of our developers manage complex development environments with hundreds of services, while making a minimum of effort. This is a story about how a crazy idea became a reality. This is the story of
Carnotzet .
It all started about five years ago (approx. Publication date August 3, 2017), Swissquote quickly grew and developed. At that time, we had about 70 developers working on large (approx. Monolithic) Java projects, requiring each of them to spend 1 to 2 days to configure the launch of the project locally. We hate wasting time on repetitive tasks! So, it was decided to improve this process by using Vagrant and Chef to automate the deployment of our local development environment. This marked the beginning of our first Sandbox project (Sandbox, approx.)
We would like to share lightweight, reproducible, isolated, and portable development and testing environments.
For a while, everything was smooth. Developers could simply download the project, execute “vagrant up” and get to work. We used this approach for about two years and were quite pleased.
Subsequently, working on these large applications became more difficult due to the fact that business logic became more complex (white-labeling of our trading platform). And we decided to do what most organizations did back in 2014: break the logic into microservices.
Everything went well, and by 2016 we had about 150 (micro, approx.) Services in production. But the Chef scripts used to configure the virtual machines have grown and are out of control. Development environments have become much more complicated for some applications with about 30 dependencies. We also found out that knowledge of Chef was not part of the standard set of skills of our developers. For this reason, many people just made the script work, at times, by copying bad examples. Trying not to break the configuration for other teams, they created long-lived brunches for their own team needs, and, as a result, the development environment configuration code was no longer public.
We had to fix it.
The following weaknesses of our sandbox were identified:
- Need to know Chef. And for existing developers, and for beginners, learning this framework was quite an expensive pleasure.
Also, this was redundant only for the development / testing environment. - There was no way for easy abstraction / composition or reuse of part of the configuration of the environment. This led to the fact that people had to delve into all the details and nuances of the dependency configuration of their projects, which were usually supported by other teams. This led to disagreement and unnecessary interaction between the teams.
In order to fix the first point, it was decided to switch to using Docker. It has a lower entry threshold and also provides other benefits, such as standardization, PaaS support, and easier deployment across multiple environments.
We also considered using only docker-compose. It looked very promising, but despite its name, it lacked such things as the possibility of composition, and, at the same time, abstraction and reuse. It is those aspects that we needed. We dreamed of being able to take the existing development / testing environment, add one service to it, and optionally redefine part of its configuration. We wanted to achieve this without having to copy the configuration of the added service or to delve into the details of transitive dependencies and their configuration.
It was then that we had this idea: “What if we take Maven, the dependency management system we use to build our java applications, and start using it to build our environments?” This would allow us to abstract our transitive dependencies, package configurations, create versions of them, and reuse them. There will also be no need to learn a new dependency management system.
After a number of attempts, we came to the following solution:
- Each Maven artifact represents a fully functional environment that can be used as a dependency in other environments. (minimal environment can be represented by one service, for example, a database)
- JAR files contain configuration. Environment variables, application configuration files, image Docker name, etc.
- This configuration can be overridden in modules located higher in the dependency hierarchy, if necessary.
- Java library can lift the complete environment (using docker-compose under the hood) from one Maven module (GAV or pom.xml)
Maven makes it possible to distribute and version these artifacts (“environment artifacts”) and we can easily share the source code between the teams. The process became simpler and developers began to use modules of other teams. And also create and maintain your own.
As a bonus, all tools facilitating work with Maven can be used here. As an example, the dependency graph generated using IntelliJ IDEA now shows us a diagram of the architecture for the interaction of our dependencies :)
Architecture of a
voting application example
from docker-composeAt the same time, there were some implementation difficulties, such as forcing all teams to pack applications using Docker (dockerize) and pack configurations into separate Maven modules. By all accounts, it has become easier to import applications from other teams and support complex development environments.
We started working in this direction about a year ago, and now we have about 200 of these modules used for development and testing. We came to the conclusion that this library is also great for managing the temporary environment for end-to-end testing on our CI platform.
We are currently studying how to reuse this technology to manage our UAT / integration environments by launching containers in Kubernetes instead of docker-compose, and thus ensure rolling-updates and computing elasticity on significantly larger environments.
I hope you will be glad to know that our project was opened under the Apache 2.0 license and is available on Github:
github.com/swissquote/carnotzetUse :)