A new software development process starts
Oliver, how come Coop wanted to develop a new software development standard based on this project? What role did Unic play in this?
Coop asked us to support them in the area of AEM development. This revealed various problems that were not solely AEM-specific. For example, there were problems with modules that overwrote each other. The need arose to reduce the «integration problems» between these modules and thus to increase the quality of the artifacts delivered in order to achieve more stable operation.
Coop then came up with the idea of developing a new software development process as part of the new FOOBY project, which would then be used as an example for other new AEM projects. As Unic, we were allowed to accompany Coop on this path with our expertise and experience. We supported them in the implementation of this new software development process.
How did you proceed in setting up the new development process? How have you organised yourselves?
Together we checked what software development tools and techniques were already in use and then outlined the target state. We then implemented the individual tasks (automatic tests, continuous integration, continuous delivery & deployment) step by step in the FOOBY project.
Since we decided to proceed according to Scrum, we were also able to evaluate at regular intervals what worked well and what didn't work as well, and to adjust corresponding points.
What was the aim of the new process?
To increase the quality of software artifacts and to design the software development process in such a way that the software can be put into operation in appropriate quality.
Developers from Unic and Coop worked on the application at the same time. How was the cooperation in the joint team for you personally?
Not only Unic and Coop team members were present. We were a cross-company, cross-functional team. In addition, we had interfaces to various peripheral systems, which is why other people were involved in the project.
The work in the Dev team was great and it gave me great pleasure to work with such different people. We needed some sprints until we «warmed» up to each other, but this is normal for a new team. The regular retrospective meetings of Scrum have helped us to grow together.
I had the feeling that there was a good team atmosphere: people were prepared to help each other, even if there were jobs that were not necessarily in their own «area of work». That was a good, reassuring feeling. Because about 3 months before the planned golive my wife and I got our daughter. I was glad that the team gave full speed and I could enjoy my paternity leave.
Why automated tests are so important
Let's get to the technical aspects: For engineering you have introduced automated unit tests and integration tests based on ScalaWebTest. Why, what are the advantages and how have the testing technologies proven themselves?
The main reason for the general introduction of automated testing was that we wanted to improve the quality of software artifacts and have mutual assurance. For example, the tests took into account dependencies between the various components in the system.
The introduction of automated tests has the following advantages:
- Writing automated tests forces a developer to build software that is easy to test. Thus, more robust software is already being built in the core.
- With automated tests we can ensure the integration of other modules.
- The use of so-called «mocks» for surrounding systems – these are simulated surrounding systems – allowed us an implementation based on clear interfaces. It also made us more independent: We were able to continue working on the train without Internet access because we did not need a connection to a peripheral system.
For the unit tests we have chosen well-known technologies such as JUnit, Mockito, AssertJ or SoapUI for the mocking of peripheral systems. We also use AEM/Sling specific mocks.
We chose ScalaWebTest for the integration tests. This enabled us to write very expressive tests. Since ScalaWebTest is based on the well-known testing framework ScalaTest, we are also well equipped for the future. ScalaTest is actively developed by a large community. It allows to write other styles of tests, which makes it very flexible. It is nicely integrated with the Selenium testing framework, respectively it integrates the WebDriver API in a simple way.
The system must be running and usable for the end customers
How have you implemented continuous integration (CI) and continuous delivery & deployment?
Continuous integration and continuous delivery&deployment are basic techniques, but also basic ways of thinking that pursue the idea of bringing software to the end customer in the highest possible quality and as quickly as possible (business-driven). Software only adds value if it takes the needs of end customers into account and thus makes itself available to them. That means: The system runs and is usable. With just a few sketches or clickable prototypes, the end customer still do not receive any functional software that solves their problems.
For CI and CD we used «Apache Maven» as build tool. This tool manages the building, testing, packaging and deployment of the software.
Strictly speaking, we have not implemented continuous deployment fully yet. Continuous deployment would mean that all necessary tests would be run automatically shortly after a code commit, including acceptance and performance tests. The new code would then automatically go live so that the end customers could access the latest functions after a few minutes. However, we still have some intermediate steps in between. For example, we can automatically deploy the software to the various environments using scripts.
Keyword modularization: How modular did you implement the Fooby project?
Hmm....hard to say. There are different levels of modularization:
- At system level, we have integrated various data from surrounding systems, such as enterprise search, recipe services or the central authentication infrastructure. We did not want to solve these requirements in AEM.
- At product level, we have made the necessary separation between the logic part (OSGi services, models etc.) and the display part (templates, components).
- At the code level, we tried to group the code according to features, for example at package level or OSGi bundle level. For example, we have outsourced the entire topic of enterprise search integration to our own OSGi bundle, so that this can easily be used by other projects. This means that other projects could and can benefit from the fact that we have already solved this problem here.
Building software is a deeply social art
How far have you come and what else do you want to accomplish?
I think we were able to implement the main part. As developers, we now have a new software development process that is fully integrated: running automatic tests, building and deploying release packages, and semi-automated deployment. Especially the part with the automatic tests was a core of the implementation. It is a process and constant learning how to write good tests.
Ultimately, it is always a matter of writing well understandable and testable software that is sustainably maintainable and remains adaptable. And this can only be done via coding, coding, coding...learning, learning, learning and again coding, coding, coding...
As already mentioned, we could still fully automate the deployment. This means that a stable deployment at the push of a button should be possible by a product owner or a system engineer (operation). We're not there yet. For example, acceptance tests from the department (automated as far as possible), automatic performance tests or automatic rollbacks in the event of an incorrect deployment are still missing.
It is also always a balancing act between the costs of maintaining a production environment and the costs of having to deploy several times if something does not work correctly. Deployment stability with AEM was also a challenge. Depending on how the «file-based» deployment in AEM has proven itself, one would have to look again.
One of my personal concerns was that we learn to develop software «with more love», entirely in the sense of «Love your code and teammates!» And this concerned not only the professional and technical aspects, but also, or even above all, the organisational and social aspects. Building, constructing and developing software is a profoundly «social art». I think we have achieved this as a team and this is something worth spreading.
As fooby.ch’s technology partner, Unic ensured a smooth user journey thanks to its technical solutions created on the Adobe Experience Manager (AEM).