Agile Software Development

In this article, I wanted to write down some ideas and opinions I stumbled upon while discussing agile software development with colleagues or reading books and articles. I have been dealing with agile project management in software development for some time now. It all started with pure interest, but recently this topic has become more important for my work. That's why I'm also participating in discourses with several people about this topic. Anyway, in this simple article, I would like to sort out and elaborate on my findings and the different perspectives I have encountered.

Project Management and Software Development

We encounter project management in various professions, wherever ever-larger visions or tasks are realised by a group of people or even companies. It not only concerns the area of software development, but we can find this term in other sectors like business, industry, construction, or even in school.

However, I want to focus one software development here and project management represents a fundamental aspect of it, at least if we can assume a certain size (in terms of task and/or team). It describes the methodologies that structure the work of developers and other involved people. This structuring aims to make software development easier, which hopefully results in easy-to-understand tasks and a good overview of open tasks and milestones that should be achieved.

The term software can represent a single product or feature, and it also can be just a fraction of the overall system a company is using or selling. Nevertheless, there is often one fundamental characteristic in all these software or software systems: They are developed continuously. Even if there is a bigger vision for a product or software system, it generally starts with a small feature set that grows over time. Nearly no one plans and implements software (or things in general) on a very large scale and if so has a very high risk of failing. That's why project management is such a fundamental aspect because it tackles exactly this characteristic.

Software development without project management is simply impossible from a certain size of the developer team or the tasks. Accordingly, it is also essential to deal with this topic in a differentiated manner because I may doubt that chaos is rarely a good choice for management here.

However, there is still something I want to point out before taking a closer look at project management itself. It's the term development in software development. When we develop something, we are talking about the fact that a result has been created at the end of this development process. This can be simply an opinion, idea or also a realization   - yes, for me, it can also be the realization to have developed nothing at all ;-). But basically, the term differs decisively from the term production or to produce! With development, we describe something that arises without knowing exactly how the final result might look, e.g. like a tree that grows, we know what will arise from a sprout, but not exactly how it will look in the end (not in every detail we just have an idea). With production, this is quite something else because here, we know exactly what will be produced or created; the coincidence is left here with little to no leeway.

Agile Methods in Software Development

This difference between development and production is also decisive for project management and explains the change of mindset there in the recent decades and the success of agile project management methods.

A very long time ago - and even today - models like the waterfall model described how to develop software best. I don't even want to talk about this being fundamentally a misconception because developers can still do small or simple projects very well with the waterfall model. But this is also the reason, why I don't want to talk about these small or simple cases. The only misconception I want to talk about here is that small or simple projects don't scale linearly to large or complex projects in terms of project management. From experience and I think many other developers may agree with me on this, the capability with which people can plan projects and anticipate risks and difficulties is independent of the size or complexity of the projects. This capability is almost the same for each person so that the size or complexity of the project does not change how much that person can foresee or anticipate. I believe that in terms of the number of developers involved in planning, this is logarithmic and capped by a maximum. What this means is that one developer or project manager or whoever can only go and develop software in small (iterative) steps.

This problem of predictability is also the reason for the success of agile project management methods. The focus here is not to plan the entire project at the beginning or before the actual implementation but continuously throughout the whole development process. However, I would also like to state that this is understood very differently by some agile methods and in my opinion, often misses the core of the actual problem. I sometimes find the way some agile experts interpret these basic ideas or concepts to be completely alienating. For example, I have read about ideas where corresponding experts think that there is no longer any need for something like documentation or planning as a whole.

Basically, I don't want to give an assessment of the different methods such as Scrum, XP and similar at this point because I simply lack the experience with most of these methods and only know Scrum in some ways. Furthermore, I don't want to present and explain explicit methods here but instead talk about specific core ideas and concepts crucial for software development.

Splitting Software Development Into Cycles

It should have become understandable that the idea of thoroughly planning a large or complex software, product or feature and then implementing it without difficulties is unrealistic. Here the agile methods are showing us a way out of the misery because it is dividing the entire development into parts/cycles that each person involved in the project can oversee fundamentally. A developer or project manager no longer needs to have an overview of the entire project and development at the beginning, but only one part of the process at a time.

In Scrum, for example, there is the product backlog, which describes individual components of the product, mostly in form of requirements (usually user stories or something similar). During the sprints in Scrum, only a part of the product backlog and thus only a part of the total work needed for the project is processed or performed. This ensures, for example, that only a part of the work has to be planned and monitored at any one time.

It is important to understand here that the product is constantly evolving throughout the entire development, i.e. throughout all cycles, and therefore the achieved results must also be reflected and monitored continuously. I also want to mention that some decisions must be taken into account as early as possible in the project planning, even if their effects or implementations do not occur until later. These include, for example, some but not necessarily all decisions regarding the architecture of the software.

What Does It Take for a Good Agile Software Development?

I sometimes struggle a bit with some agile methods and the ideas or notions of some agile experts because I often have the impression that they very much arise from the perspective of project managers. If this is done, in my opinion, a certain aspect of the development itself is overlooked, because in this case the goals to be achieved or the end results are usually in the foreground again, not the process itself. Therefore, it is crucial to include the perspective of the developers to the same extent.

If you were to ask me what the core characteristics and ideas of good software development are, they would be flexibility, trust, and openness. It is important to emphasize that these characteristics must be demanded of everyone involved, from the project manager to the stakeholders to the developer.

For me, flexibility refers to the general agile basic idea. That means being flexible in dealing with the product vision, the requirements analysis and the implementation, which also means always reacting flexibly to problems or events. In other words, wanting to implement the project in a structured manner through individual development cycles and letting the product development itself. This also means giving the product (and therefore the developers) time to develop. This doesn't mean we don't need deadlines or estimations, that is wrong because without this we lose our focus on what to prioritise.

This brings me directly to the next point, trust. I do not refer this to as the trust of developers among themselves, but rather the trust of all participants, in the development process itself. There must be mutual or general trust that the intermediate results will develop more and more through the individual development cycles into the desired end product. It might seem like developers or the product losing track of the desired results but that's no big deal because that just part of the overall way the development process is going and developers can correct the way they are going in the next cycle.

But what is the best way to create this trust? I would suggest openness. The development of the software or even the product, features, or releases must take place openly. Everyone involved must have a clear idea of what the goal is, in which steps they want to get there (at least roughly), and of course, they must also be able to address problems or difficulties openly. Furthermore, all participants need to be able to influence the development process itself.

Amusing how all of these things blend into each other or even seem to blur together, right? But I think it's only fitting because everything goes hand in hand, in my opinion, and ultimately always has to be applied together to make this whole development a success. You might be able to add other terms to this list and this is absolutely fine, I wouldn't say that only these three terms make up good agile software development, but these are the terms that (currently) matter the most for me.

Lessons Learned

In software development, large or complex projects cannot simply - or only rarely - be planned completely in advance and then implemented. It turns out that it makes sense to proceed iteratively or incrementally with such developments, which also explains the success or trend of agile project management methods. However, agile methods are not a foregone conclusion and flexibility, trust, and openness are generally crucial for software development.

Software Development Agile