Best Practices For Software Design And Development

— Chapter 2 —

«prev          next»

Author’s Note: This chapter is now freely available to the public. All paywalls have been removed.

In the previous chapter I complained a bit about the clean code principles not being given the necessary attention they deserve in software organizations during the software development and/or engineer hiring process. But what exactly are these clean code principles? Well, this is what this chapter is all about.

However, before we come to the actual clean code principles, there are three important things I need to mention.

First of all, this is one of the most technical chapters in this book. I tried to make it as simple and clear as possible when writing it. However, there is only so much I can do. These are technically heavy subjects by their very nature. Going into this chapter, it would help to have some basic knowledge of software concepts like classes, functions, object oriented programming, composition, and inheritance. The good news is that the chapters following this one will be more palatable for the readers with no background in software engineering, so please try to hang on.

The second important thing I wanted to mention is this: When you are a newcomer to a team and look at their code base for the first time, it never looks clean or easy to understand. This is true whether you are a software engineer with decades of experience or a brand new Computer Science graduate. If you are a brand new graduate, no code with more than a couple of hundred lines would look very understandable to you anyway. The ability to read someone else’s code is an important skill that is only learned with lots of practice.

Grady Booch says in his book named Object-Oriented Analysis and Design with Applications, “Clean code is simple and direct. Clean code reads like well-written prose. Clean code never obscures the designer's intent but rather is full of crisp abstractions and straightforward lines of control.”¹

Well, unfortunately in all of my entire software engineering career, I have not come across any code base that reads like well-written prose. (And this includes my own code.) Neither did I come across any other software engineer praising their codebase as a well-written novel or short story.

Making sense of a code base is always difficult, no matter how well it is written.

Nevertheless, some of the code I’ve come across was cleaner than others. In some cases it took me a couple of days or weeks to come up to speed on a codebase (depending on its size). And in some cases, looking at the codebase for months wouldn’t make much of a difference. In the end, somehow, I found ways to make it through such codebases. Sometimes, it helped having long meetings with the code author who would explain their code to me, in the rare times they were around and available.

The difference between dealing with these two kinds of codebases is thinking on one hand “well, that took some time, but I believe I understand what’s going on here”, and thinking on the other hand “what are the choices I made in my life that brought me to this miserable point?” No one wants to be in that existential anxiety questioning the choice of one’s career. Unfortunately, software engineering is one of those careers that can bring great joy as well as great misery to its practitioner. And unfortunately, it all depends on the people you are working with, and how well they have written their code.

The final important thing I wanted to mention is this: I have learned all of these software engineering best practices by reading from many talented authors and experienced software engineers over the years. Nothing happens in a vacuum. The best practices that I mention in this book have been developed over multiple decades of collective software engineering experience and wisdom. As some say, software engineering is more craftsmanship than science.

Some amazing books that I’ve read and have inspired me on this subject include Code Complete by Steve McConnell², The Mythical Man-Month by Fred Brooks³, The Pragmatic Programmer by Andrew Hunt and David Thomas⁴, The Passionate Programmer by Chad Fowler⁵, Effective Java by Joshua Bloch⁶, Refactoring by Martin Fowler⁷, and Design Patterns by Erich Gamma, Richard Helm, Ralph Johnson, and John Vlissides⁸. Robert “Uncle Bob” Martin is another author I read, and his books are titled Clean Code⁹ and Clean Architecture¹⁰. I am sure I am forgetting many other amazing software engineering authors that I’ve read along the way.

So, without further adieu, let’s get into the clean code best practices. These are the ones that I believe to be the most important and fundamental ones. Of course there could be countless more that I might have overlooked.

Clean Code Best Practices

There is an old joke: There are 2 hard problems in Computer Science… Naming things, cache invalidation, and off-by-one errors.

While this joke might be a tad nerdy, it touches on some truth: Naming things is a hard problem. And it is a very important clean code best practice. When writing code, one must be careful in naming all the classes, methods, functions, fields, and variables. The names should show their intent very clearly. Looking at the name of a class, method, or variable, you should be able to see what exactly they intend to do.

Do not assign cryptic or nonsensical names to variables. Also try to avoid assigning one letter names like x, y, z. There are exceptions to this rule though: In some languages it is conventional to use one-letter variable names, like the Go language. And in most languages, letters like i and j are frequently used as loop indices. As long as it fits the convention, and it is comprehensible by the people who read the code, it is ok to use the one-letter variable names. Otherwise, try to keep your variable names expressive and easy to understand.

Speaking of variables: You should try to keep the variable definitions local, and close to where they are used in the code. Let’s say you define a local variable and then use it (or reassign to it) 50 lines later in the code somewhere. This would seem confusing to a reader of this code, who would find it difficult to keep track of such variables, particularly if there are too many of them flying around. It is much easier to understand a variable that is defined, assigned, and used all within a couple of lines.

Keeping the variable definitions local would limit them to a single scope (e.g. within a single function definition, or within a loop definition, etc.) Variables in a limited scope are easier to keep track of, compared to let’s say a class field that is updated and read in multiple different methods of that same class.

Global variables should be avoided. By definition, a global variable can be changed and reassigned to another value anywhere in the entire codebase. This makes them very dangerous. It is difficult enough to keep track of variable updates in a 50-line method. It is pretty much an impossible task to keep track of updates to a global variable in the entire codebase. This is just a recipe for disaster, making it very easy to add bugs to the codebase during development.

Global constants on the other hand, are fine to have. Unlike variables, constants are assigned once during the entire run of the program, at the point of their definition. Their value never changes after their initial assignment. Therefore, it is safe to define a global constant and use it from anywhere else in the program. The one potential downside to this is, if it’s a large enough codebase, there might be multiple global constants that might be inadvertently defined to represent the same thing. Conversely, there could be a single constant used at too many places, while separate constants should be defined in its place: Like a timeout value that should be specific to each separate component. When something is used globally within the codebase, even if it is a constant, there might be problems associated with it. Oftentimes it makes sense to limit the scope of a constant to a single class or module.

It is much better to define and use a constant rather than write the same number or string value all over the place. For instance, if a timeout value in your component is 1500 milliseconds, don’t write 1500 all over your codebase. Instead define a TIMEOUT constant with a value of 1500, and use that instead. Use of constants also makes the code easier to read. It is easier to read and understand constant names rather than trying to make sense of some enigmatic numerical values all over the codebase.

Another important advice is to keep the functions, methods, and classes short. Shorter functions and classes are easier to understand, and would contain less variables to keep track of. The rule of thumb is to not have any functions longer than the length of your screen (25-40 lines), and to not have any classes more than a couple of hundred lines.

While keeping the functions and classes shorter than a certain number of lines, it is also important to keep these lines themselves short. Each line of code should do only one thing, execute only one statement. Some languages allow multiple statements to be stuffed in one line of code. That is what I am warning against here. The principle idea is to keep the code readable, not just to keep a method or a class below a certain number of lines. Let’s try to avoid the loopholes, and stick to the fundamental principles.

Functions and methods should do one thing and one thing only, without any side effects. Let’s say you have a function that reads a value from a database and returns it. That function shouldn’t do anything else, like writing a value to somewhere else in that database. And if that function is meant to write a certain value or a related set of values to a database, then that is the only thing that that function should do. Functions that do multiple things are hard to keep track of in our minds, and could easily cause bugs to proliferate in the codebase. Someone could call such a function forgetting it does multiple things, which could cause unintentional actions to take place in the code.

As always, there is an exception to this rule: It is ok for a function to do logging to a log file. Logging is usually very helpful to us when we need to do debugging. A function or a method could be doing logging in addition to that one thing they do. Writes to logs should not have any adverse effects on the working functionality of a code.

Similar to a function, each class should also have only one single responsibility. I am going to cover more of this in the next subsection, when I am talking about the SOLID design principles, and the Single Responsibility Principle, which happens to be the letter S of the SOLID acronym.

Functions and methods should not contain too many arguments in their definition. If a function has to be called with too many arguments, this makes the code harder to understand. Any function with 5 or more arguments becomes confusing. There are several ways to overcome this. One way is to group the arguments into a more coherent data structure, and pass that data structure instead. To show you what I mean, instead of passing in a street address, city, and zip code separately, imagine you could instead pass in a single address object into a function. Another way could be to split the function into separate different functions, each of which take fewer arguments.  Also, if it is a method that you’re dealing with, it might be easier to use a class field instead of passing in that value as a separate method argument. You are going to have to get a bit creative when trying to make your code easier to read and understand. But that is one of the great joys of software engineering. A clean and readable code that works like a clockwork makes all these efforts worthwhile in the end.

You should avoid an extensive list of conditions in your code. A code section with 10 if-then-else statements would make it difficult to read and understand. One solution, if you can, is to replace them with switch-case statements. There are more creative solutions: If the numerous if-then statements are operating on a single variable value, maybe use a hash-map where that variable is the key, and the behaviors you want to execute are the values on the hash-map, such as callable function-objects. But try not to get too carried away with over creative solutions. In the end, the code should be as simple and clear to understand as possible. If your code contains numerous if-then statements, the best solution could just be to reorganize that code into multiple separate functions or methods that handle those conditions separately, if it’s possible. You should apply your best judgment into how the code should be reorganized.

You should also avoid extensive nested conditions or extensive nested loops. For example, too many for-loops within each other makes the code hard to read. It might also be a sign of inefficiency in your algorithm. If you have 3 nested loops, your runtime might be O(N^3), which would be a really bad thing. In this case, it might be a good idea to rethink your algorithm.

Try to use comments effectively. Code comments should explain the overall intent of the code. Each public class and public method should contain comments documenting its functionality, as well as its inputs and its outputs where applicable. Comments can also be used to explain why a certain section of code was necessary, like how it was added to solve a very specific bug, to give an example. But do not write comments to explain what the code does in a step by step fashion. Reading the code itself should be enough to do that. Also, the code itself might change in the future, while the step by step explanatory comments may not be updated. This would lead to a very confusing situation.

In the end, your code should be clean, well organized, and easy to understand. Working with such a codebase is much more fun than working with a bad alternative.

Immutability

I talked about why global variables should be avoided while it is ok to have global constants. This ties into a more wider-reaching principle: Immutability.

It is a best practice to define as many of your data structures and their fields as immutable.

A variable is considered immutable when its value is only assigned at the time of the variable’s creation, and cannot be changed in any way whatsoever afterwards. Each programming language usually has a way to define a variable as immutable. (E.g. Java has the “final” keyword.) A data structure or a class is considered immutable when all of its underlying fields (i.e. the data fields in the data structure) are each defined as immutable. The data of an immutable object cannot be altered in any way once it is instantiated.

In a way, each field of an immutable class is similar to a constant. However, true constants are not tied to a particular class instance (a.k.a. an object), they are at best defined under a class namespace. In other words, a particular constant has the same value throughout the entire program. On the other hand, an instance1 of an immutable class can have fields with different values than those of an instance2 of the same class. It’s just that these field values can never be altered in any way once their respective objects have been created/instantiated.

Why are immutable data structures better to deal with? For one, it is much easier to introduce a bug in a codebase where an object’s fields are mutable, or changeable. One can accidentally end up changing the value of a field in an object when they’re not supposed to. With immutable objects, once they are created, it is impossible to do that. Their field values are set and immutable for the lifetime of the object.

Immutable objects are also great to use in multithreaded programs. This is actually one of their greatest strengths.

You can think of a multithreaded program as a program where multiple threads of execution are running simultaneously side by side. There could be situations where multiple threads try to access the same object instance and update a particular field value. In this case, the threads are in a race to update the same field value, and one of the threads gets to make the update first. The thing is, you don’t know which order the threads are going to make the updates. The order is completely random. Which way the field value will end up cannot be known. This is a dangerous bug known as “race condition”. If you have a multithreaded program that is randomly failing sometimes but succeeding to run in other times, you might have a race condition.

When an object is immutable, no thread can make any updates to it whatsoever. Race conditions are not a possibility with immutable objects. It is completely safe to pass around immutable objects between multiple threads.

As a side note, the subject of multithreading is a very complex and interesting one. There is a lot to be said about multithreading, which goes a bit beyond the scope of this book. So, I’m not going to write a lot about this subject here, but I highly recommend you to read about it elsewhere.

There are times when you cannot define every field on a data structure as immutable. Sometimes, some fields of an object need to be updated after the object’s creation for valid reasons. That is ok. In this case, just try to define as many fields on the object as immutable, as the structure of your code and algorithm permits.

DRY

DRY is an important clean code best practice, which is an acronym for Don’t Repeat Yourself. Its purpose is to reduce duplication in software.

Any time you see a section of code repeating itself in various different places in the codebase, that’s a good time to apply the DRY principle and reduce that duplication. You should create a separate function and sometimes even a separate class to hold that functionality, and then move the code in question there. Then every place in code where that code used to be call the new function or class method from there instead. This is a form of refactoring that is used to apply the DRY principle.

The DRY principle should also apply when you see duplicate sections in the same class or method. Then you should create a separate private method to refactor that duplicate code out. The new method should be private since the duplicate code was limited within the same class, so no other outside class or entity should be able to make any calls on this refactored method.

The DRY principle applies to classes and packages, even at the architectural level. When you notice duplications or repetitions in your large-scale architecture, feel free to refactor those architectural elements out to reduce the duplication. For example, if there are database calls being made throughout your codebase to the same database by using different classes or packages, then those classes should probably be consolidated in a single database package to be used by the entire codebase.

Software is usually never static. It changes throughout time, as new features are added, and as old bugs are fixed. When some code is duplicated, it becomes easy for each of its duplicates to be changed in different ways. Someone might apply some fixes to one duplicate code section, while forgetting to apply the same fixes to another duplicate section. This is a recipe for disaster, potentially adding more bugs to the system. However, if the duplicates are reduced and the relevant code is in a single function, then any future fixes or changes are made to the code in a single place, avoiding these kinds of bugs.

It is possible to go a bit overboard with DRY. Throughout your codebase, you might come across similar looking code, constants, etc. which makes you think they should be consolidated in one place. However, these code sections or constants could be used for different purposes. Their similarity could be a coincidence. In this case, it might be best to leave them separate, and let each of them evolve in their own way as the codebase changes over time. You must apply your best judgment to each situation, as each situation is different.

KISS and YAGNI

KISS is an acronym for Keep It Simple, Stupid! It’s a design principle that got popularized by the U.S. Navy in the 1960s. It emphasizes keeping the software design as simple as possible, and not adding any unnecessary complexity. YAGNI is a principle that arose from the Xtreme Programming fame of the more recent times. It stands for You Aren’t Gonna Need It. It states that you shouldn’t add any functionality to your codebase until it is deemed necessary.

In software engineering, it is easy to over-engineer things. Oftentimes, it is difficult to see how the system is going to evolve in the future, and what kinds of features are going to be needed to be implemented. So you try to anticipate the future, and try to add more functionality, more classes and modules into software design. However, oftentimes, the future you anticipate doesn’t come to pass. Turns out you aren’t going to need all that extra functionality. Turns out it’s better to keep the design as simple as possible.

However, please don’t use these principles as an excuse to avoid your responsibilities to the code cleanliness and architectural soundness. Always try to design your software with changeability and evolvability in mind. Do not design yourself into a corner. This means you should not design software that is difficult to change just so that you can “keep things simple”. Always have and use your best judgment. If it turns out that you were wrong and “you are going to need it after all”, then try to refactor your software as soon as you can.

Clean Architecture Best Practices

Most software grows as time passes. More features are added, as more bugs are fixed. As the software grows, it becomes important to give some thought to its large-scale architecture, and how the different classes, modules, and components all interact and fit together. It becomes important to think about clean architecture.

The primary purpose of a clean large-scale architecture is to enable the software to be changeable as it evolves. Badly designed software architectures are harder to change. It is harder to figure out where in the codebase to make the necessary modifications. It is also much harder to add any feature to a badly designed software without causing serious bugs. A cleanly designed architecture makes it easier for us to change the software.

The SOLID design principles were developed to help make object oriented designs more understandable, flexible, and maintainable.¹¹ Since a lot of the software development in the world today uses object oriented programming paradigm (with a sprinkle of functional programming paradigm), these design principles make a good foundation for designing a clean software architecture.

SOLID is an acronym which stands for the following set of design principles:

• Single Responsibility Principle

• Open Closed Principle

• Liskov Substitution Principle

• Interface Segregation Principle

• Dependency Inversion Principle

Single Responsibility Principle

The Single Responsibility Principle is one of the most important design principles. I do not say this lightly, as I have observed throughout my career that the violation of this principle is responsible for the many ills in software design and development in the world today. Violation of this principle is the reason why I came across that 6,000 line class, and why you might come across a 30,000 line class in the future.

The Single Responsibility Principle states that every class or module in a computer program should have responsibility over a single part of that program’s functionality. In other words, each class should have only one reason to change.

It has been my observation that pretty much every class starts out its life as small and simple. Then it starts growing as more code is added to it, as more features are added to the overall software. And pretty soon, in a couple of months or years, you get the 10,000 line monstrosities.

Let’s say you are developing an Account class. It is supposed to represent a customer’s account. It starts out as a simple design: It probably has fields representing an account id, the date opened, the account type (e.g. premium, basic, free-tier, etc.), the name of the customer associated with the account, and maybe the address associated with the account, and so forth.

After a while, you start to add methods to this class to write the Account to a database and read it from that database. And then after some more time, maybe you add some behavior to validate the address information. The address validator connects to another remote service and tells you whether the account’s address is a valid one.

You keep adding more and more functionality to this Account class. And then inevitably, it turns into a huge monstrosity. It becomes harder and harder to understand its code, and add more functionality to it. Anytime someone makes an update to it, the tests start failing, and the chances of a production server failure increases. At that point, you realize that the class has become a gigantic bug-prone mess.

This is because too many responsibilities were added to this class. There should have been an Account class that only serves as a data object. It should have been only responsible for holding the account information, that’s all. There should have been separate AccountDataWriter and AccountDataReader classes that are responsible for writing the account to and reading it from the database. There should have been a separate AddressValidator class that is responsible for validating the address on the account.

Any time you’re adding a new field or method to a class, you need to consider whether you are violating the Single Responsibility Principle. If that’s the case, then it is a good time to create a new class that holds that separate responsibility.

As might be expected, refactoring a class into multiple fine-grained classes would eventually bring its own problem: Too many classes that could be hard to keep track. At this point, it would be a good idea to organize these classes into separate packages. And further reorganize the packages into hierarchies of packages, to make sense out of them. The data objects could be in one package, validators could be in a separate package, while the database related objects could be in a package of their own. You could further create reader and writer packages under the database package, for the relevant database operations. These are just examples, but you get the idea. You should also make sure that the packages themselves don’t violate the Single Responsibility Principle. For example, the data objects package (containing the Account class, etc.) shouldn’t contain any class related to the actual database read/write operations. Any given package should contain a reasonable number of classes or other packages. The Single Responsibility Principle doesn’t just apply to classes. It applies to everything, and helps us make better sense of the software.

Open Closed Principle

Open Closed Principle is another cornerstone principle which tries to emphasize that software architecture should be easy to change. Officially, it states that software entities (classes, modules, functions, etc.) should be open for extension, but closed for modification. This means that if you want to add a new feature to some software, you should be able to do it by writing some new code rather than changing or modifying much of the existing code.

This can be best understood with some examples.

Let’s say you’re working on a software that deals with customer accounts. (I know this is a boring example, but it’s a very common example in the software development world.) Let’s also say that this particular software wasn’t very well designed. One day, the business folks come to you and say they have come up with a new account type that has a different pricing tier than the existing ones. Now you’re going to have to look through your entire codebase where all the pricing calculations are done, and try to modify them. Due to the haphazard nature of this software design, this might mean a lot of modifications. A lot of modifications mean a lot more opportunities to add new bugs to the system. Definitely not something to be desired.

But let’s say your software was actually well designed. The account pricing calculations are all done in one place: In a class with a well defined interface. Depending on the account type, a separate implementation class is used to implement that price calculation interface. Now when the business folks come to you with this new requirement, all you have to do is write a new implementation class for the new account type’s pricing calculation, and make sure it is instantiated and used appropriately in the system. (Needless to say, you also need to make sure it is tested correctly, with a full proper suite of unit and integration tests. More about this in later chapters.)

This is the Open Closed Principle in action. In the second scenario above, the software was open for extension, but closed for modification. All you had to do was to write an implementation class to extend the software properly. You didn’t have to touch or modify the rest of the codebase.

Here is another example: Let’s say you are working on a customer account code again, and you want to model accounts as active or inactive. Let’s also say you have been under a lot of deadline pressure, you have been working late at the office almost every day, and your brain has been fried from all this overwork. You just want to get this task over with, as quick as you can. So you choose the easiest solution: Just define a boolean variable named isActive to represent the account state. If isActive true, the account is active, if it’s false, the account is inactive.

Then a couple of weeks or months later, they come to you and say they want to model suspended accounts and deleted accounts as well. Now you are in a bit of a bind. You could define new boolean variables to represent these account states, like isSuspended and isDeleted. But then you have to be very very careful. Each time the account state is changed or used in the code, you have to make sure everything is consistent. To give you an idea, it’s easy to set the isSuspended variable to true while forgetting to set isActive to false. An account cannot be both suspended and active at the same time. Logically, only a single account state boolean can be true at any given time. Such an oversight can easily cause weird bugs to arise in the system.

The correct thing to do would have been to define the account state as an enum instead of a boolean from the very beginning. You could have defined an AccountState enum with values like ACTIVE and INACTIVE. Then when the new requirements came, it would be easy to add the new enum values of SUSPENDED and DELETED. You wouldn’t have to worry about keeping the account state consistent. The enum variable could only possess a single value at any given time anyway. This is an example of a well designed software that is easy to change, in line with the Open Closed Principle. The design is open to having new enum values added to it later on.

And on a completely separate note, this is a perfect example of how bad management decisions can lead to a software best practice being violated, resulting in low quality software. It is a terrible thing for managers to overwork their employees in death marches and crunch times. A software engineer who is overworked, stressed out, and under deadline pressure is bound to make many mistakes while designing the software, rendering it harder to change over time. These mistakes add up and become very costly to the organization months and years down the line, as it becomes harder to add new features to the software without also adding lots of bugs. In the end, the organization has to spend a lot of time and resources rewriting the software from scratch. Such rewrites bring their own problems, as it is challenging to rewrite and redeploy a badly designed system that is currently under use. All of this starts from deadline pressures, death marches, and crunch times. And all of this could be avoided if the engineers could take enough time to design the software properly from the very beginning.

Liskov Substitution Principle

The Liskov Substitution Principle was introduced by Barbara Liskov in a 1988 conference. Barbara Liskov is a Turing award winning computer scientist. Her principle is about object oriented inheritance.

The Liskov Substitution Principle states that the objects of a base class should be replaceable with objects of its derived classes without causing bugs in the software.¹² In other words, we should be able to make the same method calls to the derived class objects as we make to the base class objects without causing any issues.

The canonical example here is one of a Rectangle vs a Square. Let’s say you have defined a Rectangle class with the methods that enable you to set its height and width. It also has a method that calculates and returns its area, which happens to be its height multiplied by its width. Let’s also say you also need to define a Square class. Seeing that a square looks very much like a rectangle, you think that it would be a good idea to reuse the existing Rectangle code here by using inheritance, and extend the Square class from the Rectangle class. However, in reality, this would be a very bad idea.

A square is not exactly a rectangle. A rectangle could have a height different from its width, while a square has to have all of its sides equal to each other. With a rectangle you can set its height and width to different values and then calculate its area. If you use a Square object in that same code instead of the Rectangle object, setting its height and its width to different values would overwrite whichever value was written first. The Square’s area calculation would just be done with the latest height or width value set, which would yield a different result than a Rectangle’s area calculation. In other terms, you could not use a Square object in the place of a Rectangle object. If you define an array of Rectangles in your code, you cannot insert a few Squares in that array and expect whatever algorithm you’re running to run correctly. Deriving Square from Rectangle violates the Liskov Substitution Principle.

A better idea would be to define a Shape base class, and derive both the Square and Rectangle classes from it. The Shape class would have no methods to set its width or height, since Shape is a very abstract concept, and not all shapes have widths or heights. (For instance, a Circle is a Shape, and it has a radius or a diameter only.) Each Square and Rectangle class would have its own way of setting its dimension parameters. However, a Shape class could have a calculateArea() method, which Square, Rectangle, and all other subclasses could inherit. This way, you could use Square and Rectangle objects anywhere a Shape object is used. You could define an array of Shapes, insert any Square or Rectangle objects in that array, and calculate areas for all the shapes in that array. Your algorithm would run perfectly fine, since this particular software design is all in line with the Liskov Substitution Principle.

As a software engineer, it is easy to get tempted and just derive a Square class from a Rectangle class. You can tell yourself that you will save lots of time this way, why not just do it. You can say you will only use the setWidth() method on the Square and not touch the setHeight() method, so there will be no issues in the area calculations. Or alter the code so that an exception will be thrown if the height and width are ever set to different values. Your algorithm might even work like this for a while. But then you might transfer to a different team, or leave the company altogether, and someone else might come to take over the code. Then that person could easily end up calling the setHeight() method and setting the height to a different value than the width. Then they would be wondering why some of the tests started failing with weird exceptions, and why the algorithm is not running correctly anymore. Because of the shortcuts you took, the code could be behaving in ways no one would expect. This is the sign of a bad software design.

Interface Segregation Principle

The Interface Segregation Principle states that no code should be forced to depend on interface methods that it does not use. This principle encourages us to split the large interfaces into smaller and more specific interfaces, so the clients will only depend on the methods that they need to use.

If you think that the Interface Segregation Principle is very much like the Single Responsibility Principle, you would be correct. It is very much similar to the Single Responsibility Principle that is applied to interfaces. It also involves simplifying the dependency relationship between two entities.

To provide a simple example, let’s say you want to execute some database operations. In your case, you only want to read certain values from a certain database. As it happens, the database interface class that you want to use also contains methods for writing to the database. It even contains methods to do batch writes to the database. These are operations that you do not need. If these interfaces ever have to undergo any updates, your code’s binary will have to be recompiled & rebuilt, or it would be incompatible with the new interface. This is an example of different modules of code that are closely coupled with each other. Closely coupled code is harder to change, refactor, and redeploy.

A better option would be to depend on a database interface for just read-only operations. If such an interface is designed and offered to us, this would make everyone’s life better. On the library-side, the methods to write or do batch writes to the database would be defined on separate interfaces, implemented by separate classes. If any of those interfaces have to change, it wouldn’t affect us in any way, since we only depend on the interface that does reads from the database.

Dependency Inversion Principle

Dependency Inversion Principle states that high-level modules should not depend on low-level modules. Instead, they should depend on abstractions. This principle helps with the loose coupling of software modules.

If your software follows the Dependency Inversion Principle, the high-level classes should not depend on any implementations of the low-level classes. Instead, they should depend on abstract interfaces. Let’s say that a class you’re working on writes to a database. You should not be calling the database implementation directly from your class. Instead, you should be calling a well-defined database interface class. This would make your class loosely coupled from the database implementation.

Let’s say in the future you decided to use a different database implementation. If your class was instantiating and calling the database implementation directly, then now you’ll have to instantiate a different database implementation, and make sure you call it correctly. On the other hand, if your class was just calling a well-defined database interface, then you wouldn’t have to make any changes to your class code at all. Assuming that the new database implementation also implements the exact same database interface, your class could use the new implementation seamlessly by calling the same database interface methods. (The only caveat is, wherever your class is instantiated in the codebase, the new database implementation has to be passed into your class instead of the old implementation.)

In the first scenario here, the modules are not loosely-coupled. A change in the database implementation causes your class code to change. In the second scenario, the modules are loosely coupled. A change in the database implementation causes no change to your class code. In the second scenario, the software is easier to change. There are less opportunities to add new bugs to the system. In general, loosely coupled modules are what we wish to design when it comes to good software design.

High Cohesion and Loose Coupling

The SOLID design principles lay a good foundation for software design best practices. But there are a few more design principles that need to be mentioned here. They all tie into the various SOLID design principles as well. All of these principles are interconnected in multiple ways.

When covering the Dependency Inversion Principle, I mentioned loose coupling. Now I’m going to expand on this a little more, and touchbase a very important and fundamental design principle:

Good software design has high cohesion and loose coupling.

Cohesion refers to the degree to which the elements inside a class or a module belong together. High cohesion means: Is the class focused on what it should be doing? Do its methods and fields make sense together? Are they cohesive? As you might have guessed, high cohesion is closely related to the Single Responsibility Principle.

Coupling is the degree of interdependence between software modules and classes. It is a measure of how closely connected classes or modules are. Does changing something in a class affect another class? Loose coupling means changes made in a class are not going to require more changes to be made elsewhere. This means less chances of inducing bugs into the system. This means the software is easier to change.

In case of low cohesion, a class might have fields or methods that don’t really belong to it, but in fact belong to another class. In such a case, making changes to one class would result in changes needed to be made in another class, since its supposed fields & methods could be all over the place. This also happens to be an indication of tight coupling between these classes. Thus, cohesion and coupling are related to each other. Fixing one may end up fixing the other as well.

Now, there are going to be situations where a new feature or fix is going to require changes in multiple places in the codebase no matter what. Such situations are sometimes unavoidable. For instance, adding a new text-box on a webpage is probably going to require changes in the UI code, backend implementation, and require addition of a new field to the data objects (e.g. the JSON or the protobuf) transmitted between the two. Updates to the interfaces between software components end up in changes to the implementations of those components as well.

However, not all software updates should cause widespread changes in the codebase. Certain changes made to a backend implementation should be isolated to one or a few implementation classes. If you have ever come across a codebase where making a simple fix or adding a simple feature causes widespread changes across components each and every time, you might be facing a violation of the High Cohesion and Low Coupling design principle. In such codebases, more changes being made means more potential bugs. As the system becomes more complex, interdependent, and tightly coupled, it becomes more brittle and more difficult to change. If nothing is done to fix this, such codebases turn into special “legacy software” that shouldn’t be touched in any way whatsoever by anyone in the organization.

Before you reach this stage, it might be time to think about redesigning that software and paying back some technical debt.

Encapsulation

Encapsulation is a concept that helps us achieve loose coupling. It involves bundling data together in a data structure (i.e. a class), and most importantly, restricting access to that data as well as the other implementation details of the class as much as possible. Only data and behavior that are needed to be used by other classes are made available through public methods. Data and behavior that are only going to be used within the class itself are made private, rendering them inaccessible by any other class.

Encapsulation enables bundling of related data and behavior together and prevents classes from accessing each other’s private data and behavior. In other terms, only the public interface of a class is exposed to other classes, while the implementation details are kept hidden and private. Thus, it enables high cohesion within a class and loose coupling between separate classes. It is a very effective tool for good software design.

Composition Over Inheritance

In the section about the Liskov Substitution Principle, we talked about the importance of designing our inheritance relationships correctly. Inheritance is a powerful tool for code reuse. When you inherit from a base class, you can reuse the methods and fields of that base class, thus avoiding code duplication and complying with the DRY principle.

However, it is possible to get too carried away with using inheritance. When you’re a software engineer who deals with object oriented code, it is an occasional occurrence to come across a codebase that has overdone inheritance. A codebase where one class inherits from another class, which inherits from another class, and so forth. These huge inheritance trees make the code more difficult to understand. The reason for that has to do with the one huge downside of inheritance: Inheritance breaks encapsulation. By its very nature, inheritance causes all the classes inheriting from each other to be tightly coupled.

When you’re using inheritance, your derived class can access all the methods and fields of its base class hierarchy (at least those that haven’t been declared as private). Thus, when you’re dealing with a huge inheritance tree, the derived classes can access the methods & fields of all of their base classes up through the entire inheritance tree. When one of the derived classes is calling a method or using a field, you don’t really know which of the base classes defines/contains that method or field. This means you don’t really know what kind of side effects there might be with calling that particular method, or changing that particular field value. As you all know by now, when the code starts to become very confusing to you, it becomes a breeding ground for technical debt, code rot, and bugs.

So what can one do to avoid the huge inheritance trees? There is another software design principle named Composition over Inheritance. As the name implies, you should prefer to use composition instead of inheritance. Instead of a class inheriting from another class, that class should be a private instantiated field in the other class. Thus instead of huge inheritance trees, you should have a reasonable hierarchy of classes composed of each other.

Let’s say a class A contains another class B, which in turn contains another class C. The parent A can only see the public methods of its child class B. It has zero direct access to class C. Only class B can see the public methods of its child class C. Thus, the principle of encapsulation is preserved. Such a composition hierarchy is more loosely coupled than an inheritance hierarchy. The chances of inducing bugs into this system is much less.

Certainly, I am not saying avoid inheritance altogether. Inheritance has its place in software design. It’s still a very strong tool for code reuse. It’s ok to have the occasional abstract class which contains the common code for its derived classes. A lot of the software design patterns involve the use of inheritance.

In many languages, interfaces and their implementations work through the mechanism of inheritance. Interfaces are essentially reference types similar to classes with just the method signatures, devoid of any actual implementation. The implementation classes inherit from the interfaces and implement those methods. Therefore, inheritance is a very useful tool, especially for following the design principles such as the Dependency Inversion Principle.

Just try to avoid the humongous inheritance trees in your software design, and everything will be alright, hopefully.

Acyclic Dependencies

No software exists on its own. The code you’re working on is always going to depend on other classes, modules, and packages that are present in separate libraries. This means you’re going to include or import (depending on your computer language) those other libraries/packages in your code. You’re also going to indicate those dependencies in your Build files (for Maven, Gradle, Bazel, Makefile, or whatever build automation tool you’re using).

When you are working on a very large-scale software, it might be up to you to determine how the various classes and packages will be organized into separate libraries, and how those libraries are going to depend on each other. Architecting such a large-scale dependency graph is no easy task. You need to make sure that there are no circular dependencies. Let’s say you create a library A which depends on library B, which in turn depends on library C. And let’s say library C turns out to depend on library A. That’s a circular dependency between the libraries. Such a circular dependency makes it pretty much impossible to reuse either library independently of each other. Libraries A, B, and C become tightly coupled thanks to this issue. Circular dependencies can also result in other unexpected failures during compilation and program runs.

If you’re designing a dependency graph of packages, libraries, or components, that dependency graph needs to be a directed acyclic graph (DAG). This is known as the Acyclic Dependencies Principle, which is another software design principle.¹³

Microservices vs Monoliths

The software you write eventually has to be deployed into production, so it can be used by other people. For a UI software, the choice is pretty clear: A web client software runs on the user’s browser. A mobile app runs on a mobile device of course. A video game might run on a variety of devices: A PC, console, or again, a mobile device. In either case, it is a single software binary that is running on a single particular device, machine, or web browser.

When it comes to backend software however, things could get a little more complicated.

In a lot of cases, backend software runs as a single binary on a single server, similar to a UI software. These are called monoliths, or monolithic applications. Yet, there are a lot of other cases where the backend software is distributed among many servers. These are multiple software binaries running on separate servers, communicating with each other through network links. Sometimes a software product is distributed among a considerable number of binaries deployed on separate servers, each providing a different service as a part of the whole product. Such deployments are called microservices.

In the past decade or so, there have been many advances in virtualization and containerization technologies. Nowadays it has become more manageable to create multiple containers on the same server hardware, where each container runs a separate server. The hardware is now one more abstraction layer away from the server software, which makes creating and orchestrating multiple server containers relatively easier. (I wouldn’t call this a trivial task though. It still takes a great deal of work and skill.)

These technological advancements made the task of server deployments relatively easier and cheaper. As a resulting side effect of this, building and deploying microservices became more fashionable.

It is a very common theme in humanity’s history that when something becomes fashionable, we get quite carried away with it, taking it to the extremes. That is exactly what happened with Microservices. And Monolith ended up becoming a dirty word.

Do you need to send emails to your users? That could be done in a separate microservice. Do you need a new scheduling/ticketing system for your clients? That’s a separate new microservice. Do you need a new catalog for your products? Yes, you guessed it: A new microservice.

As the software world went crazy with microservices, the engineers responsible with building them have started to realize something: Implementing a monolithic application is hard enough. On the other hand, microservices and their distributed systems bring a whole new set of unique challenges to the table.¹⁴

In a microservice architecture, you have to deploy multiple software binaries to the production servers. Oftentimes, the deployment schedules are not simultaneous. One binary could be pushed to production, followed by another binary hours or sometimes days later. These two servers could be communicating with each other using RPC or a RESTful API, as a part of their regular operations. What if there were changes made to their APIs? Some new fields could be added to the data objects that they send to each other. In such a scenario, both of the servers calling each other must have their binaries updated at the same time, otherwise those remote API calls may not be able to go through. One server might be expecting a certain new field populated in the data object it receives from the other server, while that other server may not be populating that new field if it’s still working with an older binary version. The first server’s software might end up crashing in this scenario, depending on how its algorithm is implemented.

Let’s say you managed to synchronize the deployment schedules of these two dependent servers. There could still be issues. Now let’s say in the same deployment with the API changes, an unrelated bug was accidentally introduced to the binary of one of the servers. The bug was not caught during testing unfortunately, so it ended up in the production binary. (It happens. In some dysfunctional organizations, it happens way too frequently.) Now you have to roll back the binary of that server to its previous version. But what about the other server with the API changes? When you roll back server 1, server 2 is still going to have the new binary running with the updated API. The aforementioned software crash might end up happening, due to the version mismatch between the servers’ binaries.

You might then say, ok, I will roll back both of the servers simultaneously then. But what if there were more than two servers with multiple simultaneous API changes? Things would get a lot more hairy. More servers to deploy and maybe rollback simultaneously. You get the picture.

With microservices, you need to take extra care writing the software for each server, making sure it could still work with the former versions of its API. So that if another interacting server gets its software binary rolled back to a previous version, there are not going to be any crashes.

With a monolithic server, none of these issues would arise. There would be only one server to deploy or rollback if needed in this case. Writing software for a monolithic server is much easier and straightforward.

Microservice architectures could have other, even more serious issues, particularly when it comes to data coherency during distributed database transactions.¹⁵ Let’s say two different servers are required to make two separate writes to their respective databases, or maybe even to the same remote database. (It wouldn’t matter in this scenario.) And let’s say these two write operations are required to be done atomically. This means both of the writes have to go through together, as a part of the same transaction. This could be due to many different reasons, for many different kinds of applications. For instance, it could be a bank transaction. The two servers might have access to two separate bank accounts. When a certain money amount is subtracted from one bank account, that exact same account has to be added to the other bank account, within the same transaction.

In a distributed system like a microservices architecture, it is extremely difficult to guarantee the atomicity of a database transaction. In our example, let’s say one server ended up subtracting from one bank account before the other server was able to add to the other bank account. Now let’s say there were read operations on the database(s) for these two particular accounts, and the read operations went through between those two write operations. In a distributed system, the remote operations could be arriving in any random order to its recipient. The read operations would show that the first account was missing some money account while the second account had no changes (since its write operation did not go through before its read operation). The entity doing the reads would think that some money could be missing from the bank.

A monolithic server on the other hand could be dealing with only one database. With a single database and a single server application operating on it, it is very much possible to do a successful database transaction that involves multiple operations. A transaction with two writes to two different accounts on the same database would go smoothly without a hitch, every single time, since the transaction is originating from a single server.

When you need atomic transactions for multiple database operations, monolithic servers are the way to go.

This is the lesson that I’m trying to convey here: Monolith should not be a dirty word. Monoliths have their place, as well as microservices. There should be a balance to how services should be distributed among multiple servers.

If your team is starting a new project, there is nothing wrong with starting a project as a monolith. There is even nothing wrong with keeping it as a monolith in the future. And if you think that you need to transition into a microservice architecture in the future, you can keep your software architecture modular and flexible as you design your software system. Separate library packages of today can turn into separate server binaries of tomorrow.

If it’s a large project in a sizable organization, it could make sense to separate the project into microservices. In this case, it would make sense to split the services among the team boundaries. Each team should work on the same particular service. This would make the regular ongoing deployment and on-call schedules much easier to arrange, among other things.

And as I mentioned, definitely do try to place the complex database transactions all on the same service if you can afford to. I have heard of many organizations large and small who try to tackle distributed database transactions, and end up spending lots of engineering hours trying to deal with the unavoidable subsequent issues. The headache is not worth it in my opinion.

Final Words on Software Best Practices

The Software Development Best Practices listed here are some of the ones that I thought were very important. There are many more. I wrote down what I could think of in a single chapter in this book, but you could fill entire books with them. Software engineering, including the engineering of complex and very large-scale systems, is a skill-set that is learned through an entire career. And the learning never stops. To this day, I am still learning of better ways to architect, design, and develop software.

These Best Practices are mostly geared towards developing clean and understandable software code and architecture. There are other, very important and critical best practices, such as software test development and CI/CD DevOps processes. Again, entire books can be (and have been) dedicated to these subjects. I am going to go over these in the upcoming chapters. Test development has a special place in my heart, as I spent a considerable portion of my career developing software test frameworks. Therefore, I dedicated an entire chapter to Test development alone. In my opinion, each engineer must be responsible for developing the test code (for both unit and integration tests) along with their production code. Indeed, under most circumstances, most of the developed code must consist of test code, even more than the production code itself. Anything less would be adding to the overall technical debt of the software system.

An engineer might think “Oh well, I just need to write this code here very fast. It is not going to be around for too long anyway, since we’re going to replace it with something else fairly soon.” There is a good chance that this engineer and their team can move onto some other high priority projects, and that code keeps staying around. Then other engineers start to build on top of that hastily developed low-quality code, and you can guess where all this goes. Over time, such decisions and their resulting bad code keep accumulating. Building on top of badly designed software is like building a skyscraper with a cracked foundation. It will inevitably collapse at some point. At best, such bad software is going to turn into “legacy code” which is impossible to change. It will just keep on running, but no one in the organization will be allowed to touch it or change it in any way. If any engineer attempts to touch that codebase, they will tell that engineer things like “that system runs on prayer, no one understands how exactly it works, do not touch it under any circumstances!” At worst, the system is going to become completely dysfunctional, the project is going to be canceled, and the organization is going to die off. In the very worst case, your bad software is going to actually end up killing someone.

It is one of the unchanging physical laws of the universe: Entropy always increases. By entropy, physicists mostly mean the state of disorder and randomness of atoms and molecules. The Second Law of Thermodynamics says the disorder state of particles always increases. The same law also applies to information theory and telecommunication signals. It seems like the same law applies to software engineering as well. Software always seems to grow in complexity and disorder. If not enough care is given, in time all software, even the very good ones, will become more disorderly and nonsensical. It will all turn into a total mess.

Everyone in the organization, from the engineer working on the code, to the PMs and managers associated with the project, all the way to the executives, should be aware of this fact. They need to care for the health of the software in their organization. They need to allocate enough priority to reducing the entropy of the software, reducing the technical debt, and improving the cleanliness of the code and the architecture. A good percentage of the quarterly objectives in the organization, such as 25-30%, should be allocated to improving the software quality and reducing the technical debt on a continuous basis.

Here is another analogy: Technical debt is like credit card debt. You can put off paying it back in the short term and try to carry on with it. But in the long term, you’re going to have to pay it back with interest. And in the very long term, the high interest payments are going to destroy you.

An ongoing commitment to software quality helps in another way: Even if you cannot come up with a really great software design from the very beginning, you are going to have other chances to improve your design in the future. Oftentimes the design choices become clearer as you start writing the software anyway. There is no need to come up with the “most perfect design” from the very beginning.

Regardless, it helps a great deal to spend some time thinking at the beginning of a project and get the major aspects of the design correct. Making major changes to the design of your project later on is going to be costly, from whichever angle you look at it. There is a balance to determining what is a major design change and what is a minor one. Determining this balance correctly can only come from experience.

A decent software design can go a really long way. On the other hand, a bad design can haunt your system months or years down the line, unless it’s properly addressed and fixed continuously.

«prev          next»

1. Booch, Grady. Object-oriented analysis and design with applications. Benjamin/Cummings Publishing Company, 1994.

2. McConnell, Steve. Code complete. Microsoft Press, 2004.

3. Brooks, Frederick P. The Mythical Man-Month: Essays on Software Engineering. Addison-Wesley Publishing Company, 1975.

4. Hunt, Andrew, and David Thomas. The Pragmatic Programmer. Addison-Wesley, 2000.

5. Fowler, Chad. The Passionate Programmer: Creating a Remarkable Career in Software Development. Pragmatic Bookshelf, 2009.

6. Bloch, Joshua. Effective Java. Addison-Wesley, 2018.

7. Fowler, Martin, and Kent Beck. Refactoring. Addison-Wesley, 1999.

8. Gamma, Erich, et al. Design Patterns. Edited by Erich Gamma, Addison-Wesley, 1995.

9. Martin, Robert C. Clean Code: A Handbook of Agile Software Craftsmanship. Edited by Robert C. Martin, Prentice Hall, 2009.

10. Martin, Robert C. Clean Architecture: A Craftsman's Guide to Software Structure and Design. Prentice Hall, 2018.

11. See footnote 10.

12. Liskov, Barbara H., and Jeannette M. Wing. “A behavioral notion of subtyping.” ACM Transactions on Programming Languages and Systems, vol. 16, no. 6, pp. 1811-1841. 10.1145/197320.197383.

13. See footnote 10.

14. Kleppmann, Martin. Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems. O'Reilly Media, 2017.

15. See footnote 14.