Introduction
Creating good, successful software is hard - very hard.
For every programmer, it is therefore important to know, understand and apply fundamental software development pragmatics - practical advice and rules that have proved their usefulness and help us to create the best possible software in the shortest possible time.
In this article, I have tried to assemble a set of basic pragmatics which I consider to be the most relevant ones. If you are an experienced programmer, then you might be familiar with most/all of them. If you miss some advice, then please share it with us by leaving a comment.
Please note:
- The pragmatics in this article are about designing and writing code. There are other important aspects for software development projects to be successful (such as good user interfaces, dealing with people, etc.), but they are out of the scope of this document.
- The following pragmatics only cover basic principles that are meant to be generally applicable. There is no advice for specific programming environments (programming languages, libraries, tools, and architectures).
The tips are divided into three categories:
- General Guidelines
- Data Design
- Writing Code (will be published in part 2 of this article)
Let’s get started.
General Guidelines
Everything should be as simple as possible, but not simpler! - Albert Einstein
Simple tools and concepts can be understood quickly, are easy to use, less error-prone and make us more productive.
We all like simplicity. Simplicity makes work and life more enjoyable.
However, we must be aware of oversimplification, as indicated at the end of Einstein’s beautiful quote.
Therefore: Keep it simple, but not simplistic!
Many famous people advocate simplicity. Here are a few examples:
Quote:
Simplicity is prerequisite for reliability.
Simplicity and clarity … decide between success and failure.
— Edsger W. Dijkstra
Dutch computer scientist; author of 'Go To Statement Considered Harmful'
Quote:
Controlling complexity is the essence of computer programming.
— Brian W. Kernighan
Computer scientist and writer; co-developer of Unix
Quote:
Simplicity is the ultimate sophistication.
— Unknown
Quote:
Simplicity transforms ordinary into amazing.
— Scott Adams
creator of Dilbert
Quote:
If you can’t explain it simply, you don’t understand it well enough.
— Albert Einstein
Physicist; genius
Quote:
Truth is ever to be found in simplicity, and not in the multiplicity and confusion of things.
— Isaac Newton
Mathematician; astronomer; theologian; author and physicist
If It’s doomed to fail, then Fail fast!
Most software projects fail. This is a sad and undeniable fact.
If a project is deemed to fail, then it should fail as fast as possible, in order to limit the damages and free up time and resources for other projects that will (hopefully) not fail.
Failing fast in this context means that problems should be detected and dealt with as soon as possible. The longer we wait to address the problem, the more time, energy and resources will be wasted. The cumulative loss increases exponentially with time.
For example, correcting a design flaw in the design phase is easy and cheap. But once the software is in production and used by many people, it is often very expensive and frustrating to fix the error.
Quote:
The sooner we 'fail' and the faster we learn, the greater the chances for success. Failing early saves you time and money.
— Demian Borba
product manager at Adobe
Quote:
Test fast, fail fast, adjust fast.
— Tom Peters
author of the bestseller book 'In Search of Excellence'
Strive for 'good enough', not for 'perfect'. Then release!
Creating perfect software (no bugs, all features fully implemented, optimal user interface, excellent documentation, etc.) is extremely time-consuming and expensive, unless we are working on a very small project. In most cases, it is impossible to achieve perfection because of practical limitations.
Even the major players in the software industry with the best developers and dream-budgets don’t write perfect software. That’s why they constantly provide patches and new versions.
Therefore, proceed like this:
- Set goals and priorities
- Create a prototype
- Deliver 'good enough' software
- Improve continuously (see the next item)
|
|
|
|
| Prototype
| Good enough
| Better
|
Quote:
You cannot write 100% perfect code. Even if you do, in 6 months time it will not be perfect.
— Chris Maunder
Co-founder of CodeProject
Quote:
No one in the brief history of computing has ever written a piece of perfect software.
— Andy Hunt
Writer; co-author of 'The Pragmatic Programmer'
Quote:
90% of the functionality delivered now is better than 100% of it delivered never.
— Brian W. Kernighan
Computer scientist and writer; co-developer of Unix
Quote:
Searching for perfect solutions often will lead to stagnation and frustration. Perseverance, tolerance for less than perfection, the pursuit of improvement, and commitment to doing the very best you can, all are healthy, and most likely to yield the best results.
— Albert Ellis
Psychotherapist
Listen to the users!
- Software developers can’t anticipate everything the users really want.
- Users often don’t know exactly what they want, unless they have used the software for some time.
- The users' satisfaction is a determinant factor for the software’s success.
We want happy users. Therefore the best approach is an iterative one that goes like this:
Quote:
After spending a lot of time with trying out some marketing tricks he [Joel Spolsky, co-founder of Stack Exchange] concludes (after 5 years): Nothing works better than just improving your product. Make great software that people want and improve it constantly. Talk to your customers (users) and listen. Find out what they need.
— book 'Coders at work'
Quote:
... we build prototypes for every major product feature. We test with pre-release users and key customers very early on and definitely before we get to implementation. And yes, we do 'fail' a lot, which is great! It’s great because we learn a ton in the process and minimize the risk of failure in the long run. In the end, we tackle real customer needs, with empathy, in an innovative way.
— Demian Borba
product manager at Adobe
Quote:
The most important property of a program is whether it accomplishes the intention of its user.
— Tony Hoare
Computer scientist; ACM Turing Award 1980
Data Design
Before writing code, design your data carefully!
Whenever you create an application, start by carefully designing the data structures and their relationships. Do this before writing code.
Well designed data structures lead to simpler and more maintainable code, less bugs, better performance and less memory consumption.
The difference can be striking.
Quote:
Study after study shows that the very best designers produce [data] structures that are faster, smaller, simpler, clearer, and produced with less effort. The differences between the great and the average approach an order of magnitude.
— Fred Brooks
book 'No Silver Bullet'
Quote:
Show me your flowcharts (code) and conceal your tables (data structures), and I shall continue to be mystified. Show me your tables (data structures), and I won’t usually need your flowcharts (code); they’ll be obvious.
— Fred Brooks
book 'The Mythical Man-Month'
Quote:
Rule of Representation: Fold knowledge into data so program logic can be stupid and robust.
Data dominates. If you’ve chosen the right data structures and organized things well, the algorithms will almost always be self-evident. Data structures, not algorithms, are central to programming
— The Art of Unix Programming
Quote:
I’m a huge proponent of designing your code around the data, rather than the other way around.
Bad programmers worry about the code. Good programmers worry about data structures and their relationships.
— Linus Torvalds
creator of Linux
Quote:
If you get the data structures and their invariants right, most of the code will just write itself.
— Peter Deutsch
book 'Coders at work
Make all data structures immutable, unless there is a good reason to make them mutable!
Immutable data structures are simpler to understand, simpler to use and less error-prone because:
- Once created, the state doesn’t change anymore. There are no state transitions, no temporary invalid states and there is no need for synchronisation or locking or defensive copying.
- Immutable data can be freely shared in concurrent/parallel computing environments - there is no risk of corrupted data, deadlocks or other nasty problems that can be very difficult to tackle down and fix.
- Computed results based on immutable data can easily be cached for better performance.
However, immutable data structures are not always the best choice. For example, cloning a whole structure for every single change can be very costly (in time and space). In some cases (e.g. game or GUI applications) mutable data structures are better suited.
Moreover, if two objects need to directly refer to each other, then the objects must be mutable. Examples would be a parent and child node in a tree referring directly to each other, mutual friends (A points to B, and B points to A), etc. If immutability must be preserved in such a case, then a possible solution would be to have an additional data structure that describes the relationships, such as a set of tuples that represent a pair of objects related to each other.
Limit the set of allowed values to the smallest possible one!
Limiting the set of allowed values for a data type:
-
documents and helps to understand the data type
-
eliminates the risk of misbehavior or severe failures due to wrong values
-
simplifies the code that has to deal with the data
For example, consider the case of a name field for data type employee. By allowing any string to be stored in name , the following can happen:
-
A long string can lead to a buffer overflow (depending on the programming language), memory exhaustion, or other software failures.
-
Characters that are invalid in a name, but used as valid symbols in Javascript and SQL (such as <, > and ") can open the door for attacks such as SQL and/or script injections.
-
An empty or null string can lead to bugs if the code doesn’t explicitly handle these values correctly.
To avoid these risks, the name field should be constrained. For example, the following simple regular expression eliminates all above mentioned problems:
[a-zA-Z ]{1,70}
This regex limits the name to a maximum of 70 characters, requires at least one character and allows only letters and spaces. Note, however, that the above regex would be over-simplified and not suitable in real-life applications that must allow names containing hyphens, apostrophes, etc. See comment below with more information from Chad3F.
Protecting data against invalid values (especially in case of data read from external sources) is often cited as the most important rule for writing secure software. (for example, see OWASP Top 10 Critical Web App Vulnerabilities and Top 10 Secure Coding Practices)
Example of an attempt for SQL injection in a data entry form:
Example of a JavaScript injection attempt:
Most of the time, a default value should be the strictest among the set of allowed values.
By choosing the strictest possible value as default value, we are always on the safe side.
More permissive values should need to be explicitly stated (in the code, in configuration files, etc.).
-
A function that writes to a file should, by default, not overwrite an existing file.
-
All compiler warnings should be enabled by default.
-
If a new user is added in a multi-user application then the lowest privileges should be granted by default.
The last example demonstrates that strict default values are not always the best choice. They can be annoying. If the multi-user application is only used by a single person on his/her PC, then the user obviously wants to have full rights by default. Hence, the best default value sometimes depends on several factors.
Example of a strict default value in a GUI:
Avoid data redundancy!
Storing copies of mutable data at different locations is error-prone, laborious and expensive because:
-
In case of data changes there is a risk of not updating all locations - due to forgetfulness, technical problems, security concerns, etc. This can lead to data inconsistencies and corruption, and can ultimately result in severe software malfunctions.
-
Sometimes a locking/synchronisation mechanism needs to be implemented and activated for every data modification (create, update, and delete operations), in order to avoid accessing invalid data during the data modification process. Implementing these mechanisms can be very tricky and error-prone.
-
Additional memory is needed to store the copies.
Banal example: A salesman misses an important appointment. Reason: He entered the appointment into his agenda on his PC, but forgot to synchronize the data with another agenda on his mobile phone. If the data for both agendas were stored in one place (e.g. in the cloud), he wouldn’t have missed his appointment.
Consider storing data in simple text files using universal, standardized formats!
Using text files as storage media has many advantages:
-
Text files are fully supported by all operating systems. There is no need to install and configure additional software such as a database server.
-
Text files can easily be read and manipulated by humans. This is very convenient for debugging purposes.
-
They can also easily be read and manipulated by many third-party applications (written in any programming language) and tools. For example, Unix provides many useful tools for text manipulation, such as grep, awk, sed, etc.
-
Using standard formats such as JSON, XML and CSV enables the data to be queried, sorted, filtered, searched, printed and converted by many existing applications. For example, a CSV file can easily be used in a spreadsheet application.
However, text files also have severe limitations, especially in cases of big data. Complex queries (with filters and joins), update and delete operations, transactional processing, data encryption and other functionalities might need to be implemented manually and can be very inefficient. When it comes to big data then storing them in a database is often the only viable choice.
Moreover, storing data as characters (instead of bits) can consume much more space and time. Therefore, binary data are sometime unavoidable.
A nice example of using text to represent graphics is SVG (Scalar Vector Graphics).
Here is a simple example of an SVG file (SVG_example.svg):
<?xml version="1.0" encoding="UTF-8"?>
<svg xmlns="http://www.w3.org/2000/svg"
width="140"
height="140" >
<circle cx="70" cy="70" r="40" stroke="red" stroke-width="4" fill="yellow" />
</svg>
Opening the file (for example with your web browser) displays the following image:
Quote:
Write programs to handle text streams, because that is a universal interface.
— Doug McIlroy
Inventor of Unix pipes
Quote:
Unix tradition strongly encourages writing programs that read and write simple, textual, stream-oriented, device-independent formats.
— Eric Steven Raymond
The Art of Unix Programming
Writing Code
Please continue with part 2.
