Crafting Software: Introduction

Background

I'm a software development team lead and I often find it necessary to explain what we devs are doing in context of the bigger picture. This is my thought-experiment about how I might go about explaining all the things we do to newer devs.

Blog A Book : Crafting Software

I figured a nice way to capture the lessons learned over the 25 years of my career was to take the challenge of writing it all up as a book. This is my shot at that and I've decided to blog the book before I officially self-publish the entire book.

This is the introduction and it is way too long, but I've created some natural breaks with some images in case you want to read it in a few different sessions.

I’ve worked in IT for twenty-five years, most of them as a Software Developer. I’ve noticed that most beginning Software Developers (and many experienced ones) think their job is to produce code. However, in reality, one of the most valuable things you might do as a developer is to warn against the creation of code.

There’s not much guidance for Software Developers on where their roles begin and end, but growth means that your role must broaden (into a Senior Software Developer, Software Architect or Solution Designer). However, the lines between roles are blurry making it difficult for companies and individuals to determine where any particular Software Developer stands.

Purpose Of This Book

Because the roles are blurry there is often a lot of disagreement about what Software Developers do and how they do it. Is there any process to software development? Some companies lean toward the belief that software development is an organic thing and developers are artists who need creative freedom. Just start hacking away and see what we get. Other companies believe developers are assembly line workers who should sit in cubicles and write the code they are told to write. The truth is probably somewhere in the middle.

A Flexible Process Which Provides Guidance

What we need is a flexible process which can provide guidance while not crushing creativity. This process can provide a loose working structure that can be used by individual developers even if no one else in the company is adhering to a particular methodology. By the time you finish reading this introduction you’ll have a better understanding of the process I propose. I believe (and hope) you’ll see the great benefit this loose structure can bring to the software development process.

Even if you decide not to continue further into the book, I hope you’ll find this introduction an interesting discussion that brings a new angle to software development as a process. To start our discussion, let’s go way back to the basics.

What Do Software Developers Do?

I believe we need a simple, single line definition of what software developers do. This definition will serve as a foundation we will continually return to in this book to determine where we are at.

First Tenet of Software Development

 

Software developers write code to turn manual processes into automated processes.

That’s really all we do.

You may think that is too simple to cover all software. However, if you consider any software, even a game, from an abstract viewpoint, you’ll find that it is really just a manual process that has been turned into an automated process.

Pac-Man As A Manual Process

Take a look at Pac-Man video game. Originally, Pac-Man could’ve been a board game where players roll dice to move Pac-Man and the ghosts. The player could decide the direction for Pac-Man and you could spin a spinner to get the direction for each ghost movement. As you move the Pac-Man along the path on the board you would simply tally up how many dots he eats with each roll of the die. Next, you’d roll the dice for each ghost and spin the spinner to indicate movement direction. Each turn the ghosts could possibly land where the Pac-Man token is and destroy him.

Of course, the object would be to get your Pac-Man to eat all of the dots before he is eaten by the ghosts. (See https://en.wikipedia.org/wiki/Pac-Man^ for more on the Pac-Man video game.)

Now that we know those things, it would be far easier to write the video game. The video game simply automates this process. The code is “rolling the dice” and “spinning the spinner” to move the ghosts behind the scenes. It’s also updating the location of the ghosts each time making it seem as if they are moving. You see? It all really is just automation. However, you’ve learned another important axiom related to software development.

Second Tenet of Software Development

Second Tenet of Software Dev:

Everything the program is supposed to do must be known by the software developer.

Just as the designer of the board game has to know exactly how the rules work so she can create the game, so too the software developer must know what the program will do so she can create the code.

Immediately, A Problem Arises

Here’s the problem. A lot of what we just talked about in the description of the Pac-Man board game seems obvious. It’s the nature of how human brains work.

“Of course you have to roll dice,” someone says. “How else would you know how far to move Pac-Man and the ghosts each time?” How things should be done may seem obvious to one person on the team. Those requirements that seem so obvious may not even be discussed much. The problem occurs later when the obvious assumptions are actually different in numerous team members’ minds.

Without Detailed Definition, The Developer Does It His Way

Maybe the Pac-Man board game doesn’t use any dice. Maybe the ghosts move X units every time. The point is that without the actual explanation and description the game builder (analogous to software developer) may do it however he decides. However, his way may be completely wrong.

If the designer and the builder are two different people, the builder’s decision may be entirely wrong. Furthermore, unless the design element is described properly it may force the developer (builder) to make a decision that she believes is correct. For example suppose the description for movement is something like the following:

Each turn each ghost will move a random number of units in a random direction.

No maximum movement has been defined so now any ghost may move up to infinite units in a direction. So now, the game is created and Pac-Man moves then a ghost moves infinite units towards Pac-Man and destroys him every time.

That’s ridiculous. I know. But let’s circle back to the original definition of what Software Developers do:

Developers write code to turn manual processes into automated processes.

Automate the Wrong Process

Now we are seeing that if the manual processes are not described properly or are not described at all, the software developer (builder) can automate the wrong process.

That’s why an important part of Crafting Software is : Communication.

Of course, communication is at the root of all business challenges. That's because we never really know what the other person knows or exactly thinks even though we may be communicating directly to him. For Software Development, there are two things we can focus in an effort to make our projects move more smoothly.

Two Core Challenges of Communication

1. Creating a common understanding
2. Capturing the appropriate details

Creating Common Understanding

Why do we have meetings? It's so we can discuss things. Why do we discuss things?

Three Reasons For Discussing Things

The following three things are probably basic drivers for the most meetings.

1. Obtain a baseline of understanding : Determine what others think or know about the subject
2. Get to a common understanding about the subject under discussion
3. Decide on a plan of action - what will we do after the meeting?

Meeting Failure

The problem that often occurs is that a team can generate thousands of words and hundreds of ideas in a short meeting. Often, a stakeholder may take a dev team into a meeting to explain what she wants from the product. Numerous ideas are discussed and people are nodding their heads and the meeting feels like a success. However, no one can actually be sure that they all have the same understanding of what the stakeholder wants. It is likely that they do not have a common understanding.

Were the Salient Details Captured?

Also, consider how most teams capture the details of a meeting. Maybe someone jots down some notes. The stakeholder may pause and ask, "Are the devs getting this so they can create the functionality?"

Very sharp devs who have a lot of experience can probably wing it and get fairly close to some percentage of the desired functionality. The problem is that there are so many details. How do you capture it all?

Other devs are completely disinterested in this entire process.

Ugh! Communication

Designer / creative types (Software Developers, portrait painters, song writers, etc) don’t normally want to talk a lot. That’s because they are busy making stuff. These devs know that talking about the details and gathering the requirements is Business Analyst work.

At times software developers require extremely specific details and creating and gathering this kind of detail can be:

• Difficult to get from business users
• Boring (from a Dev's viewpoint)
• Extremely difficult to track
• Easily forgotten
• Did I mention boring? :)

For all of these reasons (and more) Software Developers try to ignore these things. After all, software developers want to be about the business of writing code. However, writing the wrong code isn’t helpful, even though it can feel like progress. "Hey, I wrote some code that does some stuff."

You Can’t Wait Until You Have Everything

The counterpoint is that you cannot wait until you have everything before you start coding, because you will never have everything.

Again, let’s circle back to my second tenet of software development:

Second Tenet of Software Development

Everything the program is supposed to do must be known by the software developer.

As a software developer, you cannot just start typing code. Well, you can. And, unfortunately, many software developers (and dev groups) do. But it is generally a terrible idea. I mean if you literally know nothing about the project and you’re already typing code then you have little chance of success. You must have at least the germ of an idea of what you are attempting to create.

Remember, software devs must write code which creates software that :

• Fulfills stakeholders' expectations

But, as we said you will never actually know everything the software will do before you begin to build. Isn’t this all just a paradox? I admit that it does feel that way, but there is actually a way out of this swamp.

The Team Needs a Process

Everything I’ve explained so far is why you must have:

1. A process (method)
2. Structured yet flexible design which allows you to change things as you go

Those two little items look so short on the page but there are entire books within each of them. Many of you with more experience in the IT industry may have bristled when you read the word method. Please notice that the word is not capitalized and it is not intended to invoke any defined Methodology (ISO 9000, Six Sigma, Agile Scrum).

Jack Ganssle, Embedded Systems Design (September, 2003):

One way to identify an amateur organization of any sort, be they accountants, lawyers, craft shops, or software developers, is a lack of process.

I’m talking about a much more basic meaning of the word method.

Dictionary.com defines method, perfectly or our purposes:

Dictionary.com:

“a procedure, technique, or way of doing something, especially in accordance with a definite plan”

For our purposes, in Crafting Software, we will alter that definition slightly to show the tight focus we’ll have in this book:

Crafting Software - methodology

A way of developing a system (software) guided by an appropriate plan.

A Process That Solves Both Problems

At the beginning of this article I mentioned two problems:

1. Creating a common understanding
2. Capturing appropriate details

Crafting Software guides you into a process that will help you resolve both of these challenges by showing you effective ways to capture the appropriate details using specific artifacts and user stories. Chapter 1 will make all of this clear.

There are three things you can do to instantly make your communication more effective:

1. use a common language - every role must use the same terms to talk about the desired business solution ( no tech jargon)
2. focus on nouns (the things you want in the system)
3. focus on verbs (the functionality you want from the system)

As we move through Crafting Software we will discuss these in far more detail.

Hit Your Target

If your devs do not have the same understanding that the stakeholder has for the product, you will never hit your target. If you have not captured the appropriate details in a manner that builds a common understanding of the expected result, you will never hit your target.

Again, that is all based upon the Second Tenet of Software Development:

Second Tenet of Software Dev:

Everything the program is supposed to do must be known by the software developer.

Crafting Software helps create a process that creates guidance, like a map.

Software Development Map

Crafting Software will provide a map which will allow you to create an appropriate plan for developing your specific system.

Isn’t a Map Too Limiting?

I believe the the word map is quite appropriate in our context. Consider a geographic map. It provides a set of specific details but it does not tell you where to go. Nor does the map tell you how to get to your destination. Instead, it lays out the current situation as a simpler abstraction of the reality that exists.

The map user decides upon the destination and the path to the destination.

The map provides guidelines but it does not enforce a specific route. The map may indicate routes that are simpler (paved roads) or it may indicate areas that are impassable by automobile (mountain cliffs). However, if a map user decides that he will take a parachute and jump off the mountain cliffs and parachute into the town below, the map does not forbid it.

That’s exactly how Crafting Software will work. It will provide you with guideposts along the way that will show you the easiest way -- the smoothest route for most travelers. However, there will be some paths that you and your team may decide not to take. There are many times you just want to take the shortcut and it makes sense. But, you’ll consciously know you took the shortcut and be able to explain why you did so.

Crafting Software Will Provide the Map

Crafting Software will provide the larger map as guidance. However, each reader will hash out her own and exact route, deciding on the exact artifacts she will use to describe how the software will be created and which decisions were made along the way.

There are portions of the map that every software development team will use.

Examples of those items are things such as:

• database schemas
• class diagrams
• requirements documents (made up of user stories)

Please don’t get too caught up on specific types of artifacts at this time because if you do you may think I’m just talking about doing a lot of documentation and that I’m just disguising some other Methodology and it’s all the same and why bother.

Unnecessary Documentation Kills Creativity

That’s not what I’m talking about at all and I believe you’ll agree by the time you complete this introductory chapter. I am a software developer first and I hate unnecessary documentation too. It kills creativity.

What Do Software Developers Really Need?

There are a few things that every software developer needs.

1. A basic starting point - what are the basics of what we are supposed to build?
2. A way forward - how should the code be created?
3. A way to continue - A basic process for the daily work the software developer will do
4. A way backward - what did we actually do to create the software?

• • Examples of things you might need to know as you look backward:
    • Configuration - so you can deploy
    • OOP Design -- so you can add enhancements, or more easily determine why a bug is occurring
    • DB Schema -- so you know where data is stored, for examination
    • Etc.

Crafting Software will focus on fulfilling these to guide the software developer into always knowing where she is in the overall System Development Life Cycle and what she should be focusing on.

Now, let’s close the loop on the other item on our two item list (structured yet flexible design) and talk about what it means to us in Crafting Software.

Structured Yet Flexible Design

This special item described here as structured yet flexible design provides the way forward for software developers.

Software Design : Large & Growing Body of Knowledge

The ability to create a structured yet flexible design pushes a developer to learn a specific set of concepts which comes from a large and growing body of knowledge.

Crafting Software will have a strong focus on learning OOP principles and design patterns and more, but always with a focus of why we might choose one design over another. We will always focus on applying designs as a way forward in creating working software. We will only move forward with designs after we understand why we’ve chosen a specific way forward while understanding the trade-offs we’ve chosen.

The things we will learn about in Crafting Software will help you as you create software systems of every kind.

Here’s a list of some of the things we will cover (there will be far more in the completed book):

• Requirements: What Do They Mean to Developers?
  • I thought the BA (Business Analyst) would provide these. (Insert maniacal laughter of experienced software developer here.)
• How can I get people to tell me what they really want?
• Is there an easy way to get a summary of the system?
• How can I easily track requirements (since no one else is)?
• Real Object Oriented Programming (OOP)
  • OOP Is For Humans, Processors Don’t Care (OOP is a way to organize your code)
  • SOLID Principles
• The Power of Flexible Design Using Interfaces
• Writing Less Code Means You’re Advancing
• Read Your Code Like a Book - readable source means less docs.
• Code Reuse: The Holy Grail!
• Handle Errors Because You Know About Them
• TDD, BDD - Building quality into the design
  • Code coverage - touch all your code & know what hasn’t been touched
• Patterns of Software Development
  • New ways to think about patterns -- so many devs think of these as some kind of magic written about in the GoF (Gang of Four Design Patterns book). Here you will learn the why of patterns.
  • Reusable OOP structures which help you quickly solve problems and communicate solutions to other developers MVC - Heavy emphasis on Separation of Concerns (SoC) in modern software
• Real Software Design
  • The real way forward : decomposing requirements (user stories) into domain model classes
  • Sticky note method: nouns and verbs
• Database As A Developer
  • What is the point of a relational database? Answer: Structure
  • Basic DB Design, Connection, etc
  • Repository Pattern
  • Tools to make you faster
    • Roundhouse
    • Localdb (Microsoft “file” database)
    • Special section on Sqlite
• Version Control (Mercurial or Git)
• VCS (Version Control System) is non-negotiable for professional devs It’s not overhead, good VCS is a developer’s best friend
• Enterprise Patterns
  • Reusable system structures which help you solve Enterprise challenges
• Integrating solutions - One of the main challenges is that data is stored in disparate formats and yet the Enterprise needs to share certain data elements with various systems
• Deliverables / Milestones
• Working software (Minimum Viable Product (MVP))
• continuous delivery and the necessity of tooling (build & deploy)
• Appropriate UML Docs
  • You will have to explain things to other people in various roles. This is how you choose the appropriate document to show the person.
  • Highest level of abstraction first, drill down into detail only as necessary - speeds up communication, makes meetings faster!
  • You must use the language of the business domain. You must get everyone to use the same language (ubiquitous language) so that everyone can talk about things the same way.
  • These UML docs will help create our Software Development Map
  • Best software for creating UML - Enterprise Architect -- non-negotiable if you’re a professional dev
    • Also provides nice code reverse engineering and code generating
  • Every UML document explained with a why -- why you should produce the document
    • A large portion of developers do not know why they create these documents. After you read Crafting Software you will be able to produce and explain every UML document covered and defend your reasons for not creating specific documents should you decide not to.

How will I show you all of those things I’ve just listed? Will Crafting Software simply provide dry descriptions of what you would do if you were developing software? No.

Crafting Software : Design and Code Real Apps

As you read through Crafting Software we will be designing and writing actual code which I will provide on github. All of these applications will be linked in some way. In other words, they will share data and be available via desktop apps and also accessible via web and/or Android and iPhone/iPad apps. This will allow us to see the software dev map in action and also see how certain components (OOP structures) can be reused across language and platform boundaries very easily. That’s the power of process and great OOP.

Now, let’s wrap this introduction up.

Was This Some Kind of Agile Trick?

Earlier I told you that this book is not based upon some giant Methodology. However, I have found after a lot of reading about best practices and numerous methodologies I am aligned in all of my foundational process philosophy with the original principles of the Agile Manifesto. Before you balk and think I’m trying to force Agile down your throat, please read the twelve simple principles from the Agile Manifesto: http://agilemanifesto.org/principles.html^

They are truly fantastic because they are simple. They are short. They are not wrapped up in some five hundred page book. The reason I mention them is because Crafting Software is focused heavily on one specific principle from that manifesto. It’s the one that all Software Developers should care about the most.

Crafting Software: The Focus Is Always On Working Software

The Crafting Software method will always focus on what helps us to deliver working software.

That’s one of the twelve original Agile Manifesto Principles:

Agile Principle

Working software is the primary measure of progress.

As we move through the book, we will always focus on the MVP (Minimum Viable Product) and working software. I’m hoping that will help make Crafting Software a lively read since readers will always have their focus on developing a working software system.

We've covered a lot here because a Enterprise Software Development touches a lot of areas. I've attempted to create discussion that will make you think a bit differently about process and hopefully that will make you want to read more. If so, let's jump into the first chapter where we'll begin at the beginning: requirements. I promise it'll be a fast chapter and it'll make you think about requirements in a different way.

History

First published : 2017-05-03 -- originally published at my site : http://raddev.us/CraftSW/post/introduction-what-s-it-all-about^