Why CRUD might be what they want, but may not be what they need | Ian Cooper


A brief history of CRUD

The Rolling Stones once sung:

“You can’t always get what you want, But if you try sometimes, well you just might find, You get what you need”

I think the same is true of CRUD. Our users seem to ask for it and we blindly agree to their wishes and build it; however, its regularly not what they need.

The File Store

When computers first arrived the early commercial use was to try and remove the need for paper files. Some of us may be old enough to remember that offices has filing departments with staff, where when we began a piece of work we went to get the records that the company had on that customer, or policy, or project. There were problems with this approach: file stores took up a lot of space, and the mangement of paper based records, particulary archival material is expensive; often, someone else had the file out, buried on their desk, and it was not always clear who – an early form of pessimistic concurrency control – which often stopped you responding to requests; files were only as useful as the discipline of maintaing them, and keeping them in order – searching was difficult within a file if it was not ordered and there was no idea of seaching the file store itself.

So it was natural when we began to think about ways in which computers could help in the workplace to think about electronic filing systems. We would record the data around the customer, policy, or project on an electronic form instead of a paper one and the computer could store it for us. We saved paper, we saved staff, we saved time, and we added new functionality such as the ability to search.

The UIs that we designed were optimized to enter data. After all there was a lot of data to enter just to catch up on. The essential questions for UX design became tab order or number of clicks – just how fast could we enter data. We wanted bulk data entry grids, and forms that mirrored our paper based forms, because that was what we needed to get out of the physical filing store and into our electronic store

Throughout the paper told us what to do and the process was built around the form. The form was passed from person to person. Stickers attached to the form, or boxes on the file defined the workflow. The paper-trail was the business process. Indeed, early analysis techniques simply followed the paper trail. Initial systems just followed the existing paper based process. Someone took the paper, the insurance application form, and type it in to the terminal. Then a batch run moved the record to the next step, so the next person in line could perform the same step that they had at the computer.

But no one really thought of this workflow as something the computer could handle. We just did handoffs the same way we did with paper. Indeed we sometimes used paper as the ‘token’ that we passed around for the workflow and told you that you needed to enter the system and do some work. Or, later we became more sophisticated, and used email to do the same thing.

And for this perspective – the electronic filing store – Create, Read, Update, and Delete (CRUD) was the paradigm we needed. It described the lifecycle of a paper file perfectly and with our goal to automate that physical file store it was a capable model – or so it seemed.

Brave New World

Users tend to create electronic documents today. There is less paper. Inputs come from phone, email, the consumer directly accessing a web-based interface. Removing the phsyical file store is no longer behind the business case to provide IT systems. We no longer want to remove the old filing room and its associated clerks. Of course we might want to move away from our new shadow IT world of documents on shared drives and Excel spreadsheets.

The business goals around automation have changed too. To obtain the savings that justify the cost of IT savings, we want to enable smaller numbers of people to carry out the tasks formally done by many through not just electronic filing but through business process automation.

But once we shift the goals away from data entry to process automation, the shape of our architecture changes too. Less of the value comes from data entry today. It comes from providing a key business differentiator that allows us to outperform the competition in service, control of costs, management of supply chain, enabling new business models through disintermediation etc.

CRUD – a model for removing the old file store is less useful as a paradigm when we come to think about analysing and architecting a response to this kind of business process.

Think about how we force users to interact under a CRUD paradigm. They have procedures written in a manual, a checklist for the process. They have to edit this page,then edit that page etc. The system consists of CRUD screens where the users do the data entry. People run their workflow using email. Business process is the preserve of a priesthood who maintain their position by understanding ‘how things are done here’.

The problem here is that the behavior is not captured in the system, it is captured in the manual at best, more likely in the oral transmission of process from one worker to another. The business understands that it has workflow and behavior – they defined process with copies of memos, forms etc. We have not removed those steps by automating them, we have just  given them electronic forms. Pejoratively we are little better than the shadow IT offering written in Access; why engage an IT expert at all for that, all you do is add cost.

Much of the issue here is sheer laziness on our part. When I see tweets and blog posts saying: most business apps are CRUD, I hear, “we don’t want to work with the business to show them what IT could do for them”. Part of the reason is because it requires little skill to model CRUD; modelling business process is harder – but that is where the value lies. Think of early software development approaches that began with modeling the data, and then identified processes to act on that data. Is it any wonder we built CRUD based systems if we started by thinking about data. Data Flow diagrams, Process diagrams. These were the cornerstones of how we thought about the design phase of the software lifecycle and we have failed to escape from it. However, for our users recording data itself is rarely the end goal any more, completion of a business process is the goal.

Now sure, in some cases we might still have CRUD, for some kinds of reference data, and I’m sure a lot of people will respond to this with straw men of CRUD only applications. But be careful. In a lot of cases the customer thinks about records management, and they don’t know you could help them with processing, business rules, workflow (or are frightened of what that might mean for their jobs). Anywhere where there is a process in someone’s head, a word document, or excel spreadsheet that says, when this happens, you need to go into this page and fill in these values on the form, you have behavior, or intent that could and should be captured.

Let’s take a simple example: changing the state of a purchase order. What we tend to be saying is: the workflow for this process has reached a certain point and so it can move into this state. We can immediately see that we must have a number of things for this to occur: the purchase order must have some data that we need to validate it; we can also see that a number of things may be predicated on this occurring: we need to send the order to fulfilment, we need to begin the billing process etc. So this act, completing a purchase order, has both a workflow or business process and rules. A CRUD based approach to design tends to encourage us to leave these outside the system, with the user, in some set of manual checks or switching between parts of the application. What we want to look at is: is this process something we could automate, could the workflow and rules live in the system?

Of course a traditional objection to putting workflow and rules ‘in the system’ is that they can change. This causes some folks to suggest that they ought to be left out.

Unfortunately in the typical business this means that a given named person Alice, Mary or Bob is the person that understands how to drive the process, and is mission critical to the success of the enterprise. If they get hit by a bus its all over, and they can pretty much ask for a raise whenever they like, because we can’t afford to lose them. Process diagrams often end up with a stick figure which imply that ‘Alice does her magic’. Often that person is uncommunicative and secretive about the process – to ensure that they remain essential to the business. When they are absent, things fall apart.

No business really wants that.

And the reality is that we have plenty of options we can use to make workflow and rules more configurable – though we need to look at the tradeoffs between the cost to change the app against using a rules engine or workflow engine carefully by understanding the amount of likely change.

Enter Tasks: What to you want to do today.

We need to drop the focus on CRUD and move to a focus on tasks: what is it that the user wants to accomplish, and how can we help them accomplish that?

Let’s think about the case of a family doctor. In an appointment they tend to take the patient’s history, diagnose the illness, and prescribe a course of treatment. Now, in the UK the paper based way to achieve this used to be the Lloyd George envelope. Simply replicating the functionality of the Lloyd George envelope using a CRUD approach would give us screens for entering consultation notes, prescriptions… But there is a lot more we could do to help. If we treat diagnosis as a task, then we ought to consider searching previous medical history to see if anything shows up related to the patient’s complaint that may be of concern. There may be guidelines for these symptoms that we should be following to check that we are not missing illnesses like meningitis. When we prescribe we can list common drugs for the illness, and look for incompatibility with drugs that the patient is currently being prescribed in a drugs database. That may lead to us offering further advice, for example if the patient is using the contraceptive pill and we prescribe them antibiotics.

Or perhaps consider an on line retailer where the customer can go in and change their account details. If the customer changes their address, we might want to prompt the customer for in-flight orders and ask where they want them to be delivered – the old address or the new address.

Ask yourself the question: am I thinking about the intent of what the user wishes to accomplish?

The grid is particularly dangerous ground in this regard. It is rare that you can capture intent through a grid. if you are using a grid you are probably not focused on a task but on CRUD

Greg Young has a great discussion on the Task-based UI. It is one of the building blocks in a journey towards understanding the why of CQRS