Reliability and Effectiveness

Many times when presenting either the Rightshifting curve:

or the Marshall Model:

I have been asked to define “Effectiveness” (i.e. the horizontal axis for both of these charts). I have never been entirely happy with my various answers. But I have recently discovered a definition for effectiveness, including a means to measure it, which I shall be using from now on. This definition is by Goldratt, as part of Theory of Constraints, and appears in his audiobook “Beyond the Goal”.

Measurements

Measurements serve us in two ways:

1. As indicators of where we are, so we know where to go. For example, the dials and gauges on a car’s dashboard.
2. As means to induce positive behaviours.

We must always remember, though, that we are dealing with humans and human-based organisations:

“Tell me how you measure me and I’ll tell you how I behave.” ~ Goldratt

We must choose measurements to induce the parts to do what’s better for the company as a whole. If a measurement jeopardises the performance of the system as a whole, the measurement is wrong.

Companies already have one set of measurements which measure their performance as a whole: their Financial measurements: e.g. Net profit (P&L) and investment (Balance Sheet)

What about when we dive inside the company as a whole, though? We then have two areas in which we have to conduct measurements:

1. Support for and evaluation of management decisions
2. Oversight on execution (how well are we executing on the decisions we’ve made?)

We generally don’t have good measurements in terms of decisions, nor good measurements in terms of execution.

We have to remember we’re dealing with human beings. And as long as we’re dealing with human beings, we have to realise that by judging any person on more than five measure, we’re creating anarchy. Simply because, with more than five measurements, people can basically do whatever they like, and likely still score high on one of them. And their bosses can nail them on some measurement they fail to deliver against. More than five measurements is conceptually wrong.

Categories of Measurement

So, how to categorise thing so that human beings can grasp the situation? Can we do better than we do now? Theory of Constraints suggests we can.

What resources do we have to help us formulate measurements in each of the above two areas; management decision-making, and execution of those decisions?

• For decision-related measurements – there are lots of resources available to help e.g. books on Throughput Accounting.
• For execution-related measurements – there is next to nothing published anywhere.

Continuous Improvement

I’ll not make the case for continuous improvement here. But if we wish to induce people to continuously improve, where should we focus our measurements? On things that are done properly, or on things that are not done properly? Which of these two foci better drives action? Focussing on the things we’re doing properly tends not to drive improvement. So we must concentrate on things that are not done properly.

How many things are not done properly? Kaplan suggests that in most businesses, there are more than twenty categories of things that are not done properly. But for humans to grasp our measures, we have already decided we need at most five categories, categories that completely cover everything that is not done properly, with zero overlap or duplication. Finding a way to categorise things that meets our criteria here is a nontrivial challenge.

Goldratt says there are only two categories:

1. Things that should have been done but were not.
2. Things that should not have been done but nevertheless were done.

Just two categories, with zero overlap. Beautifully simple.

And each of the above two categories already have a word defining them:

1. Things that should have been done but were not – unreliability.
2. Things that should not have been done but nevertheless were done – ineffectiveness.

Let’s swap these around into positive terms: Reliability, and Effectiveness.

Lovely.

Reliability and Effectiveness

Can we find measures to quantify Reliability and Effectiveness? How can we put numbers on our reliability? How can we put numbers on our effectiveness? Because, without numbers, we’re not measuring.

Let’s consider what is the end result of being reliable, in terms of the system as a whole. And what is the end result of being effective, in terms of the system as a whole? Not in financial terms though, as reliability and effectiveness are not financial things. We know this intuitively.

Reliability

Things that should have been done but were not.

The end result of being unreliable, in terms of the system as a whole, is that the company fails to fulfil its commitments to the external world. In other words, the company fails to ship on time. Do we already measure on-time shipment? Yes. We call it Due Date Performance. That’s a measure of how much we ship on time. “Our company Due Date Performance is 90%”. The unit of measure is almost always “percent”. What behaviour does this unit of measure trigger? Does it trigger behaviour that is good for the company? No. It encourages us to sacrifice on-time shipment of difficult, larger shipments in favour of smaller, easier shipments. So the dollar value of the sale must be part of any reliability measurement. We cannot ignore the dollar value. And neither is time is a factor in percent units. How late is each late shipment? We must include time, too. So, let’s change our “Reliability” units from “percent” to “Throughput dollar days” – the sales dollar value of each orders that is late, multiplied by the number of days it is late, summed across all late orders. The sum total is the measurement of our (un)reliability.

This is of course  a new unit of measure: Throughput-dollar-days. To infer trends, or to compare the performance of e.g. groups or companies, we will need time to train our intuition in the significance of this new unit of measure. As we begin to get to grips with this new unit of measure, it can help to present it as an indicator (a number in some fixed range, say 1-10, or as we use in Rightshifting, and the Marshall Model, 0-5) until we have adjusted to the Throughput-dollar-days measure.

Effectiveness

Things that should not have been done but nevertheless were done.

If we do things that we should NOT have been doing, what is the end result? Inventory. Do we already measure inventory? Of course we do. But how do we presently measure inventory? Either in terms of a dollar value, for example “$6 million of finished good inventory”, or in terms of a number of days, for example “60 days of finished goods inventory”. But both dollars AND time are important. Existing units of measurement for inventory drive unhelpful local behaviours like over-production and poor flow. So, how to measure to induce helpful behaviours? For each item of inventory, let’s use the dollar value of the inventory multiplied by the number of days that we’re holding that inventory under our local authority. We’ll call this unit of measure “Inventory-dollar-days”.

And one more measure of effectiveness: local operating expense. (For example, scrap, or salaries – with a given subunit of the company).

Note: We can fold quality into these measures simply by not recognising a sale, or a reduction in inventory, until the customer accepts the items (i.e. until the items meet the customer’s quality standards).

Summary

Now we have means for defining effectiveness, (and reliability) in a way in which we can also measure it. I feel very comfortable with that.

– Bob

Further Reading

Beyond the Goal ~ Eliyahu M. Goldratt (Audiobook only)

Relevance Lost: Rise and Fall of Management Accounting ~ Kaplan & Johnson

The Goal ~ Eliyahu M. Goldratt

Throughput Accounting ~ Thomas Corbett

The Balanced Scorecard: Translating Strategy into Action ~ Kaplan & Norton

Forecasts, Estimates and Cost Accounting

#NoEstimates?

I’ve tried to avoid getting involved in the ongoing #NoEstimates debate. It seems more like a religious war than a discussion with much prospect of a useful outcome. And a classic case of the Analytic-minded folks butting heads with the Synergistic-minded (and a few Ad-hoc perspectives thrown in for extra confusion).

For me, it also seems like a non-argument. By which I mean that all the knowledge is out there, should folks only but seek to look. For myself, I have several perspectives, drawn from these bodies of knowledge, that I shall continue to apply in the context of estimating and #NoEstimates.

The Theory Of Constraints Perspective

I don’t recall much in Goldratt’s teachings about estimates, per se. But he has written much about the futility of forecasting, e.g. customer demand for products. I suggest his arguments also hold true for forecasting costs (estimating). For more info you might like to take a look at his books, and in particular “It’s Not Luck”.

The Systems Thinking Perspective

Systems Thinking has a relevance to cost estimation, in that systems thinking (c.f. Goldratt, Ackoff) observes that a system is a collection of parts, such that improving the performance of the parts of a system taken separately will negatively impact the performance of the whole. In fact, such “local” improvements can entirely destroy an organization.

Cost Accounting assumes that the cost of each part, each operation, can be known separately (“local costs”). This is a false assumption. I suggest that this means the estimation of costs can, in reality, only produce useful numbers when considered in the context of the system (organisation) as a whole.

See also: “Throughput Accounting” ~ Corbett

The Nonviolent Communications Perspective

From this perspective, we can choose to see folks’ requests for estimates as a means for meeting some of their needs. I’d suggest that some other folks see this means as sub-optimal, in that these other folks believe that there are better means for those folks to get their needs met than through estimates and estimating. And I’d also suggest that for those other folks, having to provide estimates is not meeting their needs. Which is triggering in them various negative feelings, possibly including anger, frustration, hostility and anxiety.

So, applying this knowledge, we might choose to discuss what needs all these folks have, which ones are being met and which not, and some options for effective means for getting everyone’s needs met. Hopefully this might lead to an outcome where folks can agree on a mutually joyful way forward.

The Covalent Perspective

In any non-trivial endeavour, there may be some number of different stakeholders and stakeholding communities, each with their own set of needs. These different needs can and will, at least from time to time, conflict in possibly mutually-exclusive ways. The Covalent approach recognises this and focuses on making folks’ needs explicit and visible, such that these conflict can be resolved, to the extent that is ever possible.

See also: “Competitive Engineering” ~ Tom Gilb

– Bob

Productivity

For all the angst and discussion around how to make organisations, teams and people more productive, we might be forgiven for thinking that the idea of “productivity” was commonly understood and agreed.

However, this is not so.

For example, classical economics has a markedly different definition than does Theory of Constraints (TOC). And if you ask someone – in particular managers demanding “higher productivity” – for an operational definition, you may get a blank look, or other definitions again.

“An operational definition is a procedure agreed upon for translation of a concept into measurement of some kind.”

~ W. Edwards Deming

I’m not arguing for one, common, consistent, clear definition. Rather, I’m drawing attention to the confusion over the term – confusion compounded by many folks taking it for granted that they’re all talking about the same things, that they’re all using the same definitions.

“There is no true value of any characteristic, state, or condition that is defined in terms of measurement or observation. Change of procedure for measurement (change of operational definition) or observation produces a new number.”

~ W. Edwards Deming

Here are just some (differing) definitions I found on the Web:

• Productivity = Value / Time
• Productivity is a ratio of production output to what is required to produce it (inputs)
• Productivity  = average output per period / total costs incurred or resources (capital, energy, material, personnel) consumed in that period
• The amount of output per unit of input (labor, equipment, and capital).
• The amount of work done, related to the time, effort, money, and energy that was needed to produce it.
• Productivity = Throughput / Operating Expenses
• Productivity = Delivery Rate

So, what is productivity? I’m confused now. Are you?

My Own Definition

When I’m talking about productivity, for example in my presentations and workshops on organisational effectiveness and Rightshifting, I have a particular definition clearly in mind:

“Productivity is the act of bringing a company closer to its goal”

~ Jonah, in The Goal by Eliyahu M Goldratt

Personally, although fully agreeing with Goldratt on this definition, I find it’s hard to use in an explanation or discussion, especially with folks unfamiliar with Goldratt’s work. As the Jonah character goes on to say:

“Productivity is a meaningless concept unless the folks in an organisation understand what their goal is.”

And the book (The Goal) takes a whole book to explain how to discover the Goal in any given organisation.

Aside: This definition of productivity makes it closely congruent with “organisational effectiveness”. See this chart:

So for the purposes of discussion, I sometimes use another definition, derived from the TOC formula:

Productivity = Throughput / Operating Expenses

where:

Throughput = Sales – Totally Variable Costs

(a.k.a. the rate at which the system generates money through sales).

and

Operating Expenses = all the money the [organisation] spends in order to turn inventory into throughput.

So, my simpler definition is:

Productivity is how much it costs an organisation to move one unit (measurable step) closer to its goal (whatever its goal may be – for example, getting a particular product to market).

Note: “Cost” here, and below, is in the most general of terms, maybe a composite function of the Five Capitals, and not necessarily just in financial terms e.g. money or cash.

Or, as an (almost) operational definition, where the goal is (improving) organisational effectiveness:

Cost to move the organisation one unit (say, 0.1 of a rightshifting index point) to the right on the effectiveness axis of the Rightshifting chart.

Note: This is much the same as the Rightshifting measure named “Drag”.

Productivity is a Property of the System

By which I mean, that productivity is never a property of an individual or team, but of the whole system of work within which individuals and teams do their work. This is often referred-to as Deming’s 95% rule.

Productivity in Knowledge Work

Taiichi Ohno said “People don’t go to Toyota to ‘work’ they go there to ‘think’”.

If we take this at face value, then The Goal of Toyota, at least from Ohno’s point of view, was to get its people to ‘think’ (which I take to mean, study the system – the way the work works – and improve it).

Make it Clear

So next time, and every time, the topic – or issue – of productivity comes up, think “Are we all on the same page about what this word actually means?”

Some discussion, to ensure everyone is talking about the same thing, pays major dividends. And, yes, increases productivity.

– Bob

Postscript

Since I first wrote this post, it occurs to me that some readers may infer that I believe productivity is an “unalloyed good thing”. Inasmuch as productivity meets the needs of some folks in an organisation, we might choose to accept this at face value. Personally, I reject worshiping at the altar of productivity, and chooser rather to appreciate that a blind pursuit of productivity at the expense of folks’ wider needs can do much more harm than good.

Post-Postscript

In December 2019 I wrote a post titled “Your Real Job” to highlight just how irrelevant productivity is in most organisations. You might like to take a look.

Further Reading

Cost Accounting is Productivity’s Public Enemy Number One – Abonar’s Blog
Theory of Constraints: Bottom Line Measurements – TOC Guide

Quantification vs Measurement

“If you think you know something about a subject, try to put a number on it. If you can, then maybe you know something about the subject. If you cannot then perhaps you should admit to yourself that your knowledge is of a meager and unsatisfactory kind.”

~ Lord Kelvin, 1893

Some folks seem to mix up the idea of quantification with the idea of measurement.

“Why does it matter?” I suspect you might ask. I’ll leave you to be the arbiter of that.

I just wanted to flag that in my view (and in the dictionary), there’s a difference:

Quantity

“A fundamental, generic term used when referring to the measurement (count, amount) of a scalar, vector, number of items or to some other way of denominating the value of a collection or group of items.”

Quantification

“The act of assigning a quantity to (something).”

Tom Gilb defines quantification thusly:

“Quantification, even without subsequent measurement, is a useful aid to clear thinking (what is this about?) and good communication (this is the goal, gang).”

~ Tom Gilb

Measurement

“To ascertain the quantity of a unit of material via calculated comparison with respect to a standard.”

In A Nutshell

In a nutshell, the two terms differ in that:

• Quantification is about a way to have more meaningful discussions, less obscured by subjective language, whilst
• Measurement is about seeing more objectively what’s happening in your world.

In general we can fairly quantify anything; measuring things is often more problematic.

If you have your own definitions which you prefer more, or any other feedback, I’d love to hear from you.

– Bob

Further Reading

Principles of Software Engineering Management ~ Tom Gilb
Competitive Engineering ~ Tom Gilb
Software Metrics ~ Norman E. Fenton
Quantifying Stakeholder Values ~ Tom Gilb (pdf)
Making Metrics More Practical in Systems Engineering ~ Tom Gilb (pdf)