Are ‘software developer competency” and “software product delivery” complementary, compatible, or oppositional?
Agile Expert, Deprecated
Are you an Agile Expert? Do your revenues depend on your Agile chops? Do you see the writing on the wall?
Newsflash: You’re as obsolete as a buggy-whip maker. Agile Main Street is closing. Your markets are shrinking. You’ll only be dealing with the laggards and bad-faith actors from now on.
As a player in technology markets, you know all about technology adoption curves. We’re fifteen years or more past the Chasm for Agile. And tech markets move much faster than FMCG, manufacturing and other such markets.
What’s the next technology curve where your existing cumulative assets can play best and earn the greatest margins? What legacy issues are holding you back? Are you another victim of the Sunk Cost fallacy? What new assets must you acquire to play well on the new curve? What actions must you take today to best position yourself for tomorrow? Times they are a’changing (as always). Are you changing with them?
And what resolutions do these questions bring to mind?
The software industry is not the only domain in which dogma and conservatism combine to defeat effectiveness. Here’s an article on how the US Army (and USMC) are using Mission-type Tactics (Auftragstaktik) in name only (MTTINO).
and a backgrounder on auftragstaktik:
And see also: Product Aikido for insight into (real) mission-type tactics for product development.
If you love books, and in particular books about business, organisations and software and product development, you’ll probably love Memeology. There’s more than 90 chapters, and almost every chapter has a host of references, in context.
(Aside: I’ve personally read almost every book and paper referenced in Memeology).
PS. I expect Memeology to reach 100% completion in just a few days now.
Category errors (or category mistakes) abound in life, and work. In particular, there’s the category error of regarding collaborative knowledge work – such as software or product development – as like some other category of work. And then expecting the work to work like that other – probably more familiar – category of work.
This only and ever leads to management monstrosities.
Here’s the thing: does your organisation make this kind of category error? And the consequences?
“In reality, the Western way of managing is still based on the principles of Scientific Management where the goals of management is control and compliance to rigorous processes. The Toyota paradigm is much different; it is based on ongoing learning, capturing the learning for wide reuse, and continually improving the knowledge across generations of products. To do this takes a lot of management understanding and commitment to guide the change.”
~ Michael Kennedy
I’ve not seen the idea of “Constant state of ship” mentioned much in the past few years, so maybe it’s time for a quick reminder?
How To Predictably Deliver Great Software
What is “Great Software“?
Aside: I invite you to consider how the Needsscape plays into the above definition.
Ironically, when we dig into the needs of all the Folks That Matter, we find that great software often means no software. Strange?
Further, we regularly find that the needs of the developers and ancillary development staff trump the needs of the users and other customers. So we get software, even though users and other customers rarely need it.
Let’s assume for a moment that predictability (e.g. due date performance) is among the needs of some of the Folks That Matter. In this case, predictability is part of the “great” we were just talking about.
Predictability comes from anticipating and managing risks – actively, deliberately and competently. Formal approaches such as set-based concurrent engineering (SBCE) help manage the risk of finding oneself unable to bring a particular aspect of the solution to completion, for a variety of reasons. Identifying the Folks That Matter and their needs helps manage the risks involved in building the wrong thing, as does consideration of the Cost of Focus. Predictability demands we keep on top of all significant risks. (See: the All Holes in the Boat principle Cf. Gilb).
Know what needs you’re attending to. And whose.
This is not always possible, a priori. So identify as many of the Folks That Matter as you can (expect more to come to light as you proceed). Concurrently, investigate their needs through quickly delivering early exploratory things such as mock-ups, paper-prototypes, sketches of various kinds, and conversations. “Quickly” here means in a few hours, or days at most. Expect to have to iterate on this.
Many developers assume that someone else will be handing them a list of the Folks That Matter along with a definitive statement of the needs of those folks. If that other party is competent and sufficiently resourced, this can work. I’ve never seen it. I prefer to have the developers own this crucial information, and the gathering and updating thereof too. This is not popular in many organisations, shining a light as it does on the inadequacies of the way the work works, the management, the analysts, the product owner(s) and so on.
In parallel with these investigations, make a start on building the solution. In particular, those parts of the solutions which seem relatively stable, and where you feel the needs are relatively well understood. This can even involve writing some software – if you really must. Manage the risk of not knowing how to build certain things through e.g. “Spikes” and other risk mitigations.
“Surely there’s more to it that this?’ I hear you ask.
Well, actually, no. We’ve been hoodwinked into thinking that software development is “the most complex endeavour known to man” ~ Jim McCarthy
Actually, if tackled appropriately it’s quite a pussy cat. People make it much more difficult than it really is. Will the industry ever grow up?
Memeology For Developers
“The greatest determinant of organisational performance is the way we think about the design and management of work.”
~ John Seddon
And, therefore, the greatest determinant of the performance of self-organising software development teams is the way those teams think about the design and management of their work.
Aside: For teams and individuals that are not self-organising, the original quotation pertains.
Memeology is For Whole Organisations, not Teams
I suspect many developers and development teams might see “Memeology” (my new book) as irrelevant to them. As a book that lists memes of little interest, and as a book that surfaces assumptions and beliefs more relevant to senior management – which, to be fair, is the books primary audience.
And yet, as organisational psychotherapy speaks to an organisation’s collective psyche, “Memeology” invites addressing of the collective assumptions and beliefs of everyone in the organisation. So, as far as involving people in the discussions around “Memeology”, I’d suggest encouraging everyone to become involved.
In most organisations, however, local rules apply. Different groups may well be interested in memes specific to their own specialisms. For example: Finance, Operations, Logistics, HR, and yes, Product and Software Development.
I’m chary of promoting local optimisations – for example, the improvement of software development practices in isolation from the rest of the organisation. But developers in Analytic-minded organisations outnumber by far those in Synergistic-minded organisations (the latter being more prone to taking system-wide issues into consideration). Are we to deny a self-help book such as “Memeology for Developers” on the grounds that discussing development-specific memes in isolation might perpetuate specialisms and concomitant local optimisations confined to the “development silo”?
Here’s just a few of the many memes that “Memeology for Developers” might cover:
- Requirements (when to capture them, who’s responsible for such capture, formalisms and representations, etc.)
- Code ownership (individual, group, other, and the trade-offs and consequences)
- Defect tracking
- Performance (of the software when in production)
I have a long list of such memes. I’m sure you can suggest memes for such a list, too.
Does the idea of “Memeology for Developers” pique your interest? Would it be a useful book, promoting as it would not only the surfacing and reflection on the assumptions and beliefs developers hold in common, but also promoting experimentation to improve self-organising software teams and the way they work?
I’d love to hear your thoughts.
I was reading the other day about how Elon Musk “reasons from first principles“. And I was thinking, “Well, d’oh! Doesn’t everyone do that? I know I do.” And then, upon reflection, I thought, “Hmm, maybe most folks don’t do that.” I certainly have seen little evidence of it, compare to the evidence of folks reasoning by extension, and analogy. And failing to reason at all.
Now, allowing for journalistic hyperbole and the cult of the celebrity, there may just still be something in it.
So, in case you were wondering, and to remind myself, here’s some first principles underpinning the various things in my own portfolio of ideas and experiences:
The Antimatter Principle
The Antimatter Principle emerges from the following basic principles about us as people:
- All our actions and behaviours are simply consequent on trying to get our needs met.
- We are social animals and are driven to see other folks’ needs met, often even before our own.
- We humans have an innate sense of fairness which influences our every decision and action.
Flowchain emerges from the following basic principles concerning work and business:
- All commercial organisations – excepting, maybe, those busy milking their cash cows – are in the business of continually bringing new products, or at least new product features and upgrades, to market.
- When Cost of Delay is non-trivial, the speed of bringing new products and feature to market is significant.
- Flow (of value – not the Mihaly Csikszentmihaly kind of flow, here) offers the most likely means to minimise concept-to-cash time.
- Autonomy, mastery and shared purpose affords a means for people to find the intrinsic motivation to improve things (like flow).
- Building improvement into the way the work works increases the likelihood of having sufficient resources available to see improvement happen.
Prod•gnosis emerges from the following basic principles concerning business operations:
- All commercial organisations – excepting, maybe, those busy milking their cash cows – are in the business of continually bringing new products, or at least new product features and upgrades, to market.
- Most new products are cobbled together via disjointed efforts crossing multiple organisational (silo) boundaries, and consequently incurring avoidable waste, rework, confusion and delays.
- The people with domain expertise in a particular product or service area are rarely, if ever, experts in building the operational value streams necessary to develop, sell and support those products and services.
Emotioneering emerges from the following basic principles concerning products and product development:
- People buy things based on how they feel (their emotional responses to the things they’re considering buying). See: Buy•ology by Martin Lindstrøm.
- Product uptake (revenues, margins, etc.) can be improved by deliberately designing and building products for maximum positive emotional responses.
- Quantification serves to explicitly identify and clarify the emotional responses we wish to see our products and service evoke (Cf. “Competitive Engineeering” ~ Gilb).
Rightshifting emerges from the following basic principles concerning work in organisations:
- The effectiveness of an organisation is a direct function of its collective assumptions and beliefs.
- Effectiveness is a general attribute, spanning all aspects of an organisation’s operations (i.e. not just applicable to product development).
The Marshall Model
The Marshall Model emerges from the following basic principles concerning work in organisations:
- Different organisations demonstrable hold widely differing shared assumptions and beliefs about the world of work and how work should work – one organisation from another.
- Understanding which collection of shared assumptions and beliefs is in play in a given organisation helps interventionists select the most effective form(s) of intervention. (Cf. The Dreyfus Model of Skills Acquisition).
Organisation psychotherapy emerges from the following basic principles concerning people and organisations:
- The effectiveness (performance, productivity, revenues, profitability, success, etc.) of any organisation is a direct function of its collective assumptions and beliefs about the world of work and how work should work.
- Organisations fall short of the ideal in being (un)able to shift their collective assumptions and beliefs to better align with their objectives (both explicit and implicit).
- Having support available – either by engaging organisational therapists, or via facilitated self-help – increases the likelihood of an organisation engaging in the surfacing, reflecting upon, and ultimately changing its collective assumptions and beliefs.
Software Development at Scale
Scaled Agile – or scaling agile – seems like a hot potato at the moment. Here’s a few
Scaled Agile? Don’t. Just don’t. For GOD’S sake don’t.
Agile even at a small scale doesn’t work and scaling something that doesn’t work just leads to an even bigger mess that doesn’t work. Take a look at SAFe if you don’t believe it. Or just listen to Ackoff:
“The righter we do the wrong thing, the wronger we become. When we make a mistake doing the wrong thing and correct it, we become wronger. When we make a mistake doing the right thing and correct it, we become righter. Therefore, it is better to do the right thing wrong than the wrong thing right.”
Mindset (a.k.a. culture, memeplex) over methods, tools, etc.. What an organisation, collectively, believes and assumes has far more impact on e.g. productivity, quality, etc. than any methods or tools. See: Organisational Psychotherapy, the Marshall Model, and my new book, “Memeology“.
The only thing of real importance that leaders do is to create and manage culture.
~ Edgar Schein
The thing I have learned at IBM is that culture is everything. It took me to age fifty-five to figure that out.
~ Lou Gerstner, CEO, IBM
If you get the culture right, most of the other stuff will just take care of itself.
~Tony Hsieh, CEO, Zappos.com
And Schein also provides us with a definition for “the culture of an organisation”:
Culture is the deeper level of basic assumptions and beliefs that are shared by members of an organisation, that operate unconsciously and define in a basic ‘taken for granted’ fashion an organisation’s view of its self and its environment…
It’s a pattern of shared basic assumptions invented, discovered, or developed by a given group.
~ Edgar Schein
Goldratt wrote the book on Flow Efficiency (see “The Goal”, in case you’re interested).
Flow efficiency is miles better than resource efficiency (a.k.a. utilisation). What is flow efficiency? I’m sure you can look it up. But for the lazy: Flow efficiency suggest a focus on keeping work moving through its various value-adding steps, with minimal or zero queues and wait times, thereby attending to folks’ needs (value to the Folks That Matter) – and prompting feedback – as quickly as possible.
Formalise innovation, evolve into continuous organisational transformation.
What do I mean by “innovation”?
In the context of organisations, here are a few possible definitions of “innovation”:
- A process by which a method, a product, or a service is renewed and refreshed by applying new ideas or introducing new techniques to create new value.
- Turning an idea into a solution that adds value from the perspective of the folks that matter
- A new strategy (means) for attending to the needs of the folks that matter.
- The application of ideas that are novel and useful.
- Staying relevant – keeping pace with changes to the (business) environment.
- The implementation of something new – until it’s implemented it’s just an idea.
- Stop talking about innovation. Focus on organisational transformation.
Here’s my preferred definition:
Innovation: Delivering against an idea which meets a specific set of needs, for all the Folks That Matter.
And “formalise”? What the hell does that mean, exactly? In effect, institute trainings, standard work, measures, etc., around the whole innovation (and, a.s.a.p., organisational transformation) piece.
I’ve been brief here to avoid a rant. Do get in touch if you’d like me to elaborate on some of these themes.
Artefact Driven Delivery
My preference when approaching solution delivery (I use that term in preference to software delivery, because #NoSoftware) has for the past 25 years centred on artefacts as opposed to tasks. I’m not going to retread here the arguments in favour of the artefact as the unit of progress. This post covers the use of artefacts in incremental development environments, and lists the core artefacts we use in our approach to solution delivery.
Delaying work on implementing and delivering a solution until we have fully defined the requirements, designs, etc., for that solution magnifies the Cost of Delay, defers feedback significantly, and inflates other risks too. Yet we don’t want to skip having clear requirements and designs, either.
The approach we adopted starting circa 1994 is to establish a set of standard artefacts, at the outset of work on each new solution. From Day One, these artefacts will be empty scaffolds, based on standard templates, each artefact being elaborated just-in-time, immediately in advance of their contents being needed for implementation and delivery purposes.
In this way, we avoid B*UF (e.g. Big Design Up Front, Big Requirements Up Front, etc.) and the delays and risks associated with a Big Bang approach.
The following is a list of all the standard artefacts we create on Day One of the inception of each new solution on which we embark. Note that each artefact is based on a standard template, with, from the outset, little or no solution-specific content (i.e. empty).
- Control Document
- Articles of Understanding
- Glossary of Terms
- Statement of Purpose
- Case for Action
- Folks That Matter™ and their Needs
- Risk Parade
- Top Risks
- Functional Requirements
- Non-functional Requirements aka QQOs
- Critical Success Factors
- Feature Schedule
- Quality Plan
- Test Plan
- Change Control
- Cycle Plans
- Cycle Reviews
Artefacts and Deliverables
We share some or all of the above artefacts with our clients (the folks on whose behalf we are developing solutions) continually, and at their request. These artefacts are available for sharing throughout the duration of the development. And serve as a running history of the endeavour, too. The deliverables of any solution (code, data, policies, documentation, configs, databases, etc.) augment these standard, evolving artefacts. Typically, a set of deliverables will fall out of the work according to some cadence or rhythm (for example, weekly or every two weeks).
For a fuller (if rather dated) explanation of each particular artefact (some now carry slightly different names), see the Javelin white paper.
In a following post I’ll be showing how you might insinuate the Antimatter Principle into your existing approach to developing solutions, using the Artefact Driven Delivery approach.
Cost of Focus Revisited
[Recent conversations suggest that my post on Cost of Focus failed to explain the idea clearly enough for readers to grasp easily or quickly. As, for me at least, it’s a simple idea, I thought I’d summarise it in brief with a new post (this one).]
Cost of Focus
Let’s start with a definition in a nutshell:
Cost of Focus is the cost incurred when we fail to include key stakeholders* in our deliberations**.
* I generally refer to these folks, collectively, as The Folks That Matter™.
** By deliberations, I have in mind what old-school folks call “requirements capture” or “requirements analysis”, and what I, nowadays, refer to as “needs investigation”.
Put another way:
Cost of Focus is a way of communicating the impact – on the outcomes we hope to achieve – arising from excluding or including specific folks and their needs.
Note: I could have chosen the name “Cost of Flawed Focus” (the cost of focussing on less relevant stakeholders and less relevant requirements), but this seemed a little less snappy than “Cost of Focus”.
Typically, the costs in question accrue from rejection of part or all of the delivered project / system / product / software application by one or more key parties (such as users) whose needs have not been adequately addressed – and these costs can be massive. In any number of cases, whole systems have had to be abandoned because one or more key stakeholder groups have refused to use the new system. And even when not totally abandoned, oftentimes major costs have accrued from the delays and extra work required to remediate the original errors of focus. (See also Cost of Focus’ kissing cousin – Cost of Delay).
The Folks That Matter™
[The following excerpt first appeared in my blog post The Folks That Matter™. I repeat it here for the convenience of the reader:]
Cost of Focus
Don Reinertsen states that the Cost of Delay – the financial or economic cost of delaying a given feature by prioritising another – is rarely considered in most organisations. Put another way, the way in which delivery priorities are selected and adjusted, the frequency and means of such adjustments, etc., are rarely discussed, and rarely even discussable.
I propose that Cost of Delay is a subset of the wider question stated above, i.e. the question of Cost of Focus.
By definition, we are failing to meet some folks’ needs when we choose to or otherwise exclude certain folks with their particular needs from the set of The Folks That Matter™.
Maybe those excluded folks and their needs are indeed irrelevant, or their exclusion has little impact – financial or otherwise – on the success of our endeavour. But maybe, contrariwise, some of those excluded needs are in fact critical to our “success”. How would we know? The arguments for Cost of Focus are much the same as for its golden child, Cost of Delay.
FWIW, I’ve seen countless projects stumble and “fail” because they inadvertently omitted, or chose to omit, some crucial folks and their needs from the their list of The Folks That Matter™. Get Cost of Delay wrong (prioritise less valuable features), and we lose some money. Sometime a little, sometime a lot. Get Cost of Focus wrong, and we more often lose big time. Cost of Focus often has a much more binary, black-and-white impact.
What is Cost of Focus?
Cost of Focus is a way of communicating the impact – on the outcomes we hope to achieve – arising from excluding or including specific folks and their needs. More formally, it is the partial derivative of the total expected value with respect to whose needs we focus on.
“Cost of Delay is the golden key that unlocks many doors. It has an astonishing power to totally transform the mind-set of a development organisation.”
– Donald G. Reinertsen
Similarly, I’d say that unless and until we have a handle on Cost of Focus, the golden key of Cost Of Delay remains firmly beyond our grasp.
Put another way, until we have a means for deciding to whose needs to attend, the particular order in which we attend to those needs (cf. priority, Cost of Delay) is moot.
What Is A Customer?
In the world of Agile, and the world of business too, we hear a lot about “customer value”. Folks seem to have some kind of handle on “value” (although not everyone can agree on that one – see my post “What Is Value” for my take, based on Goldratt and his Theory of Constraints).
And for the record, we might also choose to frame the question of value within the Antimatter Principle frame, and vocabulary:
Value: The degree to which folks’ needs, in aggregate, are being (or have been) met.
But what about “customer”? So simple and straightforward. Do we even need to define it? I thought not, until a recent conversation on Twitter gave me pause for reconsidering. Specifically, the idea that maybe folks are talking at major cross-purposes, with significantly differing assumptions and definitions for the term. If we can’t agree on a basic term like “customer”, what chance alignment of a whole host of fundamental questions about software, products and business generally?
Here’s my definition, again using the Antimatter Principle as a frame:
Customer: Someone (could be either a person, or a collection of people) whose needs we’re attending to.
I’m pretty sure you’ll have a different definition of customer. I’d love to hear your take.
Before I close this post, here’s a different definition, informed by Crosby and his Zero-Defects (ZeeDee) approach to quality:
Customer: Anyone who receives or anticipates receiving something (e.g. a good or a service) from someone else.
This definition canonises Crosby’s idea that we’re all customers. And we’re all suppliers, too. And as suppliers, it falls to us to ensure that what we’re supplying is what our immediate customer needs to supply their customer(s).
Why Reason When Faith is So Much More Comfortable?
I’ve become very bored trying to explain why Agile – even when practised as the Snowbird Gods intended – is a dead-end and why we might choose to bark up a different tree for progress in improving the effectiveness of software development organisations.
Firstly. No one seems at all interested in “improving the effectiveness of software development organisations”. Yes, there does seem to be some interest in being seen to be doing something about improving the effectiveness of software development organisations. Hence SAFe, DAD, LeSS – and Agile itself. None of these approaches do anything about actually improving the effectiveness of software development organisations, of course. But that’s not the point. Improvement *theatre* wins the day in just about every case. Irrespective of practices done “right”, or more often, done “in name only” (Cf AINO).
To actually do anything about improving the effectiveness of software development organisations requires we remove some fundamental system constraints, including:
- Optimising parts of the organisation in isolation
- Pursuit of specialism (vs generalists)
- Control (as in Command & Control)
- Annual budgeting
- Extrinsic motivation
- Ignorance of the special needs/realities of collaborative knowledge work
- Separation of decision-making from the work
- Decision-makers’ ignorance of and indifference to customers’ needs
- Seeing performance as consequent on the efforts of individuals and “talent”
- Discounting the paramountcy of social interactions and inter-personal relationships
And that ain’t gonna happen.
Second, improving the effectiveness of software development organisations kinda misses the point. In that software development is part of the problem. Making it more effective is just – as Ackoff would say – doing more wrong things righter.
Instead, a focus on meeting folks’ needs, or at least, as a minimum, attending to their needs, would serve our search for effectiveness rather better. And that generally requires less software, and placing software development last in terms of priority, way before understanding customers’ needs (and more generally the needs of The Folks That Matter).
Given that the software industry’s revenues are contingent on producing software (see: Upton Sinclair’s Dictum) that ain’t gonna happen, either.
Third, if we regard improving the effectiveness of software development organisations as our aim, and limit our ambitions to that part of the organisation concerned directly with software development (i.e. the IT department or the Product Development department) then, at best, we’ll only ever see a local optimisation. Which as Ackoff tells us, only makes matters (i.e. the effectiveness of the whole organisation) *worse*. To improve organisational effectiveness (not to mention supply chain effectiveness, customers’ effectiveness) requires us to consider the organisation as a system, and focus on the systemic relationships between the parts, rather than on the parts taken separately. And given that systems thinking has failed to gain much traction in over fifty years of trying, THAT ain’t gonna happen either.
I’ll just leave this here:
“If you could reason with Agile people, there would be no Agile people.”
It all looks a bit bleak, doesn’t it? Another method isn’t going to help much, either. Unless it addresses the three points outline above. As a minimum.
That’s why I have been for some years inviting folks to consider Organisational Psychotherapy as a way forward.
But reason, rationality, and a cold hard look at reality and the shortcoming of the status quo ain’t gonna happen. Until organisations see a need for that to happen.
It seems in vogue to extol the praises of “outcomes” when discussing e.g. software and product development. Setting aside the challenges of defining what we might mean by “outcomes” (I dislike getting into rabbit-hole discussions of semantics), there’s one key aspect of this debate that seems to escape folks’ attention. W Edwards Deming nailed it decades ago with his First Theorem:
“Nobody gives a hoot about profits.”
Even so, Deming said little about what folks (managers, in his frame) DO give a hoot about. We can turn to Russell L. Ackoff for an insight into that:
“Executives’ actions make sense [only] if you look at them as taken in order to maximise the executive’s well being.”
As Dr. Ackoff says, a secondary focus on profits is just the cost executives must pay in order to maximize their rewards. The actions taken would be different if the well being of the organization was primary and the well being of senior executives subservient to that aim.
So, to outcomes. The outcomes that delight will be those that maximise the executives’ (and other folks’) well being. When developing a piece of software, how often do the specifics of the well being of the Folks That Matter get discussed? Indeed, is the subject even discussable? Or is it taboo? In your organisation?
How unsurprising then, that software as delivered is so often lacklustre and uninspiring. That it fails to address the core issues of the well being of the Folks That Matter? That it’s the wrong software.
As a developer or team, do you ever afford your customers (a.k.a. the Folks That Matter) the opportunity to talk about their well being? And how what you’re doing for them might contribute to that well being?
So, might I invite you to stop talking about specious and illusory “outcomes”. And start asking the difficult questions of your customers (and yourselves)?
Here’s a possible opener:
“Would you be willing to discuss what it is you need for your own well being?”
The Big Shift
Let’s get real for a moment. Why would ANYONE set about disrupting the fundamental beliefs and assumptions of their whole organisation just to make their software and product development more effective?
It’s not for the sake of increased profit – Deming’s First Theorem states:
“Nobody gives a hoot about profits”.
If we believe Russell Ackoff, executives’ motivation primarily stems from maximising their own personal well being a.k.a. their own quality of work life.
Is There a Connection?
Is there any connection between increased software and product development effectiveness, and increased quality of work life for executives? Between the needs of ALL the Folks That Matter and the smaller subset of those Folks That Matter that we label “executives”? Absent such a connection, it seems unrealistic (understatement!) to expect executives to diminish their own quality of work life for little or no gain (to them personally).
Note: Goldratt suggests that for the idea of effectiveness to gain traction, it’s necessary for the executives of an organisation to build a True Consensus – a jointly agreed and shared action plan for change (shift).
Is Disruption Avoidable?
So, the question becomes:
Can we see major improvements in the effectiveness (performance, cost, quality, predictability, etc.) of our organisation, without disrupting the fundamental beliefs and assumptions of our whole organisation?
My studies and experiences both suggest the answer is “No”. That collaborative knowledge work (as in software and product development) is sufficiently different from the forms of work for which (Analytic-minded) organisations have been built as to necessitate a fundamentally different set of beliefs and assumptions about how work must work (the Synergistic memeplex). If the work is to be effective, that is.
In support of this assertion I cite the widely reported failure rates in Agile adoptions (greater than 80%), Lean Manufacturing transformations (at least 90%) and in Digital Transformations (at least 95%).
I’d love to hear your viewpoint.
Organisational Cognitive Dissonance ~ Think Different blog post
Something’s Gotta Give
“The things businesses have to do to make software development successful are well known. And equally well known is the fact that businesses will absolutely not do these things.”
This reality puts us in a bind. We find ourselves in a position where we have to trade off successful development against conforming to organisational norms. We can have one – or the other. It’s not a binary trade-off, we can for example relax some norms and gain some (small) improvements in success. But by and large it’s a zero sum game. At least from the perspective of those folks that find value in everyone conforming to preexisting norms.
I don’t think many business folks realise this trade-off exists. Almost all the business folks I have met over the years seem unaware that their norms are what’s holding back their success in software (and product) development. I put this down to the absence of any real understanding of the fundamentally different nature of collaborative knowledge work (different to their experiences and assumptions).
Some of the Things
By way of illustration, here’s just a few of the things that are necessary for successful software (and product) development, that businesses just won’t do:
Removing stressors (things that create distress) from the workplace. These things include: job insecurity; being directed and controlled; being told where, when and how to work; etc..
Stressors serve to negatively impact cognitive function (amongst other things). See also: Amygdala hijack.
Placing trust in the folks actually doing the work. We might refer to this as a Theory-Y posture.
Finding out through disciplined and systematic experimentation what works and what doesn’t. See: the Toyota Improvement Kata.
Embracing what it means to be human; seeing employees as infinitely different, fully-rounded human beings with a broad range emotions, needs and foibles (as opposed to e.g. interchangeable cogs in a machine).
Relying on intrinsic motivation to encourage and support a disciplined approach to work.
Talking about what’s happening, the common purpose, and what the problems are.
Realising the limitations with numbers, dashboards, KPIs and the like and finding other ways to know whether things are moving in the “right direction”.
Prioritising Interpersonal Relationships
In collaborative knowledge work (especially teamwork), it’s the quality of the interpersonal relationships that’s by far the greatest factor in success.
If your organisation needs to see more success in its software (and product) development efforts, then something’s gotta give. Specifically, some of its prevailing norms, assumption and beliefs have gotta give. And given that these norms come as a self-reinforcing memeplex (a.k.a. the Analytic Mindset), a piecemeal approach is highly unlikely to afford much in the way of progress.
Standard Work and Collaboration
[Tl;Dr: Ad-hoc and impromptu collaboration is a signal – that our standard work is incomplete or insufficient, and that we don’t understand as much about what we’re doing and how, as we’d like to think.]
Standard work (also known as Standardized Work) is an operational definition of how the work works today. Best written and maintained (studied, updated) by the folks actually doing the work. Toyota defines Standard Work as ”the steps one needs to walk in order to complete a process”. Mike Rother defines Standard Work as the “Target Condition” in the Improvement Kata. This seems to me to make some sense.
“There is something called standard work, but standards should be changed constantly.”
~ Taiichi Ohno, Workplace Management
In slightly more detail: “Standardized work answers the 5W+1H of a process – the who, what, when, where, why, and how. Who operates the process, and how many people does it take? What does the final product look like, what are the quality check points, what are the tools required to complete the job? When is a part completed and ready for the next step (how long should the cycle time and takt time be)? Where is this process completed and what does this location look like (standardized work cell, point of use storage of tools, etc)? Why is this step necessary or value-adding, or why is this a quality check point?”
“When there is no standard [work], there is no Kaizen (continual improvement).”
~ Taiichi Ohno
In other words, when a process is performed unsystematically in different ways, then:
- There can be no basis for comparison (before/after)
- One cannot objectively tell if there was a difference or change
- No improvement is possible in regards to Time, Quality, Quantity, Cost, etc.
Collaboration is Waste
So, where does collaboration come into the picture? If the standard work specifies “collaborate here” (with 5W+1H or an operation definition for the collaboration) for a particular step, then all is fine and dandy.
But often, in software development particularly, there is no standard work, or the standard work lacks the detail which might suggest the 5W+1H of the collaboration. Exceptions which come to mind are: the daily standup (Scrum), sprint planning (Scrum) and sprint retrospectives (Scrum) (i.e. the various Scrum ceremonies – for which teams rapidly find their own work standards or de facto operational definitions).
Consequently, collaboration in software development is most often ad-hoc. Someone might run into a problem or challenge, and ask a colleague e.g. “Hey, can you help me with this?” or “Can we pair on this for half an hour?” or “Let’s get together and figure out what to do here”.
If we had clearly defined standard work, the specifics of what to do and who to call on when a problem arises would already be defined. Without such standard work, the coordination (set-up, figuring-out) of the necessary collaboration is waste, and interrupts the flow (both of value, and in the Mihaly Csikszentmihalyi sense of the word).
Do I hear you rail against this idea? Do you believe it’s impossible to foresee where and when collaboration might be necessary? Do you enjoy collaborating so much that you’re prepared to dismiss its negatives? May I put it to you that in such circumstances, you don’t actually know what y’all are doing? That you have little or no clear idea how to get from the start of sprint (or longer term) to the end, to the delivery of value? That you’re making much of it (“the way the work works”) up as y’all go along?
“…this model of ‘standards’ as something for compliance is a cancer that is holding us back in our quest to establish a new level of understanding around what ‘continuous improvement’ really means.”
~ Mark Rosenthal
The Bottom Line
This may all seem rather esoteric. How much can it matter whether collaboration costs us a few dollars or hours? For me, ad-hoc and impromptu collaboration is a signal – that our standard work is incomplete or insufficient, and that we don’t understand as much about what we’re doing and how, as we’d like to think.
Does it matter? I leave that to y’all to decide.
How well does the almost universal Agile practice of “build it and see if they come” serve us (as developers, as customers)?
I suggest it’s time to rethink our belief that customers (and developers, for the most part) “don’t know what they want until they see it”.
My late, great colleague and friend Grant Rule used to refer to the practice, common in the Agile domain, of building (a portion of) something to see if the customer likes it as “random walks through the problems-solution space”.
Quality Demands Requirements
Philip Crosby, a widely acclaimed “guru” of Quality Management, defined quality as “conformance to requirements”. As simple and blunt as that.
Recently, I’ve been reflecting on my experiences with software product development, especially the development of “quality” products that customers love. In Javelin, we place special emphasis on de-risking delivery through explicitly defining the customers and their respective requirements. Not big-bang, up-front stylee, but incrementally, just enough each couple of days to build a little more of the product and deliver it to the customer(s) for their delight, confidence, and feedback.
But in our approach, requirements (in the frame of the Antimatter Principle we call these needs) precedes building anything. Agile shops these days seems to major in building something before discussing requirements (if they ever get discussed at all). BDD offers an exception, but how many shops do BDD?
Aside: In Javelin, we identify all stakeholders (a.k.a. all the Folks That Matter), discuss their needs (“Stakeholders’ Needs”, in Javelin parlance) and quantify them (a la Gilb – see: Competitive Engineering) in the form of Quantified Quality Objectives. Although:
- This all generally proceeds incrementally, rather than in a big batch up front.
- The information is always to hand by the time someone gets around to building the relevant part of the thing in question.
- The requirements come from dialogue(s) with the relevant Folks That Matter.
- The requirements need not get written down (documented) unless there are some Folks That Matter that need them to be.
People work from the requirements. Always.
Random Walks are not Our Bag
Random walks are not our bag.
By cleaving to the belief that customers “don’t know what they want until they see it”, and structuring the whole approach to development around this belief, Agile shops have no incentive to improve the way they work with customers to understand their needs. No incentive to improve requirements elicitation and capture. No incentive – or means – to prevent defects and deliver zero-defects quality. Indeed, this belief and its associated practices blocks us from working to continually find better ways to create useful requirements (formal statements of folks’ needs) from which to drive quality (cf Crosby) and the improving of relationships with each other (developers, ops) and with customers.
Is this emphasis on working-from-clearly-stated-and-agreed-requirements better? Well, in my experience it makes for happier customers, happier developers, and more successful products. I’ll leave it to you to decide whether and how that’s “better”.