I’m Done

Memeology, Quintessence

I’m done with inviting folks to discover better ways to run collaborative knowledge work businesses and other organisations. 

The Antimatter Principle

I’m done with inviting people to build more humane, engaging organisations.

Rightshifting

I’m done with illustrating the gulf in performance and effectiveness between the average organisation or business, and the best. And how much productivity just goes begging.

The Marshall Model

I’m done with inviting people to understand the role of collective assumptions and beliefs in the effectiveness of their organisations.

Effectiveness

I’m done with even mentioning effectiveness. No one seems to need it or want it or even to understand what it is and its role in organisational success.

Emotioneering

I’m done with inviting organisations to consider the way people actually go about buying goods and service, and the role of emotions therein.

FlowChain, Prod•gnosis, Flow•gnosis

I’m done with providing food for thought on how the work in collaborative knowledge work organisations can work awesomely better.

Product Aikido

I’m done with inviting folks to look more deeply into the principles of product development and what makes for more effective product development.

The Giants

I’m done with mentioning the Giants such as Ackoff, Deming, Drucker, et al.

Software

I’m done with software and helping people improve software development, reliability, quality, predictability, etc.. #NoSoftware’s the thing.

Recruiters and the Job Market

I’m done with know-nothing recruiters only focused on their next commission. And a totally broken job market focussed on mediocrity and the status quo. Oh, and CVs too. #NoCV.

The Closed-Minded

I’m done with people that are happiest sitting on their arses (metaphorically speaking) and keeping their eyes, ears, and minds closed to possibilities. Which is everybody, AFAICT.

The Unreliable

I’m done with people that promise to do things, and then, silently, do fuck all.

Agile

I’m done with Agile. Actually, as you’re probably aware, I’ve been done with Agile for a decade and more. I’m just adding it here for the sake of completeness. Oh, and I’m SO done with ignorant people who continue to promote the Agile busted flush.

I’m Done With Better Ways

I’m done with it all. Given there’s zero demand for “better”, better ways seem entirely irrelevant.

And good luck with that status quo. 

– Bob

Quintessential Product Development

In my most recent book “Quintessence” I map out the details of what makes for highly effective software development organisations.

As fas as software development organisations are concerned, it’s a bit of a moot point – as software is generally something to be avoided, rather than sought (see also: #NoSoftware).

“The way you get programmer productivity is by eliminating lines of code you have to write. The line of code that’s the fastest to write, that never breaks, that doesn’t need maintenance, is the line you never had to write.”

~ Steve Jobs 

Foundational Concepts

There are just a few complementary concepts that mark out the quintessential product development company. These are:

• Whole Product.
• Systematic Product Management.
• Whole Organisation (systems thinking).

Whole Product

The quintessential product development organisation embraces the concept of “whole product”. Which is to say, these organisations emphasise the need to have every element of a product i.e. core product elements plus a range of “intangibles” – everything that is needed for the customer to have a compelling reason to buy (Mckenna 1986).

Systematic Product Management

Quintessential product development organisations take a systematic approach to flowing new product ideas and features through a number of stages – often in parallel (Ward 1999) – to predictably arrive at a successful new product in the market:

• Inception – spotting a gap in the market, a.k.a. some (potential customer) needs going unmet, interesting enough to do some discovery.
• Discovery – uncovering and proving the real needs of customers, the things they value, the likely usability of possible solutions, the feasibility of meeting everyone’s needs, and the viability of a product as a means to these ends. In essence, the key risks facing the proposed product. 
• Implementation – building a whole product solution, i.e. both core elements and “intangibles”.
• Launch – Placing the product on sale (or otherwise making it available to customers).
• Feedback – Seeing how the market responds.
• Pivot or Augmentation – Acting on feedback to either reposition the solution (in response to unfavourable feedback) or to incrementally update / extend the “whole product” offering to continually strengthen the product’s value proposition and appeal.
• Cash Cow – Reap the commercial rewards of a strong product and market share.
• Sunsetting – Wind down the product in a way that meets the ongoing needs of all the Folks That Matter™️ (e.g. continued support, spare parts, etc.; easing customers’ transition to newer products; etc.). 

Whole Organisation

It’s common for organisations to think in terms of silos. A Product Management or Product Development silo being but one more silo in a long and ever-lengthening list. 

In the quintessential organisation, the whole organisation is geared around – amongst other things – the task of regularly and predictably getting new products and new product features/updates out the door and into the hands of customers. In the longer term, new products are the life blood of most organisations, especially in the technology industries.

We only have to look e.g. Toyota and their TPDS (Toyota Product Development System) to see both an example of how this works in practice, and the huge benefits of the whole-organisation approach.

Quintessential product development organisations embrace a range of progressive ideas such as Prod•gnosis and Flow•gnosis.

– Bob

Further Reading

Marshall, R.W. (2013). Product Aikido. [online] Think Different Available at: https://flowchainsensei.files.wordpress.com/2013/04/productaikido041016.pdf [Accessed 13 Jan. 2022].

Mckenna, R. (1986). The Regis Touch: New Marketing Strategies for Uncertain Times. Reading, Mass.: Addison-Wesley Pub. Co.

Perri, M. (2019). Escaping The Build Trap: How Effective Product Management Creates Real Value. O’Reilly.

Ward, A.C. (1999). Toyota’s Principles of Set-Based Concurrent Engineering. [online] MIT Sloan Management Review. Available at: https://sloanreview.mit.edu/article/toyotas-principles-of-setbased-concurrent-engineering/. [Accessed 13 Jan. 2022].

Incompatible

Ever wondered why so many “Agile Adoptions” end up in the crapper?

Here’s the thing: Agile development, as described in the Agile Manifesto, and as aspired to by the gullible, is fundamentally and irredeemably incompatible with traditional management approaches (commonly known as command and control, Taylorism, Scientific Management, or some such).

This is neither a matter of opinion nor of experience (although I have many such experiences to relate), but of logic. I’ll not make the logical connections here, although I’m happy to do so if anyone is interested. I predict no one will be so interested.

This fundamental incompatibility is the reason we see so many failed Agile adoptions. Management is almost never going to swap out its existing mental models of how an organisation must be run, and so almost never will we see effective software development. (And almost no one seems in the slightest bit bothered).

BTW This incompatibility accounts for the approximately 80% failure rate of Agile adoptions we now know of.

The Gullible

The real kicker, though, is how Agile, as a local optimisation of the first order, will never deliver the benefits its proponents claim. Even those instances that have some impact on the traditional management worldview will only ever serve to enhance, and only marginally, the effectiveness of the development silo. And have little to no effect on the wider organisation. So much effort and risk of failure for so little gain.

The Alternative

I’m often asked “What’s the alternative, then, Bob?”. In a kind of despairing tone, as if it’s impossible to imagine a viable alternative at all.

I suggest at least one alternative is to look at the organisation as a (whole) system. And apply the precepts of flow to bring that system towards the Quintessential. If you need help with that, I’d be delighted to assist.

– Bob

ICYMI: The Future of Work

“The most important, and indeed the truly unique, contribution of management in the 20th century was the fifty-fold increase in the productivity of the MANUAL WORKER in manufacturing. The most important contribution management needs to make in the 21st century is similarly to increase the productivity of KNOWLEDGE WORK and the KNOWLEDGE WORKER.”

~ Peter F. Drucker

And in case you missed the original post:

The Future of Work

ICYMI: Product Development Flow

In case you missed it:

Product Development Flow

 

First Principles

I was reading the other day about how Elon Musk “reasons from first principles“. And I was thinking, “Well, d’oh! Doesn’t everyone do that? I know I do.” And then, upon reflection, I thought, “Hmm, maybe most folks don’t do that.” I certainly have seen little evidence of it, compare to the evidence of folks reasoning by extension, and analogy. And failing to reason at all.

Now, allowing for journalistic hyperbole and the cult of the celebrity, there may just still be something in it.

So, in case you were wondering, and to remind myself, here’s some first principles underpinning the various things in my own portfolio of ideas and experiences:

The Antimatter Principle

The Antimatter Principle emerges from the following basic principles about us as people:

• All our actions and behaviours are simply consequent on trying to get our needs met.
• We are social animals and are driven to see other folks’ needs met, often even before our own.
• We humans have an innate sense of fairness which influences our every decision and action.

Flowchain

Flowchain emerges from the following basic principles concerning work and business:

• All commercial organisations – excepting, maybe, those busy milking their cash cows – are in the business of continually bringing new products, or at least new product features and upgrades, to market.
• When Cost of Delay is non-trivial, the speed of bringing new products and feature to market is significant.
• Flow (of value – not the Mihaly Csikszentmihaly kind of flow, here) offers the most likely means to minimise concept-to-cash time.
• Autonomy, mastery and shared purpose affords a means for people to find the intrinsic motivation to improve things (like flow).
• Building improvement into the way the work works increases the likelihood of having sufficient resources available to see improvement happen.

Prod•gnosis

Prod•gnosis emerges from the following basic principles concerning business operations:

• All commercial organisations – excepting, maybe, those busy milking their cash cows – are in the business of continually bringing new products, or at least new product features and upgrades, to market.
• Most new products are cobbled together via disjointed efforts crossing multiple organisational (silo) boundaries, and consequently incurring avoidable waste, rework, confusion and delays.
• The people with domain expertise in a particular product or service area are rarely, if ever, experts in building the operational value streams necessary to develop, sell and support those products and services.  

Emotioneering

Emotioneering emerges from the following basic principles concerning products and product development:

• People buy things based on how they feel (their emotional responses to the things they’re considering buying). See: Buy•ology by Martin Lindstrøm.
• Product uptake (revenues, margins, etc.) can be improved by deliberately designing and building products for maximum positive emotional responses.
• Quantification serves to explicitly identify and clarify the emotional responses we wish to see our products and service evoke (Cf. “Competitive Engineeering” ~ Gilb).

Rightshifting

Rightshifting emerges from the following basic principles concerning work in organisations:

• The effectiveness of an organisation is a direct function of its collective assumptions and beliefs.
• Effectiveness is a general attribute, spanning all aspects of an organisation’s operations (i.e. not just applicable to product development).

The Marshall Model

The Marshall Model emerges from the following basic principles concerning work in organisations:

• Different organisations demonstrable hold widely differing shared assumptions and beliefs about the world of work and how work should work – one organisation from another.
• Understanding which collection of shared assumptions and beliefs is in play in a given organisation helps interventionists select the most effective form(s) of intervention. (Cf. The Dreyfus Model of Skills Acquisition).

Organisational Psychotherapy

Organisation psychotherapy emerges from the following basic principles concerning people and organisations:

• The effectiveness (performance, productivity, revenues, profitability, success, etc.) of any organisation is a direct function of its collective assumptions and beliefs about the world of work and how work should work.
• Organisations fall short of the ideal in being (un)able to shift their collective assumptions and beliefs to better align with their objectives (both explicit and implicit).
• Having support available – either by engaging organisational therapists, or via facilitated self-help – increases the likelihood of an organisation engaging in the surfacing, reflecting upon, and ultimately changing its collective assumptions and beliefs.

– Bob

Solutions Demand Problems

I’m obliged to Ben Simo (@QualityFrog) for a couple of recent tweets that prompted me to write this post:

I very much concur that solutions disconnected from problems have little value or utility. It’s probably overdue to remind myself of the business problems which spurred me to create the various solutions I regularly blog about.

FlowChain

Problem

Continually managing projects (portfolios of projects, really) is a pain in the ass and a costly overhead (it doesn’t contribute to the work getting done, it causes continual scheduling and bottlenecking issues around key specialists, detracts from autonomy and shared purpose, and – from a flow-of-value-to-the-customer perspective – chops up the flow into mini-silos (not good for smooth flow). Typically, projects also leave little or no time, or infrastructure, for continually improving the way the work works. And the project approach is a bit like a lead overcoat, constraining management’s options, and making it difficult to make nimble re-adjustments to priorities on-the-fly.

Solution (in a Nutshell)

FlowChain proposes a single organisational backlog, to order all proposed new features and products, along with all proposed improvement actions (improvement to the way the work works). Guided by policies set by e.g. management, people in the pool of development specialists coalesce – in small groups, and in chunks of time of just a few days – around each suitable highest-priority work item to see it through to “done”.

Prod•gnosis

Problem

Speed to market for new products is held back and undermined by the conventional piecemeal, cross-silo approach to new product development. With multiple hands-offs, inter-silo queues, rework loops, and resource contentions, the conventional approach creates excessive delays (cf cost of delay), drives up the cost-of-quality (due to the propensity for errors), and the need for continual management  interventions (constant firefighting).

Solution (in a Nutshell)

Prod•gnosisproposes a holistic approach to New Product Development, seeing each product line or product family as an operational value stream (OVS), and the ongoing challenge as being the bringing of new operational value streams into existence. The Prod•gnosis approach stipulates an OVS-creating centre of excellence: a group of people with all the skills necessary to quickly and reliably creating new OVSs. Each new OVS, once created, is handed over to a dedicated OVS manager and team to run it under day-to-day BAU (Business as Usual).

Flow•gnosis

Problem

FlowChain was originally conceived as a solution for Analytic-minded organisations. In other words, an organisation with conventional functional silos, management, hierarchy, etc. In Synergistic-minded organisations, some adjustments can make FlowChain much more effective and better suited to that different kind of organisation.

Solution (in a Nutshell)

Flow•gnosis merges Prod•gnosis and FlowChain together, giving an organisation-wide, holistic solution which improves organisational effectiveness, reifies Continuous Improvement, speeds flowof new products into the market, provides an operational (value stream based) model for the whole business, and allows specialists from many functions to work together with a minimum of hand-offs, delays, mistakes and other wastes.

Rightshifting

Problem

Few organisations have a conscious idea of how relatively effective they are, and of the scope for them to become much more effective (and thus profitable, successful, etc.). Absent this awareness, there’s precious little incentive to lift one’s head up from the daily grind to imagine what could be.

Solution (in a Nutshell)

Rightshifting provides organisations with a context within which to consider their relative effectiveness, both with respect to other similar organisations, and more significantly, with respect to the organisation’s potential future self.

The Marshall Model

Problem

Few organisations have an explicit model for organisational effectiveness. Absence of such a model makes it difficult to have conversations around what actions the organisation needs to take to become more effective. And for change agents such as Consultants and Enterprise Coaches attempting to assist an organisation towards increased effectiveness, it can be difficult to choose the most effective kinds of interventions (these being contingent upon where the organisation is “at”, with regard to its set of collective assumptions and beliefs a.k.a. mindset).

Solution (in a Nutshell)

The Marshall Model provides an explanation of organisational effectiveness. The model provides a starting point for folks inside an organisation to begin discussing their own perspectives on what effectiveness means, what makes their own particular organisation effective, and what actions might be necessary to make the organisation more effective. Simultaneously, the Marshall Model (a.k.a. Dreyfus for Organisations) provides a framework for change agents to help select the kinds of interventions most likely to be successful.

Organisational Psychotherapy

Problem

Some organisations embrace the idea that the collective organisational mindset – what people, collectively believe about how organisations should work – is the prime determinant of organisational effectiveness, productivity, quality of life at work, profitability, and success. If so, how to “shift” the organisation’s mindset, its collective beliefs, assumptions and tropes, to a more healthy and effective place? Most organisations do not naturally have this skill set or capability. And it can take much time, and many costly missteps along the way, to acquire such a capability.

Solution (in a Nutshell)

Organisational Psychotherapy provides a means to accelerate the acquisition of the necessary skills and capabilities for an organisation to become competent in continually revising its collective set of assumptions and beliefs. Organisational Psychotherapists provide guidance and support to organisations in all stages of this journey.

Emotioneering

Problem

Research (cf Buy•ology ~ Martin Lindstrom) has shown conclusively that people buy things not on rational lines, but on emotional lines. Rationality, if it has a look-in at all, is reserved for post-hoc justification of buying decisions. However, most product development today is driven by rationality:

• What are the customers’ pain points?
• What are the user stories or customer journeys we need to address?
• What features should we provide to ameliorate those pain points and meet those user needs?

Upshot: mediocre products which fail to appeal to the buyers’ emotions, excepting by accident. And thus less customer appeal, and so lower margins, lower demand, lower market share, and slower growth.

Solution (in a Nutshell)

Emotioneering proposes replacing the conventional requirements engineering process (whether that be big-design-up-front or incremental/iterative design) – focusing as it does on product features – with an *engineering* process focusing on ensuring our products creaate the emotional responses we wish to evoke in our customers and markets (and more broadly, in all the Folks That Matter™).

The Antimatter Principle

Problem

How to create an environment where the relationships between people can thrive and flourish? An environment where engagement and morale is consistently through the roof? Where joy, passion and discretionary effort are palpable, ever-present and to-the-max?

Solution (in a Nutshell)

The Antimatter Principle proposes that putting the principle of “attending to folks’ needs” at front and centre of all of the organisation’s policies is by far the best way to create an environment where the relationships between people can thrive and flourish. Note: this includes policies governing the engineering disciplines of the organisation, i.e. attending to customers’ needs at least as much as to the needs of all the other Folks That Matter™.

– Bob

Holistic Solutions for Product Development Businesses

Several people have been in contact this week to say “It’s all very well talking about holistic solutions for software development, but who really has any notion of what that might even look like?”

Which is a fair question.

Aside: Let’s note that the phrase “holistic solution for software development” is an oxymoron of the first order. By definition, software development is but part (and generally a small part) of any solution that addresses customer or market needs. Even software “pure plays” have a lot of non-software components.

So, I thought I’d describe just one holistic solution for whole organisations that develop new products containing some software. For want of a better name, I’ll call it this example “Flow•gnosis”. With this example, I hope that maybe one or two readers might find a spark of insight or inspiration to broach the question of a True Consensus in their own organisation.

I’ll try to keep this post brief. I’d be happy to come talk with you about holistic product development solutions for organisations, in person, if you have any real interest.

Flow•gnosis?

Flow•gnosis is a hybrid born of FlowChain and Prod•gnosis. FlowChain is not specifically a holistic solution, focussing as it does on improving the flow of knowledge work through a development group or department whilst moving continuous improvement “in-band”. Prod•gnosis is specifically a holistic solution for effective, organisation-wide product development, but says little about how to organise the work for e.g. improved flow or continuous improvement.

Together, they serve as an exemplar of a holistic solution for knowledge-work organisations, such as software product companies, software houses, tech product companies, and organisations of every stripe with in-house product development needs.

Let’s also note that Flow•gnosis is just an example, to illuminate just a few aspects of a holistic solution. Do not under any circumstances consider copying it or cloning it. Aside from the lack of detail presented here, you’d be missing the whole point of the challenge: building a True Consensus, as a group, with your people, in your context.

What’s the Problem?

Before talking more about a solution, what’s the problem we’re trying to solve?

I’ll work through an almost universal problem facing organisations that “do” software development. A problem that some enterprising supplier or management team might choose to address with an “innovative” solution.

Understanding the Problem

Here are some typical Undesirable Effects (UDEs) I hear from many companies involved in software and product development:

UDE: Delivery is not fast enough (long time-to-market)

UDE: New products cost too much to develop (high cost to bring to market)

UDE: We struggle to keep our products at the cutting edge

UDE: We always drop balls in hand-offs of tasks (i.e. between specialists, or business functions)

UDE: Continual contention for in-demand specialists

(For the sake of brevity, I’ll skip the building and analysis of the cause and effect graph. Your analysis will be different, in any case).

Root cause: Developing new software, products and services through a byzantine labyrinth of tasks and hand-offs, between and across dozens of specialisms within the company and its network of suppliers and distributors. This approach causes many delays (waste of time), mistakes (waste of effort, money), contention of resources, etc.

Conflict: A) Specialists must work together in their own specialist groups or silos to maintain their cutting-edge skills and know-how. B) Specialists from across all specialisms must work together as a group else handoffs and queues will cause many delays and mistakes.

Conflict arrow: Specialists cannot work in their own groups advancing their specialist knowledge at the same time as they work together with other kinds of specialist on new product ideas.

Flawed assumption at the root of the conflict: Specialists must work together. What makes this so? Only the old rules of the organisation. That silos (a.k.a. business functions) “own” their clutch of specialists. What if we changed that rule? (Note: this change is one key element of Toyota’s TPDS). And if we changed that rule, what other rules would we need to change too?

A Bird’s Eye View of Flow•gnosis

Prod•gnosis looks at an organisation as a collection of parallel Operational Value Streams, each dedicated to the selling and support of one of the organisations’ (whole) product or service lines. And each with its own team or collection of people doing the daily work of that operational value stream. Further, Prod•gnosis asks “How do these operational value streams come into being?” And answers with “They are made/created/developed by a dedicated Product Development Value Stream.”

FlowChain describes a way of working where a pool of self-organising specialists draws priority work from a backlog, executes the work, and delivers both the requested work item, and posting new work items (for improving the ways the work works) into the backlog. In this way, FlowChain both improves flow of work through the system, and brings continuous improvement “in-band”

By merging Prod•gnosis and FlowChain together into Flow•gnosis, we have an organisation-wide, holistic example which improves organisational effectiveness, reifies Continuous Improvement, speeds flow of new products into the market, provides an operational (value stream based) model for the whole business, and allows specialists from many functions to work together with a minimum of hand-offs, delays, mistakes and other wastes.

The latter point is perhaps the most significant aspect of Flow•gnosis. Having customer, supplier, marketing, sales, finance, logistics, service, billing, support and technical (e.g. software, usability, emotioneering, techops, etc.) specialists all working together (cf. Toyota’s Obeya or “Big Room” concept) enables the evolution of “Mafia Products” and “Mafia Offers” which the more traditional silo-based models of business organisation just can’t address effectively.

Out With the Old Rules, In with the New

Flow•gnosis is an innovation. Irrespective of the promised benefits of Flow•gnosis, we have learnt in recent posts that adopting an innovation ONLY brings benefits when we change the rules.

Let’s apply the four questions to our Flow•gnosis innovation and see what rule changes will be necessary to truly reap the benefits.

Q1: What is the POWER of Flow•gnosis?
A1: Flow•gnosis makes it commercially feasible for a company to repeatably come to market with new “Mafia Products”.

Mafia Product: “A product (or service) so compelling that your customers can’t refuse it and your competition can’t or won’t offer the same.”

Q2: What limitation does Flow•gnosis diminish?
A2:  Serialisation of specialist work (passing things back and forth between various specialist and business functions).

Q3: What existing rules served to help us accommodate that limitation?
A3: Handoffs. Queues. Batches. Separation of command and control from the work. Process. Process conformance. Local measures. Constant expediting. Critical path planning (Gannt charts, WBS). Local optima. Functional management (discrete management of each separate business function).

Q4: What (new) rules must we use now?
A4: Flow. Value streams. SBCE. Holistic measures. Ubiquitous information radiators. Cost Of Delay prioritisation. Buffer management. Constant collaboration. Dedicated teams of generalising specialists. Self-organisation. In-band continuous improvement. Resource levelling. Limits on WIP.

Summary

This post has been a – necessarily brief – look at one holistic solution to (software) product development. Crucially, we have seen how old rules have to be replaced, and what many of the replacement new rules might look like.

I invite you to remember that Flow•gnosis is just an broad-brush example of a holistic solution. Please don’t consider copying it or cloning it. And remember the real challenge: building, as a management group, a True Consensus.

– Bob

Further Reading

Meeting Folks’ Needs At Scale – Think Different blog post

The Simplest Thing That Could Possibly Work

Here’s an excerpt (pp 206) from “Birth Of the Chaordic Age” by Dee Hock, about “an odd project management scheme” they adopted in the early days – circa 1974:

“Swiftly, self-organisation emerged. An entire wall became a pinboard with every remaining day calendared across the top. Someone grabbed an unwashed coffee cup and suspended it on a long piece of string pinned to the current date. Every element of work to be done was listed on scraps of paper with the required completion date and the name of the person who had accepted the work. Anyone could revise the elements, adding tasks or revising dates, providing they coordinated with others affected. Everyone, at any time, could see the picture emerge and evolve. They could see how the whole depended on their work, and how their work was connected to every other part of the effort. Groups constantly assembled in front of the board as need and inclination arose, discussing and deciding in continuous flow; then dissolving as needs were met. As each task was completed its scrap of paper would be removed. Each day, the cup and string moved inexorably ahead.”

I’m struck by the similarities with FlowChain, and as with FlowChain, it seems an exemplar of simplicity and flow. I also note the implicit “Advice Process” vibe, and the emphasis on “making the work visible” (Cf Personal Kanban).

– Bob

Further Reading

Birth of the Chaordic Age ~ Dee Hock

Meeting Folks’ Needs At Scale

Scaling Agile is one of those oxymorons that has, nevertheless, consumed endless column-inches and hours of debate. I have no time for it. Agile was meant for development-in-the-small. For teams of five to seven people, give or take. Even at that “scale”, I have serious reservations about its efficacy and effectiveness. The idea of scaling it up to multiple dependent teams, or even to whole development departments or groups of hundreds or thousands of people, seems just crazy.

The Demand

Yes. There are organisations with hundreds or thousands of developers working on more or less dependent things. Huge systems. Ginormous products. And these organisations have problems. Boy, do they have problems. They’ve had the same kind of problems for decades now. Advances in tech and tools seems to have made little dent in those problems. Similarly, advances in methods and processes have barely scratched the surface.

So there is a demand. A demand for something better. A demand for a packaged solution that can be simply (ha!) “installed” and run with. And the folks with the problems have money. Lots of money. Bags of moolah.

No wonder it looks like an interesting problem to solve. The thing is, I don’t see many people, either on the demand or the supply side, that actually understand the nature of the problem.

Absent an understanding of the problem, and given their desperation, the demand side will embrace any and all solutions that bear even a vague whiff of credibility. And the “Agile” label these days confers that smell.

Needs

All product development is, in essence, an exercise in attending to folks’ needs. The more kinds of folks with needs, the more of their needs we bring into scope, and the quicker we want to see those needs met, the more people we might choose to commit. The question of agile (or not) is a huge irrelevance. Things like communications overheads, coordination, partitioning of needs, a regular cadence, flow, and effective use of resources (time, money, equipment) hold sway. TPDS had these issues sorted out (mostly) years ago with e.g. the Obeya or “big room” concept, set-based concurrent engineering, etc.

I have suggested that something like FlowChain might afford an effective model for organising to meet folks’ needs at scale. I have yet to hear anyone suggest why this model is flawed. Until that day, I will continue to invite you to consider its merits.

The demand side continues to have needs that are not being well-met (due to e.g. a host of pathological beliefs). The supply side has needs that are being more-or-less well met by selling solutions largely disconnected from the real problem. Scaling agile seems to me a very one-sided, cynical and exploitative bargain.

– Bob

Further Reading

Moving Past the Scaling Myth ~ Michael Feathers
Lean Product and Process Development ~ Allen Ward

Emergence

You may have noticed that I write regularly about the different mindsets that explain the relative effectiveness of the organisations we work for and with. Things like Theory-X (strong in the Ad-hoc and Analytic mindsets) vs Theory-Y (Synergistic and Chaordic mindsets), and organisations-as-machines vs organisations as social/biological/complex adaptive systems.

One difference I have not touched on much is the part that emergence has to play in the effective organisation.

Gall’s Law

“A complex system that works is invariably found to have evolved from a simple system that worked. The inverse proposition also appears to be true: A complex system designed from scratch never works and cannot be made to work. You have to start over, beginning with a working simple system.”

~ John Gall

Does Your Organisation Embrace or Ignore Gall’s Law?

Ad-hoc and Analytic minded organisations generally believe that systems are best designed, up front, with acts of conscious will and intent. Be they organisational structure, policies, products or a myriad of other systems upon which an organisation depends.

Synergistic organisations learn, by degrees, that John Gall nailed it – complex systems that work require evolution from simple systems that work. For effective (working) organisations, we need to embrace emergence. We need to allow our systems – and our thinking – to evolve to the point where emergence is working for us. This is hard.

Messy

Emergence seems messy. Allowing things to take their own course is hard for folks who seek certainty and control as means to getting their needs met. It can often feel like a crowd of people trampling over your nice, neat, manicured lawns. But the properties of beauty and simplicity can emerge more or less unbidden, too.

Whilst we opposed emergence, we lock ourselves into relatively ineffective ways of thinking, and thus, of working.  Only when we embrace and encourage emergence, do we open the door to more effective ways of thinking and working.

FlowChain

One of the fundamental guiding principles of FlowChain is to encourage emergence:

• Emergence of products
• Emergence of teams
• Emergence of methods (“the way the work works”)
• Emergence of systems
• Emergence of priorities
• Emergence of flow
• Emergence of needs (and e.g. stakeholders)
• Emergence of purpose (the “why”)
• Emergence of ideas (i.e. creativity)

Would you be willing to consider, and share, where your organisation is at regarding the role of emergence?

“My ideas have undergone a process of emergence by emergency. When they are needed badly enough, they are accepted.”

~ R. Buckminster Fuller

– Bob

Further Reading

Obliquity ~ John Kay
Systemantics ~ John Gall

Out Of House FlowChain

When I conceived of FlowChain, some six years ago now, my immediate context was development shops with their own in-house developers, and other supporting staff.

But it strikes me that with just a few adjustments, it’s also suitable for organisations that sub-contract out most or all of their development projects to third parties.

These adjustments centre around arranging for the various third parties (assuming, in the likely case, that there’s more than one) to each contribute staff to the “Pool” (see diagram). These arrangements include:

Commercial

How will the third parties be paid? Some UK government functions use function points as a measure of “work done”, with a set price for each function point “delivered”. See: Output-based contracts. I can imagine other contractual arrangements, too.

Social

How will the third parties’ staff integrate or form healthy relationships with the in-house commissioning staff (a.k.a. product owners)? Will there be shared spaces? Regular visits to and fro? Some more technical forms of communication (Twitter, chat, video conferencing, etc.)? Remember, each backlog item is sized for two to three people working together for two to three days.

Tooling

Third parties remain free to pull items from the backlog as they see fit (just as with in-house FlowChain), and use their own tools, languages, etc.. I foresee some advantages in having a common code repository, coding and other standards, agree requirements around test suites, and so on.

Delivery Into Production

Maybe the organisation contracting the third parties has its own Ops department. In this case the interface between development (teams, third parties) and Ops would probable best be standardised and agreed (like an API). If the third parties have their own Ops folks, then they can do DevOps in their own space and time, and serve the “production” services – or even micro-services – they each operate, directly to users.

Shared Backlog

For clarity, this variant of Flowchain retains the enterprise-wide backlog, with user stories, improvement stories, etc. being prioritised by some black box (or white box) prioritisation algorithm, committee, manager, or whatever. The only real change is in how the Pool is constituted. Note: I see no particular reason why the general FlowChain principle of “ANY unassigned development folks from the Pool can coalesce around each new top backlog item” cannot stand, here.

There may even be emergent advantages in having e.g. developers from different third parties coming together to collaborate on specific backlog items. How weird would that be? Again, policy would guide folks’ actions here.

Who would “manage” the backlog?  This could be done by a small in-house staff, or itself subcontracted out to one or more third parties. Note: the backlog in FlowChain is largely self-managing, in any case, given an effective prioritisation algorithm or approach.

Flow

Flow (of e.g. software into the hands of those whose needs are being attended to) remains the key objective of the whole approach.

Growing An In-house Capability

For organisations without an in-house development capability, this approach provides a simple(r) path to establishing and growing an in-house development capability. In-house developers can be added, one by one, as and when circumstances (budgets, priorities, etc) allow. These new folks can work alongside – and learn from – third-party staff already used to Pool working, and the balance between in-house and out-of-house staff, skill sets, etc. adjusted dynamically as needs dictate.

Drawbacks

The key drawback I foresee is in the matter of dev-ops integration (DevOps). This could prove more difficult, in the case where developers, etc. are out-of-house and Ops in-house. This seems a special case of the issues of outsourcing and offshoring, generally. But I’m sure a bunch of smart developers and smart ops can work this out, especially with some help and guidance.

– Bob

NeedsFlow

[Tl;Dr: The essence of software and product development is about continuously exploring and meeting folks’ needs. How about as in industry we stop shilly-shallying and place this reality at the core of our approach to development?]

The Essence Of Software And Product Development

Essentially, every approach to software and product development is about a flow of things (including services) whereby a development team, group or organisation attempts to satisfy some selection of folks’ unmet needs.

In batch-and-queue (e.g. Waterfall, etc.) approaches, needs are batched-up into large batches, and flow (not very effectively) through a sequence of queues. Each batch (in the pathological case, only one) eventually gets dropped onto those assumed to have said needs, and that’s about it.

In iterative (e.g. Scrum, Kanban, etc.) approaches, needs are batched up into many, relatively smaller, batches, and flow (rather more effectively) through a sequence of queues (at its simplest, e.g. Backlog, Doing, Done). Each batch gets dropped onto the folks assumed to have said needs, and these folks get the chance to say if their needs have been met well enough, or not. When budgets and timescales allow, a later iteration can then have another go at better meeting those needs deemed to have been poorly-met, along with a new attempt to address any other priority needs that might have emerged along the way.

Different approaches differentiate themselves on e.g. batch sizes, selecting of the kinds of folks whose needs will be attended to, cadence and strategies.

Needs Trump Value

Why Needs flow? Why not just stick with the more common “value” flow? Well, for me there’s a whole bunch of issues around the notion of value. Including:

• Value for whom?
• Who gets to specify how any particular “value” is measured or quantified (most often folks ill-equippped to do so).
• The whole notion of “value” (often overlooked, often misunderstood).
• Value is too often though of in simple economic or financial terms. Human beings perceive value differently, and much more richly (e.g. emotionally).
• How can we tell when we’ve delivered “value”?

So, I propose that needs trump value. And therefore, that when focussing on flow, it’s more useful – and effective – to focus on the flow of needs.

The above diagram shows a system boundary – this could be the whole organisation, the development organisation, or just one development team. Bundles of external needs flow into the system, and merge with other bundles of needs coming from inside the system. These bundles pass though various stages (and queues) where “development” strategies are uses in moving the bundles from stage to stage, and from queue to queue. Ultimately, candidates (things we hope will meet the needs that entered the pipeline) flow back to external parties, and also to internal stakeholders, such as the developers themselves.

Impediments

This perspective allows for the simple integration of dealing with impediments. Things that are impeding e.g. flow or the outflow of effective candidates may themselves be needs. And may enter the pipeline much as any other needs. Of course, there may be some( or many) such impediments than no one has any need of dealing with. These may be filed or ignored.

Abstract FlowChain

Whilst FlowChain was conceived to be awesomely effective at handling bundles of e.g. User Stories, Use Cases, Improvement Stories, etc., it can very easily embrace the unit of flow being bundles of needs, or – in the single-piece-continuous-flow scenario – individual needs.

Limiting WIP, Making Things Visible

From the diagram, you may begin to see how we might simply make the flow of needs visible, should we so choose. And personally, I’d like to make visible the ultimate fate of the candidates too, with a little expansion of the scope of the diagram.

And the queues (particularly the first) can serve as a convenient means to limit Work In Progress (a.k.a. Needs in Progress).

No Projects

It may not be immediately obvious, but the NeedsFlow perspective implicitly removes the need for projects. NeedsFlow decouples development (flow) from products and product lines (each of which meet a more or less related collection of needs).

– Bob

No Projects

The idea of “projects” in software and product development is a glaring anachronism, traceable back to the days when organisations saw each new project as “the last one we’re ever likely to run”. Absent the expectation of ever running another project, why bother moving to a more continuous, flow-based (non-project) set-up?

And of course, the idea of breaking things down into parts, and managing those parts separately, is another glaring anachronism, and one still grasped so tightly by those of the Analytic mindset.

But even the briefest of reflections about the nature of development work in organisations reveals a simple truth: just about every organisation today has a more or less continual flow of work – of new ideas transforming into products, of concepts becoming cash revenue generators.

I won’t dwell further here on the case for #NoProjects – Grant made the case quite well in his piece “What’s Wrong With the Project Approach?”. Maybe you’d like to consider the relative (dis)merits of “projects” – compared to a more flow-oriented approach?

May I just invite you to consider whether there may be other, maybe better ways of getting folks’ – and organisations’ – needs met?

And, by the by, offer up FlowChain as a simple – and complete – example of what I’m talking about in terms of one such better way.

– Bob

Seven Changes To Improve Flow In Your Software Development Process

Many folks drinking the Lean coolade seem to believe that removing waste is at the heart of the Lean approach. I beg to differ. I’d say that improving flow is the heart of Lean.

Here’s seven ways in which your team or, preferably, your organisation as a whole, might go about improving flow:

1. Adopt a small thing as the universal unit of work. And by universal, I mean some unit of work that everyone across the whole organisation can recognise and adopt. This could be Use Cases, User Stories, or something else. Just keep the “small thing” small (never more than a couple of days work for a couple of people). cf Heijunka, FlowChain.
2. Make flow visible. In particular, make e.g. queues, queuing times, and end-to-end cycles times visible for all to see.
3. Know your WIP (work in progress) and work to reduce (limit) it. Cf. Little’s Law.
4. Use demand to “pull” units of work through the system (as opposed to “pushing” work through).
5. Eliminate – or at least minimise – hand-offs. That is, having work pass from one specialist to another. Each hand-off typically introduces another queue, with the inevitable costs and delays. One way to do this involves multi-disciplinary teams, or better still, up-skilling individuals so each person can competently take on a variety of specialist tasks.
6. Identify the goal; understand demand (by various means – for example follow individual “demands” through the system, end-to-end;) identify the constraint; and apply the Five Focussing Steps (repeatedly). Cf. Theory of Constraints
7. Experiment continually: trial possible improvements to flow, one by one, to assess their actual efficacy, in isolation from one another. Cf. PDCA a.k.a. the Shewhart Cycle.

And of course, none of the above suggestions will do much good, or even get acted on, unless and until the folks doing the work internalise a basic appreciation of the very notion of flow. And that’s unlikely to happen unless and until the work environment supports and nurtures folks’ curiosity and innate desire to do a good job.

Further Reading

The Principles of Product Development Flow ~ Donald G. Reinertsen
Seven Changes To Remove Waste From Your Software Development Process ~ Cecil Dijoux
Product Development Flow ~ FlowchainSensei
LondonCD Talk based on this post ~ Video
Getting People to Limit Their Work In Progress ~ Ben Linders

A System of Continuous Improvement

 

“If continuous improvement isn’t in-band then realistically it ain’t gonna happen.”

 

This post is about FlowChain (which btw, is the inspiration for my Twitter handle, @flowchainsensei). Not only does FlowChain afford a means for moving continuous improvement in-band, but also does away with the need for projects, and offers a means for dramatically improving product development flow.

These changes mean:

• Shortest possible concept-to-cash times.
• Steady, reliable flow of new features into the market.
• Earliest possible return on product/software development investment.
• Standardised, reliable numbers to manage by.
• No more project overheads.
• Simple coordination of work streams (no more PMO overheads).
• Improved business agility.
• Compatible with Agile development (team) methods.
• Your highly-utilised specialists are always working on the company’s most important (highest cost-of-delay) features.

Continuous Improvement

A.K.A. Continual Improvement – for those who prefer that variant.

I wrote last year about continuous improvement as a vaccine. This was an attempt to raise the profile of the value of a systemic approach to making organisations work better and better (more and more effectively). In a related post I’ve explained how it’s fruitless to try to change people, but very fruitful to change the way the work works (“the system”) so as to effect changes in folks’ behaviours, without any coercion, obligation or other forms of violence (in that the folks doing the work are the same folks that choose and effect the changes to their collective way of working). And with transparency – both in terms of intent (to change behaviours) and so that everyone can see what is going on, every minute of every day.

FlowChain Basics

In FlowChain we see these themes (and others, not covered here) woven together. Flowchain illustrates a system of work in which continuous improvement is integrated right there into the way the work works, on a daily basis. The following diagram illustrates the general FlowChain idea:

Here we see the enterprise-wide backlog. This contains all the work-items – a.k.a. deliverables – the enterprise is presently interested in producing. Through BAU, the Operational Value Stream Owners, engaged as they are, daily, in dealing with the constraints (cf. TOC) in their respective value streams, make requests for changes to their products, line-of-business applications, and so on. Let’s assume they choose to do this through a succession of “User Stories”. The enterprise backlog, then, allows the organisation as a whole to review and prioritise this workload – most likely according to some set of agreed policies. (We might choose to think of this as black-box backlog prioritisation).

The folks who choose to participate in the Pool (of e.g. “engineers” – in the most general sense), when not working on an existing item, are free to “pull” an item from at or near the head of the enterprise backlog and start working on it. These folks will get to know that:

• Their skills and experience – plus personal interests and enthusiasms – will suggest which items they might best pull.
• Some folks might like to work with the same folks on each successive item (standing teams).
• Some might choose to work with different folks from item to item (fluid, self-assembling teams).
• Some might choose to flit from one mode to another, item by item, as the fancy takes them.

Note: As a rule of thumb, each work item will be of a size requiring the efforts of around three to four people for one to three days. There may be benefits in making work items a consistent size, and even in balancing the flow through e.g. Heijunka

Aside: The nature of FlowChain is also informed by an old Familiar policy:

“Absolutely no work in the organisation is done off-plan.”

Flow

The yellow arrows in the above diagram illustrate the flow of i.e. product development work through the enterprise. As each e.g. user story is pulled from the backlog and worked-on by folks from the pool, ideas and innovations pertinent to improving the way the work works – most likely item-dependent – may be generated. These potential improvement ideas are captured in the form of e.g. “improvement stories”.

(Remembering that each “story” is a placeholder for a conversation.)

These improvement stories then also find their way into the enterprise backlog (the blue arrow shows this route). This allows the enterprise – through the backlog prioritisation algorithm (set of policies) – to balance and adjust the flow of user stories and improvement stories in real-time.

The backlog “management” may also include visualisation of the demand, work and flow – through e.g. an enterprise kanban board with kanban (cards) for the user stories and improvement stories both.

The Holy Grail – In-band Continuous Improvement

In this scheme of things, then, we integrate continuous improvement into the heart of the flow of BAU. This allows Flowchain-type organisations to square the circle and effectively serve the two masters of production and improvement with full transparency, non-violence, real-time flexibility and control.

Incidentally, this also allows FlowChain organisations to do away with the whole rotten edifice of projects, programmes, Programme Offices, internecine strife over resources, and so on.

My thanks again to all those folks that have made this post possible. I’d love to hear your thoughts.

– Bob

Ambitions

Not in the sense of being ambitious, but in the sense of having some things in mind that I’d like to see come to pass. You might call that an agenda. Or some needs which, upon being met, might make my life more wonderful…

Whatever you choose to call it, here’s my list:

• An implementation of FlowChain. I’ve had this model rattling round inside my brain for years now. I see little prospect of someone else taking the necessary leap of faith and implementing it – although the Reaktor folks seem to have evolved something similar, independently –  so it’s down to me.
• An illustration of just how much like product development is software development. And an illustration of the value – and relevance – of applying decades of well-evolved product development practices to the betterment of software development and a business.
• A systems thinking approach to a) product development and b) (more ambitious) running a business. I refer to this as “Prod•gnosis“.
• Learning how practical all the above ideas really are, in the crucible of real life. And how much – and in what regards – they need modifying when they come into contact with the “enemy’s main strength”.

“No plan of operations extends with certainty beyond the first encounter with the enemy’s main strength.”

~ Helmut von Moltke

This is my agenda.

As I explained recently in my post “Introducing Rightshifting“, I have no desire to foist these things upon people. But (in the context of my new job) I HAVE been asked to bring my experience and insights to bear. And to “innovate”. The above list seems to be a start, at least.

And just in case you’re wondering about my motivation, it’s not all selfish. I’ve made no secret over the years about what drives my work: to see an end to the egregious waste of human potential happening in software organisations everywhere, today. I believe the above ideas, implemented, will contribute significantly to this aim.

If someone asked me what needs of theirs their participation might serve, I’d offer the following list of possibilities:

• The opportunity to learn lots of new things.
• A chance to master the art of software development (esteem).
• Making a difference. Advancing the art, illuminating the possible, and inspiring others.
• The prospect of much fellowship, positive stress, self-actualisation (cf Maslow) and  fun!

Am I an idealist? A dreamer? You may choose to make that judgement. Although such a choice (i.e. to judge) would make me feel sad.

“Observing without evaluating is the highest form of human intelligence.”

~ Jiddu Krishnamurti

So who’d like to join me on this journey? How do you feel about my agenda? What’s your agenda? What needs can we meet, together? What might make your life more wonderful? And how might we help each other?

– Bob

Further Reading

Prod•gnosis in a Nutshell – blog post

What’s Wrong with the Project Approach?

Some time back the late, great Grant Rule wrote this paper on the problems with “projects” as an approach to organising software development. As the original has now ceased to be, I’ve taken the liberty of reproducing it here for posterity:

What’s wrong with the project approach to software development?

January 11th, 2011
Author: pg_rule

Pretty much since “software” was first invented (60 years ago?), numerous folk have been promoting an ‘engineering-led’ approach to ‘software projects’. Yet this advice goes largely unheeded, with the result that the relative success of IT projects is poor, and has improved very little during all my years in IT (38 and counting). Given that such admonishments seemed to have had such little effect in all that time, I also find myself asking, “Do I think it likely that further exhortations to those involved in ‘software projects’ to change their project practices is likely to achieve improved value delivery to stakeholders?”

And I have to conclude that the answer is “No”.

Following Albert Einstein’s adage that, “The definition of insanity is doing the same thing over and over again and expecting different results”, it seems to me that we need an entirely new approach. A new approach which goes to the root causes of what actually goes wrong in the end-to-end process. Why are the honest endeavours of software developers often so disconnected from the delivery of customer and stakeholder value?

Observation of what actually happens in organisations suggests there are two root conditions to the problem:

1. Those responsible for business strategy are disengaged from, and know relatively little about, software-intensive systems and technology. So they structure their organisations so that software & technology people are segregated into silos. In those silos, the inmates talk amongst themselves in whatever arcane language they choose. But importantly, they don’t communicate (or interfere) with the ‘real business’.
2. Everyone conspires to pretend that software-related activities should be managed as ‘projects’. That is, as chunks of work that start and end (on more or less clear and agreed dates), that have more or less well-defined goals, that contain a list of activities (tasks) that are assigned by a ‘project manager’ to a project team to which ‘resources’ are assigned for a limited period.

The results of studies too numerous to mention shows that most software projects are ‘challenged’ or ‘fail’. One study suggested that the majority of experienced project managers (and I am sure, folk playing other roles) expect at least 1 in 3 of the projects they lead to fail! As systems become more complex, and larger, they employ more teams combining projects into programmes… which further reduces the likelihood of successful achievement of the overall goals.

My conclusion is that we need a complete change of mindset. We need to move away from the inherently batch & queue concept of the ‘project life-cycle’ (as promoted by organisations including BCS, APM, PMI, OGC, SEI, NCC, ISO, IEEE, IET, etc. etc.) to a different approach.

I suggest that the required new mindset will accommodate the ideas of flow production and lean systems thinking that first began to be developed systematically (in e.g. automotive engineering) around 100 years ago. (Of course, one can trace elements of flow production & lean at least back to Carthage c.300-200 BC, but let’s skip over the history for now.)

I posit that Tom Gilb’s Evo method, and other agile methods such as XP, Scrum, Flowchain, and software Kanban, etc., begin to achieve ‘better’ results compared to ‘traditional’ big-design-up-front, wholly batch & queue methods, precisely because they encourage workers to focus on smaller batches of stakeholder value. In other words, value in terms the software developer can get to grips with.

Agile methods are one or more steps nearer to the ideal of ‘single piece continuous flow’. BUT… they are inherently limited because they continue to create & disband teams, to establish & abandon value streams, to create & throw away know-how, at – it seems – every opportunity. And crucially, they allow the C-suite and ‘business-side’ managers to ignore their responsibilities for the system of work and for the desired outcome.

Flow production (toward which Evo, Flowchain and Kanban currently make the nearest approaches IMO) would:

A) Make the entire end-to-end, whole-life, ‘concept to consumption & retirement’ process of defining, deciding, acquiring, designing, developing, operating, supporting, maintaining & replacing software- intensive systems a visible, inherent part of normal business operations… forcing issues onto the management horizon so they can be addressed as business issues – and not just something technologists worry about.

B) Because it would be apparent that software & IT issues were causing interruption to (or even cessation of) the flow of value, C-suite executives would have to recognise the pressing need to engage with software & IT related issues just as much as they do with other kinds of business issue. Conversely, the engagement of the ‘systems and software engineers’ with ‘the business’ would also be stimulated, the role of each and the communication between them finally becoming acknowledged as a main artery of the organisation’s lifeblood.

Flow production can only work effectively with the active engagement of all involved. For this reason it is a far more sustainable business model than other, perhaps more familiar, approaches. It embeds the ability to flex and respond to market forces deeply into the organisational culture. The focus on ‘the unforgiving minute’ forces constraints and problems out into the open where everyone can see them. It won’t allow problems to be swept under the carpet, or passed from one department to another like the proverbial buck. Hidden problems will always result in debts in one or more of the five kinds of capital. And such hidden debts, whether financial, technical, intellectual, social or environmental, all too often bite you when you least expect it. They will injure or kill the project – and destroy stakeholder value. Even if the project avoids repaying its hidden debts, this usually means that the debt has been passed-off onto one or another unsuspecting stakeholder group (sometimes, the end-customer or taxpayer). Which must be judged as unethical behaviour.

Unfortunately, not only have most people in the software industry been taught to sit in their silos and focus largely on coding, they and their masters have developed a cultural love affair with the project concept. To the extent that everyone assumes that all work has to be compartmentalised into projects, the very epitome of batch & queue thinking. Tell me what you think. Has the software project had its day, or is there another way of revolutionising workpatterns in the software industry?

– P Grant Rule

Dreaming

 

If we wanted to have a highly effective software development or knowledge-work organisation, aside from the necessary mindset, what would the-way-the-work-works look like? What would our dream be?

Well, for me it would include certain essential (and therefore, non-negotiable) aspects:

• In-band change (seamless integration of day-to-day management and continuous improvement).
• Covalence (attending to the composite needs of all stakeholders, concurrently).
• Value-driven (deliveries systematically prioritised according to covalent value).
• Smooth, continuous flow (single-piece flow, if this best meets the stakeholders’ needs).
• Maintenance of adequate slack.
• Optimisation of delays and cycle (feedback delay) times.
• The people (front-line, workers) doing the work take all decisions about the work (guided by covalent value and the organisation’s purpose). Cf. Auftragstaktik, Drive
• A pervasive belief that the system (the way the work works) is the overriding determinant of the effectiveness of the organisation. Cf. W E Deming.
• Widespread understanding that collective mindset determines the way the work works.
• An appreciation that product characteristics (look, feel, utility, evocativeness) derive from the way the work works (Conway’s Law).
• Explicit limits on work in progress.
• Key (engineering) information is visible in real-time.
• A multi-skilled (generalising specialist, Cthulhu-shaped) workforce.
• Esprit de corps and a sense of fellowship exists at the organisational level (as opposed to having e.g. standing teams). Cf. The Regiment (British Army).
• Individuals strive for personal fulfilment through e.g. mastery
• Continuous innovation, no only on product and technology, but more importantly in the way the work works and its institutions and other sacred cows.

Ackoff calls this approach – imagining what the ideal future might look like, and working backwards from there to where we are today – “Interactive Planning“.

Unsurprisingly perhaps, the above attributes form the core blueprint for FlowChain.

– Bob

Perspectives on Rightshifting

Index

As this is a long post, here’s an index to each slide / dimension in the post:

Background
Introduction
Dimension: The Software Development Life Cycle
Dimension: Flow Mode
Dimension: Feedback Delay
Dimension: Administrative Project Management
Dimension: Perspective on the Individual
Dimension: Measurement
Dimension: Inductive vs Deductive
Dimension: Toolheads
Dimension: Quality and Testing
Dimension: Development Focus
Dimension: Risk Awareness
Dimension: Systematic Learning
Dimension: Design Loopbacks
Dimension: Conformance to Schedules
Dimension: Use of Third Parties
Dimension: Deployment Problems
Dimension: Variability in Project Success
Dimension: Metaphor in Use

Background

Way back in 2008, the first public outing for my ideas about Rightshifting was a forty-five minute presentation at Agile North 2008. The slides for this presentation have been online at AuthorSTREAM ever since (including, incidentally, a Part 2, that was not presented, featuring an introduction to FlowChain).

The presentation was very well received, but one thing that has rankled me since then has been the absence of any narrative to accompany the slides. I can appreciate that this absence limits the usefulness of the slide pack. As a remedy, I have reproduced the slides here, accompanied by a brief commentary, or explanation, for each slide.

[Note: the slides in this first draft, acting as placeholders, are taken from the original presentation. I may update them later, to the more recent 3D-effect format, if there’s any demand for that.]

Introduction

The presentation as a whole attempts to address the question “given that there is such a wide range of  effectiveness between different knowledge-work organisations out there in the world, how does life – and work – in these organisations differ? What makes Rightshifted organisations so different (and thus, more effective) from their less effective cousins?”.

What is a Dimension?

What is a “dimension“, in this context? It’s a slice through – or aspect of – how things look or work in knowledge-work organisations everywhere. We might imagine mapping each organisation in the real world to a (fuzzy) n-dimensional point in an n-dimensional hypercube. This mapping reveals certain clusters, or commonalities between organisations.

The slides, one by one, each illustrate a different dimension of life and attitudes to work in organisations, accompanied by a commentary.

Note: These charts and their accompanying narratives illustrate tendencies, not so much hard and fast delineations.

Dimension: The Software Development Life Cycle

This chart illustrates the kind of software lifecycle prevalent in organisations at various stages of effectiveness:

“Code and fix” refers to the disorganised, seat-of-the pants approach to developing software systems and products. Some folks refer to this as “cowboy coding”.

“Waterfall” (more accurately described as “batch and queue”) refers to those particular approaches to software development where each stage of transformation e.g. Analysis, Coding, Testing, etc.) is completed as a large single batch of work, before passing on to the next stage.

“Agile” refers to the various approaches to software development where work is conducted incrementally and iteratively, with early and regular delivery (into production) of increments in e.g. functionality.

“Beyond” alludes to other approaches to software development “beyond agile”.

Note: This slide preceded the Marshall Model by some two years. Even so, one can see the boundaries of the fours mindsets emerging.

Dimension: Flow Mode

This chart illustrates the prevailing (collective) mental model (and thus operational practices) with respect to how value flows through the organisation (e.g. from order to delivery, or “from concept to cash”):

“Random” refers to the absence of any understanding about “Flow” (of value) , and thus an absence of any specific practices to enable flow, meaning that flow of value within the organisation happens at random. For example, the actual schedule of product releases – or due date performance – will be highly variable and unpredictable, to the point of being essentially random.

“Batch and Queue” Value flows through these organisations in (often, large) batches, with each batch – for example, an entire software product – queueing at various points during its passage through the organisation. These organisations generally have little conscious understanding of the idea of flow, of queueing theory, or of the other issues that contribute to smooth, predictable flow. Consequently, the actual schedule of product releases – or due date performance – will show marked variation and a significant lack of predictability.

“Sprints, etc” These organisations have a conscious understanding of the advantages of flow, and structure their operations around improving the flow of value through the organisation.  However, these organisations have not yet transcended siloisation to the point where they can optimise flow across the whole organisation as a joined-up system. Thus, we may see the use of agile practices, such as iterative – or even continuous – delivery of user stories, features or use cases. As a  consequence, the actual schedule of product releases – or due date performance – will show limited variation and reasonable predictability.

“Systems Thinking” refers to the mindset that embraces the whole organisation as a system, and optimises flow through this system as a whole. In concert with techniques like Statistical Process Control (SPC), this means that the actual schedule of product releases – or due date performance – will show generally predictable and minor variation.

Note: The boundary between “Sprint, etc.” and “Systems Thinking” segments may lie somewhat further to the left of the 3.2-3.5 position than it appears on the above chart.

Dimension: Feedback Delay

This chart illustrates organisational thinking on how long the feedback loops in the organisation should be:

“Random” refers to the absence of any conscious attention to feedback and the length of feedback loops. Thus any feedback received from e.g. retail channels or customers about new products, product features, and the like will be acted on (or ignored) essentially at random, with the timescales (delays) for such action also, essentially, random.

“3 – 6 Months” these organisations regard a three to six month time frame for acting on e.g. customer feedback as quite normal and acceptable. Produce release cycles are typically geared around this timeframe. The concept of “cost of delay” is not often known.

“2 -4 Weeks” here, the concept of “cost of delay” is understood, and these organisations work towards quantifying and tracking these costs, and base their product investment and prioritisation decisions, at least in part, on these factors. This typically sees dramatic reductions in cycle times, bringing the time it takes to incorporate feedback from the market down to less than a month.

“Daily” highly-effective organisations tend to have a very clear understanding of their own cost-of-delay, and of the impact of feedback, and feedback delays, on their effectiveness. Not least because these kinds of organisation tend to be in the web space (c.f. Forward, Facebook, Salesforce.com, etc.), where cost-of-delay can be high, these organisations focus on cycle times and feedback delays of a day or less.

Dimension: Administrative Project Management

This chart illustrates organisational thinking on how work (in particular, product development work) should be structure and managed:

“APM” shows the prevalence of Administrative Project Management as correlated with organisational effectiveness. Least-effective organisations have little or no project management, nor indeed even projects, as such. Moving to the right, some slightly more effective organisations adopt the idea of conducting work within structures or containers called “projects”. Pretty soon after this comes the full panoply of Administrative Project Management, as typified by e.g. PRINCE2, PMBoK, etc.. As organisations’ effectiveness continues to improve, these (fewer) organisations come to understand the limitations of both the project concept itself, and the dysfunctions inherent in Administrative Project Management. The role of APM thus tails off.

“Fun” shows that although quite (relatively) ineffective, organisations to the left (little APM) are fairly fun places to work. People have a degree of autonomy, rules are absent or at least lax, and work is not so regimented or controlled. As APM increases, fun goes into a tailspin, reaching its nadir as APM reaches its zenith. This is no mere coincidence. As APM goes into decline, further to the right, fun rises again, and indeed reaches new heights, driven on by the satisfaction inherent in doing good work, delivering real value, and generally making a real difference. Highly-effective organisations tend to provide high levels of job satisfaction (aka fun).

“Wasted Potential” illustrates the correlation between APM and the waste of people’s innate potential (e.g. for doing good work). The key mechanism here is engagement. As fun drops (in line with rising APM), engagement with the work also drops away, and people have less incentive, motivation and thus inclination to do good work.  See Dan Pink’s book “Drive” (and associated videos, etc.) for an in-depth explanation of the role of Autonomy, Mastery and Purpose in the intrinsic motivation of knowledge-workers.

Dimension: Perspective on the Individual

This chart illustrates how organisations at different levels of effectiveness have different attitudes towards individuals (e.g. workers):

“Respect” maps the degree of importance which organisations attach to the idea of respect for the individual. The chart illustrates how the least-effective organisations, on the left-hand side, have some level of respect for their staff. This may be patchy, but overall, it’s about what you’d expect to find in wider society. As we consider slightly more effective organisations (progressing to the right), here we see respect for the individual decreasing as effectiveness increases. Respect reaches a nadir around 1.5 or the chart, (see also the preceding chart on Administrative Project Management) – here organisations tend to treat people as fungible, interchangeable “cogs” in the “machine” of the organisation. As this machine view of organisations begins to wane (further to the right again), respect accorded to folks in the organisation rapidly rises, easily exceeding the levels seen in wider society.

“Heroism” portrays the way in which highly-ineffective organisations attribute success and e.g. productivity to the heroic acts of individual “rock-stars”. As organisations progressively become more effective (rightshift), they likewise progressively tend to realise the role played by the system (the way the work works) relative to the contribution of “heroic” individuals. This realisation has knock-on effects on hiring, remuneration and a host of other organisational policies.

Dimension: Measurement

This chart illustrates the preferred role of measurement a.k.a. metrics in organisations at different levels of effectiveness:

“Metrics Effort” illustrates how highly ineffective organisations place very little emphasis on measuring things, and thus on the place of evidence, facts and data, more generally, in the operation of the organisation. When organisations (eventually) do begin to value measurement, they tend to go overboard on the idea, spending much effort on collecting all kinds of measures, much of which has little relevance or utility. As effectiveness continues to increase, organisations’ focus tends to resolve onto the measures with most relevance to the effectiveness of the organisation, whittling-away the less useful measures. Also, these more effective organisations tend to embed measurement – and the use of measures – into daily operations (business as usual), rather than have special (out-of-band) measurement efforts.  c.f. Basili et al –  GQM.

Dimension: Inductive vs Deductive

This chart illustrates the balance of focus on working practices (sometimes called “best practices” as against principles (the ideas underlying working practices) – in organisations at different levels of effectiveness:

Here we see a direct correlation between effectiveness and a focus on principles over practices. That is to say, highly-effective organisations understand the principles underpinning their working practices, whereas ineffective organisations have little or no understanding of the fundamental principles involved. The latter organisations are much more likely to simply copy “best practices” from others. Often this amounts to no more than “cargo-culting“.

See also: “The Inductive Deductive Schism” for more context.

Dimension: Toolheads

This chart illustrates the general disposition towards the buying and using of tools – whether physical tooling (plant), software tools or indeed, methodologies – in organisations along the Rightshifting axis:

Highly ineffective organisations tend to see little value in buying or using tools to e.g. improve productivity or reduce variation. As effectiveness improves, organisations tend to go overboard, buying tools left, right and centre in the belief that tools improve efficiencies, and that tools compensate for a lack of specialists and their know-how. For organisations that continue to improve their effectiveness, however, comes the realisation that a blanket predilection for tools does more harm than good, and these organisations become much more selective about the tools they acquire and use, even to the point of retiring or disposing of much of their existing tooling.

See also: “Watch Out For the Toolheads” article by John Seddon

Dimension: Quality and Testing

This chart illustrates the general attitude towards quality, and incidentally, the role of testing, in organisations distributed along the Rightshifting axis:

“Quality philosophy” speaks to organisations’ general philosophy on the matter of quality. Highly ineffective organisations, if they have any overt philosophy at all regarding quality, tend to believe that quality can be tested into their products and services (despite, incidentally, more than thirty years of TQM, Crosby et al., advising to the contrary). Highly effective organisation come to the realisation that quality is – at least in part – an economic concern, and whereas sometimes it may be cost-effective to retain some testing, more often the effective path to quality lies in reducing or eliminating defects.

“Testing effort” reflects the cost of testing, as seen in organisations at different levels of effectiveness. Highly ineffective organisation have low testing costs, simply because they have little or no testing (or any other quality efforts, for that matter). Organisations of moderate effectiveness tend spend a great deal of time, money and effort on testing things, primarily because they have little or no focus on reducing or eliminating defects, and thus have to rely on testing (a.k.a. inspections) to prevent defects reaching their customers. Highly effective organisations have discovered that by reducing or eliminating defects at source, the need for testing (a.k.a. inspections) reduces markedly.

“Defects see by users” illustrates the combined effect of an organisation’s quality philosophy and testing effort. Customers of highly ineffective organisations tend to see many defects and quality problems,  whereas customers of highly effective organisations tend to see far fewer quality issues.

Dimension: Development Focus

This chart illustrates the typical stance of developers and product development groups in organisations distributed along the rightshifting axis:

“CV-centric” refers to the tendency of developers and other technical specialists in e.g. highly ineffective organisations to focus on selecting and using technologies and tools that will enhance their CVs and give them interesting and cool new things to “play” with.

“Code-centric” describes the tendency for technical staff in low-effectiveness organisations to believe that code and code quality is the be-all and end-all with regard to producing successful software products and services.

“Requirements-centric” relates to moderately effective organisations’ belief that ongoing commercial success stems from understanding customers’ requirements and delivering against those requirements. Note: This does not necessarily imply a big-design-up-front or batch-and-queue approach to requirements gathering. Indeed, many requirements-centric (development) organisations quickly learn that iterative approaches to exploring requirements can afford more effective means for understanding.

“Learning-centric” pertains to the focus of highly effective organisation on continual, organisation-wide learning – including learning about customers and markets and their evolving needs and perceptions of value, but more importantly, continually learning more about how best to make the whole organisation work ever more effectively.

Dimension: Risk Awareness

“Greater risk brings greater reward, especially in software development. A company that runs away from risk will soon find itself lagging behind its more adventurous competition. By ignoring the threat of negative outcomes—in the name of positive thinking or a can-do attitude—software managers drive their organisations into the ground.”

This chart illustrates the awareness of, and approach to handling, development risk in organisations across the spectrum of organisational effectiveness:

Highly-ineffective organisations not only remain unaware of risk and risk management disciplines, but often have a pathological fear of even discussing issues from a risk perspective (hence the negative portion of the line on the chart). Risk awareness rises oh-so-slowly as organisational effectiveness increases, with only the reasonably effective organisations achieving significant levels of awareness (and hence, effective ways to handle risk). The line tails off for the highly effective organisations, as these eschew some aspects of risk management in favour of effective and disciplined means of opportunity management.

See also: “Waltzing With Bears” by DeMarco and Lister.

Dimension: Systematic Learning

learning (ˈlɜːnɪŋ)
— n
1. knowledge gained by study; instruction or scholarship
2. the act of gaining knowledge
3. (psychology) any relatively permanent change in behaviour that occurs as a direct result of experience

This chart illustrates the typical attitude, of organisations distributed along the rightshifting axis towards, systematic (i.e. deliberate, organised and organisation-wide) learning:

Highly ineffective organisations tend to be blind to the value of systematic learning. Moderately effective organisations, once awake to the possible commercial advantages of a systematic approach to learning, begin to institute means to encourage such learning. Highly effective organisations recognise the need for such learning to be integrated with Business as Usual (BAU) and to ensure that what is discovered is actually “learnt” – i.e. new knowledge actually modifies organisational behaviour.

Dimension: Design Loopbacks

“One of the fundamental problems companies have is this practice of continual loopbacks, where they think they made the right decision, but it was the wrong decision and they end up continually in firefighting mode, fixing problems on the back end.”

“If you look at the continual state of loopbacks and lost knowledge in companies, something like 70 percent of engineering talent is used to solve problems that should have been solved early on.”

 ~ Michael Kennedy

This chart illustrates the frequency and impact of “design loopbacks”, in organisations at different levels of effectiveness:

See also: “Product Development for the Lean Enterprise” by Michael Kennedy

Dimension: Conformance to Schedules

This chart illustrates the ability of organisations, at different stages of effectiveness, to deliver new products into production on time (i.e. on schedule, or on the due date):

Here we see how well organisations meet their own development schedules. It’s probably no surprise that highly ineffective organisations struggle to deliver anything on time – with high variation and low predictability in their schedule conformance. But most (averagely-effective) organisations do little better. And few of these less-effective organisations realise that the best performers (the highly effective organisations) can have highly reliable and predictable schedule conformance as high as 98%.

Note: It may be apparent that to achieve such high levels of schedule conformance requires fundamentally different approaches to product design and development that those more commonly employed. Such approaches can include Set-based concurrent engineering (SBCE a.k.a. set-based design), trade-off curves, and other measures seen in e.g. the Toyota Product Development System (TPDS).

See also: Lean Product and Process Development by Dr Allen C. Ward

Dimension: Use of Third Parties

This chart illustrates the strategic role of third-parties (specialist suppliers, consultants, sub-contracting companies, etc.) as seen by organisations distributed along the Rightshifting axis:

No organisation, however large or diverse, can hope to have all the specialist skills and know-how that might be needed to design and deliver new products and services into ever-changing markets. Thus working with specialist third-parties is often a necessity. Highly-ineffective organisations have little or no understanding or capability for finding, and working with third parties. Generally, these organisations will treat each such relationships as an entirely novel and unusual situation, discovering how to make it work as they go along. And repeating the whole exercise the next time…ad infinitum. Thus, these organisations, also often victims of NIH (not invented here) syndrome,  rarely use third parties.

Moderately effective organisations come to realise that  working with third parties is an inevitable part of doing business and evolve means to make this part of Business As Usual. Thus, these organisation come to use and rely-on third parties in many aspects of their business.

Highly-effective organisations, not least because of their fundamentally different approaches to doing things, find it increasingly difficult to find third-parties with the necessary specialist skills and cultural (mindset) “fit”. Hence, these organisations find themselves using third parties less than they perhaps might like.

Dimension: Deployment Problems

This chart illustrates the likelihood that organisations, at different stages of effectiveness, will have significant problems with their new products (or updates) after they’ve “gone live”:

Many highly-ineffective organisations see it as inevitable that their customers, users, etc. will find problems with their new product designs when released (put into live production). Moderately-effective organisations begin to regard this as undesirable, realising the cost involved – both remediation costs and reputational costs, not least. These organisations, however, typically have an uphill struggle to reduce their level of deployment problems, basically because of their piecemeal approach to the “whole product” notion, borne of years or decades of incrementalism and local optimisations. Highly-effective organisations, often by dint of radical overhaul of their approach to “whole product” issues, have minimal deployment problems.

Dimension: Variability in Project Success

This chart illustrates variation in project (i.e. new product or service development) success, along the effectiveness spectrum. More significantly in my view, it also illustrates the different causes to which organisations at different levels of effectiveness attribute such variation:

Here we see that highly-ineffective organisations have high levels of variation (and thus low levels of predictability, certainty) in their new product development efforts. Levels of variation fall in line with increases in organisational effectiveness.

As to causes, highly-ineffective organisations tend to attribute success (and variability thereof) to the heroic (or paltry) efforts of specific individuals. Moderately effective organisations tend to let go of that simplistic notion, but often get lost in their search for the root causes of the variability in their record of success. Highly-effective organisations have discovered that, as Deming suggests, circa 95% of their success at delivering projects is down to their organisational systems – or “the way the work works”.

Dimension: Metaphor in Use

This chart illustrates the prevailing metaphor for knowledge work, as it varies in different organisations along the effectiveness axis:

Highly ineffective knowledge-work organisations have yet to realise even the nature of the work in which they are engaged, choosing, mostly by default, to regard it as just another kind of “office work”. This choice of metaphor leads to certain choices regarding e.g. the layout of the work space (cube farms, segregation of specialists, absence of team spaces, etc.).

Organisations of limited effectiveness choose to adopt the “software factory” metaphor for work, with an abundance of manufacturing/factory related metaphors for all aspects of work, such as “production line”, “batch and queue”, “conformance”, etc.

Reasonably effective organisations eschew these metaphors in favour of work as “product design” or  the “design studio”, choosing to regard the workers as “creatives”, and understanding the value of flow (in the Mihály Csíkszentmihályi sense of the word), creativity and innovation.

Highly-effective organisations, whilst appreciating the “design studio” metaphor and values, choose to adopt a “value stream” or “value network” metaphor for work, and place emphasis on the flow of value.

See also: “Principles of Product Development Flow” by Don Reinertsen.

– Bob