Archive

Flowchain

Holistic Solutions for Product Development Businesses

Several people have been in contact this week to say “It’s all very well talking about holistic solutions for software development, but who really has any notion of what that might even look like?”

Which is a fair question.

Aside: Let’s note that the phrase “holistic solution for software development” is an oxymoron of the first order. By definition, software development is but part (and generally a small part) of any solution that addresses customer or market needs. Even software “pure plays” have a lot of non-software components.

So, I thought I’d describe just one holistic solution for whole organisations that develop new products containing some software. For want of a better name, I’ll call it this example “Flow•gnosis”. With this example, I hope that maybe one or two readers might find a spark of insight or inspiration to broach the question of a True Consensus in their own organisation.

I’ll try to keep this post brief. I’d be happy to come talk with you about holistic product development solutions for organisations, in person, if you have any real interest.

Flow•gnosis?

Flow•gnosis is a hybrid born of FlowChain and Prod•gnosis. FlowChain is not specifically a holistic solution, focussing as it does on improving the flow of knowledge work through a development group or department whilst moving continuous improvement “in-band”. Prod•gnosis is specifically a holistic solution for effective, organisation-wide product development, but says little about how to organise the work for e.g. improved flow or continuous improvement.

Together, they serve as an exemplar of a holistic solution for knowledge-work organisations, such as software product companies, software houses, tech product companies, and organisations of every stripe with in-house product development needs.

Let’s also note that Flow•gnosis is just an example, to illuminate just a few aspects of a holistic solution. Do not under any circumstances consider copying it or cloning it. Aside from the lack of detail presented here, you’d be missing the whole point of the challenge: building a True Consensus, as a group, with your people, in your context.

What’s the Problem?

Before talking more about a solution, what’s the problem we’re trying to solve?

I’ll work through an almost universal problem facing organisations that “do” software development. A problem that some enterprising supplier or management team might choose to address with an “innovative” solution.

Understanding the Problem

Here are some typical Undesirable Effects (UDEs) I hear from many companies involved in software and product development:

UDE: Delivery is not fast enough (long time-to-market)

UDE: New products cost too much to develop (high cost to bring to market)

UDE: We struggle to keep our products at the cutting edge

UDE: We always drop balls in hand-offs of tasks (i.e. between specialists, or business functions)

UDE: Continual contention for in-demand specialists

(For the sake of brevity, I’ll skip the building and analysis of the cause and effect graph. Your analysis will be different, in any case).

Root cause: Developing new software, products and services through a byzantine labyrinth of tasks and hand-offs, between and across dozens of specialisms within the company and its network of suppliers and distributors. This approach causes many delays (waste of time), mistakes (waste of effort, money), contention of resources, etc.

Conflict: A) Specialists must work together in their own specialist groups or silos to maintain their cutting-edge skills and know-how. B) Specialists from across all specialisms must work together as a group else handoffs and queues will cause many delays and mistakes.

Conflict arrow: Specialists cannot work in their own groups advancing their specialist knowledge at the same time as they work together with other kinds of specialist on new product ideas.

Flawed assumption at the root of the conflict: Specialists must work together. What makes this so? Only the old rules of the organisation. That silos (a.k.a. business functions) “own” their clutch of specialists. What if we changed that rule? (Note: this change is one key element of Toyota’s TPDS). And if we changed that rule, what other rules would we need to change too?

A Bird’s Eye View of Flow•gnosis

Prod•gnosis looks at an organisation as a collection of parallel Operational Value Streams, each dedicated to the selling and support of one of the organisations’ (whole) product or service lines. And each with its own team or collection of people doing the daily work of that operational value stream. Further, Prod•gnosis asks “How do these operational value streams come into being?” And answers with “They are made/created/developed by a dedicated Product Development Value Stream.”

FlowChain describes a way of working where a pool of self-organising specialists draws priority work from a backlog, executes the work, and delivers both the requested work item, and posting new work items (for improving the ways the work works) into the backlog. In this way, FlowChain both improves flow of work through the system, and brings continuous improvement “in-band”

By merging Prod•gnosis and FlowChain together into Flow•gnosis, we have an organisation-wide, holistic example which improves organisational effectiveness, reifies Continuous Improvement, speeds flow of new products into the market, provides an operational (value stream based) model for the whole business, and allows specialists from many functions to work together with a minimum of hand-offs, delays, mistakes and other wastes.

The latter point is perhaps the most significant aspect of Flow•gnosis. Having customer, supplier, marketing, sales, finance, logistics, service, billing, support and technical (e.g. software, usability, emotioneering, techops, etc.) specialists all working together (cf. Toyota’s Obeya or “Big Room” concept) enables the evolution of “Mafia Products” and “Mafia Offers” which the more traditional silo-based models of business organisation just can’t address effectively.

Out With the Old Rules, In with the New

Flow•gnosis is an innovation. Irrespective of the promised benefits of Flow•gnosis, we have learnt in recent posts that adopting an innovation ONLY brings benefits when we change the rules.

Let’s apply the four questions to our Flow•gnosis innovation and see what rule changes will be necessary to truly reap the benefits.

Q1: What is the POWER of Flow•gnosis?
A1: Flow•gnosis makes it commercially feasible for a company to repeatably come to market with new “Mafia Products”.

Mafia Product: “A product (or service) so compelling that your customers can’t refuse it and your competition can’t or won’t offer the same.”

Q2: What limitation does Flow•gnosis diminish?
A2:  Serialisation of specialist work (passing things back and forth between various specialist and business functions).

Q3: What existing rules served to help us accommodate that limitation?
A3: Handoffs. Queues. Batches. Separation of command and control from the work. Process. Process conformance. Local measures. Constant expediting. Critical path planning (Gannt charts, WBS). Local optima. Functional management (discrete management of each separate business function).

Q4: What (new) rules must we use now?
A4: Flow. Value streams. SBCE. Holistic measures. Ubiquitous information radiators. Cost Of Delay prioritisation. Buffer management. Constant collaboration. Dedicated teams of generalising specialists. Self-organisation. In-band continuous improvement. Resource levelling. Limits on WIP.

Summary

This post has been a – necessarily brief – look at one holistic solution to (software) product development. Crucially, we have seen how old rules have to be replaced, and what many of the replacement new rules might look like.

I invite you to remember that Flow•gnosis is just an broad-brush example of a holistic solution. Please don’t consider copying it or cloning it. And remember the real challenge: building, as a management group, a True Consensus.

– Bob

Further Reading

Meeting Folks’ Needs At Scale – Think Different blog post

The Simplest Thing That Could Possibly Work

TinCup

Here’s an excerpt (pp 206) from “Birth Of the Chaordic Age” by Dee Hock, about “an odd project management scheme” they adopted in the early days – circa 1974:

“Swiftly, self-organisation emerged. An entire wall became a pinboard with every remaining day calendared across the top. Someone grabbed an unwashed coffee cup and suspended it on a long piece of string pinned to the current date. Every element of work to be done was listed on scraps of paper with the required completion date and the name of the person who had accepted the work. Anyone could revise the elements, adding tasks or revising dates, providing they coordinated with others affected. Everyone, at any time, could see the picture emerge and evolve. They could see how the whole depended on their work, and how their work was connected to every other part of the effort. Groups constantly assembled in front of the board as need and inclination arose, discussing and deciding in continuous flow; then dissolving as needs were met. As each task was completed its scrap of paper would be removed. Each day, the cup and string moved inexorably ahead.”

I’m struck by the similarities with FlowChain, and as with FlowChain, it seems an exemplar of simplicity and flow. I also note the implicit “Advice Process” vibe, and the emphasis on “making the work visible” (Cf Personal Kanban).

– Bob

Further Reading

Meeting Folks’ Needs At Scale

Scaling Agile is one of those oxymorons that has, nevertheless, consumed endless column-inches and hours of debate. I have no time for it. Agile was meant for development-in-the-small. For teams of five to seven people, give or take. Even at that “scale”, I have serious reservations about its efficacy and effectiveness. The idea of scaling it up to multiple dependent teams, or even to whole development departments or groups of hundreds or thousands of people, seems just crazy.

The Demand

Yes. There are organisations with hundreds or thousands of developers working on more or less dependent things. Huge systems. Ginormous products. And these organisations have problems. Boy, do they have problems. They’ve had the same kind of problems for decades now. Advances in tech and tools seems to have made little dent in those problems. Similarly, advances in methods and processes have barely scratched the surface.

So there is a demand. A demand for something better. A demand for a packaged solution that can be simply (ha!) “installed” and run with. And the folks with the problems have money. Lots of money. Bags of moolah.

No wonder it looks like an interesting problem to solve. The thing is, I don’t see many people, either on the demand or the supply side, that actually understand the nature of the problem.

Absent an understanding of the problem, and given their desperation, the demand side will embrace any and all solutions that bear even a vague whiff of credibility. And the “Agile” label these days confers that smell.

Needs

All product development is, in essence, an exercise in attending to folks’ needs. The more kinds of folks with needs, the more of their needs we bring into scope, and the quicker we want to see those needs met, the more people we might choose to commit. The question of agile (or not) is a huge irrelevance. Things like communications overheads, coordination, partitioning of needs, a regular cadence, flow, and effective use of resources (time, money, equipment) hold sway. TPDS had these issues sorted out (mostly) years ago with e.g. the Obeya or “big room” concept, set-based concurrent engineering, etc.

I have suggested that something like FlowChain might afford an effective model for organising to meet folks’ needs at scale. I have yet to hear anyone suggest why this model is flawed. Until that day, I will continue to invite you to consider its merits.

The demand side continues to have needs that are not being well-met (due to e.g. a host of pathological beliefs). The supply side has needs that are being more-or-less well met by selling solutions largely disconnected from the real problem. Scaling agile seems to me a very one-sided, cynical and exploitative bargain.

– Bob

Further Reading

Moving Past the Scaling Myth ~ Michael Feathers
Lean Product and Process Development ~ Allen Ward

Emergence

emergentpath

You may have noticed that I write regularly about the different mindsets that explain the relative effectiveness of the organisations we work for and with. Things like Theory-X (strong in the Ad-hoc and Analytic mindsets) vs Theory-Y (Synergistic and Chaordic mindsets), and organisations-as-machines vs organisations as social/biological/complex adaptive systems.

One difference I have not touched on much is the part that emergence has to play in the effective organisation.

Gall’s Law

“A complex system that works is invariably found to have evolved from a simple system that worked. The inverse proposition also appears to be true: A complex system designed from scratch never works and cannot be made to work. You have to start over, beginning with a working simple system.”

~ John Gall

Does Your Organisation Embrace or Ignore Gall’s Law?

Ad-hoc and Analytic minded organisations generally believe that systems are best designed, up front, with acts of conscious will and intent. Be they organisational structure, policies, products or a myriad of other systems upon which an organisation depends.

Synergistic organisations learn, by degrees, that John Gall nailed it – complex systems that work require evolution from simple systems that work. For effective (working) organisations, we need to embrace emergence. We need to allow our systems – and our thinking – to evolve to the point where emergence is working for us. This is hard.

Messy

Emergence seems messy. Allowing things to take their own course is hard for folks who seek certainty and control as means to getting their needs met. It can often feel like a crowd of people trampling over your nice, neat, manicured lawns. But the properties of beauty and simplicity can emerge more or less unbidden, too.

Whilst we opposed emergence, we lock ourselves into relatively ineffective ways of thinking, and thus, of working.  Only when we embrace and encourage emergence, do we open the door to more effective ways of thinking and working.

FlowChain

One of the fundamental guiding principles of FlowChain is to encourage emergence:

  • Emergence of products
  • Emergence of teams
  • Emergence of methods (“the way the work works”)
  • Emergence of systems
  • Emergence of priorities
  • Emergence of flow
  • Emergence of needs (and e.g. stakeholders)
  • Emergence of purpose (the “why”)
  • Emergence of ideas (i.e. creativity)

Would you be willing to consider, and share, where your organisation is at regarding the role of emergence?

“My ideas have undergone a process of emergence by emergency. When they are needed badly enough, they are accepted.”

~ R. Buckminster Fuller

– Bob

Further Reading

Obliquity ~ John Kay
Systemantics ~ John Gall

Out Of House FlowChain

When I conceived of FlowChain, some six years ago now, my immediate context was development shops with their own in-house developers, and other supporting staff.

But it strikes me that with just a few adjustments, it’s also suitable for organisations that sub-contract out most or all of their development projects to third parties.

These adjustments centre around arranging for the various third parties (assuming, in the likely case, that there’s more than one) to each contribute staff to the “Pool” (see diagram). These arrangements include:

Commercial

How will the third parties be paid? Some UK government functions use function points as a measure of “work done”, with a set price for each function point “delivered”. See: Output-based contracts. I can imagine other contractual arrangements, too.

Social

How will the third parties’ staff integrate or form healthy relationships with the in-house commissioning staff (a.k.a. product owners)? Will there be shared spaces? Regular visits to and fro? Some more technical forms of communication (Twitter, chat, video conferencing, etc.)? Remember, each backlog item is sized for two to three people working together for two to three days.

Tooling

Third parties remain free to pull items from the backlog as they see fit (just as with in-house FlowChain), and use their own tools, languages, etc.. I foresee some advantages in having a common code repository, coding and other standards, agree requirements around test suites, and so on.

Delivery Into Production

Maybe the organisation contracting the third parties has its own Ops department. In this case the interface between development (teams, third parties) and Ops would probable best be standardised and agreed (like an API). If the third parties have their own Ops folks, then they can do DevOps in their own space and time, and serve the “production” services – or even micro-services – they each operate, directly to users.

Shared Backlog

For clarity, this variant of Flowchain retains the enterprise-wide backlog, with user stories, improvement stories, etc. being prioritised by some black box (or white box) prioritisation algorithm, committee, manager, or whatever. The only real change is in how the Pool is constituted. Note: I see no particular reason why the general FlowChain principle of “ANY unassigned development folks from the Pool can coalesce around each new top backlog item” cannot stand, here.

There may even be emergent advantages in having e.g. developers from different third parties coming together to collaborate on specific backlog items. How weird would that be? Again, policy would guide folks’ actions here.

Who would “manage” the backlog?  This could be done by a small in-house staff, or itself subcontracted out to one or more third parties. Note: the backlog in FlowChain is largely self-managing, in any case, given an effective prioritisation algorithm or approach.

Flow

Flow (of e.g. software into the hands of those whose needs are being attended to) remains the key objective of the whole approach.

Growing An In-house Capability

For organisations without an in-house development capability, this approach provides a simple(r) path to establishing and growing an in-house development capability. In-house developers can be added, one by one, as and when circumstances (budgets, priorities, etc) allow. These new folks can work alongside – and learn from – third-party staff already used to Pool working, and the balance between in-house and out-of-house staff, skill sets, etc. adjusted dynamically as needs dictate.

Drawbacks

The key drawback I foresee is in the matter of dev-ops integration (DevOps). This could prove more difficult, in the case where developers, etc. are out-of-house and Ops in-house. This seems a special case of the issues of outsourcing and offshoring, generally. But I’m sure a bunch of smart developers and smart ops can work this out, especially with some help and guidance.

– Bob

NeedsFlow

[Tl;Dr: The essence of software and product development is about continuously exploring and meeting folks’ needs. How about as in industry we stop shilly-shallying and place this reality at the core of our approach to development?]

The Essence Of Software And Product Development

Essentially, every approach to software and product development is about a flow of things whereby a development team, group or organisation attempts to satisfy some selection of folks’ unmet needs.

In batch-and-queue (e.g. Waterfall, etc.) approaches, needs are batched-up into large batches, and flow (not very effectively) through a sequence of queues. Each batch (in the pathological case, only one ) eventually gets dropped onto those assumed to have said needs, and that’s about it.

In iterative (e.g. Scrum, Kanban, etc.) approaches, needs are batched up into many, relatively smaller, batches, and flow (rather more effectively) through a sequence of queues (at its simplest, e.g. Backlog, Doing, Done). Each batch gets dropped onto the folks assumed to have said needs, and these folks get the chance to say if their needs have been met well enough, or not. When budgets and timescales allow, a later iteration can then have another go at better meeting the poorly-met needs, along with including any other priority needs that might have emerged along the way.

Different approaches differentiate themselves on e.g. batch sizes, selecting of the kinds of folks whose needs will be attended to, cadence and strategies.

Needs Trump Value

Why Needs flow? Why not just stick with the more common “value” flow? Well, for me there’s a whole bunch of issues around the notion of value. Including:

  • Value for whom?
  • Who gets to specify how any particular “value” is measured or quantified (most often folks ill-equippped to do so).
  • The whole notion of “value” (often overlooked, often misunderstood).
  • Valeu is too often though of in simple economic or financial terms. Human beings perceive value differently, and much more richly (e.g. emotionally).
  • How can we tell when we’ve delivered “value”?

So, I propose that needs trump value. And therefore, that when focussing on flow, it’s more useful – and effective – to focus on the flow of needs.

Diagram

The above diagram shows a system boundary – this could be the whole organisation, the development organisation, or just one development team. Bundles of external needs flow into the system, and merge with other bundles of needs coming from inside the system. These bundles pass though various stages (and queues) where “development” strategies are uses in moving the bundles from stage to stage, and from queue to queue. Ultimately, candidates (things we hope will meet the needs that entered the pipeline) flow back to external parties, and also to internal stakeholders, such as the developers themselves.

Impediments

This perspective allows for the simple integration of dealing with impediments. Things that are impeding e.g. flow or the outflow of effective candidates may themselves be needs. And may enter the pipeline much as any other needs. Of course, there may be some( or many) such impediments than no one has any need of dealing with. These may be filed or ignored.

Abstract FlowChain

Whilst FlowChain was conceived to be awesomely effective at handling bundles of e.g. User Stories, Use Cases, Improvement Stories, etc., it can very easily embrace the unit of flow being bundles of needs, or – in the single-piece-continuous-flow scenario – individual needs.

Limiting WIP, Making Things Visible

From the diagram, you may begin to see how we might simply make the flow of needs visible, should we so choose. And personally, I’d like to make visible the ultimate fate of the candidates too, with a little expansion of the scope of the diagram.

And the queues (particularly the first) can serve as a convenient means to limit Work In Progress (a.k.a. Needs in Progress).

No Projects

It may not be immediately obvious, but the NeedsFlow perspective implicitly removes the need for projects. NeedsFlow decouples development (flow) from products and product lines (each of which meet a more or less related collection of needs).

– Bob

No Projects

The idea of “projects” in software and product development is a glaring anachronism, traceable back to the days when organisations saw each new project as “the last one we’re ever likely to run”. Absent the expectation of ever running another project, why bother moving to a more continuous, flow-based (non-project) set-up?

And of course, the idea of breaking things down into parts, and managing those parts separately, is another glaring anachronism, and one still grasped so tightly by those of the Analytic mindset.

But even the briefest of reflections about the nature of development work in organisations reveals a simple truth: just about every organisation today has a more or less continual flow of work – of new ideas transforming into products, of concepts becoming cash revenue generators.

I won’t dwell further here on the case for #NoProjects – Grant made the case quite well in his piece “What’s Wrong With the Project Approach?”. Maybe you’d like to consider the relative (dis)merits of “projects” – compared to what?

May I just invite you to consider whether there may be other, maybe better ways of getting folks’ – and organisations’ – needs met?

And, by the by, offer up FlowChain as a simple – and complete – example of what I’m talking about in terms of one such better way.

– Bob

%d bloggers like this: