Archive

Flowchain

The Simplest Thing That Could Possibly Work

TinCup

Here’s an excerpt (pp 206) from “Birth Of the Chaordic Age” by Dee Hock, about “an odd project management scheme” they adopted in the early days – circa 1974:

“Swiftly, self-organisation emerged. An entire wall became a pinboard with every remaining day calendared across the top. Someone grabbed an unwashed coffee cup and suspended it on a long piece of string pinned to the current date. Every element of work to be done was listed on scraps of paper with the required completion date and the name of the person who had accepted the work. Anyone could revise the elements, adding tasks or revising dates, providing they coordinated with others affected. Everyone, at any time, could see the picture emerge and evolve. They could see how the whole depended on their work, and how their work was connected to every other part of the effort. Groups constantly assembled in front of the board as need and inclination arose, discussing and deciding in continuous flow; then dissolving as needs were met. As each task was completed its scrap of paper would be removed. Each day, the cup and string moved inexorably ahead.”

I’m struck by the similarities with FlowChain, and as with FlowChain, it seems an exemplar of simplicity and flow. I also note the implicit “Advice Process” vibe, and the emphasis on “making the work visible” (Cf Personal Kanban).

– Bob

Further Reading

Meeting Folks’ Needs At Scale

Scaling Agile is one of those oxymorons that has, nevertheless, consumed endless column-inches and hours of debate. I have no time for it. Agile was meant for development-in-the-small. For teams of five to seven people, give or take. Even at that “scale”, I have serious reservations about its efficacy and effectiveness. The idea of scaling it up to multiple dependent teams, or even to whole development departments or groups of hundreds or thousands of people, seems just crazy.

The Demand

Yes. There are organisations with hundreds or thousands of developers working on more or less dependent things. Huge systems. Ginormous products. And these organisations have problems. Boy, do they have problems. They’ve had the same kind of problems for decades now. Advances in tech and tools seems to have made little dent in those problems. Similarly, advances in methods and processes have barely scratched the surface.

So there is a demand. A demand for something better. A demand for a packaged solution that can be simply (ha!) “installed” and run with. And the folks with the problems have money. Lots of money. Bags of moolah.

No wonder it looks like an interesting problem to solve. The thing is, I don’t see many people, either on the demand or the supply side, that actually understand the nature of the problem.

Absent an understanding of the problem, and given their desperation, the demand side will embrace any and all solutions that bear even a vague whiff of credibility. And the “Agile” label these days confers that smell.

Needs

All product development is, in essence, an exercise in attending to folks’ needs. The more kinds of folks with needs, the more of their needs we bring into scope, and the quicker we want to see those needs met, the more people we might choose to commit. The question of agile (or not) is a huge irrelevance. Things like communications overheads, coordination, partitioning of needs, a regular cadence, flow, and effective use of resources (time, money, equipment) hold sway. TPDS had these issues sorted out (mostly) years ago with e.g. the Obeya or “big room” concept, set-based concurrent engineering, etc.

I have suggested that something like FlowChain might afford an effective model for organising to meet folks’ needs at scale. I have yet to hear anyone suggest why this model is flawed. Until that day, I will continue to invite you to consider its merits.

The demand side continues to have needs that are not being well-met (due to e.g. a host of pathogenic beliefs). The supply side has needs that are being more-or-less well met by selling solutions largely disconnected from the real problem. Scaling agile seems to me a very one-sided, cynical and exploitative bargain.

– Bob

Further Reading

Moving Past the Scaling Myth ~ Michael Feathers
Lean Product and Process Development ~ Allen Ward

Emergence

emergentpath

You may have noticed that I write regularly about the different mindsets that explain the relative effectiveness of the organisations we work for and with. Things like Theory-X (strong in the Ad-hoc and Analytic mindsets) vs Theory-Y (Synergistic and Chaordic mindsets), and organisations-as-machines vs organisations as social/biological/complex adaptive systems.

One difference I have not touched on much is the part that emergence has to play in the effective organisation.

Gall’s Law

“A complex system that works is invariably found to have evolved from a simple system that worked. The inverse proposition also appears to be true: A complex system designed from scratch never works and cannot be made to work. You have to start over, beginning with a working simple system.”

~ John Gall

Does Your Organisation Embrace or Ignore Gall’s Law?

Ad-hoc and Analytic minded organisations generally believe that systems are best designed, up front, with acts of conscious will and intent. Be they organisational structure, policies, products or a myriad of other systems upon which an organisation depends.

Synergistic organisations learn, by degrees, that John Gall nailed it – complex systems that work require evolution from simple systems that work. For effective (working) organisations, we need to embrace emergence. We need to allow our systems – and our thinking – to evolve to the point where emergence is working for us. This is hard.

Messy

Emergence seems messy. Allowing things to take their own course is hard for folks who seek certainty and control as means to getting their needs met. It can often feel like a crowd of people trampling over your nice, neat, manicured lawns. But the properties of beauty and simplicity can emerge more or less unbidden, too.

Whilst we opposed emergence, we lock ourselves into relatively ineffective ways of thinking, and thus, of working.  Only when we embrace and encourage emergence, do we open the door to more effective ways of thinking and working.

FlowChain

One of the fundamental guiding principles of FlowChain is to encourage emergence:

  • Emergence of products
  • Emergence of teams
  • Emergence of methods (“the way the work works”)
  • Emergence of systems
  • Emergence of priorities
  • Emergence of flow
  • Emergence of needs (and e.g. stakeholders)
  • Emergence of purpose (the “why”)
  • Emergence of ideas (i.e. creativity)

Would you be willing to consider, and share, where your organisation is at regarding the role of emergence?

“My ideas have undergone a process of emergence by emergency. When they are needed badly enough, they are accepted.”

~ R. Buckminster Fuller

– Bob

Further Reading

Obliquity ~ John Kay
Systemantics ~ John Gall

Out Of House FlowChain

When I conceived of FlowChain, some six years ago now, my immediate context was development shops with their own in-house developers, and other supporting staff.

But it strikes me that with just a few adjustments, it’s also suitable for organisations that sub-contract out most or all of their development projects to third parties.

These adjustments centre around arranging for the various third parties (assuming, in the likely case, that there’s more than one) to each contribute staff to the “Pool” (see diagram). These arrangements include:

Commercial

How will the third parties be paid? Some UK government functions use function points as a measure of “work done”, with a set price for each function point “delivered”. See: Output-based contracts. I can imagine other contractual arrangements, too.

Social

How will the third parties’ staff integrate or form healthy relationships with the in-house commissioning staff (a.k.a. product owners)? Will there be shared spaces? Regular visits to and fro? Some more technical forms of communication (Twitter, chat, video conferencing, etc.)? Remember, each backlog item is sized for two to three people working together for two to three days.

Tooling

Third parties remain free to pull items from the backlog as they see fit (just as with in-house FlowChain), and use their own tools, languages, etc.. I foresee some advantages in having a common code repository, coding and other standards, agree requirements around test suites, and so on.

Delivery Into Production

Maybe the organisation contracting the third parties has its own Ops department. In this case the interface between development (teams, third parties) and Ops would probable best be standardised and agreed (like an API). If the third parties have their own Ops folks, then they can do DevOps in their own space and time, and serve the “production” services – or even micro-services – they each operate, directly to users.

Shared Backlog

For clarity, this variant of Flowchain retains the enterprise-wide backlog, with user stories, improvement stories, etc. being prioritised by some black box (or white box) prioritisation algorithm, committee, manager, or whatever. The only real change is in how the Pool is constituted. Note: I see no particular reason why the general FlowChain principle of “ANY unassigned development folks from the Pool can coalesce around each new top backlog item” cannot stand, here.

There may even be emergent advantages in having e.g. developers from different third parties coming together to collaborate on specific backlog items. How weird would that be? Again, policy would guide folks’ actions here.

Who would “manage” the backlog?  This could be done by a small in-house staff, or itself subcontracted out to one or more third parties. Note: the backlog in FlowChain is largely self-managing, in any case, given an effective prioritisation algorithm or approach.

Flow

Flow (of e.g. software into the hands of those whose needs are being attended to) remains the key objective of the whole approach.

Growing An In-house Capability

For organisations without an in-house development capability, this approach provides a simple(r) path to establishing and growing an in-house development capability. In-house developers can be added, one by one, as and when circumstances (budgets, priorities, etc) allow. These new folks can work alongside – and learn from – third-party staff already used to Pool working, and the balance between in-house and out-of-house staff, skill sets, etc. adjusted dynamically as needs dictate.

Drawbacks

The key drawback I foresee is in the matter of dev-ops integration (DevOps). This could prove more difficult, in the case where developers, etc. are out-of-house and Ops in-house. This seems a special case of the issues of outsourcing and offshoring, generally. But I’m sure a bunch of smart developers and smart ops can work this out, especially with some help and guidance.

– Bob

NeedsFlow

[Tl;Dr: The essence of software and product development is about continuously exploring and meeting folks’ needs. How about as in industry we stop shilly-shallying and place this reality at the core of our approach to development?]

The Essence Of Software And Product Development

Essentially, every approach to software and product development is about a flow of things whereby a development team, group or organisation attempts to satisfy some selection of folks’ unmet needs.

In batch-and-queue (e.g. Waterfall, etc.) approaches, needs are batched-up into large batches, and flow (not very effectively) through a sequence of queues. Each batch (in the pathological case, only one ) eventually gets dropped onto those assumed to have said needs, and that’s about it.

In iterative (e.g. Scrum, Kanban, etc.) approaches, needs are batched up into many, relatively smaller, batches, and flow (rather more effectively) through a sequence of queues (at its simplest, e.g. Backlog, Doing, Done). Each batch gets dropped onto the folks assumed to have said needs, and these folks get the chance to say if their needs have been met well enough, or not. When budgets and timescales allow, a later iteration can then have another go at better meeting the poorly-met needs, along with including any other priority needs that might have emerged along the way.

Different approaches differentiate themselves on e.g. batch sizes, selecting of the kinds of folks whose needs will be attended to, cadence and strategies.

Needs Trump Value

Why Needs flow? Why not just stick with the more common “value” flow? Well, for me there’s a whole bunch of issues around the notion of value. Including:

  • Value for whom?
  • Who gets to specify how any particular “value” is measured or quantified (most often folks ill-equippped to do so).
  • The whole notion of “value” (often overlooked, often misunderstood).
  • Valeu is too often though of in simple economic or financial terms. Human beings perceive value differently, and much more richly (e.g. emotionally).
  • How can we tell when we’ve delivered “value”?

So, I propose that needs trump value. And therefore, that when focussing on flow, it’s more useful – and effective – to focus on the flow of needs.

Diagram

The above diagram shows a system boundary – this could be the whole organisation, the development organisation, or just one development team. Bundles of external needs flow into the system, and merge with other bundles of needs coming from inside the system. These bundles pass though various stages (and queues) where “development” strategies are uses in moving the bundles from stage to stage, and from queue to queue. Ultimately, candidates (things we hope will meet the needs that entered the pipeline) flow back to external parties, and also to internal stakeholders, such as the developers themselves.

Impediments

This perspective allows for the simple integration of dealing with impediments. Things that are impeding e.g. flow or the outflow of effective candidates may themselves be needs. And may enter the pipeline much as any other needs. Of course, there may be some( or many) such impediments than no one has any need of dealing with. These may be filed or ignored.

Abstract FlowChain

Whilst FlowChain was conceived to be awesomely effective at handling bundles of e.g. User Stories, Use Cases, Improvement Stories, etc., it can very easily embrace the unit of flow being bundles of needs, or – in the single-piece-continuous-flow scenario – individual needs.

Limiting WIP, Making Things Visible

From the diagram, you may begin to see how we might simply make the flow of needs visible, should we so choose. And personally, I’d like to make visible the ultimate fate of the candidates too, with a little expansion of the scope of the diagram.

And the queues (particularly the first) can serve as a convenient means to limit Work In Progress (a.k.a. Needs in Progress).

No Projects

It may not be immediately obvious, but the NeedsFlow perspective implicitly removes the need for projects. NeedsFlow decouples development (flow) from products and product lines (each of which meet a more or less related collection of needs).

– Bob

No Projects

The idea of “projects” in software and product development is a glaring anachronism, traceable back to the days when organisations saw each new project as “the last one we’re ever likely to run”. Absent the expectation of ever running another project, why bother moving to a more continuous, flow-based (non-project) set-up?

And of course, the idea of breaking things down into parts, and managing those parts separately, is another glaring anachronism, and one still grasped so tightly by those of the Analytic mindset.

But even the briefest of reflections about the nature of development work in organisations reveals a simple truth: just about every organisation today has a more or less continual flow of work – of new ideas transforming into products, of concepts becoming cash revenue generators.

I won’t dwell further here on the case for #NoProjects – Grant made the case quite well in his piece “What’s Wrong With the Project Approach?”. Maybe you’d like to consider the relative (dis)merits of “projects” – compared to what?

May I just invite you to consider whether there may be other, maybe better ways of getting folks’ – and organisations’ – needs met?

And, by the by, offer up FlowChain as a simple – and complete – example of what I’m talking about in terms of one such better way.

– Bob

Seven Changes To Improve Flow In Your Software Development Process

Many folks drinking the Lean coolade seem to believe that removing waste is at the heart of the Lean approach. I beg to differ. I’d say that improving flow is the heart of Lean.

Here’s seven ways in which your team or, preferably, your organisation as a whole, might go about improving flow:

  1. Adopt a small thing as the universal unit of work. And by universal, I mean some unit of work that everyone across the whole organisation can recognise and adopt. This could be Use Cases, User Stories, or something else. Just keep the “small thing” small (never more than a couple of days work for a couple of people). cf Heijunka, FlowChain.
  2. Make flow visible. In particular, make e.g. queues, queuing times, and end-to-end cycles times visible for all to see.
  3. Know your WIP (work in progress) and work to reduce (limit) it. Cf. Little’s Law.
  4. Use demand to “pull” units of work through the system (as opposed to “pushing” work through).
  5. Eliminate – or at least minimise – hand-offs. That is, having work pass from one specialist to another. Each hand-off typically introduces another queue, with the inevitable costs and delays. One way to do this involves multi-disciplinary teams, or better still, up-skilling individuals so each person can competently take on a variety of specialist tasks.
  6. Identify the goal; understand demand (by various means – for example follow individual “demands” through the system, end-to-end;) identify the constraint; and apply the Five Focussing Steps (repeatedly). Cf. Theory of Constraints
  7. Experiment continually: trial possible improvements to flow, one by one, to assess their actual efficacy, in isolation from one another. Cf. PDCA a.k.a. the Shewhart Cycle.

And of course, none of the above suggestions will do much good, or even get acted on, unless and until the folks doing the work internalise a basic appreciation of the very notion of flow. And that’s unlikely to happen unless and until the work environment supports and nurtures folks’ curiosity and innate desire to do a good job.

Further Reading

The Principles of Product Development Flow ~ Donald G. Reinertsen
Seven Changes To Remove Waste From Your Software Development Process ~ Cecil Dijoux
Product Development Flow ~ FlowchainSensei
LondonCD Talk based on this post ~ Video
Getting People to Limit Their Work In Progress ~ Ben Linders

%d bloggers like this: