I first encountered the phrase “Shadow Organisation” in Mark Pelling’s 2011 “Adaptation and Climate Change: From resilience to transformation”
However, the generation of novel ideas or practices that are in conflict with or undervalued by canonical organisation often first emerges from the unmanaged space of shadow organisations. Shadow systems are supportive of innovation because they are typically rich in trust, cut across canonical organisational structures and are hidden from formal oversight, allowing experimentation and risk-taking with novel ideas and practices (Shaw, 1997). Successful experiments in shadow systems may in turn become co-opted and formalised within the canonical system. This can provide opportunities for the replication of adaptations, but through formalisation of individual roles and relational commitments will limit flexibility and change the social relations which led to the original innovation and potentially undermine long-term sustainability. Alternatively, shadow systems can remain marginalised and informal, operating in parallel with canonical systems. This is especially so under transitional and transformative adaptation where emergent forms are a site for the challenging of established discursive and material power (Pelling, et al, 2007).Shaw (1997) Intervening in the Shadow System of Organizations: Consulting from a complexity perspective. Journal of Organisational Change, 10 (3): 235 -50.
YES, I thought. This is about interstitial spaces, hinterlands and bohemias. Places at the critical edge, where new ideas can be nurtured, battered around and so on; means by which powerful bureaucrats can absorb ideas from outsiders and then say (to themselves and others) that they themselves came up with.
But, as Pelling says, these groups can be (or be perceived as) a threat to the ideological/political power of those On Top.
Reflexive adaptation, especially that which seeks to challenge existing canonical institutions, is strengthened by a strong shadow system. The key challenge for organisations is how to support – but not manage – the shadow system…. Working to support boundary organisations and to better understand the shadow systems are two ways in which resilience as adaptation can be supported and potentially allowed to contribute towards wider movements of transitional and transformative social change.
Boundary riders have gotta be tough.
Wenger’s concepts of boundary organisations and individuals that can work to transfer information between epistemic communities helps to identify a key resource in adapting to climate change where conversations between science and policy are required to prevent maladaptation. They can also help to overcome the limited conceptualisation of climate change adaptation which continues to be framed as a primarily technical rather than a social and political agenda.
Page 67 (The citation is to E Wenger (1999) Communities of Practice: Learning, Meaning and Identity)
Reading Pelling led on, as these things do, to an internet search around “Shadow organisations”. From an interesting powerpoint presentation entitled Policy Development, Power and the Shadow Organisation by Graham Durant-Law (2009) comes this quote from a chap called Garry Robins
“Whatever a central management imposes, informal networks develop in ways that shape how an organisation works. These multiple networks involve information-flow, knowledge transfer, work cooperation, support, friendship and antagonisms. They are crucial to organisational functioning. … A multiple network perspective demands the examination of networks jointly, not one at a time. How networks associate and disassociate has implications for an organisation’s performance and the outcomes for its members.”
And google brought me to a conference paper called Interaction of the Legitimate Sysytem and the Shadow System in Organisations and in Haplin and Hanlon (2008) presented a paper with the following abstract –
This research examines the relationship between the Legitimate and the Shadow Systems in organisations: an interaction that can result in bringing an organisation into a state of bounded instability, and therefore increased creativity and innovation. The Legitimate System consists of the formal hierarchy, bureaucracy, rules, controls and communication patterns in an organisation.
A properly functioning Legitimate System is vital for the conduct of business in an organisation in order to ensure its survival and efficiency. The Shadow System is a term coined by Stacey (1997) to describe the informal network of relations within the organisations that are evident in casual hallway conversations, along the grapevine, through the rumour mill and in the informal procedures for getting things done. It harbours such diversity of thought and approach that it is often the place where much of the creativity resides within an organisation. Hence, it can be a great source of innovation if leaders could learn to listen to and tap into it. Stacey proposes that when the Legitimate and Shadow Systems are at a level of optimum interaction, an organisation can sit at the Edge of Chaos or be in a state of bounded instability. In this state the organisation hovers between equilibrium and chaos and is the ideal setting in order to promote change and maximise innovation and creativity (Brown and Eisenhardt, 1997).
Their paper is very much dealing with intra-organisational ‘shadows’, rather than relations with the world beyond. That’s because, after a very handy literature review, it’s case studies about two advertising/design companies…
“While chaos theory is primarily about deterministic non-linear systems that are mostly mechanical in nature, complexity theory accounts for social systems and human behaviour in organisations. It recognises the self- organisation and the emerged aspect in human organisations, and the fact that organisations are paradoxically stable and unstable and thus the potential for prediction and control is limited.”
(Stacey et al, 2000).
What’s “bounded instability”? I’m glad you asked. Halpin and Hanlon write:
At their outer edges, complex systems border on a state of chaos. The Edge of Chaos is a phase transition as a system moves from stability to instability (bounded instability). It provides a rich environment for creativity and the emergence of new behaviours. It is in this state of bounded instability that a system is capable of escalating tiny changes into radical transformations – such as the Butterfly Effect as discussed earlier. According to Casti emergence is defined as “an overall system behaviour that comes out of the interaction of many participants
– behaviour that cannot be predicted or ‘even envisioned’ from a knowledge of what each component in a system does in isolation” (in Lissack 1999, p11).
But for all the talk about innovation, who really likes being taken out of their comfort zone, eh?
According to Dervitsiotis (2005), living systems thrive only when pushed away from their comfort zone, where they must reconfigure themselves. The Legitimate System will ensure survival in the short term, but will not allow for the leeway for enduring growth or progression in the long term.
Attempts to introduce change into the organisation can produce anxiety. Anxiety reducing techniques take the form of defence mechanisms. Delahaye (2002) identifies two defence mechanisms which result in single loop learning or negative feedback loops:
• undiscussables: labelling certain activities or philosophies as ‘undiscussable’- no one can discuss them in the Legitimate System;
• defensive routines: The Legitimate System automatically commences activities that are designed to subvert the new idea
o ignore mode: Any attempt to introduce a new idea is simply ignored in the hope that it will go away;
o genuine management activities: committees can be formed but never comes to a conclusion. These committees are based on negative feedback loops and this soon crushes any creativity.
page 12 Halpin and Hanlon
They go on to talk about Self Organising Groups
(SOGs) can emerge in the Shadow System. The need for self-organisation is evident in unstable conditions and this raises the issue of the sort of leadership skills necessary to stimulate and facilitate such processes. Managers face two challenges; the ability to identify the emergence of SOGs and the ability to encourage SOGs to survive by providing energy, and therefore, positive feedback loops (Delahaye, 2002).
page 13 Halpin and Hanlon
And, also interesting, they write
Sbarcea (2003) suggests that there is such thing as a complexity leader. She suggests that the complexity leader has an enhanced awareness and intuition which will allow the leader to dance between chaos and stasis. Her concept of the complexity leader is as follows:
• the complexity leader should provide guidance, mentoring and coaching (not management or control);
• the complexity leader should allow for paradoxes; recognising the need for structure yet less structure; leaders know but also guide into the unknown; leaders have authority but not control; leaders explore possible future scenarios without committing to one, fixed reality.
And beating people up/finding a scapegoat when things go wrong.
Failure has a natural tendency to raise anxiety levels, in this regard every interviewee was asked how they felt their organisation and the people in it reacted to failure. According to Delahaye (2005) those who use punishment as a reaction to failure tend to focus too much on the Legitimate System and rely on negative feedback loops. Those who see failure as merely one step on the path to learning, tend to have more active Shadow Systems.
If nobody’s got your six, you’re gonna be very very risk averse.
So, there’s more to be done on this – especially reading Ralph Stacey. For me, the key task in the coming two years (asides from the Masters, being a better husband etc) is to create an effective shadow organisation that can survive its founder’s departure…
Durant-Law, G. (2009) Policy Development, Power, and the Shadow Organisation. GET WEB ADDRESS accessed
Halpin, C and Hanlon P, (2008) Interaction of the Legitimate System and the Shadow System in Organisations. Presented at 11th Annual Conference of the Irish Academy of Management, Dublin City University, September 2008. WEB ADDRESS date accessed
Pelling, M. (2011) Adaptation to Climate Change Routledge
Dervitsiotis, KN (2005), Creating Conditions to Nourish Sustainable Organisational Excellence,
Total Quality Management, Vol 16, Iss 8-9, pp. 925-943, Oct- Nov,
Sbarcea, K (2003) Living leadership: the dance between chaos and stasis. A guide for complexity leaders, http://thinkingshift.com/web/page.php?key=26
Stacey, R. D., (1997) The Implications of Complexity Theory for Psychoanalytical Thinking about Organisations, http://www.ispso.org/Symposia/Philadelphia/97stacey.htm
Stacey, R.D, Griffin, D, Shaw, P, (2000), Complexity and Management: Fad Or Radical Challenge to Systems Thinking? Routledge
Stacey R. D. (2003) Complexity and Group Processes: A Radically Social Understanding of Individuals, Psychology Press