At writing-about-testing and other places, we've talked about improv. I get the impression that some people freak out when they hear the notion of improv being applied to software creation. "Improv! That's where people just start making up random stuff outta the middle of nowhere! We can't have that in our nice, disciplined, 100% predictable software production process! That would be chaos!"
The short answer is that improv isn't about guessing, or grasping at straws, or pulling stuff out of the air. Improv isn't a crap shoot. There are solid underpinnings to improv, which is why skilled performers can reliably do it.
First, a bit on Improv
Improv is something that happens in most every field of endeavour, not just those we think of as artistic. It's common in music; famously so in jazz. Improv in the kitchen is a familiar notion; tweaking a recipe or putting together a unique new dish both spring to mind. The Army, on the other hand, isn't usually thought of as a hotbed of creativity, but you just have to speak the jargon: look for the term "field expedient", and you'll find improv being used for everything from building customized radio antennae to repairing vehicles with gun tape and para cord. It's an age-old approach to solving problems in a small-a agile manner, and it has a discernible modus operandi.
Improv Involves Preparation
As oxymoronic as it may sound, improv requires preparation. When the time comes to improvise, you must be ready with the knowledge and tools you need to act. A sig (military communications tech) who wants to create an improvised antenna needs the formulae that dictate its dimensions as well as the wire and sundry bits required to construct it. A chef who wants to create a meal that will address a diner's special needs must have ingredients prepped and at the ready. A jazz musician who wants to be able to solo needs to know their way around their instrument, not just how to play a scale.
Improv Involves Expertise
To improvise effectively, you need expertise and judgement to know how to apply your ability to the situation at hand. A sig needs the wherewithal to know when it's useful to set up an improvised radio antenna and which types the local terrain will support. A chef must know how different flavours hang together, how different ingredients of a dish complement each other, and what kind of meals they can assemble with what they have on hand. A jazz performer needs to know where they can take a tune given its melody, chord progressions, mood, and so forth.
Improv Involves Creativity
Creativity is improv's defining element. It involves taking your preparation and expertise and, in the moment, creating something that addresses the unique situation at hand. A sig comes up with an antenna that gets the radio range they need while blending into the surrounding landscape. A chef comes up with an off-menu dish that looks spectacular and tastes amazing, but doesn't contain the nuts or eggs their customer is allergic to. A jazz musician blows a solo that's unique, expresses their style and mood, and gels with the tune, the rhythm section, the previous solo, and the atmosphere of the evening.
Improv Involves Style
Finally, every practitioner has a different approach to improv. Whether they dub it their style, their reperoire, or their bag of tricks, different improvisors will come up with different things in similar situations. Sigs may have a favourite set of go-to antenna designs; chefs specialize in different regional styles of cooking; jazz musicians have trademark riffs. Styles are unique to each person and their background, and while they seem to bring an element of randomness into the equation, they're internally consistent -- styles evolve over time, but don't change at the drop of a hat. Moreover, oftimes style is a factor of the improvisor bringing their most exceptional or best-practiced skills to bear on a particular problem.
It's not an accident that skilled improvisers can reliably and repeatably come up with something when called upon to perform. While improvisations are not reproduceable in a carbon copy sense, neither are they arbitrary, capricious, or based around mere hope or chance. They're based on a foundation of preparation, judgement, creativity, and individual skill. Far from being irresponsible, improv highlights how prudent it is to be prepared and agile -- it makes it possible to react to a situation quickly with a response that is masterful, timely, and appropriate.