Agile Software Development is dead . If you practice that, you are a doorstop. If you manage that way, you are a boat-anchor. The wave has ended, it is over, and if you went for the head-fake you wasted some money. Soon recruiters will be putting your resume in the circular storage container . I have been warning you for some time, and the day is here. Hah, you should have listened. Move along .
In truth, it will probably take a while. Many big things keep moving after death, like giant zombie worms. Big corporations, government agency business processes and obsolete technologies can be that way. Some small bits may live on forever, like the anti-hero’s hand or eye in a horror film.
All these hyped trends have a lifespan. Management fads especially have a lifespan. In the modern environment these waves are closer together, and closer, and closer. The end of the curve can mean unpopularity, few sales, reduced margins i.e "death". No more big money to be made, margins and fees reduced. This is opposed to the geek meaning, more akin to "brain death", implying the thing has no more technical merit. [Note: I ultimately had to spell this out explicitly for some of our less accomplished readers. So much for sarcastic subtlety.]
So the job of the consultant is to grab a wave and make money while he can , then grab another. He may not serve customer interests much, but he can eat. At the beginning of some waves you will find engineering and technical rigor and proof, but at the end more concern for consultant fees, revenue and profit with some normal concerns regarding the extension of marketing and sales.
These waves can be seen as the superimposition of Gartner Hype Cycle ™ curves for multiple technologies. Did you think it would last forever? Nothing lasts forever, but things based on solid engineering last longer. Guess where that puts Agile Software Development? In the trash. Manifesto indeed, I hope it felt good.
Who said Agile is dead? The founders of Agile and its practitioners said it, not me. Don’t go thinking I made this up. (I claim nothing myself regarding its current death, I just report the claims of many developers. It’s dead with or without me or my post. As for my interpretation of "dead", see above.)
- Dave Thomas was one of the original 13 who proposed Agile Software Development. He says it is dead . Hate to read? See the Video .
- Richard Bishop wrote an angry developer response , agreeing.
- William Edmonson says Agile is dead.
- Kent Beck, one of the 13, had little to say about if Agile was a success. "I don’t have a sound-bite answer for you on that."
- Andrew Hunt of the 13 said it’s dead .
- Scott Ambler is now pushing Disciplined Agile Delivery because the original stuff didn’t cut it for enterprise class systems .
- Hayim Makabee says it died of oversimplification, and provides myths .
- Andrew Binstock says Agile was corrupted in Dr. Dobbs
- Mike Hadlow explains why agile failed .
- Stuart Eccles says " The more you try and practice Agile the less agile you become. And vice versa ."
- Darren says Agile should not have been a management process , roughly.
- Nathan Dintenfass says the language used is broken .
- Garreth Dottin says it’s destructive .
- Kristopher Wilson blames the stand-up meeting .
- Erik Dietrich provides great sarcasm as to why it was all silly.
- The Editors at Software Development Times say mainly the term is dead.
- DogitalRoot says it’s too convoluted , and therefore dead.
- These guys have a whole website to take advantage of the death.
- There is a Twitter Feed .
- I left out several hundred. Sorry for that.
The death occurred some time ago, though few noticed. I thought its demise was inevitable myself. Yes, I participated some early on, then stepped back from the media event boondoggle . Why?
- Agile had asweet-spot, and a range of inapplicability, but everyone wanted to ignore that.
- I knew it was beinghype-marketed, and thefacts were secondary.
- There was a sacred mythology, strange terminology, special sacred tools and other weird cult behavior .
- I saw everyone hammering it to fit withmuch more importantcorporate controls. Clearly we hadsuboptimizationwith development thinking they were the most important part of the ecosystem.
- I participated in an Agile development effort where the focus on the user was clearly highlighted as a myth in transformation initiatives. The end-users had no idea that their organizational function was obsolete and redundant. There was no reason to automate it. Millions were wasted.
- We really need repeatable process, maturity , but as soon as you turn Agile into process you have the compromised junk design that results from trying to bridge a dichotomy .
- Agile did not support strategic goals well. Attempts to scale Agile to reach strategy fell short. There was no mechanism foralignment.
- Agile had (still has) an inherent problem with fiduciary responsibility. You do not know what the software will do before you build it, and you cannot estimate its effect on operations. Therefore you cannot calculate ROI before you start. You are writing a blank check for an uncertain return.
- It never did really work well forbig, complex systems. (Yeah, tons of people did an anti-agile manifesto , that was just my take.)
- The Agile over-reliance on user stories interfered withtest and evaluation, especially higher level test and evaluation and security testing. Insufficient testing is a problem bigger than the scope of developers .
- I knew emergent design wasjunk architecture.
- There were so many big Agilefailures that I lost track .
- Software quality has noticeably decreased over the last decade. MS Word 2003 is better than 2010. Lists of security defects for major software companies show high levels of open deficiencies. Even websites more often work poorly. This has occurred despite claims of higher quality in Agile, probably because various non-functional and derived requirements are simply never discovered.
- Then the new FITARA law seemed to be in conflict , so the big bucks in government contracting were at risk. For me, the Fat Lady was singing .
Others were skeptical too, of course. Here is a great article expressing a range well thought out of misgivings. Yet Agile has become the most popular software project management paradigm, certainly exceeding my expectations. People commonly fix some of these issues above, ignore the manifesto, and call it Agile anyway (AINO). Market penetration is high, real applicability is limited, problems have not been solved, the original supporters have left, the name is tarnished, implementations are highly modified away from original goals. To continue selling, marketing money will be shifted to supporting a new product name and highlighting the problems of the old product. Agile will be obsolete . This is how the market for such methodologies and services works.
What Can be Saved
I am certain parts can be saved . I participated in a large project about 1994 with small teams, daily standups (we sat down) for problems and weekly progress checks. It worked great. I also participated in what was essentially scrum development with a genius team in 2005- without a manifesto or various nonsense . In both cases the architect knew what the thing would do before effort began, and you could identify ROI. Surely there is more that works fine and can be saved if you take out the insane bits of cult logic.
User stories for example are fine, if you use them to produce real requirements. Poor requirements and poor testing have driven the software quality problems related to Agile. I do not here refer to very low-level testing which might be automated, but the rest of it.
The name of Count Agile may live forever, both undead and brain-dead. I recently heard I could program in FORTRAN- oops- Fortran again for 6 figures.
However the Agile Manifesto should be replaced with reputable research findings and serious management. This "manifesto" eschewed all management and engineering rigor in favor of laziness. Some of it should probably also be burned, buried and then a very big rock placed on top. Then a warning to future generations should be carved in the rock. ‘Something like "Naive oversimplified management ideology does not sell services forever, especially when promoted by non-managers. BOHICA!"
So next comes the DevOps wave . There is still a chance to fix this one.
- We couldtake a wider focus than last time on enterprise considerations.
- We could integrate corporate controls .
- We could recognize the real reason enterprise software exists , wheremost of this custom coding happens , and focus on code relevance .
- BiModal IT could fix some of it, if any of these analysts could grasp the full lifecycle and where each method fits for but a moment.
- We could lose the "full stack developer" mythology.
In the meantime, if you are currently using AINO (like my government colleagues) you are at the state of the art, and doing the right thing. Be prepared to adjust DevOps the very same way, accommodating governance and controls. Consider relaxing mandatory methodology rules to allow the PM to customize for the situation at hand. Consider that Agile may not be best for large, complex projects.Read this.
However if you ask me what direction we should be going in, I do have an opinion. While in manufacturing approaches like Lean and Six-Sigma are meant to improve quality, Agile and Dev-Ops are meant to increase volume. But the quality of our software has decreased, and the requirement has increased. The number of defects , including vulnerabilities is far too high . In enterprise software, the quality of software measured as "fit for purpose " is also become too low, decreasing operational improvements. If I were king, I would prefer to see a methodology using the (ISC)2 recommended practices in the SDLC, and the emerging OWASP practices , with more certifications like the CSSLP . I would like to see us turn away from greater volumes of sloppy software before it destroys civilization .
The Agile hype-wave is over. There were serious, fundamental problems. I really doubt anyone will fix those issues in DevOps. Enterprise customers will have to create hybrid approaches, just like Agile. Ourcurrent culturein software technology is not about real improvements. We currently prefer comfy culture over strategy and effectiveness .
If you really want to fix it, I’ll be right here to talk about that. In the meantime you might look up systems engineering, CMMI, corporate controls, security testing, operational evaluation and start thinking how that can be bolted on to DevOps. Or you can work in a product oriented software development company and stay away from enterprise software implementation. That oversimplified crap may work inside a software development company, but not in an enterprise customer environment. (Toby Wootton recentlyexpressed it well, with a PM spin.) Big enterprises have switched to AINO (Agile In Name Only).
In the meantime when you say "Agile Software Development" everyone will know you are referring to just another methodology, one that failed to produce the promised results, one that was widely implemented inadequately, one no better than Waterfall or Spiral overall, one with certain relative strengths and even more weaknesses. ‘No more magic dust. Several of the founders of Agile Software Development and many other influential developers have pronounced it dead. Only consultants and managers with a vested interest in the brand-name "Agile" still want it alive.
Here is the summary:
(1) Agile became a brand-name, with marketing hype. It therefore became subject to the rules of all such hyped products. First there is great acceptance, then a crash. A period of long term acceptance may occur, or not, based on real results. However the days of fanaticism and huge hype will be over forever. This is "death" in a marketing sense.
(2) As this happened, the deep programming thinkers who created and adopted the methodology became disillusioned as none of the vision was being implemented. Substitutions and changes did not accomplish the goals of the proposal. It was abandoned by those deep thinkers. The core technical merit was lost. "Death" in a the geek sense.
(3) Those who remained sold Agile in a highly modified sense, keeping the name but little of the original vision. This pragmatic view was driven by sales and marketing, as the name still had time left in the cycle to be exploited. Customers adopted Agile with massive changes to fit in with real enterprise governance and corporate controls, for example. See my post on AINO.
This last is the nature of current adoption, but the hype wave has ended or is ending. This is a post about hyped technologies and methodologies, marketing, and the end-game of a brand-name abandoned by its originators. It is about the cynical nature of what remains. It is about the old brand being currently superseded by the new brand, with fundamental problems unaddressed.
(Note: Just to be clear, software development haslittle to do with the common daily duties of an enterprise level architect . (EA does often keep the list of organizational standards. Agile could be among those. There is some possible relevance.) But, as some of you know, I was long ago a PM for integration projects, a solution architect, and did some of my own prototyping, wrote some drivers, did some assembler for a couple years. Hence my interest in this topic. I do not think of this as one of my many EA posts. No more flames claiming I think dev is EA . Thanks.)
(Note: This was partly meant as a public service to my colleagues who are still seeking the mystical one, wherein the screen and keyboard disappear and only your mind and the code remain. You published truth and the marketing machine ignored it. I took my shot. I did not expect all the management consultants still pushing Agile (to make money) to launch desperate attacks and claim it has an endless bright future. This is not why I selected the image, it was just not expected to be such an incendiary topic. Really. I just randomly picked that image. No?)