Agile is Dead
Enterprise Architecture Evangelist MSEA BSEE CEA3 CISSP-ISSAP PMP ITIL
2016 年 4 月 20 日
Are you an IT consultant or contractor? Agile Software Development work is dead. If you practice that, you are a doorstop. If you manage that way, you are a boat-anchor. The wave has ended, it is over, and if you went for the head-fake and bought certifications, you wasted some money. Soon recruiters will be putting your resume in the circular storage container. I have been warning you for some time, and the day is here. Hah, you should have listened. Move along.
[For the less intuitive, please note that these overstatements are intentionally somewhat facetious. Some detail oriented readers completely missed the point of the sarcasm here. Agile is not a hamster and cannot die dead, splat, all at once. Death is an inevitable process, and it will continue to die. See below.]
In truth, it will probably take a while. Many big things keep moving after death, like giant zombie worms. Big corporations, government agency business processes and obsolete technologies can be that way. Some small bits may live on forever, like the anti-hero’s hand or eye in a horror film.
All these hyped trends have a lifespan. Management fads especially have a lifespan. In the modern environment these waves are closer together, and closer, and closer. The end of the curve can mean unpopularity, few sales, reduced margins i.e “death". No more big money to be made, margins and fees reduced- this will happen when the market penetration is very high, at maximum, and then begins to fall. There will be too many people in the market, and this causes the salaries and fees to drop. This is opposed to the geek meaning, more akin to “brain death", implying the thing has no more technical merit. [Note: I ultimately had to spell this out. So much for sarcastic subtlety.]
So the job of the consultant is to grab a wave and make money while he can, then grab another. He may not serve customer interests much, but he can eat. At the beginning of some waves you will find engineering and technical rigor and proof, but at the end more concern for consultant fees, revenue and profit with some normal concerns regarding the extension of marketing and sales.
These waves can be seen as the superimposition of Gartner Hype Cycle ™ curves for multiple technologies. Did you think it would last forever? Nothing lasts forever, but things based on solid engineering last longer. Guess where that puts Agile Software Development? In the trash. Manifesto indeed, I hope it felt good.
這些潮流在不同科技的 Gartner 炒作週期曲線中看得出來。你認為會永久持續的技術都不會永久，基於工程的事物會比較長久。猜猜看敏捷在這條曲線的哪裡？在垃圾區。只是像口號一樣，希望我這樣說它不會受傷。
Anyway there is a cycle, so it will eventually die in a marketing sense- replaced by the next thing.
Who said Agile is dead? The founders of Agile and its practitioners said it, not me. Don’t go thinking I made this up. (I claim nothing myself regarding its current death, I just report the claims of many developers. It’s dead with or without me or my post. As for my interpretation of “dead", see above. This section explains a second meaning of “dead".)
# Dave Thomas was one of the original 13 who proposed Agile Software Development. He says it is dead. Hate to read? See the Video. (He said roughly that they had lost control of the term “Agile" so another term must be used. Quickly, by the rules of such things, only the soundbyte remained, that they had lost control of the term “Agile". Agile was dead, per his title.)
# Richard Bishop wrote an angry developer response, agreeing.
# William Edmonson says Agile is dead.
# Kent Beck, one of the 13, had little to say about if Agile was a success. “I don’t have a sound-bite answer for you on that." So much for brand enthusiasm.
# Andrew Hunt of the 13 said it’s dead.
# Scott Ambler is now pushing Disciplined Agile Delivery because the original stuff didn’t cut it for enterprise class systems.
# Hayim Makabee says it died of oversimplification, and provides myths.
# Andrew Binstock says Agile was corrupted in Dr. Dobbs
# Mike Hadlow explains why agile failed.
# Stuart Eccles says “The more you try and practice Agile the less agile you become. # And vice versa."
# Darren says Agile should not have been a management process, roughly.
# Nathan Dintenfass says the language used is broken.
# Garreth Dottin says it’s destructive.
# Kristopher Wilson blames the stand-up meeting.
# Erik Dietrich provides great sarcasm as to why it was all silly.
# The Editors at Software Development Times say mainly the term is dead.
# DogitalRoot says it’s too convoluted, and therefore dead.
# These guys have a whole website to take advantage of the death.
# There is a Twitter Feed.
# I left out several hundred. Sorry for that.
# 開始推廣敏捷開發的十三位初始成員之一的 Dave Thomas。他說敏捷已死，還有影片為證。他很概括地說他們已經失去對於敏捷開發這個名詞的掌控權，所以應該要另外定義一個名詞。
# Richard Bishop 以一位憤怒的開發者的角度回覆，同意這種觀點。
# William Edmonson 也說敏捷已死
# Kent Beck 同樣是十三位成員之一，對關於敏捷是否成功表示遲疑。
# 同樣是十三位成員之一的 Andrew Hunt 表示敏捷已死
# Scott Ambler 正在推動 Disciplined Agile Delivery 因為原本的東西不符合企業等級的系統。
# Hayim Makabee 說簡單來說敏捷已死 https://effectivesoftwaredesign.com/2014/03/17/the-end-of-agile-death-by-over-simplification/
# Andrew Binstock 在 Dr. Dobb’s 說敏捷已經崩壞
# Mike Hadlow 解釋為何敏捷開發失敗
# Stuart Eccles 說你越是想要認真執行敏捷開發，你就距離敏捷更遠。
# Darren 說敏捷不應該是一個管理流程
# Nathan Dintenfass 說敏捷這名詞已經被用壞了。
# Garreth Dottin 說敏捷是毀滅性的。
# Kristopher Wilson 譴責站立會議。
# Erik Dietrich 挖苦為何敏捷是如此的蠢
# Software Development Times 的編輯說這名詞已經死亡
# DogitalRoot 說敏捷正在兜圈子，接著會死亡。
The moral of this part of the story is that if you want to use a specific brand/term (Agile) in a specific commercial domain (software development) with a specific meaning (manifesto), then get a trademark from USPTO. Otherwise your naive efforts will be co-opted, and your control/meaning will soon be lost as money is involved.
The meaning (of the brand name “Agile) was lost, the technical merit was diluted, and those looking for technical excellence abandoned the effort, but for the efforts of “true believers" it might be history already.
The geeky death of the brand-name occurred some time ago, though few noticed. I thought its demise was inevitable myself. Yes, I participated some early on, then stepped back from the media event boondoggle. Why?
# Agile had a sweet-spot, and a range of inapplicability, but everyone wanted to ignore that.
# I knew it was being hype-marketed, and the facts were secondary.
# There was a sacred mythology, strange terminology, special sacred tools and other weird cult behavior. (For example many Agile practitioners are bullies, and want to browbeat you into agreeing with them. They will attack you or undermine your credibility when you disagree. I was just threatened yesterday 4/28 by one young acolyte.)
# I saw everyone hammering it to fit with much more important corporate controls. Clearly we had suboptimization with development thinking they were the most important part of the ecosystem.
# I participated in an Agile development effort where the focus on the user was clearly highlighted as a myth in transformation initiatives. The end-users had no idea that their organizational function was obsolete and redundant. There was no reason to automate it. Millions were wasted.
# We really need repeatable process, maturity, but as soon as you turn Agile into process you have the compromised junk design that results from trying to bridge a dichotomy.
# Agile did not support strategic goals well. Attempts to scale Agile to reach strategy fell short. There was no mechanism for alignment.
# Agile had (still has) an inherent problem with fiduciary responsibility. You do not know what the software will do before you build it, and you cannot estimate its effect on operations. Therefore you cannot calculate ROI before you start. You are writing a blank check for an uncertain return.
# It never did really work well for big, complex systems. (Yeah, tons of people did an anti-agile manifesto, that was just my take.)
# It did not work well in enterprise environments, where I do most of my work.
# The Agile over-reliance on user stories interfered with test and evaluation, especially higher level test and evaluation and security testing. Insufficient testing is a problem bigger than the scope of developers.
# I knew emergent design was junk architecture. They call it the Big Ball of Mud pattern.
# There were so many big Agile failures that I lost track. The success rate for Agile projects was 67% in 2012, not much improvement, aided by a lack of concrete completion criteria and dates no doubt, but the failures are mostly big and complex projects, highly visible, with big budgets. (Allen Woods finds this example enlightening.
# Software quality is low. The industry is not focused on quality. Lists of security defects for major software companies show high levels of open deficiencies. The ever increasing backlog of defects in Agile can be seen in the problems with the “technical debt crisis". Even websites more often work poorly. This has occurred despite claims of higher quality in Agile, probably because various non-functional and derived requirements are simply never discovered. It has affected development cost.
# Then the new FITARA law seemed to be in conflict, so the big bucks in government contracting were at risk. For me, the Fat Lady was singing.
The Myth of Emergent Design and the Big Ball of Mud
# 我沒有追蹤很多失敗的敏捷案例。但我知道在2012年敏捷專案的成功率是百分之六十七，而且沒有怎麼進步，大多數失敗是在複雜的大專案，無法忽視又有高預算的。（Allen Woods找到了證據）
Others were skeptical too, of course. Here is a great article by David Longstreet expressing a range well thought out of misgivings. Yet Agile has become the most popular software project management paradigm, certainly exceeding my expectations. People commonly fix some of these issues above, ignore the manifesto, and call it Agile anyway (AINO). Market penetration is high, real applicability is limited, problems have not been solved, the original supporters have left, the name is tarnished, implementations are highly modified away from original goals.
To continue selling, marketing money will be shifted to supporting a new product name and highlighting the problems of the old product. Agile will be obsolete. This is how the market for such methodologies and services works.
[Whatever the next wave is, there is certainly are a mass of deficiencies to highlight in the Agile wave. Surely they will be exploited? The point of this section is less that I am calling the baby ugly to anger the fanatic parents- but rather that others are probably coming to do so. They have plenty of ammo.]
The moral of this part of the story is that if you produce some compact politicised ideology (manifesto) consisting of principles and rules there will be unintended consequences. Success creates a religion or cult, and defeat is being ignored. No such doctrine is perfect. Thinking you will change the world with a manifesto is naive, and if you succeed you may not have improved the world.
What Can be Saved
I am certain parts can be saved. I participated in a large project about 1994 with small teams, daily standups (we sat down) for problems and weekly progress checks. It worked great, and I was a big fan. I also participated in what was essentially scrum development with a genius team in 2005- without a manifesto or various nonsense. In both cases the architect knew what the thing would do before effort began, and you could identify ROI. Surely there is more that works fine and can be saved if you take out the insane bits of cult logic.
我確信其中一部份是應該留下來。我在1994年跟幾個小團隊參與一個大型專案，每日站立會議（但其實我們是坐著）解決問題確認每周進度。運作得很好，我很支持這樣的做法。我也在2005年跟天才團隊參與核心的 Scrum 開發，沒有照著甚麼宣言或無意義的規範。在兩次經驗中，在實做之前都知道該做甚麼，也很清楚產出的效能數據。當然假如我們把瘋狂跟儀式化的邏輯拿掉，很多部分都可以保留下來。
User stories for example are fine, if you use them to produce real requirements. Poor requirements and poor testing have driven the software quality problems related to Agile. I do not here refer to very low-level testing which might be automated, but the rest of it.
The name of Count Agile may live forever, as an ideal, both undead and brain-dead. I recently heard I could program in FORTRAN- oops- Fortran again for 6 figures. Yet the specific term “Agile" as a brand probably has to go- Dave Thomas was right.
理想中敏捷會永存，也有可能腦死。就像是我現在仍可以用 FORTRAN 噢不是 Fortran 寫程式。當然這個敏捷的招牌必須拿掉，正如 Dave Thomas 所說的一樣。
However the Agile Manifesto should be replaced with reputable research findings and serious management. This “manifesto" eschewed all management and engineering rigor in favor of laziness. Some of it should probably also be burned, buried and then a very big rock placed on top. Then a warning to future generations should be carved in the rock. ‘Something like “Naive oversimplified management ideology does not sell services forever, especially when promoted by non-managers. BOHICA!"
So next comes the DevOps wave. It is the “heir apparent". If you buy software development services at a medium or large scale then someone will be in your office selling you DevOps this year. (When they do, they will say it is better than mere Agile. You have to buy it.) There is still a chance to fix this one.
# We could take a wider focus than last time on enterprise considerations.
# We could integrate corporate controls.
# We could recognize the real reason enterprise software exists, where most of this custom coding happens, and focus on code relevance.
# BiModal IT could fix some of it, if any of these analysts could grasp the full lifecycle and where each method fits for but a moment.
# We could lose the “full stack developer" mythology.
# BiModal IT 也許可以解決一些問題，假如它的分析家可以掌握生命週期，同時知道每個方法在哪個時候適合使用。
There are also various modified forms of Agile resulting from practical experience and improvement. Maybe someone will make a real marketing effort with one of those.
The moral of this part of the story is that if you started adopting Agile now, you would be a very late adopter. Adopting any such sweeping change has a high startup cost. You may be better off skipping a wave and moving directly to whatever comes next.
In the meantime, if you are currently using AINO (like my government colleagues) you are at the state of the art, and doing the right thing. Be prepared to adjust DevOps the very same way, accommodating governance and controls. Consider relaxing mandatory methodology rules to allow the PM to customize for the situation at hand. Consider that Agile may not be best for large, complex projects. If they force you to use Agile, agree and adapt whatever you need to to make it work- regardless of purists. Read this.
假如你現在其實行事就夠敏捷（像我在政府單位工作的同僚一樣），你就已經內化了敏捷的精神，這就很好了不需要改變。在 DevOps 這一波來時用同樣的方法去面對它，適應即將來到的管理及外部控制。試著針對方法學保持適度的彈性，允許專案經理對狀況做客製化。試著理解敏捷方法其實對大專案並不是最好的。假如他們逼迫你使用敏捷開發，表面上對信徒們表示同意，然後私底下在調整後的範圍內工作。讀讀這篇文章：
However if you ask me what direction we should be going in, I do have an opinion. While in manufacturing approaches like Lean and Six-Sigma are meant to improve quality, Agile and DevOps are meant to increase volume. But the quality of our software is too low, and the requirement has increased. The number of defects, including vulnerabilities is far too high. In enterprise software, the quality of software measured as “fit for purpose" is also become too low, decreasing operational improvements. If I were king, I would prefer to see a methodology using the (ISC)2 recommended practices in the SDLC, and the emerging OWASP practices, with more certifications like the CSSLP. I would like to see us turn away from greater volumes of sloppy software before it destroys civilization.
但假如你問我，若是要認真實行敏捷，我們該如何進行，我還真的有一點意見。當在公司內部想要推廣精實，或六西格瑪來改善品質時，推廣敏捷及 DevOps 來增加總量時。但我認為我們的軟體品質太低，需求常常不斷增加，臭蟲包含安全性問題的數目則太高。在企業級軟體中，軟體常常只要你"適合需求"，此時品質必定低落，同時會很難在操作上改進。假如我有權力改動，我就會在軟體開發週期使用這兩篇 ISC 提到的方法論並推薦認證過的軟體生命週期規範（Certified Secure Software Lifecycle Professional）中的幾篇文章。我們就能避開那些將摧毀一切的軟體開發方法。
These 5 Facts Explain the Threat of Cyber Warfare
The Agile hype-wave is over. There were serious, fundamental problems. I really doubt anyone will fix those issues in DevOps. Enterprise customers will have to create hybrid approaches, just like Agile. Our current culture in software technology is not about real improvements. We currently prefer comfy culture over strategy and effectiveness.
敏捷的炒作曲線已經結束。它有一些嚴重且基礎的問題。我很懷疑在 DevOps 中，這些問題有人能解決。企業級的顧客必須採取混合的做法，其實就像敏捷的精神一樣。我們現在使用的軟體科技環境並非希望真實改善情況。而是期待安全勝過於策略及效率。
If you really want to fix it, I’ll be right here to talk about that. In the meantime you might look up systems engineering, CMMI, corporate controls, security testing, operational evaluation and start thinking how that can be bolted on to DevOps. Or you can work in a product oriented software development company and stay away from enterprise software implementation. That oversimplified crap may work inside a software development company, but not in an enterprise customer environment. (Toby Wootton recently expressed it well, with a PM spin.) Big enterprises have switched to AINO (Agile In Name Only).
假如你真的想要改善問題，我接著才談這個問題。同時你必須查看系統工程，CMMI，公司控制，安全測試，操作的評估，然後開始想想看那些東西如何會在 DevOps 中發生衝突。你也可以改在一間產品導向的軟體開發公司工作，免去企業軟體的問題。在純軟體開發中會大大簡化我們遇到的問題。因為那些產品並非給企業客戶使用。（Toby Wootton 在這篇中：https://www.linkedin.com/pulse/dont-join-cult-stop-worshipping-agile-craftsman-deliver-toby-wootton?trk=prof-post 用一個專案經理的角度闡述了這個說法）。大公司其實已經夠敏捷了。
In the meantime when you say “Agile Software Development" everyone will know you are referring to just another methodology, one that failed to produce the promised results, one that was widely implemented inadequately, one no better than Waterfall or Spiral overall, one with certain relative strengths and even more weaknesses. ‘No more magic dust. Several of the founders of Agile Software Development and many other influential developers have pronounced it dead. Only consultants and managers with a vested interest in the brand-name “Agile" still want it alive.
Here is the summary of what I said:
(1) Agile became a brand-name, with marketing hype. It therefore became subject to the rules of all such hyped products. First there is great acceptance, then a crash. A period of long term acceptance may occur, or not, based on real results. However the days of fanaticism and huge hype will be over forever. This is “death" in a marketing sense.
(2) As this happened, the deep programming thinkers who created and adopted the methodology became disillusioned as none of the vision was being implemented. The name Agile was co-opted. Substitutions and changes did not accomplish the goals of the proposal. It was abandoned by those deep thinkers. The core technical merit was lost. This was “death" in a the geek sense.
(3) Those who remained sold Agile in a highly modified sense, keeping the name but little of the original vision. This pragmatic view was driven by sales and marketing, as the name still had time left in the cycle to be exploited. Customers adopted Agile with massive changes to fit in with real enterprise governance and corporate controls, for example. See my post on AINO.
This last is the nature of current adoption, but the hype wave has ended or is ending. This is a post about hyped technologies and methodologies, marketing, and the end-game of a brand-name abandoned by its originators. It is about the cynical nature of what remains. It is about the old brand being currently superseded by the new brand, with fundamental problems unaddressed.
Read the follow-on post, “Agile Is Dead 2″.
Then read Real Agility.
And also “The Enterprise Quality Gap".
(Note: Just to be clear, software development has little to do with the common daily duties of an enterprise level architect. (EA does often keep the list of organizational standards. Agile could be among those. There is some possible relevance.) But, as some of you know, I was long ago a PM for integration projects, a solution architect, and did some of my own prototyping, wrote some drivers, did some assembler for a couple years. Hence my interest in this topic. I do not think of this as one of my many EA posts. No more flames claiming I think dev is EA. Thanks.)
備註：這邊說清楚，軟體開發跟一般的企業等級架構運作稍有不同。（EA倒是有一個組織標準：敏捷就在其中。確實有一些相關性）但是，你們中的一些知道，我以前曾是專案經理，架構師，我以前也實作原型，好幾年寫驅動級組語。因此本篇算是我的興趣。我不認為這代表 EA 的我，請不要說這代表 EA 的開發態度。謝謝。
(Note: This was partly meant as a public service to my colleagues who are still seeking the mystical one, wherein the screen and keyboard disappear and only your mind and the code remain. You published truth and the marketing machine ignored it. I took my shot. I did not expect all the management consultants still pushing Agile (to make money) to launch desperate attacks and claim it has an endless bright future. This is not why I selected the image, it was just not expected to be such an incendiary topic. Really. I just randomly picked that image. No?)
…Another item off my “bucket list".