James Coplien

> Either you're writing tests or you're manually running the software (manual testing) There's nothing wrong with manual testing. In fact, unless you have good set of E2E tests (most companies don't), you still need manual testing. At some point manual testing may become tedious and it's can be worthwhile to automate it to certain degree, typically with a few integration tests. But well-architected systems need very few tests to be reliable. Having a lot of tests is not something you should celebrate, it's a signal your architecture sucks. And if you find yourself writing a lot of unit tests, please read this: https://rbcs-us.com/documents/Why-Most-Unit-Testing-is-Waste.pdf

I'm a huge fan of the following essay. It provides a set of very rational, well considered and compelling arguments for less unit tests: https://rbcs-us.com/documents/Why-Most-Unit-Testing-is-Waste.pdf

Also, there's a followup by Coplien which has not been discussed: https://rbcs-us.com/documents/Segue.pdf So maybe this document should be the focus of the discussion now.

I submit that testing a bunch of trivial things isn't actually going to make it easier to catch bugs. I recommend this article: http://rbcs-us.com/documents/Why-Most-Unit-Testing-is-Waste.pdf

There was also a follow-up article [1]. My take on the two articles is that he argues that integration tests should be able to replace unit tests in most cases. However, in my own experience, both kinds of tests have their palces. Why unit tests are good: - You get well-tested parts that you can use in your integration tests, so that the integration tests truly catch the problesm that couldn't be caught at a lower level. This makes trouble-shooting easier. - Decoupled design - one of the key advantages of TDD - Rapid feedback. Not all integration tests can be run as quickly as unit tests. - Easier to set up a specific context for the tests. There are more details in the blog post I wrote as a response [2]. [1] http://rbcs-us.com/documents/Segue.pdf [2] https://henrikwarne.com/2014/09/04/a-response-to-why-most-unit-testing…

> I have a serious question, given that this was written by a consultant, is it possible that tests get in the way of completing a project in a timely manner, thus causing a conflict of interest in terms of testing? Doubt so, he works for a software testing consultancy. They've been around since 1994 apparently. http://rbcs-us.com/

The primary link for this should be to http://www.rbcs-us.com/documents/Why-Most-Unit-Testing-is-Waste.pdf . Coplien write a follow-up at http://www.rbcs-us.com/documents/Segue.pdf . Links to those, and more by the author are at http://www.rbcs-us.com/software-testing-resources/articles . There was also a long discussion about the original posting, almost exactly a year ago, at https://news.ycombinator.com/item?id=7353767 , with few hundred comments.

> Some people involved in the project are talking about "100% unit test coverage" as a target. 100% targets ignore that software engineering is different from programming: costs in engineering hours, resource invested, maintenance burden etc. all should lower the target. Also think hard if unit test is suitable for your program. Most compilers aren't unit tested. Distributed system unit testing requires an absurd mock infrastructure. > The project include some fault-tolerant distributed servers, proxies and various API proxies/converters Make the unit test target ~20% and focus rest of the time you would've spent on writing unit tests to making the system debuggable: all the best distributed systems are extremely debuggable, since there are no unit tests that will help you once there's an RPC (such things subvert any and all invariants you programmed) - https://rbcs-us.com/documents/Why-Most-Unit-Testing-is-Waste.pdf also p. fun read

Didn't know this article thanks! "Why Most Unit Testing is Waste" https://rbcs-us.com/documents/Why-Most-Unit-Testing-is-Waste.pdf

This paper has been public for years at http://rbcs-us.com/documents/Why-Most-Unit-Testing-is-Waste.pdf

James O. Coplin has a long of experience and is steeped in theory and practice. His list of publications is about 200 entries long: https://sites.google.com/a/gertrudandcope.com/info/Publications Unit tests are not free as they are also code that much is obvious. Coplin however delves also into less obvious aspects of impact of unit tests on design and also the organizational aspects. Ultimately coding patterns are going to reflect the incentives that govern the system. Software development is a lot about trade-offs. There is plenty to be learned here how to do it. A addendum by him can be found here: http://rbcs-us.com/documents/Segue.pdf but the meat is in the 2014 article.

Write them in terms of user requirements. When someone does X, they expect Y to happen. If a test breaks, it should mean that a user somewhere would think there's a problem. See for instance this: http://rbcs-us.com/documents/Why-Most-Unit-Testing-is-Waste.pdf You can write integration tests with the same tools as unit tests. Personally, I try to make sure all major paths are covered, but I won't test every little UI detail. It's faster to test those manually. But YMMV.

the core of TDD was to iterate by making a failing test, then making it pass Yes and when I first heard that 15 years ago (or whenever it was) it sounded like genius and I did it on a few projects. Its a useful crutch maybe if you're doing something very unfamiliar But as a dogma that must always be followed? And having a test for every single 'specification' and edge case? No - screw that, you end up faffing around for ages setting up mocks and testing stuff that doesnt deserve it. And you end up with a massive library of tests that get on your nerves. Also everything got completely bent out of shape for a few years because everything had to be injectable and testable. https://dhh.dk/2014/tdd-is-dead-long-live-testing.html http://web.archive.org/web/20180215225218/https://iansommerville.com/s… http://blog.cleancoder.com/uncle-bob/2016/03/19/GivingUpOnTDD.html https://rbcs-us.com/documents/Why-Most-Unit-Testing-is-Waste.pdf

I put together a reading list of the best writing on automated testing I know. Most of this is agnostic wrt the language you use, though of course examples are written in a language. ## Monographs Test Driven Development By Example (Kent Beck) Unit Testing: Principles, Practices, and Patterns (Vladimir Khorikov) The Art of Unit Testing 2nd ed (Roy Osherove) Working Effectively with Legacy Code (Michael Feathers) ## Various resources [Unit Testing Best Pratices](https://www.youtube.com/watch?v=k_ItB5btREU&t=2931s&ab_channel… (Vladimir Khorikov) [Testing Without Mocks: A Pattern Language](https://www.jamesshore.com/v2/blog/2018/testing-without-mocks) (James Shore) [TDD Lunch & Learn](https://www.jamesshore.com/v2/projects/lunch-and-learn) (James Shore) [Test Driven Development - What? Why? And How?](https://www.youtube.com/watch?v=llaUBH5oayw&ab_channel=ContinuousD… (Dave Farley) [The 3 Types of Unit Test in TDD](https://www.youtube.com/watch?v=W40mpZP9xQQ&t=22s&ab_channel=C… (Dave Farley) [TDD and the Lump of Coding Fallacy](https://www.geepawhill.org/2018/04/14/tdd-the-lump-of-coding-fallacy/) (Geepaw Hill) ## Against Unit Tests [Why Most Unit Testing is Waste](https://rbcs-us.com/documents/Why-Most-Unit-Testing-is-Waste.pdf) (James O Coplien) [On unit testing and TDD](https://www.youtube.com/watch?v=21JlBOxgGwY&ab_channel=JeruSanders… (Jonathan Blow)

> His solution works, but realistically only for small numbers. He never specified an input domain, and during development never tried to identify ambiguities between the API 'promise' that the function would take int32, the expectation by most that it's fast to factor 2,147,483,647 (2^31-1), and the reality that his code would take a long time for that. I think in one of the videos I watched, he explicitly said, that this is lame implementation and can be optimized (for example by looping until square root of the number). The interesting point, is that one wouldn't have to change tests for that. One would want to write a benchmark for that, to use instead of test. As in: normal tests are tests for logic, while benchmarks are tests for performance. Additionally, it is always true, that premature optimization is not a good idea. If you have an evidence, that current algorithm is not fast enough (knowledge of your dataset sizes + understanding on what would the reasonable speed be + benchmark to verify all of that), then you optimize, without behavior change. > I think Martin's TDD style works because the problem he uses are "simple" in an algorithm sense, which is why he can talk about a 'Transformation Priority Premise'. Martin says "Refactorings are simple operations that change the structure of code without changing its behavior", but as I wrote before, 'substitute algorithm' is not a simple operation. Funny thing I have found about 'SubstituteAlgorithm' refactoring, is…