Explores ideas that help to produce Better Software Faster: Continuous Delivery, DevOps, TDD and Software Engineering.
Hosted by Dave Farley - a software developer who has done pioneering work in DevOps, CD, CI, BDD, TDD and Software Engineering. Dave has challenged conventional thinking and led teams to build world class software.
Dave is co-author of the award wining book - "Continuous Delivery", and a popular conference speaker on Software Engineering. He built one of the world’s fastest financial exchanges, is a pioneer of BDD, an author of the Reactive Manifesto, and winner of the Duke award for open source software - the LMAX Disruptor.
Dave is passionate about helping software development teams around the world improve the design, quality and reliability of their software, by sharing his expertise through his consultancy, KZread channel, and training courses.
Пікірлер
You simply cannot measure developer productivity by the number of lines of code written (aka. SLOC, which stands for Source Lines of Code). This metric results in developers gaming the system by writing inefficient code just to inflate their metrics. In the worst case, developers will duplicate procedures and introduce unique bugs into various copies of the code. I once inherited a code base written by 32 developers over the course of 3 years. It was a nightmare. The combined work made up the code for two desktop applications and a communications server that ran as a Windows NT service. Neither of the desktop applications had been successfully compiled in more than a year. To top it off, there were no formal design specifications. After noticing a few duplicated procedures which had different parameters and minor differences in the internal code, my next step was to analyze the code base. I discovered that the majority of procedures had been duplicated between 3 and 14 times. While the intent of each procedure was to perform the same function, the procedures had to be compared against each other to determine which one should be retained. Then all of the code referencing the retained procedures had to have the order of parameters aligned (and in some cases created). Of course, there was no standard convention for naming variables or objects. My manager complained after two weeks about my productivity. He was trying to measure me by SLOC while I was converting the code base to an object-oriented design and removing thousands of no longer needed lines of code per day. I will provide an example to explain how this was possible. One procedure contained 8 lines of code which were repeated so many times with only minor changes that it required 66 sheets of paper to print it to hardcopy. This was rewritten as a single function called in a loop to populate a linked list which held the results. This printed on less than a single sheet, but it didn't help the metrics I was being held to. Fortunately, the senior management and customers were very impressed when I delivered new applications featuring modern user interfaces that loaded data into a tree view from a relational database in less than one second (where the last version that would compile took more than 80 seconds) and a communications server that no longer thread locked after 255 calls. Over the next two months I added the complete backlog of features requested by customers. I measure developer productivity by the number of elements that users can see, the features in the business logic layer and data access layer that they can't, and by customer satisfaction. If the product doesn't meet customer expectations, then nothing else matters.
Fantastic video! Thanks for covering this Dave. Observability is so important and I've seen the benefits of this in production - before and after.
Preach!! 🙏
Great video !
I have to disagree mightily here.. Engineers have a thorough understanding of their area of knowledge from first principles and are primarily designers. Craftsmen have experience and a visceral feeling for their chosen medium and are primarily makers. Both are creative and innovative.. As a wise man once told me, anyone can understand how to lay bricks, few can lay them fast enough to make a living
What we do is programming. That is a profession on its own and it is around since decades. We are not engineers, we are not craftsmen, as we are not salespersons and we are not artists. We do programming aka software development, and usually for a living. Software development comes with its own challenges, while it can involve several skills and activities, which are also helpful in other professions: Creativity, planning, experimenting, measuring, reasoning, testing, communication (between humans) and so on. These are human skills and activities, not attributes of a specific profession. This should not be confused. A rigid view of the world does not reflect its very nature and not our own. So we usually do programming for a living and that is all there is. The hardware which we are programming is completely deterministic. We make sure by excluding randomness with checksums. Measurement and reality checks are deeply integrated into the core of our hardware. The basic building blocks of our profession are very obvious, there is no room and no need for philosophical controversy because of the fundamental principles of what we do for a living. We are not philosophers. So of course we have to build a scientific approach and reality checks into what we create on top ("engineering") when we are interested in results, by thinking and acting in a goal-oriented way. To deny that is just obviously careless. Of course it helps to strive for software to be "well-crafted". What that means may depend on the particular goal, while a well-structured piece of software seems to be more helpful than a chaotic approach. This is just obvious and not a contradiction at all. The universe will have evaporated into nothingness some day, going from order to entropy and beyond. Order is what is needed to resist that a little longer. That insight can and should be applied to software, too. Everyone can establish or become a member of an organization of professionals in particular field. That is a good thing. Call it a "guild", why not. Other professions have that as well. Just do not impose a certain definition upon others by excluding a scientific view of the world. Complete your personal view of the world and go beyond an over-simplified claim, which may even be a good one but not enough by itself. And there is always change. We do not need prejudice against an objective aka scientific view of the world. That is just from the past.
What ever happened to system testing?
Both people in the video are completely wrong. Software IS art not engineering. Here’s the proof. Ask 5 structural engineers to engineer a bridge and you get the same more or less bridge in terms of load, materials, tolerances, etc. Because there are maths involved based on physics and the limits of raw and engineered materials. Ask 5 software “engineers” to write a program and you will have completely different programs and hence interpretations of the requirements. Real engineers aren’t creative they have to adhere to limits of physics for both safety and functionality. When a civil engineer designs something that won’t work because of the limits of physics, they don’t just invent a new molecule or bend the laws of physics they must conform. When a software engineer can’t express what they want in a language they just invent a new one so they can. Perl, node.js, Java, Haskell, LIsP, and dozens of other languages are testimony to the art of software and not the engineering.
What are your thoughts on backlog refinement? Need the whole dev team to have a meeting every Tuesday / Thursday to groom the backlog?
I'd be curious to hear what you think of the book "Software Craftsmanship: A New Imperative" by Pete McBreen and Mike Hendrickson (forward by Dave Thomas). I like their approach to guiding one's own personal development as a programmer, and his notion that educating new programmers should look more like apprenticeship than credentialing, but that could be my own biases as a self-taught programmer showing through. I don't think craftsmanship and engineering are mutually exclusive but rather complementary. Anyway, thanks for the interesting conversation.
Micro-services are not a new idea. UNIX was built around micro-services, only back then they were called processes. So if we want to have a truly useful micro-service architecture, we can learn a lot by looking at the features that the UNIX architecture provided. UNIX offers solutions for: * Configuration of services * Starting and Stopping services in a pre-defined order * Interactively monitoring, starting and stopping services. * Interactively monitoring the input and output to services. * Logging. * Setting a unified Time for the services. * Offering a unified method for reading and writing data, which the user can (re)direct, store and replay as he/she pleases. This is very useful during testing and trouble-shooting. * Methods for authentication & authorization. As it stands, UNIX is locked into the single "computer" paradigm. I wonder why people aren't more interested in e.g. Plan9, which elevates the UNIX principles into a Cloud operating system.
I think the tension between craft and (aspirationally) engineering for software is found even in other media. I remember the cartoons that came with the Metrowerks CodeWarrior products in the 1990s. CodeWarrior (IDE, compilers, etc.) had these cartoons of a factory worker - manually creating giant CDs by hand. Metrowerks was so small I wonder if one of their developers (and making dev tools no less) was involved. I do know there was an open secret that the character was named "Arnie", after the movie star everyone knows about.
Interesting perspective! I would say the craftsmanship manifesto is more informative about what that movement was about, and maybe the name wasn't the best.
I have met software engineers, because I worked in aerospace, but 99% of those who call them selves such are just stealing valor. Software engineers working under a licensed professional engineer with liability insurance to sign off on the systems design and architecture, sure, you pass the gate and are engineers. If you don't know what licensed means or why liability insurance is involved, you are not an engineer.
"I generally prefer to have a layer of my own code between almost any 3rd party code, and the rest of my system". This is the heart of something pretty deep and difficult, I think. I think it'd make a great initial assertion for a whole episode.
The interesting aspect of 'Engineering'-ensuring that an app is robust, secure, scalable, and meets all technical specifications-is that it inherently relies on 'Craftsmanship.' Attention to detail, crafting tailored solutions, maintaining high code quality, and focusing on user satisfaction are indispensable for achieving these engineering goals. Therefore, the notion that engineers do not engage in craftsmanship can be misleading. While terminology preferences vary-for instance, preferring specific professional titles like 'Informatics Engineer' over general terms like 'developer'-it's essential to recognize that both craftsmanship and engineering are fundamental to the success of software development.
I think you might be using craft in a very specific way rather than in its more general use to describe something one practices to get better at. Such as 'writing is a craft'. I find the comparison to childhood creations off the point. Software is so ephemeral that it works like a mirror- descriptions of it often illuminate the describer more than the described. So, since my my major was literature, I have always treated software development like any other writing assignment. And that has worked very well.
Support comment, please ignore
I'm not sure how we can implement ATDD in a microservices architecture without essentially conducting end-to-end tests. If our acceptance tests require interactions between all our microservices and the frontend, doesn't that mean we are essentially performing end-to-end testing?
I grabbed hold of "software craftsmanship" when the idea first arrived. I was trying to convince small businesses that the custom applications I could provide were superior to the fragile Excel solutions that my competitors offered... I even tried to join The Guild of Master Craftsmen.
this is pretentious and fart sniffing. this is all about ego, and no one who is a serious person has time for that noise.
Why do you/we have to break our heads so much about craft vs this-that-or-the-other? You think engineering is the height of human creativity, really, how self congratulatory... and software engineering? But otherwise I agree with all else in this episode.
A great video, wish someone had told me these things a decade ago.
Are you cooler than the Primeagen?
Me, I only write ARTISANAL software.
There is little engineering in software outside of the biggest/most advanced shops. We have no national/international guidelines and rules. We arenot nearly as rigid as we should be when bringing in random third party dependencies. We do not hold our vendors to account for failures or bugs. We are do not take accountability for our bugs etc. Engineering isn't a creative field. Engineering is about applying tried and true guidelines. Engineering research can be creative. But no one builds an X on the hopes and wishes it doesn't break.
I see no problem in romantizicing. I believe in a term equality between "well engineered" and "well crafted" when it comes to software engineering. Semantics matter here, software development is very opinionated, rarely scientific. Code Readability and good design has little to do with science and a lot to do with craftsmanship. I know some people frawn upon concept of beautiful software but pursue of elegance feels just right to me. I'm not alone. Leave me alone with my craft that you consider "just engineering".
I agree with both of you Dave and Emily on a lot of the items that you bring up. However to me in order to be called an Engineer I feel there needs to be a regulatory body or licensing aspect to the title. Much like other Engineering occupations. Until there is regulatory or licensing requirements it really is just code craft. It would be awesome to have some sort of regulatory organization that would allow for developers to get a standardized score for something instead of mistakenly applying domain expertise and tenure for the measure of "how good one's programming skills are".
This seems to be more about association, and the question is a linguistic one. To me craftsmanship is about a few things, mostly creation and application. Both of which apply to engineering. To me, engineering is a more specific version of craftsmanship. I've had the pleasure of working with civil engineers at my current job, and to me it's just another form of creation and a craft. I think the reason why programming is special in this is because it's deceptively less theoretical compared to other forms of engineering for most people, I consider some code to even have artistic value in a sense. Although the audience for this "art" is quite narrow compared to more common forms of art. In the end programming is on a spectrum, from having very strict design requirements to having full freedom. And I think the more you go to the free side of the spectrum the more it looks and feels like craftsmanship. To me it seems your perspectives are ones that relate more to the restrictive side of programming, which is completely fine and understandable, but if I'm right not representative of programming as a whole.
You guys lost the plot. Maybe you should reread the pragmatic programmer and recontextualize this discussion. There are often many ways to solve the same problem, a mathematician would argue infinitely many ways, craftmanship among other things is coming up with rather arbitrary rules and assumptions most people share to e.g. make code more maintainable. And if you think pair programming is a good idea, you're also a craftsman, that's how craftsmen and women were taught for ages.
Who gaf. Do what you love and get paid. The only thing that matters is the impact you make with software, and the value it adds to your own life.