Jenkins is a security dumpster fire. I haven't worked with a single company in the past decade that has consistently kept Jenkins updated, much less the myriad of plugins that do most of the heavy lifting in Jenkins. The last company I worked with, a health insurance company, was running a version of Jenkins that was a couple years behind the current release and publicly reachable from anywhere on the Internet. When I informed the development teams that we were going to adopt a more frequent update cadence for Jenkins, they pushed back because they were afraid that any changes to the build environment would trigger failures in their builds.

@ancbi

3 ай бұрын

Could you give specifics of how Jenkins is the source of security problem? Does its update really break pipeline often or not? So far it your story the blame seems to lie with Jenkins' user.

@carpdog42

3 ай бұрын

@@ancbi I think we have a bit of a double problem here. Yes a lot of blame lies within the organizations that are deploying and failing to secure or keep their system updated. However, we also know from decades of examples that, on average, all installs will be done as default as possible, and thought of as little as possible because people install tools to use them, not for the love of installing and updating tools. Realistically, I think the real genius of modern workflows is taking away the whole concept of upgrading software and moving to we just use the latest version all the time. Don't give them the option to be afraid.

@ancbi

3 ай бұрын

To give context, I'm not asking to pinpoint blame. I'm asking because I'm considering trying out Jenkins in the next project. I want to know if Jenkins give reasonably secure default values and have good documentation.

@carpdog42

3 ай бұрын

@@ancbi I can't speak to that as I have never actually installed it myself. My experience actually comes from working at a company whose name was the same as one of the major software products it sold... which was also the example usename and password in the docs, and well... the production username and password at some shocking places.

@JohnDoe-bu3qp

3 ай бұрын

We use Jenkins where I work and from my limited experience sometimes API method signatures and thrown exceptions change between versions, to the extent that auto-retry mechanisms may fail depending on the versions used for example. I think the key problem is the plugins. If you focus more on using Docker containers to do things instead of Jenkins-level abstractions you'll have less blockers to keep Jenkins updated and a more loosely-coupled CI system overall. But even that comes through a plugin, so you will definitely still use some.

CI software is just the tip of the iceberg. We also have other supply chain attacks. I was amazed today to find that a brand new container i built based upon latest stable Debian had two critical vulnerabilities in it. I wonder what that would be like if i didn't update after a few years. Software BOMs are becoming more and more important. Sometimes easy to automate but you have to be prepared to deal with the upgrades, new ways of doing things and regressions. That's why it's engineering and not art.

Props for dropping Tricentis as a sponsor, if that's what happened

@Yxcell

3 ай бұрын

What happened with Tricentis? I didn't see any news about something bad happening with them.

@brandonmansfield6570

3 ай бұрын

Could also be: Payback on the sponsorship isn't providing the returns that they hoped. Recent video cycle has had lower viewership, making them reconsider.

@ContinuousDelivery

3 ай бұрын

I am not sure what you are referring to. We are very grateful for Tricentis' sponsorship over several years. We parted on good terms.

Jenkins kept up-to-date in environments running windows, using same user by everyone for both RDP access and jenkins running just as-is, without other nodes in various companies? Wishful thinking for most companies :)

Bitbucket pipelines were just down this morning. Some bug caused everyone’s pipelines be stuck in a “pending” state. I wonder if they’ll release the root cause.

@JeremyAndersonBoise

3 ай бұрын

🤣😂🤣

After my experience last week with GitLab running my own servers. Its most likely hacked too.

Issues like this is why I do not give my credentials for e.g. pushing changes to my repo to a tool like my IDE. I will use the git CLI for pushing code to servers, thank you very much. Security starts with limiting the number of parties you trust with your credentials. I will not entrust an IDE with my credentials, and I think it is wrong of Jetbrains, in this instance, to ask for them. Same for SSH private keys. Of course, the danger from CI / CD pipe-lines is even greater, as these need to have credentials to access the production pipeline. I hope this episode will teach people to be more careful with what they entrust their tools with.

@mudi2000a

3 ай бұрын

May I ask how did you do your setup so that you can use the gut cli but the IDE can’t push to git? The way how I set it up I configure only the git cli and then I can commit from both the cli and all IDEs. No IDE has ever asked for credentials as it doesn’t need them, it is handled by git itself.

Well, the more dependency you have, the larger your attack surface is. Building a huge network of build tools dependency and package dependencies, with untold injection points, while keeping it zero day safe? That's not a pipeline, it's a pipedream... Like anything, when you over buy into the hype train, you later find out you solved a lot of problems you had and gained even more you didn't have before.

Wonderful video! Sadly @,@!

first

The best attack are code contribution to open source software.

@damoates

3 ай бұрын

The University of Minnesota Linux incident is my go to story for showing how even the most strict open source projects aren't necessarily safe.

@lucasrecoaro1701

3 ай бұрын

your comment is simply non-sense.

@llothar68

3 ай бұрын

@@lucasrecoaro1701 did you not listen to Edward Snowden? 10 years later and people are still stupid and ignorant

@d3stinYwOw

3 ай бұрын

@@llothar68 You mean dude who run to russia and selling their stories? :P