Jeff, ya dummy!

@zombie_pigdragon

5 ай бұрын

Nobody's gonna run one of these things in production!

@JeffGeerling

5 ай бұрын

Haha not only topically relevant, but FIRST!

@YodaComedy

5 ай бұрын

40th like

@pearcomputers

5 ай бұрын

:D subbing, twice :)

@ubbgn

5 ай бұрын

@@JeffGeerling paying license fees? lmao

I bought your ansible book in 2018 which, in effect, got me to my job where I'm now. There, I'm responsible for a giant ceph cluster with 1250 OSDs . Enjoy your time with ceph, it can be a scary software when something goes sour :D ....and also a big thank you for helping me become a cloud engineer!

@KimYoungUn69

5 ай бұрын

Send money not words

favorite quote today,"....but my home network is only 10GB..."

@JeffGeerling

5 ай бұрын

Hehe... someday I'll upgrade to 40 or 100 Gbps...

The most complex raspberry pi setup doesn't exi....... This is above my pay grade.

You lost me at licensing :) If this becomes a brick if you dont pay licnese fees, that becomes a problem. I remember Datto doing that to many customers. Their bare metal restore servers were literally bricks.

@jani140

5 ай бұрын

Not different from cloud services and saas. I've been told, thats what people want theese days.

@markarca6360

5 ай бұрын

Also NetApp storage, which requires licensing in order to use their hardware with protocols like NFS, iSCSI, and SMB/CIFS.

@DallanLoomis

5 ай бұрын

I use 3 datto servers are my proxmox cluster XD 2 use standard asrock mobos, 1 uses a modified asrock mobo that took an asrock mobo bios chip

@PeterRichardsandYoureNot

5 ай бұрын

What world are you living in? Anything that is commercial and made for enterprise class is by subscription. It’s just the way they make money and keep the development cycle alive.

@TheLukernator

5 ай бұрын

​@@PeterRichardsandYoureNotthe development of ceph itself isn't funded by that subscription. the hardware itself is a bit meh (hot swapping disks is not possible) and the software itself is just a wrapper around ansible + cephadm.

Ceph is my fav for homelab - and now I have a small cluster at the office too! Finally some love.

As a "very large utility company" we had a 24 pi cluster doing predictive math for peak shaving.

@kevinthorpe8420

5 ай бұрын

Shame Sony took that away from the Playstation. Oh well their loss.

@Roxor128

5 ай бұрын

Could the work have been done on a GPU using OpenCL, or was the job something branch-heavy, necessitating the use of the cluster?

Jeff, Pi clusters aren’t crazy. The people who build them are. I should know. I have two of them. Keep the posts coming and stay well!

@JeffGeerling

5 ай бұрын

Takes one to know one lol

@jhvhest

3 ай бұрын

Only thing that is crazy are the prices. This hardware Is not in my budget anymore. It only becomes more expensive.

I think your videos are very informative.. So I would never call you a dummy..just tease you about hoarding all the Pi's..lol

@Roy_1

5 ай бұрын

How much of the total Rpi in existence does he have? What if it's like 10-15%? I realize that's probably not true, but it would be interesting to calculate.

@JeffGeerling

5 ай бұрын

@@Roy_1Heh, judging by the millions that are out there, it's less than .0000001% :) I'm not even in the triple-digits! (And still well below 50 total)

@Enixious99

5 ай бұрын

​​​@@JeffGeerlingFair I think a lot of us honestly think you have like 150 of them which in hindsight seems a bit unrealistic haha

@D3M3NT3Dstrang3r

5 ай бұрын

Jeff takes the " You will never know till you try" And I love it. Instead of asking why he asks why not and tries it. That's my type of fun!

@D3M3NT3Dstrang3r

5 ай бұрын

​@@JeffGeerlingYou know you have hoarded all the cm4s you just wont tell us 😅

Wow. I've introduced to the Ambedded in 2018, back then when the Ambedded's founder husband was teaching me Openshift & Ceph. He told that her wife is starting a ARM-based ceph cluster in a box. At that time, i believe it is using Cortex A7 or A53, but with almost the same rack chassis and 8 node config. in 2019, I've met her on the Taiwan Computex showcasing the Mars 400

@JeffGeerling

5 ай бұрын

This chassis is that! And it's pretty neat! Would love to see them build a new 1U clustered model with more speeed.

So any use of this "appliance" requires payment of monies to Mars in terms of license fees, even after an initial subscription? If I am correct, then the forever pay aspect of cloud storage is an integral part of this solution. I think the concept of fully owning your own hardware inherent in a Raspberry Pi cluster must not be forgotten or lost.

Funny story. A couple years ago I wanted a NAS but I wanted it open source. I was thinking of a TruNAS build until a Jeff on KZread mentions “ceph cluster”. The video of your build wasn’t even out yet but I was off to Google to learn about Ceph and before long I decided to build a 3 node Ceph cluster out of small PCs for my home lab. So far so good!

@jeremyjedynak

Ай бұрын

If you have a 4th node in the cluster, in the event that one node becomes unavailable -- intentional maintenance or unexpected node failure -- you can still operate in a cluster state (three nodes) that can absorb a single node failure.

Love to see how much you progressed on your Ceph journey. Keep up the good work!

Firstly I just want to say I love this. It is awesome to see more arm clusters. That being said, I think starting at $3,500 this feels a bit lacking. Especially the hard drive mounting seem super clunky. The other thing that kinda grinds my gears is why documentation is "liscense" based. Where you need to pay for support to get their documentation. Actual support I understand, but the manual and faq etc. seems a bit excessive.

@JeffGeerling

5 ай бұрын

Can't disagree here. Also note this machine was launched in 2019 I think. It's been a while and their newer boxes are a bit friendlier.

@penguinnh

3 ай бұрын

@@JeffGeerling Speaking of ARMs, has anyone noticed yet that you have three of them?

We opened up a vendor's box and there was an RPi 4 compute node in there, doing all the work

I love watching Jeff videos as it reminds me of the challenges I faced when writing acceptance tests for 5ESS-PRX switches in Saudi Arabia 40 years ago. They ran UNIX RTR and if one of the two main processors failed, theoretically only one call was lost. Never managed to prove that!

The Mars 400 really makes for the textbook example of why industrial ARM has taken so long to take off, and why Ampere has done so well in contrast. It has no resale value, the company behind it has no real reputation to speak of and they gave it a heavily used product name, it has no refurb or reapplication opportunity, and it's running on non-distribute closed blobs with minimal oversight. It's an expensive e-waste hulk to dispose of the moment Ambedded decides to drop support in any form or fashion and by extension a ticking time bomb for any business that decides to rely on it for mission-critical tasks, not to mention a security risk. As a CSE, I wouldn't want that thing anywhere near any network, cluster, or farm I'm responsible for.

@kevinthorpe8420

5 ай бұрын

It's running on basically stock Ubuntu + Ceph. You have full access to the innards of their software. It happily runs several other stock Linux deployments if you want to reuse the box. Our next closest option at the time was over a quarter of a milion quid and ran proprietary software.

@Nephiaust

5 ай бұрын

It could be the same for any other company...... Take Cisco and their $%$&%y HyperFlex product line; my last job just installed it ~3 years ago and Cisco announced this year that they are dropping it completely. And that product is so properity its not funny..... to the point where you cant do half the standard VMware management without using their backwards and for the most part broken services. Add to the point that unless you have an active support with them, you dont even get root access to the services on your own network (they recently changed it but *unless* you rebuilt or fresh installed, it wasnt easy to get the password).

How do I get extra arms, Jeff?!? - Reminds me of a "Rick & Morty" episode on "Gazorpazorp"!!!! LOL - Love You, Sir!

It's nice to know I'm not the only one that _still_ forgets sudo sometimes. It's been 30 years, I'll remember eventually! :D

@JeffGeerling

5 ай бұрын

I think my command history is like 90% up-arrow, and 9% 'sudo !!'

@Scriven42

5 ай бұрын

@@JeffGeerlinglolol! Command History is such a wonderful gift to the world. I can't remember how many times I just grep the history for that command I did 2 weeks ago!

That's sexy gear. I wish more people knew about these ARM solutions. I know people in IT maintaining tons of servers and they have no clue what an ARM SoC is. Many of their tasks could be offloaded by low-end to mid-range ARM devices consuming about 10 x less. But changing somebody their views takes time. This video can help for sure. 👍

@JeffGeerling

5 ай бұрын

And this server is actually an older model-the newer ones are much faster, and even more power-efficient.

@sazma

5 ай бұрын

Is it though? No SAS. Very poorly designed drive bays. License(s?). More drive bays than you can use. And 5 rusty spinners are going to give you maybe 800MB/s sequential writes. Doesn't seem very sexy for a base price of $3500 plus licensing.

@kevinthorpe8420

5 ай бұрын

@@sazma Why SAS? Each node has internal Flash, an M.2 and possibly an HDD or SSD. The aggregation is at the network layer. It's a textbook Ceph deployment with lots of small servers handling a smallish amount of storage each. The only major issue of this is slow rebalancing times. You can load 8 HDDs on them depending on the role of the specific node. Since this is a single chassis I would guess that the three missing HDDs are the bays for the management nodes. If you have the recommended minimum three chassis then you'll have one HDD missing per chassis. I agree it's not full enterprise grade kit. But it fills a niche that a few years ago nobody else supported.

@sazma

5 ай бұрын

@@kevinthorpe8420 Because it's Enterprise price. Perhaps you're right about 8 hdds, but from what I understood from the video, 3 supervisor/mon/whatever are suggested, so that seems like a possibility rather than a likely scenario. It's a fun tech demo, for sure, but it's definitely WAY overpriced and FAR better solutions can be had for FAR cheaper.

@ramdynebix

5 ай бұрын

@@sazmaClaiming it can be had cheaper/better somewhere else without proof is just hot air moving.

I ran Ceph on old Dell gear years ago. Very cool fs. 10gb network is required to get decent IO out of it.

It's amazing how well done your videos are, I always learn something. And it's just so much fun to watch, not the least because you have a great talent in sharing knowledge.

The lack of vibration dampening is a concern in any data center. Vibration can be a major issue when you fill out a rack if you're still using spinning disk media. However, it might not be significant if you only have a single 1u device with 8 drives. When you scale it up, that's when you're going to have problems.

@JeffGeerling

5 ай бұрын

It's a good reason to only buy Exos or other rated HDDs, they are built for high-vibration environments. And racks can (and should) have a little isolation... but they still have vibration induced by fans, other drives, etc.)

@xani666

5 ай бұрын

No, that's a very small problem. Much, much bigger one is lack of hot-swap. ANYTHING down in that server and you are for server-out operation. So the entire benefit of having 1 OSD per server (not having to worry about having multiple OSDs down when server dies), is entirely moot here, as you need to take it out to do even simple disk swap. Like, you could risk it doing it when it is on but that needs those rails that allow cables to be still connected when rail is fully extended... but that's PITA to cable and have shit airflow Also having option of hot-swapping (even if it was a pair-at-a-time) would also solve vibration as those trays almost always have some elasticity in...

@kevinthorpe8420

5 ай бұрын

@@xani666 Nope, as long as your cables are long enough you can access the cabinet to change hard drives, M.2 drives or even an entire node without a power down.

@xani666

5 ай бұрын

@@kevinthorpe8420 we had few servers mounted like that but it's just extra work to install that also blocks airflow, all to make servicing more difficult and time consuming. It's not worth to save few bucks, and looking at price of those you won't even be saving all that much. You can get say S5TH | D52T-1ULH for similar money but with drive shelf built into the server and 12 slots instead of 8 in 1U

@georgebrandon7696

5 ай бұрын

@@kevinthorpe8420 yeah. You can have cables long enough to slide it out. However, as mentioned, it's a PITA to cable manage, and it hinders airflow at the back of the rack. Congrats. You solved the storage access issue with even bigger issues. A solution like this is fine in a 1U chassis. IF you have an identical setup in another rack already to go to switch over when SHTF. Preferably three identical. This way you still have redundancy when one is down. Think Kubernetes. Think ETC clusters. Think anything HA.

Thats such a great little 10Gbit switch from Mikrotik!

You make me want to try this for myself! Now THAT'S awesome influencing! Mahalo for you videos!

This is a really cool storage appliance for a homelab! I didn't know that I needed one until you reviewed it And thanks for the free eBook!

Finally, i was waiting for something like this again

Sick video Geff very nice lay out wonderful design

I really need to spin up an ARM powered Ceph cluster of my own when I'm finished with the Switchwire...

@JeffGeerling

5 ай бұрын

Definitely! It's gotten a lot better-I remember a few years ago it was a struggle to get it running. Now it's almost on par with the x86 experience, and you fight with the normal things you fight with in all clusters, instead of incompatibility issues.

@socketwench

5 ай бұрын

@@JeffGeerling Hopefully it won't generate enough heat to melt the 3D printed server rack it's in. I really should have used ABS instead of PLA for it...

@boneappletee6416

5 ай бұрын

@@socketwench Sounds like it's time to prep another rack... ;)

I've ran CEPH on my ProxMox 7 node cluster a few years ago. While it worked well but it had performance issues whenever it's re-balancing. From what I understand that CEPH recently has gotten alot better at that so may give it another try on my backup Proxmox cluster at work. CEPH is great for what it is but you really have to know it's in and outs to get the most out of it. Which is why companies like this offer support contracts.

I would imagine the dual 10Gbps ports out the back are so you can connect them up with multi-chassis link aggregation to the upstream switches.

@jeremyjedynak

Ай бұрын

Also, network math within the device: 8x nodes with 2x 2.5GbE ~= 2x 10GbE

Nice video. Though I don't work with Pis, I find the whole cluster concept fascinating.

I understood about 4% of that. 4.5% tops. But it's a pretty box.

@JeffGeerling

5 ай бұрын

Honestly if there's one takeaway, it's that manufacturers should have a go at different color schemes for the front of their boxes :D

@domramsey

5 ай бұрын

@@JeffGeerling I have a Cobalt RaQ4 from the 90s that used to host all my websites. It died years ago, but it's such a cool looking blue 1U case with green lighting. I can't throw it away. I should really fill it with Pi's.

I don't even know what ceph is but this video is fascinating 0oO

Link aggregation is still worth it sometimes! even though the core of your network is only 10Gbe, having the storage linked to the switch at X2 (or higher) to the switch means that a single node on the network can only ever consume 50% (or less) of the total available throughput, it's good at least for multi user access. in your case though maybe not worth it, but just thought i'd mention it!

I will have to look for the previous video where you presented this six CM 4 compute module mother board. Looks interesting. I would like to learn Ceph just for fun and curiosity. Anyway this 1 U server looks promising, but for the company the version with hot swap drives that you can easily change on the fly is much more convenient. Of course it depends and this is just my private opinion. Thanks for sharing.

That is the reason is pretty hard to find one for our projects, he bought it all

I've unfortunately not yet had the pleasure of running a CEPH cluster... but I'd definitely love to set one up!:D

this chanel has opened my eyes to low power computing i now have about 10 "compute devices" ... nothing abouve 65w i am going to pick up 2 pi 5 to replace 4 of those devices i hope one day to have everthing running of battery and solar

Thanks Jeff. This video is informative and enjoyable. I know Ambedded is partnering for marketing purposes, but you don't make it a sales pitch. Great tech talk and honest reviews/testing. These kind of videos are why I have you in my notifications.

@JeffGeerling

5 ай бұрын

Yep! They were kinda surprised I wanted to test their hardware. I mean, I would be too haha :D It's not like the common homelab or Pi user would buy enterprise storage appliances! But I thought it was a unique enough machine, and had some interesting parallels to the Pi clusters I build a lot, and they were game! They're a neat company, and I know the engineers there do solid work :)

Who needs this? Raises his hand …slowly at first, and then yells “I DO!”

Liked 'cause the nice mention of Network Chuck! Already subbed.

We've been running those for a few years. They do what it says on the box. At the time we started this journey there was nothing in the price range to touch them, or space (8 nodes in 1U), or power ~100W per chassis. We've started having issues with hardware failures now they're older and we've overloaded them so we've had to move some of the ceph nodes (index nodes, not storage) off this hardware, but it still works. Our biggest problem by far is that support in a UK time slot is not easy to manage. If they had a support centre in Europe then I think we'd be much happier.

@georgebrandon7696

5 ай бұрын

"At the time we started this journey there was nothing in the price range to touch them" And that right there is why some people don't understand the price isn't actually expensive. Just the power draw alone pays for itself within a year compared to running multiple Ampere platforms. (Which weren't really available at the time these things were first a thing.) I swear those who complain about prices have never actually worked in a medium to large business to know what those budgeting reports look like. I see this same thing with professional camera equipment from hobbyist users. Complaints about how a camera body costs $6000 or a lens costs $10,000. A few grand for a piece of equipment is pennies on the dollar when compared to wages at home. Not to mention that money spent makes money. And many (most) companies don't have shoestring budgets. They actually have profit margins worth a darn.

you are living a week in the life of some of my testing! fun surprises sometimes.

☺️😌 he showed my blog diagram about Ceph 👍🌷 I’m so honored

@JeffGeerling

5 ай бұрын

Thank you for publishing it!

The Ansible part of the video was your "Squirrel!" moment 😂 Also, I understood maybe 25% of the whole thing, maybe because I don't know much about networking. Regardless, it was a fascinating video. 👍

@JeffGeerling

5 ай бұрын

Haha I know; I was setting the thing up and was (pleasantly) surprised to see ansible output right in the UI they built!

"You're crazy" "No poor people are crazy, Jack. I'm eccentric." (Speed)

I like the sound of the fans with a picture of still fans...

@JeffGeerling

5 ай бұрын

Those are actually solid 'ducts' on the back of those server fans (they're deep!). The front has the actual spinning fan blades, then at the back I believe the ducts help create a more turbulent flow to make sure the air back there mixes (or it might be something to help with static pressure... or both!). But the best thing is if you touch that your fingers don't get chopped off. Not quite the same on the other side of those fans!

I think this is the bare minimum for Klipper !

I maintain our companies ceph storage with >6TB and confused about the "disk per host-ratio" in this setup. There is a sweet spot with disks@hosts and sites. I would be scared to run and maintain this in production.

4:54 Kids these days with their "full-sized" 3.5" drives. Back in my day, we only had 5.25" drives in 3.25" high by *8 INCHES DEEP* bays... and we liked it. 5MB should be more than enough for anyone!!

@kevinthorpe8420

5 ай бұрын

Meh, back In my day a 160MB drive was the size of a washing machine drum. Which was the style at the time. (ICL 2900 series)

Love the concept, less excited in the licensing aspect of it but appreciate that I as a lowly home labber am not the audience for this product.

@kevinthorpe8420

5 ай бұрын

Buy is and don't licence it then. It won't brick. You can still install stock Linux. You just don't get support.

Over 400 pages? Oh my Jeff...

The color of that server reminds me of the old SGI machines...

Love your videos Jeff. Lol! Calls himself a dummy while successfully setting up a Pi cluster server. Me: watches videos on YT with a Pi 400. P.S.: I'm getting 1080p playback without a hitch on YT now. Perhaps the latest updates updated the Mali driver?

While I have no good use for this I can see the value in a pi cluster as a learning tool for those still using on prem hardware. My place of employment went to the cloud a few years ago though so we can pay a lot more for the same server specs as we had in our still running data center. Government work at its finest lol

Perfect thing to run a homebrew VTL on.

Hey man, the reason you are seeing 55k on FIO under Linux terminal is Linux BuffMem cache, i recommend you to remove your test file each time you run the test that way its not able to cache inside of your memory. Awesome video.

I feel stupid, I thought you said the vpn software was called timegate, I am glad you posted the url here, because I did look for timegate, was like how is a time sheep software vpn software. Thanks Jeff

@JeffGeerling

5 ай бұрын

Heh, you could travel through time with timegate!

Great video Jeff and looks like some interesting hardware. Who do you think would be the target audience for an machine like this? I'm not familiar with Ceph myself so don't know of it's benefits, but looking at your speeds I can't help feeling that a regular server would outperform in transfer speeds and come at a lower cost, although with higher power usage.

@JeffGeerling

5 ай бұрын

Generally if you needed some extremely reliable archive storage, you might consider a Mars 400 (or three!). For higher performance (like for active use, or for VMs in a cloud), you would want their faster machines (or other faster hardware, generally), so you could get lower latency and higher throughput. But for almost all home users, and for a lot of small business use cases, a typical NAS would be quite adequate.

Hmmm Twingate looks interesting. I've been considering setting up my Mac Mini to do that sort of thing, Twingate sounds like a much simpler solution than fighting with Apple bullshit from 2006, and at least the hardware side for it would be cheaper than buying basically anything x86. I'll need to come up with something to do with the ancient Mini though.

@JeffGeerling

5 ай бұрын

I used my 2011 Mini for a few years as a simple file server. Some of the Intel minis could run Linux and even do okay with some things like Jellyfin or Plex!

@annieworroll4373

5 ай бұрын

@@JeffGeerling I should look into more options for it, though mine is a fair bit older than yours. I do know I can upgrade the CPU and storage, if only there were quad cores that would run in it, that would probably be better for any likely task I'd put to it since I'm not a vintage Mac software enthusiast where I'd run into lots of stuff that isn't multithreaded enough to benefit from the extra cores.

Sillicon Graphics (SGI) Purple 😅. What do you use Ceph for?

@JeffGeerling

5 ай бұрын

Kubernetes / distributed storage. Right now in my 'production' homelab I'm still running mdadm RAID and Samba. But I'm considering moving my main storage to Ceph at some point. Not sure if 'forever' but it would be nice to run a local bare-metal Ceph cluster in production. Most of my usage has been on managed instances, and they're less fun :) (and more expensive).

Looking forward to someone selling Raspberry pies with dedicated GPUs too lol

Twingate doesn't inherently offer enhanced security by default. If you gain console access, such as through SSH for debugging purposes, you can easily move to other services and machines within the same Layer 2 network segments, bypassing TwinGate's access control. In contrast, I prefer Zerotier, which also supports ingress and egress filtering on a per-network basis, though it's not stateful, but it's free and can be self-hosted. NOW ... I'm getting myself a MARS400 for the office - definitely some good stuff! 😉

Do they ship these with a list of installed MAC addresses so you can set IP reservations before plugging it in? How are you liking that CRS309? I have two of these, one for my lab (XCP-NG on used HP servers) and one for work's production XCP-NG (as top of rack on new Supermicro servers).

@JeffGeerling

5 ай бұрын

I didn't ask about pre-configuring, but the one they sent me was configured with a couple customizations out of the box, so it's probably an option to at least have them send you the MACs. I love the CRS309-I have three of them now, they're quite handy!

Would love to see perf tests on this with sata ssds

This is pretty good, the 4GB of memory per node worries me though. If it's just the storage node, maybe it's fine.

@5:48 did Jeff suggest that keeping a larger number cool is 'Orwell and good'?

thanks for causing the pi shortage

I don't know much at all about anything and I enjoy your videos a lot. Never touched a raspberry pi even once lol. I love purple.

Well, It's nice to have connections and get free stuff.

you are truly the Raspberry Pi Lord. Your videos are awesome. EDIT: i actually commented before watching the video, even if there are not actual Raspberry Pis, i still think you are awesome

How do you upgrade this to a newer ceph version, what linux is running on these nodes, does it run a vanilla kernel?

Does ceph + (offsite backup or tape) count as 3-2-1 backup? There are more than 2 copies on ceph and the offsite backup is a different storage media

@kevinthorpe8420

5 ай бұрын

Ceph clusters can be multi-location and you can define the crush map to keep copies in both locations. However that can slow things down if the network link is slow. There are also replication strategies you can use for DR backups or archiving.

"my home network is only 10gig"... :P

A simple answer as to why do anything is "fun"

I would really like to see you run ceph on a 45drives server. It would be cool to see over 2 pb of data storage linked together.

@JeffGeerling

5 ай бұрын

For now my HL15 and XL60 servers are going to get deployed a bit more traditionally. But follow Network Chuck - he has a set of 45Drives servers he's working on building into a massive Ceph cluster!

Hmmmm. An idea for this could be to replace a dead board in a consumer(?) nas like synology or asustor. Those only generally do gigabit and have 2-4 drives. Might be a way to reuse the same form factor.

Seems like a good idea but wish there was a version that seemed more appropriate for home servers.... might just get the blade if it still exists when I manage to get a new job 😅

Hello jeff, can you explain how to check the computational capacity / calculation per second of a computer / server. Is this good benchmark to select better system for application or web server.

Nice Sun-inspired purple...

Morning Jeff, wow, you're doing fantastic work. Do they have Amiga on twingate?😂

@JeffGeerling

5 ай бұрын

Haha!

Hello Jeff, I'm currently training for the RHCE exam with all the ansible stuff. I was just thinking because you're mentioned in the book wrote by Sander van Vugt for the exam prep. Do you know him or it just happened that he mentioned you as author for ansible roles (not directly but as geerlingguy) ?

@JeffGeerling

5 ай бұрын

I've communicated with him before, but I don't personally know him-I think there are probably more than 50% of my Ansible-related contacts who I've never actually met in person :(

I get why YT keeps sending me these videos of advertisements for products I would never buy. This is big money. 💵

Now they only need to become widely available cheap on the second hand market with good support for running Debian on the individual arm nodes

@kevinthorpe8420

5 ай бұрын

They used to run CentOS and now run Ubuntu. You're fine. Debian probably works out of the box. I know Ubuntu does.

Next up, you're going to link a raspberry pi cluster to a cluster of raspberry pi clusters over 10g to create a megacluster.... or a clusterf**k.... :P

Nice appliance, Jeff! Do I understand it the right way: Ceph is a storage area networking (SAN) solution implemented completely in software?!

@kevinthorpe8420

5 ай бұрын

Indeed it is. The software on the MARS400 runs basically stock Ubuntu and Ceph. You can build your own cluster if you have a pile of servers kicking around but that's a pile of servers if you need the resilience, then rack space and, power.

put soft foam in the gap to avoid hitting the hard disk.

lol oh boy. The first couple times I thought you were saying CIF really weirdly

You get to play with the best toys.😀

0:06 well there's your problem, RSJ has taken over your account again!

Jeff , you're the main reason there's a raspberry pi shortage ! 🤣

Jeff . Youu and trafficlightdoctor should do a colab ...peope keep saying raspberry pi shoukd be used in traffic lights..love your input as I respect your wealth of knowledge.

yes dad i need this for school

I think computing companies should try the transputer archetecture again. the idea being that you have a proccessor with it's own onboard ram and switch to allow for any number of processors to work in parallel.

@another3997

5 ай бұрын

The transputer was, and is, an interesting concept, however, it didn't succeed in getting a foothold. A modern cluster of networked nodes could in theory make use of that concept, making each node equal and responsible for sorting things out with it's neighbours, but I imagine there's a lot of latency involved in the process. Even if the architecture were scaled down to modern CPU die sizes, would they be any faster than current multicore, multiprocessor designs? Bus contention and latency are big issues in all high performance systems.

@kevinthorpe8420

5 ай бұрын

@@another3997 That's how the computers on the USS Enterprise work. So maybe.Edge computing is a big thing nowadays.

NICE!

Jeff, you silly goose! No one is going to use that stuff in a real production environment.

A single Intel i9 CPU consumes twice as much as this entire appliance! Can't complain about its efficiency.

Interesting, but a little bigger than what I could use. Jeff can you do some content on half rack 1u equipment? I don’t know about the rest of the home lab community but I think that would be more interesting to us

@JeffGeerling

5 ай бұрын

It would certainly fit more racks more easily :)

what on earth this thing uses less power than my desktop idling.

@JeffGeerling

5 ай бұрын

Heh same here for my 4090 desktop. But you can get down pretty low on mid-range PCs these days! I still wish they focused more on idle power consumption, especially for peripherals like graphics cards!

@Adam-zf3bv

5 ай бұрын

@@JeffGeerling yeah my intel arc 770 uses 40W just idling but its memory bus is unable to clock down probably due to its design, 12VO looks like a very good standard for idle power.