The waitgroup verison of the example is MUCH easier to reason about. I would hate to have to debug that function.

@IamI16

2 ай бұрын

me too

@pmartin

Ай бұрын

If the function is implemented by a random CoWorker, with a silly name, obscur pattern etc. Yeah obviously, I prefer the WaitGroup version. But, If Parallel becomes an actual thing in std, I'll be happy to use it. It's like Generic, random package using Generic for nothing or container that I never needed, is a redflag for me. But, Generic helpers like `cmp.Or` or the `slices` package is actually a great thing which remove a lot of boilerplate.

I think a lot of the potential harm that could be caused by this feature would be almost entirely mitigated with a strong standard library implementation of many/most of the most common use cases. I think this approach is fine, and generally needed when writing a lot of iterator-heavy code. This seems fairly reasonable. What I get scared of is every sophomore programmer thinking they need their own implementation of exotic iterators, writing their own package, and it just creating a bunch more headache that would've been prevented with a simpler or more explicit approach. Not everyone should get equal access to parallel iterators! That comes with some very real risks!😆

@sevenmore3355

2 ай бұрын

This feature sounds more and more like templates in c++

@kenneth_romero

2 ай бұрын

also couldn't he just put the code in the for loop in its own function to make the logic more readable? seems like so much work to create all that code to have an iteratable function

The syntax is not the most readable, but overall, I'm in favour of the proposal. It will be a very useful tool for the Go devs and for library devs like myself. Endusers don't have to deal with the inner complexity as much and can have a more homegeneous experience dealing with ranges. It's up to the library devs to write self-documenting and readable iterators.

This is really powerful. But it feels very un-Go-y……eg it’s too much magic. Obfuscation isn’t good for Go, I know verbosity gives it a bad repo, but I also think this forces better code

@l_unchtime

Ай бұрын

That's why it's experimental, and will improve over time :)

Nice feature to extend loops, but such an implementation feels difficult to handle and violates the value of simplification. So, whatever the Go Community prefers :)

Go is such a cool language, I just started learning it so this was very helpful. Thanks Dreams!

Elliott, pls, make a step by step guide on how to use pprof professionally.

@flannn6

2 ай бұрын

THIS! cpu and memory profiling.

@wodxgod

Ай бұрын

yesss

Just what I needed! TY TY TY

very interesting! great quality content as always. thanks

Python and C++ (modern C++, anyway) make fairly heavy use of generators (in C++ up to 20 they were implemented with input iterators; C++23 has an explicit std::generator class). It's a very powerful pattern for parsing data streams, for example. Node has had something similar since basically forever.

This will be very nice to work with when using remote data sources - just treating them as arrays, imstead of something like 'for val, err := source.Next(); err == nil { dostuff(val) }

Nice to see Go finally getting generators. I use this pattern a lot when working in PHP, so good to see it's going to be possible to do it in Go.

5:40 that moment when I'm finally stupid enough for Go.

what do you use to animate the code in your videos?

The syntax of "functions returning functions accepting functions that you can pull values from" is too complicated and magic-y. It seems like they're trying to stretch the extra-simplistic syntax of Go1, when it might be better to learn from its shortcomings to come up with Go2 But that's my humble opinion

@ForeverZer0

2 ай бұрын

They try a little *too* hard to never add any new keywords or core types to the language, and always reuse what already exists, in this case "for..range". Sometimes it would be better to just expand the language a bit for the sake of clarity. Reading the developer discussions on various language proposals is a frustrating experience: practically everything is rejected if would require anything new whatsoever to the syntax of current lexicon/vocabulary of the language. I appreciate that they err on the side of simplicity, it is one of Go's greatest strengths, but then there is examples such as this, where it actually makes the language more complicated by being so strict.

Parallel processing can definitely get interesting when hidden somewhere down in code you don't know about there is a shared resource. Something like a memory allocator perhaps.

This video is great. The explanation and animation is great. One nit, the Parallel implementation is wrong. It works because it is unlikely that that race condition will be met once you've canceled the context, but another goroutine could be yielding in as you cancel. This is a classic case where you need a mutex. A potential solution could look like this: func Parallel[T any](list []T) func(func(int, T) bool) { return func(yield func(int, T) bool) { var ( wg sync.WaitGroup m sync.Mutex cancel = make(chan struct{}) isCanceled bool ) wg.Add(len(list)) for i, e := range list { go func() { defer wg.Done() select { case

@what1heh3ck

2 ай бұрын

but it will no longer run in parallel right?

@daviddesmarais-michaud4345

2 ай бұрын

@@what1heh3ck well this example is trivial. Technically you would want to be doing doing some work to generate the items you yield. In this case we are doing no work, and since we need to synchronize at the moment of yielding as per the spec or risk a panic, then in this case there is no advantage to using concurrency and only serious drawbacks. But as an example of how concurrency could be built into function iterators it’s great! TLDR: we don’t want concurrent yielding, but concurrent generation/work.

@TheArtikae

2 ай бұрын

I’m not super familiar with Go, but isn’t your code holding the lock while it calls yield? Yield being the function that runs the body of the for loop, right? The entire point of this wrapper is to run the body of the for loop multiple times in parallel. Your function doesn’t do that, as far as I can tell. All your version seems to do is run a single-threaded for loop split across N threads. I’m also not clear on what race condition you’re trying to avoid. The entire purpose of this wrapper is to call yield from multiple threads simultaneously, so that can’t be it. I suppose there is the case where one thread calls cancel while another thread is in the middle of calling yield. However, that’s an unavoidable consequence of the task at hand. If you run a for loop in parallel, you don’t get any guarantees about iteration order. Break could never work like it normally does.

@daviddesmarais-michaud4345

2 ай бұрын

@@TheArtikae you are correct. However the code shown in the video is not safe. The context does not definitely block two goroutines from yield racing whilst one returns false. I am not sure this parrallel can be safe in Go using function iterators. Mine basically just made it serial

@TheArtikae

Ай бұрын

@@daviddesmarais-michaud4345 Oh dear, I assumed that yield was safe to call from multiple threads, seeing as that was the entire point of the Parallel wrapper shown in the video. I was wrong. You're absolutely right about the issue, and the data race. The wrapper could never possibly be correct with function iterators.

Every time I'm surprised by the fact that go hasn't all the things I take for granted now in Rust and Kotlin. I need to make my own stuff too much, it feels like left-pad all over again.

yaaa.... it think go should bench this one. it just seems like the plot has been lost in this feature. i know go wants to be this gc lang that has first class support for concurrency to make up for that but it also promises to stay simple like c. It feels like too big of a sacrifice of the former in favor for the latter....

So it seems like function integrators are, generally, a good feature. The concern seems to come if an iterator implements concurrency because the concurrent logic is obfuscated behind the function. I’d love to see some high level iterators implemented by the std lib but default (or only) be synchronous. Maybe there could be a pattern to “enable” or “opt in” to concurrency on the new iterators. That way it would be obvious that they’re concurrent without having to read how exactly that iterator you’re using works

Maybe it's the specific example, but this implementation made the code far more complicated and less obvious than it was when using the async and go func directly. It can be bad when abused.

@BosonCollider

2 ай бұрын

I mean, the video specifically intended to show a footgun example. There's other examples that swing the other way like iterating over the results of postgres cursor without leaking it. This makes it possible to write code that will not leak resources needed by an iterator (unless you use iter.pull)

Please create a premium course on Go. I am all ready to subscribe to it. 🎉🎉🎉🎉🎉You are probably the best teacher who teaches golang perfectly.

So long as the standard library gets a good set of iterators covering most use cases this could be quite good without introducing too much magic you need to lookup to the language. My favorite thing about go is it’s dead simple and you can at a glance see exactly what the code is doing.

if go manages to polish typing system or move to traits (which is unlikely), it'll be hands down the best language

Yes, this video made me appreciate Rust even more. Thanks!

@MrlegendOr

Ай бұрын

Rust is trash

@nubunto

Ай бұрын

L

@heldim92

Ай бұрын

I know nothing about neither Rust nor Go, but this code seemed complex to me, maybe because I have zero contact with the language. My question is, when a person gets to the point where they need to use such functionalities as shown in the video, is Rust a more practical language when it comes to achieving the same results?

@nubunto

Ай бұрын

@@heldim92 I'm not saying it's _practical_, I believe that would just be untrue (depending on your definition of "practical"). But what I came to enjoy about Rust is that code is explicit: you know at compile time that it is race condition free (at least in your own usage, i.e. multithreaded code) , mutation-free, or explicitly signaled as a mutation. Is it more practical? Most of the time, no. But it's usually correct. And iterators are already baked into the language and a joy to use

This experimental feature challenges boilerplate oriented programming. Jokes aside. It's more convenient for sure. In my opinion it's bad for go's simplicity.

@ForeverZer0

2 ай бұрын

yeah, it does feel like a little out-of-place for go. This would perhaps be a good candidate for a dedicated keyword (i.e. "iter" instead of reusing "range") to make it abundantly clear what is going on, and/or some refinement on the syntax, which feels a bit clunky. While i understand it is not a requirement to write it as such, realistically the syntax encourages that this is most often going to be written as layers of anonymous nested closures, which feels "off" as a core language feature.

@IgoR.R.

2 ай бұрын

​@@ForeverZer0 Dedicated keyword could be too confusing. It's feels weird that range for an array or a slice yields index and value, but just value for functions. Take python, for example, it has "for x in iterable". It doesn't matter if iterable is a generator or an array. But definitely less confusing because of consistency.

MANN I WAS JUS ABOUT TO COMMENT IF U COULD MAKE MORE GO ADVACNED CONTENT SOO GOOOD

One of the ways I've been identifying goroutine usages obfuscated by a function call is to require a context passed in as the first parameter.

Can you make a neovim for elixir tutorial? I keep getting stuck on it myself.

I highly prefer the boiler plate "heavy" way of doing this. No reason to abstract away easy patterns everyone knows

This is great, gonna make some things way easier to do

func func func func func func

it seems rust is becoming easier and clear than go. The only good thing i like about go is compile time.

@omgnifty5957

2 ай бұрын

As someone who stills prefers rust... go has way, way, way easier concurrency. Doing async stuff in rust is a huge headache if you go beyond trivial examples.

@funky_hedgehog

2 ай бұрын

@@omgnifty5957 In Go you will get panic in production. In Rust compilation error.

@TheArtikae

2 ай бұрын

This is just try_for_each tho, isn’t it? What’s the big deal?

@sdramare864

Ай бұрын

@@omgnifty5957why can't you use tokio select and mpsc channels to achieve the same concurrency approach?

I don't know much about Go, so forgive this potentially stupid question: are these basically generators as defined in Python?

1. Goroutine G starts iteration, fall into default section in select and runs yield function 2. Goroutine H starts iteration and fall into default section in select 3. Goroutine G receives false from yield function and cancels context 4. Goroutine H call yield function and must panic, because it is yield after break Parallel does not work :( To fix that, we need blocking yield implementation, and await on yield channel in select

Finally Go has gotten a feature that existed in other languages like Python or C# 20 years ago.

this is huge

the less boilerplate the better IMO. makes it easier to understand and to learn

3:24 what happens if we just call wg.Done() after calling process? Ie without defer?

@IllllIIllllI

2 ай бұрын

The only difference is that defer will execute even if `process` panics. Without defer, it would possibly cause a deadlock

@dreamsofcode

2 ай бұрын

defer is a little safer, especially if the logic changes later on. However, in this example, it'll also work fine.

@vikingthedude

Ай бұрын

I love how youtube decides i should see these replies only after 6 days

How to configure vs code so it doesn't show syntax "error" when writing iterators for rangefuncs? For running I know it's just GOEXPERIMENT=rangefunc .

@dbalencar

18 күн бұрын

I'm getting: cannot range over backwards(nums) (value of type func(func(i int, x int) bool)) compiler (InvalidRangeExpr)

I think this could be really powerful but it's kind of complicated, I would use it sparingly and wouldn't introduce custom iterators to a project if I didn't have to. Old-school for loops and indexing can be cumbersome at times but even a monkey understands what it's doing.

I need a slice keyword because i’m slow and confuse a lot of things 😂. This feature is great but i need something more declarative where im just calling a keyword & it just works! Yes im a bad developer but swiftui’s declarative nature has ruined me and made me lazy 😂. No more imperative button code. You just call button and it’s there. I need this in a package or more preferably in the language. Ok it started off minimal but when i first used rust is was basic, now you barely see lifetimes because they have slowly abstracted them into keywords where you can literally build your own state manager with miner lifetime usage to get it done. Pack the language with declarative stuff! rant over. 😂

need this shit.

So it took them just 15 years to discover iterator infrastructure... Now we wait for local-scope level for defer (that is calling on end of the current scope, not the whole function), mutual module inclusion, configurable ciphersuites in TLS1.3, modern error handling and a few others.

@dreamsofcode

2 ай бұрын

Another 15 years for null safety!

@tomas120

2 ай бұрын

@@dreamsofcode is null safety really needed? I mean, c++ doesn't have it and also you have to deal with null pointers. I think Go was designed with that kept in mind

@cylian8422

2 ай бұрын

The thing I'm waiting for the most is sane enums. Just do what Rust is doing in this context

@PawelKraszewski

2 ай бұрын

@@dreamsofcode Null safety? I'm merely asking for const ptrs... func (hs *const HugeStruct) CantDamageHS() {}

@PawelKraszewski

2 ай бұрын

@@tomas120 C++ has references that guarantee null-safety (if not abusing some UBs), if you fear pointers.

Looks a lot like what is done in C# / Dotnet. I like the comment you mention, it's hard to tell how the underlying code is run. I would say good naming could help with this. But for love of god, please don't do like dotnet, by making everything a Task and have async / await all the way up the stack.

Still waiting for arena in go

Did they hire a JavaScript programmers in the core team?

Great but I'll stick to the first implementation.

I think Go is only fun when you learn about the syntax. When I use it, I don't get it, it's like a primitive thing. Most Go developers won't update Go to a new version because they don't need it.

that's a lot of steps to run a par...

While I've always been a critic of Go for being anemic in features, this one feels out of touch with the rest of the language. It shies away from Go's main idea of simplicity.

As it stands, I'm not sure I like it. Go's power comes from the verbosity, and obfuscating that behind a convenience interface seems more like other languages for little benefit

@dreamsofcode

2 ай бұрын

Yeah agreed. I'm in two minds about it. I think for the standard library to tidy up the many iterators patterns we currently have is a good thing, but for concurrency it might hide too much.

@AmirHosseinHonardust

Ай бұрын

Have you ever used them? There is not much hidden. All the logic behind these functions are obvious.

wow, Go has finally got yet another feature that exists in other languages for like forever

oh... they finally discovered operations on iterators...?

Personally, I prefer generators in the style of python for something like this (e.g. the nicety of just using "yield" instead of this strange function stuff), but I guess this'll do. I think the best part that will come out of this is watching all the gophers complaining for years about how this too complex... "What's next???? Usable Enums?!?!? Preposterous I say"

ah yes, features that exist in other languages for years

Nice video, but I'm too stupid to understand it.

I hope you start making Rust content

Go lang and programmers slowly realizing have some more features may actually be good 😆 Watch it basically doing what other languages do but worse because they didnt accept complexity from the start and think "simplicity" is cutting corners

@fsharplove

2 ай бұрын

Even "they didnt accept complexity from the start" is debatable. if err != nil all over the place.

@sevos

2 ай бұрын

Ah yes, another one screaming "quit having fun"

@jaymartinez311

2 ай бұрын

in the end it’ll have to add the other stuff anyway. So i like how it started but they need to rev-up some new declarative functionality. We are in the age of AI. This function should just work. I shouldn’t need to know the details of it unless i click through the different function signature docs in the ide.

This is not the go way. One of the key distinguishing features of go is its relentless simplicity. We don't need new syntactic sugar. What we need is more speed and less memory consumption.

WE DON'T WANT TO CHANGE THE WAY TO WRITE LOOPS! THAT'S THE ENTIRE POINT OF GO!

Govascript 😅

this seems to be the worst implementation of c# iterators (or js generators) I’ve seen so far. by a lot. This must be the most convoluted and arcane syntax to solve the problem in any language. I would have expected the func to return an interface that for .. range can iterate over, and you’d yield a value in your func to the consumer. the compiler would build a hidden struct that implements the interface. (haven’t written go in a while, last time, they didn’t even have a standard interface for something that you can loop over) that way you can process a stream in your func and consume each item in a for loop the moment they’re read without any expensive extra threads or synchronization (when using go routines and channels) go is such a bizarre language…

Rust will be easier to use than this.

the syntax is terrible

Congrats on making Go look just as shitty and unreadable as Scala. Generics was such a mistake, dear lord.

I dont like it

uu can this change it so that we don't write fucking loops no more? :D

They could have simply added generator rather than this ugly shit. A simple yield statement is simpler and far more readable

There is no point in using Golang, when there is Rust. Go is just worse at everything

@dreamsofcode

2 ай бұрын

I like Rust, but I think they both have their own strengths and weaknesses. There's no equivalent yet for Go+Templ+HTMX when it comes to Rust. Additionally, Concurrency in Go is better than async in Rust imho.

@iatheman

2 ай бұрын

@@dreamsofcode Agreed. Async and lifetimes are great ideas, but in practice they make everything exponentially more complex and boring to read and write as a dev. Rust did simplify lifetimes as much as they could, though... maybe it's the same with async, but the end result is still far from enjoyable or productively readable.

@baxiry.

2 ай бұрын

May the Almighty God bless your steadfast faith in Holy Rust ✝🛐

I hate golang, for very deep reasons. From the fact that it’s shitty by design, and the fact they cover it up with “simplicity” (see clojure for REAL simplicity, and not just easiness), all the way to the fact that this language forces programmers to be primitive code monkeys that cant do anything besides write lines of unexpressive gibberish. verbosity language has the repetitiveness and verboseness of assembly, but without the benefits that verbosity gives to other languages. It has the shortsighted sloppy design of C, but without the ability to do what C is capable of. I hope they keep piling features on top of Go so that all these new features will eventually reveal more and more of the fundamental problems with the language, just like once happened with C++. Trying to evolve C is a bad idea, it’s proven by all the garbage latguages we have today. The only ones that are designed relatively well are Erlang, Clojure and Common Lisp.

@neonraytracer8846

2 ай бұрын

Calm it down.. I think the languages you mentioned as having good design, has horrible syntax and is tough to read. We all have different opinions, but nobody agrees that C is sloppy or shortsighted. That's simply not correct especially given the history of things. Take a chill pill mate

@iatheman

2 ай бұрын

What a salty way to praise your personal taste.

@baxiry.

2 ай бұрын

The beauty of Haskell, Erlang, Clojure... comes from the fact that they tolerate the problem of copying data everywhere. And eat up as much memory as possible. If these languages try to solve this problem...then they will fall into what you call: bad design.

@apestogetherstrong341

2 ай бұрын

@@baxiry. Clojure solved this problem without falling into bad design. Please don't show your lack of knowledge. Clojure data structures don't copy data everywhere thanks to what they call „persistent data structures”. Erlang employs this technique too, where it's important. I never mentioned Haskell, you did. I don't like Haskell. Do your research before talking. Study efficient persistent data structures, hash-array mapped trie, rbb trees and others. There even are implementations in C++ (see „immer” library), where these exact data structures allow for massively efficient code (See the talk „Postmodern immutable data structures”). What you described is false and your opinion is based on misinformation. Have a good day.