Pamela McANulty - Things I Wish They Told Me About The Multiprocessing Module in Python 3

"Speaker: Pamela McANulty
If you haven't tried multiprocessing or you are trying to move beyond `multiprocessing.map()`, you will likely find that using Python's `multiprocessing` module can get quite intricate and convoluted. This talk focuses on a few techniques (starting, shutting down, data flow, blocking, etc) that will maximize `multiprocessing`’s efficiency, while also helping you through the complex issues related to coordinating startup and especially shutdown of your multiprocess app.
Slides can be found at: speakerdeck.com/pycon2019 and github.com/PyCon/2019-slides"

Пікірлер: 13

  • @TimRichardsonGooglePlus
    @TimRichardsonGooglePlus5 жыл бұрын

    Tips begin at 9:48

  • @oldmankatan7383
    @oldmankatan73834 жыл бұрын

    Thank you for including examples and patterns in your talk! So many of these talks discuss the theory, but don't offer a simple practical example to help relate the principle to the practice.

  • @equinox4467
    @equinox44675 жыл бұрын

    Thank you for this Pamela! This is a lot clearer than most examples and talks I've seen on the subject.

  • @pamelamcanulty5236

    @pamelamcanulty5236

    5 жыл бұрын

    You're welcome!

  • @michaelhackman3195
    @michaelhackman31953 жыл бұрын

    Very valuable knowledge! Thanks Pamela!

  • @nokkonokko
    @nokkonokko4 жыл бұрын

    Thanks, Pamela!

  • @MattLayman
    @MattLayman5 жыл бұрын

    There's a lot of great information in here. Nicely done! With some of the tips provided, it seemed to me like Pamela described the Actor model using multiprocessing. Wikipedia has a very dry presentation of that design style at en.wikipedia.org/wiki/Actor_model, but, the short version is that using messages and queues/mailboxes is a way to share state between processes in a way that will parallelize well.

  • @bennguyen1313
    @bennguyen13133 жыл бұрын

    Using threads vs multiprocessing... or queues vs pipes... or when to use .join have always been a source of confusion for me! I understand that multiprocessing's Queue will .get() one element at a time... But suppose you're writing several bytes worth of data in a queue (in the form of a tuple)... what happens if try to read from the queue, while the write hasn't finished the tuple? Does it block until the tuple is finished, does it return empty, or does it return the partial bytes in the tuple?

  • @SarfarazAhmad89
    @SarfarazAhmad894 жыл бұрын

    Conferences are a million times better than "nuggets". Instead of cut-down tailored unrelate-able tech notes you get to witness first hand the other person's personal experience.

  • @minyakonga8897
    @minyakonga88972 жыл бұрын

    Dont share, pass messages, Just like Golang

  • @fringefringe7282
    @fringefringe72822 жыл бұрын

    This Pamela had a voice mutation?

  • @doc0core

    @doc0core

    Жыл бұрын

    She was declared and malloc'ed as one type and type casted into another much much later.

  • @fringefringe7282

    @fringefringe7282

    Жыл бұрын

    @@doc0core 🤣

Келесі