This video is sponsored by Skillshare. The first 1,000 people to use this link will get a 1 month free trial of Skillshare: skl.sh/arjancodes09211

You can make it even more flexible by passing the factory to the 'initialize' method in the plugin from the loader. That way you don't have to import the factory. If the factory changes, you don't have to touch the plugin code at all.

@ArjanCodes

2 жыл бұрын

Great suggestion!

@Thristle

2 жыл бұрын

Exactly! plugins should just implement a required interface (preferably also announce what version of the main program they are expecting) but not import anything

@enriquerodriguez9723

2 жыл бұрын

@@ArjanCodes thanks! Perhaps this video can be a good segway into the topic of clean architecture, by Robert C. Martin, since it uses the idea of building software into pluggable components. You could show how one could implement this architecture in python. Great stuff, keep it up!

@pokemusicfan

2 жыл бұрын

I mean you always have to worry about the register function of the factory still being defined but good idea

@Zeedox

2 жыл бұрын

Wouldn’t you lose type checking in that case? The Factory function call is the place where the data class is checked for compatibility with the protocol.

This is an application of the Open/Closed Principle of SOLID. Quoting Uncle Bob: “Open for extension. : This means that the behavior of the module can be extended. Closed for modification.: Extending the behavior of a module does not result in changes to the source or binary code of the module.”

Many classes out there but very few of them touch the architecture, and very few of those explain it that great. Great job! Thanks for the euphoria per video!

For complex / long typing hints, you can make custom typing definitions and use them.

Awesome! So far you have been choosing examples that fit well to the object oriented programming paradigm, where you got clear objects like Characters or Customers or VideoExporters. Im wondering if you can discuss and come with a counter example where you would rather not use classes but just functions separated to different files / namespaces. Thanks for great work Arjan!

@ArjanCodes

2 жыл бұрын

Great suggestion, thanks!

@VivekYadav-ds8oz

2 жыл бұрын

This should still more-or-less work in languages with no-inheritance-OOP, like Go and Rust (all that I know).

Thanks for another great video. I liked the creative use of the PluginInterface class to provide useful static type info for a dynamically loaded module (plugin).

This is neat, indeed! Thank you for providing us with new tools and mental frameworks to expand our software development skills!

@ArjanCodes

2 жыл бұрын

Thanks Alessandro, glad you liked it!

I really love how you teach an architecture!

@ArjanCodes

2 жыл бұрын

Thank you, glad you like it!

thank you so much for your lessons on decoupling and composition, it helped me a lot in my projects and made refactoring/improving components of my application much easier !

@ArjanCodes

2 жыл бұрын

Great to hear! I'm happy the videos are helpful.

You are the man Arjan! As an intermediate hobbyist developer, your series are all super super helpful. This plugin architecture layout is beautiful.

@ArjanCodes

2 жыл бұрын

Thanks a ton! Glad you're enjoying the content!

Honestly, amazing videos! I've only seen 3 videos of yours so far and I was not expecting such great, methodical explanations followed by actual clean, useful code and advice. Moreover, spoken by a person with, what looks to be, an arsenal of experience and insight. In the KZread space, there is plenty of bad coding channels, which for me as a software developer is so frustrating. THIS channel is definitely one of the best I've stumbled upon. Keep up the interesting stuff!

Very cool. I'll have to experiment with this design pattern.

Seems interesting. In my code, I recently needed to do the same thing (adding variations without changing the existing code). I solved it using inheritance and reflection. I searched for all the subclasses of the base class and in every class I have the name as a static variable. Obviously the main issue is that it requires that all subclasses would be in the same module so they are loaded. I might try to use a factory like you did, which seems much cleaner because every subclasses would be in its own module with its own imports.

This was good on Plugin dynamic loading and creating it. You explain it in very simple language. I was wondering whether you have any similar video on building python framework. Particularly on builds and distribution.

Awesome content, Arjan! Keep them coming!

@ArjanCodes

2 жыл бұрын

Thank you Johan, glad you like the content!

This opened my mind. Thanks!

9:30 don't mix broad exception catching for "is my factory function name in the dict" with exception catching from running the factory function itself. If the factory throws a KeyError it will get misdiagnosed as a missing character type.

@aflous

Жыл бұрын

Exactly! easily fixed by moving the creation and return outside the try/except block

Cool! I think i've started implementing a project with an architecture that reminds this one ... but without the importlib mechanism. Thanks!

Awesome ! This was really helpful ! I was trying to do something similar for my project

@ArjanCodes

2 жыл бұрын

Thanks, glad you liked it!

Hi Arjan, really appreciate the videos. A suggestion for a video topic is the motivation behind choosing good architecture and design patterns. I've been reading the Pragmatic Programmer and they start with a section about how all of this is motivated by a desire to increase the "ease to change" code. I see that you understand this and as you often reach the point where you say, "ah now it's very easy to change x without changing y" but your viewers may not explicitly understand this. Thanks again!

@DonGioification

2 жыл бұрын

I think this is a very good point!

Great video as always! Some suggestions: 1. To avoid the type ignore (15:40) I would make the PluginInterface a Protocol decorated with runtime_checkable. 2. To avoid using global objects, instead of defining the initialize as taking no arguments, I would pass an "App" object exopsing a public API of what can be changed. This may have been too much for this example but it seems like running something without parameters encourages people to use a lot of global vars and polution of the namespace 3. A more strict and parametrizable way of defining callables (5:25) is using ParamSpec or Protocols with __call__. Maybe instead of using plain dictionaries which are hard to type strictly we could have defined another Protocol called GameCharacterConfig and make GameCharacter have that as an example. That would allow dependency injection of configuration while still keeping everything type safe. When adding a new character, one should not only define a CustomCharacter class but also a CustomCharacterConfig that implements the GameConfig protocol. That way the type of that callable would be Callable[[GameCharacterConfig], GameCharacter]. If more strict typing is needed we could use a TypeVar but I believe it is not needed here.

That's very neat content. Thank you Arjan.

@ArjanCodes

2 жыл бұрын

Thanks so much! 😊

This is gold, thanks

Very nice video Arjan! Also just looking for a more pythonic approach, example.py is doing a lot of things (declaring and registering classes) and since this is a plugin approach a "plugin.base" file with all those initial classes (Wizard,Witcher and Sorcerer) from line 12 to 36 could be added and then add it into the json file plugin list as you did with the "plugin.bard". All the initial registration from lines 43-45 are no longer needed as well as the lines from 12-36 as they're going to be defined in that "new plugin.base" file, letting the loader (line 52) to handle it Really good job, thanks a lot for your work

Hey! I've got another suggestion: Have you covered the singledispatch function from the functools package in one of your videos before? If not, I'd really recommend it! It really goes hand in hand with the plugin architecture too, since it lets you define generic functions and then the plugins can register their own versions of that function to handle the new classes they create (eg new characters in your example). I personally think it could potentially be a great topic for a future video, in case you're interested!

@ArjanCodes

2 жыл бұрын

Hi Fabrice, thanks for the suggestion, I'll take a look!

Awesome videos Arjan ! Nice channel !

@ArjanCodes

Жыл бұрын

Thanks so much, glad the content is helpful!

plugin architecture is very useful to develop modular and adaptative systems. your overview is clarifying and helped to recap it. thanks for the video!

@ArjanCodes

4 ай бұрын

Glad you found my video to be insightful! Thank you for the comment, Lucas :)

I'm implementing something currently that uses a modified version of this to load a plugin from a file path after checking its content hash to make sure it has been validated/authorized. The idea is to make sure that only signed plugin code is run to avoid nefarious code from hijacking the application. (The plugins are actually contracts that specify correspondent time banking relationships, so there is an additional incentive for the system to be hijacked.)

Awesome video arjan!

@ArjanCodes

2 жыл бұрын

Thank you, Scott, glad you enjoyed it!

Arjan, do you have any intention of developing some videos on TypeScript (or other languages)? It would be nice to see your approach on those. (The channel is great anyway, congrats and thanks :) )

This was very helpful, thank you

@ArjanCodes

2 жыл бұрын

You’re most welcome!

"Toss a plugin to your code" 👍🏼👍🏼👍🏼

Arjan! YOU THE MAN!

@ArjanCodes

Жыл бұрын

Thank you Jake, glad you liked the video!

the tutorial was awesome

Nice work thanks!

@ArjanCodes

Жыл бұрын

Thanks happy you’re enjoying the content!

I like the background Picture that says Toss a coin to your code!

Thank you very much for teaching design patterns and architecture, I think these are critical components to be a good software engineer..

@ArjanCodes

Жыл бұрын

You're very welcome!

I wrote a mock for an embedded application using python. Since I wanted to implement a way to change the behavior like you showed in this video I also used the importlib module. However, I required a specific name for the files and functions. It is horrible! I will try the plugin architecture. Thanks for the great content!

@ArjanCodes

2 жыл бұрын

You're welcome, glad you liked it!

Great video! I'd suggest adding those mini code project to a github account for everyone to enjoy.

@ArjanCodes

2 жыл бұрын

Thanks, and good point! I normally put the link to the repository in the video description, but forgot to do it here. It's been added now.

The way you organize code is magical.

@ArjanCodes

2 жыл бұрын

Thanks, glad you like the video.

@cryp0g00n4

2 жыл бұрын

@@ArjanCodes yea itz solid

definitely had to review this one a bit, but really 😎

Love it amazing!

@ArjanCodes

2 жыл бұрын

Glad you like it!

Hey Arjan, than you for the Content. I'm a big fan and I'm really learning new stuff. Can you make a Video about Project Structure? Like how to name things, what files belong in what folder? Where to put interfaces, dataclasses etc.? I have troubles structuring bigger projects. They start small but then once they are too big they get really messy.

@ArjanCodes

2 жыл бұрын

Glad you like the content and thanks for the suggestion, Antonio - noted!

this channel is a real treasure

@ArjanCodes

2 жыл бұрын

Thank you so much!

Toss a plugin to your code! Nice one!

Waw! Nice one :)

Zelfs als ervaren dev heel nuttig Obligatory: Wanneer komt de TickTick 2015 XNA naar Python editie? :)

I’ve been waiting for this one! I assume that there should also be some test code that generates the strangest plugin that will compile to pyc and sees your code-base fail gracefully, correct?

@ArjanCodes

2 жыл бұрын

Yes, that's a good way to test various edge cases of what a plugin should or shouldn't do!

If I could I'll give it ten likes. Mindblowing. Excellent explanation. Thanks.

@ArjanCodes

Жыл бұрын

Thanks Jose, happy you’re enjoying the content!

very interesting vid, as always

@ArjanCodes

2 жыл бұрын

Thanks, happy you enjoyed it!

Can we use namespace packaging to create pluggable architecture?

About 15:40 - To force the IDE Type Checker to accept your typing, you can use `from typing import cast`.

it’s good to listen to you

@ArjanCodes

2 жыл бұрын

Thank you, glad you like the videos!

I learned that try block should be as sort a possible. So it would be good an idea, to move the return out of it, after the except in def create(...). Whats your opinion?

This seems like it would cause conflicts if you have multiple modders wanting to edit the same json file. Avoiding such conflicts would probably need a dedicated directory from which you load every json file rather than just a specific one.

@k98killer

2 жыл бұрын

Maybe. For Skyrim, there are tools that each compiles a master list of plugins to avoid conflicts, and these use metadata to figure out the load order; if one plugin depends upon another, it cannot be loaded first. This potentially violates the SOLID principle, but at the same time being unable to reuse plugin code violates DRY. Every architecture will have different trade-offs.

Great video

Do you have any information anywhere about how you set up your IDE? It looks like you're using Visual Studio Code, but I'd love to know what, if any, plugins you have for it in these videos

In english translation of Witcher series Jaskier's name has been translated to Dandelion for easier pronunciation and easier reference. Anyway beside thanks for sharing this stuff, will certainly have that in mind when I add new projects in the future.

@ArjanCodes

2 жыл бұрын

I probably still won’t be able to pronounce that correctly, haha.

@Alche_mist

2 жыл бұрын

In Czech, the bard is Marigold. Which obviously clashes with Triss's second name when comparing to English - it's Triss Ranuncul in Czech. Note: Czech is very close to Polish and our translator was amazing - Sapkowski himself once (in the far history before the Witcher games and series) stated that only Czech and Polish readers read what he wrote.

really nice! However, I'd register the dataclass using a class decorator which takes character_type as argument.

Could you please point me out to an explanation of what 'Protocol' is and where it's defined? Is it just what interface means in C++ or Java?

Thanks - great video! One qustion: I've incorporated this into my own project which I'm trying to package with pyinstaller. However the packaged program can't find the plugins folder when import_module() is called, even though I've added it through --hidden-import. Any idea?

Hi Arjan, Thank you so much for this. It is exciting. I am hoping to use it in a project very soon. While looking at your example code, I ran into an error in the loader.py: " def load_plugins(plugins: list[str]) -> None: TypeError: 'type' object is not subscriptable " It was not happy with the function parameter hint for plugins: "list[str]" I was running version 3.8.6 and noticed you were running version 3.9.6. When I upgraded to your version, the code worked perfectly. I was wondering if you could help me on how to find the lowest version of python this functionality would work on is. I am more asking for a method or approach on how to find this out than the specific codebase so that I can empower myself for next time :) Thank you for your amazing content and hard work putting these out. Kind Regards Ryan Julyan

Nice witcher references :)

@ArjanCodes

2 жыл бұрын

Thank you :).

I love your tutorials, you're great at explaining! Just the jump between showing the code and showing you is quite jarring and jumpy sometimes. For a tutorial you're almost doing too much camera work, especially as you're typing really fast (wow!). I'm about half a quick and need to swap between the browser window and my text editor very often to pause the video.

Awesome, I've seen you from the Write better Python series, keep it up.

@ArjanCodes

2 жыл бұрын

Thanks, will do!

Thanks for this (again) amazing video! I was wondering, is there a reason why the new Character do not inherit from the GameCharacter class? Thanks again!

@djl3009

2 жыл бұрын

I think it is because he chose to use the Protocol class to define the GameCharacter interface instead of ABC inheritance. He provides some guidance on when to use the Protocol class vs (ABC) inheritance in another video (kzread.info/dash/bejne/rHuWssZtirOZabw.html). I also found the official Python docs explanation useful regarding the rationale behind the Protocol class: www.python.org/dev/peps/pep-0544/#rationale-and-goals

@nandoflorestan

11 ай бұрын

The reason is, GameCharacter is a Protocol. And now you have to go read about protocols in Python.

Instead of type: ignore, you can use the cast function from the typing module

@ArjanCodes

2 жыл бұрын

Good point!

Ok this video project looks like labor of love :D

@ArjanCodes

2 жыл бұрын

It certainly is :)

At 9:17, I see an opportunity to improve the code. The return statement (and calling the character constructor) should definitely be outside of the try block. Always remember to check for this mistake: do not over-catch exceptions.

very nice

@ArjanCodes

2 жыл бұрын

Thank you!

The whole creation of the characters it's pure magic. Congrats! The way the creator_func creates the objects its beautiful!

Isn't there any dedicated library for this?

Thanks for the content. As a developer coming from a Java background, why exactly do we need copy here? Is it just because we want to pop the type and for some reason keep the original dict?

@nandoflorestan

11 ай бұрын

Yes

Really cool demonstration of how to build plugins in python. I am, however, not a fan of the use of Protocols here, since it is easy to miss adding the initialize method. It is also not explicit that the registration in the factory should occur in there. I prefer the more explicit solution of having a base class (say RegisteredCharacter) that implements __init_subclass__ to automatically register its derived types in the factory ;)

@ArjanCodes

2 жыл бұрын

Great solution! I agree that’s a really nice way to implement autoregistration.

@virtualraider

2 жыл бұрын

I also have to agree that protocol use is actually making the code more esoteric. I really enjoy your earlier videos when you can see everything tied in nicely in the IDE. But here it doesn't help you auto discover these relationships and requirements. Protocols seen like a lightweight solution that might help when execution speed is of the essence, but it seems like too much of a tradeoff against ease of development. Am I overreacting?

This looks great and all but, is it just me or these patterns combined make it very hard to create comprehensive unit tests? How would you accunt for your builder behaving as expected with such flexibility while importing modules and classes?

How does that "raises ValueError(... " Works? From None? That's not usual for me. Thanks

Here the PluginInterface class is an Interface working exactly like a protocol ? I find it strange that « class PluginInterface: » and « class PluginInterface(Protocol): » seems to do the same So what the difference ?

I have not watched the video yet but I am sure it will be exciting. Thank you Arjan and keep on building :)

woo, functional programming!

@ArjanCodes

2 жыл бұрын

you bet ;)

Hi Arjan, I subscribed to your newsletter but did not receive the 7step pdf yet

@ArjanCodes

Жыл бұрын

Did it by any chance end up in your spam? If you haven't received anything, send me an email (support@arjancodes.com) and I'll make sure you receive the guide.

In fact this is a cross cutting concern thing. Because you can make plugins for any layer not only for domain as in this video.

a better version of importlib.import_module would support generics, to be able to specify the return type explicitly

Great

@ArjanCodes

2 жыл бұрын

Thank you very much!

This was a really cool video! But doesn't this open up our code to malicious code? From my PHP times everything inside me starts screaming when we take a string and pretty much execute it :D. I know this scenario is different because the end user doesn't really see the config file and plugin directory. Nevertheless, we allow arbitrary code to be executed purely by adding a line of code to our config. What are your thoughts on that?

@ArjanCodes

2 жыл бұрын

This is certainly possible. It’s therefore also important to have processes in place to protect yourself against that. For example, if a plug-in is to be distributed, you can require to check the code beforehand (kind of what is happening in app stores). You can also limit the things that a plug-in can access via the interface that you provide, but since Python has little protection in terms of access restrictions, this is dangerous.

@malteplath

2 жыл бұрын

@@ArjanCodes Python's introspection features will get you anywhere once the main program calls your code. That does not make the architecture invalid, it only means that you will have to have some process to vet the plugins. Like you suggested. There is no way Python and Python code can ensure 100% security against malicious plugins. (I guess this holds for any interpreted language.)

@Ryndae-l

2 жыл бұрын

Well, if someone can drop code in your plugin folder, what is stopping them from modifying your base code ?

@maximilian1344

2 жыл бұрын

@@Ryndae-l simply shipping a malicious plugin from some website and letting the user "install" it, is a lot easier than modifying the base code somehow. But thanks for the replys, everyone :)

My python code is the neanderthal equivalent of your modern day humans python code... But I'm learning 🙂... love the channel.

@ArjanCodes

2 жыл бұрын

Glad you enjoy it!

Toss A Plugin To Your Code. Noice!

Hi @ArjanCodes and thanks for sharing your knowledge. I download the project, but when I run it with python3 I got the following error: TypeError: 'type' object is not subscriptable in the factory module: in this line : character_creation_funcs: dict[str, Callable[..., GameCharacter]] = {} I wonder what I'm doing wrong.

@josetapias2339

Жыл бұрын

I found the error: The error occurs because the code uses "dict" instead of "Dict"; also uses "list" instead of "List". the initial letters in lowercase are the built-in datatype but the initial letters in uppercase are the corresponding type. I made this little change and it works.

@ArjanCodes

Жыл бұрын

Glad that you made it work!

most of the peeps here are suggesting, adding, modifying code, FCS, he KNOWS all of it, but he needs to have a base code simple for explanation, just enjoy the masterclass!

While this architecture allows a truly extensible framework, how would you defend the host computer against plugins that contain malicious code?

@fat_pigeon

2 жыл бұрын

Sandboxing. Use a declarative plugin interface, and also execute all of the plugins in a separate, sandboxed process. Essentially, the subprocess only has permissions to communicate with the host process. This requires explicitly defining an API for the plugin to call into, which is a good idea anyway for compatibility (plugins will be coded to an interface that you promise to support across versions, even if you refactor the host app), but reduces flexibility (plugins can only modify your app's behaviors in ways you've explicitly planned for) and takes work.

hello, your tutorials sound very interesting to me but i'm having a hard time understanding. It’s interesting sometimes to start from scratch.

Why you didn't do the GameCharcter class an ABC class?

Great video! (This is the third time I try to leave a comment, they seem to disappear as soon as I write them! Or are they getting removed? Am I going crazy?)

@FabriceNormandin95

2 жыл бұрын

I was suggesting the "entry_point" feature of setuptools, which is a great way to add plugins from outside your package!

@ArjanCodes

2 жыл бұрын

Sometimes KZread filters out comments, but I’m not sure how it works exactly. Entry_point is interesting, I’ll take a look, thanks for the suggestion!

Great game 😁😁😄

what exactly does "{!r}" do in 09:51?

@josetapias2339

Жыл бұрын

It just calls the repr of the value supplied.

My favorite part. "Toss a plugin to your code."

nice

@ArjanCodes

2 жыл бұрын

Thanks!

wouldn't a better decoupled plugin design not force the new Character classes to know anything about the mechanism of the plugin factory, but rather just have a factory that searches through a designated directory, instantiating any classes it finds (with certain limitations for safety, perhaps), and registering the classes it finds that implement the given Protocol or ABC, and ignoring/noting any others it finds that do not comply?

seems like an observer pattern with dinamic load to me

Interesting. Everything else I've read tries to make their loader search the file structure for plugins, but yours simplifies the loader code a ton by having the plugin be registered in the config file...

@ArjanCodes

2 жыл бұрын

I often like introducing some kind of intermediate data representation to decouple things, in this case a configuration file. Later on, you can write code that automatically generates the config file contents, but the rest of the code doesn’t need to know anything about that.