THANK YOU for this video! I've been saying for a while in my company how the repository's purpose is to serve the Write (aka. consistency) side of the application and how it should NOT be a container for dozens of query methods which most of the time shouldn't even be using EF. It's never been about replacing the database; it's about helping enforce that consistency boundary for commands. I'll pass this along :)

We have a couple rules around data access: 1. Don't retrieve more than you need to, don't write more than you need to, and minimize database hits. 2. Any dependencies (transactions!!!) must be explicit. 3. Do not write your program like a bad version of dynamic typing. #1 is basic performance rules. You're allowed to break them in certain scenarios, especially for data and operations only admins and CSR can access. #2 should just be obvious. Implicit coupling is the worst type. It makes codebases scary to change. Some people in Java like to use @Transactional annotations, but that implicitly couples the databases your repositories use: if you switch one to redis then you lost your consistency guarantees and you probably didn't even know it. This Is Bad. And damn near impossible to write automated tests for. I've seen a lot of code violate #3: "Here's my Basket, but sometimes it has items and sometimes it doesn't". Don't. You're violating static typing. Your types are lying to you. If you feel the need to do this, you should probably have used a programming language with dynamic typing. When using static typing explicitly name things. But it isn't as clean. We might have Basket. But we might also have BasketWithItems. Depending upon the domain, this can get ugly. I've worked on some projects where we have something like "BasketWithEverything" that just has everything. It's ugly, but the code isn't lying to you, and it doesn't perform like garbage. You can probably tell from all this... no, it's not easy to just swap a database in our projects. Every decision has tradeoffs. Including "standard practices".

@GerardJaryczewski

Жыл бұрын

Awesome summary for my private experience and reflections - thank you! I also have a feeling, that many programmers feel bad if they make repetitions or if they write a stupid code. They choose "deduplication" over readability. In large codebases it is justified, but in others?...

@kabal911

Жыл бұрын

I love this thinking!!! I say “rules” are meant to be broken. The only rules that exist on projects are those that the project teams/organization define. It’s not haphazard though. When you think like this you can build simple codebases that are easy and safe to change I think the “BasketWithEverything” is actually a great example. I like to say that just because 2 queries look the same, it doesn’t mean they are the same, and why then making this a shared query can cause problems.

@brandonpearman9218

Жыл бұрын

I like the way you have phrased this. Im going to steal it :) A rule that is big for me, is Do not blindly trust the data. Often I see applications that assume that because they wrote the data into the db that they can cleanly read it out, and this is true, until it is not. There are a number of things that can cause your data to go crazy like bugs, migrations, manual db changes, model changes, etc. So I start with distrust of data and program defensively here, the same way I would as input into the application.

This is a really great video. “Potentially swopping the database” (which has never happened to me in 20+ years 😉)has never been a consideration of mine when choosing to use a repository or not, because I am already using EF Core that does that exact thing. The bonus is that it also happens to behave like an incredibly powerful query repository. But this video gave me some perspective on the potential value of aggregate root for commands.

Yes. I replaced it during unit and integration tests. Devs are still yet to understand this simple concept that the database is being replaced during tests as well. SMH.

I have worked in a project where we needed to change from SQL Server to Oracle and using the repository pattern wasn't as helpful as people thought it would be. I can say that the effort of not using it would have been the same.. However for unit testing having this abstraction layer was very helpful!

@garylee867

Жыл бұрын

For DB related unit testing, I prefer using test containers. It is much simpler than having an extra abstraction and mocking

@JohnZombek

Жыл бұрын

@@garylee867 Can you explain it a bit more?

@barrymanilow3893

Жыл бұрын

Unit testing an abstraction is nonsense.

@garylee867

Жыл бұрын

@@JohnZombek It means using docker to create a temp DB container for unit testing. There are libraries called "TestContainers" to do this very convinently. You can google it.

@tomaszenko13

Жыл бұрын

​@@garylee867 It's no longer a unit test. But yeah, it is good for integration testing

I have liked this video before I've been able to see it. For testing purposes, the repository pattern works: mock it up or set up fake data collections during design, then use that interface when building the real DB connections. But if you are changing prod-DB, you want to use the perks of the new DB and then the repository interface could have to change. Also, if you don't get hung up on the particular interface when changing DB, you can use the opportunity to improve performance or iron out some eventual inconveniences from the first design.

I have mostly given up on a repository layer for attempting to abstract away the persistence technology. If I were to change from sql to mongodb, it would be more than exchanging a repository implementation. I'm still using a repo, but more for organizational purposes, just like some people like to use extension methods as an alternative. I'm embracing the persistence technology in the interface as well. So for me, a "repository" is basically a simple, mockable piece of code which does not even attempt to be neutral.

@GerardJaryczewski

Жыл бұрын

"which does not even attempt to be neutral" - excellent.

@kabal911

Жыл бұрын

I like this take.

Generic repositories do not accomplish all the goals of using a repository. A generic repository does not create an abstraction of your persistence. It is only a thin wrapper around your ORM. Use of your generic repository still relies on ORM implementation details like LINQ query translation. A repository should not be an ORM, wrapped or not. The ORM is an implementation detail of your repository that the repository needs to hide.

@kabal911

Жыл бұрын

I 100% agree here. That is why I think generic repository is pointless. You are basically implementing a forward-proxy decorator.

I've successfully done this a couple of times on fairly large systems and the impacts were incredibly minimal. I generally keep my domain objects dumb and instead favor having a service/business layer that handles all of that. The service only ever interacts with repository interfaces that all accept/return the shared domain objects. This keeps all of the EF and DB stuff isolated to just the repository, so it can be replaced seamlessly. In one instance, I replaced most of the repository's EF queries with high performance stored procedures (our object model was quite complex with over 40 tables in some cases and EF just isn't great for this). In another case, I swapped out a backend DB for a set of external services that returned the same data. Had we exposed the repository implementation upstream, it would have required changes all over the place to do this replacement, but since the only link was an interface and our domain objects, it was simple and low risk.

@StrandedKnight84

Жыл бұрын

This is the way. I realize that DDD die-hards will say that anemic domain models is an anti-pattern, but in my experience having dumb domain objects and dumb repositories makes it a lot easier to replace the DB implementation. It also makes it easy to set up mocks and write powerful unit tests. As always, it depends on context and personal preference of course.

@Fafix666

Жыл бұрын

@@StrandedKnight84 What do dumb domain models help you with? Their logic should remain unchanged. What the data store and its models do is not something domain models should care about.

Interesting video thank you. As you alluded too, "Repository" for fetching or saving domain objects is not the same as "Repository" for data and it's not helpful to conflate the two concepts under the same term "Repository.". I prefer to reserve "Repository" for the concept of working with data in the data store. I prefer to use different terms for the concepts of fetching or saving domain objects / aggregate roots. When thinking about fetching a domain object it would be more appropriate in my view to use a `Factory`. e.g `CustomerFactory` or `OrderFactory`. Those factories could inject repositories because they need to solve the problem of loading and returning a consistent aggregate root, given the "data" in the data store. The "Repository" is the abstraction for the data store. One problem here is the object-relational impedance mismatch problem - the data store is concerned with modelling data and that usually means normalisation etc. The domain object is concerned with modelling behaviours, and data normalisation is not it's primary concern. This results in two different structures and mapping has to occur. The idea behind the IRepository is that it is supposed to deal with the data side of the equation - like in this case supplying the data based on a query. The idea behind the Factory is it is meent to return an instance of the domain object, and that means it has to be able to take the data from the data store (IRepository) and create / map it to a domain object. I say `Factory` here, but the factory "logic" could be implemented in a variety ways such as a factory static method e.g `Order.Get(1)` or a factory class `new OrderFactory.Get(1)` etc and even there, the factory could end up delegating the hydration of the aggregate to the aggregate itself, for example `order.Load(data); // order aggregate knows how to poplate its inner workings from its state`. When talking about saving domain objects back to the database, again in the domain layer, that is not a repository concern. It's not a `Factory` either because `Factory` is typically associated with object creation. I tend to model saving changes by having a `SaveChanges` on the aggregate root in the domain layer. The aggregate root could for example inject the required repository and when save changes is called, persist it's state, to the data store via the IRepository. In my view the aggregate itself is the best object to encapsulate the logic in terms of hydrating itself from a given data state (supplied via an IRepository), as well as persisting itself by saving it's data state back through the IRespository.

Another amazing video. It was this channel that made me look at my system with new eyes. And how different parts communicate across boundaries.

Nice to see the quality increasing (visually)

Repository is thrown around a lot and mean different things to people - still they talk about it like they mean the same. And what you have previously said about having a separate model for reading data has really stuck with me. There is no shame in having to different interfaces when it is solving a problem that has been imposed by the first interface.

@CodeOpinion

Жыл бұрын

Yup. It's a different concern. As always, depending on your context, it maybe feasible to use the same abstraction for simplicity... or maybe it's not.

@marna_li

Жыл бұрын

​@@CodeOpinion Yes. Data Access vs mainly retrieving Aggregates in DDD.

@kabal911

Жыл бұрын

Well said

In my case I work in a project that is installed as a nuget package. Beyond the "Core" packages we provide 2 nuget packages for storage, one for EF6 and one for EF Core. But since those only implement our Repository interfaces from the core package, a developer can ignore our prebuilt ones and implement a Dapper version instead if they need.

Answer to the first question is YES. I have. Multiple times (same project, full cloud migration). The repo's interface only uses application/domain objects, that's why it worked. What I don't subscribe to however, is focusing on Entities. That would have made it a nightmare. Don't implement repos as crud interfaces, implement them as data services and enforce those transactional boundaries. TL;DR - combine the ideas.

In my case I have a porject wich uses diferent DBs depending on the environment. In the cloud uses cosmos db an ind development it uses LiteDb. If needed we could add a Relational Db in the mix ;) Its more a "unit of work" and "query" than a plain repository

I recently implemented a helper library which enables me to transparently store full structs into a kvs DB in one call, the coolest thing about that is i can take advantage of extremely fast key based iteration, or if I want i can directly access individual field values with basically 0 overhead, unlike running a SQL select for just a single column on a table. I have an example "repository" which i had to make generic to demonstrate how it could be used. Are we saying that unless your repo wraps up some useful internal logic as part of it's operations it's not really a repository?

I use them only when the repositories are specific, used only for the write side and in combination with domain models. Another nice feature is to hide cross cutting concerns with the decorator pattern.

@kabal911

Жыл бұрын

Decorator is indeed a great pattern to with with repository. One of my favorite design patterns.

The value added by repository pattern sometimes (in many cases) is not justifiable by the the complexity it introduces, I think, and am eager to find out any unique use cases that would have value over cost.

So for commands we should use a repository to fetch the entire aggregate and make changes, and for queries we could expose an interface of the dbcontext using efcore for example? my doubt doing this is if we expose the dbset we can access to the write methods of it (add, update, delete) of any entity

@CodeOpinion

Жыл бұрын

Use an aggregate if you need to form a consistency boundary around state changes. Using a repository is a way to fetch and persist aggregates. For queries, use what makes sense the most. That could be exposing the dbcontext (overriding savechanges to throw). Beware of how your coupling to it and the degree of it. You may choose to use a repository to return a data model instead.

hey, could you tell me how to achieve ACID transactions in a repository interface with only Get and Save methods? because you usually would want to select for update first and then make the changes.

A few questions... 1. Why are you equaling domain objects to database objects? This creates coupling, but also forces your domain objects to contain audit fields like creation dates or creators. 2. Why does your application layer know about the presentation layer's models?

@CodeOpinion

Жыл бұрын

1 - I'm not. 2 - View model generated in the handler as apposed to mapping to yet a other object?

@Fafix666

Жыл бұрын

@@CodeOpinion1. Ah, had to misunderstand, cheers. 2. Yep. It feels a bit stinky tbh. Is performance your only consideration, or is there something else?

@kabal911

Жыл бұрын

For 2, you could think of it rather like an api contract, instead of saying the application knows about the presentation models.

@Fafix666

Жыл бұрын

@@kabal911 so now your contracts are coupled with your logic layer.

On reading you need flexibility. On writing you need control. Good video.

Might be interested in some analysis of MS's "reliable web app" pattern. Perhaps a good topic for a video?

I have a question, how to abstract ORM specific features? For example, in mongodb you have AsyncCursor, even if you expose that in a wrapper object, it won't solve the problem, because other pairs of ORM-DBs like EF-SQL doesn't have the concept of cursor like mongodb. I said abstraction, not a repository because it may belongs to Query side.

@kabal911

Жыл бұрын

I’m of the opinion of why even try. The hard fact is that you most likely WONT want/need to ever switch. Otherwise you build technical debt for “just in case”

In the end, it's not mandatory to write a repository for your aggregate. But it might help you organise along with some DTOs, for example. You might be better off using rich domain models and skip the repository entirely. Everything will depend on what you're doing. I also don't think that you'll ever have 100% decoupled software. You need to have it decoupled enough so you don't have a hard time with dependencies and unit tests. That's it.

@g.v.m7935

Жыл бұрын

You could do the dtos through services.

@EldenFiend

Жыл бұрын

@g.v.m7935 Yes, of course.

Repositories is clearly defined in the DDD book and yet .NET developers read repositories and nothing else and have spent 20 years implementing it wrong.

@kabal911

Жыл бұрын

I just don’t care about DDD 😅

@allyourfuturebelongstochina

Жыл бұрын

@@kabal911 Then you don't need repositories.

"Okay, so let's pretend we're in an episode of 'The Office' here for a second. You're Dwight and I'm Jim. I've just heard you say 'hi-archy' instead of 'hi-ur-archy' and now I'm looking directly into the camera with that classic Halpert smirk on my face. Now, I would launch one of my infamous pranks. I'd probably create an entirely new office policy, stating that all words should be pronounced incorrectly, using 'hi-archy' as the leading example. I'd even make a fake memo from Corporate, just to sell it.

Thanks,do you have this code on the github?

Repository pattern is for hiding and abstracting away persistence from the application and for providing the methods with actual meanings (like GetOrder, FindOrdersByFilter, etc...) and not exposing underlying logic (like First, FirstOrDefault) to the consumer of the repository. Also keep in mind that you might have MULTIPLE data sources including API calls, for which your app should NOT care about. Overfetching is caused by exposing the underlying mechanism (a violation of repo pattern) of getting data and is solved by providing actual methods which an application uses (instead of list orders which return everything, create GetOrders, GetUserOrders, ... etc, methods which explain the use!) Database "switching" was always an example and not a real use case, this is the real use case.

@sorvex9

Жыл бұрын

I hate programmer terminology so fuckin much "hide away implentation from the app, the app doesn't care about how the data is fetched" Of course the app cares, it is the app that executes the code that fetches the data. Stop using dumb as FK terminology, please.

@allyourfuturebelongstochina

Жыл бұрын

No.

@sorvex9

Жыл бұрын

@@allyourfuturebelongstochina Don't even get me started on severless.

@genechristiansomoza4931

Жыл бұрын

Yes.

@allyourfuturebelongstochina

Жыл бұрын

@@genechristiansomoza4931 no. He’s misunderstood the DDD book.

But what if I want to use a in memory database during tests?

A few points worth noting here. First of all, the problem isn't making your database swappable for your tests. That is all well and good, because you know what the options are that you are working with and can take their differences into account when designing your abstraction layer. The problem is trying to make your database swappable for unknown mystery alternatives. Usually when people do this, one of two things happens. Either they lock things down so far that they lose the ability to fine tune things for performance, and end up with a system that grinds to a halt as a result, or else they allow so many different details of the underlying implementation leak through that the abstraction layer becomes completely useless. And in both cases, it adds so much extra complexity to the codebase that it slows down development and makes simple, everyday routine changes horribly painful. Secondly, people sometimes try to defend all this chicanery by saying "But I have worked on a project where we've had to replace the database." This misses the point completely. The point is that even if you have to do this kind of thing, there is no evidence whatsoever that the approach you're adopting to try and facilitate this actually makes any difference when you have to do it. In any case, even if you do have to do it, it's something you end up doing once or twice in the lifetime of a project at most. On the other hand, the routine changes that it makes difficult and frustrating are changes that you have to work with every single day. Thirdly, making your database swappable for unknown mystery alternatives is not a technical decision; it is a business decision. There are valid reasons why your stakeholders may be asking for such a requirement, but they usually only apply to shrinkwrap software that gets deployed to multiple sites where different customers have different database technologies that they need to support. They rarely if ever apply to software as a service or web based software in general. If your stakeholders are asking for this, they need to be made aware that it will significantly increase development time and this needs to be factored into their budget. Most people who advocate this approach miss this step, and this being the case they are not delivering the best value for money to their clients. Finally, this approach is frequently justified by crying "separation of concerns." But it is important to remember that separation of concerns is a means to an end and not an end in itself, and the most important thing that you need to do with separation of concerns is to understand what you are separating and why you are separating it. Just making things swappable for the sake of it is not "separation of concerns" at all -- it is speculative generality: separation of concerns's evil twin.

@ruyvieira104

Жыл бұрын

I hand-rolled a "orm" which allowed me to move from sql to firestore, now it has some function which are so specific to firestore that I cannot move to any other database type without some hardcore changes about my "data modeling" (e.g: no "structured" objects)

For my understanding the repository is not concerned with the data abstraction layer, the ORM etc is for that. People seem to use repositories to test their business logic. So the initial question is irrelevant in practice.

Ditched both UoW and repository pattern years ago and haven't looked back ever since.

Basically, never. I use the repo pattern to make it easier to mock the database, thats it. I know one company who changed databases after I left, but they were using stored procedures in the same way one might use the repository pattern - and they were nutters

I've never seen anyone use the repo pattern and use it to replace a db. I've always thought the repo pattern was a giant waste of time and effort.

yes

The title is biased. The reason for not mixing business logic with ORMs is the rule of not mixing levels of abstractions. What if a file is our db? Do you open and save and configure buffers in the business logic? ORM is just same level of abstraction as File class in C#. Not once I've seen changing ORM on read side to raw sql. None of business classes that used the method abstracted behind the interface were affected, nor any unit or integration tests.

I replace DB with in-memory implementation every day in my unit tests

Repository pattern is to separate persistence from business logic right?

@CodeOpinion

Жыл бұрын

Check out this video where I talk about where I find the value in a Repository. kzread.info/dash/bejne/YmWg28mxpsXHn5c.html

Cammon guys, just keep it simple, CQRS is not a common user case. Don't need the whole aggregate root? Just forget this dogma from DDD and create a repository for the entity you need. You can name it entityDao if makes you feel better.

I personally would love to get rid of the repositories that we use. But it is really troublesome to do unit test.

Saying that implementing the repository pattern it will be easier to change the underlining database is just a fantasy

@CodeOpinion

Жыл бұрын

It really depends on your context and how you're using it, and what the underlying implementation is and what you're moving to.

@leonardomangano6861

Жыл бұрын

@@CodeOpinion What about moving from a SQL db to a NOSQL db?

@CodeOpinion

Жыл бұрын

It depends on what your abstraction looks like in the first place. Check out kzread.info/dash/bejne/o5l-x7VmZ8_bn5s.html

@nbkredspy3726

Жыл бұрын

I can't think of one project that I've worked on that has actually changed databases. Greenfield or not. I've had a couple of projects where the customer included it in the spec, but it was simply extra development and maintenance cost for a capability that was ultimately not used.

@kabal911

Жыл бұрын

@@nbkredspy3726 exactly. “Being able to switch the database” would rank extremely low in my decision making process

Repository Pattern is a fantasy pattern.. Like so many things in development, it "sounds" good on paper; but it rarely actually works in the real world..

It really seems many are not even using EF Core and the DB context correctly if you're doing it this way. Sorry but the Data model is the DB Context and a repository is just the abstraction to do the context calls so devs do not need to know anything about the actual DB Design. If your using generic Services then you are good to go as you will have error checking in that then it calls your context. I seem to wonder where you get this as hey here is what people are doing when we aren't. There are also DTO's you should be using to hide your exact model. This video came up as suggested but it really seems your going to the fringe of people using things incorrectly and people that do not know what they are doing.

Repository is the bad excuse not to use stored procedures.

@BosonCollider

Жыл бұрын

This. It is an excuse to try to remote control the database across a process or network boundary to compose data that you need, instead of just running a function on the DB itself that fetches exactly what you need.