We used to use the SQLite in-memory DB for our tests. Once we started switching to using TestContainers (since our actual DB was Postgres), we learned that a lot of our tests were actually invalid because SQLiite (and the EF in-memory DB) don't enforce a lot of things, like string length limits and foreign keys.

@umutkayatuz9963

9 күн бұрын

The in-memory provider will not behave like your real database in many important ways. Some features cannot be tested with it at all (e.g. transactions, raw SQL..), while other features may behave differently than your production database (e.g. case-sensitivity in queries). While in-memory can work for simple, constrained query scenarios, it is highly limited and we discourage its use.

@brianm1864

9 күн бұрын

@@umutkayatuz9963 Yep... we learned that the hard way. But thanks to Nick and TestContainers we are going down the right path now!

That password joke was so smooth. I can't stop laughing

I always use a real database for my integration tests, but often I'll write higher-level behavioral unit tests for which I will use an in-memory database (being mindful of the limitations) to keep them quick and minimizing the need to mock.

@AlgoristHQ

10 күн бұрын

I consider the database to be part of the application and part of the "unit" so I include it in my unit tests.

How do you handle large systems with setup and inserting setup data with 100s of tests? Do you use a transactional context that rolls back after a test or regular transactions? How do you do parallel testing if you are sharing the same instance?

@queenstownswords

10 күн бұрын

In our testing, the answer is 'it depends'. It might make sense to spin up separate containers for tests running in parallel... or not. You may need to split out some tests from each other so you can run separate tests in separate containers in parallel. As for the test data creation, I suggest a simple script to run in the before of the test.

@Crozz22

10 күн бұрын

Containers start very fast, you can run tests in parallel and have a new container for every test

@Wouldntyouliketoknow2

9 күн бұрын

One strategy is to avoid tests interfering with eachother is to partition the data based on some scope like using different user, or tenant id's for different tests - if you can. Otherwise I use xunit collections to run tests in a sequence rather than concurrent. Final option is multiple instances of the DB but that is obviously a lot more resources intensive.

@pablolopezponce

7 күн бұрын

@@Crozz22 They are not so fast. We are talking seconds instead of miliseconds. When you have +1000 tests, running them all can take minutes and be very resource intensive (having several apps + databases running in parallel). The sqlite in-memory is much faster. I love using the real db, but it's just not feasible to test everything at that level, you need to test edge cases more unitarily in large projects.

Been doing this for the last few years! Did a lot of discussion on it with EF maintainers and some other thinkers in the space. It works very well.

I also suggest specifying to test container builder a concrete image tag of the db that corresponds to your prod db version.

Feels like Deja Vu... Is this a re-upload?

@nickchapsas

10 күн бұрын

Have you taken my integration testing course?

@SwampyFox

10 күн бұрын

Nick has covered Test Containers in the channel before. There is a good video about using Bogard's Respawn

@VINAGHOST

10 күн бұрын

i think about this right after i saw sqlite in memory stuff

@albe1620

10 күн бұрын

Same here 😅 liked twice onto the upload timestamp 😬

@senriofthealexis3998

10 күн бұрын

Definitely a re-upload

Ok, i have a question, if I set up containers for my tests but let's say that my integration tests require something to be configured in database like a stored procedure or something like that, how should I do it? EF core uses migrations to keep up the database state, is it necessary to run all migrations before the test in that start method or what would be the best solution here?

It would be great if there was any option to do this without docker at all. Although, most of ci's supports it, the key word is 'most'. Moreover, I don't see a reason to use docker for testing purposes only if your app is not contenerized itself and you haven't considered it as a needed feature of the pipeline initially

Hey Nick, I have used this TestContainers but when having many tests more than 100 lets say, its creating a lot of containers in docker, its taking 100% of cpu and memory running all of them and some of them will fail too. Do you know how can we optimize?

I'd like to see how to use Podman instead of Docker Desktop.

I write my Integration tests from scratch, without using any extra library. Excellent results so far.

Nice video but I really feel like it's missing the WHY in memory db tests are a bad idea (like I saw in some comment about constraints and other)

I usually use Sqlite provider with the "memory" connection string. One advantage of the in memory provider though: the error messages are actually better than the TRASH Sqlite provider's error messages. One that comes up most often for me is foreign key violations. Testing with the actual provider you use in production has obvious advantages, but it's still relatively slow.

TestContainers = mindblown, thank you for this!

We tried InMemory db for unit tests for our EF6 db. The in memory db did not respect check constraints, default constraints or triggers; I guess this was obvious. We use the localdb which even works on azure devops.

Our production is Azure Sql, devs and tests all use localdb. Just a difference in connection string and all works great.

That's great. In memory db doesn't really convert the linq to real sql and therefore may hide potential issues when writing complex linq which may call on other methods in dotnet (which may or may not be valid for linq conversions). I will definitely start adopting this in my workplace.

I don't understand. If code first is an option, shouldn't there be a reliable library that can do all the constraint checks also in memory or am I missing something? I already have a real database for integration testing for a legacy codebase but its a hell when it comes to speed. It literally takes around 3 to 5 minutes to execute all the integration tests.

How does this compare to SQLite Integration tests? We use SQL Server with TestContainers, but we aren't able to use it in our pipeline without changing it to a linux agent.

@nickchapsas

10 күн бұрын

Same problem.

tryied to replace my in-memory database with container but ended up with `Cannot resolve scoped service from root provider` even when i almost copied your code (important parts) ... :(

The only way to mock is to never mock

Okay but for me the bigger problem when testing EF Core is testing code that uses views. We use database-first + scaffold, but the test database has no information about the views. We have to mock the view db sets with lists, forwarding the IQueryable and IEnumerable. But you can't test when using views in a more complex query

@viniciusvbf22

10 күн бұрын

Wow! I thought the database-first users were extinct. Thank God I'm not alone 😊 The sad reality is: the tools are not made for us anymore. If you're not developing microsservices + .NET 8 + EF + Code First, you're doomed. You're being forced into an architecture by the tools, and not the other way around, which is crazy if you think about it for a sec.

@coloresfelices7

10 күн бұрын

Just set up the views using the appropriate SQL commands when starting the container.

@davidantolingonzalez4929

10 күн бұрын

Populating the tables of the test database it is something you have to do no matter if you are using code first or database first. You can do it via executing an SQL script or by instantiating your DB context, adding the data to the corresponding DBSet and executing SaveChanges.

I used an InMemory db for some "integration" tests at work until I had introduced a temporal table (InMemory db doesn't support temporal tables). So I had to overwrite everything to use test containers. Now I don't use InMemory db even for unit tests

Love the ease and speed of the InMemory variant though

IMO: In-memory database is for unit tests! Testcontainers are for integration tests. Question - are you able to run these tests in parallel? Is it one database per test, if so what is the overhead of doing that for 100-500 tests?

@darioferrai8691

10 күн бұрын

You can use a CollectionFixture to "group" tests together and have them all use a single db. You can then launch those groups in parallel

@lepingouindefeu

10 күн бұрын

I just don't see the point in unit testing something that uses EF. Just do integration tests.

@dy0mber847

10 күн бұрын

Testing out-of process dependencies with unit tests🤦‍♂️

@llindeberg

10 күн бұрын

​@@darioferrai8691recipe for flaky tests

@llindeberg

10 күн бұрын

​@@lepingouindefeusometimes agree

I feel like a SQLIte database in memory mode (“DataSource=:memory:”) is what most devs think an in-memory database is. But the in-memory provider for EF Core just holds objects in memory and acts upon those objects in memory when you call the various DbSet methods. I don’t think it even generates any SQL. But that SQLite approach might work, if containers aren’t an option.

What IDE do you used?

@tera.

8 күн бұрын

That was JetBrains Rider in the video

Is this dependency implicitly creating and executing the migrations?

You omit a lot of things. How the test container picks up the db schema?

@Ry4nWTF

10 күн бұрын

he probably uses code first.. would like to see someone do this with DB first, where the schema/db migrations lives somewhere else

@liquidpebbles

10 күн бұрын

You know these videos aren't meant to be all encompassing right? They more just shed light on things that you should be aware so that you can go do your own deep dive on if you need to.

@Ry4nWTF

10 күн бұрын

@@liquidpebbles he doesn’t need to spend 10 minutes showing how to use an in-memory dbcontext or barely using test containers.. plenty of time to go more in depth, but also keeping it abstract enough to not go too much into implementation

In which cases can we use the In-Memory Database? What is the correct usage of it?

I do not think there is any place for any kind of replacement, mocking or anything for the integrations test. If one is testing the interaction between 2 components, how replacing one of them would make any sense? You are running MS SQL in prod, but using MySQL/SQLite in integration tests? How is that an integration test? Quote from the in-memory db package page: "This database provider allows Entity Framework Core to be used with an in-memory database. While some users use the in-memory database for testing, this is discouraged"

I understand the premise and agree that it's not testing the actual conversion of SQL, but you know what? I don't think it's that bad. Surely it has a few drawbacks, but so does every approach. Some of the benefits I like are: - It's portable, and runs straight from the IDE or pipeline without needing to set up a container runtime. - The tests are usually quicker, as they can be easily parallelizable (Especially with SQLite provider), while actual DBs need to run mostly sequentially to avoid having the data of one test impacting the other. - Easy to run + Quicker feedback loop = Better developer experience. The TestContainer approach also has some drawbacks: - Some teams don't have EF migrations, but the DB is versioned in SQL Scripts that run a separate pipeline, sometimes in a different repo. Creating a TestContainer with the correct schema may be more difficult. - Now that Docker Desktop require licenses, some companies are using containerd under WSL2, which (AFAIK) would not run natively unless it's configured to have the socket exposed in a local port. - Some CI/CD setups don't support containers. You also mention that unless it's using the actual database, we don't have good integration tests. I'm not too sure I agree with this point, for a couple of reasons. One of them is the difference between Unit and Integration tests. I'm aware of some heat in the community over the past years regarding the Chicago vs London school of thought (a.k.a., "Sociable Unit Tests") when it comes to the definition of unit test, but it's arguably that if Unit tests are supposed to test just one unit, then if you have more than one concrete (not mocked) service class instance being test you have an integration test. The other reason is... does it really matter? Saying that you don't have good integration tests unless you use the same DB is like saying you don't have good integration tests if you don't use WebApplicationFactory and call your API in your tests because that's what your clients use. I'm not too keen on this mindset, as the core of our solution should be the domain, the business logic. That's the critical part, that's what needs a comprehensive test. Entrypoint layer (being it HTTP, gRPC, Service Bus events) and persistency (being it SQL, NoSQL or simple text files) are implementation details, coupling most of the test with those seems a bit odd. And making a test that is supposed to verify some rules under a chain of responsibility go all the way from API to the Persistency layer when just the raw classes would suffice seems like unnecessary overhead and hurts developer experience. At the end of the day, there's no right or wrong. Each team uses what suits them best. I just don't think actual DBs on TestContainers are better than an in-memory provider, it just has different pros and cons.

It's exactly how we are handling integration tests in my company (in combination with Respawn) :)

The mail chimp to get the source code doesn't work for me. I press "email me the source code" and nothing happens. At least give me an error, I mean, come on!

Always test against the real thing! @nick I thought you were leading us down the wrong path, but then you switched it to testcontainers, nice!

I don’t see any problem with setting up real integration tests. The application should be ready to switch the connection and just works. You will end up also test migrations. Only benefits with this approach

The second solution has only one problem - docker desctop are denied for corporate users. So or buy license or run ubuntu with docker inside WSL2)))

@nickchapsas

10 күн бұрын

You can use podman

Cool technology, I had tried it before, and will give it another try buuut I am struggling to understand what the real world usefulness of this is. It’s not useful for unit tests, don’t need to hit databases for unit tests and far too slow. Limited to Not useful for integration tests because the database is not likely in a testable state and would be super arduous to get it there. Sure it is cool to have a database that can be scrapped after the tests, but using snapshots I can do that too, and the database can be in more useful states by using a real db image. Can you give some more context or cool examples of how to actually make integration tests easily using this?? - nick I know you can- go on! 😊

@z0nx

10 күн бұрын

Why would a db not be in a testable state? We make our own docker image that has a dacpac pre-applied, for example. This is just a way to write more self-contained integration tests i guess, so as long as you do integration tests all is fine.

Unit Testing is not really worth it in most modern applications where the domains are well separated and more easily testable. Integration tests and functional tests are FAR more cost effective, and can be done in parallel by different people, and are resilient to massive refactors. It makes your code better too, not to have to be worried about stupid things like TDD and unit test coverage.

I think of In-Memory as a fake for unit tests, instead of using mocks for repositories, you use a "real" implementation that behind the scenes calls the EF's In-Memory provider.

@z0nx

10 күн бұрын

Ye, a big antipattern is using mocks (dynamic mocks like Moq, NSubstitute) for setting up repositories in unit tests. Using dynamic mocks couples your tests to the implementation of the SUT. You really feel that pain once you modify/refactor the implementation.

Just override connection string Environment.SetEnvironmentVariable(“connection_string", "…"); instead of reassigning dependency

"flaky tests" means they pass/fail intermittently/randomly without changes to the code

Make simple... guys, make simple.. The customer/employer does not care how complex or well thought out the solution is. All they care about is 'Is it working? Are our customers satisfied?'.

Once you use EF.... you are EF'd

As not native speaker I like your videos that were made 2 years ago. You were talking not so fast and it was easier to undestand. Now I have to set speed to 0.75 (or even to 0.5) and you look drunk.

@nickchapsas

10 күн бұрын

It’s because I am drunk

Or: get rid of EF entirely

@Rick104547

10 күн бұрын

Why exactly? I am pretty happy with EF provided you don't make the in memory provider mistake ofc.

And again huge fat dislike for annoying ads of your Domtrain