"MONGO" is an anagram for "OMG NO."

@mauricewebb1098

Жыл бұрын

Lmao wow

@riccardotoniolo9604

Жыл бұрын

I am using it just to learn it. Why is it so bad?

@daniedmola7465

Жыл бұрын

🤣

@TheJobCompany

Жыл бұрын

@@riccardotoniolo9604 This video goes over why Mongo is recommended against at 17:22. To reiterate, DocumentDBs have valid use cases, however their major problem is that they make relationships hard to handle, which makes them an inefficient tool for most applications, because lots of problems require relations.

@ru2979

Жыл бұрын

😮😮😐🤭🤭

There were some incorrect facts about Hasura: - It is not proprietary, but Open Source Apache 2.0 and MIT licensed - You are not locked into proprietary infrastructure, you can host it yourself, with anything that can run a docker container (Kubernetes, Cloud Run, Railway, Render, Digital Ocean, AWS, etc) - It connects to standard databases: Postgres, MS SQL Server, MySQL, Big Query, etc, so you can replace Hasura at any time, and use Prisma if you prefer. - It provides both GraphQL and REST APIs for accessing data - You can put a server in front of it anytime, just for some router, or right from the beginning, and treat it has a GraphQL abstraction layer over SQL, similar to what you would be doing with a non-standard ORM interface.

@BosonCollider

Жыл бұрын

We are hosting Hasura ourselves and it is still a bad idea from our experience, but for entirely different reasons, where it seems to have a weirdly bad interaction with postgresql and pgbouncer, and the specific way in which it handles roles and schema separately from the database is really annoying. From what I hear, postgraphile is nicer. We're replacing it with postgREST for the rest API bit and pg_graphql for the graphql query bit for those people that liked it. We haven't run into any scale issues with postgrest so far, and running on essentially the same stack as supabase is generally comforting.

I understand your issues with mongo, but there are reasons to use mongo apart from relationships. - You cant treat relationships in DocumentDBs the same way you do in SQL based, the base for the relationships are different, in documentDB you design your relationships in a way that makes it easier to query and get the data you need for that query. based on your example you store the user reactions inside the comment with the user info inside the reaction object. If you don't need to update the information in a sub collections very frequently then separate them in a new collection otherwise if that data is mostly read them put them in the object it self. - If your data structure is not consistent 1 object has 3 fields others have 5 for example, in this cases SQL based DBs are not the right option for you. I may be wrong here but it seems you are looking at document databases with the same mentality and structure as you do for SQL based ones and that is the wrong way to go about it. In SQL based databases you store the user in a table and you create relationships to other tables to point to your user, in document databases you can put the user info inside the object you want to show. In you example the user info could be inside the reaction object, the comments object also has the user info for the user commenting. In documentDBs you dont separate data based on their relationships but rather based on the info accessibility. If you need to update the related info very frequently separate that into a different collection otherwise just put it inside the document, the concept of a single source of truth does not exist in documentDBs. In my experience most of the codebases I see with mongoDB that have problems is mostly because people design their databases based on the SQL principles and dont want to repeat info inside the databases. Imagine a marketplace with 10M products, if you put this into SQL you will have a bunch of tables that need to be queried every time you want to list 1 or more products, if you use documentDB you can include all product info inside 1 object and get the catalog with 1 simple query, on the other hand if you want to change the name of a category in the product it is easier to use SQL, you change it in 1 place and all documents are updated, on a DocumentDB you will need to get all products that have that category and update all products info. Yes it is more complex to update but it is way faster to get the info. As a rule if you need to read 100 times for every write you do, them documentDB may be the best option for you.

@seanchen9771

Жыл бұрын

What is your opinion on using SQL in the back and then cache the more "high level" results in Mongo, like using Mongo as a cache. All mutations go to SQL, then Mongo updates based on the new changes from SQL. Queries go straight to Mongo.

@MiguelPintolookatitude

Жыл бұрын

@@seanchen9771 I would not use mongo in front on a SQL database, unless you have a very specific situation where that makes sense, if you want to use cache use Memcached, or even Redis in front of your SQL database, another solution is to store the query results in elasticsearch. There are tons of solutions for caching SQL databases out there.

@eizen8774

Жыл бұрын

"our data structure is not consistent 1 object has 3 fields others have 5 for example, in this cases SQL based DBs are not the right option for you." What? is this serious because if thats your problem just create 5 fields then just return 3 fields if thats the only thing you want. { id: 1, name: John, comments: [ { id: 1. description: Sample, } ] } This kind of structure can only be found on simple backend. In real life example, comments are not tagged in users alone because its tagged in thread which has multiple comments from different users. How can you achieve that through documents?

@MiguelPintolookatitude

Жыл бұрын

@@eizen8774 in such a structure and if you follow proper SQL based design rules the comments in your example should be in a separate table, and you would have a table that would relate the comments with the users with many-to-many relationship. Indexing array values in SQL is expensive. In a document DB you would list the comments as a collection and ever document would have the information regarding that comment inside of it including the user info: { id: "comentid", description: "your comment", user: { id: "userId", name: "john"}} If there is another comment you repeat the user info for that new comment. A simple query like get all comments where user.id == "someUserId" is quite easy to write either in mongo, dynamo or even firestore. It is just a different paradigm, it works in a different way you should not think about it the same way you do when structuring your data for SQL. There are good and bad use cases for both SQL or document based DBs. In general if you just need to get information about a certain document (for example products in an ecommerce scenario) if you have a catalog with 100 products is fairly safe to say you can do it with SQL but not at a big scale, those joins you use in SQL to grab related info will compound really quickly when you start entering the 100k to 1M marks specially on reading. This would give a much longer discussion than this comments allow to... But if you need to read a lot it is better to avoid dealing with relationships on every read.

@NeuralNotes69

Жыл бұрын

Bad luck, he doesn't reply criticism only compliments. Proves he is very ignorant.

As a senior dev, I have to commend you man. Your chats are great. Even if I already 90% of the stuff I still listen to you from beginning to end.

@windwardhive7363

Жыл бұрын

You really didn't sit down and watch all 1 hour of this

@sircalvin

Жыл бұрын

@@windwardhive7363 a lot of people watch long form content, for a relevant example i watched all of theos 4 hour ama yesterday

Hey Theo, I am an engineer working in the core FaunaDB team and I think you have gotten couple things wrong when describing Fauna. First of all, you can put a server in between your Client and FaunaDB nothing is preventing you from doing that. Second, Fauna is not like Hasura, yes we do have a GraphQL api and you can choose to connect to your database directly from Client but you don't have to. Fauna cloud offering is no different that DynamoDB or Mongo Atlas cloud offering. And I am not sure why you think you can't control data permission with Fauna.

@sohn7767

Жыл бұрын

I did get the feeling that he just threw Fauna into the Hasura category without fully knowing what it is

@matheus-kirchesch

Жыл бұрын

He talked random nonsense of mongo too, then proceeded to draw an extremelly dumb relational db diagram, he is clueless, probably just wanted to convince his audience of new devs that he knows stuff

@user-sh3uq1ct8t

Жыл бұрын

@@matheus-kirchesch absolutely YES, and it is a reason why dislikes were extremely usefull on youtube.

@musashi542

Жыл бұрын

@@user-sh3uq1ct8t he has 87 dislikes right now , so what are u on about ???

@jesseliverless9811

Жыл бұрын

@@musashi542 People people don't dislike anymore, since it's pointless, since it doesn't show up. You can only get it through extensions (and the numbers aren't even correct), which very few people got through the trouble with, because (going back to my first point), people don't care to dislike anymore. So, yeah, that's why the dislikes were extremely useful on youtube.

Thank you for this! The video made me pause/think more deeply about my choice to go w/ Mongo for an indie project I'm working on, and I started writing lots of code to handle relations, even after it started to become apparent not the best tool for the job (but was too lazy to want to do anything about it). Nothing against Mongo, I need it for certain applications, but this video helped motivate to bite the bullet and switch over to relational db, and no question it was the right choice after taking the change. Thanks again!

Just remember ya'll there really is no such thing as making a "wrong" choice. If you went with Firestore, Mongo, or any of the others spoken badly about that does not mean you need to rewrite everything. Some solutions may be suboptimal but all that goes down to trying out technologies and learning the intricacies of each solution. For 99.9 percent of the projects, they will never reach the scale of teasing out some of the problems with individual solutions. And if that .1% happens, take it as a learning for next time.

@philsburydoboy

Жыл бұрын

Yeah. Just chalk it up to experience. Using the wrong tool for the job is part of learning what the right tool is.

Shared this with my dev friends, I think this was the best hour of content on this channel so far 😎

Thank you a lot for your insides, been waiting for this a while. Please more on databases and why to choose them and the ease of use in comparison to the feature set. 👏🏾🙏🏾

Thank you for making these videos! I’m trying to learn web development and your videos have been very helpful!

Well, ya gave me the confidence to not think I need to use Firebase whenever I come around to starting a side project. Also, the Railway website looked tight to get some stuff up and running quickly. I definitely struggle most with figuring out authentication and CI/CD.

Neo4J is amazing for analysing data that can be conceptualised as graphs. I was doing some research-like work (investingating RNA-protein-disease-pathway-compound networks), and it was so nice for graph searches. Though it does require you to re-conceptualise some things (say, large enums, such as caregories of functionality) as abstract nodes - but when you get it right, you can zoom around the network SO fast! It's great. But yeah, I wouldn't use it for an app, unless I had a very good reason to

For MongoDB or any other NoSQL, you design the database based on the questions you asked, ie. your queries. For SQL database, due to its relational nature, you design based on how the data relates to each other. For NoSQL, designing data by questions means denormalising a lot of the data by access patterns. This also means that there can be data duplication, and you have to account for that upfront in terms of data cleanliness. That means more work that you might need to do for benefits that you might not need. You also giving up the very flexible nature of SQL for this. The benefits of doing that is increased read throughput since all the data that is accessed together is clumped together. Is it worth it? I would say it depends on the business and should be a business question whether it’s worth it or not. Now, can this be implemented in a SQL database anyway? Well yes. But the nature of something like MongoDB is that it’s much more seamless to use it this way since it’s designed around that. Now, when you approach on using MongoDB like it is a SQL database of course it going to be complicated. It will feel like a hack rather than what’s it’s designed to do. In other words, don’t use MongoDB like its some sort of a relational DB. It will just hurt.

@t3dotgg

Жыл бұрын

Totally agree. I've been convinced by some very smart people that Mongo is ROUGHLY in the same category as Dynamo of "if you know what you're doing, good shit. Ignore my video and do that" Real talk tho: how many Mongo devs have you seen using Mongo correctly? I literally never have in my life 😅

@BosonCollider

Жыл бұрын

I think one particularly interesting thing that rarely gets touched upon here is that there are two subtly different way to do relational databases that are VERY different from each other. On one hand you have traditional ER diagram models that naturally end up in 3NF. On the other hand, which includes the largest actual data sets, you have completely denormalized uses of relational DBs using star schemas where the dimension tables form a common bus (Kimball architecture), and where the data set is *a lot* easier to query and a lot easier to add/remove tables to/from, but where you have slightly weaker enforcement of constraints due to being denormalized. Dimensional modeling and fact tables is imho a good fit for many cases where people go with noSQL just to denormalize. You do not need to abandon relational databases, you can just use them differently.

@user-ge2vc3rl1n

Жыл бұрын

@@t3dotgg We use mongo at work and for a solid year we had good API design and we were able to fetch data logically. At some point it started getting so convoluted that it was far easier to work with TS and filter data with TS than it was to make a query. This was simply a skill issue.

@CodeVault

11 ай бұрын

​@@t3dotgg In a volatile environment where it's difficult to know the schema beforehand I find it much easier to work with Mongo (or any document based database). At the company I previously worked for we had an entity called an Order. This Order had so many sub-entities linked to it that it was basically more than half of the database itself (40+ tables just for this entity and its subentities, and we might have had to add more). Any query, any modification of that Order was a huge pain in SQL because you always had to use all the subentities. The same thing in Mongo would've have been incredibly easy

@Davide73

8 ай бұрын

You cannot write in the same sentence "NoSQL" and "design" !

Theo you are dev youtuber we didn't know we needed, but we truly appreciate!! Def not for beginners but there are a bunch of other great channels for that need. Thank you for being constant at being you and putting yourself in a video! lol

@jww0007

Жыл бұрын

also for beginners too that understand first principles & the presence of tradeoff

@mianala

Жыл бұрын

true

I would love way more backend content from you, Theo. I find that I learn quicker from your explanations than I do from most other content creators and I am way weaker on backend than I want to be.

MongoDb still the good choice it have lot of functions to relate the documents and we can use populate for get those relations it has lot more functions like that.

Your talks are perfect to listen to, I'm almost treating them like podcasts and I can learn some interesting things I wouldn't have thought about. Thanks for creating such a a great content.

Dude I have been thinking about this for so long! I always liked SQL, it works great for every project and there's ton of official documentation and forums out there, way more than Firestore alone. Besides we get more compatibility with legacy systems and you can still control your APIs and make the necessary changes as they come. Wow It feels good to get that out of the chest hahaha

Thanks for the vid I was already on the "I run my DBS on bare metal", but as dev u heard of lot of new tech everyday and you cleared some shit of my mind thanks for that !

One of the advantages of document based databases like MongoDB is that you do not necessarily need migrations, instead you can use different schema versions of your document simultaneously and manage the migrations in the application layer without the need of running and syncing huge migrations at once. In relational databases on the other hand you need a migration for every single field you add rename, table you add etc. Also the absence of foreign keys (and migrations) is perfect for sharding and partitioning and therefore scalability. I think in relational databases migrations tend to be rather painful, since you are forced to migrate your schema and need to sync all your nodes. I really like the video, but the MongoDB bashing seems a bit too subjective to me, and I assume you did never design a sophisticated document based schema. E.g. in a real world document schema you would probably not add all comments to the user collection unless you expect not more than a few comments per user and also need the comments to be available everytime you load a user document. There is an old talk from one of the creators of reddit here on youtube in which he explains that they had to create one giant, flexible table for almost all of their models in order to ease migrations and scalability, which for me sounds like they tried to replicate a document database with a relational database. I think you should choose your database depending on the requirements and there are pros and cons for both concepts (many more than I've described).

@carlosemvqueiros

Жыл бұрын

He doesn't seem to know that Mongo has references.

@xDJxG0LD3Nx

Жыл бұрын

Mongo literally can be a "relational" database. That second diagram he did for SQL is Mongo too, the User object can have an array of commentIds, then to get those comments you just pass those IDs to the Comments collection

@JudahHolanda

Жыл бұрын

And he made the worse model possible for his problem.

@amd9918

Жыл бұрын

@@carlosemvqueiros yeah he doesnt know shit about mongo.

@adrycough

Жыл бұрын

I agree with what you said about sharding / scaling out horizontally. Maybe I'm just wrong, but wouldn't it be much easier to just do one huge migration rather than needing all these "what ifs"(which I assume you are persisting forever) inside the application layer to make sure your schema version is up to date? And then, when you start adding new features, you have to always take into account that some document might have an older schema version. I think his main point was that relational databases are faster and more efficient in production, especially for big data. I do think it's silly that he says you shouldn't use MongoDB because of that, as not everybody is trying to be big tech/data (even if you were, it's probably not too hard to transform your data to fit into a relational model by the time there's any real benefit to swapping). I almost exclusively use MongoDB because of how easy it is to get up and running. I don't want to sit here for an entire day+ every time I work on a new project to draft up some ER, etc, diagrams, just so I can start working on it (All so I can save some CPU cycles and queries? No thanks).

I feel like I need to say something.. The idea that document dbs are completely a functional subset of sql dbs is not entirely informed. The functionality of Mongo aggregation queries _cannot_ be fully replicated in SQL - Take the “Users” and “Comments” example, you would either need to join Users to Comments and pull the same user record for each comment record (denormalized, and therefore larger result), or do one query for users and one query for comments (normalized, but multiple calls to the database). In mongo, if you had the same setup (a Users collection, and a Comments collection, just no relation enforcement) you could join the Users collection to the Comments collection with a $lookup aggregation and pull down a completely normalized result in a single query. Wether or not/where this is useful is up for debate, but it is certainly something a document db can do that a sql db cannot do.

@Channel-yl9fh

Жыл бұрын

Great comment!

@mileselam641

10 ай бұрын

With CTEs, jsonb functions, etc. Postgres could 100% do this "only a document DB can do this" query with a relational model. Is it as easy as pulling out of a document DB, no, unless… …you store documents inside jsonb columns, in which case it's about the same complexity and speed as MongoDB.

@the-avid-engineer

10 ай бұрын

I am not very familiar with CTEs or all of the josnb functions available to postgres, but perhaps my example is not a good one and the boundaries are being blurred for specific implementations of SQL and specific functionalities of mongo’s aggregation pipeline. My main point is that the aggregation pipeline is not a functional subset of SQL; $lookup is but one of the many operators available, and I’m not convinced that SQL can do everything that a document db can do

@mileselam641

10 ай бұрын

@@the-avid-engineer Then the focus should be on finding a query pattern that document DBs can do-or do substantially quicker-but something like Postgres can't in a timely fashion. So far, I haven't seen such cases, but my expertise is in SQL engines, not MongoDB. An example outside of this discussion, Graph databases, shows how optimizing for the connections rather than the nodes can provide substantial performance wins (though recursive CTEs are still able to do the same tasks, only orders of magnitude slower).

🔥Firebase sounds amazing, can't wait to try it out! Thanks!

20:00 Jesus, if you design your data structure like that, congratulations on not forgetting how to breath, but don't blame mongo for not knowing what you are doing

@amd9918

Жыл бұрын

yeah hes kinda dumb hahaha I already stop watching on that part..

@voidemon490

Жыл бұрын

ikr, like you define a normalized schema and you except to work. it's the whole point of using mongodb to have denormalized data as much as possible to ease scaling...

@user-ge2vc3rl1n

Жыл бұрын

This is just a reminder to never blindly follow tech youtuber advice without a grain of salt. This is absolutely NOT how you would want to use mongo.

Excellent video and excellent comment section. Thank you to the engineers in here adding their own experience & clarifications to the discussion.

Thank you very much Theo for the video about DBs. I wish someone had told me all this years ago (of course none of us knew). It seems that almost all of us have suffered the same path of adoption, hype, disenchantment and technological debt. It is sadder in cases like mine where despite having more than 10 years of experience with SQL DBs we jump to Mongo DB and other document-based databases as if they were a panacea. Regardless of what you say, there is a MongoDB usage pattern that we use a lot in particular: A SQL database with a relational model with a centralized Back Office and Mongo Servers in the different regions in which we model the data exactly as the views (normally from ecommerce) require to paint it (as a "complex document cache of JSON", usually with the SLUG as the collection's primary key). In this way a getServerSideProps (to put it in terms of NextJS SSR) only has to do a findOne (by id) to a MongoDB Server located in the same region to fetch all the data required by the “view” (and some times throw query’s to overwrite some of the data with user’s personal pre-rendered information). But I wouldn't recommend anyone to use Mongo's indexes and aggregations to do Joins and expect it to scale. Not to mention the architectural disaster of sharding and clustering.

@adrycough

Жыл бұрын

I think the switch to document-based databases is due to the highly sped-up development time, originally I thought it was silly, however, it is much easier than having to draft up ER, etc, diagrams to even get started. I don't think most people are pushing their hardware to the limits anyways, and it seems like it should be easy enough to transform your document-based data into relational data if/when the time comes. Why do you think sharding/clustering leads to "architectural disasters"?

@fizzcochito

Жыл бұрын

@@adrycough computers are also so fast nowadays that even inefficient DB architectures are good enough for 90% of people

@Isweir

Жыл бұрын

Why don’t you use mongo partitions? And caring about sharding so much?

watching this as I am helping to build an application on Firebase using Firestore (previous CTO decision). Very helpful, will keep the issues in mind, and try to isolate all of the database components so that it will be easy to move off later when we re-build the application. Will look out for costs and security.

Thanks for this rant sir! This helps me a lot even if I need to do some more database homeworks yet again 🙄😅...

Just started watching this right after choosing firebase for my next project. Was able to switch to vercel etc once you pointed out the flaws in some of the things related to firestore -- thanks!

Some of your sentences and paragraphs transverses years of education and experience. There are only so many people that are truly a gift for youtube viewers like this guy right here. Thanks man!

I am just so thankfull for Theo existence, he explains things so well and shares best use cases for everything. Just thank you!

Thanks for so thorough review!

It's looks like you've never done MongoDB property before. You can do relationships and use "aggregations" which is much more powerful then anything SQL can output.

@liquidcode1704

Жыл бұрын

Yeah, I think he was trying too hard to sell his points at that point lol.

@matheus-kirchesch

Жыл бұрын

Yeah, take a look at the relation structure he designed at 21:00 the man is a living joke trying to teach others.. fuck sake

@amd9918

Жыл бұрын

yup hahaha hes a dum dum

@adrycough

Жыл бұрын

Did he actually say you can't do relationships? In his MongoDB example(timestamp in description), he seems to understand that MongoDB can have relationships. I do remember reading, multiple times, that Mongo's strong suit is querying for a single document rather than multiple. I think he puts way too much weight on speed though, as I highly doubt he recommends writing out a server in C++, or even lower-level languages.

Did you try Neo4j? I think it's awesome. But i guess you only see the full potential in apps needing graph search algorithms (like a social). The cypher language is much easier and intuitive than it seems at a first glance

In terms of NoSQL, the main issue was you still have relational thoughts on document-based design. The NoSql design was not correct.

You should take a look at databases that support the OpenCypher query language. Thinking in graphs comes just so much more naturally than any ORM or SQL.

Ive been using firestore as data transport between the backend and the frontend rather than a "database". The backend only writes to it and the frontend only reads. This also helps keep security rules pretty minimal. If I want to get rid of it cant I just choose a different transport layer?

From a Data 4 Analysis person the answer is always PostgreSQL. The way you can use Partition Windows is perfect, better than Oracle and M$. I know this video is not about representing any SQL DB, but you gave me so much ammunition against Document Stores that I have to thank you. I now understand that the message system that we were trying to build in MongoDB will be done better by simply using a K/V in the cloud or Redis on premise. Utilize technology without locking yourself into a platform. Robust yet nimble. I can get behind this concept of builders instead of endedness.

I totally agree wrt mongo, firebase, and fauna. But not so much with Hasura. 1. Hasura gives you a default GQL API for crud operations and basic permissions, making easy things easier, but this is JUST THE BEGINNING. It makes hard things easy too. 2. You can add your own custom GQL mutations/queries implemented as an http endpoint. You can do anything, including accessing your the SQL database directly with custom SQL queries. 3. In addition to #2, you get *** tRPC *** -like functionality with the "hasura actions codegen" command with the "typescript-express" template, so along with other GQL codegen tools, you can call a server side function on the client side as a direct function call. 4. Hasura allows you to extend your GQL schema with another remote schema. For example, you could extend the Hasura schema with an Apollo Server for your more complex cases. 5. Hasura has remote webhook triggers on CrUD operations, so you can run custom code on data changes. Once I had my tools configured, Hasura made easy things easier, tedious things fun, and hard things easy (or at least no harder than other means).

@balajibobby8530

Жыл бұрын

💯

@santicomp

Жыл бұрын

I agree 100%, I have been using it before 1.0. It's flexible and can be integrated with actions and events to lambdas or anything. I host my own instances, never had an issue. Rock solid 👌 💪 🙌

@perryyyy

Жыл бұрын

true!

I've been a programmer a long time, and this is by far the best explination I've seen of KV/Document DB / SQL. Love the Venn diagram explanation. That's so true. Funny that I've never seen anyone else explain it that way.

Nice to see there is still a strong love for SQL in the nodeJS community, I am primarily backend JAVA developer who's thinking of switching to NodeJS on the backend, I am still on the fence. But this rant opened my eyes to a world of all sorts of crazy stuff! WOW!

Thanks Theo, this is super useful and timely content as I am thinking about the stack for a serverless sveltekit / graphQL project. What do you think about running Prisma in conjunction with Supabase as opposed to Supabase's built-in `pg_graphql`?

For my first big project I ended up going with postgres on EC2 and i wish I went a more managed route. One of the strangest ones i've used was a google sheet that i web scraped to manip data because i was too lazy to swap from a spreadsheet to something actually good and ended up just making more work for myself.

@abbe00abbe

Жыл бұрын

Haha there is atleast an api if you for some godforsaken reason wanna use google sheets as a database

@gingerbeargames

Жыл бұрын

@@abbe00abbe i remember using it and it being very janky to do what i wanted to do so just webscraped instead def wouldn't recommend.

Thanks! You just saved me from learning Mongo for an app when I already have a decent grasp on SQL!

Ah your videos are honestly always so good

Hasura is opensource. You don't need a proprietary client for hasura, the datastore is just postgres, it's not Proprietary. I have a decent experience of hasura in production, I can talk a bit about the not so good parts, they are not the one you talked about. Except the authentication part, it was a bit restrictive to have to use JWT because hasura only works with JWT, where our application has no need for it. Permissions of hasura has been not really an issue for us, the column level permission are very granular and expressive. We selfhost everything, optimizing query is doing postgres indexes it's not that abstract, using hasura does not mean you can't use a server too 🤷‍♂ Bad points for us was: - easy to write complex queries, it's harder to realize the cost of things - the given console can be painfully slow, have not seen much love in the 3 years of using the db - JWT only - Subscription are NOT FAST and scale super bad, use with caution. I would not re-pick hasura for my other projects today, but i had a good experience with the technology and it allowed to move fast when we needed it the most. I still see it as technical debt

@jorgemacia9902

Жыл бұрын

Hi GodOfMacro, thanks for your comment. What alternatives to Hasura would you pick for other projects (if in need for complex queries and subscriptions)? Thanks

@GodOfMacro

Жыл бұрын

@@jorgemacia9902 I do not have experience from a successful "generic subscription" tool, I would say, not firebase haha. I tried pocketbase, it's not ready for production, the subscription is not reliable (updates sometimes skipped / missing). If I had to do it again, I would do subscription myself directly.

@jorgemacia9902

Жыл бұрын

@@GodOfMacro Great, thanks!

@mileselam641

10 ай бұрын

@@jorgemacia9902Postgraphile

Yo, I have been watching this videos and on twitch for some time now, and I love it. You are so unhinged is super relatable. Keep up the good work man.

This was great, I've used most flavors of SQL but never heard of half of these and always had some vague feeling like maybe I'm missing out. Now that feeling is gone :) that firestorm and fauna stuff seems quite hellish

I've used Mongo fairly heavily. I wouldn't knock it that hard. You can band-aid a wide variety of its problems. It's better designed for more monolithic data, obviously, but there are a ton of things you can do with Mongo that feel simply magical. This kinda all goes down to a data storage level, though. It's a special use tool that people outgrow because they can prototype so quickly and end up stuck with a prototype their management doesn't understand wasn't designed for production.

One argument I'd like to make for mongo is that You can use mongoose ODM to map relationships between datatypes with the 'ref' type and the related object would have the same ID while remaining in different tables. I'm not that experienced but I'd like to know if you have actually worked with mongoose.

@abdimussa8932

Жыл бұрын

Yes this. I don't know if there's any downside

Thanks for this video, I appreciate it!

Just for the sake of simplicity around 5 years ago I started using NameOfTableId to describe foreign keys with the lazy purpose of seeing the field and automatically recognize it as a foreign key, still use that pattern today on my sql databases, but I didn't remember seeing anyone using like that before. Anyway happy to find a kindred soul

Thanks. I'm new to cloud, so it was helpful. Quick question: For my project, I was considering using a remote file system (like AWS S3) and it seems it would fit your definition of database. As it's mostly to store logs (a lot of writes, not a lot of reads), I felt like it was a valid option. Am I too old school ? :)

@t3dotgg

Жыл бұрын

For logs and "dumb writes", S3 is a totally fine option! There's a few other things like this (Snowflake and BigQuery come to mind). Generally I only care this much about "active data", like the things that put my business under if they fail. I feel significantly less strongly when it comes to what you're describing

This channel is a gem for beginners!

I used to have mongodb in my project, but later, i think mySQL is still best. And there are so much job in indonesia for SQL db rather than noSQL. For playground, fauna is good and easy to integrate. There are also CouchDB, database that can connect with rest api

"they kinda suck, but they dont suck that bad" :D i love it. Thanks for another informative video theo!

There is one issue for folks trying to build something quickly with SQL. It is offline first app. You'll need a DB on the device and another one on the server and figure out how to sync them. I hope a solution exists for quickly prototyping an offline first app. I believe MongoDB Atlas and FireStore are presenting themselves as a solution to that.

@mekelius

Жыл бұрын

This so much. I tried to write the syncing code myself but oh god what a clusterfuck. Firebase solved it for me.

@AlexanderSuraphel

Жыл бұрын

@@mekelius hopefully SurrealDB does it right.

Mongo is great but you need to think how to dessign your collections. You can't just put everything into the arrays of the documents. There are some limitations to documents and developer should know those limitaitons and design everything properly. There are relationships in mongo but it's quite different than what we have in SQL dbs

MongoDB is legit my go-to for pretty much anything and now I'm feeling worried to hear why it shouldn't be...

@shioli3927

Жыл бұрын

Honestly for me too. I don´t think most of the points he had are a problem with mongo specifically rather than people treading mongo like any relational sql database and designing a mess. You can build horrible sql databases too. In fact when I started working where I do right now the database was mssql and it was slow af and it was a mess beyond "fixing it with migrations". To be clear not saying mssql is a mess, the database model they conjured up was bad as nobody had any idea what the program is actually REALLY gonna do at the time of making it up and leaving all the type decisions to an ORM is not ideal either. I do agree that too many people jump on hype trains for no reason. But imo mongo is legit a very good db, with very good sdks (although i mostly know the c# one, the c# one is great though) and it does not vendor lock you to a specific cloud.

@steventerry4117

Жыл бұрын

@@shioli3927 Yes! Thank you. After watching the video I honestly thought his issues with MongoDB seemed a little offbase. Yeah, MongoDB is bad when you use it for things you shouldn't use it for. That's just a redundent statement. Need a LOT of relational connectivity? Well SQL isn't your friend either, and a graph DB is what's appropriate. Good luck even getting your query right with its ridiculous joins, never mind the performance. Again, "don't use a thing for something it's not very good at" is not compelling. I get the point that, yeah, as a project grows often it turns into a case where SQL might be the best choice. I would just argue that you can re-frame a lot of use-case problems and make a document DB not just fit, but actually be really good choice over SQL. Where rapid iteration and flexibility are concerned especially, SQL just doesn't make sense. As a last comment it feels like he framed his argument around the idea that document DBs have zero relational capacity. You actually don't have to dump everything into a single document type. You can certainly store ref ids so long as it's not an attempt to overly flatten your model. Yet again, don't use it for what it's not meant for.

learning so much. thank you!!!

Great video thanks! One questions though. I had used Hasura in a tutorial and enjoyed it. However I did a coding bootcamp and used raw SQL queries, which I also enjoyed. However I haven't had to do a ton of different joins and difficult queries. You can use Hasura with SQL databases and I thought it just translated your queries to SQL and didn't realize that it created proprietary language on the database server. You also don't use a Hasura database so not sure what you were referring too there. You can use any database server as far as I know. So I am not understanding something here. I will have to go back and look at the database I used for that project but pretty sure the data looked like any other PostgreSQL database and Hasura was just the middleman.

I actually think MongoDB is far on the left of the spectrum and yes almost all applications data do have relations. But MongoDB is NOT about having no relations. People who complain about MongoDB either embeds all relational data into a single document(this is not even possible to due to single document size limit of 16MB) making queries impossible or normalize all the data just like relational databases, ending up with way worse (JOIN) performance. The super cool thing with MongoDB is that you can start out with a straight forward relational database like normalized design and can incrementally optimize according to the application needs. For example, in the case of Users and Comments, yes u would definitely store comments in separate documents. There is absolutely no reason to embed ALL the user's comments in a single user document. BUT if you have an API that fetches user data with the recent 10 comments and this API is getting a ton of requests, then based on these metrics you get an option to duplicate and embed the recent 10 documents. Now you just have to fetch the user document once with no JOINs, its just fetching it from the SSD and done. And you still want the comments stored in a separate collection, so that you can fetch older comments. Now this performance optimization leads to data duplication, and yes this can be quite challenging to manage, if you dont know what you are doing. But as I said, you can start designing your database in "normalized relational style" and evolve your schema according to how your performance bottlenecks. BTW I find designing relational database very straight forward. People seem to think its more difficult cuz you have to actually create the tables, set up the foreign key etc. Also for usual applications, you have way more reads than data mutations(create, update, delete). Think about a blog post, that shows the number of comments, three recent comments, and the writer details. While you write the blog post once, maybe update it a few times, hundreds or millions of users could read that blog post. Imagine you have a relational database for this. You would either have to do all these duplicated JOINs, which will definitely kill your database or you would have to add another cache layer, just to solve the database issue. Now with this cache layer, which is very common with relational databases, even with a relational database you have duplicate data and have to manage cache invalidation(which is somewhat similar to duplication management) So even though data duplication can be additional work for the developer, its super beneficial once your app scales. Also comparing PostgreSQL JSON column with MongoDB is really not fair. If you look at SQL, its "API" is optimized on reads and does all the data processing on every "Read", on the other hand MongoDB focuses on data mutation. You get to update the complex JSON data with a single update atomically. In fact I think SQL(relational) databases are more useful for data engineers/scientists/analysts. For them its not about getting the data fast. Unlike the application, the set of "read queries" are not fixed and can change a lot. Well, in this case, it would be more like columnar databases such as BigQuery and Redshift rather than traditional database. Im talking more about the SQL in this case.

@t3dotgg

Жыл бұрын

This all makes sense. Thankfully solutions like Vitess on MySQL can keep SQL read perf pretty nuts. I'd put you in the category of the DynamoDB folks fwiw

@sihoonkim1502

Жыл бұрын

@@t3dotgg Interesting, looks like it enables horizontal scaling with sharding to make it fast. I should take a look at that. I was always very skeptic with horizontal scaling with relational databases. I was skeptic, cuz one of the reason why RDBs' queries are fast, is because all the processing happens in a single machine. RDBs are very old technology, built for vertical scaling, which was more than enough at that time. However, if you look at how sharded cluster works, unlike replica sets, each shard is a separate machine and has unique set of data. Basically, the rows in a single table is stored in different machines(shards). All the data processing(JOINs, GROUP BY, ORDER BY etc) can no more be done in a single machine, leading to a ton of network request just to process a query. So even though it could improve writes(more primaries), I was skeptic with faster queries And as far as I know, another nightmare is that if you were following the convention to make primary keys with AUTO INCREMENT.., you would have to switch them all to something like uuid, cuz u have multiple machines that writes to ur database at the same time, meaning that unless the shards communicate each other to make sure that AUTO. INCREMENT is "incrementing" to a unique value, which degrades the benefit of horizontal scaling.

@fadelibrahim7663

Жыл бұрын

why bother dude, for me whenever someone repeat the famous misconception about mongo "not relational" it is end of discussion, they dont care, you will not convince them to invest a single unit of brain energy on the subject anymore. And for them it is like mongodb magically vanished and dynamodb is the only nosql left. Just like they convinced the beginners to learn mongo at first in the couple of years ago without telling them about the big bottlenec of connection pooling, now they will again make a mistake of recommending against it while it is the right time to use mongo since now it is optimized for serverless and every use case.

As a firebase/firestore user there are gooduse cases for the platform, some of them are not provided by supabase because supabase is not a 1 to 1 map to firebase but my Pain with the queries, relationships, even when they have an SDK that allow you to use it in your own server and write and API if the query you bid is complex and dynamic (like a sort in dynamic fields) your call fails in runtime because you have to buil the index to. and is a hard vendor lock. At the end if is for prototyping a quick idea is good but otherwise it will hurt you.

Your video saved my time. I mean a lot. Thank you very much. I subbed u from now.

I do agree that with mongoDB it is not as seamless as in sql to define relations between tables. However, I still use mongo, because it is very much possible and mongodb atlas comes with the ability to perform (potentially very complex) fuzzy search queries, which i need in almost every application I work on. Saves a lot of time not having to setup things like elasticSearch or using half-baked postgresql extensions

Used Fauna at a few startups. It was cool at first. All the easy stuff worked fine. Right when I started getting into more complex logic, it became a nightmare to work with.

@lutfiikbalmajid

Жыл бұрын

Hows it compare with SQL?

@anyadatzaklatszjutub

Жыл бұрын

@@lutfiikbalmajid same experience... plus it's just so annoying to take up mental space with learning it's syntax, that does not translate to anywhere else

@Khari99

Жыл бұрын

@@lutfiikbalmajid The syntax will break your brain lol. You're better off with SQL. I personally switched to Dgraph because I wanted to keep GraphQL but Theo hates that too lmao.

@Khari99

Жыл бұрын

@@anyadatzaklatszjutub Lol FQL gets insane. There's a lot of cool stuff you can do with it but for what it is, I'd rather split apart my queries in a better designed language

@t3dotgg

Жыл бұрын

@@Khari99 to be clear I really like GraphQL - I just like to use it for what it was built for. Not as an SQL replacement. I ranted about this on stream yesterday lol

Question: What is your favorite SQL db? Your go to for quick/smaller projects & The db you would pick for an enterprise.

@mileselam641

10 ай бұрын

Quick/smaller: SQLite Enterprise: Postgres

Your best video so far! When do we Get the Nestjs rant?

I like arangodb a lot. It's super underrated and worth checking out. Open source multi model db (graph too) and very dev friendly. Though expensive if you use their managed hosting - much cheaper to host it yourself on aws or gcp or something.

@Khari99

Жыл бұрын

Im slowly coming back around to ArangoDB

@azdy68

Жыл бұрын

NOOOOOOOO, pls noooo

@Khari99

Жыл бұрын

@@azdy68 😂😂😂

thank you very much, that was very informative.

Thanks for this chat! Didn't realize the bottlenecks of firebase

Hey Theo, just found your channel and I’m loving the videos. I actually got fired a few weeks ago for arguing with management about some database decisions they were making. I still regret getting fired, but this was a nice sanity check that my concerns were not completely invalid.

@MysterCannabis

Жыл бұрын

You probably got fired for the way you communicated your disagreement

@pastafarian7

Жыл бұрын

@@MysterCannabis lol. Probably some truth there. I was mad at them for a couple reasons, could have handled it better. Next time management wants to drive off a cliff I’ll just keep my mouth shut. CMA moving forward

Can you elaborate on the dynamo take ? When I’m in need of persistent storage in aws land I go to s3 or dynamo depending on the time it’s needed. Not sure of other parameters involved but I would love for anyone to add their thoughts

Hey Theo, loved the video! What database would you recommend for a social media network if starting from scratch using the latest database choices on the market today?

great content was very interesting to watch and think about, thanks!

Hey theo ! Im new to SQL and Im learning postgres. Do you know how or where I can practice more of SQL/Postgres

Ive used Postgre for like 14 years in big expensive mission enterprise apps, whiches last minimum 15 years in production, and they have really stand the test of time. Rock solid in every aspects. Btw, you can put AWS Aurora Postgre/MySql in the managed DB category,

firebase has an admin sdk for many languages which makes it really easy to work with if you have a service in python or go or whatever. also it can be very convenient for small projects, you just pay what you use (even supabase is 0 -> then 25$).

Interesting explanation about functionality subsets of DB, it is more clear now for me which solution to pick! Thank you for explanation! KV (small features) -> Document DB -> SQL (all features + standard = best)

So in the end, what’s the best solution for a frontender with basic knowledge of backend(express, postres etc), but not confident with building scalable systems. Is it supabase? It looks quite good, but it being in “beta” and not disclosing pricing for going over the “pro” amount concerns me

Theo, do you export these drawings into a repo? It would be great to go back and look at it as once piece.

@lasfito

Жыл бұрын

I was wondering the very same

What's your take on event databases (eventstore db) for example?

I'm getting frustrated with supabase client and looking at moving off to prisma ATM.. looking tricky given auth and schema limitations in prisma, and hackyness with RLS. Kinda at a 'burn it all down' phase and thinking of next-auth, prisma and just hitting postgres directly within next APIs. I've quickly found I'm not using the supabase client a whole heap on frontend anyway, and leaning heavily on next API routes.. speaks to your comments around APIs being good. The irony is that I'm very much a backend leaning dev and went for supabase (running locally) for batteries included to focus on learning frontend again... little did I know 😅

@t3dotgg

Жыл бұрын

I am not a fan of third party clients like firebase and Supabase fwiw - I mostly recommend Supabase due to it’s Postgres underneath. Prisma’s a great bet 🙏

How does security work on all of these Dbaas offerings? I have always had RDS running in a private subnet, which is only communicated with from services within that subnet (lambdas etc). Planetscale/supabase just let you connect with user/password from anywhere? Or am I missing something

@chastriq

Жыл бұрын

Not sure about these specific providers, but that's a very common model. Usually you can set up IP whitelisting, or often if you're integrating from a public cloud you can do e.g. network peering and keep the DB off the public internet.

Never knew about railway. Going to use on personal projects now.

REDIS stack has document database, graph and more. Fully persisted, replicated and highly available. Everything stays in memory so it’s super fast.

This answer is always PostgreSQL (if you don't know the problem). Its easier to migrate that data/tables into other DBs. You can recruit at your local college or universities and find someone that can extend postgresql into some other popular Db.

Thanks guys you saved me a good 50mins of my time just reading the comments first! I thought his take on documentDB’s was a bit biased 😂

"Almost EVERY data problem has relations of some form" - THIS! Knowing this can save a lot of pain in projects.

the standards in sql is really good what sql needs is to become more serverless for resource but keeping all the standards.

I learnt the hard way with Firebase... you cannot leave that platform unless you decide to start over

I really want to use supabase but I'm worried it will get acquired and then so man things about it will change, that's what happened to firebase and with their focus on how profitable they are it seems like that's the goal anyway.

Old is gold, I love MySQL.. and now comes SurrealDB, such a dream

Really useful video!!

Thanks. Rn I’m on amplify, literally just serving my static app. In the process of moving it to either Django or rails, but want to avoid the rtc of elastic beanstalk or ec2. Didn’t realize I could just store my actual db off of AWS.

Thanks for sharing!

So what would you recommend when you start your project? Supabase?

is sql still better for something like a chat app? or something with a lot of data being exchanged? I know discord uses something like casandraDB for example Also You said at one point that the 10gb that planetscale gives you is a lot of data since you shouldn't be storing images or data intensive files like that in a database anyway, but what about text based content such as chat messages? that can add up right? I made a random calculation: if an average user sends 1500 characters a month and a character is 4 bytes, then it would take 1,790 MAU to send 1gb. after 10 months of back storage you will have reached your limit and then it will cost 0.14 cents per user per month for storage on planetscale which is pretty low i guess to be fair. But is it the optimal use-case? I am a beginner to databases I would say

Which one you will prefer for making rest api with PostgreSQL. Golang or Node js ?

I'd love to see a video outlining server options if you don't have that already!

Would be nice you will make a review of Redis stack. You have a very good point of view.