Your explainations are the best, right to the point, well illustrated, real world examples. Thank you.

Thank you so much for the effort. appreciate it Loay from Egypt

Please also make a few videos on Low Level Design and CDNs.

thanks alot!I was struggling to anwer these questions in interview .I must say it's a easy way to understand these concepts.

Thanks a lot for these wonderful videos. I've become a better Engineer with the help of your videos. Keep up the awesome work 👏

Wow terrific to the point with clear and clean message.

Some information on how the backend queries will manage limit and offset would have been helpful to tie in the solution end to end.

@anothermouth7077

2 жыл бұрын

Same, at moment I'm making query string and based on given filters I conditionally appending where clauses. Could you let me know your approach?

@johnrodger5467

2 жыл бұрын

Can validate the request param in the api using a validation library, to only accept values in the allowed range. If the request is outside the range you can either return a response to the client telling them it's an invalid request, or send the request to the DB but with the max range on the query. Something like that. You could also write your own DB query wrappers/facades that have the limit already set there, and only use those in the application code rather than using the DB client directly.

You did a great job with this video, thank you

This video was very helpful and got more info . Thanks

Expecting implementations of concepts as well in all videos

This is nice topic. Sometime I feel that pure REST approach is not suitable for business need. It mainly revolve around single entity but business application has complex need like to filter. Let's say for example Employee Belong to specific company and organization and there are multiple attribute associated with that and we have to pass multiple filter attribute. I feel that sometime pragmatic approach will make more sense and expose end-point like /search / filters etc will make more sense.

This explanation is so valuable thank you for that , also I would like to suggest if possible to do a video demo with a limited paging server response for example : API like Zoho, or deluge code based API's , how to use the explained methods on a study case like this

Great and clear explanation 👏👏 Thanks a lot

Thanks for posting this one

Dope quality Explanation.

Love this video. Can you please create a video dedicated to API Security? How do you secure an API?

This is really wonderful explanation. I know its not in scope of this series but out of curiosity would you mind throwing some light on how backing code for pagination can be implemented? Will it use some sort of sorted cache from where it returns results based on offset and max limit or is there a better way to get this done?

In backend for offset and limit pagination we can use db limit and offset parameter while creating a query to fetch that many rows and this quite inefficient in certain scenarios. Keyset pagination which is quite efficient but can’t directly query a particular page. Either go forward or backward in rows with a particular limit.

Thank you so much 🔥

by your presentation we easily got the concepts, thanks to your editor😅😅

Wonderful explanation

Great. Wonderful explanation.......

mam you explain awesome. Thanks

just one word Awesome.

Amazing explanation

Thanks !!

greatly explained

Found 2 channels useful so far .. Ur channel and another.. kudos

@aatiqaghazali7850

2 ай бұрын

Plz mention the other one

@rahulsen002

2 ай бұрын

@@aatiqaghazali7850 Rahul nath

Nice. Any video on cursor vs offset based pagination

Very helpful video

Explaination is excellent please provide practical video if possible

One more way to reduce payload is sending it back as a stream, can you share use case for the same? Could it be video/audio streaming?

amazing video thank you!

@sudocode

2 жыл бұрын

My pleasure!

Thanks

Clear and crisp explanation but I've one question. Do we need to hit db for every request, Is it a good idea ? If not then what are the other ways we fetch x amount to data in once(certain threshold) and serve in 20-20 chunks ?

@m13m

9 ай бұрын

Definitely we could fetch a threshold number of rows and warumup your cache and grab any new data only through but offset-limit pagination is quite inefficient imagine there is no cache is such case every single query of limit and offset you are hitting db grabbing all the rows and then applying limit and offset on them. Also if you are inserting new items as well there can be cases where you could have duplicate items among your list with offset-page pagination Other technique that could be useful keyset pagination where you can’t actually pass offset but can behaviour of pagination with previous and next page with a certain limit just that can’t jump to a particular page but definitely traverse them limit you have set at a time.

you're super underrated

Can you please cover REST API security also?

Super :)

Thank u

nice

please cover sorting also

Hi,your explanation is good. I have issue with pagination. I using one azure api to get all items. In azure portal, 500 items are there but I am getting 200 items while querying api. I applied limit and offset as well but not working. Please help.

Hi Yogita In case of pagination are all responses for all pages cached at server side at once? And then they are dispatched to client page by page as per the request? If they are cached on Server side when will the cache cleared? Thank you for all your videos 🙏

@_tachyons

2 жыл бұрын

This is unlikely, it is mostly implemented by directly executing query on database, caching layer is added to improve performance, but caching entire thing in the cache defeats the purpose of caching

Hi teacher thanks❤

Mam'm how could I understand my frontend before making backend server. If I created a registration page and login page are on different pages then it's important to make the same thing on the backed side. So, I will have to login and registration page (register, update, compare password pages) are on the different pages or same page in backed side.

Designing the APIs are fine, but how to implement such things? What is the right approach. Let's assume that I am building an ecommerce site and where from the database I want top 20 filtered items, how to do that in code? Thanks for the awesome content. PS: All theory and no code, makes jhony a dull boy 😉

Thank you for your video but there is some limitation in LIMIT and OFFSET. it will be huge cost involved for RU/s

Is there any specific form of response code sent by the server when the limit set by the client in the query param is larger than the max limit set in the server. I was thinking if it will send either of 400 ( Bad request ) or 413 ( payload too large ), but again you get a response with valid set of items in the server. So it seems to use some specific 2XX code itself. Could you please suggest or throw some pointer on what that might be?

@UtkarshSinghPaliwal

2 жыл бұрын

Probably a 422 would do.

@AjaSiva

2 жыл бұрын

You can throw any status code😅

Does api works to show products which are on low prices due to price error listed on amazon or flipkart?

On the server in every request we will be fetching all the data(from database) then paginating it right, assuming no caching is done??

@UtkarshSinghPaliwal

2 жыл бұрын

No. We can write the DB query which take the limit and offset numbers as arguments and fetch only the required data from the Database.

@shailjakantupadhyay5183

2 жыл бұрын

We never do that, consider server contains trillion data in table if you fetch all the data then it will go down. Read @Utkarsh comment.

Hi I have a quick question. So if i have 50 orders fetched from api and are available in dom. Now i want to implement search the order by id. In this case shall I do a internal filter in the dataset or shall I fetched filtered data from api? What would be the best approach to do it?

@AjaSiva

2 жыл бұрын

Fetch it from the api. You may not know what ids are there in the current data you fetched. So it can give no result too if searched by id. Also id's are indexed in databases. So fetch call is not that costly.

hey i was asked a question where i told about offset for pagination, but interviewer told the data is added on api all the time, so when i call the next page , i might get the same data in the next offset as well coz new data was appended by time i clicked on next page, so how to handle such cases???

@II-ii2um

Жыл бұрын

simple function that checks if the previous data shown != the new data being shown this time? Source: Copy-Paste Buffers in Systems use this all the time to prevent the same shit from being copied and pasted over and over again... Have you figured out a better answer for this yet?

💕

Are there any issues if we use post instead of get in this and send all the required data in requestBody rather than query params?

@_tachyons

2 жыл бұрын

That will be violation of rest conventions, but if the params is too big, then you may have to use POST

@UtkarshSinghPaliwal

2 жыл бұрын

Bad Practice

can we send a json data in filter if filtering data is complex? I currently POST and json body for it currently

@UtkarshSinghPaliwal

2 жыл бұрын

Generally, there sbould not be a payload associated with a GET request. For a POST request, filtration might not be something that is required in the usual scenarios.

Critical: what if you request, say first 10 (sorted by latest) orders and then 1 new order received. Now if you request next page then 1 order (1st of second page) will be repeated because thats 11th order now (not 10th anymore). How can we fix that.

@_tachyons

2 жыл бұрын

This is not necessarily a technology issue, this could be an expected behavior as well. But if you want more predictable behavior use cursor based pagination

Mam have you created video on implementation?

It's clear ..but there is more link to fetch more data in some pagination APIs

how does this compare to graphQL?

How do I get to know till what number should I make the pagination buttons? Because if I don't specify it then the next button will return an empty page sometime in other words the grt request will return an empty array

@ivocosta7281

2 жыл бұрын

When you receive paginated data, the resultset should also set total records , what is the pagesize. Once you have these two data point, you know which will be the last page

Nic mam

Using an OFFSET for huge data will still be inefficient for MILLIONS of rows as the database still needs to skip all the rows specified by offset. I expected you to give a solution to this problem in this video as the title specified "millions".

What is there is no pagination.. the data is loaded based on the user scrolling the screen..

actually i learned these terms pagination, limit, offset, filters while learning DBMS and i think they're more related to that that rest Api

You did not discuss this scenario: if client request for 100 items and server max-limit is 20, how client will be able to request items 20 to 39 in the next request? How client will set its offset ? I am assuming client logic should have offset=lastoffset+numberOfItemsReceived

@sudocode

Жыл бұрын

That’s implied my friend.

U looks like NAIROBI of money heist 😀

will this work for mongodb?

😊

I have few questions on pagination. Can you explain about the page count. How to determine how many number of pages for each query. Say the query returned 90 rows, and if the results per page is 10. I need to show 9 pages (1 2 3 4 .. 9) 2. See this scenario. I have a query which returns 100 records and on top of that I need to apply some business logic and filter some data, so after filter I got 70 records. Now how pagination work when user click on page 2. (Point to be noted there might be thousands os records returned at a time for a query).

@UtkarshSinghPaliwal

2 жыл бұрын

Pagination should be applied on the final records being exposed to the clients. In the case where only 70 records are accessible by the client, pagination should create only 7 pages in totality. Moreover, in these cases, the client should never know that there are actually 100 records on top of which a biz. logic filters them down to 70. Otherwise, this would be an abstraction leak.

@sumanthkeith1

2 жыл бұрын

@@UtkarshSinghPaliwal yup I agree. But my question is when I click on page two, should query the db get 100 and apply business logic again to filter and show the contents of next set of records?? If we do this there will be performance impact. One solution is to store the filtered logic and Show the 2 nd page and so on. Should we use Redis or java cache us my next question.

@UtkarshSinghPaliwal

2 жыл бұрын

@@sumanthkeith1 That's a pretty decent problem statement. We need to paginate and we don't have the records to send back without applying some business logic. In this case I'd say to choose either of two things, may be not use pagination at all or cache the indexes of the records and some meta if required. I'm presuming that querying data more than once is very expensive so that logic is not an option. Now with both the cases suggested, two problems arise, one with both the paths. If no pagination used, data may not even fit in a single response in which case you may provide a download link instead. In the other case of caching, you'll have to deal with all the caching related issues like data consistency etc. So you'll have to choose either of the two evils depending on your particular costs and usage.

CDN can you share

Do you offer a service to integrate an API from my distributor to my website for orders and dropshipping

Is using OFFSET a good idea to skip resources? I have heard that using OFFSET in SQL can reduce the performance of DB as if there are a large no. of records, the DB has to pull the tuples from the disk but not include them in the RESULT SET

in this video u r looking lean

Didn't work.. getting complete result

Nice explanation but only for a very primitive audience to understand. It does not contain any technical details on how it works or implemented . If you could add more details on server side services to map to your URI examples that would be helpful.

@sudocode

Жыл бұрын

Sure, you can create a video on that and I will be happy to endorse it 😅