can you please do "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks"

quite detailed and clear explanation. And I like the brief idea of the paper introduced in the beginning of the video~

Using marginalized probability is a good idea. Brilliant!

This content here, is gold!

Excellent video man! Thank you so much!

I'd imagine REALM is pronounced like the work ‘realm’ (sounds like ‘relm’), given it seems to be a pun on the definition of realm, ‘a field or domain of activity or interest.’

Love it :)

This seems on the surface similar to the idea Yannic had in the GPT-3 ML Street talk video.

For "Salient span masking", "BERT-wwm" "SpanBERT" "ERNIE" "RoBERTa" all adopted mask-based modifications in improving BERT.

Yannic - how do we get to know which research paper(RP) is more relevant. Only a miniscule fraction of RPs published make an impact. In earlier comments I had requested if you could somehow make a mapping of RP video to citations or anything similar, it would be great. Otherwise it is difficult to select videos from so many :).

@herp_derpingson

3 жыл бұрын

Checkout arxiv sanity preserver made by Karpathy. Its intended to serve this purpose. www.arxiv-sanity.com/

@MrAlextorex

3 жыл бұрын

use www.arxiv-sanity.com/ .The paper basicly must be accepted to conferences and must have many citations

Could you do a video on ORQA?

Thank you for sharing your clear paper explanation! even thought your clear explanation, I have one thing not to be clear to me... 34:17 I can't understand. I think the equation means that more retriever put relevant answer, more the r(z) value is going to be high, and that means gradient too is going up! But In my understanding, it is natural that more models are learned, more the gradient have to be small! So I'm so confused right now. Is what I know wrong? help me please..

Great explainer! One thing that I’m not sure about. How are the 3 models connected, is this end to end? How does the retrieval work in that case?

@YannicKilcher

3 жыл бұрын

Yes, this is end to end.

@bdennyw1

3 жыл бұрын

@@YannicKilcher Thanks, I'll have to dig into the paper. The retrieval step doesn't seem like it's differentiable, so there is something I'm missing.

@user-nc5cq9yu2c

2 жыл бұрын

@@bdennyw1 I don't see how the retrieval step is differentiable, either😂

Can you do one with ColBERT as well?

nice

Hey Yannic! Great video! I really appreciate the work you are doing to make research more accessible to everyone! By the way, I don't see a 1080p (HD) option for this video. Is it possible to watch this video in 1080p? Thank you! :)

@YannicKilcher

3 жыл бұрын

Thanks for telling me, I didn't even see that. No idea why this happens

@tarunpaparaju5382

3 жыл бұрын

@@YannicKilcher Thanks for your reply! Keep making awesome videos, it really helps me a lot :)

Yannic, instead of using a neural retriever, wouldn't a probabilistic retrieval framework, such as BM25 give a similar result?

@MrAlextorex

3 жыл бұрын

BM25 is too simplistic. BERT also has some understanding of meaning of words in the context of document

@abhilashnandy

3 жыл бұрын

@@MrAlextorex I have seen BM 25 augmented with T5 generated questions perform really well on MSMARCO

@MrAlextorex

3 жыл бұрын

@@abhilashnandy MSMARCO is based on questions sampled from bing searches which are quite factual and simplistic

60 degrees or 300 degrees?

@sandeepunnikrishnan8806

11 ай бұрын

How would it be 300?

what does marginalizing mean in this context?

@YannicKilcher

3 жыл бұрын

I'm not sure what context you mean, could you clarify?

It is pronounced "relm". ah when knowing only english is actually helpful ;)