Transformers are a type of machine learning model used primarily for natural language processing (NLP) tasks. They have revolutionized the field of NLP due to their ability to handle long-range dependencies and capture complex linguistic patterns. Here are key points about transformers: 1. **Attention Mechanism**: Transformers use an attention mechanism that allows them to weigh the importance of different words or tokens in a sequence when processing input data. This mechanism enables the model to focus on relevant information while ignoring irrelevant or redundant parts. 2. **Self-Attention**: In a transformer model, self-attention refers to the process of computing attention scores between all pairs of words or tokens in an input sequence. This mechanism allows the model to capture dependencies between words regardless of their positions in the sequence. 3. **Multi-Head Attention**: Transformers often employ multi-head attention, where multiple attention heads operate in parallel. Each attention head learns different aspects of the input data, enhancing the model's ability to extract meaningful information. 4. **Encoder-Decoder Architecture**: Transformers typically consist of an encoder-decoder architecture. The encoder processes the input sequence, while the decoder generates the output sequence. This architecture is commonly used in tasks like machine translation and text generation. 5. **Positional Encoding**: Since transformers do not inherently understand the order of tokens in a sequence like recurrent neural networks (RNNs), they use positional encoding to provide information about token positions. This allows the model to consider sequence order during processing. 6. **Transformer Blocks**: A transformer model is composed of multiple transformer blocks stacked together. Each block contains layers such as self-attention layers, feedforward layers, and normalization layers. The repetition of these blocks enables the model to learn hierarchical representations of the input data. 7. **BERT and GPT**: Two popular transformer-based models are BERT (Bidirectional Encoder Representations from Transformers) and GPT (Generative Pretrained Transformer). BERT is designed for tasks like sentiment analysis and question answering, while GPT focuses on generating human-like text. Transformers have significantly advanced the capabilities of NLP models, leading to breakthroughs in areas such as language translation, text summarization, sentiment analysis, and dialogue systems.

@user-jl5gj4mv1z

11 күн бұрын

you put great effort on writting this

In our graduation years we used to learn about something called codec, as in coder-decoder (something like modem for modulation-demodulation or balun for balanced-unbalanced in the domain of communication technology. So as I can understand from the video is that the transformers are just a fancy and advanced name for a codec, which functions at much bigger capitalistic scale.

Thanks you for your enthusiasm and explanation , by far the best

Hi Martin from the Homebrew Challenge! ML and beer clearly go hand in hand!

I really love your videos as they are really friendly to understand. Really graceful for the high quality of the synthesis of key messages on AI/ML/DL. I am a medical doctor and biomedical researcher. I can see the great potential of using the different technics to further develop a bunch of areas, for example: economic evaluations based on modeling (using a combination of approaches in the sensitivity analysis to find out the internal consistence of the predictions…to gain internal validity as a cornerstone to have external validity). So, looking forward to learn more through your channel. Thank you, again for sharing good quality knowledge. L.

@ms.barrio4402

Жыл бұрын

Congratulations to all the team work!, I will keep learning more. Thank you all, Leslie.

What a great way to introduce the topic. First 4 seconds made me laugh out loud. Well done (and the rest of the video as well)

Banana joke GPT-4: Sure, here's a banana joke for you: Why did the banana go to the doctor? Because it wasn't peeling very well!

@evaar440

Жыл бұрын

Good transformer 🤣

like to see the energy which you put on to it, Thanks for this.

Dr. Ashish Vaswani is a pioneer and nobody is talking about him. He is a scientist from Google Brain and the first author of the paper that introduced TANSFORMERS, and that is the backbone of all other recent models.

@user-uv2sy5je4z

8 ай бұрын

Agreed

@AK-ex5md

2 ай бұрын

He should be documenting his work like our guy, and make interesting vids. Hope it happens.

I have been more then blown away by the unfathomable exponential growth from just increasing transformers an a few weights lol

Are encoders and decoders both RNN? Plz clear my doubt.

Need Explanation for GRU , BERT , LSTM

Thank your video,your video really easy understand

Thanks for this video - a simple and concise introduction to transformers. Do large language models really possess reasoning capabilities? Or, the way they operate makes it seem so.

Thank you!

Perhaps the AI made the banana joke as a subtle way to tell us humans that we are a cruel species that mash anything we come across. The AI finds it funny because the banana would rather cross the road and take on the high likelihood of being mashed violently by a vehicle to avoid the certain mashing by humans. Perhaps the AI identified with the banana 🤔

@st0a

Жыл бұрын

Next level empathy: thinking about a banana's perception of reality 🧠

@drewsteinman1898

Жыл бұрын

Q

@zainkhalid5393

Жыл бұрын

You guys are overthinking it. 😁

@gohardorgohome6693

Жыл бұрын

that's how I interpreted it too - like yeah, the AI knows the banana doesn't want to be mashed by a car, neither do I

@l4l01234

Жыл бұрын

No, you’re definitely overthinking it. The AI doesn’t think anything because it is incapable of such context like “we are a cruel species that mash anything we come across”. Unless you specifically input that in the prompt, it has no mechanism to even conceive of the phrase.

Great video! I'm stumped on how you made this. Did you really write backwards? Can you reveal your magic trick?

@JoshWalshMusic

Жыл бұрын

You write it naturally and then flip the video when editing.

@AK-ex5md

2 ай бұрын

Exactly what's gng on in my mind lmao

This was greate!!!

How does the transformer take something of variable length (like a sentence) and shove it into a neural network (which requires a fixed number of inputs)?

@anushka.narsima

11 ай бұрын

Generic NNs take only fixed inputs but this is one of the specialities of these types of models! RNNs (the older model used for NLP) were created back in the 80s addressing mainly this issue, along with memory being important for sequences. LSTMs n now transformers came in to solve the issues with RNNs

This is the pinnacle performance of training.

Actually, I'm interested in the hand writing presentation style. How is it made ?

is the fact that he is able to write text mirrored incredible or is there a simple trick here?

@IBMTechnology

Жыл бұрын

There is a trick. Hint: he's not left handed.

@vaibhavthalanki6317

Жыл бұрын

its flipped and rotated, done through editing

@leihejun844

Жыл бұрын

@@IBMTechnology yeah, I though he can't be left handed.

@leihejun844

Жыл бұрын

@@vaibhavthalanki6317 it's not a glass, it's a mirror I think.

@somehhakarima5408

Жыл бұрын

@@IBMTechnology thought he was left handed

Sir Will u give me a research topic in transformer

how do i use transformers on a new pair of language?

The Transformer technology is the reason why you see AI everywhere.

How do you get loads of loss on on a neural network in given ways for analytics

Hi, what does an autoregressive language model mean?

can we use transformers over spacy for NER?

You are mirror writing, Great skill🤩

I think he write on the glass normally and the camera got it backword so they montage it to be flipped so the written words can be shown notmally.

how can text algorithm (transformer) work in image domain like vision transformer over CNN

@ChocolateMilkCultLeader

2 жыл бұрын

Transformers are being used in many ways. For example you could take a bunch of vectors (representing image features extracted from Convolutions) and feed them into Transformers to decode as text. This gives you a lot of power combining the NLP and Computer Vision Domain

@strongsyedaa7378

Жыл бұрын

@@ChocolateMilkCultLeader Generic features or specific?

@ChocolateMilkCultLeader

Жыл бұрын

@@strongsyedaa7378 what do you mean?

Does this mean they can fix my adhd? I don't quite know why but all this transformer tech helps me understand my own glitched mind better

Hey its the guy from the beer channel...

Lol, i find the banane joke funny :)

How are you able to write a mirror image of the words so effortlessly? :O

So is it like ... a layered, parallelized autoencoder?

Aged like a wine!

“Before too long, they might even be able to come up with jokes that are actually funny.” 2 years later, here’s the banana joke ChatGPT 4 (already 1y old) came up with for me. > Why did the banana go to the doctor?> Because it wasn't peeling well! I think we can call that a win.

Can we use this method to detect outliers in time series data

@TheShawMustGoOn

2 жыл бұрын

While you can use Transformers for Time Series, I'm not sure why you'd want some network architecture to look for outliers instead of regularizing it and let the network learn to ignore those during optimization.

@coffle1

Жыл бұрын

Transformers are a bit overkill for anomaly detection. A lot of time more traditional methods might perform better faster (especially if the resources for training the models are constrained like not having dedicated chips or an insufficient amount of training data)

Why did the attention mechanism NOT cross the road? Because it was paralyzed!😜😁 BTW did I hear that part correctly near the end of the video?

Things are judged by their appearance. And this video looks way way better than it actually is. That explains the views.

This was prophetic. I wonder whether at that time you realized that Transformer would revolutionize the world.

Where can I summon autobot?

I just have to say it TRANSFORMERS MORE THEN MEETS THE EYES!

In reference to the summary of an article example, How does that work? How does the program know to summarize the article and not continue it? Also, how do you go from language processing to playing chess or other games or functions?

@damianliew5243

Жыл бұрын

I'm not a machine learning expert so I can't verify the validity of this answer, but from my POV I think these questions about "how the program... instead of..." is generally dependent on 1. The actual architecture of the model (in this case, a transformer) 2. The input data it's based upon (text vs maybe piece type and board position labels for a chessboard) 3. The output data it's trying to predict (predict a summary text vs predict the next words in an article) Because such supervised/semi-supervised learning models learn off labelled data, (to a certain extent, for semi-supervised learning), all the model is really doing is mapping an input to an output. Think of it like a maths graph (which is actually exactly what it is); given a dataset with many points, you'd want to find a "best fit" line that models the rough trend accurately without over or underfitting. Machine models do this but on many axes (due to the use of vectors, some with just an insane number of dimensions). Of course there are many other things like hyperparameters, activation functions, loss functions, and nuanced variables to each model architectures, but hopefully this gives you a good understanding of ML in general.

@xerxel69

Жыл бұрын

A summary is a continuation of the text in that case. Consider a webpage on the internet which has an article and then at the bottom of the page it says, "here is a summary of the key points we learned above" and it goes on to summarise. This is an example of the kind of content the AI is trained on. So as long as you do some Prompt Engineering then you can ask your question in such a way that the answer comes from completing the text! It's like magic! 🙂

@andrewnorris5415

Жыл бұрын

@@xerxel69 Yeah, articles do often contain a summary section at the end. Or parts of an essay say, "To summarise so far". Not sure if it can learn this totally unsupervised. Mu guess is summaries are a popular feature - so they will train it specifically to look for them and learn from them i na focused way. Not sure though.

Transformers: More than meets the eye...

IBM: Next-Level Tech explained. Chat: How does he write backwards on that invisible board?

@IBMTechnology

11 ай бұрын

See ibm.biz/write-backwards

Now it can indeed write funny banana jokes!!

this dude can write reversely. so awsome

@andrewnorris5415

Жыл бұрын

ha. it looks the right way around to him. The final image is inverted in the video we see. Fun trick.

I do searches for Transferormer in Machine learning.and in my mind same those transformer there and video starts with the same.

Do I get this right, that a transformer is a special case of a state machine, which is designed to learn on, or update it's weights on demand, and is still general enough to cover most data?. Wouldn't an FPGA be optimal to implement such a state machine in flip flop, so that you can generate with 100mhz.

@nestorlopez7071

Жыл бұрын

It really all boils down to performing matrix multiplications. GPUs are best at that. An FPGA can be a GPU if it wants to (:

Latest update on banana humor of AI Why did the banana go to the doctor? Because it wasn't peeling well! - GPT 3.5 11th January 2024, 23:06 IST

the joke was just too deep for your puny mind to get

Is he writing on reverse so we can see it correctly?

Correct me if I'm wrong but it seems that translating a document would require a human doing Quality Control right before publishing. Transformers are impressive in how close they come to mimicking humans but they seem to be The Great Pretenders. Now, how does that QC step get implemented in real time?

Are you guys open to Guest Speakers

Optimus Prime

the banana … skidded …

@normacenva

2 жыл бұрын

it wanted to split

Instead of the content I started thinking how this guy writing in opposite direction 😭😂😂 Is this some AI trick or fr?!

@IBMTechnology

8 ай бұрын

See ibm.biz/write-backwards

They've got better jokes now. 😂

You do realize the joke about the chicken crossing the road is a suicide joke right? He wanted to get to the other side?

And with this simple idea the civilization ends. No, kidding, the AI will be so smart, it will leave us alone as we will be like bugs to it.

I came here to understand how on earth he writes backwards or what camera trickery I am obviously missing, LOL.

@IBMTechnology

11 ай бұрын

See ibm.biz/write-backwards

@MikeHowles

11 ай бұрын

@@IBMTechnology LOL thanks!!! I suppose it shouldn't surprise me there is a video about that. Very cool and elegant technique.

But how does the human write backwards?

@IBMTechnology

11 ай бұрын

See ibm.biz/write-backwards

I don't like people ripping Me off, whether IBM or Google.

Indian SME's might be able to create this and be a unicorn. Easily.

I didnt get it

I dont think the joke was that bad. Picture meatwad from AquaTeen Hunger Force, but very pale beige.

The banana joke is an instance of an “anti-joke”… just like the chicken joke.

it doesn't "come up" with a thing, it regurgitates what it's learned. It's nothing but a copy machine and being made out to be much more than it really is by all the AI hype machine artists.

not very descriptive.. it is for those who already are studying deeply about sequencing, encoder decoder etc.

@amudhanbakthavathsalu5308

3 ай бұрын

may be i am not smart enough to understand..

Why do they always translate English sentence to French in every video that explains Transformers :D

@alexandrav1020

8 ай бұрын

🤣😂

ok but how about a more detailed explanation ?

Well, jokes are hard. Kids take several years to learn how to be funny.

@gohardorgohome6693

Жыл бұрын

KIDS ARE OBSOLETE, AI IS BETTER

I can't concentrate I can't understand how he manages to write backwards

are you writing backwards in real time? because if so..... 🤯

@IBMTechnology

7 ай бұрын

See ibm.biz/write-backwards

@jayseph9121

7 ай бұрын

@@IBMTechnology one of the few times in my life I wish to be lied to 😂

wait, does this guy write backwards?

@IBMTechnology

7 ай бұрын

See ibm.biz/write-backwards

The joke would’ve worked if it was a potato. Pretty close though.

ask it why did the potato cross the road

Did I miss something? This didn't seem to give any clue as to how transformers actually work.

Your skills in writing backwards were really distracting.

@IBMTechnology

10 ай бұрын

See ibm.biz/write-backwards for how it's done

You didn't really explain anything.

didn't find it helpful to conceptually understand transformers

To reach the other bunch. chat gpt3.5