This is the second of 5 videos where Phil examines the ins, outs, and struggles of AI! Join us every Tuesday in April for more. And watch the first one, on the difficult task of drawing hands, here: kzread.info/dash/bejne/ZGitzLSbcsTOY9o.html

@justinmanzo3945

Жыл бұрын

Love you vox for these great videos, keep making more!

@user-os4yd5tj2k

Жыл бұрын

You guys should not pin this as to allow more time for comments by others to have screen time in a more favorable way.

@Octobris

Жыл бұрын

He can examine my ins and outs

@evansusukumaran5123

Жыл бұрын

Is there Malayalam

I imagine these models may actually help smaller languages to thrive. Imagine training them only enough to translate whole books with reasonable accuracy, keeping the native speakers with content.

@AnimilesYT

Жыл бұрын

I remember watching a video recently about a language which is only spoken. There's currently no way to write the language, so it'll be hard to train a model on those languages. Other than that I do agree with you. Even if it may not help all smaller languages to thrive, it may help some or a lot of them :)

@MangoMotors

Жыл бұрын

Smaller languages might not have enough data points considering how surprisingly inaccurate machine translation still is with large languages with a ton of data set.

@MateusSFigueiredo

Жыл бұрын

It's an interesting idea but also a dangerous one. The AI is not a native speaker and might damage the language. This reminds me of the non-Scot speaker that wrote a lot of Scot-wikipedia based on what they THOUGHT Scottish was, and did a lot of damage there.

@nickc3657

Жыл бұрын

@@MateusSFigueiredo I was thinking the same thing. The language is called Scots btw.

@jqzhu8348

Жыл бұрын

exactly. this was my immediate reaction. we could use LLMs to sustain smaller languages.

This is not just a problem for the AI. As a language nerd, I have faced this problem with low-resource languages before (namely Hawaiian, where I was born and raised). As much as the linguistic community likes to talk about saving endangered languages, I feel like too few people actually do the first, most important work: Write the language down! Write and write and write! I'm glad to see the people here are taking the work with AI seriously, and I look forward to the day chatgpt can help me learn some of the fascinating low-resource languages of the world.

@yucol5661

Жыл бұрын

@@TylerMarkRichardson probably easier than recording. You probably could pay people for recordings of their phone calls tough. Or get them to donate them at the end of the call. Shady but easy

@Eggmancan

Жыл бұрын

@@TylerMarkRichardson I'm not sure what you mean. Are you talking about historical Hawaiian? Because, they didn't have writing. But Hawaiian still exists, and if people intend to save it, it needs to be written down. And I'm not sure what you mean by "a modified version" since languages literally change every single day, so of course any Hawaiian written down now will be different from the past. That doesn't make it any less valuable to write down.

@rogerwilco2

Жыл бұрын

And also record the language in other ways, the sound, the body language, etc. Language is much more than gets captured in written text.

@youtubedeletedmyaccountlma2263

Жыл бұрын

I’m writing hokkien, henghua down

@Ellary_Rosewood

Жыл бұрын

I've been using it to further my studies of Georgian, which is unfortunately difficult to find decent recourses for. It's been quite helpful so far. I've tested some of the responses with my Georgian friend and he said it was pretty accurate. I look forward to seeing it continue to improve!

As a fellow Jamaican software engineer, I am EXTREMELY proud of Ruth-Ann! 👨🏽‍💻🇯🇲

@clowneryandbuffonery

Жыл бұрын

🇯🇲❤❤❤ it’s awesome that Patois is getting this love

@deliocache2528

Жыл бұрын

Thats awesome

As a native Filipino (tagalog more specifically), I find it interesting because Filipinos sometimes mix both english and filipino when speaking so I can somewhat assume that some blogs or informal news sites could be used as part of the sample and it could create problems for GPT-3/4 when understanding and decyphering which paragraphs/sentences are pure Filipino or "TagLish" (basically English and Filipino in the same sentence) On a side note: Does anyone have a link to the data set he used in the video because I am now curious with my own language and how it's used in the Dataset

@guillerhonora717

Жыл бұрын

They had yet to replicate some of our 195 languages

@fanofgaming8403

Жыл бұрын

@@guillerhonora717 exactly, which is why I'm interested on how they will translate those types of sentences for now.

@WinTeRzZ47

Жыл бұрын

Imagine Malaysia and Singapore

@mikicerise6250

Жыл бұрын

I don't think it would be a problem for GPT. The problem is only a lack of readily available data to feed it of written Filipino speech. If enough data were available, I'm sure GPT could master Tagalog while also understanding when to intersperse English or how to code-switch without getting confused. That is what GPT is best at. I'm surprised at how much of the common crawl is English, and even other big languages make up a small percentage.

@Tustin2121

Жыл бұрын

My grandmother immigrated from Italy in the 40’s. This was before Italy pushed to unify the language across the country, so she spoke a regional dialect that no one in Italy speaks anymore. In addition, when she arrived, she had to learn English solely by immersion learning. So now when she speaks her Italian dialect, about 40% of it is just English words and phrases dropped into it. So there’s a mix of still-used Italian words, Italian words from a dead dialect, ‘Italianified’ English words (basically adding an -o or -a suffix), and just plain English words and phrases. There’s like a total of 20 people who maybe understand and/or speak this weird conglomeration of languages developed due to circumstance. There’s basically no way any AI would ever understand her.

I'm Catalan and just yesterday I was testing Chat GPT's Catalan capabilities, which made me wonder how well trained it had been. Catalan has a relatively strong online presence, as a lot of work has been put into this by the local government, so I thought it was trained on a lot more than just a few pages... Now I know

@monopicator9314

11 күн бұрын

Same thing for me! When I heard it was trained on 140 pages of data I was like: "Això sí que no m'ho esperava"

That woman has amazing patience to go through those data sets. I wonder how they are addressing the regional differences just within England or the UK. in the North of England ‘pants’ are your jeans or joggers, whereas down south your ‘pants’ are your undergarments!

@andrewvirtue5048

Жыл бұрын

Dialect. You are wondering how they handle Dialect.

@jamesgreenldn

Жыл бұрын

I thought trousers are called pants in the U.S? What do you call pants up North?

@andrewvirtue5048

Жыл бұрын

@@jamesgreenldn pants are pants in the US. From the north to the south I've never heard someone in the US say trousers for non-comedic/sarcastic purposes.

@jamesgreenldn

Жыл бұрын

@@andrewvirtue5048 yes I know thanks I was asking the O.P

@joepopplewell680

Жыл бұрын

No one in the north or south call them that. It's trousers as a general term everywhere in the UK with slight regional variations such as keks up north, or specific words for the type of garment like joggers or jeans. Undergarments are your underwear, and most people in the UK call them pants(M/F)/panties(F) or knickers(older F).

The machine translations for Finnish have been a target of constant jokes since 1990's and those translators still are not perfect today. So I believe when some AI software starts speaking Finnish the results are hilarious.

@fz1576

Жыл бұрын

Just wondering if Deepl does a better job with Finnish translations?

@ema2031

Жыл бұрын

@@fz1576 I think the problem is the structure of the language, a lot of words depend on the context of the sentence so you cant just put a bunch of words one after the other like in english.

@anomanees

Жыл бұрын

Mitä ite kokeilin ChatGPT:tä, niin osasi muistaakseni suomea yllättävän hyvin.

@charlieangkor8649

Жыл бұрын

I am volunteer working on a machine translator Khmer-English called Sugar Palm Tree Translator. I am anxious about being outperformed by Google and turned into irrelevance. What warms my heart is that out of 7 most basic and often used phrases in Khmer, Google Translate fails on 6, while mine does all 7 correctly: older sibling, younger sibling, older brother, older sister, younger brother, younger sister, siblings. Also, Google Translate once translated "the police shot at the thief" as "the thief shot at the police". Comes handy when you are reading the news reports.

@chayaalexandriaakin429

Жыл бұрын

Machine translations of Hebrew and other Jewish languages are also infamously bad to the point where it's a joke on the internet. Even a lot of yiddish words that have been absorbed into standard English or just English Jewish words aren't recognized by auto-correct, let alone Siri.

Some people are mentioning how AI can help small languages survive. It's an interesting idea but also a dangerous one. The AI is not a native speaker and might damage the language. This reminds me of the non-Scot speaker that wrote a lot of Scot-wikipedia based on what they THOUGHT Scots was, and did a lot of damage there.

@DLCS-2

Жыл бұрын

Yes definitely

@aedeatia

Жыл бұрын

The AI speaks English pretty well, I guess it comes down to if they can obtain enough training data for the rare languages.

@xXxSkyViperxXx

Жыл бұрын

lol theres a lot of misconceptions out there. there are people too that arent very clear on how chinese languages work and they get easily influenced too by some people's agendas based on how their perspective works

@seanleo1285

Жыл бұрын

Damage? So when a non speakers use your language, they damage your language?

@MateusSFigueiredo

Жыл бұрын

@@seanleo1285 usually, no. But for endangered languages, maybe. Specially when it's not clear that it's a non speaker speaking. Read about Scots Wikipedia and the damage done there.

Unicode originally thought it only needs 65,536 code points to represent all the writing systems in the world. That was proven to be false so they just created new encoding systems and can now represent millions of code points. Researchers and language advocates continue to add obscure writing scripts to the Unicode, like the Linear B ideograms which are not yet deciphered yet but are now part of the Unicode character set. It only worked this way because Unicode is a non-profit entity. And that's what needed in large language models which are used in chat AI. A non-profit entity that can add even extinct languages to these models. A non-profit organization that is accessible to researchers and language advocates, while at the same time sets the standard on how large language models should behave.

@merry_christmas

Жыл бұрын

I vouch heavily for LLM and a Chat AI standard to remain non-profit and preferably open source. But I can imagine Google, Microsoft, Amazon & Meta to try everything they can to prevent that. Hopefully government regulations will be put in place for EU, as we barely have a say in what those American hypercompanies decide with the future of internet.

@oceanicmage

Жыл бұрын

oh if only this commentwas written in November last year.....

@SolomonUcko

Жыл бұрын

OpenAI kinda is? They're not being very open though, and they have a for-profit subsidiary OpenAI Limited Partnership...

@nemesi55

Жыл бұрын

I’m absolutely in favor of this approach. I think AI has the potential to be a transformative tool when it comes to preserving languages, but it must be managed wisely.

Another language related issue is how precise a language is inherently. We could expect different results in therm of quality or efficiency in specific fields based on language specificities. Let's take Pirahã as an example, they only have 2 colours in their languages light and dark. They would have a harder time express visual art related specificities in therm of colour. Inversely they might allow more creativity to the AI to try things... so it's a mix bag and every languages will have blindspots like that.

@polecat3

Жыл бұрын

I don't think that's an issue

@petyamiteva2382

Жыл бұрын

Languages just are. If they exist in the form that they do, it’s because that’s sufficient for speakers to communicate effectively. It’s not AI’s or anyone’s role to determine how precise or not those languages are.

@incognitotorpedo42

Жыл бұрын

@@polecat3 lol ok, that's your opinion. It isn't mine.

@qingyangzhang6093

Жыл бұрын

It helps when translating through two non-English languages doesn't have to route through English. For one, when translating between French, German and Russian, the tu/vous, du/Sie, ты/вы distinction would be lost if we have to do a translation to English first (all translate to you)

@hdjwkrbrnflfnfbrjrmd

Жыл бұрын

Ok but Piraha is a language of stone age primitives, an extreme outlier.

I tried speaking to bing (by hitting the mic button) in Vietnamese. It not only recognized what I was asking but it gave an appropriate response with the correct diacritics.

@Boobeinstein

Жыл бұрын

Same with Kiswahili. All LLMs are pretty good with it with only some minor grammatical errors once in a while. Maybe our languages are just simpler to learn.

@onlysublime

Жыл бұрын

@@Boobeinstein I know it did a better job than Google translator or bing translator

It is worth noting that the classification of a language as high or low resource can vary depending on the context and criteria used. (by AI )

It's so wonderful to know that people are working so hard to make sure that all speakers of all languages, even the most obscure ones, will also have their jobs taken by natural language models! Equality!

I'm now imagining a job people may have one day, to sit down in a massive office with others, literally just chatting with one another about anything, or writing books, or poetry or whatever, while an LLM catalogs all of it. I feel like those sorts of jobs could jumpstart low resource language sets. Too bad our tech companies don't prioritize stuff like that.

@hundvd_7

Жыл бұрын

You have no idea how much training data LLMs need do you? You could pay starvation-level pay for those native workers, and you would still bankrupt a company the size of Microsoft if you wanted to do that at scale And then again, why _should_ they prioritize that? I'm not a native English speaker either, and it'd be great if we could keep obscure languages alive, but why take it out on AI companies? They have no obligation to do anything, and AI isn't _by far_ the best way to achieve this goal. Not even if AI was genuinely perfect. What use is there an AI speaking it, if people just never want to learn it? Just put that ridiculous amount of money in education instead...

@Grymgar

Жыл бұрын

@@hundvd_7 Your lack of imagination does not surprise me. And yes, obviously a single paragraph I wrote after seeing this video is the exact blueprint I think should be used going forward. That is exactly what I said. 🙄

I was not expecting open source to be brought up but I'm very glad it was.

Absolutely off topic, but @5:50-- I really like her paintings 😁

Nice one there. Yeah, a spotty dataset is a problem for language processing. I personally know people, who speak a dialect of an high resource langugage, but because the language they speak is written down standardized, the dialect/language they speak is basically not represented in the dataset. e.g. good luck getting your point across to the computer speaking irish english or swiss german.

Hope this gets addressed for the low resource languages. Here in Canada, many of the First Nations have been challenged to get younger generations learning their languages so AI is something else for them to bolster their efforts.

@Student0Toucher

Жыл бұрын

Native Americans need to move on lol Canada is being taken over by Africans and Asians

@navin750

Жыл бұрын

Only solution is to generate more content, and frankly it is impossible to just transalte the internet and everything in it to every language, as you need to invent new words for it.

Awesome vid right here! :) I learned a lot here! :) Thank you and cheers to the uploader! :)

Such an important and insightful video! Thank you for sharing I look forward to learning more.

0:35 Chat GPT is based in GTP 4 only for the paid version, for the free version it's GPT 3.5

This is really intriguing for English variants. If English is allowed to change over time and become something new then AI should be learning the process in countries where English hybridised with other languages like Singapore and Hongkong. Perhaps one day the will be a language model that has no problem blending languages and adapting to new grammatical rules. Language is not static after all

@mikicerise6250

Жыл бұрын

ChatGPT already has no problem blending languages. I speak to it in English and Spanish, and it has no trouble understanding me, even when I'm using both in the same context, and no trouble using English terms when it thinks it appropriate. Sometimes, for example, I might be asking for help in English and say "my system language is Spanish, so tell me where to find this option in program X with Spanish menu options." No problem at all. ChatGPT4 has perfect English, Spanish and German from what I can tell. I don't know about other languages, but I've read that its proficiency goes down the less common the language is on the Internet.

@noahmethven

Жыл бұрын

@@mikicerise6250 Yeah obviously it has no problem with those languages because they're dominant global languages with a lot of source material. The point of the video is that languages that haven't been exported around the world or aren't represented by countries with a lot of political power are being left behind

@BlackEagle352

Жыл бұрын

We should speak binary

Very nice explanatory video. I am catalan, and I too was surprised by how well chatGPT could write in Catalan, though it still had some mistakes. Let’s hope we can add some clarity and transparency to this issue

Always love your stuff Phil❤

Never expected to find a Yaadie 🇯🇲 researcher on Vox.. big up

That was SO interesting . Thank you

After learning anything about languages, I always wonder just how many have been lost to the sands of time... never to be heard again... sad and tragic, but also inevitable for most languages as time marches on.

@kevingeoghegan294

2 ай бұрын

All languages in time

when you started explaining entailment, it made me glad to see a 'useless' specialisation like linguistics intersect with the biggest tech trend

In addition to this whole mess I would say, there are words that you cannot directly connect by meaing to other languages, you would need to guestimate them all. For example you can say that russian "smekalka" is something like "savy" but historically and culturally it's very different.

@MateusSFigueiredo

Жыл бұрын

I think the AI "thinks" in each language. It doesn't translate, unless you ask it to.

@danser_theplayer01

Жыл бұрын

@@MateusSFigueiredo Right but it cannot inherit what people think of the word, the AI can't find the underlying context. Idk how to explain it properly english is not even my first language.

@MateusSFigueiredo

Жыл бұрын

@@danser_theplayer01 I agree. The robot has read every book but doesn't really understand any of them, much less understand the feelings, the subjective interpretations, the insinuations...

@lucaskp16

Жыл бұрын

I dont think it matters when you say historically and culturally really different. For example i am argentinian from the capital buenos aires. The chat box needs to understand formal neutral spanish the most not our own sub language lunfardo of which you can speak almost whole sentences with no actual Spanish in them. This words also dont translate well to 1 single word in spanish or english.

I guess it's time to learn Gaelic for the future machine wars

@Game_Hero

Жыл бұрын

Cha chuir dad stad air a’ Ghàidhlig!!!

Top-10 the most used languages on the Internet are 1. English - 56.1% 2. Russian - 5.1% 3. Spanish - 4.8% 4. French - 4.2% 5. German - 4.2% 6. Japanese - 3.5% 7. Turkish - 2.4% 8. Portuguese - 2.2% 9. Persian - 1.9% 10. Italian - 1.8%

@Nabrolo

Жыл бұрын

Chinese not being there is a bit suspect. Their internet usage is very large.

@yanikkunitsin1466

Жыл бұрын

Chinese don't have internet?

@Max_Jacoby

Жыл бұрын

China is number 11 with 1,5%. Although Chinese is spoken by 1.5 billion people their usage of Internet is heavely restricted by government.

@yanikkunitsin1466

Жыл бұрын

@@Max_Jacoby they restrict only some access to western resources, they have way more internet usage than Japan. Also no Hindu and Arab. That's absolutely wrong statistics you found.

@aedeatia

Жыл бұрын

@@Max_Jacoby I don't think that they are restricted from using the internet, it's just that there's the Great Firewall that restricts access to foreign websites. I looked up your list and it seems to report the percentage of websites, not content. From the methodology section "We investigate technologies of websites, not of individual web pages. If we find a technology on any of the pages, it is considered to be used by the website.". So Chinese only being 1.5% could be because they only have a few large websites that everyone uses instead of a whole bunch of smaller websites.

Thanks Vox for highlighting the genuine concerns surrounding language models and the significant influence that large corporations have on the source data that is used to train AI models.

I work at an AI startup as a prompt engineer. I think the main reason AI LLM (Large language models) don't know every language has to do with the linguistic knowledge of the people compliling the LLMs. Rarer languages have less data to be scraped, so you need to try harder to find the content to generate new data. However, there are some languages, such as Hebrew or Basque, that models such as GPT4 know quite well, and they have fewer than 10 million native speakers, each.

@DLCS-2

Жыл бұрын

Not exactly about less data , mostly is about which data you chose to train the LLM with . I remember an article that came out two ir three weeks ago which was about kinda close to the same topic. In the article they were stating that sometimes the wirds that were being translated weren't always the correct ones , and that ked to problems They also stated that for the LLM to understand the words that needed ro finetune / filter the data that it was fed or feed a smaller bach if correct data.

@dibbidydoo4318

Жыл бұрын

"I work at an AI startup as a prompt engineer." that's a job? does your ai company even train models?

Thank your for the insight.

Really tough job done, by you and your team. Congratulations 💐

I study reactions to AI and I love this series for explaining AI to my family, Phil is the best

Do the video editors at Vox use Adobe software? Keep up the great content

Thank you Vox for this insightful video. Always relevant and makes my day :) ❤

Dear Mr. Phil, Congratulations on your newest KZread video! I wanted to take a moment to extend my warmest congratulations to you on this exciting accomplishment. Your dedication and hard work really shine through in your videos, and it's evident that you're passionate about creating content that resonates with your audience. Your latest video was engaging, informative, and truly entertaining. The way you effortlessly connect with your viewers and share your unique perspective is truly impressive. I know how much effort and creativity go into producing high-quality content, and your latest video is a testament to your talent and commitment to excellence. You continue to impress and inspire your viewers, and I'm sure your KZread channel will continue to grow and thrive. Once again, congratulations on your latest KZread video, Mr. Phil! I'm excited to see what you have in store for your future videos, and I wish you continued success in all your endeavors. Best, ChatGPT

Omg what cameras do you guys use? The quality and the processing are absolutely stunning 🩷

I've discussed this with my Cherokee philosophy professor. ChatGPT will just confidently lie about Cherokee translations.

@ugotgamesonurphonee

Жыл бұрын

It’s not necessarily a ”lie” it’s called AI hallucination, u should search it up it’s super interesting to read about :3

@michaelrenper796

Жыл бұрын

Because ithas almost no trainign data on Cherokee.

My like is for Ruth and the JamPatoisNLI, I never heard of this

An additional problem for low resource languages is that they're often not standardized, so there may be different writing conventions, different words depending on local dialects, etc. All of this makes it extremely difficult even to get good data to train on.

An oddity for you: when my father graduated from college, in pharmacy, in the 1930s, his first job was translating English-language pharmacy texts into Welsh. He lasted two weeks in the job, he told me, and quit, believing the whole thing was useless. My former partner's first job, when she first left higher education because of the civil war, was translating English language texts into Tik-Monjiaang, ("Language of The People," aka "Dinka"). Exactly parallel story: she quit within weeks, believing that the whole effort was going nowhere. (Today, a generation later, both efforts, in Welsh and in Dinka, are back in operation -- at scales comparable perhaps to that in Catalan in this excellent video.)

That's very important for indigenous minority and endangered languages like Occitan, Breton, Cherokee, Ojibwe, Paiwan, etc

I remember a friend who lived in the country of Georgia back in the early 2000s. He said technology companies didn't have Georgian language settings in their devices. So many Georgians used Russian or English on their devices.

That stache is getting out of control 🎉 Love the videos

Literalmente le pregunté Chat GPT ¿Cuáles son los canales que consumirias si ipoteticamente fueras una persona? Y me recomendó tu canal

Interesting video. My question is why do you bleep out the percentage of GPT-3 that is made up of Common Crawl?

@timseguine2

Жыл бұрын

Probably because he signed an NDA when he talked to them.

@merry_christmas

Жыл бұрын

Because it's 69 🙃

@stevemawer848

Жыл бұрын

@@merry_christmas Not 42?

you could start answer queries from a vector database (i.e. [x] language dictionary, common expressions, Q&A's etc.) and gradually train it the GPT with those precise recalls till it reaches optimal "comprehension" levels.

finnaly a good video on the topic of AI and LLMs, I actually might have found a solution to this it is not just the data but there are other fundamental problems too (it's more related to indexing and embeddings)

Actually AI and chatgp[t like applications can be a saviour of small languages. I'm catalan, and never imagined I coul have the option to read any documentation about any software application for example... now I can. I get the same quality of answerd in English that in Catalan.

When I use ChatGPT I use English and then whenever I need to show the result to other people I use Google translate on a browser level to convert the whole conversation to Korean(my native language). I think general-purpose chatbots like ChatGPT is the niche where it makes sense to imitate what has been done on a dataset that is mostly English and replicating the same task on a different geographic context. It does work as a localized replacement rather than a redundant vanity project.

Come to think of it, a lot of best selling books have been published in dozens of languages. It might be a good evaluation for multi lingual systems to find inconsistencies between versions that we already know of.

Big Up Jamaica!!! Big up Ruth-Ann!!

My friend is working for a local Indonesian AI startup. He was majoring Indonesian Literature and actually the perfect major for teaching AI about Bahasa Indonesia structure, vocab, and rules

We Appreciate You

imagine javanese language that has 3 extreme level of register language for daily conversation that sometimes sounds like a whole different language

It's a good thing to deal with such issues in first hand. AI should be a platform where all the people can do everything.

I was wondering how language models deal with primarily spoken language ? if we manage to add audio in training dataset, it can make a huge difference. Some languages have a lot of variations in “speaking/writting” form or “colloquial/formal” form such as cantonese or javanese… Also imagine if languages models can output regional vocabularies and mimic regional accents… it will change the translation app, the way we learn languages, etc…. 😎

This is a great video talking about what gpt actually is. I think most people think it is sentient computers when all it is is trying to model language like a kid in a library trying to learn but having no actual experience

I'm a bit surprised there was no mention of embeddings at all in this video. Once a large language model has learned a diversity of different types of languages, there's a lot of transferable knowledge that it can pull from for low resource languages. For example the reason why it's performing so well on Catalan given only 140 pages of example is because it's pulling from similar languages that it knows in much more detail. This will be doubly true for the pidgin creole languages. I agree with the fundamental premise of the video being that data diversity is key. Since all our languages are related at some level then transferable knowledge means that learning new languages will bolster the quality of the whole.

That is very interesting, I didn't know this was a think

Gpt-4 is brilliant with Romanian! It is a very very difficult language to write correctly and it gets it spot on every time.

@TikoVerhelst

Жыл бұрын

Same with Dutch. I can write the most informal stuff and ChatGTP just gets it! I have never experienced that before with software.

Sometime ago, I've simulated the same narrative using GPT-3, firstly using English and then Portuguese. I've noticed that in English it was able to create not only way better dialogues, but also a superior quality narrative, more informal and creative.

Phil is my motivation for waking up in the morning.

7:21 I wasn't expecting to see Hugging Face featured in this video! Props for giving them the coverage they deserve. 👍

while most people wonder if the chat is sentient, you hit the nail on the head and understood how it works, when it doesn't and its social impact. very professional, journalistic aproach. Vox is really one of the best sources of information these days. it is not perfect... nor does it cease to be a company... but it does have good people working on good stories

AGI is coming. It will change the world. It will help choosing the right job and answering difficult questions. It will also open many possibilities.

I asked chat gpt on percentages of language in GPT-2. It tild me the following GPT-2 was trained on a large and diverse corpus of text data from various sources, which includes a significant amount of text in English. GPT-2 was trained on a web crawl corpus that contained a diverse set of texts from websites, news articles, books, and other sources. The corpus includes texts in several languages, but the majority of the data is in English. According to the OpenAI team, the corpus used to train GPT-2 included around 40 GB of text in English, out of a total of 45 GB of text data. It is worth noting that GPT-2 was designed to generate high-quality text in English, and its performance in other languages may not be as good. However, GPT-2 can still generate text in other languages, and the quality of the output depends on the quality and quantity of the training data available for each language.

The fact that GPT can do a passable job of writing in languages for which it has very small datasets makes me wonder if it has discovered something consistent among all languages.

languages evolve. we are now moving into a more international relationship with each other, and for whatever reason, a lot of that seems to be done in english. I could see an agrument made in regards to that, for sure. if the point of a language is understanding eachother, over tradition or culture, then theres some sense to make out of streamlining whatever language we settle on. 'Not comforting, given the notion that a language helps to frame how people think. Contextualizes the text by its implications past the dictionary meanings of a word or phrase. What im curious to see is how much more technical language might become more common place. As a computer geek I have been seeing more of technobable already bleeding its way into common parlance. I wonder if we'll hit a point where some more of the computer geek/cyberpunk style slang becomes more common place, like "meatspace" or "IRL", for example, to suggest that its not on zoom, or virtual in some way, etc. Or even how much of it will be similar but bastardized into something else. Ive heard a lot of older people point at their computer towers and call it a "harddrive" ...which is not totally wrong, but also not really correct, either. And how much of that will be picked up by our AI/tech tools, and (correct or not) and pushed out to people.

Thanks for this. I’ll just say that I was surprised when ChatGPT was able to translate English statements to Cree.

I’ve been watching so many Phil videos lately that I was worried he wasn’t making a lot of videos for Vox

6:51 Interesting to know how small a percentage Wikipedia forms a part of the GPT3 training dataset. I wonder whether they make use of Wikidata too, and how.

Are there higher value sources vs lower value sources? Like are there tags for verified trustworthy or high usefulness or other important markers? Also, do they include dictionaries and thesauruses?

great video gary oldman..

The information that the next most common language for words in GPT-3 was German - as a German speaker - absolutely baffled me... Especially that it's ahead of Spanish!

In Mexico, we have 60 languages, and a lot gonna disappear, I saw hope in AI, but with this I don't think so more.

My academic background is in Philosophy of AI and language. I don't know if we will learn anything about "artificial intelligence", but we are going to learn a lot about natural stupidity. This "AI histeria" is going to be the mother of all investment scams.

@errgo2713

Жыл бұрын

AI doesn't actually need to be intelligent by the normative standards of human intelligence in order to be *effectively* intelligent in the context of common types of work. I sense you're implying that AI ought to be, a bit like Chomsky, when that isn't even necessary.

I noticed that when you asked the GPT-4 version of Chatbot science related questions it would actually just translate them into English, answer them and then translate them back but this leaves it with a ton of nonsense because it'll use English names for constants and confuse itself.

... same reason AI (still) doesn't speak whale🐳: data. but that's a different story 😉

good. keep it that way

This is super interesting. Would love to see a bit more on how AI is or isn't accommodating people with disabilities i.e. deaf, hard of hearing, blind, etc.

@borissand3891

Жыл бұрын

I'd assume it can understand French so that's one disability covered

4:20 what does lol mi just wake up mean? Lol as in lots of laugh or is it a jamaican word?

Accessibility is great and has to be pursued but at the same time how do you protect the language, its culture and the people that produce/reproduce it? all those AI make so much easy to track and control what is said online while deciding what can be read or watch and removing, hiding or blocking the access for any other information and certain languages. It's known that intelligence services in the 60's and 70's sent anthropologist to many third countries to learn about their languages and ideologies, we don't have access but the data is there and more likely they're already using it.

Algorithmic punch! (Concise scan of something complex!)

You forgot to mention it doesn’t understand english either

Esperanto restos homa eterne!

1:40 He arrange documents on the floor Me: *A Bungou Stray Dogs reference intensifies*

Thank you

The Bofa language of south-central Cameroon is famously known among linguists as being impossible to learn unless you were born into the Bofa culture. Figuring it out will be the true litmus twst for AI.

@Student0Toucher

Жыл бұрын

Well to bad for them…they better learn a known language

@jeffersonclippership2588

Жыл бұрын

@@Student0Toucher bofa deez nuts gottem

That will change in a few years, exponentially so.

Interesting idea, if we develop these models well enough, could we get them to learn these languages through apps like DuoLingo and others? Like maybe humans could train an LLM like a person by going through a long conversation with it and then it could retain that information

*One reason maybe is that AI speech tools rely heavily on the use of big data sets to learn a language, its pronunciation, grammar and semantics. The ability or quality of the resulting tools is mainly limited by how much data is available (and how “good” it is).*

@polecat3

Жыл бұрын

It's almost like the video was talking about that exact phenomenon

@mryochi_yuzhe8873

Жыл бұрын

bro didn't watch the video 💀

I know so many non native English speakers that put all their digital content in English to reach a wider audience, and as a result all of their content is in English and not their native language.

You changed your thumbnail. lol GOOD CALL

"Unsupervised Neural Machine Translation with Universal Grammar" by John Hewitt, Christopher D. Manning, and Percy Liang. Noam Chomsky meets GPT tech... It seems foreseeable that some day GPT10 or something like that won't require much of a dataset to understand a new language very well!

Kevin Scannell or Mícheál Ó Meachair would be the ones to speak to about Irish/Gaeilge (Gaelic) if you'd like to focus further on minority languages in data sets