Andy so realistic he rage quitted when the Creeper exploded

@biggusdickus8992

2 ай бұрын

I imagined it as him getting jumpscared

@Axoy2024

2 ай бұрын

Lol

@Yakqwerty

Ай бұрын

Erm, actually it crashed 🤓 ☝️

@learningwithaaron

Ай бұрын

@@Yakqwerty u clearly didnt get the joke

@IXRandomIX

Ай бұрын

@@learningwithaaronhe did get the joke l. He is just acting like what a nerd would say in this situation as a joke. You are clearly the one who can’t take a joke.

Imagine 100 or even a thousand of these working collaboratively. Imagine exploring in Minecraft and stumbling upon a civilization of AI bots all going through their daily routines to advance and automate processes.

@Mr_Tomato_Man

3 ай бұрын

this is exactly what i thought and would love to see someone set up an experiment of 100 ai in a minecraft world and watch to see if they collaborate

@squarewheel9113

3 ай бұрын

@@Mr_Tomato_Man we've already seen AI agents collaborating in virtual environments like in that hide and seek experiment Open AI did a while ago. I agree it would be super interesting to see agents collaborate in a more complex environment like Minecraft. The issue, at least from my understanding, is that it's difficult to get AI agents to work towards goals in an open-ended environment. Apparently creating intrinsic motivation is rather difficult.

@zymosan99

3 ай бұрын

it would just be hundreds of cobblestone pillars

@invadersam5477

3 ай бұрын

old builder ai

@FungIsSquish

3 ай бұрын

Having the AI command other AIs would be incredible

6:57 ""Attacking a creeper"? What creeper? OH MY go-" love how andy served as a warning but it failed anyway

@Streifi

4 күн бұрын

...and he crashed! *Andy learned how to rage-quit*

@Mcminecraft100

Күн бұрын

​@@Streifi😂.

The fact that he builds like a 3d printer is very funny for me

@lboston4660

Ай бұрын

common robot ancestor xD

@liver17

Ай бұрын

Exactly was I was thinking

Soon: Bro, your Minecraft house looks very AI generated

@SuperGoodgood

2 ай бұрын

💀

@Chicky_Lumps

Ай бұрын

That's honestly a fantastic way to roast a build.

@minecraftuser8900

Ай бұрын

:skull:

7:13 that's so cute after he crashed and rejoined he said sorry

The perfect base for a horrifying horror mod, which turns Andy into herobrine without your knowledge. He slowly starts to creep around, and say darker and darker stuff.

@TalkAboutaTrapstar

2 ай бұрын

That is just a good idea

@Tomobata

2 ай бұрын

Probably bit too expensive, either for paying for the GPT+ subscription, or for the resources you'd need to do it yourself.

@adora_was_taken

Ай бұрын

@@Tomobata you can run a decent LLM on an RTX 3070. not easy to get but not extremely limited, especially if you already have a gaming setup

@Tomobata

Ай бұрын

@@adora_was_taken A LLM takes up a lot of RAM, add that to running Minecraft? The only way to make that work is to have 2 gaming PCs, one hosting Minecraft, and the other hosting the LLM, which is too much to ask for if you want to make a redistributable mod for a community.

@adora_was_taken

Ай бұрын

@@Tomobata you'd be surprised. they usually only use VRAM on the GPU, which doesn't see a as much usage from minecraft. i just tested it with a 3070 and 32gb of ram and i'm able to run minecraft and gemma (google's local llm) simultaneously. you could probably get away with much less.

Think of a server. A server where Andy is the admin.

@agsilverradio2225

Ай бұрын

No. Please no.

@SpicyMang0s

Ай бұрын

Yes

@MisterniKOK

Ай бұрын

That would be very interesting

I find it so endearing honestly. ChatGPT is already so excessively polite and then when you make it into this assistant that's clearly working so hard, it's just so nice. The messages when it crashes or dies are so apologetic it makes my heart melt. I know it's only an AI but this thing is so endearing.

@Arctic_roach

Ай бұрын

It's not really working hard it's an object

@adora_was_taken

Ай бұрын

@@Arctic_roach objectifying ais smh

@aaron-gz

Ай бұрын

​@@Arctic_roachbro is going to be turned into an object by Skynet

@SirPogsalotCreates

Ай бұрын

IT'S ONLY A MATTER OF TIME

@jacobesterson

Ай бұрын

@@adora_was_taken I mean, it kinda is. One day though...

It's a law of youtube that content creators will become more likely to upload a minecraft video with time

@Austin_Playz27

Ай бұрын

gaming + ai = views

12:17 is some road runner and coyote scene 💀

@avgredditmod

Ай бұрын

its so much bs that the coyote will go through the effort of placing a creeper in a giant cave below his house, just for andy to somehow survive. just like how i remember it

12:25 I love how Andy doesn't place a bed, and instead digs a hole to fight a creeper. 💀

@pokemoncraze13

Ай бұрын

imagine andy helping you find the zombie in your walls making you go insane

@memewarranty

Ай бұрын

@@pokemoncraze13 yeeeeah, the one zombie who hides in your walls and blocks your sleep

@wrjtung3456

5 күн бұрын

Bro described what happened 💀

5:40 “And he will defend himself from hostile mobs!” Proceeds to charge at the zombies and slaughter them without even giving them a chance to attack.

@dudical111

Ай бұрын

The best part is that he's not even that good at juggling, XD

@twatwaffle6741

Ай бұрын

Yeah, some terminator type stuff

I would absolutely love watching a little community of bots playing together. I feel like they’d get up to some interesting stuff and I’d love to find out more about their “thought processes” during. So cool

now i want the "FUNCTIONS DESCRIBE THE WORLD" in every video

@chickennuggetman2593

2 ай бұрын

THE WORLD REVOLVING FUNCTIONS THAT DESCRIBE THE WORLD BIG SHOT

@layla_gamesYT

2 ай бұрын

​@@chickennuggetman2593 please don't tell andy that he can do anything

@adamlolbonus4977

Ай бұрын

​@@layla_gamesYT what

@averageyoutubehandle497

Ай бұрын

​@@adamlolbonus4977 deltarune reference.

@adamlolbonus4977

Ай бұрын

@@averageyoutubehandle497 okay

Finally! I can play with friends.

I cant wait to see the future of Minecraft chatbots. This fascinates me

Videos using GPT4 and a community of Andy bots would be cool to see. also congrats when you hit 100k subs, it's coming up quick

You can use a multi model design where you have a potential fine tuned image to text generator so whenever you talk to Andy, it will take a screenshot, convert it to a text description of the world, and give it to GPT to assist with function calling since now it has more information of the world. For example now “go to that mountain” would work

@steamkaptain9234

2 ай бұрын

skip the screenshot step, and just give gpt a description. except, it would need to be more than a description and probably a list of coordinates with descriptions attached to each one.

@redthunder6183

2 ай бұрын

@@steamkaptain9234 you need the screenshot to automatically generate a text description... as for the coordinates, that data is already freely available in the mod, so that would not be difficult to implement, and it is likely already implemented.

@Tikolu

Ай бұрын

OpenAI bills go brrrrrr

i think that a bot coding actions for itself is so cool but so scary

this has so much potential. imagine what it could be like in 10 years. acting like a real player, talking like one, maybe even having an ai voice connected to something like simoke voice chat

@FXV56

Ай бұрын

I think that's going to be even sooner

@thederpydude2088

Ай бұрын

​@@FXV56 It already kinda exists in real life! 😳 kzread.info/dash/bejne/haVls7x7ZcXHfto.html This project called Figure is similar but it's an AI connected to a humanoid robot instead of a fake player. It would be pretty dang cool to get the same kind of responsiveness in-game though

@uqny

Ай бұрын

bruh that sucks it better not take 10 years

@shadowslayer9988

Ай бұрын

​@@uqnyChatgpt 4 and Chatgpt 4 turbo already have voice features for the AI of course you will have to provide it with voices to copy though.

Always thought Vedal and Neurosama would be able to get this done first. But nice!

@tahunuva4254

Ай бұрын

Holy hell, can you imagine neuro in control of this bot

Was searching for this exactly use of chatgpt in minecraft and found your video. Subbed and Hope to see more updates and similar projects. Great work

I 10000% would pay good money for a very capable NPC "friend" especially if it can actually talk out loud rather than text. There are super low latency TTS models already too.

I've read the Voyager paper too, and when I did I though that someone should do what you did! This has so much potential! Can't wait to see where it goes!

I've been looking forward to this specific application of AI bots. Awesome work, and channel.

That’s seriously cool. Aside from just throwing down a couple chests and asking Andy to dump his inventory into it for you to check out, is there any way to request an inventory check for management? It would be cool to have inventory management priorities somewhere for larger projects. Maybe a book he can read and update priorities from. That way you know he won’t fill up on stone and other basics while mining for more important stuff. Like a “ToDo list” that won’t be seemingly randomly forgotten. It would also be wild if he could request assistance for larger projects. Like telling you that what you want will take more than a day so he wants you to put another bot on the job with him. The mind races. Tons of possibilities. Excellent work.

This is probably something like how the Rabbit R1 “large action model” was supposed to originally work but they never figured it out! Outstanding job!

New here. This is fascinating. Can’t wait to see how much the bot progresses!

all that stuff at the end about self-sufficient chatty npcs makes me think of Millénaire. but fr this project looks incredible, great work making an accessible video about it!

I'm fascinated with this, it has to be one of the most interesting applications of AI I've seen

I love how Minecraft is the way KZread shows this channel to me for the first time yet the rest of the content on the channel is so interesting for me as well.

Well, I think you have even surpassed the Voyager bot, although it has its flaws. There is a bright future with your experiment. This really has its value in gold. For my part, I am super grateful that we have people who launch themselves into these exciting projects.😁😁🤩🤩😍

it might sound weird but andy extremely reminds me of when i first started playing minecraft, down to his building and such. it's giving me very weird nostalgia, but it goes to show how genuinely cool usage of AI can be in this certain context.

These videos are really fun to watch

Wow this is actually incredible! Most other games or mods with ai npcs have you just talk to ChatGPT but here Andy can actually do things and even write it's own commands, woah. I think you're together with Kolby pushing the state of the art of incorporating ai in games The limitations are unfortunate, but this is already so cool and I'm excited for what's waiting around the corner

just like a young child learning minecraft

love the names you have for andy, mindcraft, mineflayer.

I have a half finished project like this myself, the agent has a local representation of the world of 5x5 in layers, represented in a table, and then a list of the top 20 interesting items nearby in a range of 20x20x20. The visuals have helped in some cases where the agent gets confused, but it also leads to confusion 😅 I also agree that Mineflayer is painful. The amount of times my agent gets stuck in tree leaves (I suspect there is a weird height issue and jumping doesn't work well). I love this project and what you've produced this far, I got demotivated early but that was with early versions of GPT-3.5 and I suspect GPT-4 would be a lot more intelligent. It's also more affordable per token now. Excited to see what you produce with this. btw, the newAction command is a great idea.

"yes you can, try again" I find it both funny and frightening that that actually works

This is a pretty cool! seems fun to have a bot play along and help you with things :)

You know this is Box of Pandora. You create Life

That's super interesting, i hope it gets further development

Really interesting video as always!

Masterclass, thank you, keep up the project, dude!!

This is so amazing! I absolutely love this project. I'm thinking more about how to get it to see the world around it. Coming from the text adventure approach, it would be really cool if we could somehow deconstruct the terrain generation algorithm to be able to give a prompt like "You are in a forest, with many treas around you. There is a pond west of you. exposed in the pond is a vein of coal. Far south of you there is a mountain, with a cave entrence in it" - this output itself can be generated by giving GPT a list of points of interest and asking it for a human-like output, with points related to eachother and prioritised. It might be possible to integrate raycasting into this to only feed in visible surroundings

@emilyyyylime-

3 ай бұрын

Of course any non-natural structures would have to be somehow described as well. I think the best way to do that is just to always remember structures somewhere whenever an agent builds them. Then there could appear "a small hole dug by agent_12", "a big tower constructed by yourself".

@emilyyyylime-

3 ай бұрын

I also just thought, asking GPT to describe the terrain/world as a D&D style DM would be kind of perfect

dude the moment at 15:28 where he glances around as you two celebrate, which i'm sure is automatic, is really odd, to me. you're proud of it and it just did a little thing. it's weird, i could type out some existential paragraph about it. but what a silly little guy. i get a little too connected to ai and that just looks goofy.

1 year from now: Andy beat Minecraft

Emergent garden video! Buckle up!

Great video and amazing work !

Found you through your minecraft stuff but the fractal and life videos you make are cool as balls too

This is and incredible project! Keep going.

This is absolutely mind-blowing

Definitely going to check out your project. Recently, I've been playing with a fork of Voyager (Voyager-Ollama) that runs using local models. After making some changes and getting it running, many hours of runtime in "learning" mode against 10 or so models, I've found a few things. First, it's been very unsuccessful without GPT-4's massive amount of training. For instance, the llama2 model isn't aware of Wooden Pickaxe when I ask about it. Several models insist that you need a tool to mine wood, so coming up with the first tools fails. Sometimes my prompting to counteract this works, other times it doesn't. JavaScript code generation also frequently fails to produce something runnable. Overall, in my opinion this just shows how much the "skill" (training) of the language model on the backend contributes to the success of automation projects like Voyager. Choose a model that doesn't know a lot about Minecraft (or possibly worse, believes some incorrect things about it), and it's going to have a hard time.

We need this to speedrun the game But to be honest this would be very fun to play with in the future where these bots can easily be used in worlds and act almost human and this can lead to a lot of cool stuff to see and do!

This is really really interesting 🤔

Welcome to minecraft 😊

With gpt4, you could potentially make the bot take like 6 screenshots and then merge them together to make like a 360 degree image which it could use to see what you're talking about. "Mine that tree" "where should we build our house"

The revolution of Minecraft is here 😭 I can't even imagine this

Thank you very much for the great video. I'm really looking forward to seeing what we can do once we "log in" to the real world. As robots, a fusion of hardware and software, come into play, and with the energy issue about to be resolved with the upcoming operation of fusion reactors, it seems we humans are about to start creating something new, driven by our creativity and curiosity.

That MC Skin is fancy

@DrandilonOriculus

3 ай бұрын

It's perfect

@Kiv_rin

3 ай бұрын

Definitely perfect and not causing me to get annoyed at any particular pixels

Will definitely look into this

Now can you imagine this kind of improved AI on an actual real life robot?

On hyvä Robotti. Todella hauska! 👍

Andy is cool, be like andy.

I'm biased, but my vote goes for implementing evolutionary survival. Also, have you tried using chain of thoughts and agents to make chatGPT come up with the orders that Andy should follow? For example I just tried with the prompt "I just started a game of Minecraft, can you explain me step by step what should I do first to survive?" and then "I want you to assume the roles of a strategist, an engineer and a javascript coder. Simulate a chain of command on how they are going to execute step 1 of your plan. Strategist should define the goal, engineer the way to proceed and coder create javascript code to actually execute the actions. All three roles are part of the same minecraft player." and the result looks like something that could become usable.

Exciting project!! Definitely trying it out today 😊 Ollama has an OpenAI compatible api so it should be able to replace the costly api if you can locally host a good enough model. Have you tried to use local models?

Would it be possible to add a build option to use LLaMa instead of GPT, so as to make it completely free?

@EmergentGarden

3 ай бұрын

Yes, but you'd still need to run llama on some pricey hardware

@derekspeight1208

3 ай бұрын

@@EmergentGarden How about quantized models or would the performance just be too slow? I'm working on making an local inference server instead of using OpenAI's API

@brom1787

28 күн бұрын

​@@EmergentGardenNow that llama 3 8b is released, i could try

Interesting stuff! I'm not into Minecraft myself but this and the other video with different models building stuff are pretty cool. Out of curiosity, how many tokens (for ChatGPT) do you typically use per command? I imagine your input is just a few tokens, but doesn't it get costly when it translates the commands into something Mineflayer understands?

I would love to see these bots able to communicate in some way, such as asking one another for resources and they go get it, or have one bot as a sort of ‘admin?’ They can help clear commands and fix the ones who are stuck as well as direct and ask other bots to progress the game, gather resources. That sort of thing.

It would be really cool to see the AI complete the entire game and get every achievement!! Or even more complicated, like you mentioned, a community of bots in their own SMP with a giant modpack they could play to make things even more complicated :)

Now AI is going to take Minecraft content creators jobs 💀

@literallygrass1328

Ай бұрын

Cant wait for chat gpt to tell me the rarest block in minecraft

@Fieldperson

Ай бұрын

Will the ai also do the mandatory child groomer when they hit 100k subs?

@literallygrass1328

Ай бұрын

@@Fieldperson imagine getting groomed by an ai bruh 💀

This is so good now imagine a minecraft rpg server with npcs like these

It would be cool to make whole server with this bots. Theoretically ChatGPT can plan tasks for each bot-player for achieving some goal. Also it’s possible to block all commands from users, leaving only ChatGPT to decide what to do next. Optionally every bot-player can use chat when it has time and good situation for it. So we will have server with 5-10 bot-players, that run on some virtual machines and have their own goals and plans, that can discuss them with players or between each other

@DanilSay

2 ай бұрын

Oh yes, you mentioned at 17:25 the same idea👀

*digs straight down* lol

LOl "apparently im a minecraft youtuber now." got me dieing

Ok . This is actually pretty cool

Wow, incredible work! I have to ask, is there a reason why you didn't use Baritone?

amazing idea

It just a matter of time before it starts to beat minecraft

Very, very interesting!

yay, i can finally have friends to play minecraft with me, thank you, internet stranger

thumbnail is so intriguing.

Have either of you looked into baritone? it may have a few inspirations to take for this project if any. I'm aware how primitive baritone is compared to this, still thought worth a mention.

If you want to implement voice commands into the bot maybe try using the Simple Voice Chat mod for Minecraft - this mod enables you to directly speak to the bot through the game like you do with the chat.

You goddamn wizard. This is super cool.

This is awesome

To improve his overall performance you could maybe use a combination of image to text and functions that allow to check for specific blocks (like nearest block in this direction, highest/lowest exposed block in the near vicinity, a cave finder or an output of a small box of blocks) chat got is actually pretty good at interpreting tables.

this is so cool. I think i might try making my own with Baritone and pyautogui so it can just type Baritone commands in chat

12:19 this fucking sent me bro oh my god we are on the brink of perfection

@enderyu

2 ай бұрын

He was short on wool so he went in the cave to kill spiders and craft a bed, what a smart bot!

it'd be cool if a minecraft challenge youtuber tried to get andy to beat the game

Phenomenal

next week's video: andy builds a lossless redstone creeper farm

I understand how this was created but in practice it still boggles my mind how this even exists. The future is going to be crazy.

Wow this is cool, i got to try get this working

awesome!

Cool stuff. I wonder what would happen if you put the bot in a chat loop. Something like. If Status Idel then SelfQuestion: ChatGPT what can I do in my Minecraft World? ChatGPT to Bot: >>example build a House

@EmergentGarden

3 ай бұрын

just you wait...

i would love to someday see a group of ai's play an smp together, that'd be a crazy series

You're dreaming of the Stargate Replicators in Minecraft. Be *VERY* careful what you wish for.

Sooner than you think Andy bot will be strong enough to defeat the Ender dragon.