They: make Devin closed source Me: Devin, build your twin app

@StefanReich

3 ай бұрын

Big brain move

Mathew, Please get a free trial of Devin. Have it use all open source components, create a UI as beautiful as Devin, and open source the working product. Thanks!

@StefanReich

3 ай бұрын

Haha. "Please produce the holy grail for us. Thanks!"

@wurstelei1356

3 ай бұрын

Problem with commercial coding assistants is: they rely on heavily on open stuff from Github etc. Its a huger code theft as for now and even Microsoft is providing their basic coding-assistant for free with the vs code API. Still Chat-GPT 4 cant tell you whether a specific code sequence is GPL or not and thusly your project has to be GPL'ed too. Not to talk about the other software licenses lets say from Unity engine... Devin *should* already be open source.

From this video, you realize that the smart marketing is crucial in business. They are not the first but now people think they are

@TheGaussFan

3 ай бұрын

Only if you think outright lying is a good strategy. Sometimes, it backfires spectacularly . It leaves a solid core of competent people who recognize the lie.

@rootor1

3 ай бұрын

@@TheGaussFan I wish you were right but reality show every second you are wrong, people is like sheep. That's why we need AI, we need more intelligence than ourselves can show.

@drlordbasil

3 ай бұрын

Doesnt matter how good you are as a programmer sadly.

@user-jg4ci4mf8w

3 ай бұрын

They weren't the first ones but they are the first ones to do it exactly how it was supposed to be done. Anyone who used MetaGPt, GPT-Pilot, Aider etc. wish it had exactly the feature Devin has today.

@JC.72

3 ай бұрын

It really depends on how u define the criteria of first. First to show concept or first to maket or consumer. When iPhone came out although very polished and well-done I wasn't too fascinated simply because I was using PDA windows mobile years before but to be fair iPhone did it much better. And now people will remembers iPhone but will forget windows mobile lol.

I wonder if Devin can create "Devon", an AI coding agent that competes at a similar level using only open source and free to use backend softwares.

That's not a launch. That's a closed beta.

@rootor1

3 ай бұрын

That's they having 21 million$ to spend in marketing and a investor with it's own "social network".

@dirremoire

3 ай бұрын

Or even alpha. Pretty much the whole demo could be done with a bash script.

@wurstelei1356

3 ай бұрын

@@dirremoirePretty much the ultimate Github code theft. Even MS isn't that bold and is providing basic vs code for free.

That poor girl who was was up against that kid. Ooof.

@motess5304

3 ай бұрын

Bro was giving answers before I could even finish reading the question😂

@sethhorras8519

3 ай бұрын

That kid is Scott Woo, the CEO of the company that released Devin

@juliodiaz7778

3 ай бұрын

Devastation

@MrLawalker

3 ай бұрын

@@motess5304The human capacity for memory is awesome af.

So is Devin just an application linked to GPT4, browser, compiler, debugger etc.. It passes the original mission to GPT4, asks for the main steps, takes each step in turn, selects the right API for the required tool, gets the result, goes back to GPT4 to check the text for errors, fixes them, repeats the build, scrapes the browser for additional knowledge and so on? Plus an interface with all the output windows. Or is it doing a lot more than that? I don't know, anyone know how its done?

@sophiophile

3 ай бұрын

I'm guessing it's got a number of agents that are interacting with each other. It also looks like it might be using something like (I think Google's?) ScreenAI.

I really do enjoy your content and look forward to the next one

Devin, no, but there is ZERO doubt in my mind that Devin-like tools will drastically improve over the next couple years where yes, this is now inevitable. This tech is going to make a dramatic dent in available software engineering jobs. Will they be completely eliminated? No. But human software engineers will be comparable to brain surgeons today, only the absolute best will needed.

@Chuck_Hooks

3 ай бұрын

Problem with that scenario is that even the "best" might have no more than a 3-4 year "career" before they have been surpassed by AI learning to improve itself. And not much incentive for would-be programmers to take out big loans to pay for their education knowing their careers will be cut very short.

@daniel4647

3 ай бұрын

It's impossible to find "the best" when all the smartest people decide to learn something else and ignores coding. To find the best you need lots of people competing, most aren't going to compete in a field that is highly likely to be a dead end and waste of time.

@DihelsonMendonca

3 ай бұрын

Absolutely correct. I would write that, but you stated exactly. 🎉❤

@TheJscriptor09

3 ай бұрын

The whole coding thing is rapidly becoming a thing in the past especially if you live in high cost country. Forget AI for a moment .... if you are in the US for example, you have already seen how outsourcing to lower cost SW market works. AI SW engineer (Devin' is the next logical step .... It may not be perfect today, but there will be tons of money poured in this area to make it better in the very near future. The bottom line is that you are right ... the demand for typical SW engineers to do 'Devin like' work will shrink drastically. The demand for folks who can create Devin tools will always be there.

@Al-Storm

3 ай бұрын

​@@daniel4647Won't happen. There's always a market. The best will be found. Just like the best calligraphers are still found, and the market/demand for that is next to nothing.

As others have pointed out, this is a phenomenal breakdown and comparison that tears away the shiny marketing and shows us the reality! If I wasn't already subscribed, this would surely have gotten me to!

I envision Devin as a potential front-line bot on Github for addressing opened issues, capable of delving into newly opened problems and attempting to resolve them and open PR fixes, rank the severity of the issue before escalating them to an actual huaman developer Good for open source projects!

@idck5531

3 ай бұрын

Open source quality and development speed will increase drastically, it will be great for science and academy.

Highly value your content and input, thank you for posting this.

Very interesting vid, love the critical approach, do more of such

how so this channel only got 187k subs... keep your good work my friend.

What's holding a lot of other tools back is how much you need to know about tech to be able to use them. Most require python, git, pip, api keys, and debugging ect ect to make them work. The thing that will really make waves is one you can just run after hitting a "install" button or run in a browser so that most people can easily use it. LM Studio goes a long way to do this, but there's still several speed bumps in the way.

@paul1979uk2000

3 ай бұрын

True, I find that to be the case with a lot of these A.I. projects, they are not easy for most to use and even installing them can be a pain for many lol. Two things that will make A.I. really shine, one is that it's an all-in-one package, so as you say, you just install and use it without having to mess about setting many things up for it to work, whiles it also works on a lot more hardware, which many are restricted to Nvidia hardware. The second thing is where it's all running locally on your hardware, that might not be a big deal to some, but as A.I. becomes more powerful and useful, we are bound to want to use them a lot more in life, so privacy and security is going to become a lot more important, which is where locally run A.I. has a massive advantage over online central ones. Once both of these issues are solved, I can see a big uplift in use cases of A.I. for a lot of us, but for now, A.I. feels like a new toy to experiment with, the real break through is going to be when it becomes really useful for a lot of us to want to use them, which I feel we are still at the experiment stages that things are moving so fast that it's hard to keep track of it. Things are changing fast, hardware is getting more powerful and likely to be better tuned at running A.I. task, the software side will get much better that it's much easier to use, whiles it also runs on far more hardware and all this is likely to happen within 5 years, I suspect much sooner than 5 years but we'll see, but for now, A.I. is good but it's still a bit too technical for most users, especially when trying to run it at a local level, once these walls are broken down, we are going to see an avalanche of use cases when it comes to A.I. and that will be the real game changer, because like anything, the real game changer is when it's mass market with the mainstream.

@sophiophile

3 ай бұрын

That's about as low a bar for entry as I can imagine for anything coding related.

@deezplace

3 ай бұрын

what is "ect" ?

@DalmarWolf

3 ай бұрын

@@deezplace waa supposed to be etc as in et cetera.

Great video and update. Moreover, thanks for not SHOCKING us.

Love this. Thank you for the clarification.

The best review on Devin so far. Thank you!

The main point : this is the worst it will ever be. Deniers may have a point today, as Devin is only V1 ... but V3 in a couple of years will be a whole nuvver story.

@jason_v12345

3 ай бұрын

I mean, you could have said that 3 years ago. Or 10 years ago. Everyone is always saying, "Just imagine where we'll be a few years from now!" Humans have been saying that since the dawn of civilization.

@coldlyanalytical1351

3 ай бұрын

@@jason_v12345 True ... BUT the AI tools provide a non-linear advantage, so the rules have changed for the first time in centuries.

@jijijijijiji44

3 ай бұрын

​@@coldlyanalytical1351 Yeah, AI can be a huge change in the use of technology, similar to the introduction of the Smartphone, or the founding of Facebook and other social media (It effectively became the way to promote ourselves and our businesses as humans)

@brianWreaves

3 ай бұрын

Years??? I reckon it will be weeks. It can use AI to train AI and self improve itself, if they allow it to do so.

I was waiting for you to do something on Devin :D My first response to seeing a thread about Devin and all its marketing-speak was a suspicious squint. One of my side projects at work is getting agents set up as a C# dev team.

Great review and honest review, thanks a lot!

Very interesting as always.

I'm sorry to hear that you are sick 🤒 hope you will recover soon!

Great video!

Are you waiting for AGI friends?😅

@RM-B-2922

3 ай бұрын

AGI will arrive soooooon 😮.

@Nizamuddin78690

3 ай бұрын

Not yet Not soon

@eIicit

3 ай бұрын

@@Nizamuddin78690very soon

@jackflash6377

3 ай бұрын

Has AGI been defined? If so, I would like to see the official definition. What does AGI mean? Does it have anything to do with being sentient?

@RM-B-2922

3 ай бұрын

@@Nizamuddin78690 ohhhhhh 😅

Thanks for your analysis, I know why I love your videos! ❤

A team from Microsoft just released a paper titled "AutoDev: Automated AI-Driven Development". I believe the framework is similar to what Devin is trying to accomplish. I hope they publish the code soon.

@gregparrott

3 ай бұрын

Thanks for the info. Would you by chance know if AutoDev, or any other AI tool can be used to code embedded systems? For example, could someone select a specific ARM processor, define a specific set of attaached components are attached, say, memory, an A/D, a D/A, a USB and a CAN bus, along with discrete pins for input and output, and have it write sufficient code to configure and initialize the ARM?

@kevinanyanwu5427

3 ай бұрын

@@gregparrott that's a cool idea right there. i would not be suprised if there are tools like that in Xilinx

The ending was spot on from your end. It is indeed an agent, running multiple instances of variations. They use between GPT4 and Claude API. Depending on the task, strength and knowledge.

@tag_of_frank

3 ай бұрын

right so the devs lied about that plot, and people trained in ML can notice it, but the investors can't notice it. What else did they lie about? Probably 95% of requests fail in some error-loop, and they only show the 5% of requests that worked in their demos.

I hope you feel better soon

good review, thanks

@thegooddoctor6719

3 ай бұрын

yep agreed ....

You are seriously really cool. Super intelligent and fun to learn from.

Again, Matthew is the most sane commentator on AI. this channel is solid gold.

could you please make a full video on fine-tune mistral or lamma2?, ive been trying too but theres a lot dependencies errors, im sure a lot of people is trying the same and having same problems! your fallowers would love im sure!

1st iteration...1st. AI progress is becoming exponential, which makes it very hard for the human brain to conceptualize. And it's on the edge of reliability training itself. Average code monkeys will be crushed in 5-10 years... Or sooner.

Yea Matthew you are very correct. Devin should be compared with crewAI, Pythagoras and the like. Nice video!

I'm tempted to try to gain access to Devin just to see that he also cannot write an actually functional Nix flake. ChatGPT can't do it, Claude can't do it, Gemini can't do it. Humans can't do it. But some day maybe a futuristic ultra capable supermind might accomplish it.

This begs for an agent comparison video PLEASEEEEEE

can you ask Devin to implement an app like Devin?

@matthew_berman

3 ай бұрын

Someone is doing this but using another AI coding assistant framework.

@rootor1

3 ай бұрын

@@matthew_bermanShhhh... ...don't tell.

@ThomasEWalker

3 ай бұрын

@@matthew_berman When someone manages to do it thoroughly and gives it self-improvement algorithms, it will get out for free, it will work with any model you feed it, it will be everywhere overnight and the World Will Tilt On Its Axis!

Great vid Matt. Would you agree that the latest metagpt can do basically the same?

I think from someone that is more of a creative having struggled with all the other solutions which feel disjointed and still require a lot of technical knowledge, I think this team was right on the money by making everything in a SINGLE system - that is what the mainstream is looking for - ease of use - just my humble opinion

@1:47 cool that it can rerun the code with a debug print statement. But what does Devin do when it encounters an infinite recursion error?

@8:56 other possible solutions for large context issues is memgpt, if they've figured out how to set it up correctly with the right LLM, we'll see some open source implementations (actually I've already seen some them), and yep, don't forget that openai are watching!

Has anyone compared the various agent bundles? I need to use one.

Tools generally beat their competitors and win the public's opinion when they are both powerful and easy to use. It's the iPhone phenomenon. You don't have to reinvent the wheel, just smooth off the rough edges that the other tools have.

Where can I download this tool?

Holy mother of God that guy is insanely good at math.

@rolfnoduk

3 ай бұрын

yes, but mostly it just showed speed

I really appreciate your videos. You’re reasonable, you don’t make bold predictions and you know what you are talking about. I have nothing against predictions but have come to the conclusion that humans are absolutely garbage at predicting even the very near future. All these people saying what the world looks like in 5 years lack awareness of compounding and how quickly the pace of progress accelerates. Also there’s sooo much money flowing into these projects, progress is inevitable

Impressive !

I like Scott Wu. He seems humble unlike some other super smart people in the software industry.

@ontheruntonowhere

3 ай бұрын

Unlike me, for example.

HUGE +1 to open source

big +1 to comparison vs other agents

Devin will be the ChatGPT of Programming, that's if OpenAI decides not to make one of their own and disrupt Cognitive's entire business model.

@user-jg4ci4mf8w

3 ай бұрын

@@polger1739 Do what exactly?

This is already getting super crazy, and it's safe to say we've barely just reached the stone age of AI development. Considering the exponential growth in AI capabilities it's unimaginable what we'll have in the next 5 years.

@hydrohasspoken6227

3 ай бұрын

it will plateau, sooner or later, likely sooner. Ask self driving technology.

@businessmanager7670

3 ай бұрын

@@hydrohasspoken6227 one failed thing is not evidence that other things will fail lol. with tesla and self driving, they didnt innovate enough. just like how other companies didnt innovate enough and open AI showed them real innovation, example: SORA. it's just beyond any other video AI.

@hydrohasspoken6227

3 ай бұрын

@@businessmanager7670 , achieving success in one aspect does not guarantee further success, neither.

@hydrohasspoken6227

3 ай бұрын

@@businessmanager7670 , ok, alright. I will come back to this comment in 20 years to ask you if we already achieved AGI.

@businessmanager7670

3 ай бұрын

@@hydrohasspoken6227 yea sure but you are still wrong in a lot of ways. your claim already gets debunked by other evidence in ai. alphafold solved the protein folding problem in 5 years. scientists around the world attempted to solve this problem for 50+ years and was never able to solve it. this is an example of progress that went up to crazy levels, exceeding the capability of human problem solving. so not everything plateaus lol.

Loved both this and the previous video. Have a look at nVidia's NeMo and let us know if they're going to grab market share on video/graphic development. The product looks really cool.

11:15 is like the Large Action Model from the Rabbit R1

will you be testing Grok/xAi now that it will be open sourced?

Didn’t you get access to Gemini 1.5 pro?

Ai: Dev-in Me: Dev-out

I got your back Matt

Cheers for the video! 🏆 It may not have been the product it claims to be but it is certainly a lesson in the value of marketing. If they used an outside marketing agency, it is going be very busy the next few months.

@tag_of_frank

3 ай бұрын

Yes it shows you can take an existing github repo, put it a fancy sandbox, write AI on it, and investors will throw money as long as the sandbox is shiny and they think the ceo is smart

Thumbs up for a very informative assessment of an AI 'Software Engineer'. Does anyone know if such abilities have been extended to embedded systems? For example, could one select a specific ARM processor, define which components are attached, say, memory, an A/D, a D/A, a USB and a CAN bus, along with discrete pins for input and output, and have it write sufficient code to configure and initialize the ARM?

Devin as in Dev-in ... saw it from Wes Roth's comments.

@jason_v12345

3 ай бұрын

But what does "Dev-in" mean? Dev-ing? Developing?

Just imagine what would such agents show with GPT5 and more context.

great thumbnail

Agree they just did the best in programming on LLMs. In the future, there must be some better tools, that allow more visualization, control diagrams, and test case generation integrated. Currently, I can say that Devin is only available in limited cases.

The main thing I remember from their demo video was how many times they repeated the name Devin. They really wanted us to remember that Devin's name is Devin and that Devin does code, like Devin.

@ontheruntonowhere

3 ай бұрын

Thanks Devin!

how to train for velocity in this type of questions? its impresive but the question are not that dificult

Can you show us how to us Pythagora with LM Studio api, local server, or ollama.. thanks Matthew.

@HassanAllaham

3 ай бұрын

He done it (GPTPilot)... See the links in the description under the video

It is gonna take days at most after it is launched for someone to ask it to write the code.for itself and we get an open-source alternative.

I have never seen an Upwork post with enough details to know even what to build 😂

Their demo is cool. The stated results seem to be the best at the moment. But still 86% cases are failed. And the SWE-Bench is just about 2K pull requests from just 12 popular Python repositories which is not so representative.

@jichaelmorgan3796

3 ай бұрын

Should be above 50% by the end of the year, but who knows who will win

Hope you get well soon ❤ also, is the CEO himself AGI? 😛

can you use Devin to make a better Devin? One thing that I'd like to see on some of these multi agent systems is the used of different language models for different tasks. Use cheaper models for grunt work, and kick things upstairs to the more expensive models for higher functions and de-dugging. Could be especially powerful if you fine tune small open source models on specializing on specific tasks.

Also it can build only those thing on which it was trained on if somehow a new technology or language is asked to code it will faill miserably

did the upwork job start on saturday and finish it on monday😂😂😂

Just released?

The comparison you describe would be great. Maybe you can lead the way and challenge Scott Wu to participate on your terms. However, if he's as bright as it claimed, he'd never get involved.

I hate the name Devin more than Bard, Claude or Grok lol Why can't people name these models well.

@360_SA

3 ай бұрын

They need an AI to come up with a cool nanes😂

@eIicit

3 ай бұрын

Give us examples of what you would consider good names

@jichaelmorgan3796

3 ай бұрын

Fr

@bobnull

3 ай бұрын

i've seen that intro/demo video a few times and it makes me so uncomfortable. "let me show you what Devin can do" _plays sultry background music_

@bobnull

3 ай бұрын

@@eIicit I want to see some retro names like SoftwareBot3000

Please bossman, don't fire me! What am I without your boot on my neck?

So how much did Devin spend on tokens to solve that code-for-pay task? If you end up spending more on OpenAI tokens then you get in, it would still be a losing proposition. These are still early days for automated engineering, but time is accelerating so it is just a matter of time.

Demos are always impressive by the way.

Yes, but can Devin do physical whiteboard interviews? I think not.

We had two leaps in programming productivity since the invetion of computers. First higher level compilers, second knowledge exchange though internet. Now we see the first versions of tools that will bring the rhird leap. None of them are useful for real world projects yet. Even if marketing tell you it is. But we can see the idea and should adapt early.

Developing a truly autonomous AI software engineer that can handle the full breadth and complexity of software development is an extremely ambitious and challenging target with current technologies. Narrowing the problem scope to more specific and constrained fields or types of software engineering problems is probably a more pragmatic path forward, at least in the short-to-medium term future. By reducing the search space and limiting the domain boundaries, we can make the problem more tractable while still pushing the boundaries of what's possible with AI-assisted software development. Reliability, that's the crux of it. Devin's framework might be capable of pulling off some stunts, but sky-high reliability will remain elusive. More complexity equals less reliability - it's a harsh trade-off. Putting on a demo doesn't equate to real-world practicality, nor does it guarantee this framework has legs for the future. It's akin to the autonomous driving saga - ambitions inevitably get downsized to handling much humbler tasks compared to the grand vision of full autonomy.

@idck5531

3 ай бұрын

It wont happen with current LLM models, but LLMs are young, wait 2-3 years and we will have exponentially better LLMs in terms of reasoning ability.

@deter3

3 ай бұрын

@@idck5531 in the science history , when did u see a exponential development ? It always be a slow and hard development . Without another big breakthrough on algo , the development will be slow down .

I'm tempted to try to use it to debug Fortran :) But unfortunately I stopped working with Fortran long ago after a mental break down from my side ::)

So waht would would be the best one in your mind? I'll like to try to build an app/web-based. I am not a software person but I have ideas

we need the training in pythagora

4:30 shows there is no Devin, it is scott doing all the computing via a connection to the internet- a fiber cable connected to his brain.

BFD, anyone can do green field work 😂

Even if AI doesn't take your jobs, it reduce your salaries to negative numbers 😅 which is essentially the same, prepare for UBI

4:21 dude's a savant. I was entered in the national mathematics olympiad at 13 and I couldn't even read the question by the time he hit the button, let alone process it. Absolutely extraordinary. It's a multi step problem, also there are numerous methods to reach the correct answer. The quickest path I can think of: There are 10 separate positions that 1 and 2 can occupy, and 6 permutations of the other 3 spaces for each, 10x6 = 60. He had to process the question, choose his methodology and crunch the permutations, and did it all in about 3 seconds............................ffs

@frbrn

3 ай бұрын

actually very quick to calculate : permutations of 5 distinct digits = factorial(5) = 120, and symmetry guarantees there will be an equal number of sequences with 1 before 2 as there are with 2 before 1, so 60 of each.

@RandomButBeautiful

3 ай бұрын

@@frbrn elegant! I had not thought of that way to do it.

Can you make a video on gptpilot and others vs Devin as you stated. Would be good to see the what real apples to apples look like.

FWIW I built a working copy of a 3D .. not 2D .. version of Life using Claude 3 Opus. It designed it without any input from me. It took 15 minutes total .. with only a few seconds of "Claude time". I had to get Claude to fix 3 silly bugs. No Devin needed. So with experienced coders plus AI and also Devin, we can produce working code very quickly nowadays. The future for inexperienced or less capable software devs is starting to look bleak.

@cacogenicist

3 ай бұрын

The future of less-than-great software devs running one-person shops, producing something useful, would seem to be becoming less bleak.

I'd Devin to evaluate KZread "informational/educational" videos for factual accuracy, and list those in the Suggestions/Recommendations List. That would be useful.

Please made a video on Autogen using Gemini API

The revolution of nerds.

This video only reinforces why you are one of the few "AI influencers" I follow here. I just tell the algorithm not to show me more content from those who are calling it AGI.

What you're also missing is that they're claiming it's production quality off the shelf. Sure we can all build this... But they did

Is Devin GPT4 behind the scene?

Lol they couldn't show scores next to other agents, otherwise they would have abandoned the marketing statement of being "the first". Wait for version 2, then they can say there are others out there.