Unity's Groundbreaking Animation Tech You Need to See | Unite 2023 Full Interview

Unite 2023 showcased many amazing features coming to Unity but for some reason nobody is talking about this groundbreaking real-time animation tech! Facial motion capture retargeting? Sign me up! Check out my full interview with Unity ambassador, Mark Schoennagel, to learn more about it!
Learn more about Ziva Dynamics facial motion capture: zivadynamics.com/ziva-face-tr...
Thank you to the Unity Insider team (Kal and Phillip) for the amazing trip!
SUPPORT THE CHANNEL AND GET EXCLUSIVE PERKS:
💛 / iheartgamedev (Project Files Available)
❤️ kzread.info...
WANT MORE?
✅ Check out my full procedural animation series: • Animation Rigging Package
✦ Like the vid? Please consider Subscribing!
bit.ly/2YdIb6j
SOCIAL:
✦ Discord
/ discord
✦ Twitter
/ iheartgamedev
►TIMESTAMPS:
My time at Unite 2023: 0:00
How standard facial animation works: 1:03
How Ziva dynamics facial tech changes the game: 1:40
A little Q&A: 4:19
Facial animation retargeting demo: 5:40
My thoughts: 7:23
#unity3d #proceduralanimation #gamedev

Пікірлер: 117

  • @TheBlackbirdii
    @TheBlackbirdii4 ай бұрын

    Half hour ziva training cost 1500 dollars and your mesh need to follow a specifics rig which it must be exported in Maya ,it,'s crazy expensive for any indie dev

  • @iHeartGameDev

    @iHeartGameDev

    4 ай бұрын

    Woah - that price is incredibly high. Appreciate the info. Hopefully they can do something about that…

  • @lkewis
    @lkewis4 ай бұрын

    Elephant in the room here being how expensive the Ziva tools actually are and not available for free for any of the parts he mentioned as far as I'm aware. I used the Ziva Face Trainer in beta and all it gave you was video animations of the rig in action but not the actual rig.

  • @iHeartGameDev

    @iHeartGameDev

    4 ай бұрын

    Someone else pointed out the price. How can they make this more accessible is definitely a question that needs to be answered.

  • @StanTheManOfThe14thFantaCan
    @StanTheManOfThe14thFantaCan4 ай бұрын

    I used Iphone face capture for a project that was just a VR physics demo and showing the full body interaction for the first time. This will definitely be used when I remake it

  • @ColeWithAGoal
    @ColeWithAGoal4 ай бұрын

    Pretty incredible, thanks for the share

  • @DisCc
    @DisCc4 ай бұрын

    Awesome video man, thank you!

  • @elsam_
    @elsam_4 ай бұрын

    IT'S AMAZING, also congrats for 70k subs

  • @iHeartGameDev

    @iHeartGameDev

    4 ай бұрын

    It blew me away! And thank you! 30K away from the big 100!!

  • @zejugames5045
    @zejugames50454 ай бұрын

    Great work! What metrics would you want to hit to switch to full-time game dev?

  • @jozimastar95
    @jozimastar954 ай бұрын

    İ searched for that and I'm subscribed, i fell stupid İt always pothers me thay 2 min thing take massive amount of GBs , truly a life changing tool

  • @samyam
    @samyam4 ай бұрын

    was great to see you there nicky!

  • @iHeartGameDev

    @iHeartGameDev

    4 ай бұрын

    Always great to see you too, Sam! Hope to collab one day and see you at the next conference

  • @thephilosopher7173

    @thephilosopher7173

    4 ай бұрын

    This is like the first season of Silicon Valley but in gaming instead of music lol

  • @nicolasportu
    @nicolasportu4 ай бұрын

    Keep this subject. Outstanding!

  • @clarkkent1473
    @clarkkent14734 ай бұрын

    Incredible

  • @avocadopictures9706
    @avocadopictures97064 ай бұрын

    Great interview! Does anyone know the name of the free ARKit app that captures facial animation he mentioned?

  • @user-kt6hi3yo3l
    @user-kt6hi3yo3l4 ай бұрын

    Woah! This is sick - awesome tech and had no idea this was possible on the platform. As always amazing content dude. Might take some time for me to get this into my own projects lol, but incredible to see what's on the horizon :)

  • @ninjedi6710
    @ninjedi67104 ай бұрын

    Best unity channel ever!

  • @iHeartGameDev

    @iHeartGameDev

    4 ай бұрын

    thank you :)

  • @benzeji9686
    @benzeji96863 ай бұрын

    Please record a similar video like with the jump, only with forward roll

  • @thegeeeeeeeeee
    @thegeeeeeeeeee4 ай бұрын

    Very cool. Any reason you don’t try your own demo with the beta version? That would be sick

  • @avstract3085
    @avstract30854 ай бұрын

    It looks High Quality cause it takes time to process the whole thing unlike Metahuman Animator

  • @ItzSylvan
    @ItzSylvan4 ай бұрын

    crazy!

  • @iHeartGameDev

    @iHeartGameDev

    4 ай бұрын

    You know it - Cheers Syl! 🍻

  • @terryterry8744
    @terryterry87444 ай бұрын

    This feels like a technology that could be useful in a range of occupations to time save, is it machine learning based or an algorithmic based ai? (I would presume ML but AI is being flung everywhere nowadays)

  • @rkpixeldesign
    @rkpixeldesign4 ай бұрын

    i am really interested in facial animations, humanoid body animations are relatively easy to make but facial animations is a nightmare to create, if we get an AI assisted lip sync system right inside unity that would be a great help. Please tell unity to make it available to general users 😻

  • @iHeartGameDev

    @iHeartGameDev

    4 ай бұрын

    Yes! That would be awesome! Something I wish companies working on AI would learn is that we don’t want to use AI to replacement our craft (generators), rather tools to make our craft easier (enhancers)

  • @arunachalpradesh399

    @arunachalpradesh399

    4 ай бұрын

    one thing you can do is in blender with nvidia omniverse you can sync voice with mouth movement and export it to unity. and its free for nvidia cards. the only problem is have to do import export which can be time consuming but for proffesional it may be good because blender is very powerful

  • @kuromiLayfe

    @kuromiLayfe

    4 ай бұрын

    the biggest issue with facial animation is there is no standard on how to rig it: - you get the do everything using morphs method. - you have the do every single facial muscle with a bone method - you have the mocap point tracked method - and the combine all the above methods each needing more and more processing power as details are getting increased.

  • @Songfugel
    @Songfugel4 ай бұрын

    Owlcat needs to get on this asap since they are still working with Unity (although feom job postings it looks like they are changing to unreal). Just imagine getting even basic level facial animated Pathfinder/Rogue Trader type of games from them ❤

  • @iHeartGameDev

    @iHeartGameDev

    4 ай бұрын

    What’s owlcat??

  • @Songfugel

    @Songfugel

    4 ай бұрын

    Owlcat games, one of the biggest game studios using Unity @@iHeartGameDev

  • @adamodimattia
    @adamodimattia4 ай бұрын

    Ziva is not included in the engine but has a separate fee which is not cheap if I understand well?

  • @harindersingh5551
    @harindersingh55514 ай бұрын

    7:59 Motion capture is an amazing topic to cover due to the rise of many ai tool. It allows to convey complex emotion even with very primitive motion performances. We would love to have some of your very thorough explainer videos on this amazing tech.

  • @iHeartGameDev

    @iHeartGameDev

    4 ай бұрын

    Awesome to hear! I’ve always been fascinated by the tech myself so getting to jump in and try it out myself would be awesome

  • @lukegf0
    @lukegf04 ай бұрын

    Thanks for another great video! Yes, I'd like to see a video about motion capture, although I am more interested in whole body animations, as opposed to facial animations. I have been looking for a meditation animation for a long time and no luck yet. I am thinking of maybe making my own by using motion capture. I know you can do that with Xbox Kinnect, although there are probably better tools now. Any suggestions on that?

  • @PotatoManager420
    @PotatoManager4204 ай бұрын

    What does he mean by saying "it's a static mesh"? Does it mean a simple skinned mesh with rigs? Just like any ordinary animated character? Or there are no rigs and every vertex is controlled by some vertex groups and weights?

  • @iHeartGameDev

    @iHeartGameDev

    4 ай бұрын

    I'm pretty sure it's the latter

  • @npatch

    @npatch

    4 ай бұрын

    ⁠​⁠​⁠@@iHeartGameDevthe latter sounds like blendshapes, I’m thinking it’s the former. Creating blendshapes is the norm anyhow in mocap software. Analyzing vertex animation and creating a rig for it sounds like the least storage intensive solution out of the two. In mocap there are standards that might go up to 52 bledshapes for facial movap with a good degree of accuracy. And usually mocap ends up too dense in data. So if they managed to create interpolate heavy animations on bones, it would def be the win.

  • @crisgriffin3042

    @crisgriffin3042

    4 ай бұрын

    Exactly, pretty fishy at the very least. Like who ever does that? You don't bake the whole scene as frame-by-frame series of static meshes, even in cinema, cause it have practically zero sense and just a big hassle with not pros.

  • @FetaleKetones
    @FetaleKetones4 ай бұрын

    Shoutout Blender users 😂 Can’t wait

  • @iHeartGameDev

    @iHeartGameDev

    4 ай бұрын

    Blender rocks!

  • @MCA-GK
    @MCA-GK4 ай бұрын

    What about unreal engine motion capture for metahuman

  • @Diablokiller999
    @Diablokiller9994 ай бұрын

    Always wonder how americans feel about europe after a first visit, for most I know it's a completely different world for them i.e. public transportation, city building, precinct etc. Maybe you can do a video about the trip, your time at Unite and the time spent with the other content creators? It's less game related, maybe just for your Patrons, I don't know - just would be interesting to me what you think about the Netherlands :D

  • @BluesInSeattle

    @BluesInSeattle

    4 ай бұрын

    It's actually very similar to the states depending on location. But there are places in the States that are not.

  • @sarahm2005
    @sarahm20054 ай бұрын

    Watching things like this, makes me come out of my cave mind for a second, and think about how much I don't know and need to learn. Then I go back to the cave because it's easier to live there 😂.

  • @Sim2322
    @Sim23224 ай бұрын

    I'm kind of not too sure about this. My thinking is that, soon enough, AI will have no real problem just doing mouth and face animations just from an audio file with text recognition. I'm all about making mocap easier and more accessible, but it seems to me that, for the face animations themselves, this is kind of an in-between solution that won't last very long if AI tools continue to evolve at a similar rate.

  • @paul1979uk2000

    @paul1979uk2000

    4 ай бұрын

    I suspect that at the rate A.I. is developing at, it's going to be able to do most of the jobs we can do but better, faster and cheaper. I don't think there are any limits in what it can do, after all, A.I. unlike other tools we've created is flexible in that it can be used in more or less everything, that's going to make A.I. very potent in the years and decades to come that creating things would really be about skills and more about imagination. I do wonder about the job market, because there's no doubt in my mind that A.I. is going to take over a lot of jobs and it could become an avalanche in a short space of time that our system isn't designed around to cope with and I know some will say that we'll create new jobs and that's true, but realistically, any new jobs we create, A.I. is likely going to be able to do that job better, cheaper and faster than we can do, it's going to be interesting to see how the world adapts to this tech and I suspect some kind of universal basic income is going to be needed as I doubt there's going to be enough jobs to go around, after all, if you are a business, why would you employ humans when A.I. and robotics can do the same work better, cheaper and faster than we can?

  • @Tech-is1xy
    @Tech-is1xy4 ай бұрын

    Oh, so it's Unity's response to Unreal's Metahuman Animator. However instead of it being free, it comes with the bargain price of $1800. Thanks Unity 👍

  • @iHeartGameDev

    @iHeartGameDev

    4 ай бұрын

    Hopefully they can do something about that price. While the performances seem to be a higher fidelity than Metahuman animator, the cost is so high

  • @Metarig

    @Metarig

    4 ай бұрын

    Seriously? With that kind of money, you could pay someone like me to enhance the metahuman, customizing it perfectly without altering the original design, and you wouldn't have to lift a finger.

  • @containedhurricane

    @containedhurricane

    4 ай бұрын

    Adobe/ Autodesk enforced their subscription-based monetization models to please their stockholders and slow down their technological development after going public on the stock market. I wouldn't be surprised if Unity will follow suit, after making bad billion-dollar acquisitions

  • @MangaGamify

    @MangaGamify

    4 ай бұрын

    Yup, price is always the thing I try to find, and they always made me jump through hoops to find/know it. Doesn't matter if it's a little bit better of that little thing costs not so little $1800. With that price it's like they want us to be their bread-winner or child-supporter. Not saying it should be free but as someone said "If paying is not owning, then pirating is not stealing."

  • @Rollthered
    @Rollthered4 ай бұрын

    Mind you this is also the company that just axed Weta Digital almost entirely. Take that however you may.

  • @bad_uncle
    @bad_uncle4 ай бұрын

    This is old news. Once more, this isn’t Ziva motion capture. The animation comes from 4D. The highlight here is the Ziva rig. It’s very nice, but with no high fidelity system to drive the rig (such as MetaHuman animator), I’m stuck with substandard facial animation. So no mocap, and a rig that’s too expensive to compete with Metahuman, why should be interested in this? My biggest hope is that Unity sells Ziva to someone with deep enough pockets and knows what to do with it.

  • @iHeartGameDev
    @iHeartGameDev4 ай бұрын

    What do y'all think of this tech? Should we start covering motion capture animation tech on this channel? It's something I've always been interested in! I hadn't heard of Ziva prior to walking around the Unite floor but it blew me away the moment I stopped to give it a look. From some research, it looks like it's been in beta for a while but has been used in quite a few games!

  • @fuyuminekimaya7571

    @fuyuminekimaya7571

    4 ай бұрын

    is this also useful for less realistic more stylized games?

  • @mahanthkanaparthi6087

    @mahanthkanaparthi6087

    4 ай бұрын

    I have been following this channel since the first video uploaded and have been learning a lot about animations in unity. I would love content on the new tech ziva in this channel. I love your teaching style. I hope we will get a lecture on ziva in this channel soon.

  • @dreamcatforgotten8435

    @dreamcatforgotten8435

    4 ай бұрын

    While it is true that Alembic(sp) Texture animation is much larger on memory, it doesn't require much or any CPU cost, meaning lower power drain from a Mobile perspective. What I'm curious about is, what is the average CPU cost of the 50 MB solution during the same cutscene/Animation?

  • @ArnoVai

    @ArnoVai

    4 ай бұрын

    All Ziva products costs several thousand dollars to use, with continued costs the more assets you need. It's AAA and Hollywood level tech in pricing, not relevant to the rest of us.

  • @BluesInSeattle

    @BluesInSeattle

    4 ай бұрын

    I'm an indie developer, and the cost of this is out of reach.

  • @Hoptronics
    @Hoptronics10 күн бұрын

    It appears that unity kept the company but scrapped the product.. wow.. hopefully I'll be incorporated into the Unity Engine.. it'd make a great package..

  • @MrSongib
    @MrSongib4 ай бұрын

    EZ clap

  • @crazyfingers619
    @crazyfingers6194 ай бұрын

    I just want to make a fun game. Nothing Unity has done in years makes this any easier.

  • @salvadorian3288

    @salvadorian3288

    4 ай бұрын

    True, in this case its not about the engine.

  • @Hoptronics
    @Hoptronics10 күн бұрын

    So roughly 1gigabit/sec

  • @265Den
    @265Den4 ай бұрын

    Ля...Я думал, что он объяснит, как уменьшить размер билда, а тут какие-то будующие технологии... =(

  • @moritakaishida7963
    @moritakaishida79634 ай бұрын

    No ai generated facial animations are ever going to be able to beat what an artist can do with their hands. Motion capture with hand animation mixed together is still and will always be the best way to animate facial expressions in 3D animation and games, shit like this is just an excuse to be lazy

  • @iHeartGameDev

    @iHeartGameDev

    4 ай бұрын

    I think this is machine learning not so generated. It takes photogrammetry and creates a rig out of it for artists to use

  • @Nessinby
    @Nessinby4 ай бұрын

    so, it's a bunch of fancy talk for an auto-rigger?

  • @Metarig
    @Metarig4 ай бұрын

    As someone with some knowledge of 3D, I must say what I observed seemed rather amateurish and simply doesn't hold a candle to Metahuman.

  • @MangaGamify

    @MangaGamify

    4 ай бұрын

    Also, for something almost no one uploaded results, you're risking $1800. They probably want to get back all the ROI with all the tech industry they bought.

  • @GameDevBox
    @GameDevBox4 ай бұрын

    People are so negative about Unity if this tool is not for you just use another tool and stop watching Unity-related videos! some of them just started their journey and think tools should do everything they want they don't know anything about video game production. if you are not certain about it just go with other tools. there are tons of good games made with unity so there is no reason for hate or just you have skill issues.

  • @PaulMucci

    @PaulMucci

    4 ай бұрын

    Absolutely. ZIVA is amazing.

  • @elakstein

    @elakstein

    4 ай бұрын

    I love the game Dyson sphere program which is made in unity. People show hate because it seems to them the engine can go in a direction to make it the best in the market but it's not going.

  • @ragerungames

    @ragerungames

    4 ай бұрын

    Totally agree on this. I hate those who spreading hate for unity. Unity is so great. It helped me and idk how many others finding their dream career. I’m very happy using unity

  • @mohammadtausifrafi8277

    @mohammadtausifrafi8277

    4 ай бұрын

    Which engine is better than Unity? UE, maybe? If that. But what else? Is any other engine even close to Unity yet?

  • @GameDevBox

    @GameDevBox

    4 ай бұрын

    @@elakstein totally agree

  • @CouchWarrior561
    @CouchWarrior5614 ай бұрын

    Lol, they just copy pasted Unreal's Metahuman facial animation rig, truly game a changer 😂 Are they planning to fix bugs in Unity's previous "game changer" features or it will be forever in preview state usable only for demos?

  • @iHeartGameDev

    @iHeartGameDev

    4 ай бұрын

    It might be an animation rigging solution, but it’s super high fidelity. I’d be curious to do a comparison between the two

  • @Starius2
    @Starius24 ай бұрын

    i dont care how "cool" it is. Unity burned my bridge.

  • @MangaGamify

    @MangaGamify

    4 ай бұрын

    A random reminder just incase one day you're force to use it. You can always use a pirated Unity, (it's pro only versions are unlocked. )

  • @Starius2

    @Starius2

    4 ай бұрын

    @universalmuiscgroup legally I can only make personal games with that

  • @akshatmishra5527
    @akshatmishra55274 ай бұрын

    Isnt this what unreal engines power ik been doing since 2 years

  • @DaviMori

    @DaviMori

    4 ай бұрын

    yeah, but Unity isn't Unreal Engine.

  • @MangaGamify

    @MangaGamify

    4 ай бұрын

    @@DaviMori Could be great if it's cheap if not free, but with $1800 it seems they're turning us into their bread-winner or child-supporter for all the tech companies they acquired xD

  • @Camtrack
    @Camtrack4 ай бұрын

    unity next big thing to get deprecated before launch

  • @iHeartGameDev

    @iHeartGameDev

    4 ай бұрын

    Hoping the change in leadership means a re-focus on maintaining and enhancing core tech!

  • @MangaGamify

    @MangaGamify

    4 ай бұрын

    @@iHeartGameDev them short-term gains for them quarterly bonus, why wouldn't people who only last

  • @sweetybnz7482
    @sweetybnz74824 ай бұрын

    When will developers get access to it? Unity will screw developers again and charge them thousands for access.

  • @pewpew518
    @pewpew5184 ай бұрын

    Bro unreal does all than and more with an iPhone.... Yet another example of unity not knowing what the hell they are doing.

  • @emberchord
    @emberchord4 ай бұрын

    Gurl, this conference must have been awkward lol

  • @iHeartGameDev

    @iHeartGameDev

    4 ай бұрын

    To be honest, it was actually a really positive experience. Spirits were higher than expected after the whole debacle and it felt like there was a focus on steering the ship in the right direction

  • @emberchord

    @emberchord

    4 ай бұрын

    @@iHeartGameDev happy to hear that!

  • @ZaynNaz
    @ZaynNaz4 ай бұрын

    Real shame unitys rep has went downhill ever since their last CEO. I myself dropped Unity after that. Can’t trust the company.

  • @deadpianist7494
    @deadpianist74944 ай бұрын

    No thanks.