Hi thanks for posting this video. Its really helping me to understand variational autoencoders. Could you post the slides? The link doesn't have slides for Karol Gregor's presentation.

Prof. Nando, I am a data scientist that has benefited from some of your lecture/materials. Thanks for uploading this video. While to be honest, YOUR lectures are much clearer and mathematically derivable than this video though I know Karol's a big name. I personally do not agree / not understand what he said in terms of 1. adding noise will limit the number of hidden layer's representation ability, from infinite to finite. Yes, by first glance or very superficially intuitive thinking, it is true. Adding noise will change the consistency of input and make the model not able to capture the relationship between the identical input and different output, and by imagination, the model will compromise to be in some middle state and behave less overfitting. However this is not mathematically holding. The noise is random and can make 2 different input closer, but also possible to be more apart. During the gradient descent process, the model weights will swing and is possible to end up with the one when trained without noise. This is just like an Unbiased Estimator, even if you add random noise to training data, the expectation of the estimates will be the same as the ground truth, and the estimates trained by data without random noise. On the other hand, adding random noise to data is prone to increasing the variation of learned weights, such larger variation might actually increase the overfitting, which is fitting the noise, given NN is so non-linear and powerful. 2. The definition of the loss functions. All the loss functions defined in the lecture are to mimic p(Z|X) with q(z), which is aligning with the variational inference's basic thought. However, I do not see how the probability (density) wise, the value of z itself are standing for a probability, and in variational inference's theory, some hypothesis is prior and posterior distribution conjugation as well as that the joint probability of different Zi's are factorizable, but from a simple auto encoder's structure, I can't see anywhere that bear such property. Are all those a quite simplified version that reflects in nn structure? Thanks, Best regards, Isaac

So basically this is a painting algorithm which generates both the brush texture/color and the location and sharpness of the brush to reconstruct images? This might sound silly since I'm sure there are much better applications for this but, in principle, would it be possible to extract just the brush texture/color part from these as a sort of photoshop brush set? Like, for instance, train it on a bunch of grassland and you get a brush that can paint in highly convincing grass really easily, but you could still decide on the sharpness and location of all that grass on your own.

@jovonnipharr8587

6 жыл бұрын

This is not a painting algorithm.... also researchers commonly use the image use case to illustrate the point, as the data can be anything in practice. Images are usually used as a benchmark, and POC

Does someone understands something about this talk? Everything is super obscure to me. Is there anything more practical with neural networks and a little bit more of explanation on information theory? Like a paper, whatever?

@mahdiyousefi6077

8 жыл бұрын

+draguin He could describe the topic much better!!!

@luyuchen2544

8 жыл бұрын

He basically talks about minimum description length principle of autoencoder and build a connection of it to the variational lower bound.

@paulgay2805

8 жыл бұрын

You can have a look at "tutorial on variational auto-encoder" arxiv.org/pdf/1606.05908.pdf

@arturodeza3816

7 жыл бұрын

I actually think this is a good talk. Way less obscure than the original paper. I think a good call would be to brush up on concepts like Entropy and KL-Divergence.

@flamingxombie

7 жыл бұрын

Oh, and don't forget (along with carl doersch's excellent tutorial) Bishop and Murphy. Together, they remove all obfuscation.

This is a bad way of explaining. I did not understand anything.

@subschallenge-nh4xp

5 жыл бұрын

You r an awfull ingenier