Resnet_2

 While building a deep network there are two problems:

1. in the forward propagation, the last few layers have almost no information about what the original image was.

2. in the backpropagation, the first few layers near the input hardly get any gradient update due to the vanishing gradient. 

to solve this problem, resnet use highway like connection and transfer raw information from the previous layer to later layers.




Till now we have used linear or non-linear transformation but here for 1st time we are not only transforming but add the raw input into transformation.




Comments

Post a Comment

Popular posts from this blog

how to handle the imbalance class

support,  our task is to identify rare lung infection and 99% of images are of normal lung. then even we get 99% accuracy that still been useless. Even though we get 99% accuracy, how to identify, we have achieved our goal or not. The confusion matrix comes to the rescue. by this, we can find out, out of how many actual points, rare class predicted correctly. But here the question is how to identify this when the model is training, typically binary or categorical cross-entropy ensures that when misclassification is high, the loss is also very high. however, we can assign a higher weight to rarely occurring classes.  2. we can also use image augmentation, we can oversample the rare class image by increasing their mix with the overall population.
Read more

How to calculate the number of Parameters

 Suppose we have a CNN layer, input is - 1*1*28*28 (batch_size * channels * height * width ) nn.Conv2d(1, 64, kernel_size=3) means here input channel is 1 and the ouput channel is 64. the kernel is of size 3 X 3  How to find a number of parameters? (kernel width * kernel height * number of channel in the previous layer + 1)  * Number of filters of the CNN (3 * 3 * 1 + 1) * 64 = 640 parameters How to find CNN output size? [(W-K+2P) / S] + 1 if you have an image of size 1*28*28 where 1 is a channel, 28 is height, and 28 width. so Input shape = 1*28*28 W- width of an image K - Kernel size P - padding S - Stride in our case output size is [(28-3+2*0)/1]+1 = 25 Output shape : 1* 25*25
Read more