Q:

masked kernel in convolution in pytorch

Let's say:
x = torch.arange(16, dtype=torch.float).reshape(1, 1, 4, 4)

and a 2d convolution layer is:
layer = torch.nn.Conv2d(in_channels=1, out_channels=1, kernel_size=2, stride=2)
layer.weight.data[:] = 1.
layer.bias.data[:] = 0.

Normally, passing x to layer will give:
>>layer(x)    
tensor([[[[10., 18.],
              [42., 50.]]]], grad_fn=<MkldnnConvolutionBackward>)

Considering having 4 mask filters, how it is done to mask kernel in each step?
for example the following picture indicates 4 filters(white: True, black: False)

The output should be:
tensor([[[[5., 15.],
          [30., 40.]]]], grad_fn=<MkldnnConvolutionBackward>)

P.S: all masks are obtained by missing pixels in 2d input array. So 4 masks above are actually a matrix with the same shape as input.

A:

I found the solution.
Element-wise multiplication between input and the mask before feeding it to a Conv2d method would be enough.(masking input is much easier than masking kernel itself !!):
mask = torch.tensor([[[1, 1, 1, 0]], [[1, 0, 1, 1]], [[1, 1, 0, 1]], [[0, 1, 1, 1]]], dtype=torch.float, requires_grad=False).reshape(1, 1, 4, 4)

>>layer(torch.mul(x, mask))
tensor([[[[5., 15.],
          [30., 40.]]]], grad_fn=<MkldnnConvolutionBackward>)

P.S Thanks to @Shai I got the idea from partial convolution represented in this paper. However it does some extra manipulation on output. it defines a mask ratio and I guess does some weighting the final output based on it.

