[FFmpeg-devel] [PATCH 2/2] libavfilter: Removes stored DNN models. Adds support for native backend model file format in tf backend.

[Sergey Lavrushkin]

2018-08-17 17:46 GMT+03:00 Pedro Arthur <bygrandao at gmail.com>:

> Hi,
>
> You did not provided any pre trained model files, so anyone trying to
> test it has to perform the whole training!
> I'm attaching the models I generated, if anyone is interested in testing
> it.
>
> When applying the filter with tf backend there are artifacts in the
> borders, for both srcnn and espcn (out_[srcnn|espcn]_tf.jpg).
> It seems that a few lines in the top row of the image are repeated for
> espcn using native backend (out_srcnn_nt.jpg).
>

I guess, it is because I didn't add any padding to the image and tf fills
borders with 0 for 'SAME' padding in convolutions. I'll add required padding
size calculation and insert padding operation to the graph.


> The model/model_filename options are not coherent, the model type
> should be defined in the file anyway therefore there is no need for
> both options.
> It is also buggy, if you specify the model_filename but not the model
> type it will default to srcnn even if the model file is for espcn, no
> error is generated and the output ofc is buggy.
>

I think, I can remove model type and check if model changes input size.
I think all my switches for model type actually depend on this condition.

If I remove conversions inside the filter and make it to work only for
one plane, it basically will become a filter that executes neural network
for one channel input. But there is a problem with float format - it brokes
fate on some 32 bit hosts, as James stated, and I need first to fix this
issue, or, otherwise, revert to doing conversions in the filter.


> I personally would prefer to use only model=file as it is shorter than
> model_filename=file.