[RFC/PATCH] MV-HEVC decoding
Hi, this patchset adds support for decoding MV-HEVC (multiview, typically used for stereoscopic 3D video). The code currently supports at most two views, with the second depending on the first (i.e. using it for inter prediction). Both views have to be of the same size; the spec allows them not to be, but I'm not supporting that, or a lot of other insanity in there. The frames for both views are interleaved in the output, with side data indicating which one has which view ID. If the relevant SEI is present, we also add side data indicating which view is left or right. Since the API caller needs to request the views it wants, and all the multilayer configuration is potentially per-sequence, there needs to be a negotiation process where the decoder exports available views and the caller picks from them. I'm implementing this by adding several decoder-private AVOptions, meant to be accessed from get_format(). Some of them are read-only and the caller reads available view IDs (and positions, if known) from them. The others are writable, and are used to receive the caller's choice. One potential point of discussion is that the caller may want to know the view ID in its get_buffer2() callback (this is actually used in the ffmpeg CLI code). One potential option is attaching the view ID side data before calling get_buffer2(), but that is quite unusual and would require changes to the progress frame API. For now, I'm using another read-only decoder-private AVOption for that. Opinions welcome. I'm also adding nontrivial support for this to ffmpeg CLI, which mainly means extending stream specifiers to allow specifying individual views, e.g. "-map 0:v:0:vpos:left" selects the left view (see TFM for more details). The set should be mostly functional, I'm tagging it as RFC because there's a little polishing left to do, and a few tests left to add. Testing and review welcome. You can fetch the set from git://git.khirnov.net/libav, branch 'mv-hevc'. Cheers, -- Anton Khirnov
Previously one could only replace the entire array with a new one deserialized from a string. The new API allows inserting, replacing, and removing arbitrary element ranges. --- doc/APIchanges | 3 + libavutil/opt.c | 186 ++++++++++++++++++++++++++++++++++++++++++++ libavutil/opt.h | 56 +++++++++++++ libavutil/version.h | 2 +- 4 files changed, 246 insertions(+), 1 deletion(-) diff --git a/doc/APIchanges b/doc/APIchanges index 226c6f8b10..9c3eeffff5 100644 --- a/doc/APIchanges +++ b/doc/APIchanges @@ -2,6 +2,9 @@ The last version increases of all libraries were on 2024-03-07 API changes, most recent first: +2024-09-xx - xxxxxxxxx - lavu 59.36.100 - opt.h + Add av_opt_set_array() and AV_OPT_ARRAY_REPLACE. + 2024-08-xx - xxxxxxxxx - lavu 59.35.100 - opt.h Add av_opt_get_array_size() and av_opt_get_array(). diff --git a/libavutil/opt.c b/libavutil/opt.c index d515e20e97..e07ec8ce0f 100644 --- a/libavutil/opt.c +++ b/libavutil/opt.c @@ -2244,6 +2244,192 @@ fail: return ret; } +int av_opt_set_array(void *obj, const char *name, int search_flags, + unsigned int start_elem, unsigned int nb_elems, + enum AVOptionType val_type, const void *val) +{ + const size_t elem_size_val = opt_elem_size[TYPE_BASE(val_type)]; + + const AVOption *o; + const AVOptionArrayDef *arr; + void *target_obj; + + void *parray; + void *new_elems; + unsigned *array_size, new_size; + size_t elem_size; + + int ret; + + o = av_opt_find2(obj, name, NULL, 0, search_flags, &target_obj); + if (!o || !target_obj) + return AVERROR_OPTION_NOT_FOUND; + if (!(o->type & AV_OPT_TYPE_FLAG_ARRAY) || + (val_type & AV_OPT_TYPE_FLAG_ARRAY)) + return AVERROR(EINVAL); + + arr = o->default_val.arr; + parray = (uint8_t *)target_obj + o->offset; + array_size = opt_array_pcount(parray); + elem_size = opt_elem_size[TYPE_BASE(o->type)]; + + if (start_elem > *array_size) + return AVERROR(EINVAL); + + // compute new array size + if (!val) { + if (*array_size - start_elem < nb_elems) + return AVERROR(EINVAL); + + new_size = *array_size - nb_elems; + } else if (search_flags & AV_OPT_ARRAY_REPLACE) { + if (start_elem >= UINT_MAX - nb_elems) + return AVERROR(EINVAL); + + new_size = FFMAX(*array_size, start_elem + nb_elems); + } else { + if (nb_elems >= UINT_MAX - *array_size) + return AVERROR(EINVAL); + + new_size = *array_size + nb_elems; + } + + if (arr && + ((arr->size_max && new_size > arr->size_max) || + (arr->size_min && new_size < arr->size_min))) + return AVERROR(EINVAL); + + // desired operation is shrinking the array + if (!val) { + void *array = *(void**)parray; + + for (unsigned i = 0; i < nb_elems; i++) { + opt_free_elem(o->type, + opt_array_pelem(o, array, start_elem + i)); + } + + if (new_size > 0) { + memmove(opt_array_pelem(o, array, start_elem), + opt_array_pelem(o, array, start_elem + nb_elems), + elem_size * (*array_size - start_elem - nb_elems)); + + array = av_realloc_array(array, new_size, elem_size); + if (!array) + return AVERROR(ENOMEM); + + *(void**)parray = array; + } else + av_freep(parray); + + *array_size = new_size; + + return 0; + } + + // otherwise, desired operation is insert/replace; + // first, store new elements in a separate array to simplify + // rollback on failure + new_elems = av_calloc(nb_elems, elem_size); + if (!new_elems) + return AVERROR(ENOMEM); + + // convert/validate each new element + for (unsigned i = 0; i < nb_elems; i++) { + void *dst = opt_array_pelem(o, new_elems, i); + const void *src = (uint8_t*)val + i * elem_size_val; + + double num = 1.0; + int den = 1; + int64_t intnum = 1; + + if (val_type == TYPE_BASE(o->type)) { + ret = opt_copy_elem(obj, val_type, dst, src); + if (ret < 0) + goto fail; + + // validate the range for numeric options + ret = read_number(o, dst, &num, &den, &intnum); + if (ret >= 0 && TYPE_BASE(o->type) != AV_OPT_TYPE_FLAGS && + (!den || o->max * den < num * intnum || o->min * den > num * intnum)) { + num = den ? num * intnum / den : (num && intnum ? INFINITY : NAN); + av_log(obj, AV_LOG_ERROR, "Cannot set array element %u for " + "parameter '%s': value %f out of range [%g - %g]\n", + start_elem + i, o->name, num, o->min, o->max); + ret = AVERROR(ERANGE); + goto fail; + } + } else if (val_type == AV_OPT_TYPE_STRING) { + ret = opt_set_elem(obj, target_obj, o, *(const char **)src, dst); + if (ret < 0) + goto fail; + } if (val_type == AV_OPT_TYPE_INT || + val_type == AV_OPT_TYPE_INT64 || + val_type == AV_OPT_TYPE_FLOAT || + val_type == AV_OPT_TYPE_DOUBLE || + val_type == AV_OPT_TYPE_RATIONAL) { + int ret; + + switch (val_type) { + case AV_OPT_TYPE_INT: intnum = *(int*)src; break; + case AV_OPT_TYPE_INT64: intnum = *(int64_t*)src; break; + case AV_OPT_TYPE_FLOAT: num = *(float*)src; break; + case AV_OPT_TYPE_DOUBLE: num = *(double*)src; break; + case AV_OPT_TYPE_RATIONAL: intnum = ((AVRational*)src)->num; + den = ((AVRational*)src)->den; break; + default: av_assert0(0); + } + + ret = write_number(obj, o, dst, num, den, intnum); + if (ret < 0) + goto fail; + } else { + ret = AVERROR(ENOSYS); + goto fail; + } + } + + // commit new elements to the array + if (start_elem == 0 && nb_elems == new_size) { + // replacing the existing array entirely + opt_free_array(o, parray, array_size); + *(void**)parray = new_elems; + *array_size = nb_elems; + + new_elems = NULL; + nb_elems = 0; + } else { + void *array = av_realloc_array(*(void**)parray, new_size, elem_size); + if (!array) { + ret = AVERROR(ENOMEM); + goto fail; + } + + if (search_flags & AV_OPT_ARRAY_REPLACE) { + // free the elements being overwritten + for (unsigned i = start_elem; i < FFMIN(start_elem + nb_elems, *array_size); i++) + opt_free_elem(o->type, opt_array_pelem(o, array, i)); + } else { + // shift existing elements to the end + memmove(opt_array_pelem(o, array, start_elem + nb_elems), + opt_array_pelem(o, array, start_elem), + elem_size * (*array_size - start_elem)); + } + + memcpy((uint8_t*)array + elem_size * start_elem, new_elems, elem_size * nb_elems); + + av_freep(&new_elems); + nb_elems = 0; + + *(void**)parray = array; + *array_size = new_size; + } + +fail: + opt_free_array(o, &new_elems, &nb_elems); + + return ret; +} + int av_opt_query_ranges(AVOptionRanges **ranges_arg, void *obj, const char *key, int flags) { int ret; diff --git a/libavutil/opt.h b/libavutil/opt.h index cf9ebb9b12..be189f7653 100644 --- a/libavutil/opt.h +++ b/libavutil/opt.h @@ -618,6 +618,12 @@ const AVClass *av_opt_child_class_iterate(const AVClass *parent, void **iter); */ #define AV_OPT_ALLOW_NULL (1 << 2) +/** + * May be used with av_opt_set_array() to signal that new elements should + * replace the existing ones in the indicated range. + */ +#define AV_OPT_ARRAY_REPLACE (1 << 3) + /** * Allows av_opt_query_ranges and av_opt_query_ranges_default to return more than * one component for certain option types. @@ -896,6 +902,56 @@ int av_opt_set_dict_val(void *obj, const char *name, const AVDictionary *val, in av_opt_set_bin(obj, name, (const uint8_t *)(val), \ av_int_list_length(val, term) * sizeof(*(val)), flags)) +/** + * Add, replace, or remove elements for an array option. Which of these + * operations is performed depends on the values of val and search_flags. + * + * @param start_elem Index of the first array element to modify; must not be + * larger than array size as returned by + * av_opt_get_array_size(). + * @param nb_elems number of array elements to modify; when val is NULL, + * start_elem+nb_elems must not be larger than array size as + * returned by av_opt_get_array_size() + * + * @param val_type Option type corresponding to the type of val, ignored when val is + * NULL. + * + * The effect of this function will will be as if av_opt_setX() + * was called for each element, where X is specified by type. + * E.g. AV_OPT_TYPE_STRING corresponds to av_opt_set(). + * + * Typically this should be the same as the scalarized type of + * the AVOption being set, but certain conversions are also + * possible - the same as those done by the corresponding + * av_opt_set*() function. E.g. any option type can be set from + * a string, numeric types can be set from int64, double, or + * rational, etc. + * + * @param val Array with nb_elems elements or NULL. + * + * When NULL, nb_elems array elements starting at start_elem are + * removed from the array. Any array elements remaining at the end + * are shifted by nb_elems towards the first element in order to keep + * the array contiguous. + * + * Otherwise (val is non-NULL), the type of val must match the + * underlying C type as documented for val_type. + * + * When AV_OPT_ARRAY_REPLACE is not set in search_flags, the array is + * enlarged by nb_elems, and the contents of val are inserted at + * start_elem. Previously existing array elements from start_elem + * onwards (if present) are shifted by nb_elems away from the first + * element in order to make space for the new elements. + * + * When AV_OPT_ARRAY_REPLACE is set in search_flags, the contents + * of val replace existing array elements from start_elem to + * start_elem+nb_elems (if present). New array size is + * max(start_elem + nb_elems, old array size). + */ +int av_opt_set_array(void *obj, const char *name, int search_flags, + unsigned int start_elem, unsigned int nb_elems, + enum AVOptionType val_type, const void *val); + /** * @} * @} diff --git a/libavutil/version.h b/libavutil/version.h index 5ac9cc59dc..25a6f5531b 100644 --- a/libavutil/version.h +++ b/libavutil/version.h @@ -79,7 +79,7 @@ */ #define LIBAVUTIL_VERSION_MAJOR 59 -#define LIBAVUTIL_VERSION_MINOR 35 +#define LIBAVUTIL_VERSION_MINOR 36 #define LIBAVUTIL_VERSION_MICRO 100 #define LIBAVUTIL_VERSION_INT AV_VERSION_INT(LIBAVUTIL_VERSION_MAJOR, \ -- 2.43.0
Fixes operations on array UINT options. --- libavutil/opt.c | 1 + 1 file changed, 1 insertion(+) diff --git a/libavutil/opt.c b/libavutil/opt.c index e07ec8ce0f..d2af76478c 100644 --- a/libavutil/opt.c +++ b/libavutil/opt.c @@ -98,6 +98,7 @@ static int opt_is_pod(enum AVOptionType type) case AV_OPT_TYPE_DURATION: case AV_OPT_TYPE_COLOR: case AV_OPT_TYPE_BOOL: + case AV_OPT_TYPE_UINT: return 1; } return 0; -- 2.43.0
--- MAINTAINERS | 1 + 1 file changed, 1 insertion(+) diff --git a/MAINTAINERS b/MAINTAINERS index 1f758f401c..d79920d9d6 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -174,6 +174,7 @@ Codecs: h263* Michael Niedermayer h264* Loren Merritt, Michael Niedermayer hap* Tom Butterworth + hevc/* Anton Khirnov huffyuv* Michael Niedermayer idcinvideo.c Mike Melanson interplayvideo.c Mike Melanson -- 2.43.0
--- doc/APIchanges | 3 +++ fftools/ffprobe.c | 2 ++ libavfilter/vf_showinfo.c | 2 ++ libavutil/frame.c | 1 + libavutil/frame.h | 9 +++++++++ libavutil/version.h | 2 +- 6 files changed, 18 insertions(+), 1 deletion(-) diff --git a/doc/APIchanges b/doc/APIchanges index 9c3eeffff5..a8f84ca45f 100644 --- a/doc/APIchanges +++ b/doc/APIchanges @@ -2,6 +2,9 @@ The last version increases of all libraries were on 2024-03-07 API changes, most recent first: +2024-xx-xx - xxxxxxx - lavu 59.37.100 - frame.h + Add AV_FRAME_DATA_VIEW_ID. + 2024-09-xx - xxxxxxxxx - lavu 59.36.100 - opt.h Add av_opt_set_array() and AV_OPT_ARRAY_REPLACE. diff --git a/fftools/ffprobe.c b/fftools/ffprobe.c index bf5ebe3ce0..14b98d22a1 100644 --- a/fftools/ffprobe.c +++ b/fftools/ffprobe.c @@ -2920,6 +2920,8 @@ static void print_frame_side_data(WriterContext *w, } else if (sd->type == AV_FRAME_DATA_FILM_GRAIN_PARAMS) { AVFilmGrainParams *fgp = (AVFilmGrainParams *)sd->data; print_film_grain_params(w, fgp); + } else if (sd->type == AV_FRAME_DATA_VIEW_ID) { + print_int("view_id", *(int*)sd->data); } writer_print_section_footer(w); } diff --git a/libavfilter/vf_showinfo.c b/libavfilter/vf_showinfo.c index f81df9d1bf..77082505f5 100644 --- a/libavfilter/vf_showinfo.c +++ b/libavfilter/vf_showinfo.c @@ -857,6 +857,8 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *frame) case AV_FRAME_DATA_AMBIENT_VIEWING_ENVIRONMENT: dump_ambient_viewing_environment(ctx, sd); break; + case AV_FRAME_DATA_VIEW_ID: + av_log(ctx, AV_LOG_INFO, "view id: %d\n", *(int*)sd->data); default: if (name) av_log(ctx, AV_LOG_INFO, diff --git a/libavutil/frame.c b/libavutil/frame.c index 5cbfc6a48b..891909fc2a 100644 --- a/libavutil/frame.c +++ b/libavutil/frame.c @@ -46,6 +46,7 @@ static const AVSideDataDescriptor sd_props[] = { [AV_FRAME_DATA_DETECTION_BBOXES] = { "Bounding boxes for object detection and classification" }, [AV_FRAME_DATA_DOVI_RPU_BUFFER] = { "Dolby Vision RPU Data" }, [AV_FRAME_DATA_DOVI_METADATA] = { "Dolby Vision Metadata" }, + [AV_FRAME_DATA_VIEW_ID] = { "View ID" }, [AV_FRAME_DATA_STEREO3D] = { "Stereo 3D", AV_SIDE_DATA_PROP_GLOBAL }, [AV_FRAME_DATA_REPLAYGAIN] = { "AVReplayGain", AV_SIDE_DATA_PROP_GLOBAL }, [AV_FRAME_DATA_DISPLAYMATRIX] = { "3x3 displaymatrix", AV_SIDE_DATA_PROP_GLOBAL }, diff --git a/libavutil/frame.h b/libavutil/frame.h index 60bb966f8b..cea1c68df5 100644 --- a/libavutil/frame.h +++ b/libavutil/frame.h @@ -228,6 +228,15 @@ enum AVFrameSideDataType { * encoding. */ AV_FRAME_DATA_VIDEO_HINT, + + /** + * This side data must be associated with a video frame. + * The presence of this side data indicates that the video stream is + * composed of multiple views (e.g. stereoscopic 3D content, + * cf. H.264 Annex H or H.265 Annex G). + * The data is an int storing the view ID. + */ + AV_FRAME_DATA_VIEW_ID, }; enum AVActiveFormatDescription { diff --git a/libavutil/version.h b/libavutil/version.h index 25a6f5531b..7900379c12 100644 --- a/libavutil/version.h +++ b/libavutil/version.h @@ -79,7 +79,7 @@ */ #define LIBAVUTIL_VERSION_MAJOR 59 -#define LIBAVUTIL_VERSION_MINOR 36 +#define LIBAVUTIL_VERSION_MINOR 37 #define LIBAVUTIL_VERSION_MICRO 100 #define LIBAVUTIL_VERSION_INT AV_VERSION_INT(LIBAVUTIL_VERSION_MAJOR, \ -- 2.43.0
Can be used together with the split filter to decompose multiview video into individual views. --- doc/filters.texi | 3 +++ libavfilter/f_select.c | 9 +++++++++ 2 files changed, 12 insertions(+) diff --git a/doc/filters.texi b/doc/filters.texi index 2eb4a380fb..3d524e5d96 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -31043,6 +31043,9 @@ Represents the width of the input video frame. @item ih @emph{(video only)} Represents the height of the input video frame. +@item view @emph{(video only)} +View ID for multi-view video. + @end table The default value of the select expression is "1". diff --git a/libavfilter/f_select.c b/libavfilter/f_select.c index 6ba96bd7c5..ba0ae31f1b 100644 --- a/libavfilter/f_select.c +++ b/libavfilter/f_select.c @@ -93,6 +93,8 @@ static const char *const var_names[] = { "ih", ///< ih: Represents the height of the input video frame. "iw", ///< iw: Represents the width of the input video frame. + "view", + NULL }; @@ -150,6 +152,8 @@ enum var_name { VAR_IH, VAR_IW, + VAR_VIEW, + VAR_VARS_NB }; @@ -343,6 +347,7 @@ static void select_frame(AVFilterContext *ctx, AVFrame *frame) SelectContext *select = ctx->priv; AVFilterLink *inlink = ctx->inputs[0]; FilterLink *inl = ff_filter_link(inlink); + const AVFrameSideData *sd; double res; if (isnan(select->var_values[VAR_START_PTS])) @@ -381,6 +386,10 @@ FF_ENABLE_DEPRECATION_WARNINGS snprintf(buf, sizeof(buf), "%f", select->var_values[VAR_SCENE]); av_dict_set(&frame->metadata, "lavfi.scene_score", buf, 0); } + + sd = av_frame_side_data_get(frame->side_data, frame->nb_side_data, + AV_FRAME_DATA_VIEW_ID); + select->var_values[VAR_VIEW] = sd ? *(int*)sd->data : NAN; break; } -- 2.43.0
--- doc/APIchanges | 3 +++ libavcodec/defs.h | 1 + libavcodec/hevc/ps.c | 21 +++++++++------------ libavcodec/profiles.c | 1 + libavcodec/version.h | 2 +- 5 files changed, 15 insertions(+), 13 deletions(-) diff --git a/doc/APIchanges b/doc/APIchanges index a8f84ca45f..a4cf1bf69f 100644 --- a/doc/APIchanges +++ b/doc/APIchanges @@ -2,6 +2,9 @@ The last version increases of all libraries were on 2024-03-07 API changes, most recent first: +2024-xx-xx - xxxxxxx - lavc 61.12.100 - defs.h + Add AV_PROFILE_HEVC_MULTIVIEW_MAIN + 2024-xx-xx - xxxxxxx - lavu 59.37.100 - frame.h Add AV_FRAME_DATA_VIEW_ID. diff --git a/libavcodec/defs.h b/libavcodec/defs.h index 7ddfdcad0b..24250f8af5 100644 --- a/libavcodec/defs.h +++ b/libavcodec/defs.h @@ -160,6 +160,7 @@ #define AV_PROFILE_HEVC_MAIN_10 2 #define AV_PROFILE_HEVC_MAIN_STILL_PICTURE 3 #define AV_PROFILE_HEVC_REXT 4 +#define AV_PROFILE_HEVC_MULTIVIEW_MAIN 6 #define AV_PROFILE_HEVC_SCC 9 #define AV_PROFILE_VVC_MAIN_10 1 diff --git a/libavcodec/hevc/ps.c b/libavcodec/hevc/ps.c index 22042d3e62..dbdfc16cfa 100644 --- a/libavcodec/hevc/ps.c +++ b/libavcodec/hevc/ps.c @@ -29,6 +29,7 @@ #include "h2645_vui.h" #include "data.h" #include "ps.h" +#include "profiles.h" #include "refstruct.h" static const uint8_t default_scaling_list_intra[] = { @@ -244,6 +245,7 @@ int ff_hevc_decode_short_term_rps(GetBitContext *gb, AVCodecContext *avctx, static int decode_profile_tier_level(GetBitContext *gb, AVCodecContext *avctx, PTLCommon *ptl) { + const char *profile_name = NULL; int i; if (get_bits_left(gb) < 2+1+5 + 32 + 4 + 43 + 1) @@ -252,18 +254,13 @@ static int decode_profile_tier_level(GetBitContext *gb, AVCodecContext *avctx, ptl->profile_space = get_bits(gb, 2); ptl->tier_flag = get_bits1(gb); ptl->profile_idc = get_bits(gb, 5); - if (ptl->profile_idc == AV_PROFILE_HEVC_MAIN) - av_log(avctx, AV_LOG_DEBUG, "Main profile bitstream\n"); - else if (ptl->profile_idc == AV_PROFILE_HEVC_MAIN_10) - av_log(avctx, AV_LOG_DEBUG, "Main 10 profile bitstream\n"); - else if (ptl->profile_idc == AV_PROFILE_HEVC_MAIN_STILL_PICTURE) - av_log(avctx, AV_LOG_DEBUG, "Main Still Picture profile bitstream\n"); - else if (ptl->profile_idc == AV_PROFILE_HEVC_REXT) - av_log(avctx, AV_LOG_DEBUG, "Range Extension profile bitstream\n"); - else if (ptl->profile_idc == AV_PROFILE_HEVC_SCC) - av_log(avctx, AV_LOG_DEBUG, "Screen Content Coding Extension profile bitstream\n"); - else - av_log(avctx, AV_LOG_WARNING, "Unknown HEVC profile: %d\n", ptl->profile_idc); + for (int i = 0; ff_hevc_profiles[i].profile != AV_PROFILE_UNKNOWN; i++) + if (ff_hevc_profiles[i].profile == ptl->profile_idc) { + profile_name = ff_hevc_profiles[i].name; + break; + } + av_log(avctx, profile_name ? AV_LOG_DEBUG : AV_LOG_WARNING, + "%s profile bitstream\n", profile_name ? profile_name : "Unknown"); for (i = 0; i < 32; i++) { ptl->profile_compatibility_flag[i] = get_bits1(gb); diff --git a/libavcodec/profiles.c b/libavcodec/profiles.c index 44bdf6f85b..3cef82be3b 100644 --- a/libavcodec/profiles.c +++ b/libavcodec/profiles.c @@ -99,6 +99,7 @@ const AVProfile ff_hevc_profiles[] = { { AV_PROFILE_HEVC_MAIN_10, "Main 10" }, { AV_PROFILE_HEVC_MAIN_STILL_PICTURE, "Main Still Picture" }, { AV_PROFILE_HEVC_REXT, "Rext" }, + { AV_PROFILE_HEVC_MULTIVIEW_MAIN, "Multiview Main" }, { AV_PROFILE_HEVC_SCC, "Scc" }, { AV_PROFILE_UNKNOWN }, }; diff --git a/libavcodec/version.h b/libavcodec/version.h index 8b53586be1..da2264a097 100644 --- a/libavcodec/version.h +++ b/libavcodec/version.h @@ -29,7 +29,7 @@ #include "version_major.h" -#define LIBAVCODEC_VERSION_MINOR 11 +#define LIBAVCODEC_VERSION_MINOR 12 #define LIBAVCODEC_VERSION_MICRO 100 #define LIBAVCODEC_VERSION_INT AV_VERSION_INT(LIBAVCODEC_VERSION_MAJOR, \ -- 2.43.0
On Tue, Aug 27, 2024 at 05:04:46PM +0200, Anton Khirnov wrote:
--- doc/APIchanges | 3 +++ libavcodec/defs.h | 1 + libavcodec/hevc/ps.c | 21 +++++++++------------ libavcodec/profiles.c | 1 + libavcodec/version.h | 2 +- 5 files changed, 15 insertions(+), 13 deletions(-)
assuming this matches your mv-hevc branch then it fails to build /usr/bin/x86_64-w64-mingw32-ld: libavcodec/libavcodec.a(ps.o):ps.c:(.rdata$.refptr.ff_hevc_profiles[.refptr.ff_hevc_profiles]+0x0): undefined reference to `ff_hevc_profiles' collect2: error: ld returned 1 exit status make: *** [Makefile:141: ffprobe_g.exe] Error 1 probably related to --enable-small thx [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB No great genius has ever existed without some touch of madness. -- Aristotle
Quoting Michael Niedermayer (2024-09-04 23:05:28)
On Tue, Aug 27, 2024 at 05:04:46PM +0200, Anton Khirnov wrote:
--- doc/APIchanges | 3 +++ libavcodec/defs.h | 1 + libavcodec/hevc/ps.c | 21 +++++++++------------ libavcodec/profiles.c | 1 + libavcodec/version.h | 2 +- 5 files changed, 15 insertions(+), 13 deletions(-)
assuming this matches your mv-hevc branch then it fails to build
/usr/bin/x86_64-w64-mingw32-ld: libavcodec/libavcodec.a(ps.o):ps.c:(.rdata$.refptr.ff_hevc_profiles[.refptr.ff_hevc_profiles]+0x0): undefined reference to `ff_hevc_profiles' collect2: error: ld returned 1 exit status make: *** [Makefile:141: ffprobe_g.exe] Error 1
probably related to --enable-small
Fixed locally by wrapping the profile mapping code under #if !CONFIG_SMALL -- Anton Khirnov
Start by moving the DPB to it. Only one context exists for now, so decoder behaviour should not change with this commit, but that will change in the future. --- libavcodec/dxva2_hevc.c | 7 ++-- libavcodec/hevc/hevcdec.c | 63 +++++++++++++++++------------- libavcodec/hevc/hevcdec.h | 19 +++++++--- libavcodec/hevc/refs.c | 80 +++++++++++++++++++++------------------ libavcodec/nvdec_hevc.c | 5 ++- libavcodec/vaapi_hevc.c | 14 ++++--- libavcodec/vdpau_hevc.c | 5 ++- libavcodec/vulkan_hevc.c | 17 +++++---- 8 files changed, 120 insertions(+), 90 deletions(-) diff --git a/libavcodec/dxva2_hevc.c b/libavcodec/dxva2_hevc.c index 97e51d15de..d01d1e76e8 100644 --- a/libavcodec/dxva2_hevc.c +++ b/libavcodec/dxva2_hevc.c @@ -61,6 +61,7 @@ void ff_dxva2_hevc_fill_picture_parameters(const AVCodecContext *avctx, AVDXVACo DXVA_PicParams_HEVC *pp) { const HEVCContext *h = avctx->priv_data; + const HEVCLayerContext *l = &h->layers[h->cur_layer]; const HEVCFrame *current_picture = h->cur_frame; const HEVCPPS *pps = h->pps; const HEVCSPS *sps = pps->sps; @@ -163,9 +164,9 @@ void ff_dxva2_hevc_fill_picture_parameters(const AVCodecContext *avctx, AVDXVACo // fill RefPicList from the DPB for (i = 0, j = 0; i < FF_ARRAY_ELEMS(pp->RefPicList); i++) { const HEVCFrame *frame = NULL; - while (!frame && j < FF_ARRAY_ELEMS(h->DPB)) { - if (&h->DPB[j] != current_picture && (h->DPB[j].flags & (HEVC_FRAME_FLAG_LONG_REF | HEVC_FRAME_FLAG_SHORT_REF))) - frame = &h->DPB[j]; + while (!frame && j < FF_ARRAY_ELEMS(l->DPB)) { + if (&l->DPB[j] != current_picture && (l->DPB[j].flags & (HEVC_FRAME_FLAG_LONG_REF | HEVC_FRAME_FLAG_SHORT_REF))) + frame = &l->DPB[j]; j++; } diff --git a/libavcodec/hevc/hevcdec.c b/libavcodec/hevc/hevcdec.c index 4077ed3ac5..669c8f550b 100644 --- a/libavcodec/hevc/hevcdec.c +++ b/libavcodec/hevc/hevcdec.c @@ -2902,7 +2902,7 @@ static int set_side_data(HEVCContext *s) return 0; } -static int hevc_frame_start(HEVCContext *s) +static int hevc_frame_start(HEVCContext *s, HEVCLayerContext *l) { const HEVCPPS *const pps = s->ps.pps_list[s->sh.pps_id]; const HEVCSPS *const sps = pps->sps; @@ -2915,7 +2915,7 @@ static int hevc_frame_start(HEVCContext *s) if (s->ps.sps != sps) { enum AVPixelFormat pix_fmt; - ff_hevc_clear_refs(s); + ff_hevc_clear_refs(l); ret = set_sps(s, sps); if (ret < 0) @@ -2938,7 +2938,7 @@ static int hevc_frame_start(HEVCContext *s) memset(s->tab_slice_address, -1, pic_size_in_ctb * sizeof(*s->tab_slice_address)); if (IS_IDR(s)) - ff_hevc_clear_refs(s); + ff_hevc_clear_refs(l); s->slice_idx = 0; s->first_nal_type = s->nal_unit_type; @@ -2963,7 +2963,7 @@ static int hevc_frame_start(HEVCContext *s) s->local_ctx[0].end_of_tiles_x = pps->column_width[0] << sps->log2_ctb_size; if (new_sequence) { - ret = ff_hevc_output_frames(s, 0, 0, s->sh.no_output_of_prior_pics_flag); + ret = ff_hevc_output_frames(s, l, 0, 0, s->sh.no_output_of_prior_pics_flag); if (ret < 0) return ret; } @@ -2972,11 +2972,11 @@ static int hevc_frame_start(HEVCContext *s) if (ret < 0) return ret; - ret = ff_hevc_set_new_ref(s, s->poc); + ret = ff_hevc_set_new_ref(s, l, s->poc); if (ret < 0) goto fail; - ret = ff_hevc_frame_rps(s); + ret = ff_hevc_frame_rps(s, l); if (ret < 0) { av_log(s->avctx, AV_LOG_ERROR, "Error constructing the frame RPS.\n"); goto fail; @@ -3020,7 +3020,7 @@ static int hevc_frame_start(HEVCContext *s) s->cur_frame->f->pict_type = 3 - s->sh.slice_type; - ret = ff_hevc_output_frames(s, sps->temporal_layer[sps->max_sub_layers - 1].num_reorder_pics, + ret = ff_hevc_output_frames(s, l, sps->temporal_layer[sps->max_sub_layers - 1].num_reorder_pics, sps->temporal_layer[sps->max_sub_layers - 1].max_dec_pic_buffering, 0); if (ret < 0) goto fail; @@ -3158,7 +3158,8 @@ static int hevc_frame_end(HEVCContext *s) return 0; } -static int decode_slice(HEVCContext *s, const H2645NAL *nal, GetBitContext *gb) +static int decode_slice(HEVCContext *s, HEVCLayerContext *l, + const H2645NAL *nal, GetBitContext *gb) { int ret; @@ -3183,7 +3184,7 @@ static int decode_slice(HEVCContext *s, const H2645NAL *nal, GetBitContext *gb) return AVERROR_INVALIDDATA; } - ret = hevc_frame_start(s); + ret = hevc_frame_start(s, l); if (ret < 0) return ret; } else if (!s->cur_frame) { @@ -3207,6 +3208,7 @@ static int decode_slice(HEVCContext *s, const H2645NAL *nal, GetBitContext *gb) static int decode_nal_unit(HEVCContext *s, const H2645NAL *nal) { + HEVCLayerContext *l = &s->layers[0]; GetBitContext gb = nal->gb; int ret; @@ -3264,7 +3266,7 @@ static int decode_nal_unit(HEVCContext *s, const H2645NAL *nal) case HEVC_NAL_RADL_R: case HEVC_NAL_RASL_N: case HEVC_NAL_RASL_R: - ret = decode_slice(s, nal, &gb); + ret = decode_slice(s, l, nal, &gb); if (ret < 0) goto fail; break; @@ -3427,7 +3429,7 @@ static int hevc_receive_frame(AVCodecContext *avctx, AVFrame *frame) av_packet_unref(avpkt); ret = ff_decode_get_packet(avctx, avpkt); if (ret == AVERROR_EOF) { - ret = ff_hevc_output_frames(s, 0, 0, 0); + ret = ff_hevc_output_frames(s, &s->layers[0], 0, 0, 0); if (ret < 0) return ret; goto do_output; @@ -3517,9 +3519,12 @@ static av_cold int hevc_decode_free(AVCodecContext *avctx) ff_container_fifo_free(&s->output_fifo); - for (i = 0; i < FF_ARRAY_ELEMS(s->DPB); i++) { - ff_hevc_unref_frame(&s->DPB[i], ~0); - av_frame_free(&s->DPB[i].frame_grain); + for (int layer = 0; layer < FF_ARRAY_ELEMS(s->layers); layer++) { + HEVCLayerContext *l = &s->layers[layer]; + for (int i = 0; i < FF_ARRAY_ELEMS(l->DPB); i++) { + ff_hevc_unref_frame(&l->DPB[i], ~0); + av_frame_free(&l->DPB[i].frame_grain); + } } ff_hevc_ps_uninit(&s->ps); @@ -3540,7 +3545,6 @@ static av_cold int hevc_decode_free(AVCodecContext *avctx) static av_cold int hevc_init_context(AVCodecContext *avctx) { HEVCContext *s = avctx->priv_data; - int i; s->avctx = avctx; @@ -3557,10 +3561,13 @@ static av_cold int hevc_init_context(AVCodecContext *avctx) if (!s->output_fifo) return AVERROR(ENOMEM); - for (i = 0; i < FF_ARRAY_ELEMS(s->DPB); i++) { - s->DPB[i].frame_grain = av_frame_alloc(); - if (!s->DPB[i].frame_grain) - return AVERROR(ENOMEM); + for (int layer = 0; layer < FF_ARRAY_ELEMS(s->layers); layer++) { + HEVCLayerContext *l = &s->layers[layer]; + for (int i = 0; i < FF_ARRAY_ELEMS(l->DPB); i++) { + l->DPB[i].frame_grain = av_frame_alloc(); + if (!l->DPB[i].frame_grain) + return AVERROR(ENOMEM); + } } s->md5_ctx = av_md5_alloc(); @@ -3583,14 +3590,18 @@ static int hevc_update_thread_context(AVCodecContext *dst, { HEVCContext *s = dst->priv_data; HEVCContext *s0 = src->priv_data; - int i, ret; + int ret; - for (i = 0; i < FF_ARRAY_ELEMS(s->DPB); i++) { - ff_hevc_unref_frame(&s->DPB[i], ~0); - if (s0->DPB[i].f) { - ret = hevc_ref_frame(&s->DPB[i], &s0->DPB[i]); - if (ret < 0) - return ret; + for (int layer = 0; layer < FF_ARRAY_ELEMS(s->layers); layer++) { + HEVCLayerContext *l = &s->layers[layer]; + const HEVCLayerContext *l0 = &s0->layers[layer]; + for (int i = 0; i < FF_ARRAY_ELEMS(l->DPB); i++) { + ff_hevc_unref_frame(&l->DPB[i], ~0); + if (l0->DPB[i].f) { + ret = hevc_ref_frame(&l->DPB[i], &l0->DPB[i]); + if (ret < 0) + return ret; + } } } diff --git a/libavcodec/hevc/hevcdec.h b/libavcodec/hevc/hevcdec.h index 656a3ab94b..aab09bfd94 100644 --- a/libavcodec/hevc/hevcdec.h +++ b/libavcodec/hevc/hevcdec.h @@ -438,6 +438,10 @@ typedef struct HEVCLocalContext { char padding[128]; } HEVCLocalContext; +typedef struct HEVCLayerContext { + HEVCFrame DPB[32]; +} HEVCLayerContext; + typedef struct HEVCContext { const AVClass *c; // needed by private avoptions AVCodecContext *avctx; @@ -445,6 +449,10 @@ typedef struct HEVCContext { HEVCLocalContext *local_ctx; unsigned nb_local_ctx; + HEVCLayerContext layers[1]; + // index in layers of the layer currently being decoded + unsigned cur_layer; + /** 1 if the independent slice segment header was successfully parsed */ uint8_t slice_initialized; @@ -470,7 +478,6 @@ typedef struct HEVCContext { int temporal_id; ///< temporal_id_plus1 - 1 HEVCFrame *cur_frame; HEVCFrame *collocated_ref; - HEVCFrame DPB[32]; int poc; int poc_tid0; int slice_idx; ///< number of the slice being currently decoded @@ -536,7 +543,7 @@ typedef struct HEVCContext { /** * Mark all frames in DPB as unused for reference. */ -void ff_hevc_clear_refs(HEVCContext *s); +void ff_hevc_clear_refs(HEVCLayerContext *l); /** * Drop all frames currently in DPB. @@ -549,7 +556,7 @@ const RefPicList *ff_hevc_get_ref_list(const HEVCContext *s, const HEVCFrame *fr /** * Construct the reference picture sets for the current frame. */ -int ff_hevc_frame_rps(HEVCContext *s); +int ff_hevc_frame_rps(HEVCContext *s, HEVCLayerContext *l); /** * Construct the reference picture list(s) for the current slice. @@ -597,7 +604,7 @@ int ff_hevc_res_scale_sign_flag(HEVCLocalContext *lc, int idx); */ int ff_hevc_frame_nb_refs(const SliceHeader *sh, const HEVCPPS *pps); -int ff_hevc_set_new_ref(HEVCContext *s, int poc); +int ff_hevc_set_new_ref(HEVCContext *s, HEVCLayerContext *l, int poc); static av_always_inline int ff_hevc_nal_is_nonref(enum HEVCNALUnitType type) { @@ -625,8 +632,8 @@ static av_always_inline int ff_hevc_nal_is_nonref(enum HEVCNALUnitType type) * @param max_dpb maximum number of any frames that can be present in the DPB * before output is triggered */ -int ff_hevc_output_frames(HEVCContext *s, unsigned max_output, - unsigned max_dpb, int discard); +int ff_hevc_output_frames(HEVCContext *s, HEVCLayerContext *l, + unsigned max_output, unsigned max_dpb, int discard); void ff_hevc_unref_frame(HEVCFrame *frame, int flags); diff --git a/libavcodec/hevc/refs.c b/libavcodec/hevc/refs.c index 65abd09a21..4d123d6d8d 100644 --- a/libavcodec/hevc/refs.c +++ b/libavcodec/hevc/refs.c @@ -59,27 +59,29 @@ const RefPicList *ff_hevc_get_ref_list(const HEVCContext *s, return &ref->rpl_tab[ctb_addr_ts]->refPicList[0]; } -void ff_hevc_clear_refs(HEVCContext *s) +void ff_hevc_clear_refs(HEVCLayerContext *l) { int i; - for (i = 0; i < FF_ARRAY_ELEMS(s->DPB); i++) - ff_hevc_unref_frame(&s->DPB[i], + for (i = 0; i < FF_ARRAY_ELEMS(l->DPB); i++) + ff_hevc_unref_frame(&l->DPB[i], HEVC_FRAME_FLAG_SHORT_REF | HEVC_FRAME_FLAG_LONG_REF); } void ff_hevc_flush_dpb(HEVCContext *s) { - int i; - for (i = 0; i < FF_ARRAY_ELEMS(s->DPB); i++) - ff_hevc_unref_frame(&s->DPB[i], ~0); + for (int layer = 0; layer < FF_ARRAY_ELEMS(s->layers); layer++) { + HEVCLayerContext *l = &s->layers[layer]; + for (int i = 0; i < FF_ARRAY_ELEMS(l->DPB); i++) + ff_hevc_unref_frame(&l->DPB[i], ~0); + } } -static HEVCFrame *alloc_frame(HEVCContext *s) +static HEVCFrame *alloc_frame(HEVCContext *s, HEVCLayerContext *l) { int i, j, ret; - for (i = 0; i < FF_ARRAY_ELEMS(s->DPB); i++) { - HEVCFrame *frame = &s->DPB[i]; + for (i = 0; i < FF_ARRAY_ELEMS(l->DPB); i++) { + HEVCFrame *frame = &l->DPB[i]; if (frame->f) continue; @@ -123,14 +125,14 @@ fail: return NULL; } -int ff_hevc_set_new_ref(HEVCContext *s, int poc) +int ff_hevc_set_new_ref(HEVCContext *s, HEVCLayerContext *l, int poc) { HEVCFrame *ref; int i; /* check that this POC doesn't already exist */ - for (i = 0; i < FF_ARRAY_ELEMS(s->DPB); i++) { - HEVCFrame *frame = &s->DPB[i]; + for (i = 0; i < FF_ARRAY_ELEMS(l->DPB); i++) { + HEVCFrame *frame = &l->DPB[i]; if (frame->f && frame->poc == poc) { av_log(s->avctx, AV_LOG_ERROR, "Duplicate POC in a sequence: %d.\n", @@ -139,7 +141,7 @@ int ff_hevc_set_new_ref(HEVCContext *s, int poc) } } - ref = alloc_frame(s); + ref = alloc_frame(s, l); if (!ref) return AVERROR(ENOMEM); @@ -160,18 +162,18 @@ int ff_hevc_set_new_ref(HEVCContext *s, int poc) return 0; } -static void unref_missing_refs(HEVCContext *s) +static void unref_missing_refs(HEVCLayerContext *l) { - for (int i = 0; i < FF_ARRAY_ELEMS(s->DPB); i++) { - HEVCFrame *frame = &s->DPB[i]; + for (int i = 0; i < FF_ARRAY_ELEMS(l->DPB); i++) { + HEVCFrame *frame = &l->DPB[i]; if (frame->flags & HEVC_FRAME_FLAG_UNAVAILABLE) { ff_hevc_unref_frame(frame, ~0); } } } -int ff_hevc_output_frames(HEVCContext *s, unsigned max_output, - unsigned max_dpb, int discard) +int ff_hevc_output_frames(HEVCContext *s, HEVCLayerContext *l, + unsigned max_output, unsigned max_dpb, int discard) { while (1) { int nb_dpb = 0; @@ -179,8 +181,8 @@ int ff_hevc_output_frames(HEVCContext *s, unsigned max_output, int min_poc = INT_MAX; int i, min_idx, ret; - for (i = 0; i < FF_ARRAY_ELEMS(s->DPB); i++) { - HEVCFrame *frame = &s->DPB[i]; + for (i = 0; i < FF_ARRAY_ELEMS(l->DPB); i++) { + HEVCFrame *frame = &l->DPB[i]; if (frame->flags & HEVC_FRAME_FLAG_OUTPUT) { nb_output++; if (frame->poc < min_poc || nb_output == 1) { @@ -193,7 +195,7 @@ int ff_hevc_output_frames(HEVCContext *s, unsigned max_output, if (nb_output > max_output || (nb_output && nb_dpb > max_dpb)) { - HEVCFrame *frame = &s->DPB[min_idx]; + HEVCFrame *frame = &l->DPB[min_idx]; ret = discard ? 0 : ff_container_fifo_write(s->output_fifo, @@ -313,13 +315,14 @@ int ff_hevc_slice_rpl(HEVCContext *s) return 0; } -static HEVCFrame *find_ref_idx(HEVCContext *s, int poc, uint8_t use_msb) +static HEVCFrame *find_ref_idx(HEVCContext *s, HEVCLayerContext *l, + int poc, uint8_t use_msb) { int mask = use_msb ? ~0 : (1 << s->ps.sps->log2_max_poc_lsb) - 1; int i; - for (i = 0; i < FF_ARRAY_ELEMS(s->DPB); i++) { - HEVCFrame *ref = &s->DPB[i]; + for (i = 0; i < FF_ARRAY_ELEMS(l->DPB); i++) { + HEVCFrame *ref = &l->DPB[i]; if (ref->f) { if ((ref->poc & mask) == poc && (use_msb || ref->poc != s->poc)) return ref; @@ -338,12 +341,12 @@ static void mark_ref(HEVCFrame *frame, int flag) frame->flags |= flag; } -static HEVCFrame *generate_missing_ref(HEVCContext *s, int poc) +static HEVCFrame *generate_missing_ref(HEVCContext *s, HEVCLayerContext *l, int poc) { HEVCFrame *frame; int i, y; - frame = alloc_frame(s); + frame = alloc_frame(s, l); if (!frame) return NULL; @@ -372,16 +375,17 @@ static HEVCFrame *generate_missing_ref(HEVCContext *s, int poc) } /* add a reference with the given poc to the list and mark it as used in DPB */ -static int add_candidate_ref(HEVCContext *s, RefPicList *list, +static int add_candidate_ref(HEVCContext *s, HEVCLayerContext *l, + RefPicList *list, int poc, int ref_flag, uint8_t use_msb) { - HEVCFrame *ref = find_ref_idx(s, poc, use_msb); + HEVCFrame *ref = find_ref_idx(s, l, poc, use_msb); if (ref == s->cur_frame || list->nb_refs >= HEVC_MAX_REFS) return AVERROR_INVALIDDATA; if (!ref) { - ref = generate_missing_ref(s, poc); + ref = generate_missing_ref(s, l, poc); if (!ref) return AVERROR(ENOMEM); } @@ -394,7 +398,7 @@ static int add_candidate_ref(HEVCContext *s, RefPicList *list, return 0; } -int ff_hevc_frame_rps(HEVCContext *s) +int ff_hevc_frame_rps(HEVCContext *s, HEVCLayerContext *l) { const ShortTermRPS *short_rps = s->sh.short_term_rps; const LongTermRPS *long_rps = &s->sh.long_term_rps; @@ -406,11 +410,11 @@ int ff_hevc_frame_rps(HEVCContext *s) return 0; } - unref_missing_refs(s); + unref_missing_refs(l); /* clear the reference flags on all frames except the current one */ - for (i = 0; i < FF_ARRAY_ELEMS(s->DPB); i++) { - HEVCFrame *frame = &s->DPB[i]; + for (i = 0; i < FF_ARRAY_ELEMS(l->DPB); i++) { + HEVCFrame *frame = &l->DPB[i]; if (frame == s->cur_frame) continue; @@ -433,7 +437,8 @@ int ff_hevc_frame_rps(HEVCContext *s) else list = ST_CURR_AFT; - ret = add_candidate_ref(s, &rps[list], poc, HEVC_FRAME_FLAG_SHORT_REF, 1); + ret = add_candidate_ref(s, l, &rps[list], poc, + HEVC_FRAME_FLAG_SHORT_REF, 1); if (ret < 0) goto fail; } @@ -443,15 +448,16 @@ int ff_hevc_frame_rps(HEVCContext *s) int poc = long_rps->poc[i]; int list = long_rps->used[i] ? LT_CURR : LT_FOLL; - ret = add_candidate_ref(s, &rps[list], poc, HEVC_FRAME_FLAG_LONG_REF, long_rps->poc_msb_present[i]); + ret = add_candidate_ref(s, l, &rps[list], poc, + HEVC_FRAME_FLAG_LONG_REF, long_rps->poc_msb_present[i]); if (ret < 0) goto fail; } fail: /* release any frames that are now unused */ - for (i = 0; i < FF_ARRAY_ELEMS(s->DPB); i++) - ff_hevc_unref_frame(&s->DPB[i], 0); + for (i = 0; i < FF_ARRAY_ELEMS(l->DPB); i++) + ff_hevc_unref_frame(&l->DPB[i], 0); return ret; } diff --git a/libavcodec/nvdec_hevc.c b/libavcodec/nvdec_hevc.c index ce66ddcfb7..6888507535 100644 --- a/libavcodec/nvdec_hevc.c +++ b/libavcodec/nvdec_hevc.c @@ -73,6 +73,7 @@ static int nvdec_hevc_start_frame(AVCodecContext *avctx, const uint8_t *buffer, uint32_t size) { const HEVCContext *s = avctx->priv_data; + const HEVCLayerContext *l = &s->layers[s->cur_layer]; const HEVCPPS *pps = s->pps; const HEVCSPS *sps = pps->sps; @@ -225,8 +226,8 @@ static int nvdec_hevc_start_frame(AVCodecContext *avctx, } dpb_size = 0; - for (i = 0; i < FF_ARRAY_ELEMS(s->DPB); i++) { - const HEVCFrame *ref = &s->DPB[i]; + for (i = 0; i < FF_ARRAY_ELEMS(l->DPB); i++) { + const HEVCFrame *ref = &l->DPB[i]; if (!(ref->flags & (HEVC_FRAME_FLAG_SHORT_REF | HEVC_FRAME_FLAG_LONG_REF))) continue; if (dpb_size >= FF_ARRAY_ELEMS(ppc->RefPicIdx)) { diff --git a/libavcodec/vaapi_hevc.c b/libavcodec/vaapi_hevc.c index ad4cd35b26..b97e7c0343 100644 --- a/libavcodec/vaapi_hevc.c +++ b/libavcodec/vaapi_hevc.c @@ -96,7 +96,8 @@ static int find_frame_rps_type(const HEVCContext *h, const HEVCFrame *pic) return 0; } -static void fill_vaapi_reference_frames(const HEVCContext *h, VAPictureParameterBufferHEVC *pp) +static void fill_vaapi_reference_frames(const HEVCContext *h, const HEVCLayerContext *l, + VAPictureParameterBufferHEVC *pp) { const HEVCFrame *current_picture = h->cur_frame; int i, j, rps_type; @@ -104,10 +105,10 @@ static void fill_vaapi_reference_frames(const HEVCContext *h, VAPictureParameter for (i = 0, j = 0; i < FF_ARRAY_ELEMS(pp->ReferenceFrames); i++) { const HEVCFrame *frame = NULL; - while (!frame && j < FF_ARRAY_ELEMS(h->DPB)) { - if ((&h->DPB[j] != current_picture || h->pps->pps_curr_pic_ref_enabled_flag) && - (h->DPB[j].flags & (HEVC_FRAME_FLAG_LONG_REF | HEVC_FRAME_FLAG_SHORT_REF))) - frame = &h->DPB[j]; + while (!frame && j < FF_ARRAY_ELEMS(l->DPB)) { + if ((&l->DPB[j] != current_picture || h->pps->pps_curr_pic_ref_enabled_flag) && + (l->DPB[j].flags & (HEVC_FRAME_FLAG_LONG_REF | HEVC_FRAME_FLAG_SHORT_REF))) + frame = &l->DPB[j]; j++; } @@ -125,6 +126,7 @@ static int vaapi_hevc_start_frame(AVCodecContext *avctx, av_unused uint32_t size) { const HEVCContext *h = avctx->priv_data; + const HEVCLayerContext *l = &h->layers[h->cur_layer]; VAAPIDecodePictureHEVC *pic = h->cur_frame->hwaccel_picture_private; const HEVCPPS *pps = h->pps; const HEVCSPS *sps = pps->sps; @@ -208,7 +210,7 @@ static int vaapi_hevc_start_frame(AVCodecContext *avctx, }; fill_vaapi_pic(&pic_param->CurrPic, h->cur_frame, 0); - fill_vaapi_reference_frames(h, pic_param); + fill_vaapi_reference_frames(h, l, pic_param); if (pps->tiles_enabled_flag) { pic_param->num_tile_columns_minus1 = pps->num_tile_columns - 1; diff --git a/libavcodec/vdpau_hevc.c b/libavcodec/vdpau_hevc.c index b9e922ecfc..e6232567e0 100644 --- a/libavcodec/vdpau_hevc.c +++ b/libavcodec/vdpau_hevc.c @@ -35,6 +35,7 @@ static int vdpau_hevc_start_frame(AVCodecContext *avctx, const uint8_t *buffer, uint32_t size) { HEVCContext *h = avctx->priv_data; + const HEVCLayerContext *l = &h->layers[h->cur_layer]; HEVCFrame *pic = h->cur_frame; struct vdpau_picture_context *pic_ctx = pic->hwaccel_picture_private; @@ -236,8 +237,8 @@ static int vdpau_hevc_start_frame(AVCodecContext *avctx, info->PicOrderCntVal[i] = 0; info->IsLongTerm[i] = 0; } - for (size_t i = 0, j = 0; i < FF_ARRAY_ELEMS(h->DPB); i++) { - const HEVCFrame *frame = &h->DPB[i]; + for (size_t i = 0, j = 0; i < FF_ARRAY_ELEMS(l->DPB); i++) { + const HEVCFrame *frame = &l->DPB[i]; if (frame != h->cur_frame && (frame->flags & (HEVC_FRAME_FLAG_LONG_REF | HEVC_FRAME_FLAG_SHORT_REF))) { if (j > 15) { diff --git a/libavcodec/vulkan_hevc.c b/libavcodec/vulkan_hevc.c index 5228e41ad5..f4c8da2d61 100644 --- a/libavcodec/vulkan_hevc.c +++ b/libavcodec/vulkan_hevc.c @@ -731,6 +731,7 @@ static int vk_hevc_start_frame(AVCodecContext *avctx, { int err; HEVCContext *h = avctx->priv_data; + HEVCLayerContext *l = &h->layers[h->cur_layer]; HEVCFrame *pic = h->cur_frame; FFVulkanDecodeContext *dec = avctx->internal->hwaccel_priv_data; HEVCVulkanDecodePicture *hp = pic->hwaccel_picture_private; @@ -762,8 +763,8 @@ static int vk_hevc_start_frame(AVCodecContext *avctx, }; /* Fill in references */ - for (int i = 0; i < FF_ARRAY_ELEMS(h->DPB); i++) { - const HEVCFrame *ref = &h->DPB[i]; + for (int i = 0; i < FF_ARRAY_ELEMS(l->DPB); i++) { + const HEVCFrame *ref = &l->DPB[i]; int idx = nb_refs; if (!(ref->flags & (HEVC_FRAME_FLAG_SHORT_REF | HEVC_FRAME_FLAG_LONG_REF))) @@ -790,8 +791,8 @@ static int vk_hevc_start_frame(AVCodecContext *avctx, memset(hp->h265pic.RefPicSetStCurrBefore, 0xff, 8); for (int i = 0; i < h->rps[ST_CURR_BEF].nb_refs; i++) { HEVCFrame *frame = h->rps[ST_CURR_BEF].ref[i]; - for (int j = 0; j < FF_ARRAY_ELEMS(h->DPB); j++) { - const HEVCFrame *ref = &h->DPB[j]; + for (int j = 0; j < FF_ARRAY_ELEMS(l->DPB); j++) { + const HEVCFrame *ref = &l->DPB[j]; if (ref == frame) { hp->h265pic.RefPicSetStCurrBefore[i] = j; break; @@ -801,8 +802,8 @@ static int vk_hevc_start_frame(AVCodecContext *avctx, memset(hp->h265pic.RefPicSetStCurrAfter, 0xff, 8); for (int i = 0; i < h->rps[ST_CURR_AFT].nb_refs; i++) { HEVCFrame *frame = h->rps[ST_CURR_AFT].ref[i]; - for (int j = 0; j < FF_ARRAY_ELEMS(h->DPB); j++) { - const HEVCFrame *ref = &h->DPB[j]; + for (int j = 0; j < FF_ARRAY_ELEMS(l->DPB); j++) { + const HEVCFrame *ref = &l->DPB[j]; if (ref == frame) { hp->h265pic.RefPicSetStCurrAfter[i] = j; break; @@ -812,8 +813,8 @@ static int vk_hevc_start_frame(AVCodecContext *avctx, memset(hp->h265pic.RefPicSetLtCurr, 0xff, 8); for (int i = 0; i < h->rps[LT_CURR].nb_refs; i++) { HEVCFrame *frame = h->rps[LT_CURR].ref[i]; - for (int j = 0; j < FF_ARRAY_ELEMS(h->DPB); j++) { - const HEVCFrame *ref = &h->DPB[j]; + for (int j = 0; j < FF_ARRAY_ELEMS(l->DPB); j++) { + const HEVCFrame *ref = &l->DPB[j]; if (ref == frame) { hp->h265pic.RefPicSetLtCurr[i] = j; break; -- 2.43.0
Active VPS is a property of the decoding process, not of the list of parameter sets. Check that the VPS can only change in a base layer - while this can never happen currently (as no other layers can exist in the decoder), it will become useful when multilayer decoding is supported. --- libavcodec/hevc/hevcdec.c | 11 +++++++++-- libavcodec/hevc/hevcdec.h | 1 + libavcodec/hevc/ps.c | 4 ---- libavcodec/hevc/ps.h | 1 - 4 files changed, 10 insertions(+), 7 deletions(-) diff --git a/libavcodec/hevc/hevcdec.c b/libavcodec/hevc/hevcdec.c index 669c8f550b..b44bc93507 100644 --- a/libavcodec/hevc/hevcdec.c +++ b/libavcodec/hevc/hevcdec.c @@ -533,7 +533,7 @@ static int set_sps(HEVCContext *s, const HEVCSPS *sps) pic_arrays_free(s); s->ps.sps = NULL; - s->ps.vps = NULL; + ff_refstruct_unref(&s->vps); if (!sps) return 0; @@ -571,7 +571,7 @@ static int set_sps(HEVCContext *s, const HEVCSPS *sps) } s->ps.sps = sps; - s->ps.vps = sps->vps; + s->vps = ff_refstruct_ref_c(sps->vps); return 0; @@ -2911,6 +2911,12 @@ static int hevc_frame_start(HEVCContext *s, HEVCLayerContext *l) int new_sequence = IS_IDR(s) || IS_BLA(s) || s->last_eos; int ret; + if (sps->vps != s->vps && l != &s->layers[0]) { + av_log(s->avctx, AV_LOG_ERROR, "VPS changed in a non-base layer\n"); + set_sps(s, NULL, AV_PIX_FMT_NONE); + return AVERROR_INVALIDDATA; + } + ff_refstruct_replace(&s->pps, pps); if (s->ps.sps != sps) { enum AVPixelFormat pix_fmt; @@ -3505,6 +3511,7 @@ static av_cold int hevc_decode_free(AVCodecContext *avctx) pic_arrays_free(s); + ff_refstruct_unref(&s->vps); ff_refstruct_unref(&s->pps); ff_dovi_ctx_unref(&s->dovi_ctx); diff --git a/libavcodec/hevc/hevcdec.h b/libavcodec/hevc/hevcdec.h index aab09bfd94..be5d76c183 100644 --- a/libavcodec/hevc/hevcdec.h +++ b/libavcodec/hevc/hevcdec.h @@ -470,6 +470,7 @@ typedef struct HEVCContext { ///< candidate references for the current frame RefPicList rps[5]; + const HEVCVPS *vps; ///< RefStruct reference const HEVCPPS *pps; ///< RefStruct reference SliceHeader sh; SAOParams *sao; diff --git a/libavcodec/hevc/ps.c b/libavcodec/hevc/ps.c index dbdfc16cfa..99382876af 100644 --- a/libavcodec/hevc/ps.c +++ b/libavcodec/hevc/ps.c @@ -83,9 +83,6 @@ static void remove_vps(HEVCParamSets *s, int id) { int i; if (s->vps_list[id]) { - if (s->vps == s->vps_list[id]) - s->vps = NULL; - for (i = 0; i < FF_ARRAY_ELEMS(s->sps_list); i++) if (s->sps_list[i] && s->sps_list[i]->vps_id == id) remove_sps(s, i); @@ -2047,7 +2044,6 @@ void ff_hevc_ps_uninit(HEVCParamSets *ps) ff_refstruct_unref(&ps->pps_list[i]); ps->sps = NULL; - ps->vps = NULL; } int ff_hevc_compute_poc(const HEVCSPS *sps, int pocTid0, int poc_lsb, int nal_unit_type) diff --git a/libavcodec/hevc/ps.h b/libavcodec/hevc/ps.h index 17395c5510..554c86a656 100644 --- a/libavcodec/hevc/ps.h +++ b/libavcodec/hevc/ps.h @@ -449,7 +449,6 @@ typedef struct HEVCParamSets { const HEVCPPS *pps_list[HEVC_MAX_PPS_COUNT]; ///< RefStruct references /* currently active parameter sets */ - const HEVCVPS *vps; const HEVCSPS *sps; } HEVCParamSets; -- 2.43.0
--- libavcodec/hevc/filter.c | 43 ++++++++++++----------- libavcodec/hevc/hevcdec.c | 73 ++++++++++++++++++++++----------------- libavcodec/hevc/hevcdec.h | 14 +++++--- 3 files changed, 73 insertions(+), 57 deletions(-) diff --git a/libavcodec/hevc/filter.c b/libavcodec/hevc/filter.c index 56e354b486..18c73dc2fa 100644 --- a/libavcodec/hevc/filter.c +++ b/libavcodec/hevc/filter.c @@ -495,7 +495,7 @@ static int get_pcm(const HEVCSPS *sps, const uint8_t *is_pcm, int x, int y) (tc_offset & -2), \ 0, MAX_QP + DEFAULT_INTRA_TC_OFFSET)] -static void deblocking_filter_CTB(const HEVCContext *s, +static void deblocking_filter_CTB(const HEVCContext *s, const HEVCLayerContext *l, const HEVCPPS *pps, const HEVCSPS *sps, int x0, int y0) { @@ -546,8 +546,8 @@ static void deblocking_filter_CTB(const HEVCContext *s, for (y = y0; y < y_end; y += 8) { // vertical filtering luma for (x = x0 ? x0 : 8; x < x_end; x += 8) { - const int bs0 = s->vertical_bs[(x + y * s->bs_width) >> 2]; - const int bs1 = s->vertical_bs[(x + (y + 4) * s->bs_width) >> 2]; + const int bs0 = s->vertical_bs[(x + y * l->bs_width) >> 2]; + const int bs1 = s->vertical_bs[(x + (y + 4) * l->bs_width) >> 2]; if (bs0 || bs1) { const int qp = (get_qPy(sps, s->qp_y_tab, x - 1, y) + get_qPy(sps, s->qp_y_tab, x, y) + 1) >> 1; @@ -575,8 +575,8 @@ static void deblocking_filter_CTB(const HEVCContext *s, // horizontal filtering luma for (x = x0 ? x0 - 8 : 0; x < x_end2; x += 8) { - const int bs0 = s->horizontal_bs[( x + y * s->bs_width) >> 2]; - const int bs1 = s->horizontal_bs[((x + 4) + y * s->bs_width) >> 2]; + const int bs0 = s->horizontal_bs[( x + y * l->bs_width) >> 2]; + const int bs1 = s->horizontal_bs[((x + 4) + y * l->bs_width) >> 2]; if (bs0 || bs1) { const int qp = (get_qPy(sps, s->qp_y_tab, x, y - 1) + get_qPy(sps, s->qp_y_tab, x, y) + 1) >> 1; @@ -610,8 +610,8 @@ static void deblocking_filter_CTB(const HEVCContext *s, // vertical filtering chroma for (y = y0; y < y_end; y += (8 * v)) { for (x = x0 ? x0 : 8 * h; x < x_end; x += (8 * h)) { - const int bs0 = s->vertical_bs[(x + y * s->bs_width) >> 2]; - const int bs1 = s->vertical_bs[(x + (y + (4 * v)) * s->bs_width) >> 2]; + const int bs0 = s->vertical_bs[(x + y * l->bs_width) >> 2]; + const int bs1 = s->vertical_bs[(x + (y + (4 * v)) * l->bs_width) >> 2]; if ((bs0 == 2) || (bs1 == 2)) { const int qp0 = (get_qPy(sps, s->qp_y_tab, x - 1, y) + @@ -644,8 +644,8 @@ static void deblocking_filter_CTB(const HEVCContext *s, if (x_end != sps->width) x_end2 = x_end - 8 * h; for (x = x0 ? x0 - 8 * h : 0; x < x_end2; x += (8 * h)) { - const int bs0 = s->horizontal_bs[( x + y * s->bs_width) >> 2]; - const int bs1 = s->horizontal_bs[((x + 4 * h) + y * s->bs_width) >> 2]; + const int bs0 = s->horizontal_bs[( x + y * l->bs_width) >> 2]; + const int bs1 = s->horizontal_bs[((x + 4 * h) + y * l->bs_width) >> 2]; if ((bs0 == 2) || (bs1 == 2)) { const int qp0 = bs0 == 2 ? (get_qPy(sps, s->qp_y_tab, x, y - 1) + get_qPy(sps, s->qp_y_tab, x, y) + 1) >> 1 : 0; @@ -736,7 +736,8 @@ static int boundary_strength(const HEVCContext *s, const MvField *curr, const Mv return 1; } -void ff_hevc_deblocking_boundary_strengths(HEVCLocalContext *lc, const HEVCPPS *pps, +void ff_hevc_deblocking_boundary_strengths(HEVCLocalContext *lc, const HEVCLayerContext *l, + const HEVCPPS *pps, int x0, int y0, int log2_trafo_size) { const HEVCSPS *const sps = pps->sps; @@ -784,7 +785,7 @@ void ff_hevc_deblocking_boundary_strengths(HEVCLocalContext *lc, const HEVCPPS * bs = 1; else bs = boundary_strength(s, curr, top, rpl_top); - s->horizontal_bs[((x0 + i) + y0 * s->bs_width) >> 2] = bs; + s->horizontal_bs[((x0 + i) + y0 * l->bs_width) >> 2] = bs; } } @@ -822,7 +823,7 @@ void ff_hevc_deblocking_boundary_strengths(HEVCLocalContext *lc, const HEVCPPS * bs = 1; else bs = boundary_strength(s, curr, left, rpl_left); - s->vertical_bs[(x0 + (y0 + i) * s->bs_width) >> 2] = bs; + s->vertical_bs[(x0 + (y0 + i) * l->bs_width) >> 2] = bs; } } @@ -840,7 +841,7 @@ void ff_hevc_deblocking_boundary_strengths(HEVCLocalContext *lc, const HEVCPPS * const MvField *curr = &tab_mvf[yq_pu * min_pu_width + x_pu]; bs = boundary_strength(s, curr, top, rpl); - s->horizontal_bs[((x0 + i) + (y0 + j) * s->bs_width) >> 2] = bs; + s->horizontal_bs[((x0 + i) + (y0 + j) * l->bs_width) >> 2] = bs; } } @@ -855,7 +856,7 @@ void ff_hevc_deblocking_boundary_strengths(HEVCLocalContext *lc, const HEVCPPS * const MvField *curr = &tab_mvf[y_pu * min_pu_width + xq_pu]; bs = boundary_strength(s, curr, left, rpl); - s->vertical_bs[((x0 + i) + (y0 + j) * s->bs_width) >> 2] = bs; + s->vertical_bs[((x0 + i) + (y0 + j) * l->bs_width) >> 2] = bs; } } } @@ -865,7 +866,8 @@ void ff_hevc_deblocking_boundary_strengths(HEVCLocalContext *lc, const HEVCPPS * #undef CB #undef CR -void ff_hevc_hls_filter(HEVCLocalContext *lc, const HEVCPPS *pps, +void ff_hevc_hls_filter(HEVCLocalContext *lc, const HEVCLayerContext *l, + const HEVCPPS *pps, int x, int y, int ctb_size) { const HEVCSPS *const sps = pps->sps; @@ -883,7 +885,7 @@ void ff_hevc_hls_filter(HEVCLocalContext *lc, const HEVCPPS *pps, skip = 1; if (!skip) - deblocking_filter_CTB(s, pps, sps, x, y); + deblocking_filter_CTB(s, l, pps, sps, x, y); if (sps->sao_enabled && !skip) { int y_end = y >= sps->height - ctb_size; if (y && x) @@ -904,15 +906,16 @@ void ff_hevc_hls_filter(HEVCLocalContext *lc, const HEVCPPS *pps, ff_progress_frame_report(&s->cur_frame->tf, y + ctb_size - 4); } -void ff_hevc_hls_filters(HEVCLocalContext *lc, const HEVCPPS *pps, +void ff_hevc_hls_filters(HEVCLocalContext *lc, const HEVCLayerContext *l, + const HEVCPPS *pps, int x_ctb, int y_ctb, int ctb_size) { int x_end = x_ctb >= pps->sps->width - ctb_size; int y_end = y_ctb >= pps->sps->height - ctb_size; if (y_ctb && x_ctb) - ff_hevc_hls_filter(lc, pps, x_ctb - ctb_size, y_ctb - ctb_size, ctb_size); + ff_hevc_hls_filter(lc, l, pps, x_ctb - ctb_size, y_ctb - ctb_size, ctb_size); if (y_ctb && x_end) - ff_hevc_hls_filter(lc, pps, x_ctb, y_ctb - ctb_size, ctb_size); + ff_hevc_hls_filter(lc, l, pps, x_ctb, y_ctb - ctb_size, ctb_size); if (x_ctb && y_end) - ff_hevc_hls_filter(lc, pps, x_ctb - ctb_size, y_ctb, ctb_size); + ff_hevc_hls_filter(lc, l, pps, x_ctb - ctb_size, y_ctb, ctb_size); } diff --git a/libavcodec/hevc/hevcdec.c b/libavcodec/hevc/hevcdec.c index b44bc93507..4978aa745c 100644 --- a/libavcodec/hevc/hevcdec.c +++ b/libavcodec/hevc/hevcdec.c @@ -90,7 +90,7 @@ static void pic_arrays_free(HEVCContext *s) } /* allocate arrays that depend on frame dimensions */ -static int pic_arrays_init(HEVCContext *s, const HEVCSPS *sps) +static int pic_arrays_init(HEVCContext *s, HEVCLayerContext *l, const HEVCSPS *sps) { int log2_min_cb_size = sps->log2_min_cb_size; int width = sps->width; @@ -100,8 +100,8 @@ static int pic_arrays_init(HEVCContext *s, const HEVCSPS *sps) int ctb_count = sps->ctb_width * sps->ctb_height; int min_pu_size = sps->min_pu_width * sps->min_pu_height; - s->bs_width = (width >> 2) + 1; - s->bs_height = (height >> 2) + 1; + l->bs_width = (width >> 2) + 1; + l->bs_height = (height >> 2) + 1; s->sao = av_calloc(ctb_count, sizeof(*s->sao)); s->deblock = av_calloc(ctb_count, sizeof(*s->deblock)); @@ -127,8 +127,8 @@ static int pic_arrays_init(HEVCContext *s, const HEVCSPS *sps) if (!s->qp_y_tab || !s->filter_slice_edges || !s->tab_slice_address) goto fail; - s->horizontal_bs = av_calloc(s->bs_width, s->bs_height); - s->vertical_bs = av_calloc(s->bs_width, s->bs_height); + s->horizontal_bs = av_calloc(l->bs_width, l->bs_height); + s->vertical_bs = av_calloc(l->bs_width, l->bs_height); if (!s->horizontal_bs || !s->vertical_bs) goto fail; @@ -527,7 +527,7 @@ static enum AVPixelFormat get_format(HEVCContext *s, const HEVCSPS *sps) return ff_get_format(s->avctx, pix_fmts); } -static int set_sps(HEVCContext *s, const HEVCSPS *sps) +static int set_sps(HEVCContext *s, HEVCLayerContext *l, const HEVCSPS *sps) { int ret, i; @@ -538,7 +538,7 @@ static int set_sps(HEVCContext *s, const HEVCSPS *sps) if (!sps) return 0; - ret = pic_arrays_init(s, sps); + ret = pic_arrays_init(s, l, sps); if (ret < 0) goto fail; @@ -1317,6 +1317,7 @@ static void set_deblocking_bypass(const HEVCContext *s, const HEVCSPS *sps, } static int hls_transform_tree(HEVCLocalContext *lc, + const HEVCLayerContext *l, const HEVCPPS *pps, const HEVCSPS *sps, int x0, int y0, int xBase, int yBase, int cb_xBase, int cb_yBase, @@ -1391,7 +1392,7 @@ static int hls_transform_tree(HEVCLocalContext *lc, #define SUBDIVIDE(x, y, idx) \ do { \ - ret = hls_transform_tree(lc, pps, sps, \ + ret = hls_transform_tree(lc, l, pps, sps, \ x, y, x0, y0, cb_xBase, cb_yBase, log2_cb_size, \ log2_trafo_size - 1, trafo_depth + 1, idx, \ cbf_cb, cbf_cr); \ @@ -1434,7 +1435,7 @@ do { } } if (!s->sh.disable_deblocking_filter_flag) { - ff_hevc_deblocking_boundary_strengths(lc, pps, x0, y0, log2_trafo_size); + ff_hevc_deblocking_boundary_strengths(lc, l, pps, x0, y0, log2_trafo_size); if (pps->transquant_bypass_enable_flag && lc->cu.cu_transquant_bypass_flag) set_deblocking_bypass(s, sps, x0, y0, log2_trafo_size); @@ -1443,8 +1444,8 @@ do { return 0; } -static int hls_pcm_sample(HEVCLocalContext *lc, const HEVCPPS *pps, - int x0, int y0, int log2_cb_size) +static int hls_pcm_sample(HEVCLocalContext *lc, const HEVCLayerContext *l, + const HEVCPPS *pps, int x0, int y0, int log2_cb_size) { const HEVCContext *const s = lc->parent; const HEVCSPS *const sps = pps->sps; @@ -1465,7 +1466,7 @@ static int hls_pcm_sample(HEVCLocalContext *lc, const HEVCPPS *pps, int ret; if (!s->sh.disable_deblocking_filter_flag) - ff_hevc_deblocking_boundary_strengths(lc, pps, x0, y0, log2_cb_size); + ff_hevc_deblocking_boundary_strengths(lc, l, pps, x0, y0, log2_cb_size); ret = init_get_bits(&gb, pcm, length); if (ret < 0) @@ -2200,6 +2201,7 @@ static void intra_prediction_unit_default_value(HEVCLocalContext *lc, } static int hls_coding_unit(HEVCLocalContext *lc, const HEVCContext *s, + const HEVCLayerContext *l, const HEVCPPS *pps, const HEVCSPS *sps, int x0, int y0, int log2_cb_size) { @@ -2255,7 +2257,7 @@ static int hls_coding_unit(HEVCLocalContext *lc, const HEVCContext *s, intra_prediction_unit_default_value(lc, sps, x0, y0, log2_cb_size); if (!s->sh.disable_deblocking_filter_flag) - ff_hevc_deblocking_boundary_strengths(lc, pps, x0, y0, log2_cb_size); + ff_hevc_deblocking_boundary_strengths(lc, l, pps, x0, y0, log2_cb_size); } else { int pcm_flag = 0; @@ -2276,7 +2278,7 @@ static int hls_coding_unit(HEVCLocalContext *lc, const HEVCContext *s, } if (pcm_flag) { intra_prediction_unit_default_value(lc, sps, x0, y0, log2_cb_size); - ret = hls_pcm_sample(lc, pps, x0, y0, log2_cb_size); + ret = hls_pcm_sample(lc, l, pps, x0, y0, log2_cb_size); if (sps->pcm_loop_filter_disabled) set_deblocking_bypass(s, sps, x0, y0, log2_cb_size); @@ -2353,14 +2355,14 @@ static int hls_coding_unit(HEVCLocalContext *lc, const HEVCContext *s, lc->cu.max_trafo_depth = lc->cu.pred_mode == MODE_INTRA ? sps->max_transform_hierarchy_depth_intra + lc->cu.intra_split_flag : sps->max_transform_hierarchy_depth_inter; - ret = hls_transform_tree(lc, pps, sps, x0, y0, x0, y0, x0, y0, + ret = hls_transform_tree(lc, l, pps, sps, x0, y0, x0, y0, x0, y0, log2_cb_size, log2_cb_size, 0, 0, cbf, cbf); if (ret < 0) return ret; } else { if (!s->sh.disable_deblocking_filter_flag) - ff_hevc_deblocking_boundary_strengths(lc, pps, x0, y0, log2_cb_size); + ff_hevc_deblocking_boundary_strengths(lc, l, pps, x0, y0, log2_cb_size); } } } @@ -2385,6 +2387,7 @@ static int hls_coding_unit(HEVCLocalContext *lc, const HEVCContext *s, } static int hls_coding_quadtree(HEVCLocalContext *lc, + const HEVCLayerContext *l, const HEVCPPS *pps, const HEVCSPS *sps, int x0, int y0, int log2_cb_size, int cb_depth) @@ -2421,23 +2424,27 @@ static int hls_coding_quadtree(HEVCLocalContext *lc, int more_data = 0; - more_data = hls_coding_quadtree(lc, pps, sps, x0, y0, log2_cb_size - 1, cb_depth + 1); + more_data = hls_coding_quadtree(lc, l, pps, sps, + x0, y0, log2_cb_size - 1, cb_depth + 1); if (more_data < 0) return more_data; if (more_data && x1 < sps->width) { - more_data = hls_coding_quadtree(lc, pps, sps, x1, y0, log2_cb_size - 1, cb_depth + 1); + more_data = hls_coding_quadtree(lc, l, pps, sps, + x1, y0, log2_cb_size - 1, cb_depth + 1); if (more_data < 0) return more_data; } if (more_data && y1 < sps->height) { - more_data = hls_coding_quadtree(lc, pps, sps, x0, y1, log2_cb_size - 1, cb_depth + 1); + more_data = hls_coding_quadtree(lc, l, pps, sps, + x0, y1, log2_cb_size - 1, cb_depth + 1); if (more_data < 0) return more_data; } if (more_data && x1 < sps->width && y1 < sps->height) { - more_data = hls_coding_quadtree(lc, pps, sps, x1, y1, log2_cb_size - 1, cb_depth + 1); + more_data = hls_coding_quadtree(lc, l, pps, sps, + x1, y1, log2_cb_size - 1, cb_depth + 1); if (more_data < 0) return more_data; } @@ -2452,7 +2459,7 @@ static int hls_coding_quadtree(HEVCLocalContext *lc, else return 0; } else { - ret = hls_coding_unit(lc, s, pps, sps, x0, y0, log2_cb_size); + ret = hls_coding_unit(lc, s, l, pps, sps, x0, y0, log2_cb_size); if (ret < 0) return ret; if ((!((x0 + cb_size) % @@ -2524,6 +2531,7 @@ static void hls_decode_neighbour(HEVCLocalContext *lc, static int hls_decode_entry(HEVCContext *s, GetBitContext *gb) { HEVCLocalContext *const lc = &s->local_ctx[0]; + const HEVCLayerContext *const l = &s->layers[s->cur_layer]; const HEVCPPS *const pps = s->pps; const HEVCSPS *const sps = pps->sps; const uint8_t *slice_data = gb->buffer + s->sh.data_offset; @@ -2555,7 +2563,7 @@ static int hls_decode_entry(HEVCContext *s, GetBitContext *gb) s->deblock[ctb_addr_rs].tc_offset = s->sh.tc_offset; s->filter_slice_edges[ctb_addr_rs] = s->sh.slice_loop_filter_across_slices_enabled_flag; - more_data = hls_coding_quadtree(lc, pps, sps, x_ctb, y_ctb, sps->log2_ctb_size, 0); + more_data = hls_coding_quadtree(lc, l, pps, sps, x_ctb, y_ctb, sps->log2_ctb_size, 0); if (more_data < 0) { s->tab_slice_address[ctb_addr_rs] = -1; return more_data; @@ -2564,12 +2572,12 @@ static int hls_decode_entry(HEVCContext *s, GetBitContext *gb) ctb_addr_ts++; ff_hevc_save_states(lc, pps, ctb_addr_ts); - ff_hevc_hls_filters(lc, pps, x_ctb, y_ctb, ctb_size); + ff_hevc_hls_filters(lc, l, pps, x_ctb, y_ctb, ctb_size); } if (x_ctb + ctb_size >= sps->width && y_ctb + ctb_size >= sps->height) - ff_hevc_hls_filter(lc, pps, x_ctb, y_ctb, ctb_size); + ff_hevc_hls_filter(lc, l, pps, x_ctb, y_ctb, ctb_size); return ctb_addr_ts; } @@ -2579,6 +2587,7 @@ static int hls_decode_entry_wpp(AVCodecContext *avctx, void *hevc_lclist, { HEVCLocalContext *lc = &((HEVCLocalContext*)hevc_lclist)[self_id]; const HEVCContext *const s = lc->parent; + const HEVCLayerContext *const l = &s->layers[s->cur_layer]; const HEVCPPS *const pps = s->pps; const HEVCSPS *const sps = pps->sps; int ctb_size = 1 << sps->log2_ctb_size; @@ -2617,7 +2626,7 @@ static int hls_decode_entry_wpp(AVCodecContext *avctx, void *hevc_lclist, goto error; hls_sao_param(lc, pps, sps, x_ctb >> sps->log2_ctb_size, y_ctb >> sps->log2_ctb_size); - more_data = hls_coding_quadtree(lc, pps, sps, x_ctb, y_ctb, sps->log2_ctb_size, 0); + more_data = hls_coding_quadtree(lc, l, pps, sps, x_ctb, y_ctb, sps->log2_ctb_size, 0); if (more_data < 0) { ret = more_data; @@ -2628,7 +2637,7 @@ static int hls_decode_entry_wpp(AVCodecContext *avctx, void *hevc_lclist, ff_hevc_save_states(lc, pps, ctb_addr_ts); ff_thread_report_progress2(s->avctx, ctb_row, thread, 1); - ff_hevc_hls_filters(lc, pps, x_ctb, y_ctb, ctb_size); + ff_hevc_hls_filters(lc, l, pps, x_ctb, y_ctb, ctb_size); if (!more_data && (x_ctb+ctb_size) < sps->width && ctb_row != s->sh.num_entry_point_offsets) { /* Casting const away here is safe, because it is an atomic operation. */ @@ -2638,7 +2647,7 @@ static int hls_decode_entry_wpp(AVCodecContext *avctx, void *hevc_lclist, } if ((x_ctb+ctb_size) >= sps->width && (y_ctb+ctb_size) >= sps->height ) { - ff_hevc_hls_filter(lc, pps, x_ctb, y_ctb, ctb_size); + ff_hevc_hls_filter(lc, l, pps, x_ctb, y_ctb, ctb_size); ff_thread_report_progress2(s->avctx, ctb_row , thread, SHIFT_CTB_WPP); return ctb_addr_ts; } @@ -2913,7 +2922,7 @@ static int hevc_frame_start(HEVCContext *s, HEVCLayerContext *l) if (sps->vps != s->vps && l != &s->layers[0]) { av_log(s->avctx, AV_LOG_ERROR, "VPS changed in a non-base layer\n"); - set_sps(s, NULL, AV_PIX_FMT_NONE); + set_sps(s, l, NULL); return AVERROR_INVALIDDATA; } @@ -2923,7 +2932,7 @@ static int hevc_frame_start(HEVCContext *s, HEVCLayerContext *l) ff_hevc_clear_refs(l); - ret = set_sps(s, sps); + ret = set_sps(s, l, sps); if (ret < 0) return ret; @@ -2937,8 +2946,8 @@ static int hevc_frame_start(HEVCContext *s, HEVCLayerContext *l) new_sequence = 1; } - memset(s->horizontal_bs, 0, s->bs_width * s->bs_height); - memset(s->vertical_bs, 0, s->bs_width * s->bs_height); + memset(s->horizontal_bs, 0, l->bs_width * l->bs_height); + memset(s->vertical_bs, 0, l->bs_width * l->bs_height); memset(s->cbf_luma, 0, sps->min_tb_width * sps->min_tb_height); memset(s->is_pcm, 0, (sps->min_pu_width + 1) * (sps->min_pu_height + 1)); memset(s->tab_slice_address, -1, pic_size_in_ctb * sizeof(*s->tab_slice_address)); @@ -3627,7 +3636,7 @@ static int hevc_update_thread_context(AVCodecContext *dst, ff_refstruct_unref(&s->pps); if (s->ps.sps != s0->ps.sps) - if ((ret = set_sps(s, s0->ps.sps)) < 0) + if ((ret = set_sps(s, &s->layers[0], s0->ps.sps)) < 0) return ret; s->poc_tid0 = s0->poc_tid0; diff --git a/libavcodec/hevc/hevcdec.h b/libavcodec/hevc/hevcdec.h index be5d76c183..18b15e7dc1 100644 --- a/libavcodec/hevc/hevcdec.h +++ b/libavcodec/hevc/hevcdec.h @@ -440,6 +440,9 @@ typedef struct HEVCLocalContext { typedef struct HEVCLayerContext { HEVCFrame DPB[32]; + + int bs_width; + int bs_height; } HEVCLayerContext; typedef struct HEVCContext { @@ -484,8 +487,6 @@ typedef struct HEVCContext { int slice_idx; ///< number of the slice being currently decoded int eos; ///< current packet contains an EOS/EOB NAL int last_eos; ///< last packet contains an EOS/EOB NAL - int bs_width; - int bs_height; // NoRaslOutputFlag associated with the last IRAP frame int no_rasl_output_flag; @@ -649,13 +650,16 @@ void ff_hevc_luma_mv_mvp_mode(HEVCLocalContext *lc, const HEVCPPS *pps, int nPbW, int nPbH, int log2_cb_size, int part_idx, int merge_idx, MvField *mv, int mvp_lx_flag, int LX); -void ff_hevc_hls_filter(HEVCLocalContext *lc, const HEVCPPS *pps, +void ff_hevc_hls_filter(HEVCLocalContext *lc, const HEVCLayerContext *l, + const HEVCPPS *pps, int x, int y, int ctb_size); -void ff_hevc_hls_filters(HEVCLocalContext *lc, const HEVCPPS *pps, +void ff_hevc_hls_filters(HEVCLocalContext *lc, const HEVCLayerContext *l, + const HEVCPPS *pps, int x_ctb, int y_ctb, int ctb_size); void ff_hevc_set_qPy(HEVCLocalContext *lc, const HEVCPPS *pps, int xBase, int yBase, int log2_cb_size); -void ff_hevc_deblocking_boundary_strengths(HEVCLocalContext *lc, const HEVCPPS *pps, +void ff_hevc_deblocking_boundary_strengths(HEVCLocalContext *lc, const HEVCLayerContext *l, + const HEVCPPS *pps, int x0, int y0, int log2_trafo_size); int ff_hevc_cu_qp_delta_sign_flag(HEVCLocalContext *lc); int ff_hevc_cu_qp_delta_abs(HEVCLocalContext *lc); -- 2.43.0
--- libavcodec/hevc/filter.c | 29 +++++++++++++++-------------- libavcodec/hevc/hevcdec.c | 29 +++++++++++++++-------------- libavcodec/hevc/hevcdec.h | 3 ++- 3 files changed, 32 insertions(+), 29 deletions(-) diff --git a/libavcodec/hevc/filter.c b/libavcodec/hevc/filter.c index 18c73dc2fa..78f408f170 100644 --- a/libavcodec/hevc/filter.c +++ b/libavcodec/hevc/filter.c @@ -264,7 +264,8 @@ static void restore_tqb_pixels(const HEVCContext *s, #define CTB(tab, x, y) ((tab)[(y) * sps->ctb_width + (x)]) -static void sao_filter_CTB(HEVCLocalContext *lc, const HEVCContext *s, +static void sao_filter_CTB(HEVCLocalContext *lc, const HEVCLayerContext *l, + const HEVCContext *s, const HEVCPPS *pps, const HEVCSPS *sps, int x, int y) { @@ -275,7 +276,7 @@ static void sao_filter_CTB(HEVCLocalContext *lc, const HEVCContext *s, int y_ctb = y >> sps->log2_ctb_size; int ctb_addr_rs = y_ctb * sps->ctb_width + x_ctb; int ctb_addr_ts = pps->ctb_addr_rs_to_ts[ctb_addr_rs]; - SAOParams *sao = &CTB(s->sao, x_ctb, y_ctb); + SAOParams *sao = &CTB(l->sao, x_ctb, y_ctb); // flags indicating unfilterable edges uint8_t vert_edge[] = { 0, 0 }; uint8_t horiz_edge[] = { 0, 0 }; @@ -385,17 +386,17 @@ static void sao_filter_CTB(HEVCLocalContext *lc, const HEVCContext *s, src1[1] = s->sao_pixel_buffer_h[c_idx] + (((2 * y_ctb - 1) * w + x0 - left) << sh); pos = 0; if (left) { - src_idx = (CTB(s->sao, x_ctb-1, y_ctb-1).type_idx[c_idx] == + src_idx = (CTB(l->sao, x_ctb-1, y_ctb-1).type_idx[c_idx] == SAO_APPLIED); copy_pixel(dst1, src1[src_idx], sh); pos += (1 << sh); } - src_idx = (CTB(s->sao, x_ctb, y_ctb-1).type_idx[c_idx] == + src_idx = (CTB(l->sao, x_ctb, y_ctb-1).type_idx[c_idx] == SAO_APPLIED); memcpy(dst1 + pos, src1[src_idx] + pos, width << sh); if (right) { pos += width << sh; - src_idx = (CTB(s->sao, x_ctb+1, y_ctb-1).type_idx[c_idx] == + src_idx = (CTB(l->sao, x_ctb+1, y_ctb-1).type_idx[c_idx] == SAO_APPLIED); copy_pixel(dst1 + pos, src1[src_idx] + pos, sh); } @@ -412,24 +413,24 @@ static void sao_filter_CTB(HEVCLocalContext *lc, const HEVCContext *s, src1[1] = s->sao_pixel_buffer_h[c_idx] + (((2 * y_ctb + 2) * w + x0 - left) << sh); pos = 0; if (left) { - src_idx = (CTB(s->sao, x_ctb-1, y_ctb+1).type_idx[c_idx] == + src_idx = (CTB(l->sao, x_ctb-1, y_ctb+1).type_idx[c_idx] == SAO_APPLIED); copy_pixel(dst1, src1[src_idx], sh); pos += (1 << sh); } - src_idx = (CTB(s->sao, x_ctb, y_ctb+1).type_idx[c_idx] == + src_idx = (CTB(l->sao, x_ctb, y_ctb+1).type_idx[c_idx] == SAO_APPLIED); memcpy(dst1 + pos, src1[src_idx] + pos, width << sh); if (right) { pos += width << sh; - src_idx = (CTB(s->sao, x_ctb+1, y_ctb+1).type_idx[c_idx] == + src_idx = (CTB(l->sao, x_ctb+1, y_ctb+1).type_idx[c_idx] == SAO_APPLIED); copy_pixel(dst1 + pos, src1[src_idx] + pos, sh); } } left_pixels = 0; if (!left_edge) { - if (CTB(s->sao, x_ctb-1, y_ctb).type_idx[c_idx] == SAO_APPLIED) { + if (CTB(l->sao, x_ctb-1, y_ctb).type_idx[c_idx] == SAO_APPLIED) { copy_vert(dst - (1 << sh), s->sao_pixel_buffer_v[c_idx] + (((2 * x_ctb - 1) * h + y0) << sh), sh, height, stride_dst, 1 << sh); @@ -439,7 +440,7 @@ static void sao_filter_CTB(HEVCLocalContext *lc, const HEVCContext *s, } right_pixels = 0; if (!right_edge) { - if (CTB(s->sao, x_ctb+1, y_ctb).type_idx[c_idx] == SAO_APPLIED) { + if (CTB(l->sao, x_ctb+1, y_ctb).type_idx[c_idx] == SAO_APPLIED) { copy_vert(dst + (width << sh), s->sao_pixel_buffer_v[c_idx] + (((2 * x_ctb + 2) * h + y0) << sh), sh, height, stride_dst, 1 << sh); @@ -889,16 +890,16 @@ void ff_hevc_hls_filter(HEVCLocalContext *lc, const HEVCLayerContext *l, if (sps->sao_enabled && !skip) { int y_end = y >= sps->height - ctb_size; if (y && x) - sao_filter_CTB(lc, s, pps, sps, x - ctb_size, y - ctb_size); + sao_filter_CTB(lc, l, s, pps, sps, x - ctb_size, y - ctb_size); if (x && y_end) - sao_filter_CTB(lc, s, pps, sps, x - ctb_size, y); + sao_filter_CTB(lc, l, s, pps, sps, x - ctb_size, y); if (y && x_end) { - sao_filter_CTB(lc, s, pps, sps, x, y - ctb_size); + sao_filter_CTB(lc, l, s, pps, sps, x, y - ctb_size); if (s->avctx->active_thread_type & FF_THREAD_FRAME ) ff_progress_frame_report(&s->cur_frame->tf, y); } if (x_end && y_end) { - sao_filter_CTB(lc, s, pps, sps, x , y); + sao_filter_CTB(lc, l, s, pps, sps, x , y); if (s->avctx->active_thread_type & FF_THREAD_FRAME ) ff_progress_frame_report(&s->cur_frame->tf, y + ctb_size); } diff --git a/libavcodec/hevc/hevcdec.c b/libavcodec/hevc/hevcdec.c index 4978aa745c..0bb4d43ed4 100644 --- a/libavcodec/hevc/hevcdec.c +++ b/libavcodec/hevc/hevcdec.c @@ -66,9 +66,9 @@ static const uint8_t hevc_pel_weight[65] = { [2] = 0, [4] = 1, [6] = 2, [8] = 3, */ /* free everything allocated by pic_arrays_init() */ -static void pic_arrays_free(HEVCContext *s) +static void pic_arrays_free(HEVCContext *s, HEVCLayerContext *l) { - av_freep(&s->sao); + av_freep(&l->sao); av_freep(&s->deblock); av_freep(&s->skip_flag); @@ -103,9 +103,9 @@ static int pic_arrays_init(HEVCContext *s, HEVCLayerContext *l, const HEVCSPS *s l->bs_width = (width >> 2) + 1; l->bs_height = (height >> 2) + 1; - s->sao = av_calloc(ctb_count, sizeof(*s->sao)); + l->sao = av_calloc(ctb_count, sizeof(*l->sao)); s->deblock = av_calloc(ctb_count, sizeof(*s->deblock)); - if (!s->sao || !s->deblock) + if (!l->sao || !s->deblock) goto fail; s->skip_flag = av_malloc_array(sps->min_cb_height, sps->min_cb_width); @@ -140,7 +140,7 @@ static int pic_arrays_init(HEVCContext *s, HEVCLayerContext *l, const HEVCSPS *s return 0; fail: - pic_arrays_free(s); + pic_arrays_free(s, l); return AVERROR(ENOMEM); } @@ -531,7 +531,7 @@ static int set_sps(HEVCContext *s, HEVCLayerContext *l, const HEVCSPS *sps) { int ret, i; - pic_arrays_free(s); + pic_arrays_free(s, l); s->ps.sps = NULL; ff_refstruct_unref(&s->vps); @@ -576,7 +576,7 @@ static int set_sps(HEVCContext *s, HEVCLayerContext *l, const HEVCSPS *sps) return 0; fail: - pic_arrays_free(s); + pic_arrays_free(s, l); for (i = 0; i < 3; i++) { av_freep(&s->sao_pixel_buffer_h[i]); av_freep(&s->sao_pixel_buffer_v[i]); @@ -990,21 +990,21 @@ do { \ if (!sao_merge_up_flag && !sao_merge_left_flag) \ sao->elem = value; \ else if (sao_merge_left_flag) \ - sao->elem = CTB(s->sao, rx-1, ry).elem; \ + sao->elem = CTB(l->sao, rx-1, ry).elem; \ else if (sao_merge_up_flag) \ - sao->elem = CTB(s->sao, rx, ry-1).elem; \ + sao->elem = CTB(l->sao, rx, ry-1).elem; \ else \ sao->elem = 0; \ } while (0) -static void hls_sao_param(HEVCLocalContext *lc, +static void hls_sao_param(HEVCLocalContext *lc, const HEVCLayerContext *l, const HEVCPPS *pps, const HEVCSPS *sps, int rx, int ry) { const HEVCContext *const s = lc->parent; int sao_merge_left_flag = 0; int sao_merge_up_flag = 0; - SAOParams *sao = &CTB(s->sao, rx, ry); + SAOParams *sao = &CTB(l->sao, rx, ry); int c_idx, i; if (s->sh.slice_sample_adaptive_offset_flag[0] || @@ -2556,7 +2556,7 @@ static int hls_decode_entry(HEVCContext *s, GetBitContext *gb) return ret; } - hls_sao_param(lc, pps, sps, + hls_sao_param(lc, l, pps, sps, x_ctb >> sps->log2_ctb_size, y_ctb >> sps->log2_ctb_size); s->deblock[ctb_addr_rs].beta_offset = s->sh.beta_offset; @@ -2624,7 +2624,7 @@ static int hls_decode_entry_wpp(AVCodecContext *avctx, void *hevc_lclist, ret = ff_hevc_cabac_init(lc, pps, ctb_addr_ts, data, data_size, 1); if (ret < 0) goto error; - hls_sao_param(lc, pps, sps, + hls_sao_param(lc, l, pps, sps, x_ctb >> sps->log2_ctb_size, y_ctb >> sps->log2_ctb_size); more_data = hls_coding_quadtree(lc, l, pps, sps, x_ctb, y_ctb, sps->log2_ctb_size, 0); @@ -3518,7 +3518,8 @@ static av_cold int hevc_decode_free(AVCodecContext *avctx) HEVCContext *s = avctx->priv_data; int i; - pic_arrays_free(s); + for (int i = 0; i < FF_ARRAY_ELEMS(s->layers); i++) + pic_arrays_free(s, &s->layers[i]); ff_refstruct_unref(&s->vps); ff_refstruct_unref(&s->pps); diff --git a/libavcodec/hevc/hevcdec.h b/libavcodec/hevc/hevcdec.h index 18b15e7dc1..e0b026eef6 100644 --- a/libavcodec/hevc/hevcdec.h +++ b/libavcodec/hevc/hevcdec.h @@ -443,6 +443,8 @@ typedef struct HEVCLayerContext { int bs_width; int bs_height; + + SAOParams *sao; } HEVCLayerContext; typedef struct HEVCContext { @@ -476,7 +478,6 @@ typedef struct HEVCContext { const HEVCVPS *vps; ///< RefStruct reference const HEVCPPS *pps; ///< RefStruct reference SliceHeader sh; - SAOParams *sao; DBParams *deblock; enum HEVCNALUnitType nal_unit_type; int temporal_id; ///< temporal_id_plus1 - 1 -- 2.43.0
--- libavcodec/hevc/filter.c | 8 ++++---- libavcodec/hevc/hevcdec.c | 10 +++++----- libavcodec/hevc/hevcdec.h | 2 +- 3 files changed, 10 insertions(+), 10 deletions(-) diff --git a/libavcodec/hevc/filter.c b/libavcodec/hevc/filter.c index 78f408f170..4d5ef631d7 100644 --- a/libavcodec/hevc/filter.c +++ b/libavcodec/hevc/filter.c @@ -515,8 +515,8 @@ static void deblocking_filter_CTB(const HEVCContext *s, const HEVCLayerContext * int ctb_size = 1 << log2_ctb_size; int ctb = (x0 >> log2_ctb_size) + (y0 >> log2_ctb_size) * sps->ctb_width; - int cur_tc_offset = s->deblock[ctb].tc_offset; - int cur_beta_offset = s->deblock[ctb].beta_offset; + int cur_tc_offset = l->deblock[ctb].tc_offset; + int cur_beta_offset = l->deblock[ctb].beta_offset; int left_tc_offset, left_beta_offset; int tc_offset, beta_offset; int pcmf = (sps->pcm_enabled && @@ -524,8 +524,8 @@ static void deblocking_filter_CTB(const HEVCContext *s, const HEVCLayerContext * pps->transquant_bypass_enable_flag; if (x0) { - left_tc_offset = s->deblock[ctb - 1].tc_offset; - left_beta_offset = s->deblock[ctb - 1].beta_offset; + left_tc_offset = l->deblock[ctb - 1].tc_offset; + left_beta_offset = l->deblock[ctb - 1].beta_offset; } else { left_tc_offset = 0; left_beta_offset = 0; diff --git a/libavcodec/hevc/hevcdec.c b/libavcodec/hevc/hevcdec.c index 0bb4d43ed4..a20e559353 100644 --- a/libavcodec/hevc/hevcdec.c +++ b/libavcodec/hevc/hevcdec.c @@ -69,7 +69,7 @@ static const uint8_t hevc_pel_weight[65] = { [2] = 0, [4] = 1, [6] = 2, [8] = 3, static void pic_arrays_free(HEVCContext *s, HEVCLayerContext *l) { av_freep(&l->sao); - av_freep(&s->deblock); + av_freep(&l->deblock); av_freep(&s->skip_flag); av_freep(&s->tab_ct_depth); @@ -104,8 +104,8 @@ static int pic_arrays_init(HEVCContext *s, HEVCLayerContext *l, const HEVCSPS *s l->bs_height = (height >> 2) + 1; l->sao = av_calloc(ctb_count, sizeof(*l->sao)); - s->deblock = av_calloc(ctb_count, sizeof(*s->deblock)); - if (!l->sao || !s->deblock) + l->deblock = av_calloc(ctb_count, sizeof(*l->deblock)); + if (!l->sao || !l->deblock) goto fail; s->skip_flag = av_malloc_array(sps->min_cb_height, sps->min_cb_width); @@ -2559,8 +2559,8 @@ static int hls_decode_entry(HEVCContext *s, GetBitContext *gb) hls_sao_param(lc, l, pps, sps, x_ctb >> sps->log2_ctb_size, y_ctb >> sps->log2_ctb_size); - s->deblock[ctb_addr_rs].beta_offset = s->sh.beta_offset; - s->deblock[ctb_addr_rs].tc_offset = s->sh.tc_offset; + l->deblock[ctb_addr_rs].beta_offset = s->sh.beta_offset; + l->deblock[ctb_addr_rs].tc_offset = s->sh.tc_offset; s->filter_slice_edges[ctb_addr_rs] = s->sh.slice_loop_filter_across_slices_enabled_flag; more_data = hls_coding_quadtree(lc, l, pps, sps, x_ctb, y_ctb, sps->log2_ctb_size, 0); diff --git a/libavcodec/hevc/hevcdec.h b/libavcodec/hevc/hevcdec.h index e0b026eef6..22b34924d5 100644 --- a/libavcodec/hevc/hevcdec.h +++ b/libavcodec/hevc/hevcdec.h @@ -445,6 +445,7 @@ typedef struct HEVCLayerContext { int bs_height; SAOParams *sao; + DBParams *deblock; } HEVCLayerContext; typedef struct HEVCContext { @@ -478,7 +479,6 @@ typedef struct HEVCContext { const HEVCVPS *vps; ///< RefStruct reference const HEVCPPS *pps; ///< RefStruct reference SliceHeader sh; - DBParams *deblock; enum HEVCNALUnitType nal_unit_type; int temporal_id; ///< temporal_id_plus1 - 1 HEVCFrame *cur_frame; -- 2.43.0
--- libavcodec/hevc/cabac.c | 9 +++---- libavcodec/hevc/hevcdec.c | 56 ++++++++++++++++++++------------------- libavcodec/hevc/hevcdec.h | 8 +++--- 3 files changed, 38 insertions(+), 35 deletions(-) diff --git a/libavcodec/hevc/cabac.c b/libavcodec/hevc/cabac.c index 33f8241bb3..12d477b2bd 100644 --- a/libavcodec/hevc/cabac.c +++ b/libavcodec/hevc/cabac.c @@ -570,16 +570,15 @@ int ff_hevc_cu_transquant_bypass_flag_decode(HEVCLocalContext *lc) return GET_CABAC(CU_TRANSQUANT_BYPASS_FLAG_OFFSET); } -int ff_hevc_skip_flag_decode(HEVCLocalContext *lc, int x0, int y0, - int x_cb, int y_cb, int min_cb_width) +int ff_hevc_skip_flag_decode(HEVCLocalContext *lc, uint8_t *skip_flag, + int x0, int y0, int x_cb, int y_cb, int min_cb_width) { - const HEVCContext *const s = lc->parent; int inc = 0; if (lc->ctb_left_flag || x0) - inc = !!SAMPLE_CTB(s->skip_flag, x_cb - 1, y_cb); + inc = !!SAMPLE_CTB(skip_flag, x_cb - 1, y_cb); if (lc->ctb_up_flag || y0) - inc += !!SAMPLE_CTB(s->skip_flag, x_cb, y_cb - 1); + inc += !!SAMPLE_CTB(skip_flag, x_cb, y_cb - 1); return GET_CABAC(SKIP_FLAG_OFFSET + inc); } diff --git a/libavcodec/hevc/hevcdec.c b/libavcodec/hevc/hevcdec.c index a20e559353..2e620d8c4a 100644 --- a/libavcodec/hevc/hevcdec.c +++ b/libavcodec/hevc/hevcdec.c @@ -71,7 +71,7 @@ static void pic_arrays_free(HEVCContext *s, HEVCLayerContext *l) av_freep(&l->sao); av_freep(&l->deblock); - av_freep(&s->skip_flag); + av_freep(&l->skip_flag); av_freep(&s->tab_ct_depth); av_freep(&s->tab_ipm); @@ -108,9 +108,9 @@ static int pic_arrays_init(HEVCContext *s, HEVCLayerContext *l, const HEVCSPS *s if (!l->sao || !l->deblock) goto fail; - s->skip_flag = av_malloc_array(sps->min_cb_height, sps->min_cb_width); + l->skip_flag = av_malloc_array(sps->min_cb_height, sps->min_cb_width); s->tab_ct_depth = av_malloc_array(sps->min_cb_height, sps->min_cb_width); - if (!s->skip_flag || !s->tab_ct_depth) + if (!l->skip_flag || !s->tab_ct_depth) goto fail; s->cbf_luma = av_malloc_array(sps->min_tb_width, sps->min_tb_height); @@ -1882,6 +1882,7 @@ static void hevc_luma_mv_mvp_mode(HEVCLocalContext *lc, } static void hls_prediction_unit(HEVCLocalContext *lc, + const HEVCLayerContext *l, const HEVCPPS *pps, const HEVCSPS *sps, int x0, int y0, int nPbW, int nPbH, int log2_cb_size, int partIdx, int idx) @@ -1909,7 +1910,7 @@ static void hls_prediction_unit(HEVCLocalContext *lc, int x_pu, y_pu; int i, j; - int skip_flag = SAMPLE_CTB(s->skip_flag, x_cb, y_cb); + int skip_flag = SAMPLE_CTB(l->skip_flag, x_cb, y_cb); if (!skip_flag) lc->pu.merge_flag = ff_hevc_merge_flag_decode(lc); @@ -2221,7 +2222,7 @@ static int hls_coding_unit(HEVCLocalContext *lc, const HEVCContext *s, lc->cu.part_mode = PART_2Nx2N; lc->cu.intra_split_flag = 0; - SAMPLE_CTB(s->skip_flag, x_cb, y_cb) = 0; + SAMPLE_CTB(l->skip_flag, x_cb, y_cb) = 0; for (x = 0; x < 4; x++) lc->pu.intra_pred_mode[x] = 1; if (pps->transquant_bypass_enable_flag) { @@ -2234,25 +2235,26 @@ static int hls_coding_unit(HEVCLocalContext *lc, const HEVCContext *s, if (s->sh.slice_type != HEVC_SLICE_I) { const int x0b = av_zero_extend(x0, sps->log2_ctb_size); const int y0b = av_zero_extend(y0, sps->log2_ctb_size); - uint8_t skip_flag = ff_hevc_skip_flag_decode(lc, x0b, y0b, x_cb, y_cb, + uint8_t skip_flag = ff_hevc_skip_flag_decode(lc, l->skip_flag, + x0b, y0b, x_cb, y_cb, min_cb_width); x = y_cb * min_cb_width + x_cb; for (y = 0; y < length; y++) { - memset(&s->skip_flag[x], skip_flag, length); + memset(&l->skip_flag[x], skip_flag, length); x += min_cb_width; } lc->cu.pred_mode = skip_flag ? MODE_SKIP : MODE_INTER; } else { x = y_cb * min_cb_width + x_cb; for (y = 0; y < length; y++) { - memset(&s->skip_flag[x], 0, length); + memset(&l->skip_flag[x], 0, length); x += min_cb_width; } } - if (SAMPLE_CTB(s->skip_flag, x_cb, y_cb)) { - hls_prediction_unit(lc, pps, sps, + if (SAMPLE_CTB(l->skip_flag, x_cb, y_cb)) { + hls_prediction_unit(lc, l, pps, sps, x0, y0, cb_size, cb_size, log2_cb_size, 0, idx); intra_prediction_unit_default_value(lc, sps, x0, y0, log2_cb_size); @@ -2291,53 +2293,53 @@ static int hls_coding_unit(HEVCLocalContext *lc, const HEVCContext *s, intra_prediction_unit_default_value(lc, sps, x0, y0, log2_cb_size); switch (lc->cu.part_mode) { case PART_2Nx2N: - hls_prediction_unit(lc, pps, sps, + hls_prediction_unit(lc, l, pps, sps, x0, y0, cb_size, cb_size, log2_cb_size, 0, idx); break; case PART_2NxN: - hls_prediction_unit(lc, pps, sps, + hls_prediction_unit(lc, l, pps, sps, x0, y0, cb_size, cb_size / 2, log2_cb_size, 0, idx); - hls_prediction_unit(lc, pps, sps, + hls_prediction_unit(lc, l, pps, sps, x0, y0 + cb_size / 2, cb_size, cb_size / 2, log2_cb_size, 1, idx); break; case PART_Nx2N: - hls_prediction_unit(lc, pps, sps, + hls_prediction_unit(lc, l, pps, sps, x0, y0, cb_size / 2, cb_size, log2_cb_size, 0, idx - 1); - hls_prediction_unit(lc, pps, sps, + hls_prediction_unit(lc, l, pps, sps, x0 + cb_size / 2, y0, cb_size / 2, cb_size, log2_cb_size, 1, idx - 1); break; case PART_2NxnU: - hls_prediction_unit(lc, pps, sps, + hls_prediction_unit(lc, l, pps, sps, x0, y0, cb_size, cb_size / 4, log2_cb_size, 0, idx); - hls_prediction_unit(lc, pps, sps, + hls_prediction_unit(lc, l, pps, sps, x0, y0 + cb_size / 4, cb_size, cb_size * 3 / 4, log2_cb_size, 1, idx); break; case PART_2NxnD: - hls_prediction_unit(lc, pps, sps, + hls_prediction_unit(lc, l, pps, sps, x0, y0, cb_size, cb_size * 3 / 4, log2_cb_size, 0, idx); - hls_prediction_unit(lc, pps, sps, + hls_prediction_unit(lc, l, pps, sps, x0, y0 + cb_size * 3 / 4, cb_size, cb_size / 4, log2_cb_size, 1, idx); break; case PART_nLx2N: - hls_prediction_unit(lc, pps, sps, + hls_prediction_unit(lc, l, pps, sps, x0, y0, cb_size / 4, cb_size, log2_cb_size, 0, idx - 2); - hls_prediction_unit(lc, pps, sps, + hls_prediction_unit(lc, l, pps, sps, x0 + cb_size / 4, y0, cb_size * 3 / 4, cb_size, log2_cb_size, 1, idx - 2); break; case PART_nRx2N: - hls_prediction_unit(lc, pps, sps, + hls_prediction_unit(lc, l, pps, sps, x0, y0, cb_size * 3 / 4, cb_size, log2_cb_size, 0, idx - 2); - hls_prediction_unit(lc, pps, sps, + hls_prediction_unit(lc, l, pps, sps, x0 + cb_size * 3 / 4, y0, cb_size / 4, cb_size, log2_cb_size, 1, idx - 2); break; case PART_NxN: - hls_prediction_unit(lc, pps, sps, + hls_prediction_unit(lc, l, pps, sps, x0, y0, cb_size / 2, cb_size / 2, log2_cb_size, 0, idx - 1); - hls_prediction_unit(lc, pps, sps, + hls_prediction_unit(lc, l, pps, sps, x0 + cb_size / 2, y0, cb_size / 2, cb_size / 2, log2_cb_size, 1, idx - 1); - hls_prediction_unit(lc, pps, sps, + hls_prediction_unit(lc, l, pps, sps, x0, y0 + cb_size / 2, cb_size / 2, cb_size / 2, log2_cb_size, 2, idx - 1); - hls_prediction_unit(lc, pps, sps, + hls_prediction_unit(lc, l, pps, sps, x0 + cb_size / 2, y0 + cb_size / 2, cb_size / 2, cb_size / 2, log2_cb_size, 3, idx - 1); break; } diff --git a/libavcodec/hevc/hevcdec.h b/libavcodec/hevc/hevcdec.h index 22b34924d5..1d8b6daf8b 100644 --- a/libavcodec/hevc/hevcdec.h +++ b/libavcodec/hevc/hevcdec.h @@ -446,6 +446,9 @@ typedef struct HEVCLayerContext { SAOParams *sao; DBParams *deblock; + + // CU + uint8_t *skip_flag; } HEVCLayerContext; typedef struct HEVCContext { @@ -504,7 +507,6 @@ typedef struct HEVCContext { int32_t *tab_slice_address; // CU - uint8_t *skip_flag; uint8_t *tab_ct_depth; // PU uint8_t *tab_ipm; @@ -579,8 +581,8 @@ int ff_hevc_sao_offset_sign_decode(HEVCLocalContext *lc); int ff_hevc_sao_eo_class_decode(HEVCLocalContext *lc); int ff_hevc_end_of_slice_flag_decode(HEVCLocalContext *lc); int ff_hevc_cu_transquant_bypass_flag_decode(HEVCLocalContext *lc); -int ff_hevc_skip_flag_decode(HEVCLocalContext *lc, int x0, int y0, - int x_cb, int y_cb, int min_cb_width); +int ff_hevc_skip_flag_decode(HEVCLocalContext *lc, uint8_t *skip_flag, + int x0, int y0, int x_cb, int y_cb, int min_cb_width); int ff_hevc_pred_mode_decode(HEVCLocalContext *lc); int ff_hevc_split_coding_unit_flag_decode(HEVCLocalContext *lc, const HEVCSPS *sps, int ct_depth, int x0, int y0); -- 2.43.0
--- libavcodec/hevc/cabac.c | 8 ++++---- libavcodec/hevc/hevcdec.c | 11 ++++++----- libavcodec/hevc/hevcdec.h | 6 +++--- 3 files changed, 13 insertions(+), 12 deletions(-) diff --git a/libavcodec/hevc/cabac.c b/libavcodec/hevc/cabac.c index 12d477b2bd..892dd1c215 100644 --- a/libavcodec/hevc/cabac.c +++ b/libavcodec/hevc/cabac.c @@ -636,10 +636,10 @@ int ff_hevc_pred_mode_decode(HEVCLocalContext *lc) return GET_CABAC(PRED_MODE_FLAG_OFFSET); } -int ff_hevc_split_coding_unit_flag_decode(HEVCLocalContext *lc, const HEVCSPS *sps, +int ff_hevc_split_coding_unit_flag_decode(HEVCLocalContext *lc, uint8_t *tab_ct_depth, + const HEVCSPS *sps, int ct_depth, int x0, int y0) { - const HEVCContext *const s = lc->parent; int inc = 0, depth_left = 0, depth_top = 0; int x0b = av_zero_extend(x0, sps->log2_ctb_size); int y0b = av_zero_extend(y0, sps->log2_ctb_size); @@ -647,9 +647,9 @@ int ff_hevc_split_coding_unit_flag_decode(HEVCLocalContext *lc, const HEVCSPS *s int y_cb = y0 >> sps->log2_min_cb_size; if (lc->ctb_left_flag || x0b) - depth_left = s->tab_ct_depth[(y_cb) * sps->min_cb_width + x_cb - 1]; + depth_left = tab_ct_depth[(y_cb) * sps->min_cb_width + x_cb - 1]; if (lc->ctb_up_flag || y0b) - depth_top = s->tab_ct_depth[(y_cb - 1) * sps->min_cb_width + x_cb]; + depth_top = tab_ct_depth[(y_cb - 1) * sps->min_cb_width + x_cb]; inc += (depth_left > ct_depth); inc += (depth_top > ct_depth); diff --git a/libavcodec/hevc/hevcdec.c b/libavcodec/hevc/hevcdec.c index 2e620d8c4a..8f0f761ee5 100644 --- a/libavcodec/hevc/hevcdec.c +++ b/libavcodec/hevc/hevcdec.c @@ -72,7 +72,7 @@ static void pic_arrays_free(HEVCContext *s, HEVCLayerContext *l) av_freep(&l->deblock); av_freep(&l->skip_flag); - av_freep(&s->tab_ct_depth); + av_freep(&l->tab_ct_depth); av_freep(&s->tab_ipm); av_freep(&s->cbf_luma); @@ -109,8 +109,8 @@ static int pic_arrays_init(HEVCContext *s, HEVCLayerContext *l, const HEVCSPS *s goto fail; l->skip_flag = av_malloc_array(sps->min_cb_height, sps->min_cb_width); - s->tab_ct_depth = av_malloc_array(sps->min_cb_height, sps->min_cb_width); - if (!l->skip_flag || !s->tab_ct_depth) + l->tab_ct_depth = av_malloc_array(sps->min_cb_height, sps->min_cb_width); + if (!l->skip_flag || !l->tab_ct_depth) goto fail; s->cbf_luma = av_malloc_array(sps->min_tb_width, sps->min_tb_height); @@ -2383,7 +2383,7 @@ static int hls_coding_unit(HEVCLocalContext *lc, const HEVCContext *s, lc->qPy_pred = lc->qp_y; } - set_ct_depth(sps, s->tab_ct_depth, x0, y0, log2_cb_size, lc->ct_depth); + set_ct_depth(sps, l->tab_ct_depth, x0, y0, log2_cb_size, lc->ct_depth); return 0; } @@ -2403,7 +2403,8 @@ static int hls_coding_quadtree(HEVCLocalContext *lc, if (x0 + cb_size <= sps->width && y0 + cb_size <= sps->height && log2_cb_size > sps->log2_min_cb_size) { - split_cu = ff_hevc_split_coding_unit_flag_decode(lc, sps, cb_depth, x0, y0); + split_cu = ff_hevc_split_coding_unit_flag_decode(lc, l->tab_ct_depth, + sps, cb_depth, x0, y0); } else { split_cu = (log2_cb_size > sps->log2_min_cb_size); } diff --git a/libavcodec/hevc/hevcdec.h b/libavcodec/hevc/hevcdec.h index 1d8b6daf8b..59c8587787 100644 --- a/libavcodec/hevc/hevcdec.h +++ b/libavcodec/hevc/hevcdec.h @@ -449,6 +449,7 @@ typedef struct HEVCLayerContext { // CU uint8_t *skip_flag; + uint8_t *tab_ct_depth; } HEVCLayerContext; typedef struct HEVCContext { @@ -506,8 +507,6 @@ typedef struct HEVCContext { int32_t *tab_slice_address; - // CU - uint8_t *tab_ct_depth; // PU uint8_t *tab_ipm; @@ -584,7 +583,8 @@ int ff_hevc_cu_transquant_bypass_flag_decode(HEVCLocalContext *lc); int ff_hevc_skip_flag_decode(HEVCLocalContext *lc, uint8_t *skip_flag, int x0, int y0, int x_cb, int y_cb, int min_cb_width); int ff_hevc_pred_mode_decode(HEVCLocalContext *lc); -int ff_hevc_split_coding_unit_flag_decode(HEVCLocalContext *lc, const HEVCSPS *sps, +int ff_hevc_split_coding_unit_flag_decode(HEVCLocalContext *lc, uint8_t *tab_ct_depth, + const HEVCSPS *sps, int ct_depth, int x0, int y0); int ff_hevc_part_mode_decode(HEVCLocalContext *lc, const HEVCSPS *sps, int log2_cb_size); int ff_hevc_pcm_flag_decode(HEVCLocalContext *lc); -- 2.43.0
--- libavcodec/hevc/filter.c | 8 ++++---- libavcodec/hevc/hevcdec.c | 10 +++++----- libavcodec/hevc/hevcdec.h | 4 +++- 3 files changed, 12 insertions(+), 10 deletions(-) diff --git a/libavcodec/hevc/filter.c b/libavcodec/hevc/filter.c index 4d5ef631d7..379c488d78 100644 --- a/libavcodec/hevc/filter.c +++ b/libavcodec/hevc/filter.c @@ -777,8 +777,8 @@ void ff_hevc_deblocking_boundary_strengths(HEVCLocalContext *lc, const HEVCLayer int x_tu = (x0 + i) >> log2_min_tu_size; const MvField *top = &tab_mvf[yp_pu * min_pu_width + x_pu]; const MvField *curr = &tab_mvf[yq_pu * min_pu_width + x_pu]; - uint8_t top_cbf_luma = s->cbf_luma[yp_tu * min_tu_width + x_tu]; - uint8_t curr_cbf_luma = s->cbf_luma[yq_tu * min_tu_width + x_tu]; + uint8_t top_cbf_luma = l->cbf_luma[yp_tu * min_tu_width + x_tu]; + uint8_t curr_cbf_luma = l->cbf_luma[yq_tu * min_tu_width + x_tu]; if (curr->pred_flag == PF_INTRA || top->pred_flag == PF_INTRA) bs = 2; @@ -815,8 +815,8 @@ void ff_hevc_deblocking_boundary_strengths(HEVCLocalContext *lc, const HEVCLayer int y_tu = (y0 + i) >> log2_min_tu_size; const MvField *left = &tab_mvf[y_pu * min_pu_width + xp_pu]; const MvField *curr = &tab_mvf[y_pu * min_pu_width + xq_pu]; - uint8_t left_cbf_luma = s->cbf_luma[y_tu * min_tu_width + xp_tu]; - uint8_t curr_cbf_luma = s->cbf_luma[y_tu * min_tu_width + xq_tu]; + uint8_t left_cbf_luma = l->cbf_luma[y_tu * min_tu_width + xp_tu]; + uint8_t curr_cbf_luma = l->cbf_luma[y_tu * min_tu_width + xq_tu]; if (curr->pred_flag == PF_INTRA || left->pred_flag == PF_INTRA) bs = 2; diff --git a/libavcodec/hevc/hevcdec.c b/libavcodec/hevc/hevcdec.c index 8f0f761ee5..1887a44e3e 100644 --- a/libavcodec/hevc/hevcdec.c +++ b/libavcodec/hevc/hevcdec.c @@ -75,7 +75,7 @@ static void pic_arrays_free(HEVCContext *s, HEVCLayerContext *l) av_freep(&l->tab_ct_depth); av_freep(&s->tab_ipm); - av_freep(&s->cbf_luma); + av_freep(&l->cbf_luma); av_freep(&s->is_pcm); av_freep(&s->qp_y_tab); @@ -113,10 +113,10 @@ static int pic_arrays_init(HEVCContext *s, HEVCLayerContext *l, const HEVCSPS *s if (!l->skip_flag || !l->tab_ct_depth) goto fail; - s->cbf_luma = av_malloc_array(sps->min_tb_width, sps->min_tb_height); + l->cbf_luma = av_malloc_array(sps->min_tb_width, sps->min_tb_height); s->tab_ipm = av_mallocz(min_pu_size); s->is_pcm = av_malloc_array(sps->min_pu_width + 1, sps->min_pu_height + 1); - if (!s->tab_ipm || !s->cbf_luma || !s->is_pcm) + if (!s->tab_ipm || !l->cbf_luma || !s->is_pcm) goto fail; s->filter_slice_edges = av_mallocz(ctb_count); @@ -1431,7 +1431,7 @@ do { for (j = 0; j < (1 << log2_trafo_size); j += min_tu_size) { int x_tu = (x0 + j) >> log2_min_tu_size; int y_tu = (y0 + i) >> log2_min_tu_size; - s->cbf_luma[y_tu * min_tu_width + x_tu] = 1; + l->cbf_luma[y_tu * min_tu_width + x_tu] = 1; } } if (!s->sh.disable_deblocking_filter_flag) { @@ -2951,7 +2951,7 @@ static int hevc_frame_start(HEVCContext *s, HEVCLayerContext *l) memset(s->horizontal_bs, 0, l->bs_width * l->bs_height); memset(s->vertical_bs, 0, l->bs_width * l->bs_height); - memset(s->cbf_luma, 0, sps->min_tb_width * sps->min_tb_height); + memset(l->cbf_luma, 0, sps->min_tb_width * sps->min_tb_height); memset(s->is_pcm, 0, (sps->min_pu_width + 1) * (sps->min_pu_height + 1)); memset(s->tab_slice_address, -1, pic_size_in_ctb * sizeof(*s->tab_slice_address)); diff --git a/libavcodec/hevc/hevcdec.h b/libavcodec/hevc/hevcdec.h index 59c8587787..ccea7a46e1 100644 --- a/libavcodec/hevc/hevcdec.h +++ b/libavcodec/hevc/hevcdec.h @@ -450,6 +450,9 @@ typedef struct HEVCLayerContext { // CU uint8_t *skip_flag; uint8_t *tab_ct_depth; + + // PU + uint8_t *cbf_luma; // cbf_luma of colocated TU } HEVCLayerContext; typedef struct HEVCContext { @@ -510,7 +513,6 @@ typedef struct HEVCContext { // PU uint8_t *tab_ipm; - uint8_t *cbf_luma; // cbf_luma of colocated TU uint8_t *is_pcm; // CTB-level flags affecting loop filter operation -- 2.43.0
--- libavcodec/hevc/hevcdec.c | 32 ++++++++++++++++++-------------- libavcodec/hevc/hevcdec.h | 2 +- 2 files changed, 19 insertions(+), 15 deletions(-) diff --git a/libavcodec/hevc/hevcdec.c b/libavcodec/hevc/hevcdec.c index 1887a44e3e..0a3b5ba4c4 100644 --- a/libavcodec/hevc/hevcdec.c +++ b/libavcodec/hevc/hevcdec.c @@ -74,7 +74,7 @@ static void pic_arrays_free(HEVCContext *s, HEVCLayerContext *l) av_freep(&l->skip_flag); av_freep(&l->tab_ct_depth); - av_freep(&s->tab_ipm); + av_freep(&l->tab_ipm); av_freep(&l->cbf_luma); av_freep(&s->is_pcm); @@ -114,9 +114,9 @@ static int pic_arrays_init(HEVCContext *s, HEVCLayerContext *l, const HEVCSPS *s goto fail; l->cbf_luma = av_malloc_array(sps->min_tb_width, sps->min_tb_height); - s->tab_ipm = av_mallocz(min_pu_size); + l->tab_ipm = av_mallocz(min_pu_size); s->is_pcm = av_malloc_array(sps->min_pu_width + 1, sps->min_pu_height + 1); - if (!s->tab_ipm || !l->cbf_luma || !s->is_pcm) + if (!l->tab_ipm || !l->cbf_luma || !s->is_pcm) goto fail; s->filter_slice_edges = av_mallocz(ctb_count); @@ -2010,7 +2010,8 @@ static void hls_prediction_unit(HEVCLocalContext *lc, /** * 8.4.1 */ -static int luma_intra_pred_mode(HEVCLocalContext *lc, const HEVCSPS *sps, +static int luma_intra_pred_mode(HEVCLocalContext *lc, const HEVCLayerContext *l, + const HEVCSPS *sps, int x0, int y0, int pu_size, int prev_intra_luma_pred_flag) { @@ -2023,9 +2024,9 @@ static int luma_intra_pred_mode(HEVCLocalContext *lc, const HEVCSPS *sps, int y0b = av_zero_extend(y0, sps->log2_ctb_size); int cand_up = (lc->ctb_up_flag || y0b) ? - s->tab_ipm[(y_pu - 1) * min_pu_width + x_pu] : INTRA_DC; + l->tab_ipm[(y_pu - 1) * min_pu_width + x_pu] : INTRA_DC; int cand_left = (lc->ctb_left_flag || x0b) ? - s->tab_ipm[y_pu * min_pu_width + x_pu - 1] : INTRA_DC; + l->tab_ipm[y_pu * min_pu_width + x_pu - 1] : INTRA_DC; int y_ctb = (y0 >> (sps->log2_ctb_size)) << (sps->log2_ctb_size); @@ -2080,7 +2081,7 @@ static int luma_intra_pred_mode(HEVCLocalContext *lc, const HEVCSPS *sps, if (!size_in_pus) size_in_pus = 1; for (i = 0; i < size_in_pus; i++) { - memset(&s->tab_ipm[(y_pu + i) * min_pu_width + x_pu], + memset(&l->tab_ipm[(y_pu + i) * min_pu_width + x_pu], intra_pred_mode, size_in_pus); for (j = 0; j < size_in_pus; j++) { @@ -2109,7 +2110,8 @@ static const uint8_t tab_mode_idx[] = { 0, 1, 2, 2, 2, 2, 3, 5, 7, 8, 10, 12, 13, 15, 17, 18, 19, 20, 21, 22, 23, 23, 24, 24, 25, 25, 26, 27, 27, 28, 28, 29, 29, 30, 31}; -static void intra_prediction_unit(HEVCLocalContext *lc, const HEVCSPS *sps, +static void intra_prediction_unit(HEVCLocalContext *lc, + const HEVCLayerContext *l, const HEVCSPS *sps, int x0, int y0, int log2_cb_size) { @@ -2133,7 +2135,8 @@ static void intra_prediction_unit(HEVCLocalContext *lc, const HEVCSPS *sps, lc->pu.rem_intra_luma_pred_mode = ff_hevc_rem_intra_luma_pred_mode_decode(lc); lc->pu.intra_pred_mode[2 * i + j] = - luma_intra_pred_mode(lc, sps, x0 + pb_size * j, y0 + pb_size * i, pb_size, + luma_intra_pred_mode(lc, l, sps, + x0 + pb_size * j, y0 + pb_size * i, pb_size, prev_intra_luma_pred_flag[2 * i + j]); } } @@ -2178,6 +2181,7 @@ static void intra_prediction_unit(HEVCLocalContext *lc, const HEVCSPS *sps, } static void intra_prediction_unit_default_value(HEVCLocalContext *lc, + const HEVCLayerContext *l, const HEVCSPS *sps, int x0, int y0, int log2_cb_size) @@ -2194,7 +2198,7 @@ static void intra_prediction_unit_default_value(HEVCLocalContext *lc, if (size_in_pus == 0) size_in_pus = 1; for (j = 0; j < size_in_pus; j++) - memset(&s->tab_ipm[(y_pu + j) * min_pu_width + x_pu], INTRA_DC, size_in_pus); + memset(&l->tab_ipm[(y_pu + j) * min_pu_width + x_pu], INTRA_DC, size_in_pus); if (lc->cu.pred_mode == MODE_INTRA) for (j = 0; j < size_in_pus; j++) for (k = 0; k < size_in_pus; k++) @@ -2256,7 +2260,7 @@ static int hls_coding_unit(HEVCLocalContext *lc, const HEVCContext *s, if (SAMPLE_CTB(l->skip_flag, x_cb, y_cb)) { hls_prediction_unit(lc, l, pps, sps, x0, y0, cb_size, cb_size, log2_cb_size, 0, idx); - intra_prediction_unit_default_value(lc, sps, x0, y0, log2_cb_size); + intra_prediction_unit_default_value(lc, l, sps, x0, y0, log2_cb_size); if (!s->sh.disable_deblocking_filter_flag) ff_hevc_deblocking_boundary_strengths(lc, l, pps, x0, y0, log2_cb_size); @@ -2279,7 +2283,7 @@ static int hls_coding_unit(HEVCLocalContext *lc, const HEVCContext *s, pcm_flag = ff_hevc_pcm_flag_decode(lc); } if (pcm_flag) { - intra_prediction_unit_default_value(lc, sps, x0, y0, log2_cb_size); + intra_prediction_unit_default_value(lc, l, sps, x0, y0, log2_cb_size); ret = hls_pcm_sample(lc, l, pps, x0, y0, log2_cb_size); if (sps->pcm_loop_filter_disabled) set_deblocking_bypass(s, sps, x0, y0, log2_cb_size); @@ -2287,10 +2291,10 @@ static int hls_coding_unit(HEVCLocalContext *lc, const HEVCContext *s, if (ret < 0) return ret; } else { - intra_prediction_unit(lc, sps, x0, y0, log2_cb_size); + intra_prediction_unit(lc, l, sps, x0, y0, log2_cb_size); } } else { - intra_prediction_unit_default_value(lc, sps, x0, y0, log2_cb_size); + intra_prediction_unit_default_value(lc, l, sps, x0, y0, log2_cb_size); switch (lc->cu.part_mode) { case PART_2Nx2N: hls_prediction_unit(lc, l, pps, sps, diff --git a/libavcodec/hevc/hevcdec.h b/libavcodec/hevc/hevcdec.h index ccea7a46e1..5f686772c8 100644 --- a/libavcodec/hevc/hevcdec.h +++ b/libavcodec/hevc/hevcdec.h @@ -453,6 +453,7 @@ typedef struct HEVCLayerContext { // PU uint8_t *cbf_luma; // cbf_luma of colocated TU + uint8_t *tab_ipm; } HEVCLayerContext; typedef struct HEVCContext { @@ -511,7 +512,6 @@ typedef struct HEVCContext { int32_t *tab_slice_address; // PU - uint8_t *tab_ipm; uint8_t *is_pcm; -- 2.43.0
--- libavcodec/hevc/filter.c | 40 +++++++++++++++++++-------------------- libavcodec/hevc/hevcdec.c | 18 +++++++++--------- libavcodec/hevc/hevcdec.h | 5 +---- 3 files changed, 30 insertions(+), 33 deletions(-) diff --git a/libavcodec/hevc/filter.c b/libavcodec/hevc/filter.c index 379c488d78..f5224151b6 100644 --- a/libavcodec/hevc/filter.c +++ b/libavcodec/hevc/filter.c @@ -223,7 +223,7 @@ static void copy_CTB_to_hv(const HEVCContext *s, const HEVCSPS *sps, copy_vert(s->sao_pixel_buffer_v[c_idx] + (((2 * x_ctb + 1) * h + y) << sh), src + ((width - 1) << sh), sh, height, 1 << sh, stride_src); } -static void restore_tqb_pixels(const HEVCContext *s, +static void restore_tqb_pixels(const HEVCLayerContext *l, const HEVCPPS *pps, const HEVCSPS *sps, uint8_t *src1, const uint8_t *dst1, ptrdiff_t stride_src, ptrdiff_t stride_dst, @@ -242,7 +242,7 @@ static void restore_tqb_pixels(const HEVCContext *s, int len = (min_pu_size >> hshift) << sps->pixel_shift; for (y = y_min; y < y_max; y++) { for (x = x_min; x < x_max; x++) { - if (s->is_pcm[y * sps->min_pu_width + x]) { + if (l->is_pcm[y * sps->min_pu_width + x]) { int n; uint8_t *src = src1 + (((y << sps->log2_min_pu_size) - y0) >> vshift) * stride_src + @@ -351,7 +351,7 @@ static void sao_filter_CTB(HEVCLocalContext *lc, const HEVCLayerContext *l, s->hevcdsp.sao_band_filter[tab](src, dst, stride_src, stride_dst, sao->offset_val[c_idx], sao->band_position[c_idx], width, height); - restore_tqb_pixels(s, pps, sps, src, dst, stride_src, stride_dst, + restore_tqb_pixels(l, pps, sps, src, dst, stride_src, stride_dst, x, y, width, height, c_idx); } else { s->hevcdsp.sao_band_filter[tab](src, src, stride_src, stride_src, @@ -466,7 +466,7 @@ static void sao_filter_CTB(HEVCLocalContext *lc, const HEVCLayerContext *l, vert_edge, horiz_edge, diag_edge); - restore_tqb_pixels(s, pps, sps, src, dst, stride_src, stride_dst, + restore_tqb_pixels(l, pps, sps, src, dst, stride_src, stride_dst, x, y, width, height, c_idx); sao->type_idx[c_idx] = SAO_APPLIED; break; @@ -559,10 +559,10 @@ static void deblocking_filter_CTB(const HEVCContext *s, const HEVCLayerContext * tc[1] = bs1 ? TC_CALC(qp, bs1) : 0; src = &data[LUMA][y * linesize[LUMA] + (x << sps->pixel_shift)]; if (pcmf) { - no_p[0] = get_pcm(sps, s->is_pcm, x - 1, y); - no_p[1] = get_pcm(sps, s->is_pcm, x - 1, y + 4); - no_q[0] = get_pcm(sps, s->is_pcm, x, y); - no_q[1] = get_pcm(sps, s->is_pcm, x, y + 4); + no_p[0] = get_pcm(sps, l->is_pcm, x - 1, y); + no_p[1] = get_pcm(sps, l->is_pcm, x - 1, y + 4); + no_q[0] = get_pcm(sps, l->is_pcm, x, y); + no_q[1] = get_pcm(sps, l->is_pcm, x, y + 4); s->hevcdsp.hevc_v_loop_filter_luma_c(src, linesize[LUMA], beta, tc, no_p, no_q); } else @@ -590,10 +590,10 @@ static void deblocking_filter_CTB(const HEVCContext *s, const HEVCLayerContext * tc[1] = bs1 ? TC_CALC(qp, bs1) : 0; src = &data[LUMA][y * linesize[LUMA] + (x << sps->pixel_shift)]; if (pcmf) { - no_p[0] = get_pcm(sps, s->is_pcm, x, y - 1); - no_p[1] = get_pcm(sps, s->is_pcm, x + 4, y - 1); - no_q[0] = get_pcm(sps, s->is_pcm, x, y); - no_q[1] = get_pcm(sps, s->is_pcm, x + 4, y); + no_p[0] = get_pcm(sps, l->is_pcm, x, y - 1); + no_p[1] = get_pcm(sps, l->is_pcm, x + 4, y - 1); + no_q[0] = get_pcm(sps, l->is_pcm, x, y); + no_q[1] = get_pcm(sps, l->is_pcm, x + 4, y); s->hevcdsp.hevc_h_loop_filter_luma_c(src, linesize[LUMA], beta, tc, no_p, no_q); } else @@ -624,10 +624,10 @@ static void deblocking_filter_CTB(const HEVCContext *s, const HEVCLayerContext * c_tc[1] = (bs1 == 2) ? chroma_tc(pps, sps, qp1, chroma, tc_offset) : 0; src = &data[chroma][(y >> sps->vshift[chroma]) * linesize[chroma] + ((x >> sps->hshift[chroma]) << sps->pixel_shift)]; if (pcmf) { - no_p[0] = get_pcm(sps, s->is_pcm, x - 1, y); - no_p[1] = get_pcm(sps, s->is_pcm, x - 1, y + (4 * v)); - no_q[0] = get_pcm(sps, s->is_pcm, x, y); - no_q[1] = get_pcm(sps, s->is_pcm, x, y + (4 * v)); + no_p[0] = get_pcm(sps, l->is_pcm, x - 1, y); + no_p[1] = get_pcm(sps, l->is_pcm, x - 1, y + (4 * v)); + no_q[0] = get_pcm(sps, l->is_pcm, x, y); + no_q[1] = get_pcm(sps, l->is_pcm, x, y + (4 * v)); s->hevcdsp.hevc_v_loop_filter_chroma_c(src, linesize[chroma], c_tc, no_p, no_q); } else @@ -657,10 +657,10 @@ static void deblocking_filter_CTB(const HEVCContext *s, const HEVCLayerContext * c_tc[1] = bs1 == 2 ? chroma_tc(pps, sps, qp1, chroma, cur_tc_offset) : 0; src = &data[chroma][(y >> sps->vshift[1]) * linesize[chroma] + ((x >> sps->hshift[1]) << sps->pixel_shift)]; if (pcmf) { - no_p[0] = get_pcm(sps, s->is_pcm, x, y - 1); - no_p[1] = get_pcm(sps, s->is_pcm, x + (4 * h), y - 1); - no_q[0] = get_pcm(sps, s->is_pcm, x, y); - no_q[1] = get_pcm(sps, s->is_pcm, x + (4 * h), y); + no_p[0] = get_pcm(sps, l->is_pcm, x, y - 1); + no_p[1] = get_pcm(sps, l->is_pcm, x + (4 * h), y - 1); + no_q[0] = get_pcm(sps, l->is_pcm, x, y); + no_q[1] = get_pcm(sps, l->is_pcm, x + (4 * h), y); s->hevcdsp.hevc_h_loop_filter_chroma_c(src, linesize[chroma], c_tc, no_p, no_q); } else diff --git a/libavcodec/hevc/hevcdec.c b/libavcodec/hevc/hevcdec.c index 0a3b5ba4c4..02571f0607 100644 --- a/libavcodec/hevc/hevcdec.c +++ b/libavcodec/hevc/hevcdec.c @@ -76,7 +76,7 @@ static void pic_arrays_free(HEVCContext *s, HEVCLayerContext *l) av_freep(&l->tab_ipm); av_freep(&l->cbf_luma); - av_freep(&s->is_pcm); + av_freep(&l->is_pcm); av_freep(&s->qp_y_tab); av_freep(&s->tab_slice_address); @@ -115,8 +115,8 @@ static int pic_arrays_init(HEVCContext *s, HEVCLayerContext *l, const HEVCSPS *s l->cbf_luma = av_malloc_array(sps->min_tb_width, sps->min_tb_height); l->tab_ipm = av_mallocz(min_pu_size); - s->is_pcm = av_malloc_array(sps->min_pu_width + 1, sps->min_pu_height + 1); - if (!l->tab_ipm || !l->cbf_luma || !s->is_pcm) + l->is_pcm = av_malloc_array(sps->min_pu_width + 1, sps->min_pu_height + 1); + if (!l->tab_ipm || !l->cbf_luma || !l->is_pcm) goto fail; s->filter_slice_edges = av_mallocz(ctb_count); @@ -1300,7 +1300,7 @@ static int hls_transform_unit(HEVCLocalContext *lc, return 0; } -static void set_deblocking_bypass(const HEVCContext *s, const HEVCSPS *sps, +static void set_deblocking_bypass(uint8_t *is_pcm, const HEVCSPS *sps, int x0, int y0, int log2_cb_size) { int cb_size = 1 << log2_cb_size; @@ -1313,7 +1313,7 @@ static void set_deblocking_bypass(const HEVCContext *s, const HEVCSPS *sps, for (j = (y0 >> log2_min_pu_size); j < (y_end >> log2_min_pu_size); j++) for (i = (x0 >> log2_min_pu_size); i < (x_end >> log2_min_pu_size); i++) - s->is_pcm[i + j * min_pu_width] = 2; + is_pcm[i + j * min_pu_width] = 2; } static int hls_transform_tree(HEVCLocalContext *lc, @@ -1438,7 +1438,7 @@ do { ff_hevc_deblocking_boundary_strengths(lc, l, pps, x0, y0, log2_trafo_size); if (pps->transquant_bypass_enable_flag && lc->cu.cu_transquant_bypass_flag) - set_deblocking_bypass(s, sps, x0, y0, log2_trafo_size); + set_deblocking_bypass(l->is_pcm, sps, x0, y0, log2_trafo_size); } } return 0; @@ -2232,7 +2232,7 @@ static int hls_coding_unit(HEVCLocalContext *lc, const HEVCContext *s, if (pps->transquant_bypass_enable_flag) { lc->cu.cu_transquant_bypass_flag = ff_hevc_cu_transquant_bypass_flag_decode(lc); if (lc->cu.cu_transquant_bypass_flag) - set_deblocking_bypass(s, sps, x0, y0, log2_cb_size); + set_deblocking_bypass(l->is_pcm, sps, x0, y0, log2_cb_size); } else lc->cu.cu_transquant_bypass_flag = 0; @@ -2286,7 +2286,7 @@ static int hls_coding_unit(HEVCLocalContext *lc, const HEVCContext *s, intra_prediction_unit_default_value(lc, l, sps, x0, y0, log2_cb_size); ret = hls_pcm_sample(lc, l, pps, x0, y0, log2_cb_size); if (sps->pcm_loop_filter_disabled) - set_deblocking_bypass(s, sps, x0, y0, log2_cb_size); + set_deblocking_bypass(l->is_pcm, sps, x0, y0, log2_cb_size); if (ret < 0) return ret; @@ -2956,7 +2956,7 @@ static int hevc_frame_start(HEVCContext *s, HEVCLayerContext *l) memset(s->horizontal_bs, 0, l->bs_width * l->bs_height); memset(s->vertical_bs, 0, l->bs_width * l->bs_height); memset(l->cbf_luma, 0, sps->min_tb_width * sps->min_tb_height); - memset(s->is_pcm, 0, (sps->min_pu_width + 1) * (sps->min_pu_height + 1)); + memset(l->is_pcm, 0, (sps->min_pu_width + 1) * (sps->min_pu_height + 1)); memset(s->tab_slice_address, -1, pic_size_in_ctb * sizeof(*s->tab_slice_address)); if (IS_IDR(s)) diff --git a/libavcodec/hevc/hevcdec.h b/libavcodec/hevc/hevcdec.h index 5f686772c8..dde87be5be 100644 --- a/libavcodec/hevc/hevcdec.h +++ b/libavcodec/hevc/hevcdec.h @@ -454,6 +454,7 @@ typedef struct HEVCLayerContext { // PU uint8_t *cbf_luma; // cbf_luma of colocated TU uint8_t *tab_ipm; + uint8_t *is_pcm; } HEVCLayerContext; typedef struct HEVCContext { @@ -511,10 +512,6 @@ typedef struct HEVCContext { int32_t *tab_slice_address; - // PU - - uint8_t *is_pcm; - // CTB-level flags affecting loop filter operation uint8_t *filter_slice_edges; -- 2.43.0
--- libavcodec/hevc/filter.c | 2 +- libavcodec/hevc/hevcdec.c | 8 ++++---- libavcodec/hevc/hevcdec.h | 6 +++--- 3 files changed, 8 insertions(+), 8 deletions(-) diff --git a/libavcodec/hevc/filter.c b/libavcodec/hevc/filter.c index f5224151b6..12a57eb223 100644 --- a/libavcodec/hevc/filter.c +++ b/libavcodec/hevc/filter.c @@ -281,7 +281,7 @@ static void sao_filter_CTB(HEVCLocalContext *lc, const HEVCLayerContext *l, uint8_t vert_edge[] = { 0, 0 }; uint8_t horiz_edge[] = { 0, 0 }; uint8_t diag_edge[] = { 0, 0, 0, 0 }; - uint8_t lfase = CTB(s->filter_slice_edges, x_ctb, y_ctb); + uint8_t lfase = CTB(l->filter_slice_edges, x_ctb, y_ctb); uint8_t no_tile_filter = pps->tiles_enabled_flag && !pps->loop_filter_across_tiles_enabled_flag; uint8_t restore = no_tile_filter || !lfase; diff --git a/libavcodec/hevc/hevcdec.c b/libavcodec/hevc/hevcdec.c index 02571f0607..4174743eb6 100644 --- a/libavcodec/hevc/hevcdec.c +++ b/libavcodec/hevc/hevcdec.c @@ -80,7 +80,7 @@ static void pic_arrays_free(HEVCContext *s, HEVCLayerContext *l) av_freep(&s->qp_y_tab); av_freep(&s->tab_slice_address); - av_freep(&s->filter_slice_edges); + av_freep(&l->filter_slice_edges); av_freep(&s->horizontal_bs); av_freep(&s->vertical_bs); @@ -119,12 +119,12 @@ static int pic_arrays_init(HEVCContext *s, HEVCLayerContext *l, const HEVCSPS *s if (!l->tab_ipm || !l->cbf_luma || !l->is_pcm) goto fail; - s->filter_slice_edges = av_mallocz(ctb_count); + l->filter_slice_edges = av_mallocz(ctb_count); s->tab_slice_address = av_malloc_array(pic_size_in_ctb, sizeof(*s->tab_slice_address)); s->qp_y_tab = av_malloc_array(pic_size_in_ctb, sizeof(*s->qp_y_tab)); - if (!s->qp_y_tab || !s->filter_slice_edges || !s->tab_slice_address) + if (!s->qp_y_tab || !l->filter_slice_edges || !s->tab_slice_address) goto fail; s->horizontal_bs = av_calloc(l->bs_width, l->bs_height); @@ -2568,7 +2568,7 @@ static int hls_decode_entry(HEVCContext *s, GetBitContext *gb) l->deblock[ctb_addr_rs].beta_offset = s->sh.beta_offset; l->deblock[ctb_addr_rs].tc_offset = s->sh.tc_offset; - s->filter_slice_edges[ctb_addr_rs] = s->sh.slice_loop_filter_across_slices_enabled_flag; + l->filter_slice_edges[ctb_addr_rs] = s->sh.slice_loop_filter_across_slices_enabled_flag; more_data = hls_coding_quadtree(lc, l, pps, sps, x_ctb, y_ctb, sps->log2_ctb_size, 0); if (more_data < 0) { diff --git a/libavcodec/hevc/hevcdec.h b/libavcodec/hevc/hevcdec.h index dde87be5be..afaf4c1954 100644 --- a/libavcodec/hevc/hevcdec.h +++ b/libavcodec/hevc/hevcdec.h @@ -455,6 +455,9 @@ typedef struct HEVCLayerContext { uint8_t *cbf_luma; // cbf_luma of colocated TU uint8_t *tab_ipm; uint8_t *is_pcm; + + // CTB-level flags affecting loop filter operation + uint8_t *filter_slice_edges; } HEVCLayerContext; typedef struct HEVCContext { @@ -512,9 +515,6 @@ typedef struct HEVCContext { int32_t *tab_slice_address; - // CTB-level flags affecting loop filter operation - uint8_t *filter_slice_edges; - /** used on BE to byteswap the lines for checksumming */ uint8_t *checksum_buf; int checksum_buf_size; -- 2.43.0
--- libavcodec/hevc/filter.c | 16 ++++++++-------- libavcodec/hevc/hevcdec.c | 34 ++++++++++++++++++---------------- libavcodec/hevc/hevcdec.h | 4 ++-- 3 files changed, 28 insertions(+), 26 deletions(-) diff --git a/libavcodec/hevc/filter.c b/libavcodec/hevc/filter.c index 12a57eb223..14afb147b9 100644 --- a/libavcodec/hevc/filter.c +++ b/libavcodec/hevc/filter.c @@ -298,31 +298,31 @@ static void sao_filter_CTB(HEVCLocalContext *lc, const HEVCLayerContext *l, if (restore) { if (!edges[0]) { left_tile_edge = no_tile_filter && pps->tile_id[ctb_addr_ts] != pps->tile_id[pps->ctb_addr_rs_to_ts[ctb_addr_rs-1]]; - vert_edge[0] = (!lfase && CTB(s->tab_slice_address, x_ctb, y_ctb) != CTB(s->tab_slice_address, x_ctb - 1, y_ctb)) || left_tile_edge; + vert_edge[0] = (!lfase && CTB(l->tab_slice_address, x_ctb, y_ctb) != CTB(l->tab_slice_address, x_ctb - 1, y_ctb)) || left_tile_edge; } if (!edges[2]) { right_tile_edge = no_tile_filter && pps->tile_id[ctb_addr_ts] != pps->tile_id[pps->ctb_addr_rs_to_ts[ctb_addr_rs+1]]; - vert_edge[1] = (!lfase && CTB(s->tab_slice_address, x_ctb, y_ctb) != CTB(s->tab_slice_address, x_ctb + 1, y_ctb)) || right_tile_edge; + vert_edge[1] = (!lfase && CTB(l->tab_slice_address, x_ctb, y_ctb) != CTB(l->tab_slice_address, x_ctb + 1, y_ctb)) || right_tile_edge; } if (!edges[1]) { up_tile_edge = no_tile_filter && pps->tile_id[ctb_addr_ts] != pps->tile_id[pps->ctb_addr_rs_to_ts[ctb_addr_rs - sps->ctb_width]]; - horiz_edge[0] = (!lfase && CTB(s->tab_slice_address, x_ctb, y_ctb) != CTB(s->tab_slice_address, x_ctb, y_ctb - 1)) || up_tile_edge; + horiz_edge[0] = (!lfase && CTB(l->tab_slice_address, x_ctb, y_ctb) != CTB(l->tab_slice_address, x_ctb, y_ctb - 1)) || up_tile_edge; } if (!edges[3]) { bottom_tile_edge = no_tile_filter && pps->tile_id[ctb_addr_ts] != pps->tile_id[pps->ctb_addr_rs_to_ts[ctb_addr_rs + sps->ctb_width]]; - horiz_edge[1] = (!lfase && CTB(s->tab_slice_address, x_ctb, y_ctb) != CTB(s->tab_slice_address, x_ctb, y_ctb + 1)) || bottom_tile_edge; + horiz_edge[1] = (!lfase && CTB(l->tab_slice_address, x_ctb, y_ctb) != CTB(l->tab_slice_address, x_ctb, y_ctb + 1)) || bottom_tile_edge; } if (!edges[0] && !edges[1]) { - diag_edge[0] = (!lfase && CTB(s->tab_slice_address, x_ctb, y_ctb) != CTB(s->tab_slice_address, x_ctb - 1, y_ctb - 1)) || left_tile_edge || up_tile_edge; + diag_edge[0] = (!lfase && CTB(l->tab_slice_address, x_ctb, y_ctb) != CTB(l->tab_slice_address, x_ctb - 1, y_ctb - 1)) || left_tile_edge || up_tile_edge; } if (!edges[1] && !edges[2]) { - diag_edge[1] = (!lfase && CTB(s->tab_slice_address, x_ctb, y_ctb) != CTB(s->tab_slice_address, x_ctb + 1, y_ctb - 1)) || right_tile_edge || up_tile_edge; + diag_edge[1] = (!lfase && CTB(l->tab_slice_address, x_ctb, y_ctb) != CTB(l->tab_slice_address, x_ctb + 1, y_ctb - 1)) || right_tile_edge || up_tile_edge; } if (!edges[2] && !edges[3]) { - diag_edge[2] = (!lfase && CTB(s->tab_slice_address, x_ctb, y_ctb) != CTB(s->tab_slice_address, x_ctb + 1, y_ctb + 1)) || right_tile_edge || bottom_tile_edge; + diag_edge[2] = (!lfase && CTB(l->tab_slice_address, x_ctb, y_ctb) != CTB(l->tab_slice_address, x_ctb + 1, y_ctb + 1)) || right_tile_edge || bottom_tile_edge; } if (!edges[0] && !edges[3]) { - diag_edge[3] = (!lfase && CTB(s->tab_slice_address, x_ctb, y_ctb) != CTB(s->tab_slice_address, x_ctb - 1, y_ctb + 1)) || left_tile_edge || bottom_tile_edge; + diag_edge[3] = (!lfase && CTB(l->tab_slice_address, x_ctb, y_ctb) != CTB(l->tab_slice_address, x_ctb - 1, y_ctb + 1)) || left_tile_edge || bottom_tile_edge; } } diff --git a/libavcodec/hevc/hevcdec.c b/libavcodec/hevc/hevcdec.c index 4174743eb6..3561a3a3ad 100644 --- a/libavcodec/hevc/hevcdec.c +++ b/libavcodec/hevc/hevcdec.c @@ -79,7 +79,7 @@ static void pic_arrays_free(HEVCContext *s, HEVCLayerContext *l) av_freep(&l->is_pcm); av_freep(&s->qp_y_tab); - av_freep(&s->tab_slice_address); + av_freep(&l->tab_slice_address); av_freep(&l->filter_slice_edges); av_freep(&s->horizontal_bs); @@ -120,11 +120,11 @@ static int pic_arrays_init(HEVCContext *s, HEVCLayerContext *l, const HEVCSPS *s goto fail; l->filter_slice_edges = av_mallocz(ctb_count); - s->tab_slice_address = av_malloc_array(pic_size_in_ctb, - sizeof(*s->tab_slice_address)); + l->tab_slice_address = av_malloc_array(pic_size_in_ctb, + sizeof(*l->tab_slice_address)); s->qp_y_tab = av_malloc_array(pic_size_in_ctb, sizeof(*s->qp_y_tab)); - if (!s->qp_y_tab || !l->filter_slice_edges || !s->tab_slice_address) + if (!s->qp_y_tab || !l->filter_slice_edges || !l->tab_slice_address) goto fail; s->horizontal_bs = av_calloc(l->bs_width, l->bs_height); @@ -2486,6 +2486,7 @@ static int hls_coding_quadtree(HEVCLocalContext *lc, } static void hls_decode_neighbour(HEVCLocalContext *lc, + const HEVCLayerContext *l, const HEVCPPS *pps, const HEVCSPS *sps, int x_ctb, int y_ctb, int ctb_addr_ts) { @@ -2494,7 +2495,7 @@ static void hls_decode_neighbour(HEVCLocalContext *lc, int ctb_addr_rs = pps->ctb_addr_ts_to_rs[ctb_addr_ts]; int ctb_addr_in_slice = ctb_addr_rs - s->sh.slice_addr; - s->tab_slice_address[ctb_addr_rs] = s->sh.slice_addr; + l->tab_slice_address[ctb_addr_rs] = s->sh.slice_addr; if (pps->entropy_coding_sync_enabled_flag) { if (x_ctb == 0 && (y_ctb & (ctb_size - 1)) == 0) @@ -2516,11 +2517,11 @@ static void hls_decode_neighbour(HEVCLocalContext *lc, if (pps->tiles_enabled_flag) { if (x_ctb > 0 && pps->tile_id[ctb_addr_ts] != pps->tile_id[pps->ctb_addr_rs_to_ts[ctb_addr_rs - 1]]) lc->boundary_flags |= BOUNDARY_LEFT_TILE; - if (x_ctb > 0 && s->tab_slice_address[ctb_addr_rs] != s->tab_slice_address[ctb_addr_rs - 1]) + if (x_ctb > 0 && l->tab_slice_address[ctb_addr_rs] != l->tab_slice_address[ctb_addr_rs - 1]) lc->boundary_flags |= BOUNDARY_LEFT_SLICE; if (y_ctb > 0 && pps->tile_id[ctb_addr_ts] != pps->tile_id[pps->ctb_addr_rs_to_ts[ctb_addr_rs - sps->ctb_width]]) lc->boundary_flags |= BOUNDARY_UPPER_TILE; - if (y_ctb > 0 && s->tab_slice_address[ctb_addr_rs] != s->tab_slice_address[ctb_addr_rs - sps->ctb_width]) + if (y_ctb > 0 && l->tab_slice_address[ctb_addr_rs] != l->tab_slice_address[ctb_addr_rs - sps->ctb_width]) lc->boundary_flags |= BOUNDARY_UPPER_SLICE; } else { if (ctb_addr_in_slice <= 0) @@ -2555,11 +2556,11 @@ static int hls_decode_entry(HEVCContext *s, GetBitContext *gb) x_ctb = (ctb_addr_rs % ((sps->width + ctb_size - 1) >> sps->log2_ctb_size)) << sps->log2_ctb_size; y_ctb = (ctb_addr_rs / ((sps->width + ctb_size - 1) >> sps->log2_ctb_size)) << sps->log2_ctb_size; - hls_decode_neighbour(lc, pps, sps, x_ctb, y_ctb, ctb_addr_ts); + hls_decode_neighbour(lc, l, pps, sps, x_ctb, y_ctb, ctb_addr_ts); ret = ff_hevc_cabac_init(lc, pps, ctb_addr_ts, slice_data, slice_size, 0); if (ret < 0) { - s->tab_slice_address[ctb_addr_rs] = -1; + l->tab_slice_address[ctb_addr_rs] = -1; return ret; } @@ -2572,7 +2573,7 @@ static int hls_decode_entry(HEVCContext *s, GetBitContext *gb) more_data = hls_coding_quadtree(lc, l, pps, sps, x_ctb, y_ctb, sps->log2_ctb_size, 0); if (more_data < 0) { - s->tab_slice_address[ctb_addr_rs] = -1; + l->tab_slice_address[ctb_addr_rs] = -1; return more_data; } @@ -2616,7 +2617,7 @@ static int hls_decode_entry_wpp(AVCodecContext *avctx, void *hevc_lclist, int x_ctb = (ctb_addr_rs % sps->ctb_width) << sps->log2_ctb_size; int y_ctb = (ctb_addr_rs / sps->ctb_width) << sps->log2_ctb_size; - hls_decode_neighbour(lc, pps, sps, x_ctb, y_ctb, ctb_addr_ts); + hls_decode_neighbour(lc, l, pps, sps, x_ctb, y_ctb, ctb_addr_ts); ff_thread_await_progress2(s->avctx, ctb_row, thread, SHIFT_CTB_WPP); @@ -2669,7 +2670,7 @@ static int hls_decode_entry_wpp(AVCodecContext *avctx, void *hevc_lclist, return 0; error: - s->tab_slice_address[ctb_addr_rs] = -1; + l->tab_slice_address[ctb_addr_rs] = -1; /* Casting const away here is safe, because it is an atomic operation. */ atomic_store((atomic_int*)&s->wpp_err, 1); ff_thread_report_progress2(s->avctx, ctb_row ,thread, SHIFT_CTB_WPP); @@ -2778,7 +2779,8 @@ static int hls_slice_data_wpp(HEVCContext *s, const H2645NAL *nal) return res; } -static int decode_slice_data(HEVCContext *s, const H2645NAL *nal, GetBitContext *gb) +static int decode_slice_data(HEVCContext *s, const HEVCLayerContext *l, + const H2645NAL *nal, GetBitContext *gb) { const HEVCPPS *pps = s->pps; int ret; @@ -2809,7 +2811,7 @@ static int decode_slice_data(HEVCContext *s, const H2645NAL *nal, GetBitContext if (s->sh.dependent_slice_segment_flag) { int ctb_addr_ts = pps->ctb_addr_rs_to_ts[s->sh.slice_ctb_addr_rs]; int prev_rs = pps->ctb_addr_ts_to_rs[ctb_addr_ts - 1]; - if (s->tab_slice_address[prev_rs] != s->sh.slice_addr) { + if (l->tab_slice_address[prev_rs] != s->sh.slice_addr) { av_log(s->avctx, AV_LOG_ERROR, "Previous slice segment missing\n"); return AVERROR_INVALIDDATA; } @@ -2957,7 +2959,7 @@ static int hevc_frame_start(HEVCContext *s, HEVCLayerContext *l) memset(s->vertical_bs, 0, l->bs_width * l->bs_height); memset(l->cbf_luma, 0, sps->min_tb_width * sps->min_tb_height); memset(l->is_pcm, 0, (sps->min_pu_width + 1) * (sps->min_pu_height + 1)); - memset(s->tab_slice_address, -1, pic_size_in_ctb * sizeof(*s->tab_slice_address)); + memset(l->tab_slice_address, -1, pic_size_in_ctb * sizeof(*l->tab_slice_address)); if (IS_IDR(s)) ff_hevc_clear_refs(l); @@ -3221,7 +3223,7 @@ static int decode_slice(HEVCContext *s, HEVCLayerContext *l, return AVERROR_INVALIDDATA; } - ret = decode_slice_data(s, nal, gb); + ret = decode_slice_data(s, l, nal, gb); if (ret < 0) return ret; diff --git a/libavcodec/hevc/hevcdec.h b/libavcodec/hevc/hevcdec.h index afaf4c1954..b69b039655 100644 --- a/libavcodec/hevc/hevcdec.h +++ b/libavcodec/hevc/hevcdec.h @@ -458,6 +458,8 @@ typedef struct HEVCLayerContext { // CTB-level flags affecting loop filter operation uint8_t *filter_slice_edges; + + int32_t *tab_slice_address; } HEVCLayerContext; typedef struct HEVCContext { @@ -513,8 +515,6 @@ typedef struct HEVCContext { uint8_t *horizontal_bs; uint8_t *vertical_bs; - int32_t *tab_slice_address; - /** used on BE to byteswap the lines for checksumming */ uint8_t *checksum_buf; int checksum_buf_size; -- 2.43.0
--- libavcodec/hevc/filter.c | 34 ++++++++++++++++++---------------- libavcodec/hevc/hevcdec.c | 17 +++++++++-------- libavcodec/hevc/hevcdec.h | 6 ++++-- 3 files changed, 31 insertions(+), 26 deletions(-) diff --git a/libavcodec/hevc/filter.c b/libavcodec/hevc/filter.c index 14afb147b9..ad655a2b36 100644 --- a/libavcodec/hevc/filter.c +++ b/libavcodec/hevc/filter.c @@ -75,6 +75,7 @@ static int chroma_tc(const HEVCPPS *pps, const HEVCSPS *sps, } static int get_qPy_pred(HEVCLocalContext *lc, const HEVCContext *s, + const HEVCLayerContext *l, const HEVCPPS *pps, const HEVCSPS *sps, int xBase, int yBase, int log2_cb_size) { @@ -104,13 +105,13 @@ static int get_qPy_pred(HEVCLocalContext *lc, const HEVCContext *s, if (availableA == 0) qPy_a = qPy_pred; else - qPy_a = s->qp_y_tab[(x_cb - 1) + y_cb * min_cb_width]; + qPy_a = l->qp_y_tab[(x_cb - 1) + y_cb * min_cb_width]; // qPy_b if (availableB == 0) qPy_b = qPy_pred; else - qPy_b = s->qp_y_tab[x_cb + (y_cb - 1) * min_cb_width]; + qPy_b = l->qp_y_tab[x_cb + (y_cb - 1) * min_cb_width]; av_assert2(qPy_a >= -sps->qp_bd_offset && qPy_a < 52); av_assert2(qPy_b >= -sps->qp_bd_offset && qPy_b < 52); @@ -118,12 +119,13 @@ static int get_qPy_pred(HEVCLocalContext *lc, const HEVCContext *s, return (qPy_a + qPy_b + 1) >> 1; } -void ff_hevc_set_qPy(HEVCLocalContext *lc, const HEVCPPS *pps, +void ff_hevc_set_qPy(HEVCLocalContext *lc, + const HEVCLayerContext *l, const HEVCPPS *pps, int xBase, int yBase, int log2_cb_size) { const HEVCSPS *const sps = pps->sps; const HEVCContext *const s = lc->parent; - int qp_y = get_qPy_pred(lc, s, pps, sps, xBase, yBase, log2_cb_size); + int qp_y = get_qPy_pred(lc, s, l, pps, sps, xBase, yBase, log2_cb_size); if (lc->tu.cu_qp_delta != 0) { int off = sps->qp_bd_offset; @@ -550,8 +552,8 @@ static void deblocking_filter_CTB(const HEVCContext *s, const HEVCLayerContext * const int bs0 = s->vertical_bs[(x + y * l->bs_width) >> 2]; const int bs1 = s->vertical_bs[(x + (y + 4) * l->bs_width) >> 2]; if (bs0 || bs1) { - const int qp = (get_qPy(sps, s->qp_y_tab, x - 1, y) + - get_qPy(sps, s->qp_y_tab, x, y) + 1) >> 1; + const int qp = (get_qPy(sps, l->qp_y_tab, x - 1, y) + + get_qPy(sps, l->qp_y_tab, x, y) + 1) >> 1; beta = betatable[av_clip(qp + beta_offset, 0, MAX_QP)]; @@ -579,8 +581,8 @@ static void deblocking_filter_CTB(const HEVCContext *s, const HEVCLayerContext * const int bs0 = s->horizontal_bs[( x + y * l->bs_width) >> 2]; const int bs1 = s->horizontal_bs[((x + 4) + y * l->bs_width) >> 2]; if (bs0 || bs1) { - const int qp = (get_qPy(sps, s->qp_y_tab, x, y - 1) + - get_qPy(sps, s->qp_y_tab, x, y) + 1) >> 1; + const int qp = (get_qPy(sps, l->qp_y_tab, x, y - 1) + + get_qPy(sps, l->qp_y_tab, x, y) + 1) >> 1; tc_offset = x >= x0 ? cur_tc_offset : left_tc_offset; beta_offset = x >= x0 ? cur_beta_offset : left_beta_offset; @@ -615,10 +617,10 @@ static void deblocking_filter_CTB(const HEVCContext *s, const HEVCLayerContext * const int bs1 = s->vertical_bs[(x + (y + (4 * v)) * l->bs_width) >> 2]; if ((bs0 == 2) || (bs1 == 2)) { - const int qp0 = (get_qPy(sps, s->qp_y_tab, x - 1, y) + - get_qPy(sps, s->qp_y_tab, x, y) + 1) >> 1; - const int qp1 = (get_qPy(sps, s->qp_y_tab, x - 1, y + (4 * v)) + - get_qPy(sps, s->qp_y_tab, x, y + (4 * v)) + 1) >> 1; + const int qp0 = (get_qPy(sps, l->qp_y_tab, x - 1, y) + + get_qPy(sps, l->qp_y_tab, x, y) + 1) >> 1; + const int qp1 = (get_qPy(sps, l->qp_y_tab, x - 1, y + (4 * v)) + + get_qPy(sps, l->qp_y_tab, x, y + (4 * v)) + 1) >> 1; c_tc[0] = (bs0 == 2) ? chroma_tc(pps, sps, qp0, chroma, tc_offset) : 0; c_tc[1] = (bs1 == 2) ? chroma_tc(pps, sps, qp1, chroma, tc_offset) : 0; @@ -648,10 +650,10 @@ static void deblocking_filter_CTB(const HEVCContext *s, const HEVCLayerContext * const int bs0 = s->horizontal_bs[( x + y * l->bs_width) >> 2]; const int bs1 = s->horizontal_bs[((x + 4 * h) + y * l->bs_width) >> 2]; if ((bs0 == 2) || (bs1 == 2)) { - const int qp0 = bs0 == 2 ? (get_qPy(sps, s->qp_y_tab, x, y - 1) + - get_qPy(sps, s->qp_y_tab, x, y) + 1) >> 1 : 0; - const int qp1 = bs1 == 2 ? (get_qPy(sps, s->qp_y_tab, x + (4 * h), y - 1) + - get_qPy(sps, s->qp_y_tab, x + (4 * h), y) + 1) >> 1 : 0; + const int qp0 = bs0 == 2 ? (get_qPy(sps, l->qp_y_tab, x, y - 1) + + get_qPy(sps, l->qp_y_tab, x, y) + 1) >> 1 : 0; + const int qp1 = bs1 == 2 ? (get_qPy(sps, l->qp_y_tab, x + (4 * h), y - 1) + + get_qPy(sps, l->qp_y_tab, x + (4 * h), y) + 1) >> 1 : 0; c_tc[0] = bs0 == 2 ? chroma_tc(pps, sps, qp0, chroma, tc_offset) : 0; c_tc[1] = bs1 == 2 ? chroma_tc(pps, sps, qp1, chroma, cur_tc_offset) : 0; diff --git a/libavcodec/hevc/hevcdec.c b/libavcodec/hevc/hevcdec.c index 3561a3a3ad..fd5da54fb1 100644 --- a/libavcodec/hevc/hevcdec.c +++ b/libavcodec/hevc/hevcdec.c @@ -78,7 +78,7 @@ static void pic_arrays_free(HEVCContext *s, HEVCLayerContext *l) av_freep(&l->cbf_luma); av_freep(&l->is_pcm); - av_freep(&s->qp_y_tab); + av_freep(&l->qp_y_tab); av_freep(&l->tab_slice_address); av_freep(&l->filter_slice_edges); @@ -122,9 +122,9 @@ static int pic_arrays_init(HEVCContext *s, HEVCLayerContext *l, const HEVCSPS *s l->filter_slice_edges = av_mallocz(ctb_count); l->tab_slice_address = av_malloc_array(pic_size_in_ctb, sizeof(*l->tab_slice_address)); - s->qp_y_tab = av_malloc_array(pic_size_in_ctb, - sizeof(*s->qp_y_tab)); - if (!s->qp_y_tab || !l->filter_slice_edges || !l->tab_slice_address) + l->qp_y_tab = av_malloc_array(pic_size_in_ctb, + sizeof(*l->qp_y_tab)); + if (!l->qp_y_tab || !l->filter_slice_edges || !l->tab_slice_address) goto fail; s->horizontal_bs = av_calloc(l->bs_width, l->bs_height); @@ -1090,6 +1090,7 @@ static int hls_cross_component_pred(HEVCLocalContext *lc, int idx) } static int hls_transform_unit(HEVCLocalContext *lc, + const HEVCLayerContext *l, const HEVCPPS *pps, const HEVCSPS *sps, int x0, int y0, int xBase, int yBase, int cb_xBase, int cb_yBase, @@ -1133,7 +1134,7 @@ static int hls_transform_unit(HEVCLocalContext *lc, return AVERROR_INVALIDDATA; } - ff_hevc_set_qPy(lc, pps, cb_xBase, cb_yBase, log2_cb_size); + ff_hevc_set_qPy(lc, l, pps, cb_xBase, cb_yBase, log2_cb_size); } if (s->sh.cu_chroma_qp_offset_enabled_flag && cbf_chroma && @@ -1418,7 +1419,7 @@ do { cbf_luma = ff_hevc_cbf_luma_decode(lc, trafo_depth); } - ret = hls_transform_unit(lc, pps, sps, + ret = hls_transform_unit(lc, l, pps, sps, x0, y0, xBase, yBase, cb_xBase, cb_yBase, log2_cb_size, log2_trafo_size, blk_idx, cbf_luma, cbf_cb, cbf_cr); @@ -2374,11 +2375,11 @@ static int hls_coding_unit(HEVCLocalContext *lc, const HEVCContext *s, } if (pps->cu_qp_delta_enabled_flag && lc->tu.is_cu_qp_delta_coded == 0) - ff_hevc_set_qPy(lc, pps, x0, y0, log2_cb_size); + ff_hevc_set_qPy(lc, l, pps, x0, y0, log2_cb_size); x = y_cb * min_cb_width + x_cb; for (y = 0; y < length; y++) { - memset(&s->qp_y_tab[x], lc->qp_y, length); + memset(&l->qp_y_tab[x], lc->qp_y, length); x += min_cb_width; } diff --git a/libavcodec/hevc/hevcdec.h b/libavcodec/hevc/hevcdec.h index b69b039655..6eb63c1b6c 100644 --- a/libavcodec/hevc/hevcdec.h +++ b/libavcodec/hevc/hevcdec.h @@ -460,6 +460,8 @@ typedef struct HEVCLayerContext { uint8_t *filter_slice_edges; int32_t *tab_slice_address; + + int8_t *qp_y_tab; } HEVCLayerContext; typedef struct HEVCContext { @@ -511,7 +513,6 @@ typedef struct HEVCContext { VideoDSPContext vdsp; BswapDSPContext bdsp; H274FilmGrainDatabase h274db; - int8_t *qp_y_tab; uint8_t *horizontal_bs; uint8_t *vertical_bs; @@ -658,7 +659,8 @@ void ff_hevc_hls_filter(HEVCLocalContext *lc, const HEVCLayerContext *l, void ff_hevc_hls_filters(HEVCLocalContext *lc, const HEVCLayerContext *l, const HEVCPPS *pps, int x_ctb, int y_ctb, int ctb_size); -void ff_hevc_set_qPy(HEVCLocalContext *lc, const HEVCPPS *pps, +void ff_hevc_set_qPy(HEVCLocalContext *lc, + const HEVCLayerContext *l, const HEVCPPS *pps, int xBase, int yBase, int log2_cb_size); void ff_hevc_deblocking_boundary_strengths(HEVCLocalContext *lc, const HEVCLayerContext *l, const HEVCPPS *pps, -- 2.43.0
--- libavcodec/hevc/filter.c | 24 ++++++++++++------------ libavcodec/hevc/hevcdec.c | 14 +++++++------- libavcodec/hevc/hevcdec.h | 5 +++-- 3 files changed, 22 insertions(+), 21 deletions(-) diff --git a/libavcodec/hevc/filter.c b/libavcodec/hevc/filter.c index ad655a2b36..db82f2319f 100644 --- a/libavcodec/hevc/filter.c +++ b/libavcodec/hevc/filter.c @@ -549,8 +549,8 @@ static void deblocking_filter_CTB(const HEVCContext *s, const HEVCLayerContext * for (y = y0; y < y_end; y += 8) { // vertical filtering luma for (x = x0 ? x0 : 8; x < x_end; x += 8) { - const int bs0 = s->vertical_bs[(x + y * l->bs_width) >> 2]; - const int bs1 = s->vertical_bs[(x + (y + 4) * l->bs_width) >> 2]; + const int bs0 = l->vertical_bs[(x + y * l->bs_width) >> 2]; + const int bs1 = l->vertical_bs[(x + (y + 4) * l->bs_width) >> 2]; if (bs0 || bs1) { const int qp = (get_qPy(sps, l->qp_y_tab, x - 1, y) + get_qPy(sps, l->qp_y_tab, x, y) + 1) >> 1; @@ -578,8 +578,8 @@ static void deblocking_filter_CTB(const HEVCContext *s, const HEVCLayerContext * // horizontal filtering luma for (x = x0 ? x0 - 8 : 0; x < x_end2; x += 8) { - const int bs0 = s->horizontal_bs[( x + y * l->bs_width) >> 2]; - const int bs1 = s->horizontal_bs[((x + 4) + y * l->bs_width) >> 2]; + const int bs0 = l->horizontal_bs[( x + y * l->bs_width) >> 2]; + const int bs1 = l->horizontal_bs[((x + 4) + y * l->bs_width) >> 2]; if (bs0 || bs1) { const int qp = (get_qPy(sps, l->qp_y_tab, x, y - 1) + get_qPy(sps, l->qp_y_tab, x, y) + 1) >> 1; @@ -613,8 +613,8 @@ static void deblocking_filter_CTB(const HEVCContext *s, const HEVCLayerContext * // vertical filtering chroma for (y = y0; y < y_end; y += (8 * v)) { for (x = x0 ? x0 : 8 * h; x < x_end; x += (8 * h)) { - const int bs0 = s->vertical_bs[(x + y * l->bs_width) >> 2]; - const int bs1 = s->vertical_bs[(x + (y + (4 * v)) * l->bs_width) >> 2]; + const int bs0 = l->vertical_bs[(x + y * l->bs_width) >> 2]; + const int bs1 = l->vertical_bs[(x + (y + (4 * v)) * l->bs_width) >> 2]; if ((bs0 == 2) || (bs1 == 2)) { const int qp0 = (get_qPy(sps, l->qp_y_tab, x - 1, y) + @@ -647,8 +647,8 @@ static void deblocking_filter_CTB(const HEVCContext *s, const HEVCLayerContext * if (x_end != sps->width) x_end2 = x_end - 8 * h; for (x = x0 ? x0 - 8 * h : 0; x < x_end2; x += (8 * h)) { - const int bs0 = s->horizontal_bs[( x + y * l->bs_width) >> 2]; - const int bs1 = s->horizontal_bs[((x + 4 * h) + y * l->bs_width) >> 2]; + const int bs0 = l->horizontal_bs[( x + y * l->bs_width) >> 2]; + const int bs1 = l->horizontal_bs[((x + 4 * h) + y * l->bs_width) >> 2]; if ((bs0 == 2) || (bs1 == 2)) { const int qp0 = bs0 == 2 ? (get_qPy(sps, l->qp_y_tab, x, y - 1) + get_qPy(sps, l->qp_y_tab, x, y) + 1) >> 1 : 0; @@ -788,7 +788,7 @@ void ff_hevc_deblocking_boundary_strengths(HEVCLocalContext *lc, const HEVCLayer bs = 1; else bs = boundary_strength(s, curr, top, rpl_top); - s->horizontal_bs[((x0 + i) + y0 * l->bs_width) >> 2] = bs; + l->horizontal_bs[((x0 + i) + y0 * l->bs_width) >> 2] = bs; } } @@ -826,7 +826,7 @@ void ff_hevc_deblocking_boundary_strengths(HEVCLocalContext *lc, const HEVCLayer bs = 1; else bs = boundary_strength(s, curr, left, rpl_left); - s->vertical_bs[(x0 + (y0 + i) * l->bs_width) >> 2] = bs; + l->vertical_bs[(x0 + (y0 + i) * l->bs_width) >> 2] = bs; } } @@ -844,7 +844,7 @@ void ff_hevc_deblocking_boundary_strengths(HEVCLocalContext *lc, const HEVCLayer const MvField *curr = &tab_mvf[yq_pu * min_pu_width + x_pu]; bs = boundary_strength(s, curr, top, rpl); - s->horizontal_bs[((x0 + i) + (y0 + j) * l->bs_width) >> 2] = bs; + l->horizontal_bs[((x0 + i) + (y0 + j) * l->bs_width) >> 2] = bs; } } @@ -859,7 +859,7 @@ void ff_hevc_deblocking_boundary_strengths(HEVCLocalContext *lc, const HEVCLayer const MvField *curr = &tab_mvf[y_pu * min_pu_width + xq_pu]; bs = boundary_strength(s, curr, left, rpl); - s->vertical_bs[((x0 + i) + (y0 + j) * l->bs_width) >> 2] = bs; + l->vertical_bs[((x0 + i) + (y0 + j) * l->bs_width) >> 2] = bs; } } } diff --git a/libavcodec/hevc/hevcdec.c b/libavcodec/hevc/hevcdec.c index fd5da54fb1..2e31928e99 100644 --- a/libavcodec/hevc/hevcdec.c +++ b/libavcodec/hevc/hevcdec.c @@ -82,8 +82,8 @@ static void pic_arrays_free(HEVCContext *s, HEVCLayerContext *l) av_freep(&l->tab_slice_address); av_freep(&l->filter_slice_edges); - av_freep(&s->horizontal_bs); - av_freep(&s->vertical_bs); + av_freep(&l->horizontal_bs); + av_freep(&l->vertical_bs); ff_refstruct_pool_uninit(&s->tab_mvf_pool); ff_refstruct_pool_uninit(&s->rpl_tab_pool); @@ -127,9 +127,9 @@ static int pic_arrays_init(HEVCContext *s, HEVCLayerContext *l, const HEVCSPS *s if (!l->qp_y_tab || !l->filter_slice_edges || !l->tab_slice_address) goto fail; - s->horizontal_bs = av_calloc(l->bs_width, l->bs_height); - s->vertical_bs = av_calloc(l->bs_width, l->bs_height); - if (!s->horizontal_bs || !s->vertical_bs) + l->horizontal_bs = av_calloc(l->bs_width, l->bs_height); + l->vertical_bs = av_calloc(l->bs_width, l->bs_height); + if (!l->horizontal_bs || !l->vertical_bs) goto fail; s->tab_mvf_pool = ff_refstruct_pool_alloc(min_pu_size * sizeof(MvField), 0); @@ -2956,8 +2956,8 @@ static int hevc_frame_start(HEVCContext *s, HEVCLayerContext *l) new_sequence = 1; } - memset(s->horizontal_bs, 0, l->bs_width * l->bs_height); - memset(s->vertical_bs, 0, l->bs_width * l->bs_height); + memset(l->horizontal_bs, 0, l->bs_width * l->bs_height); + memset(l->vertical_bs, 0, l->bs_width * l->bs_height); memset(l->cbf_luma, 0, sps->min_tb_width * sps->min_tb_height); memset(l->is_pcm, 0, (sps->min_pu_width + 1) * (sps->min_pu_height + 1)); memset(l->tab_slice_address, -1, pic_size_in_ctb * sizeof(*l->tab_slice_address)); diff --git a/libavcodec/hevc/hevcdec.h b/libavcodec/hevc/hevcdec.h index 6eb63c1b6c..426b7968b0 100644 --- a/libavcodec/hevc/hevcdec.h +++ b/libavcodec/hevc/hevcdec.h @@ -462,6 +462,9 @@ typedef struct HEVCLayerContext { int32_t *tab_slice_address; int8_t *qp_y_tab; + + uint8_t *horizontal_bs; + uint8_t *vertical_bs; } HEVCLayerContext; typedef struct HEVCContext { @@ -513,8 +516,6 @@ typedef struct HEVCContext { VideoDSPContext vdsp; BswapDSPContext bdsp; H274FilmGrainDatabase h274db; - uint8_t *horizontal_bs; - uint8_t *vertical_bs; /** used on BE to byteswap the lines for checksumming */ uint8_t *checksum_buf; -- 2.43.0
pic_arrays_{init,free}() no longer access HEVCContext --- libavcodec/hevc/hevcdec.c | 24 ++++++++++++------------ libavcodec/hevc/hevcdec.h | 6 +++--- libavcodec/hevc/refs.c | 4 ++-- 3 files changed, 17 insertions(+), 17 deletions(-) diff --git a/libavcodec/hevc/hevcdec.c b/libavcodec/hevc/hevcdec.c index 2e31928e99..c4e3debae1 100644 --- a/libavcodec/hevc/hevcdec.c +++ b/libavcodec/hevc/hevcdec.c @@ -66,7 +66,7 @@ static const uint8_t hevc_pel_weight[65] = { [2] = 0, [4] = 1, [6] = 2, [8] = 3, */ /* free everything allocated by pic_arrays_init() */ -static void pic_arrays_free(HEVCContext *s, HEVCLayerContext *l) +static void pic_arrays_free(HEVCLayerContext *l) { av_freep(&l->sao); av_freep(&l->deblock); @@ -85,12 +85,12 @@ static void pic_arrays_free(HEVCContext *s, HEVCLayerContext *l) av_freep(&l->horizontal_bs); av_freep(&l->vertical_bs); - ff_refstruct_pool_uninit(&s->tab_mvf_pool); - ff_refstruct_pool_uninit(&s->rpl_tab_pool); + ff_refstruct_pool_uninit(&l->tab_mvf_pool); + ff_refstruct_pool_uninit(&l->rpl_tab_pool); } /* allocate arrays that depend on frame dimensions */ -static int pic_arrays_init(HEVCContext *s, HEVCLayerContext *l, const HEVCSPS *sps) +static int pic_arrays_init(HEVCLayerContext *l, const HEVCSPS *sps) { int log2_min_cb_size = sps->log2_min_cb_size; int width = sps->width; @@ -132,15 +132,15 @@ static int pic_arrays_init(HEVCContext *s, HEVCLayerContext *l, const HEVCSPS *s if (!l->horizontal_bs || !l->vertical_bs) goto fail; - s->tab_mvf_pool = ff_refstruct_pool_alloc(min_pu_size * sizeof(MvField), 0); - s->rpl_tab_pool = ff_refstruct_pool_alloc(ctb_count * sizeof(RefPicListTab), 0); - if (!s->tab_mvf_pool || !s->rpl_tab_pool) + l->tab_mvf_pool = ff_refstruct_pool_alloc(min_pu_size * sizeof(MvField), 0); + l->rpl_tab_pool = ff_refstruct_pool_alloc(ctb_count * sizeof(RefPicListTab), 0); + if (!l->tab_mvf_pool || !l->rpl_tab_pool) goto fail; return 0; fail: - pic_arrays_free(s, l); + pic_arrays_free(l); return AVERROR(ENOMEM); } @@ -531,14 +531,14 @@ static int set_sps(HEVCContext *s, HEVCLayerContext *l, const HEVCSPS *sps) { int ret, i; - pic_arrays_free(s, l); + pic_arrays_free(l); s->ps.sps = NULL; ff_refstruct_unref(&s->vps); if (!sps) return 0; - ret = pic_arrays_init(s, l, sps); + ret = pic_arrays_init(l, sps); if (ret < 0) goto fail; @@ -576,7 +576,7 @@ static int set_sps(HEVCContext *s, HEVCLayerContext *l, const HEVCSPS *sps) return 0; fail: - pic_arrays_free(s, l); + pic_arrays_free(l); for (i = 0; i < 3; i++) { av_freep(&s->sao_pixel_buffer_h[i]); av_freep(&s->sao_pixel_buffer_v[i]); @@ -3529,7 +3529,7 @@ static av_cold int hevc_decode_free(AVCodecContext *avctx) int i; for (int i = 0; i < FF_ARRAY_ELEMS(s->layers); i++) - pic_arrays_free(s, &s->layers[i]); + pic_arrays_free(&s->layers[i]); ff_refstruct_unref(&s->vps); ff_refstruct_unref(&s->pps); diff --git a/libavcodec/hevc/hevcdec.h b/libavcodec/hevc/hevcdec.h index 426b7968b0..1a43c3662c 100644 --- a/libavcodec/hevc/hevcdec.h +++ b/libavcodec/hevc/hevcdec.h @@ -465,6 +465,9 @@ typedef struct HEVCLayerContext { uint8_t *horizontal_bs; uint8_t *vertical_bs; + + struct FFRefStructPool *tab_mvf_pool; + struct FFRefStructPool *rpl_tab_pool; } HEVCLayerContext; typedef struct HEVCContext { @@ -489,9 +492,6 @@ typedef struct HEVCContext { HEVCSEI sei; struct AVMD5 *md5_ctx; - struct FFRefStructPool *tab_mvf_pool; - struct FFRefStructPool *rpl_tab_pool; - ///< candidate references for the current frame RefPicList rps[5]; diff --git a/libavcodec/hevc/refs.c b/libavcodec/hevc/refs.c index 4d123d6d8d..7b8dff4f55 100644 --- a/libavcodec/hevc/refs.c +++ b/libavcodec/hevc/refs.c @@ -95,11 +95,11 @@ static HEVCFrame *alloc_frame(HEVCContext *s, HEVCLayerContext *l) goto fail; frame->nb_rpl_elems = s->pkt.nb_nals; - frame->tab_mvf = ff_refstruct_pool_get(s->tab_mvf_pool); + frame->tab_mvf = ff_refstruct_pool_get(l->tab_mvf_pool); if (!frame->tab_mvf) goto fail; - frame->rpl_tab = ff_refstruct_pool_get(s->rpl_tab_pool); + frame->rpl_tab = ff_refstruct_pool_get(l->rpl_tab_pool); if (!frame->rpl_tab) goto fail; frame->ctb_count = s->ps.sps->ctb_width * s->ps.sps->ctb_height; -- 2.43.0
Handle them together with other sps-dependent arrays. Note that current code only allocates these arrays when hwaccel is not set, but this is wrong as the relevant code runs BEFORE get_format() is called and hence before we know whether hwaccel is in use. --- libavcodec/hevc/filter.c | 22 +++++++-------- libavcodec/hevc/hevcdec.c | 59 ++++++++++++++++----------------------- libavcodec/hevc/hevcdec.h | 5 ++-- 3 files changed, 38 insertions(+), 48 deletions(-) diff --git a/libavcodec/hevc/filter.c b/libavcodec/hevc/filter.c index db82f2319f..298f1792b2 100644 --- a/libavcodec/hevc/filter.c +++ b/libavcodec/hevc/filter.c @@ -204,7 +204,7 @@ static void copy_vert(uint8_t *dst, const uint8_t *src, } } -static void copy_CTB_to_hv(const HEVCContext *s, const HEVCSPS *sps, +static void copy_CTB_to_hv(const HEVCLayerContext *l, const HEVCSPS *sps, const uint8_t *src, ptrdiff_t stride_src, int x, int y, int width, int height, int c_idx, int x_ctb, int y_ctb) @@ -214,15 +214,15 @@ static void copy_CTB_to_hv(const HEVCContext *s, const HEVCSPS *sps, int h = sps->height >> sps->vshift[c_idx]; /* copy horizontal edges */ - memcpy(s->sao_pixel_buffer_h[c_idx] + (((2 * y_ctb) * w + x) << sh), + memcpy(l->sao_pixel_buffer_h[c_idx] + (((2 * y_ctb) * w + x) << sh), src, width << sh); - memcpy(s->sao_pixel_buffer_h[c_idx] + (((2 * y_ctb + 1) * w + x) << sh), + memcpy(l->sao_pixel_buffer_h[c_idx] + (((2 * y_ctb + 1) * w + x) << sh), src + stride_src * (height - 1), width << sh); /* copy vertical edges */ - copy_vert(s->sao_pixel_buffer_v[c_idx] + (((2 * x_ctb) * h + y) << sh), src, sh, height, 1 << sh, stride_src); + copy_vert(l->sao_pixel_buffer_v[c_idx] + (((2 * x_ctb) * h + y) << sh), src, sh, height, 1 << sh, stride_src); - copy_vert(s->sao_pixel_buffer_v[c_idx] + (((2 * x_ctb + 1) * h + y) << sh), src + ((width - 1) << sh), sh, height, 1 << sh, stride_src); + copy_vert(l->sao_pixel_buffer_v[c_idx] + (((2 * x_ctb + 1) * h + y) << sh), src + ((width - 1) << sh), sh, height, 1 << sh, stride_src); } static void restore_tqb_pixels(const HEVCLayerContext *l, @@ -343,7 +343,7 @@ static void sao_filter_CTB(HEVCLocalContext *lc, const HEVCLayerContext *l, switch (sao->type_idx[c_idx]) { case SAO_BAND: - copy_CTB_to_hv(s, sps, src, stride_src, x0, y0, width, height, c_idx, + copy_CTB_to_hv(l, sps, src, stride_src, x0, y0, width, height, c_idx, x_ctb, y_ctb); if (pps->transquant_bypass_enable_flag || (sps->pcm_loop_filter_disabled && sps->pcm_enabled)) { @@ -385,7 +385,7 @@ static void sao_filter_CTB(HEVCLocalContext *lc, const HEVCLayerContext *l, dst1 = dst - stride_dst - (left << sh); src1[0] = src - stride_src - (left << sh); - src1[1] = s->sao_pixel_buffer_h[c_idx] + (((2 * y_ctb - 1) * w + x0 - left) << sh); + src1[1] = l->sao_pixel_buffer_h[c_idx] + (((2 * y_ctb - 1) * w + x0 - left) << sh); pos = 0; if (left) { src_idx = (CTB(l->sao, x_ctb-1, y_ctb-1).type_idx[c_idx] == @@ -412,7 +412,7 @@ static void sao_filter_CTB(HEVCLocalContext *lc, const HEVCLayerContext *l, dst1 = dst + height * stride_dst - (left << sh); src1[0] = src + height * stride_src - (left << sh); - src1[1] = s->sao_pixel_buffer_h[c_idx] + (((2 * y_ctb + 2) * w + x0 - left) << sh); + src1[1] = l->sao_pixel_buffer_h[c_idx] + (((2 * y_ctb + 2) * w + x0 - left) << sh); pos = 0; if (left) { src_idx = (CTB(l->sao, x_ctb-1, y_ctb+1).type_idx[c_idx] == @@ -434,7 +434,7 @@ static void sao_filter_CTB(HEVCLocalContext *lc, const HEVCLayerContext *l, if (!left_edge) { if (CTB(l->sao, x_ctb-1, y_ctb).type_idx[c_idx] == SAO_APPLIED) { copy_vert(dst - (1 << sh), - s->sao_pixel_buffer_v[c_idx] + (((2 * x_ctb - 1) * h + y0) << sh), + l->sao_pixel_buffer_v[c_idx] + (((2 * x_ctb - 1) * h + y0) << sh), sh, height, stride_dst, 1 << sh); } else { left_pixels = 1; @@ -444,7 +444,7 @@ static void sao_filter_CTB(HEVCLocalContext *lc, const HEVCLayerContext *l, if (!right_edge) { if (CTB(l->sao, x_ctb+1, y_ctb).type_idx[c_idx] == SAO_APPLIED) { copy_vert(dst + (width << sh), - s->sao_pixel_buffer_v[c_idx] + (((2 * x_ctb + 2) * h + y0) << sh), + l->sao_pixel_buffer_v[c_idx] + (((2 * x_ctb + 2) * h + y0) << sh), sh, height, stride_dst, 1 << sh); } else { right_pixels = 1; @@ -456,7 +456,7 @@ static void sao_filter_CTB(HEVCLocalContext *lc, const HEVCLayerContext *l, (width + left_pixels + right_pixels) << sh, height, stride_dst, stride_src); - copy_CTB_to_hv(s, sps, src, stride_src, x0, y0, width, height, c_idx, + copy_CTB_to_hv(l, sps, src, stride_src, x0, y0, width, height, c_idx, x_ctb, y_ctb); s->hevcdsp.sao_edge_filter[tab](src, dst, stride_src, sao->offset_val[c_idx], sao->eo_class[c_idx], width, height); diff --git a/libavcodec/hevc/hevcdec.c b/libavcodec/hevc/hevcdec.c index c4e3debae1..e02d5bba72 100644 --- a/libavcodec/hevc/hevcdec.c +++ b/libavcodec/hevc/hevcdec.c @@ -85,6 +85,11 @@ static void pic_arrays_free(HEVCLayerContext *l) av_freep(&l->horizontal_bs); av_freep(&l->vertical_bs); + for (int i = 0; i < 3; i++) { + av_freep(&l->sao_pixel_buffer_h[i]); + av_freep(&l->sao_pixel_buffer_v[i]); + } + ff_refstruct_pool_uninit(&l->tab_mvf_pool); ff_refstruct_pool_uninit(&l->rpl_tab_pool); } @@ -137,6 +142,24 @@ static int pic_arrays_init(HEVCLayerContext *l, const HEVCSPS *sps) if (!l->tab_mvf_pool || !l->rpl_tab_pool) goto fail; + if (sps->sao_enabled) { + int c_count = (sps->chroma_format_idc != 0) ? 3 : 1; + + for (int c_idx = 0; c_idx < c_count; c_idx++) { + int w = sps->width >> sps->hshift[c_idx]; + int h = sps->height >> sps->vshift[c_idx]; + l->sao_pixel_buffer_h[c_idx] = + av_malloc((w * 2 * sps->ctb_height) << + sps->pixel_shift); + l->sao_pixel_buffer_v[c_idx] = + av_malloc((h * 2 * sps->ctb_width) << + sps->pixel_shift); + if (!l->sao_pixel_buffer_h[c_idx] || + !l->sao_pixel_buffer_v[c_idx]) + goto fail; + } + } + return 0; fail: @@ -529,7 +552,7 @@ static enum AVPixelFormat get_format(HEVCContext *s, const HEVCSPS *sps) static int set_sps(HEVCContext *s, HEVCLayerContext *l, const HEVCSPS *sps) { - int ret, i; + int ret; pic_arrays_free(l); s->ps.sps = NULL; @@ -546,30 +569,6 @@ static int set_sps(HEVCContext *s, HEVCLayerContext *l, const HEVCSPS *sps) ff_hevc_dsp_init (&s->hevcdsp, sps->bit_depth); ff_videodsp_init (&s->vdsp, sps->bit_depth); - for (i = 0; i < 3; i++) { - av_freep(&s->sao_pixel_buffer_h[i]); - av_freep(&s->sao_pixel_buffer_v[i]); - } - - if (sps->sao_enabled && !s->avctx->hwaccel) { - int c_count = (sps->chroma_format_idc != 0) ? 3 : 1; - int c_idx; - - for(c_idx = 0; c_idx < c_count; c_idx++) { - int w = sps->width >> sps->hshift[c_idx]; - int h = sps->height >> sps->vshift[c_idx]; - s->sao_pixel_buffer_h[c_idx] = - av_malloc((w * 2 * sps->ctb_height) << - sps->pixel_shift); - s->sao_pixel_buffer_v[c_idx] = - av_malloc((h * 2 * sps->ctb_width) << - sps->pixel_shift); - if (!s->sao_pixel_buffer_h[c_idx] || - !s->sao_pixel_buffer_v[c_idx]) - goto fail; - } - } - s->ps.sps = sps; s->vps = ff_refstruct_ref_c(sps->vps); @@ -577,10 +576,6 @@ static int set_sps(HEVCContext *s, HEVCLayerContext *l, const HEVCSPS *sps) fail: pic_arrays_free(l); - for (i = 0; i < 3; i++) { - av_freep(&s->sao_pixel_buffer_h[i]); - av_freep(&s->sao_pixel_buffer_v[i]); - } s->ps.sps = NULL; return ret; } @@ -3526,7 +3521,6 @@ static int hevc_ref_frame(HEVCFrame *dst, const HEVCFrame *src) static av_cold int hevc_decode_free(AVCodecContext *avctx) { HEVCContext *s = avctx->priv_data; - int i; for (int i = 0; i < FF_ARRAY_ELEMS(s->layers); i++) pic_arrays_free(&s->layers[i]); @@ -3539,11 +3533,6 @@ static av_cold int hevc_decode_free(AVCodecContext *avctx) av_freep(&s->md5_ctx); - for (i = 0; i < 3; i++) { - av_freep(&s->sao_pixel_buffer_h[i]); - av_freep(&s->sao_pixel_buffer_v[i]); - } - ff_container_fifo_free(&s->output_fifo); for (int layer = 0; layer < FF_ARRAY_ELEMS(s->layers); layer++) { diff --git a/libavcodec/hevc/hevcdec.h b/libavcodec/hevc/hevcdec.h index 1a43c3662c..0b5d87b18f 100644 --- a/libavcodec/hevc/hevcdec.h +++ b/libavcodec/hevc/hevcdec.h @@ -466,6 +466,9 @@ typedef struct HEVCLayerContext { uint8_t *horizontal_bs; uint8_t *vertical_bs; + uint8_t *sao_pixel_buffer_h[3]; + uint8_t *sao_pixel_buffer_v[3]; + struct FFRefStructPool *tab_mvf_pool; struct FFRefStructPool *rpl_tab_pool; } HEVCLayerContext; @@ -485,8 +488,6 @@ typedef struct HEVCContext { uint8_t slice_initialized; struct ContainerFifo *output_fifo; - uint8_t *sao_pixel_buffer_h[3]; - uint8_t *sao_pixel_buffer_v[3]; HEVCParamSets ps; HEVCSEI sei; -- 2.43.0
ff_hevc_get_ref_list() needs the PPS of a previously decoded frame, which may be different from the currently active one. --- libavcodec/hevc/filter.c | 4 ++-- libavcodec/hevc/hevcdec.c | 1 + libavcodec/hevc/hevcdec.h | 4 ++-- libavcodec/hevc/mvs.c | 2 +- libavcodec/hevc/refs.c | 15 +++++++++------ 5 files changed, 15 insertions(+), 11 deletions(-) diff --git a/libavcodec/hevc/filter.c b/libavcodec/hevc/filter.c index 298f1792b2..68ae0e9ef6 100644 --- a/libavcodec/hevc/filter.c +++ b/libavcodec/hevc/filter.c @@ -767,7 +767,7 @@ void ff_hevc_deblocking_boundary_strengths(HEVCLocalContext *lc, const HEVCLayer if (boundary_upper) { const RefPicList *rpl_top = (lc->boundary_flags & BOUNDARY_UPPER_SLICE) ? - ff_hevc_get_ref_list(s, s->cur_frame, x0, y0 - 1) : + ff_hevc_get_ref_list(s->cur_frame, x0, y0 - 1) : s->cur_frame->refPicList; int yp_pu = (y0 - 1) >> log2_min_pu_size; int yq_pu = y0 >> log2_min_pu_size; @@ -805,7 +805,7 @@ void ff_hevc_deblocking_boundary_strengths(HEVCLocalContext *lc, const HEVCLayer if (boundary_left) { const RefPicList *rpl_left = (lc->boundary_flags & BOUNDARY_LEFT_SLICE) ? - ff_hevc_get_ref_list(s, s->cur_frame, x0 - 1, y0) : + ff_hevc_get_ref_list(s->cur_frame, x0 - 1, y0) : s->cur_frame->refPicList; int xp_pu = (x0 - 1) >> log2_min_pu_size; int xq_pu = x0 >> log2_min_pu_size; diff --git a/libavcodec/hevc/hevcdec.c b/libavcodec/hevc/hevcdec.c index e02d5bba72..eb41f9a5d6 100644 --- a/libavcodec/hevc/hevcdec.c +++ b/libavcodec/hevc/hevcdec.c @@ -3503,6 +3503,7 @@ static int hevc_ref_frame(HEVCFrame *dst, const HEVCFrame *src) dst->needs_fg = 1; } + dst->pps = ff_refstruct_ref_c(src->pps); dst->tab_mvf = ff_refstruct_ref(src->tab_mvf); dst->rpl_tab = ff_refstruct_ref(src->rpl_tab); dst->rpl = ff_refstruct_ref(src->rpl); diff --git a/libavcodec/hevc/hevcdec.h b/libavcodec/hevc/hevcdec.h index 0b5d87b18f..a534aee60f 100644 --- a/libavcodec/hevc/hevcdec.h +++ b/libavcodec/hevc/hevcdec.h @@ -366,6 +366,7 @@ typedef struct HEVCFrame { int ctb_count; int poc; + const HEVCPPS *pps; ///< RefStruct reference RefPicListTab *rpl; ///< RefStruct reference int nb_rpl_elems; @@ -556,8 +557,7 @@ void ff_hevc_clear_refs(HEVCLayerContext *l); */ void ff_hevc_flush_dpb(HEVCContext *s); -const RefPicList *ff_hevc_get_ref_list(const HEVCContext *s, const HEVCFrame *frame, - int x0, int y0); +const RefPicList *ff_hevc_get_ref_list(const HEVCFrame *frame, int x0, int y0); /** * Construct the reference picture sets for the current frame. diff --git a/libavcodec/hevc/mvs.c b/libavcodec/hevc/mvs.c index 96d8d58f39..55f115ad0c 100644 --- a/libavcodec/hevc/mvs.c +++ b/libavcodec/hevc/mvs.c @@ -211,7 +211,7 @@ static int derive_temporal_colocated_mvs(const HEVCContext *s, MvField temp_col, #define DERIVE_TEMPORAL_COLOCATED_MVS \ derive_temporal_colocated_mvs(s, temp_col, \ refIdxLx, mvLXCol, X, colPic, \ - ff_hevc_get_ref_list(s, ref, x, y)) + ff_hevc_get_ref_list(ref, x, y)) /* * 8.5.3.1.7 temporal luma motion vector prediction diff --git a/libavcodec/hevc/refs.c b/libavcodec/hevc/refs.c index 7b8dff4f55..58edc15a6f 100644 --- a/libavcodec/hevc/refs.c +++ b/libavcodec/hevc/refs.c @@ -38,6 +38,7 @@ void ff_hevc_unref_frame(HEVCFrame *frame, int flags) av_frame_unref(frame->frame_grain); frame->needs_fg = 0; + ff_refstruct_unref(&frame->pps); ff_refstruct_unref(&frame->tab_mvf); ff_refstruct_unref(&frame->rpl); @@ -49,13 +50,13 @@ void ff_hevc_unref_frame(HEVCFrame *frame, int flags) } } -const RefPicList *ff_hevc_get_ref_list(const HEVCContext *s, - const HEVCFrame *ref, int x0, int y0) +const RefPicList *ff_hevc_get_ref_list(const HEVCFrame *ref, int x0, int y0) { - int x_cb = x0 >> s->ps.sps->log2_ctb_size; - int y_cb = y0 >> s->ps.sps->log2_ctb_size; - int pic_width_cb = s->ps.sps->ctb_width; - int ctb_addr_ts = s->pps->ctb_addr_rs_to_ts[y_cb * pic_width_cb + x_cb]; + const HEVCSPS *sps = ref->pps->sps; + int x_cb = x0 >> sps->log2_ctb_size; + int y_cb = y0 >> sps->log2_ctb_size; + int pic_width_cb = sps->ctb_width; + int ctb_addr_ts = ref->pps->ctb_addr_rs_to_ts[y_cb * pic_width_cb + x_cb]; return &ref->rpl_tab[ctb_addr_ts]->refPicList[0]; } @@ -116,6 +117,8 @@ static HEVCFrame *alloc_frame(HEVCContext *s, HEVCLayerContext *l) if (ret < 0) goto fail; + frame->pps = ff_refstruct_ref_c(s->pps); + return frame; fail: ff_hevc_unref_frame(frame, ~0); -- 2.43.0
Currently active SPS is a per-layer property. --- libavcodec/hevc/hevcdec.c | 29 ++++++++++++++++------------- libavcodec/hevc/hevcdec.h | 2 ++ libavcodec/hevc/ps.c | 6 ------ libavcodec/hevc/ps.h | 3 --- libavcodec/hevc/refs.c | 24 ++++++++++++------------ libavcodec/vaapi_hevc.c | 4 ++-- libavcodec/vdpau_hevc.c | 2 +- 7 files changed, 33 insertions(+), 37 deletions(-) diff --git a/libavcodec/hevc/hevcdec.c b/libavcodec/hevc/hevcdec.c index eb41f9a5d6..6b596f1573 100644 --- a/libavcodec/hevc/hevcdec.c +++ b/libavcodec/hevc/hevcdec.c @@ -555,7 +555,7 @@ static int set_sps(HEVCContext *s, HEVCLayerContext *l, const HEVCSPS *sps) int ret; pic_arrays_free(l); - s->ps.sps = NULL; + ff_refstruct_unref(&l->sps); ff_refstruct_unref(&s->vps); if (!sps) @@ -569,14 +569,14 @@ static int set_sps(HEVCContext *s, HEVCLayerContext *l, const HEVCSPS *sps) ff_hevc_dsp_init (&s->hevcdsp, sps->bit_depth); ff_videodsp_init (&s->vdsp, sps->bit_depth); - s->ps.sps = sps; + l->sps = ff_refstruct_ref_c(sps); s->vps = ff_refstruct_ref_c(sps->vps); return 0; fail: pic_arrays_free(l); - s->ps.sps = NULL; + ff_refstruct_unref(&l->sps); return ret; } @@ -2831,6 +2831,7 @@ static int decode_slice_data(HEVCContext *s, const HEVCLayerContext *l, static int set_side_data(HEVCContext *s) { + const HEVCSPS *sps = s->cur_frame->pps->sps; AVFrame *out = s->cur_frame->f; int ret; @@ -2846,8 +2847,8 @@ static int set_side_data(HEVCContext *s) } ret = ff_h2645_sei_to_frame(out, &s->sei.common, AV_CODEC_ID_HEVC, s->avctx, - &s->ps.sps->vui.common, - s->ps.sps->bit_depth, s->ps.sps->bit_depth_chroma, + &sps->vui.common, + sps->bit_depth, sps->bit_depth_chroma, s->cur_frame->poc /* no poc_offset in HEVC */); if (ret < 0) return ret; @@ -2932,7 +2933,7 @@ static int hevc_frame_start(HEVCContext *s, HEVCLayerContext *l) } ff_refstruct_replace(&s->pps, pps); - if (s->ps.sps != sps) { + if (l->sps != sps) { enum AVPixelFormat pix_fmt; ff_hevc_clear_refs(l); @@ -3523,8 +3524,10 @@ static av_cold int hevc_decode_free(AVCodecContext *avctx) { HEVCContext *s = avctx->priv_data; - for (int i = 0; i < FF_ARRAY_ELEMS(s->layers); i++) + for (int i = 0; i < FF_ARRAY_ELEMS(s->layers); i++) { pic_arrays_free(&s->layers[i]); + ff_refstruct_unref(&s->layers[i].sps); + } ff_refstruct_unref(&s->vps); ff_refstruct_unref(&s->pps); @@ -3620,10 +3623,14 @@ static int hevc_update_thread_context(AVCodecContext *dst, return ret; } } + + if (l->sps != l0->sps) { + ret = set_sps(s, l, l0->sps); + if (ret < 0) + return ret; + } } - if (s->ps.sps != s0->ps.sps) - s->ps.sps = NULL; for (int i = 0; i < FF_ARRAY_ELEMS(s->ps.vps_list); i++) ff_refstruct_replace(&s->ps.vps_list[i], s0->ps.vps_list[i]); @@ -3636,10 +3643,6 @@ static int hevc_update_thread_context(AVCodecContext *dst, // PPS do not persist between frames ff_refstruct_unref(&s->pps); - if (s->ps.sps != s0->ps.sps) - if ((ret = set_sps(s, &s->layers[0], s0->ps.sps)) < 0) - return ret; - s->poc_tid0 = s0->poc_tid0; s->eos = s0->eos; s->no_rasl_output_flag = s0->no_rasl_output_flag; diff --git a/libavcodec/hevc/hevcdec.h b/libavcodec/hevc/hevcdec.h index a534aee60f..e43f2d0201 100644 --- a/libavcodec/hevc/hevcdec.h +++ b/libavcodec/hevc/hevcdec.h @@ -442,6 +442,8 @@ typedef struct HEVCLocalContext { typedef struct HEVCLayerContext { HEVCFrame DPB[32]; + const HEVCSPS *sps; // RefStruct reference + int bs_width; int bs_height; diff --git a/libavcodec/hevc/ps.c b/libavcodec/hevc/ps.c index 99382876af..5fa470cea3 100644 --- a/libavcodec/hevc/ps.c +++ b/libavcodec/hevc/ps.c @@ -66,15 +66,11 @@ static void remove_sps(HEVCParamSets *s, int id) { int i; if (s->sps_list[id]) { - if (s->sps == s->sps_list[id]) - s->sps = NULL; - /* drop all PPS that depend on this SPS */ for (i = 0; i < FF_ARRAY_ELEMS(s->pps_list); i++) if (s->pps_list[i] && s->pps_list[i]->sps_id == id) ff_refstruct_unref(&s->pps_list[i]); - av_assert0(!(s->sps_list[id] && s->sps == s->sps_list[id])); ff_refstruct_unref(&s->sps_list[id]); } } @@ -2042,8 +2038,6 @@ void ff_hevc_ps_uninit(HEVCParamSets *ps) ff_refstruct_unref(&ps->sps_list[i]); for (i = 0; i < FF_ARRAY_ELEMS(ps->pps_list); i++) ff_refstruct_unref(&ps->pps_list[i]); - - ps->sps = NULL; } int ff_hevc_compute_poc(const HEVCSPS *sps, int pocTid0, int poc_lsb, int nal_unit_type) diff --git a/libavcodec/hevc/ps.h b/libavcodec/hevc/ps.h index 554c86a656..331d163476 100644 --- a/libavcodec/hevc/ps.h +++ b/libavcodec/hevc/ps.h @@ -447,9 +447,6 @@ typedef struct HEVCParamSets { const HEVCVPS *vps_list[HEVC_MAX_VPS_COUNT]; ///< RefStruct references const HEVCSPS *sps_list[HEVC_MAX_SPS_COUNT]; ///< RefStruct references const HEVCPPS *pps_list[HEVC_MAX_PPS_COUNT]; ///< RefStruct references - - /* currently active parameter sets */ - const HEVCSPS *sps; } HEVCParamSets; /** diff --git a/libavcodec/hevc/refs.c b/libavcodec/hevc/refs.c index 58edc15a6f..09d759f936 100644 --- a/libavcodec/hevc/refs.c +++ b/libavcodec/hevc/refs.c @@ -103,7 +103,7 @@ static HEVCFrame *alloc_frame(HEVCContext *s, HEVCLayerContext *l) frame->rpl_tab = ff_refstruct_pool_get(l->rpl_tab_pool); if (!frame->rpl_tab) goto fail; - frame->ctb_count = s->ps.sps->ctb_width * s->ps.sps->ctb_height; + frame->ctb_count = l->sps->ctb_width * l->sps->ctb_height; for (j = 0; j < frame->ctb_count; j++) frame->rpl_tab[j] = frame->rpl; @@ -157,10 +157,10 @@ int ff_hevc_set_new_ref(HEVCContext *s, HEVCLayerContext *l, int poc) ref->flags = HEVC_FRAME_FLAG_SHORT_REF; ref->poc = poc; - ref->f->crop_left = s->ps.sps->output_window.left_offset; - ref->f->crop_right = s->ps.sps->output_window.right_offset; - ref->f->crop_top = s->ps.sps->output_window.top_offset; - ref->f->crop_bottom = s->ps.sps->output_window.bottom_offset; + ref->f->crop_left = l->sps->output_window.left_offset; + ref->f->crop_right = l->sps->output_window.right_offset; + ref->f->crop_top = l->sps->output_window.top_offset; + ref->f->crop_bottom = l->sps->output_window.bottom_offset; return 0; } @@ -321,7 +321,7 @@ int ff_hevc_slice_rpl(HEVCContext *s) static HEVCFrame *find_ref_idx(HEVCContext *s, HEVCLayerContext *l, int poc, uint8_t use_msb) { - int mask = use_msb ? ~0 : (1 << s->ps.sps->log2_max_poc_lsb) - 1; + int mask = use_msb ? ~0 : (1 << l->sps->log2_max_poc_lsb) - 1; int i; for (i = 0; i < FF_ARRAY_ELEMS(l->DPB); i++) { @@ -354,16 +354,16 @@ static HEVCFrame *generate_missing_ref(HEVCContext *s, HEVCLayerContext *l, int return NULL; if (!s->avctx->hwaccel) { - if (!s->ps.sps->pixel_shift) { + if (!l->sps->pixel_shift) { for (i = 0; frame->f->data[i]; i++) - memset(frame->f->data[i], 1 << (s->ps.sps->bit_depth - 1), - frame->f->linesize[i] * AV_CEIL_RSHIFT(s->ps.sps->height, s->ps.sps->vshift[i])); + memset(frame->f->data[i], 1 << (l->sps->bit_depth - 1), + frame->f->linesize[i] * AV_CEIL_RSHIFT(l->sps->height, l->sps->vshift[i])); } else { for (i = 0; frame->f->data[i]; i++) - for (y = 0; y < (s->ps.sps->height >> s->ps.sps->vshift[i]); y++) { + for (y = 0; y < (l->sps->height >> l->sps->vshift[i]); y++) { uint8_t *dst = frame->f->data[i] + y * frame->f->linesize[i]; - AV_WN16(dst, 1 << (s->ps.sps->bit_depth - 1)); - av_memcpy_backptr(dst + 2, 2, 2*(s->ps.sps->width >> s->ps.sps->hshift[i]) - 2); + AV_WN16(dst, 1 << (l->sps->bit_depth - 1)); + av_memcpy_backptr(dst + 2, 2, 2*(l->sps->width >> l->sps->hshift[i]) - 2); } } } diff --git a/libavcodec/vaapi_hevc.c b/libavcodec/vaapi_hevc.c index b97e7c0343..0c5a829220 100644 --- a/libavcodec/vaapi_hevc.c +++ b/libavcodec/vaapi_hevc.c @@ -407,7 +407,7 @@ static void fill_pred_weight_table(AVCodecContext *avctx, slice_param->luma_log2_weight_denom = sh->luma_log2_weight_denom; - if (h->ps.sps->chroma_format_idc) { + if (h->pps->sps->chroma_format_idc) { slice_param->delta_chroma_log2_weight_denom = sh->chroma_log2_weight_denom - sh->luma_log2_weight_denom; } @@ -596,7 +596,7 @@ static int ptl_convert(const PTLCommon *general_ptl, H265RawProfileTierLevel *h2 VAProfile ff_vaapi_parse_hevc_rext_scc_profile(AVCodecContext *avctx) { const HEVCContext *h = avctx->priv_data; - const HEVCSPS *sps = h->ps.sps; + const HEVCSPS *sps = h->pps->sps; const PTL *ptl = &sps->ptl; const PTLCommon *general_ptl = &ptl->general_ptl; const H265ProfileDescriptor *profile; diff --git a/libavcodec/vdpau_hevc.c b/libavcodec/vdpau_hevc.c index e6232567e0..affb7e7f5a 100644 --- a/libavcodec/vdpau_hevc.c +++ b/libavcodec/vdpau_hevc.c @@ -470,7 +470,7 @@ static int ptl_convert(const PTLCommon *general_ptl, H265RawProfileTierLevel *h2 static int vdpau_hevc_parse_rext_profile(AVCodecContext *avctx, VdpDecoderProfile *vdp_profile) { const HEVCContext *h = avctx->priv_data; - const HEVCSPS *sps = h->ps.sps; + const HEVCSPS *sps = h->pps->sps; const PTL *ptl = &sps->ptl; const PTLCommon *general_ptl = &ptl->general_ptl; const H265ProfileDescriptor *profile; -- 2.43.0
From: James Almer <jamrial@gmail.com> Signed-off-by: James Almer <jamrial@gmail.com> --- libavcodec/hevc/sei.c | 55 +++++++++++++++++++++++++++++++++++++++++++ libavcodec/hevc/sei.h | 17 +++++++++++++ 2 files changed, 72 insertions(+) diff --git a/libavcodec/hevc/sei.c b/libavcodec/hevc/sei.c index e39ac0c38a..6c2b55b43c 100644 --- a/libavcodec/hevc/sei.c +++ b/libavcodec/hevc/sei.c @@ -150,6 +150,59 @@ static int decode_nal_sei_timecode(HEVCSEITimeCode *s, GetBitContext *gb) return 0; } +static int decode_nal_sei_3d_reference_displays_info(HEVCSEITDRDI *s, GetBitContext *gb) +{ + s->prec_ref_display_width = get_ue_golomb(gb); + if (s->prec_ref_display_width > 31) + return AVERROR_INVALIDDATA; + s->ref_viewing_distance_flag = get_bits1(gb); + if (s->ref_viewing_distance_flag) { + s->prec_ref_viewing_dist = get_ue_golomb(gb); + if (s->prec_ref_viewing_dist > 31) + return AVERROR_INVALIDDATA; + } + s->num_ref_displays = get_ue_golomb(gb); + if (s->num_ref_displays > 31) + return AVERROR_INVALIDDATA; + s->num_ref_displays += 1; + + for (int i = 0; i < s->num_ref_displays; i++) { + int length; + s->left_view_id[i] = get_ue_golomb(gb); + s->right_view_id[i] = get_ue_golomb(gb); + s->exponent_ref_display_width[i] = get_bits(gb, 6); + if (s->exponent_ref_display_width[i] > 62) + return AVERROR_INVALIDDATA; + else if (!s->exponent_ref_display_width[i]) + length = FFMAX(0, (int)s->prec_ref_display_width - 30); + else + length = FFMAX(0, (int)s->exponent_ref_display_width[i] + + (int)s->prec_ref_display_width - 31); + s->mantissa_ref_display_width[i] = get_bits_long(gb, length); + if (s->ref_viewing_distance_flag) { + s->exponent_ref_viewing_distance[i] = get_bits(gb, 6); + if (s->exponent_ref_viewing_distance[i] > 62) + return AVERROR_INVALIDDATA; + else if (!s->exponent_ref_viewing_distance[i]) + length = FFMAX(0, (int)s->prec_ref_viewing_dist - 30); + else + length = FFMAX(0, (int)s->exponent_ref_viewing_distance[i] + + (int)s->prec_ref_viewing_dist - 31); + s->mantissa_ref_viewing_distance[i] = get_bits_long(gb, length); + } + s->additional_shift_present_flag[i] = get_bits1(gb); + if (s->additional_shift_present_flag[i]) { + s->num_sample_shift[i] = get_bits(gb, 10); + if (s->num_sample_shift[i] > 1023) + return AVERROR_INVALIDDATA; + s->num_sample_shift[i] -= 512; + } + } + s->three_dimensional_reference_displays_extension_flag = get_bits1(gb); + + return 0; +} + static int decode_nal_sei_prefix(GetBitContext *gb, GetByteContext *gbyte, void *logctx, HEVCSEI *s, const HEVCParamSets *ps, int type) @@ -163,6 +216,8 @@ static int decode_nal_sei_prefix(GetBitContext *gb, GetByteContext *gbyte, return decode_nal_sei_active_parameter_sets(s, gb, logctx); case SEI_TYPE_TIME_CODE: return decode_nal_sei_timecode(&s->timecode, gb); + case SEI_TYPE_THREE_DIMENSIONAL_REFERENCE_DISPLAYS_INFO: + return decode_nal_sei_3d_reference_displays_info(&s->tdrdi, gb); default: { int ret = ff_h2645_sei_message_decode(&s->common, type, AV_CODEC_ID_HEVC, gb, gbyte, logctx); diff --git a/libavcodec/hevc/sei.h b/libavcodec/hevc/sei.h index c97d22d423..a233d432cc 100644 --- a/libavcodec/hevc/sei.h +++ b/libavcodec/hevc/sei.h @@ -79,12 +79,29 @@ typedef struct HEVCSEITimeCode { int32_t time_offset_value[3]; } HEVCSEITimeCode; +typedef struct HEVCSEITDRDI { + uint8_t prec_ref_display_width; + uint8_t ref_viewing_distance_flag; + uint8_t prec_ref_viewing_dist; + uint8_t num_ref_displays; + uint8_t left_view_id[31]; + uint8_t right_view_id[31]; + uint8_t exponent_ref_display_width[31]; + uint8_t mantissa_ref_display_width[31]; + uint8_t exponent_ref_viewing_distance[31]; + uint8_t mantissa_ref_viewing_distance[31]; + uint8_t additional_shift_present_flag[31]; + int16_t num_sample_shift[31]; + uint8_t three_dimensional_reference_displays_extension_flag; +} HEVCSEITDRDI; + typedef struct HEVCSEI { H2645SEI common; HEVCSEIPictureHash picture_hash; HEVCSEIPictureTiming picture_timing; int active_seq_parameter_set_id; HEVCSEITimeCode timecode; + HEVCSEITDRDI tdrdi; } HEVCSEI; struct HEVCParamSets; -- 2.43.0
On 8/27/2024 12:05 PM, Anton Khirnov wrote:
From: James Almer <jamrial@gmail.com>
Signed-off-by: James Almer <jamrial@gmail.com> --- libavcodec/hevc/sei.c | 55 +++++++++++++++++++++++++++++++++++++++++++ libavcodec/hevc/sei.h | 17 +++++++++++++ 2 files changed, 72 insertions(+)
diff --git a/libavcodec/hevc/sei.c b/libavcodec/hevc/sei.c index e39ac0c38a..6c2b55b43c 100644 --- a/libavcodec/hevc/sei.c +++ b/libavcodec/hevc/sei.c @@ -150,6 +150,59 @@ static int decode_nal_sei_timecode(HEVCSEITimeCode *s, GetBitContext *gb) return 0; }
+static int decode_nal_sei_3d_reference_displays_info(HEVCSEITDRDI *s, GetBitContext *gb) +{ + s->prec_ref_display_width = get_ue_golomb(gb); + if (s->prec_ref_display_width > 31) + return AVERROR_INVALIDDATA; + s->ref_viewing_distance_flag = get_bits1(gb); + if (s->ref_viewing_distance_flag) { + s->prec_ref_viewing_dist = get_ue_golomb(gb); + if (s->prec_ref_viewing_dist > 31) + return AVERROR_INVALIDDATA; + } + s->num_ref_displays = get_ue_golomb(gb); + if (s->num_ref_displays > 31) + return AVERROR_INVALIDDATA; + s->num_ref_displays += 1; + + for (int i = 0; i < s->num_ref_displays; i++) { + int length; + s->left_view_id[i] = get_ue_golomb(gb); + s->right_view_id[i] = get_ue_golomb(gb); + s->exponent_ref_display_width[i] = get_bits(gb, 6); + if (s->exponent_ref_display_width[i] > 62) + return AVERROR_INVALIDDATA; + else if (!s->exponent_ref_display_width[i]) + length = FFMAX(0, (int)s->prec_ref_display_width - 30); + else + length = FFMAX(0, (int)s->exponent_ref_display_width[i] + + (int)s->prec_ref_display_width - 31); + s->mantissa_ref_display_width[i] = get_bits_long(gb, length); + if (s->ref_viewing_distance_flag) { + s->exponent_ref_viewing_distance[i] = get_bits(gb, 6); + if (s->exponent_ref_viewing_distance[i] > 62) + return AVERROR_INVALIDDATA; + else if (!s->exponent_ref_viewing_distance[i]) + length = FFMAX(0, (int)s->prec_ref_viewing_dist - 30); + else + length = FFMAX(0, (int)s->exponent_ref_viewing_distance[i] + + (int)s->prec_ref_viewing_dist - 31); + s->mantissa_ref_viewing_distance[i] = get_bits_long(gb, length); + } + s->additional_shift_present_flag[i] = get_bits1(gb); + if (s->additional_shift_present_flag[i]) { + s->num_sample_shift[i] = get_bits(gb, 10); + if (s->num_sample_shift[i] > 1023) + return AVERROR_INVALIDDATA; + s->num_sample_shift[i] -= 512; + } + } + s->three_dimensional_reference_displays_extension_flag = get_bits1(gb); + + return 0; +} + static int decode_nal_sei_prefix(GetBitContext *gb, GetByteContext *gbyte, void *logctx, HEVCSEI *s, const HEVCParamSets *ps, int type) @@ -163,6 +216,8 @@ static int decode_nal_sei_prefix(GetBitContext *gb, GetByteContext *gbyte, return decode_nal_sei_active_parameter_sets(s, gb, logctx); case SEI_TYPE_TIME_CODE: return decode_nal_sei_timecode(&s->timecode, gb); + case SEI_TYPE_THREE_DIMENSIONAL_REFERENCE_DISPLAYS_INFO: + return decode_nal_sei_3d_reference_displays_info(&s->tdrdi, gb); default: { int ret = ff_h2645_sei_message_decode(&s->common, type, AV_CODEC_ID_HEVC, gb, gbyte, logctx); diff --git a/libavcodec/hevc/sei.h b/libavcodec/hevc/sei.h index c97d22d423..a233d432cc 100644 --- a/libavcodec/hevc/sei.h +++ b/libavcodec/hevc/sei.h @@ -79,12 +79,29 @@ typedef struct HEVCSEITimeCode { int32_t time_offset_value[3]; } HEVCSEITimeCode;
+typedef struct HEVCSEITDRDI { + uint8_t prec_ref_display_width; + uint8_t ref_viewing_distance_flag; + uint8_t prec_ref_viewing_dist; + uint8_t num_ref_displays; + uint8_t left_view_id[31]; + uint8_t right_view_id[31];
Should be uint16_t, like you found out.
+ uint8_t exponent_ref_display_width[31]; + uint8_t mantissa_ref_display_width[31]; + uint8_t exponent_ref_viewing_distance[31]; + uint8_t mantissa_ref_viewing_distance[31]; + uint8_t additional_shift_present_flag[31]; + int16_t num_sample_shift[31]; + uint8_t three_dimensional_reference_displays_extension_flag; +} HEVCSEITDRDI; + typedef struct HEVCSEI { H2645SEI common; HEVCSEIPictureHash picture_hash; HEVCSEIPictureTiming picture_timing; int active_seq_parameter_set_id; HEVCSEITimeCode timecode; + HEVCSEITDRDI tdrdi; } HEVCSEI;
struct HEVCParamSets;
From: James Almer <jamrial@gmail.com> Signed-off-by: James Almer <jamrial@gmail.com> Signed-off-by: Anton Khirnov <anton@khirnov.net> --- libavcodec/hevc/sei.c | 55 +++++++++++++++++++++++++++++++++++++++++++ libavcodec/hevc/sei.h | 17 +++++++++++++ 2 files changed, 72 insertions(+) diff --git a/libavcodec/hevc/sei.c b/libavcodec/hevc/sei.c index e39ac0c38a..6c2b55b43c 100644 --- a/libavcodec/hevc/sei.c +++ b/libavcodec/hevc/sei.c @@ -150,6 +150,59 @@ static int decode_nal_sei_timecode(HEVCSEITimeCode *s, GetBitContext *gb) return 0; } +static int decode_nal_sei_3d_reference_displays_info(HEVCSEITDRDI *s, GetBitContext *gb) +{ + s->prec_ref_display_width = get_ue_golomb(gb); + if (s->prec_ref_display_width > 31) + return AVERROR_INVALIDDATA; + s->ref_viewing_distance_flag = get_bits1(gb); + if (s->ref_viewing_distance_flag) { + s->prec_ref_viewing_dist = get_ue_golomb(gb); + if (s->prec_ref_viewing_dist > 31) + return AVERROR_INVALIDDATA; + } + s->num_ref_displays = get_ue_golomb(gb); + if (s->num_ref_displays > 31) + return AVERROR_INVALIDDATA; + s->num_ref_displays += 1; + + for (int i = 0; i < s->num_ref_displays; i++) { + int length; + s->left_view_id[i] = get_ue_golomb(gb); + s->right_view_id[i] = get_ue_golomb(gb); + s->exponent_ref_display_width[i] = get_bits(gb, 6); + if (s->exponent_ref_display_width[i] > 62) + return AVERROR_INVALIDDATA; + else if (!s->exponent_ref_display_width[i]) + length = FFMAX(0, (int)s->prec_ref_display_width - 30); + else + length = FFMAX(0, (int)s->exponent_ref_display_width[i] + + (int)s->prec_ref_display_width - 31); + s->mantissa_ref_display_width[i] = get_bits_long(gb, length); + if (s->ref_viewing_distance_flag) { + s->exponent_ref_viewing_distance[i] = get_bits(gb, 6); + if (s->exponent_ref_viewing_distance[i] > 62) + return AVERROR_INVALIDDATA; + else if (!s->exponent_ref_viewing_distance[i]) + length = FFMAX(0, (int)s->prec_ref_viewing_dist - 30); + else + length = FFMAX(0, (int)s->exponent_ref_viewing_distance[i] + + (int)s->prec_ref_viewing_dist - 31); + s->mantissa_ref_viewing_distance[i] = get_bits_long(gb, length); + } + s->additional_shift_present_flag[i] = get_bits1(gb); + if (s->additional_shift_present_flag[i]) { + s->num_sample_shift[i] = get_bits(gb, 10); + if (s->num_sample_shift[i] > 1023) + return AVERROR_INVALIDDATA; + s->num_sample_shift[i] -= 512; + } + } + s->three_dimensional_reference_displays_extension_flag = get_bits1(gb); + + return 0; +} + static int decode_nal_sei_prefix(GetBitContext *gb, GetByteContext *gbyte, void *logctx, HEVCSEI *s, const HEVCParamSets *ps, int type) @@ -163,6 +216,8 @@ static int decode_nal_sei_prefix(GetBitContext *gb, GetByteContext *gbyte, return decode_nal_sei_active_parameter_sets(s, gb, logctx); case SEI_TYPE_TIME_CODE: return decode_nal_sei_timecode(&s->timecode, gb); + case SEI_TYPE_THREE_DIMENSIONAL_REFERENCE_DISPLAYS_INFO: + return decode_nal_sei_3d_reference_displays_info(&s->tdrdi, gb); default: { int ret = ff_h2645_sei_message_decode(&s->common, type, AV_CODEC_ID_HEVC, gb, gbyte, logctx); diff --git a/libavcodec/hevc/sei.h b/libavcodec/hevc/sei.h index c97d22d423..a9d6a52080 100644 --- a/libavcodec/hevc/sei.h +++ b/libavcodec/hevc/sei.h @@ -79,12 +79,29 @@ typedef struct HEVCSEITimeCode { int32_t time_offset_value[3]; } HEVCSEITimeCode; +typedef struct HEVCSEITDRDI { + uint8_t prec_ref_display_width; + uint8_t ref_viewing_distance_flag; + uint8_t prec_ref_viewing_dist; + uint8_t num_ref_displays; + uint16_t left_view_id[31]; + uint16_t right_view_id[31]; + uint8_t exponent_ref_display_width[31]; + uint8_t mantissa_ref_display_width[31]; + uint8_t exponent_ref_viewing_distance[31]; + uint8_t mantissa_ref_viewing_distance[31]; + uint8_t additional_shift_present_flag[31]; + int16_t num_sample_shift[31]; + uint8_t three_dimensional_reference_displays_extension_flag; +} HEVCSEITDRDI; + typedef struct HEVCSEI { H2645SEI common; HEVCSEIPictureHash picture_hash; HEVCSEIPictureTiming picture_timing; int active_seq_parameter_set_id; HEVCSEITimeCode timecode; + HEVCSEITDRDI tdrdi; } HEVCSEI; struct HEVCParamSets; -- 2.43.0
From: Niklas Haas <git@haasn.dev> Only implementing what's needed for MV-HEVC with two views. Signed-off-by: Anton Khirnov <anton@khirnov.net> --- libavcodec/hevc/hevc.h | 9 +- libavcodec/hevc/ps.c | 307 +++++++++++++++++++++++++++++++++++++++-- libavcodec/hevc/ps.h | 62 +++++++++ 3 files changed, 364 insertions(+), 14 deletions(-) diff --git a/libavcodec/hevc/hevc.h b/libavcodec/hevc/hevc.h index 9fdbc0a224..8bd59142db 100644 --- a/libavcodec/hevc/hevc.h +++ b/libavcodec/hevc/hevc.h @@ -100,11 +100,14 @@ enum HEVCSliceType { enum { // 7.4.3.1: vps_max_layers_minus1 is in [0, 62]. - HEVC_MAX_LAYERS = 63, + HEVC_MAX_LAYERS = 63, // 7.4.3.1: vps_max_sub_layers_minus1 is in [0, 6]. - HEVC_MAX_SUB_LAYERS = 7, + HEVC_MAX_SUB_LAYERS = 7, // 7.4.3.1: vps_num_layer_sets_minus1 is in [0, 1023]. - HEVC_MAX_LAYER_SETS = 1024, + HEVC_MAX_LAYER_SETS = 1024, + // 7.4.3.1: vps_max_layer_id is in [0, 63]. + HEVC_MAX_LAYER_ID = 63, + HEVC_MAX_NUH_LAYER_ID = 62, // 7.4.2.1: vps_video_parameter_set_id is u(4). HEVC_MAX_VPS_COUNT = 16, diff --git a/libavcodec/hevc/ps.c b/libavcodec/hevc/ps.c index 5fa470cea3..0f4ae92588 100644 --- a/libavcodec/hevc/ps.c +++ b/libavcodec/hevc/ps.c @@ -308,11 +308,17 @@ static int decode_profile_tier_level(GetBitContext *gb, AVCodecContext *avctx, } static int parse_ptl(GetBitContext *gb, AVCodecContext *avctx, - PTL *ptl, int max_num_sub_layers) + int profile_present, PTL *ptl, int max_num_sub_layers) { - int i; - if (decode_profile_tier_level(gb, avctx, &ptl->general_ptl) < 0 || - get_bits_left(gb) < 8 + (8*2 * (max_num_sub_layers - 1 > 0))) { + int i, status = 0; + + if (profile_present) { + status = decode_profile_tier_level(gb, avctx, &ptl->general_ptl); + } else { + memset(&ptl->general_ptl, 0, sizeof(ptl->general_ptl)); + } + + if (status < 0 || get_bits_left(gb) < 8 + (8*2 * (max_num_sub_layers - 1 > 0))) { av_log(avctx, AV_LOG_ERROR, "PTL information too short\n"); return -1; } @@ -441,13 +447,273 @@ static void hevc_vps_free(FFRefStructOpaque opaque, void *obj) av_freep(&vps->data); } +enum ScalabilityMask { + HEVC_SCALABILITY_DEPTH = 0, + HEVC_SCALABILITY_MULTIVIEW = 1, + HEVC_SCALABILITY_SPATIAL = 2, + HEVC_SCALABILITY_AUXILIARY = 3, + HEVC_SCALABILITY_MASK_MAX = 15, +}; + +enum DependencyType { + HEVC_DEP_TYPE_SAMPLE = 0, + HEVC_DEP_TYPE_MV = 1, + HEVC_DEP_TYPE_BOTH = 2, +}; + +static int decode_vps_ext(GetBitContext *gb, AVCodecContext *avctx, HEVCVPS *vps, + uint64_t layer1_id_included) +{ + PTL ptl_dummy; + uint8_t max_sub_layers[HEVC_MAX_LAYERS]; + + int splitting_flag, dimension_id_len, view_id_len, num_add_olss, + default_output_layer_idc, direct_dep_type_len, direct_dep_type, + sub_layers_max_present, sub_layer_flag_info_present_flag, nb_ptl; + unsigned non_vui_extension_length; + + if (vps->vps_max_layers == 1 || vps->vps_num_layer_sets == 1) { + av_log(avctx, AV_LOG_VERBOSE, "Ignoring VPS extensions with a single layer\n"); + return 0; + } + + if (vps->vps_max_layers > 2) { + av_log(avctx, AV_LOG_ERROR, + "VPS has %d layers, only 2 layers are supported\n", + vps->vps_max_layers); + return AVERROR_PATCHWELCOME; + } + if (vps->vps_num_layer_sets > 2) { + av_log(avctx, AV_LOG_ERROR, + "VPS has %d layer sets, only 2 layer sets are supported\n", + vps->vps_num_layer_sets); + return AVERROR_PATCHWELCOME; + } + + align_get_bits(gb); + + /** + * For stereoscopic MV-HEVC, the following simplifying assumptions are made: + * + * - vps_max_layers = 2 (one base layer, one multiview layer) + * - vps_num_layer_sets = 2 (one output layer set for each view) + * - NumScalabilityTypes = 1 (only HEVC_SCALABILITY_MULTIVIEW) + * - direct_dependency_flag[1][0] = 1 (second layer depends on first) + * - num_add_olss = 0 (no extra output layer sets) + * - default_output_layer_idc = 0 (1:1 mapping between OLSs and layers) + * - layer_id_included_flag[1] = {1, 1} (consequence of layer dependencies) + * - vps_num_rep_formats_minus1 = 0 (all layers have the same size) + * + * Which results in the following derived variables: + * - ViewOrderIdx = {0, 1} + * - NumViews = 2 + * - DependencyFlag[1][0] = 1 + * - NumDirectRefLayers = {0, 1} + * - NumRefLayers = {0, 1} + * - NumPredictedLayers = {1, 0} + * - NumIndependentLayers = 1 + * - NumLayersInTreePartition = {2} + * - NumLayerSets = 2 + * - NumOutputLayerSets = 2 + * - OlsIdxToLsIdx = {0, 1} + * - LayerIdxInVps = {0, 1} + * - NumLayersInIdList = {1, 2} + * - NumNecessaryLayers = {1, 2} + * - NecessaryLayerFlag = {{1, 0}, {1, 1}} + * - NumOutputLayersInOutputLayerSet = {1, 2} + * - OutputLayerFlag = {{1, 0}, {1, 1}} + */ + vps->nb_layers = 2; + + if (parse_ptl(gb, avctx, 0, &ptl_dummy, vps->vps_max_sub_layers) < 0) + return AVERROR_INVALIDDATA; + + splitting_flag = get_bits1(gb); + for (int i = 0; i <= HEVC_SCALABILITY_MASK_MAX; i++) { + int scalability_mask_flag = get_bits1(gb); + if (scalability_mask_flag != (i == HEVC_SCALABILITY_MULTIVIEW)) { + av_log(avctx, AV_LOG_ERROR, "Scalability type %d not supported\n", i); + return AVERROR_PATCHWELCOME; + } + } + + if (!splitting_flag) + dimension_id_len = get_bits(gb, 3) + 1; + + if (get_bits1(gb)) { /* vps_nuh_layer_id_present_flag */ + int layer_id_in_nuh = get_bits(gb, 6); + if (layer_id_in_nuh >= FF_ARRAY_ELEMS(vps->layer_idx)) { + av_log(avctx, AV_LOG_ERROR, "Invalid layer_id_in_nuh[1]: %d\n", + layer_id_in_nuh); + return AVERROR_INVALIDDATA; + } + vps->layer_idx[layer_id_in_nuh] = 1; + vps->layer_id_in_nuh[1] = layer_id_in_nuh; + } else { + vps->layer_idx[1] = 1; + vps->layer_id_in_nuh[1] = 1; + } + + if (!splitting_flag) { + int view_idx = get_bits(gb, dimension_id_len); + if (view_idx != 1) { + av_log(avctx, AV_LOG_ERROR, "Unexpected ViewOrderIdx: %d\n", view_idx); + return AVERROR_PATCHWELCOME; + } + } + + view_id_len = get_bits(gb, 4); + if (view_id_len) + for (int i = 0; i < 2 /* NumViews */; i++) + vps->view_id[i] = get_bits(gb, view_id_len); + + if (!get_bits1(gb) /* direct_dependency_flag */) { + av_log(avctx, AV_LOG_WARNING, "Independent output layers not supported\n"); + return AVERROR_PATCHWELCOME; + } + vps->num_direct_ref_layers[1] = 1; + + sub_layers_max_present = get_bits1(gb); // vps_sub_layers_max_minus1_present_flag + for (int i = 0; i < vps->vps_max_layers; i++) + max_sub_layers[i] = sub_layers_max_present ? get_bits(gb, 3) + 1 : + vps->vps_max_sub_layers; + + if (get_bits1(gb) /* max_tid_ref_present_flag */) + skip_bits(gb, 3); // max_tid_il_ref_pics_plus1 + + vps->default_ref_layers_active = get_bits1(gb); + + nb_ptl = get_ue_golomb(gb) + 1; + /* idx [0] is signalled in base VPS, idx [1] is signalled at the + * start of VPS extension, indices 2+ are signalled here; + * we ignore all but the first one anyway */ + for (int i = 2; i < nb_ptl; i++) { + int profile_present = get_bits1(gb); + if (parse_ptl(gb, avctx, profile_present, &ptl_dummy, vps->vps_max_sub_layers) < 0) + return AVERROR_INVALIDDATA; + } + + num_add_olss = get_ue_golomb(gb); + if (num_add_olss != 0) { + /* Since we don't implement support for independent output layer sets + * and auxiliary layers, this should never nonzero */ + av_log(avctx, AV_LOG_ERROR, "Unexpected num_add_olss: %d\n", num_add_olss); + return AVERROR_PATCHWELCOME; + } + + default_output_layer_idc = get_bits(gb, 2); + if (default_output_layer_idc != 0) { + av_log(avctx, AV_LOG_WARNING, "Unsupported default_output_layer_idc: %d\n", + default_output_layer_idc); + return AVERROR_PATCHWELCOME; + } + + /* Consequence of established layer dependencies */ + if (layer1_id_included != ((1 << vps->layer_id_in_nuh[0]) | + (1 << vps->layer_id_in_nuh[1]))) { + av_log(avctx, AV_LOG_ERROR, "Dependent layer not included in layer ID?\n"); + return AVERROR_PATCHWELCOME; + } + + vps->num_output_layer_sets = 2; + vps->ols[1] = 3; + + for (int j = 0; j < av_popcount64(vps->ols[1]); j++) { + int ptl_idx = get_bits(gb, av_ceil_log2(nb_ptl)); + if (ptl_idx < 1 || ptl_idx >= nb_ptl) { + av_log(avctx, AV_LOG_ERROR, "Invalid PTL index: %d\n", ptl_idx); + return AVERROR_INVALIDDATA; + } + } + + if (get_ue_golomb_31(gb) != 0 /* vps_num_rep_formats_minus1 */) { + av_log(avctx, AV_LOG_ERROR, "Unexpected extra rep formats\n"); + return AVERROR_INVALIDDATA; + } + + vps->rep_format.pic_width_in_luma_samples = get_bits(gb, 16); + vps->rep_format.pic_height_in_luma_samples = get_bits(gb, 16); + + if (!get_bits1(gb) /* chroma_and_bit_depth_vps_present_flag */) { + av_log(avctx, AV_LOG_ERROR, + "chroma_and_bit_depth_vps_present_flag=0 in first rep_format\n"); + return AVERROR_INVALIDDATA; + } + vps->rep_format.chroma_format_idc = get_bits(gb, 2); + if (vps->rep_format.chroma_format_idc == 3) + vps->rep_format.separate_colour_plane_flag = get_bits1(gb); + vps->rep_format.bit_depth_luma = get_bits(gb, 4) + 8; + vps->rep_format.bit_depth_chroma = get_bits(gb, 4) + 8; + if (vps->rep_format.bit_depth_luma > 16 || + vps->rep_format.bit_depth_chroma > 16 || + vps->rep_format.bit_depth_luma != vps->rep_format.bit_depth_chroma) { + av_log(avctx, AV_LOG_ERROR, "Unsupported bit depth: %"PRIu8" %"PRIu8"\n", + vps->rep_format.bit_depth_luma, vps->rep_format.bit_depth_chroma); + return AVERROR_PATCHWELCOME; + } + + if (get_bits1(gb) /* conformance_window_vps_flag */) { + int vert_mult = hevc_sub_height_c[vps->rep_format.chroma_format_idc]; + int horiz_mult = hevc_sub_width_c[vps->rep_format.chroma_format_idc]; + vps->rep_format.conf_win_left_offset = get_ue_golomb(gb) * horiz_mult; + vps->rep_format.conf_win_right_offset = get_ue_golomb(gb) * horiz_mult; + vps->rep_format.conf_win_top_offset = get_ue_golomb(gb) * vert_mult; + vps->rep_format.conf_win_bottom_offset = get_ue_golomb(gb) * vert_mult; + } + + vps->max_one_active_ref_layer = get_bits1(gb); + vps->poc_lsb_aligned = get_bits1(gb); + + sub_layer_flag_info_present_flag = get_bits1(gb); + for (int j = 0; j < FFMAX(max_sub_layers[0], max_sub_layers[1]); j++) { + int sub_layer_dpb_info_present_flag = 1; + if (j > 0 && sub_layer_flag_info_present_flag) + sub_layer_dpb_info_present_flag = get_bits1(gb); + if (sub_layer_dpb_info_present_flag) { + for (int k = 0; k < av_popcount64(vps->ols[1]); k++) + vps->dpb_size.max_dec_pic_buffering = get_ue_golomb_long(gb) + 1; + vps->dpb_size.max_num_reorder_pics = get_ue_golomb_long(gb); + vps->dpb_size.max_latency_increase = get_ue_golomb_long(gb) - 1; + } + } + + direct_dep_type_len = get_ue_golomb_31(gb) + 2; + if (direct_dep_type_len > 32) { + av_log(avctx, AV_LOG_ERROR, "Invalid direct_dep_type_len: %d\n", + direct_dep_type_len); + return AVERROR_INVALIDDATA; + } + + skip_bits1(gb); /* direct_depenency_all_layers_flag */ + direct_dep_type = get_bits_long(gb, direct_dep_type_len); + if (direct_dep_type > HEVC_DEP_TYPE_BOTH) { + av_log(avctx, AV_LOG_WARNING, "Unsupported direct_dep_type: %d\n", + direct_dep_type); + return AVERROR_PATCHWELCOME; + } + + non_vui_extension_length = get_ue_golomb(gb); + if (non_vui_extension_length > 4096) { + av_log(avctx, AV_LOG_ERROR, "vps_non_vui_extension_length too large: %u\n", + non_vui_extension_length); + return AVERROR_INVALIDDATA; + } + skip_bits_long(gb, non_vui_extension_length * 8); + + if (get_bits1(gb)) // vps_vui_present_flag + av_log(avctx, AV_LOG_WARNING, "VPS VUI not supported\n"); + + return 0; +} + int ff_hevc_decode_nal_vps(GetBitContext *gb, AVCodecContext *avctx, HEVCParamSets *ps) { - int i,j; + int i; int vps_id = get_bits(gb, 4); ptrdiff_t nal_size = gb->buffer_end - gb->buffer; int ret = AVERROR_INVALIDDATA; + uint64_t layer1_id_included = 0; HEVCVPS *vps; if (ps->vps_list[vps_id]) { @@ -491,7 +757,7 @@ int ff_hevc_decode_nal_vps(GetBitContext *gb, AVCodecContext *avctx, goto err; } - if (parse_ptl(gb, avctx, &vps->ptl, vps->vps_max_sub_layers) < 0) + if (parse_ptl(gb, avctx, 1, &vps->ptl, vps->vps_max_sub_layers) < 0) goto err; vps->vps_sub_layer_ordering_info_present_flag = get_bits1(gb); @@ -523,9 +789,14 @@ int ff_hevc_decode_nal_vps(GetBitContext *gb, AVCodecContext *avctx, goto err; } - for (i = 1; i < vps->vps_num_layer_sets; i++) - for (j = 0; j <= vps->vps_max_layer_id; j++) - skip_bits(gb, 1); // layer_id_included_flag[i][j] + vps->num_output_layer_sets = 1; + vps->ols[0] = 1; + + // we support at most 2 layers, so ignore the others + if (vps->vps_num_layer_sets > 1) + layer1_id_included = get_bits64(gb, vps->vps_max_layer_id + 1); // layer_id_included_flag + if (vps->vps_num_layer_sets > 2) + skip_bits_long(gb, (vps->vps_num_layer_sets - 2) * (vps->vps_max_layer_id + 1)); vps->vps_timing_info_present_flag = get_bits1(gb); if (vps->vps_timing_info_present_flag) { @@ -557,7 +828,21 @@ int ff_hevc_decode_nal_vps(GetBitContext *gb, AVCodecContext *avctx, vps->vps_max_sub_layers); } } - get_bits1(gb); /* vps_extension_flag */ + + vps->nb_layers = 1; + vps->layer_idx[0] = 0; + for (int i = 1; i < FF_ARRAY_ELEMS(vps->layer_idx); i++) + vps->layer_idx[i] = -1; + + if (vps->vps_max_layers > 1 && get_bits1(gb)) { /* vps_extension_flag */ + int ret = decode_vps_ext(gb, avctx, vps, layer1_id_included); + if (ret == AVERROR_PATCHWELCOME) { + vps->nb_layers = 1; + av_log(avctx, AV_LOG_WARNING, "Ignoring unsupported VPS extension\n"); + ret = 0; + } else if (ret < 0) + goto err; + } if (get_bits_left(gb) < 0) { av_log(avctx, AV_LOG_ERROR, @@ -890,7 +1175,7 @@ int ff_hevc_parse_sps(HEVCSPS *sps, GetBitContext *gb, unsigned int *sps_id, sps->temporal_id_nesting = get_bits(gb, 1); - if ((ret = parse_ptl(gb, avctx, &sps->ptl, sps->max_sub_layers)) < 0) + if ((ret = parse_ptl(gb, avctx, 1, &sps->ptl, sps->max_sub_layers)) < 0) return ret; *sps_id = get_ue_golomb_long(gb); diff --git a/libavcodec/hevc/ps.h b/libavcodec/hevc/ps.h index 331d163476..bd1acf12e6 100644 --- a/libavcodec/hevc/ps.h +++ b/libavcodec/hevc/ps.h @@ -32,6 +32,8 @@ #include "hevc.h" +#define HEVC_VPS_MAX_LAYERS 2 + typedef struct HEVCSublayerHdrParams { uint32_t bit_rate_value_minus1[HEVC_MAX_CPB_CNT]; uint32_t cpb_size_value_minus1[HEVC_MAX_CPB_CNT]; @@ -153,6 +155,19 @@ typedef struct PTL { uint8_t sub_layer_level_present_flag[HEVC_MAX_SUB_LAYERS]; } PTL; +typedef struct RepFormat { + uint16_t pic_width_in_luma_samples; + uint16_t pic_height_in_luma_samples; + uint8_t chroma_format_idc; + uint8_t separate_colour_plane_flag; + uint8_t bit_depth_luma; ///< bit_depth_vps_luma_minus8 + 8 + uint8_t bit_depth_chroma; ///< bit_depth_vps_chroma_minus8 + 8 + uint16_t conf_win_left_offset; + uint16_t conf_win_right_offset; + uint16_t conf_win_top_offset; + uint16_t conf_win_bottom_offset; +} RepFormat; + typedef struct HEVCVPS { unsigned int vps_id; @@ -176,6 +191,53 @@ typedef struct HEVCVPS { HEVCHdrParams *hdr; + /* VPS extension */ + + /* Number of layers this VPS was parsed for, between 1 and + * min(HEVC_VPS_MAX_LAYERS, vps_max_layers). + * + * Note that vps_max_layers contains the layer count declared in the + * bitstream, while nb_layers contains the number of layers exported to + * users of this API (which may be smaller as we only support a subset of + * multilayer extensions). + * + * Arrays below documented as [layer_idx] have nb_layers valid entries. + */ + int nb_layers; + + // LayerIdxInVps[nuh_layer_id], i.e. a mapping of nuh_layer_id to VPS layer + // indices. Valid values are between 0 and HEVC_VPS_MAX_LAYERS. Entries for + // unmapped values of nuh_layer_id are set to -1. + int8_t layer_idx[HEVC_MAX_NUH_LAYER_ID + 1]; + + uint8_t layer_id_in_nuh[HEVC_VPS_MAX_LAYERS]; + + uint8_t default_ref_layers_active; + uint8_t max_one_active_ref_layer; + uint8_t poc_lsb_aligned; + // bitmask of poc_lsb_not_present[layer_idx] + uint8_t poc_lsb_not_present; + + struct { + unsigned max_dec_pic_buffering; // max_vps_dec_pic_buffering_minus1 + 1 + unsigned max_num_reorder_pics; // max_vps_num_reorder_pics + unsigned max_latency_increase; // max_vps_latency_increase_plus1 - 1 + } dpb_size; + + // ViewId[layer_idx] + uint16_t view_id[HEVC_VPS_MAX_LAYERS]; + + // NumOutputLayerSets + uint8_t num_output_layer_sets; + // Bitmasks specifying output layer sets. i-th bit set means layer with VPS + // index i is present in the layer set. + uint64_t ols[HEVC_VPS_MAX_LAYERS]; + + // NumDirectRefLayers[layer_idx] + uint8_t num_direct_ref_layers[HEVC_VPS_MAX_LAYERS]; + + RepFormat rep_format; + uint8_t *data; int data_size; } HEVCVPS; -- 2.43.0
SPS multilayer extension contains a single flag that we are free to ignore, no reason to print a warning. --- libavcodec/hevc/ps.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/libavcodec/hevc/ps.c b/libavcodec/hevc/ps.c index 0f4ae92588..0e084958be 100644 --- a/libavcodec/hevc/ps.c +++ b/libavcodec/hevc/ps.c @@ -1422,8 +1422,6 @@ int ff_hevc_parse_sps(HEVCSPS *sps, GetBitContext *gb, unsigned int *sps_id, if (sps->multilayer_extension) { skip_bits1(gb); // inter_view_mv_vert_constraint_flag - av_log(avctx, AV_LOG_WARNING, - "sps_multilayer_extension_flag not yet implemented\n"); } if (sps->sps_3d_extension) { -- 2.43.0
Cf. F.7.3.2.2 "Sequence parameter set RBSP syntax", which extends normal SPS parsing with special clauses depending on MultiLayerExtSpsFlag. --- libavcodec/hevc/hevcdec.c | 2 +- libavcodec/hevc/parse.c | 3 +- libavcodec/hevc/parser.c | 2 +- libavcodec/hevc/ps.c | 62 +++++++++++++++++++++++++++++++++++---- libavcodec/hevc/ps.h | 7 +++-- libavcodec/qsvenc_hevc.c | 2 +- 6 files changed, 65 insertions(+), 13 deletions(-) diff --git a/libavcodec/hevc/hevcdec.c b/libavcodec/hevc/hevcdec.c index 6b596f1573..260b9abef0 100644 --- a/libavcodec/hevc/hevcdec.c +++ b/libavcodec/hevc/hevcdec.c @@ -3256,7 +3256,7 @@ static int decode_nal_unit(HEVCContext *s, const H2645NAL *nal) break; case HEVC_NAL_SPS: ret = ff_hevc_decode_nal_sps(&gb, s->avctx, &s->ps, - s->apply_defdispwin); + nal->nuh_layer_id, s->apply_defdispwin); if (ret < 0) goto fail; break; diff --git a/libavcodec/hevc/parse.c b/libavcodec/hevc/parse.c index ec8d1aeacf..ad84b7b152 100644 --- a/libavcodec/hevc/parse.c +++ b/libavcodec/hevc/parse.c @@ -49,7 +49,8 @@ static int hevc_decode_nal_units(const uint8_t *buf, int buf_size, HEVCParamSets goto done; break; case HEVC_NAL_SPS: - ret = ff_hevc_decode_nal_sps(&nal->gb, logctx, ps, apply_defdispwin); + ret = ff_hevc_decode_nal_sps(&nal->gb, logctx, ps, + nal->nuh_layer_id, apply_defdispwin); if (ret < 0) goto done; break; diff --git a/libavcodec/hevc/parser.c b/libavcodec/hevc/parser.c index 8db56e259e..a10f38941b 100644 --- a/libavcodec/hevc/parser.c +++ b/libavcodec/hevc/parser.c @@ -209,7 +209,7 @@ static int parse_nal_units(AVCodecParserContext *s, const uint8_t *buf, ff_hevc_decode_nal_vps(gb, avctx, ps); break; case HEVC_NAL_SPS: - ff_hevc_decode_nal_sps(gb, avctx, ps, 1); + ff_hevc_decode_nal_sps(gb, avctx, ps, nal->nuh_layer_id, 1); break; case HEVC_NAL_PPS: ff_hevc_decode_nal_pps(gb, avctx, ps); diff --git a/libavcodec/hevc/ps.c b/libavcodec/hevc/ps.c index 0e084958be..0b34dd10a8 100644 --- a/libavcodec/hevc/ps.c +++ b/libavcodec/hevc/ps.c @@ -1145,12 +1145,12 @@ static int map_pixel_format(AVCodecContext *avctx, HEVCSPS *sps) } int ff_hevc_parse_sps(HEVCSPS *sps, GetBitContext *gb, unsigned int *sps_id, - int apply_defdispwin, const HEVCVPS * const *vps_list, - AVCodecContext *avctx) + unsigned nuh_layer_id, int apply_defdispwin, + const HEVCVPS * const *vps_list, AVCodecContext *avctx) { HEVCWindow *ow; int ret = 0; - int bit_depth_chroma, start, num_comps; + int bit_depth_chroma, num_comps, multi_layer_ext; int i; // Coded parameters @@ -1167,16 +1167,26 @@ int ff_hevc_parse_sps(HEVCSPS *sps, GetBitContext *gb, unsigned int *sps_id, } sps->max_sub_layers = get_bits(gb, 3) + 1; + multi_layer_ext = nuh_layer_id > 0 && + sps->max_sub_layers == HEVC_MAX_SUB_LAYERS + 1; + if (multi_layer_ext) { + if (!sps->vps) + return AVERROR(EINVAL); + + sps->max_sub_layers = sps->vps->vps_max_sub_layers; + } if (sps->max_sub_layers > HEVC_MAX_SUB_LAYERS) { av_log(avctx, AV_LOG_ERROR, "sps_max_sub_layers out of range: %d\n", sps->max_sub_layers); return AVERROR_INVALIDDATA; } + if (!multi_layer_ext) { sps->temporal_id_nesting = get_bits(gb, 1); if ((ret = parse_ptl(gb, avctx, 1, &sps->ptl, sps->max_sub_layers)) < 0) return ret; + } *sps_id = get_ue_golomb_long(gb); if (*sps_id >= HEVC_MAX_SPS_COUNT) { @@ -1184,6 +1194,28 @@ int ff_hevc_parse_sps(HEVCSPS *sps, GetBitContext *gb, unsigned int *sps_id, return AVERROR_INVALIDDATA; } + if (multi_layer_ext) { + const RepFormat *rf = &sps->vps->rep_format; + + if (get_bits1(gb) && // update_rep_format_flag + get_bits(gb, 8)) { // sps_rep_format_idx + av_log(avctx, AV_LOG_ERROR, "sps_rep_format_idx!=0\n"); + return AVERROR_PATCHWELCOME; + } + + sps->separate_colour_plane = rf->separate_colour_plane_flag; + sps->chroma_format_idc = sps->separate_colour_plane ? 0 : + rf->chroma_format_idc; + sps->bit_depth = rf->bit_depth_luma; + sps->width = rf->pic_width_in_luma_samples; + sps->height = rf->pic_height_in_luma_samples; + + sps->pic_conf_win.left_offset = rf->conf_win_left_offset; + sps->pic_conf_win.right_offset = rf->conf_win_right_offset; + sps->pic_conf_win.top_offset = rf->conf_win_top_offset; + sps->pic_conf_win.bottom_offset = rf->conf_win_bottom_offset; + + } else { sps->chroma_format_idc = get_ue_golomb_long(gb); if (sps->chroma_format_idc > 3U) { av_log(avctx, AV_LOG_ERROR, "chroma_format_idc %d is invalid\n", sps->chroma_format_idc); @@ -1225,7 +1257,6 @@ int ff_hevc_parse_sps(HEVCSPS *sps, GetBitContext *gb, unsigned int *sps_id, sps->pic_conf_win.top_offset = sps->pic_conf_win.bottom_offset = 0; } - sps->output_window = sps->pic_conf_win; } sps->bit_depth = get_ue_golomb_31(gb) + 8; @@ -1248,6 +1279,9 @@ int ff_hevc_parse_sps(HEVCSPS *sps, GetBitContext *gb, unsigned int *sps_id, return AVERROR_INVALIDDATA; } sps->bit_depth_chroma = bit_depth_chroma; + } + + sps->output_window = sps->pic_conf_win; ret = map_pixel_format(avctx, sps); if (ret < 0) @@ -1260,6 +1294,9 @@ int ff_hevc_parse_sps(HEVCSPS *sps, GetBitContext *gb, unsigned int *sps_id, return AVERROR_INVALIDDATA; } + if (!multi_layer_ext) { + int start; + sps->sublayer_ordering_info = get_bits1(gb); start = sps->sublayer_ordering_info ? 0 : sps->max_sub_layers - 1; for (i = start; i < sps->max_sub_layers; i++) { @@ -1289,6 +1326,13 @@ int ff_hevc_parse_sps(HEVCSPS *sps, GetBitContext *gb, unsigned int *sps_id, sps->temporal_layer[i].max_latency_increase = sps->temporal_layer[start].max_latency_increase; } } + } else { + for (int i = 0; i < sps->max_sub_layers; i++) { + sps->temporal_layer[i].max_dec_pic_buffering = sps->vps->dpb_size.max_dec_pic_buffering; + sps->temporal_layer[i].num_reorder_pics = sps->vps->dpb_size.max_num_reorder_pics; + sps->temporal_layer[i].max_latency_increase = sps->vps->dpb_size.max_latency_increase; + } + } sps->log2_min_cb_size = get_ue_golomb_long(gb) + 3; sps->log2_diff_max_min_coding_block_size = get_ue_golomb_long(gb); @@ -1325,6 +1369,11 @@ int ff_hevc_parse_sps(HEVCSPS *sps, GetBitContext *gb, unsigned int *sps_id, if (sps->scaling_list_enabled) { set_default_scaling_list_data(&sps->scaling_list); + if (multi_layer_ext && get_bits1(gb)) { // sps_infer_scaling_list_flag + av_log(avctx, AV_LOG_ERROR, "sps_infer_scaling_list_flag=1 not supported\n"); + return AVERROR_PATCHWELCOME; + } + if (get_bits1(gb)) { ret = scaling_list_data(gb, avctx, &sps->scaling_list, sps); if (ret < 0) @@ -1579,7 +1628,8 @@ static int compare_sps(const HEVCSPS *sps1, const HEVCSPS *sps2) } int ff_hevc_decode_nal_sps(GetBitContext *gb, AVCodecContext *avctx, - HEVCParamSets *ps, int apply_defdispwin) + HEVCParamSets *ps, unsigned nuh_layer_id, + int apply_defdispwin) { HEVCSPS *sps = ff_refstruct_alloc_ext(sizeof(*sps), 0, NULL, hevc_sps_free); unsigned int sps_id; @@ -1598,7 +1648,7 @@ int ff_hevc_decode_nal_sps(GetBitContext *gb, AVCodecContext *avctx, } ret = ff_hevc_parse_sps(sps, gb, &sps_id, - apply_defdispwin, + nuh_layer_id, apply_defdispwin, ps->vps_list, avctx); if (ret < 0) goto err; diff --git a/libavcodec/hevc/ps.h b/libavcodec/hevc/ps.h index bd1acf12e6..6f5b1f8755 100644 --- a/libavcodec/hevc/ps.h +++ b/libavcodec/hevc/ps.h @@ -521,13 +521,14 @@ typedef struct HEVCParamSets { * to an existing VPS */ int ff_hevc_parse_sps(HEVCSPS *sps, GetBitContext *gb, unsigned int *sps_id, - int apply_defdispwin, const HEVCVPS * const *vps_list, - AVCodecContext *avctx); + unsigned nuh_layer_id, int apply_defdispwin, + const HEVCVPS * const *vps_list, AVCodecContext *avctx); int ff_hevc_decode_nal_vps(GetBitContext *gb, AVCodecContext *avctx, HEVCParamSets *ps); int ff_hevc_decode_nal_sps(GetBitContext *gb, AVCodecContext *avctx, - HEVCParamSets *ps, int apply_defdispwin); + HEVCParamSets *ps, unsigned nuh_layer_id, + int apply_defdispwin); int ff_hevc_decode_nal_pps(GetBitContext *gb, AVCodecContext *avctx, HEVCParamSets *ps); diff --git a/libavcodec/qsvenc_hevc.c b/libavcodec/qsvenc_hevc.c index e6c038e67d..1479d4cb8b 100644 --- a/libavcodec/qsvenc_hevc.c +++ b/libavcodec/qsvenc_hevc.c @@ -99,7 +99,7 @@ static int generate_fake_vps(QSVEncContext *q, AVCodecContext *avctx) } get_bits(&gb, 9); - ret = ff_hevc_parse_sps(&sps, &gb, &sps_id, 0, NULL, avctx); + ret = ff_hevc_parse_sps(&sps, &gb, &sps_id, 0, 0, NULL, avctx); av_freep(&sps_rbsp.rbsp_buffer); if (ret < 0) { av_log(avctx, AV_LOG_ERROR, "Error parsing the SPS\n"); -- 2.43.0
On 8/27/2024 12:05 PM, Anton Khirnov wrote:
Cf. F.7.3.2.2 "Sequence parameter set RBSP syntax", which extends normal SPS parsing with special clauses depending on MultiLayerExtSpsFlag. --- libavcodec/hevc/hevcdec.c | 2 +- libavcodec/hevc/parse.c | 3 +- libavcodec/hevc/parser.c | 2 +- libavcodec/hevc/ps.c | 62 +++++++++++++++++++++++++++++++++++---- libavcodec/hevc/ps.h | 7 +++-- libavcodec/qsvenc_hevc.c | 2 +- 6 files changed, 65 insertions(+), 13 deletions(-)
diff --git a/libavcodec/hevc/hevcdec.c b/libavcodec/hevc/hevcdec.c index 6b596f1573..260b9abef0 100644 --- a/libavcodec/hevc/hevcdec.c +++ b/libavcodec/hevc/hevcdec.c @@ -3256,7 +3256,7 @@ static int decode_nal_unit(HEVCContext *s, const H2645NAL *nal) break; case HEVC_NAL_SPS: ret = ff_hevc_decode_nal_sps(&gb, s->avctx, &s->ps, - s->apply_defdispwin); + nal->nuh_layer_id, s->apply_defdispwin); if (ret < 0) goto fail; break; diff --git a/libavcodec/hevc/parse.c b/libavcodec/hevc/parse.c index ec8d1aeacf..ad84b7b152 100644 --- a/libavcodec/hevc/parse.c +++ b/libavcodec/hevc/parse.c @@ -49,7 +49,8 @@ static int hevc_decode_nal_units(const uint8_t *buf, int buf_size, HEVCParamSets goto done; break; case HEVC_NAL_SPS: - ret = ff_hevc_decode_nal_sps(&nal->gb, logctx, ps, apply_defdispwin); + ret = ff_hevc_decode_nal_sps(&nal->gb, logctx, ps, + nal->nuh_layer_id, apply_defdispwin); if (ret < 0) goto done; break; diff --git a/libavcodec/hevc/parser.c b/libavcodec/hevc/parser.c index 8db56e259e..a10f38941b 100644 --- a/libavcodec/hevc/parser.c +++ b/libavcodec/hevc/parser.c @@ -209,7 +209,7 @@ static int parse_nal_units(AVCodecParserContext *s, const uint8_t *buf, ff_hevc_decode_nal_vps(gb, avctx, ps); break; case HEVC_NAL_SPS: - ff_hevc_decode_nal_sps(gb, avctx, ps, 1); + ff_hevc_decode_nal_sps(gb, avctx, ps, nal->nuh_layer_id, 1); break; case HEVC_NAL_PPS: ff_hevc_decode_nal_pps(gb, avctx, ps); diff --git a/libavcodec/hevc/ps.c b/libavcodec/hevc/ps.c index 0e084958be..0b34dd10a8 100644 --- a/libavcodec/hevc/ps.c +++ b/libavcodec/hevc/ps.c @@ -1145,12 +1145,12 @@ static int map_pixel_format(AVCodecContext *avctx, HEVCSPS *sps) }
int ff_hevc_parse_sps(HEVCSPS *sps, GetBitContext *gb, unsigned int *sps_id, - int apply_defdispwin, const HEVCVPS * const *vps_list, - AVCodecContext *avctx) + unsigned nuh_layer_id, int apply_defdispwin, + const HEVCVPS * const *vps_list, AVCodecContext *avctx) { HEVCWindow *ow; int ret = 0; - int bit_depth_chroma, start, num_comps; + int bit_depth_chroma, num_comps, multi_layer_ext; int i;
// Coded parameters @@ -1167,16 +1167,26 @@ int ff_hevc_parse_sps(HEVCSPS *sps, GetBitContext *gb, unsigned int *sps_id, }
sps->max_sub_layers = get_bits(gb, 3) + 1; + multi_layer_ext = nuh_layer_id > 0 && + sps->max_sub_layers == HEVC_MAX_SUB_LAYERS + 1; + if (multi_layer_ext) { + if (!sps->vps) + return AVERROR(EINVAL); + + sps->max_sub_layers = sps->vps->vps_max_sub_layers; + } if (sps->max_sub_layers > HEVC_MAX_SUB_LAYERS) {
Not strictly related to this patch, but sps->max_sub_layers should always be <= vps->vps_max_sub_layers (see F.7.4.3.2.1). So the presence of vps should be checked for.
av_log(avctx, AV_LOG_ERROR, "sps_max_sub_layers out of range: %d\n", sps->max_sub_layers); return AVERROR_INVALIDDATA; }
+ if (!multi_layer_ext) { sps->temporal_id_nesting = get_bits(gb, 1);
Similarly (not strictly related to this patch), this needs to be 1 if sps->max_sub_layers is 0 or if vps->vps_temporal_id_nesting_flag is 1.
if ((ret = parse_ptl(gb, avctx, 1, &sps->ptl, sps->max_sub_layers)) < 0) return ret; + }
(Actually related this time) If multi_layer_ext is true, sps->temporal_id_nesting needs to be set to vps->vps_temporal_id_nesting_flag when sps->max_sub_layers > 1, or hardcoded to 1 if it's 1.
*sps_id = get_ue_golomb_long(gb); if (*sps_id >= HEVC_MAX_SPS_COUNT) { @@ -1184,6 +1194,28 @@ int ff_hevc_parse_sps(HEVCSPS *sps, GetBitContext *gb, unsigned int *sps_id, return AVERROR_INVALIDDATA; }
+ if (multi_layer_ext) { + const RepFormat *rf = &sps->vps->rep_format; + + if (get_bits1(gb) && // update_rep_format_flag + get_bits(gb, 8)) { // sps_rep_format_idx + av_log(avctx, AV_LOG_ERROR, "sps_rep_format_idx!=0\n"); + return AVERROR_PATCHWELCOME; + } + + sps->separate_colour_plane = rf->separate_colour_plane_flag; + sps->chroma_format_idc = sps->separate_colour_plane ? 0 : + rf->chroma_format_idc; + sps->bit_depth = rf->bit_depth_luma; + sps->width = rf->pic_width_in_luma_samples; + sps->height = rf->pic_height_in_luma_samples; + + sps->pic_conf_win.left_offset = rf->conf_win_left_offset; + sps->pic_conf_win.right_offset = rf->conf_win_right_offset; + sps->pic_conf_win.top_offset = rf->conf_win_top_offset; + sps->pic_conf_win.bottom_offset = rf->conf_win_bottom_offset; + + } else { sps->chroma_format_idc = get_ue_golomb_long(gb); if (sps->chroma_format_idc > 3U) { av_log(avctx, AV_LOG_ERROR, "chroma_format_idc %d is invalid\n", sps->chroma_format_idc); @@ -1225,7 +1257,6 @@ int ff_hevc_parse_sps(HEVCSPS *sps, GetBitContext *gb, unsigned int *sps_id, sps->pic_conf_win.top_offset = sps->pic_conf_win.bottom_offset = 0; } - sps->output_window = sps->pic_conf_win; }
sps->bit_depth = get_ue_golomb_31(gb) + 8; @@ -1248,6 +1279,9 @@ int ff_hevc_parse_sps(HEVCSPS *sps, GetBitContext *gb, unsigned int *sps_id, return AVERROR_INVALIDDATA; } sps->bit_depth_chroma = bit_depth_chroma; + } + + sps->output_window = sps->pic_conf_win;
ret = map_pixel_format(avctx, sps); if (ret < 0) @@ -1260,6 +1294,9 @@ int ff_hevc_parse_sps(HEVCSPS *sps, GetBitContext *gb, unsigned int *sps_id, return AVERROR_INVALIDDATA; }
+ if (!multi_layer_ext) { + int start; + sps->sublayer_ordering_info = get_bits1(gb); start = sps->sublayer_ordering_info ? 0 : sps->max_sub_layers - 1; for (i = start; i < sps->max_sub_layers; i++) { @@ -1289,6 +1326,13 @@ int ff_hevc_parse_sps(HEVCSPS *sps, GetBitContext *gb, unsigned int *sps_id, sps->temporal_layer[i].max_latency_increase = sps->temporal_layer[start].max_latency_increase; } } + } else { + for (int i = 0; i < sps->max_sub_layers; i++) { + sps->temporal_layer[i].max_dec_pic_buffering = sps->vps->dpb_size.max_dec_pic_buffering; + sps->temporal_layer[i].num_reorder_pics = sps->vps->dpb_size.max_num_reorder_pics; + sps->temporal_layer[i].max_latency_increase = sps->vps->dpb_size.max_latency_increase; + } + }
sps->log2_min_cb_size = get_ue_golomb_long(gb) + 3; sps->log2_diff_max_min_coding_block_size = get_ue_golomb_long(gb); @@ -1325,6 +1369,11 @@ int ff_hevc_parse_sps(HEVCSPS *sps, GetBitContext *gb, unsigned int *sps_id, if (sps->scaling_list_enabled) { set_default_scaling_list_data(&sps->scaling_list);
+ if (multi_layer_ext && get_bits1(gb)) { // sps_infer_scaling_list_flag + av_log(avctx, AV_LOG_ERROR, "sps_infer_scaling_list_flag=1 not supported\n"); + return AVERROR_PATCHWELCOME; + } + if (get_bits1(gb)) { ret = scaling_list_data(gb, avctx, &sps->scaling_list, sps); if (ret < 0) @@ -1579,7 +1628,8 @@ static int compare_sps(const HEVCSPS *sps1, const HEVCSPS *sps2) }
int ff_hevc_decode_nal_sps(GetBitContext *gb, AVCodecContext *avctx, - HEVCParamSets *ps, int apply_defdispwin) + HEVCParamSets *ps, unsigned nuh_layer_id, + int apply_defdispwin) { HEVCSPS *sps = ff_refstruct_alloc_ext(sizeof(*sps), 0, NULL, hevc_sps_free); unsigned int sps_id; @@ -1598,7 +1648,7 @@ int ff_hevc_decode_nal_sps(GetBitContext *gb, AVCodecContext *avctx, }
ret = ff_hevc_parse_sps(sps, gb, &sps_id, - apply_defdispwin, + nuh_layer_id, apply_defdispwin, ps->vps_list, avctx); if (ret < 0) goto err; diff --git a/libavcodec/hevc/ps.h b/libavcodec/hevc/ps.h index bd1acf12e6..6f5b1f8755 100644 --- a/libavcodec/hevc/ps.h +++ b/libavcodec/hevc/ps.h @@ -521,13 +521,14 @@ typedef struct HEVCParamSets { * to an existing VPS */ int ff_hevc_parse_sps(HEVCSPS *sps, GetBitContext *gb, unsigned int *sps_id, - int apply_defdispwin, const HEVCVPS * const *vps_list, - AVCodecContext *avctx); + unsigned nuh_layer_id, int apply_defdispwin, + const HEVCVPS * const *vps_list, AVCodecContext *avctx);
int ff_hevc_decode_nal_vps(GetBitContext *gb, AVCodecContext *avctx, HEVCParamSets *ps); int ff_hevc_decode_nal_sps(GetBitContext *gb, AVCodecContext *avctx, - HEVCParamSets *ps, int apply_defdispwin); + HEVCParamSets *ps, unsigned nuh_layer_id, + int apply_defdispwin); int ff_hevc_decode_nal_pps(GetBitContext *gb, AVCodecContext *avctx, HEVCParamSets *ps);
diff --git a/libavcodec/qsvenc_hevc.c b/libavcodec/qsvenc_hevc.c index e6c038e67d..1479d4cb8b 100644 --- a/libavcodec/qsvenc_hevc.c +++ b/libavcodec/qsvenc_hevc.c @@ -99,7 +99,7 @@ static int generate_fake_vps(QSVEncContext *q, AVCodecContext *avctx) } get_bits(&gb, 9);
- ret = ff_hevc_parse_sps(&sps, &gb, &sps_id, 0, NULL, avctx); + ret = ff_hevc_parse_sps(&sps, &gb, &sps_id, 0, 0, NULL, avctx); av_freep(&sps_rbsp.rbsp_buffer); if (ret < 0) { av_log(avctx, AV_LOG_ERROR, "Error parsing the SPS\n");
Quoting James Almer (2024-08-30 04:52:39)
On 8/27/2024 12:05 PM, Anton Khirnov wrote:
@@ -1167,16 +1167,26 @@ int ff_hevc_parse_sps(HEVCSPS *sps, GetBitContext *gb, unsigned int *sps_id, }
sps->max_sub_layers = get_bits(gb, 3) + 1; + multi_layer_ext = nuh_layer_id > 0 && + sps->max_sub_layers == HEVC_MAX_SUB_LAYERS + 1; + if (multi_layer_ext) { + if (!sps->vps) + return AVERROR(EINVAL); + + sps->max_sub_layers = sps->vps->vps_max_sub_layers; + } if (sps->max_sub_layers > HEVC_MAX_SUB_LAYERS) {
Not strictly related to this patch, but sps->max_sub_layers should always be <= vps->vps_max_sub_layers (see F.7.4.3.2.1). So the presence of vps should be checked for.
av_log(avctx, AV_LOG_ERROR, "sps_max_sub_layers out of range: %d\n", sps->max_sub_layers); return AVERROR_INVALIDDATA; }
+ if (!multi_layer_ext) { sps->temporal_id_nesting = get_bits(gb, 1);
Similarly (not strictly related to this patch), this needs to be 1 if sps->max_sub_layers is 0 or if vps->vps_temporal_id_nesting_flag is 1.
if ((ret = parse_ptl(gb, avctx, 1, &sps->ptl, sps->max_sub_layers)) < 0) return ret; + }
(Actually related this time) If multi_layer_ext is true, sps->temporal_id_nesting needs to be set to vps->vps_temporal_id_nesting_flag when sps->max_sub_layers > 1, or hardcoded to 1 if it's 1.
Honestly this feels like pointless churn. We don't support any of this temporal id stuff in the decoder, so why bother? -- Anton Khirnov
On 9/3/2024 6:39 AM, Anton Khirnov wrote:
Quoting James Almer (2024-08-30 04:52:39)
On 8/27/2024 12:05 PM, Anton Khirnov wrote:
@@ -1167,16 +1167,26 @@ int ff_hevc_parse_sps(HEVCSPS *sps, GetBitContext *gb, unsigned int *sps_id, }
sps->max_sub_layers = get_bits(gb, 3) + 1; + multi_layer_ext = nuh_layer_id > 0 && + sps->max_sub_layers == HEVC_MAX_SUB_LAYERS + 1; + if (multi_layer_ext) { + if (!sps->vps) + return AVERROR(EINVAL); + + sps->max_sub_layers = sps->vps->vps_max_sub_layers; + } if (sps->max_sub_layers > HEVC_MAX_SUB_LAYERS) {
Not strictly related to this patch, but sps->max_sub_layers should always be <= vps->vps_max_sub_layers (see F.7.4.3.2.1). So the presence of vps should be checked for.
av_log(avctx, AV_LOG_ERROR, "sps_max_sub_layers out of range: %d\n", sps->max_sub_layers); return AVERROR_INVALIDDATA; }
+ if (!multi_layer_ext) { sps->temporal_id_nesting = get_bits(gb, 1);
Similarly (not strictly related to this patch), this needs to be 1 if sps->max_sub_layers is 0 or if vps->vps_temporal_id_nesting_flag is 1.
if ((ret = parse_ptl(gb, avctx, 1, &sps->ptl, sps->max_sub_layers)) < 0) return ret; + }
(Actually related this time) If multi_layer_ext is true, sps->temporal_id_nesting needs to be set to vps->vps_temporal_id_nesting_flag when sps->max_sub_layers > 1, or hardcoded to 1 if it's 1.
Honestly this feels like pointless churn. We don't support any of this temporal id stuff in the decoder, so why bother?
I see it used in videotoolbox.c and vulkan_hevc.c, so imo better make sure it's set right, just in case.
--- libavcodec/hevc/ps.c | 178 +++++++++++++++++++++---------------------- 1 file changed, 89 insertions(+), 89 deletions(-) diff --git a/libavcodec/hevc/ps.c b/libavcodec/hevc/ps.c index 0b34dd10a8..5aebd66ba7 100644 --- a/libavcodec/hevc/ps.c +++ b/libavcodec/hevc/ps.c @@ -1182,10 +1182,10 @@ int ff_hevc_parse_sps(HEVCSPS *sps, GetBitContext *gb, unsigned int *sps_id, } if (!multi_layer_ext) { - sps->temporal_id_nesting = get_bits(gb, 1); + sps->temporal_id_nesting = get_bits(gb, 1); - if ((ret = parse_ptl(gb, avctx, 1, &sps->ptl, sps->max_sub_layers)) < 0) - return ret; + if ((ret = parse_ptl(gb, avctx, 1, &sps->ptl, sps->max_sub_layers)) < 0) + return ret; } *sps_id = get_ue_golomb_long(gb); @@ -1216,69 +1216,69 @@ int ff_hevc_parse_sps(HEVCSPS *sps, GetBitContext *gb, unsigned int *sps_id, sps->pic_conf_win.bottom_offset = rf->conf_win_bottom_offset; } else { - sps->chroma_format_idc = get_ue_golomb_long(gb); - if (sps->chroma_format_idc > 3U) { - av_log(avctx, AV_LOG_ERROR, "chroma_format_idc %d is invalid\n", sps->chroma_format_idc); - return AVERROR_INVALIDDATA; - } - - if (sps->chroma_format_idc == 3) - sps->separate_colour_plane = get_bits1(gb); - - if (sps->separate_colour_plane) - sps->chroma_format_idc = 0; - - sps->width = get_ue_golomb_long(gb); - sps->height = get_ue_golomb_long(gb); - if ((ret = av_image_check_size(sps->width, - sps->height, 0, avctx)) < 0) - return ret; - - sps->conformance_window = get_bits1(gb); - if (sps->conformance_window) { - int vert_mult = hevc_sub_height_c[sps->chroma_format_idc]; - int horiz_mult = hevc_sub_width_c[sps->chroma_format_idc]; - sps->pic_conf_win.left_offset = get_ue_golomb_long(gb) * horiz_mult; - sps->pic_conf_win.right_offset = get_ue_golomb_long(gb) * horiz_mult; - sps->pic_conf_win.top_offset = get_ue_golomb_long(gb) * vert_mult; - sps->pic_conf_win.bottom_offset = get_ue_golomb_long(gb) * vert_mult; - - if (avctx->flags2 & AV_CODEC_FLAG2_IGNORE_CROP) { - av_log(avctx, AV_LOG_DEBUG, - "discarding sps conformance window, " - "original values are l:%u r:%u t:%u b:%u\n", - sps->pic_conf_win.left_offset, - sps->pic_conf_win.right_offset, - sps->pic_conf_win.top_offset, - sps->pic_conf_win.bottom_offset); - - sps->pic_conf_win.left_offset = - sps->pic_conf_win.right_offset = - sps->pic_conf_win.top_offset = - sps->pic_conf_win.bottom_offset = 0; + sps->chroma_format_idc = get_ue_golomb_long(gb); + if (sps->chroma_format_idc > 3U) { + av_log(avctx, AV_LOG_ERROR, "chroma_format_idc %d is invalid\n", sps->chroma_format_idc); + return AVERROR_INVALIDDATA; } - } - sps->bit_depth = get_ue_golomb_31(gb) + 8; - if (sps->bit_depth > 16) { - av_log(avctx, AV_LOG_ERROR, "Luma bit depth (%d) is out of range\n", - sps->bit_depth); - return AVERROR_INVALIDDATA; - } - bit_depth_chroma = get_ue_golomb_31(gb) + 8; - if (bit_depth_chroma > 16) { - av_log(avctx, AV_LOG_ERROR, "Chroma bit depth (%d) is out of range\n", - bit_depth_chroma); - return AVERROR_INVALIDDATA; - } - if (sps->chroma_format_idc && bit_depth_chroma != sps->bit_depth) { - av_log(avctx, AV_LOG_ERROR, - "Luma bit depth (%d) is different from chroma bit depth (%d), " - "this is unsupported.\n", - sps->bit_depth, bit_depth_chroma); - return AVERROR_INVALIDDATA; - } - sps->bit_depth_chroma = bit_depth_chroma; + if (sps->chroma_format_idc == 3) + sps->separate_colour_plane = get_bits1(gb); + + if (sps->separate_colour_plane) + sps->chroma_format_idc = 0; + + sps->width = get_ue_golomb_long(gb); + sps->height = get_ue_golomb_long(gb); + if ((ret = av_image_check_size(sps->width, + sps->height, 0, avctx)) < 0) + return ret; + + sps->conformance_window = get_bits1(gb); + if (sps->conformance_window) { + int vert_mult = hevc_sub_height_c[sps->chroma_format_idc]; + int horiz_mult = hevc_sub_width_c[sps->chroma_format_idc]; + sps->pic_conf_win.left_offset = get_ue_golomb_long(gb) * horiz_mult; + sps->pic_conf_win.right_offset = get_ue_golomb_long(gb) * horiz_mult; + sps->pic_conf_win.top_offset = get_ue_golomb_long(gb) * vert_mult; + sps->pic_conf_win.bottom_offset = get_ue_golomb_long(gb) * vert_mult; + + if (avctx->flags2 & AV_CODEC_FLAG2_IGNORE_CROP) { + av_log(avctx, AV_LOG_DEBUG, + "discarding sps conformance window, " + "original values are l:%u r:%u t:%u b:%u\n", + sps->pic_conf_win.left_offset, + sps->pic_conf_win.right_offset, + sps->pic_conf_win.top_offset, + sps->pic_conf_win.bottom_offset); + + sps->pic_conf_win.left_offset = + sps->pic_conf_win.right_offset = + sps->pic_conf_win.top_offset = + sps->pic_conf_win.bottom_offset = 0; + } + } + + sps->bit_depth = get_ue_golomb_31(gb) + 8; + if (sps->bit_depth > 16) { + av_log(avctx, AV_LOG_ERROR, "Luma bit depth (%d) is out of range\n", + sps->bit_depth); + return AVERROR_INVALIDDATA; + } + bit_depth_chroma = get_ue_golomb_31(gb) + 8; + if (bit_depth_chroma > 16) { + av_log(avctx, AV_LOG_ERROR, "Chroma bit depth (%d) is out of range\n", + bit_depth_chroma); + return AVERROR_INVALIDDATA; + } + if (sps->chroma_format_idc && bit_depth_chroma != sps->bit_depth) { + av_log(avctx, AV_LOG_ERROR, + "Luma bit depth (%d) is different from chroma bit depth (%d), " + "this is unsupported.\n", + sps->bit_depth, bit_depth_chroma); + return AVERROR_INVALIDDATA; + } + sps->bit_depth_chroma = bit_depth_chroma; } sps->output_window = sps->pic_conf_win; @@ -1295,37 +1295,37 @@ int ff_hevc_parse_sps(HEVCSPS *sps, GetBitContext *gb, unsigned int *sps_id, } if (!multi_layer_ext) { - int start; + int start; - sps->sublayer_ordering_info = get_bits1(gb); - start = sps->sublayer_ordering_info ? 0 : sps->max_sub_layers - 1; - for (i = start; i < sps->max_sub_layers; i++) { - sps->temporal_layer[i].max_dec_pic_buffering = get_ue_golomb_long(gb) + 1; - sps->temporal_layer[i].num_reorder_pics = get_ue_golomb_long(gb); - sps->temporal_layer[i].max_latency_increase = get_ue_golomb_long(gb) - 1; - if (sps->temporal_layer[i].max_dec_pic_buffering > (unsigned)HEVC_MAX_DPB_SIZE) { - av_log(avctx, AV_LOG_ERROR, "sps_max_dec_pic_buffering_minus1 out of range: %d\n", - sps->temporal_layer[i].max_dec_pic_buffering - 1U); - return AVERROR_INVALIDDATA; - } - if (sps->temporal_layer[i].num_reorder_pics > sps->temporal_layer[i].max_dec_pic_buffering - 1) { - av_log(avctx, AV_LOG_WARNING, "sps_max_num_reorder_pics out of range: %d\n", - sps->temporal_layer[i].num_reorder_pics); - if (avctx->err_recognition & AV_EF_EXPLODE || - sps->temporal_layer[i].num_reorder_pics > HEVC_MAX_DPB_SIZE - 1) { + sps->sublayer_ordering_info = get_bits1(gb); + start = sps->sublayer_ordering_info ? 0 : sps->max_sub_layers - 1; + for (i = start; i < sps->max_sub_layers; i++) { + sps->temporal_layer[i].max_dec_pic_buffering = get_ue_golomb_long(gb) + 1; + sps->temporal_layer[i].num_reorder_pics = get_ue_golomb_long(gb); + sps->temporal_layer[i].max_latency_increase = get_ue_golomb_long(gb) - 1; + if (sps->temporal_layer[i].max_dec_pic_buffering > (unsigned)HEVC_MAX_DPB_SIZE) { + av_log(avctx, AV_LOG_ERROR, "sps_max_dec_pic_buffering_minus1 out of range: %d\n", + sps->temporal_layer[i].max_dec_pic_buffering - 1U); return AVERROR_INVALIDDATA; } - sps->temporal_layer[i].max_dec_pic_buffering = sps->temporal_layer[i].num_reorder_pics + 1; + if (sps->temporal_layer[i].num_reorder_pics > sps->temporal_layer[i].max_dec_pic_buffering - 1) { + av_log(avctx, AV_LOG_WARNING, "sps_max_num_reorder_pics out of range: %d\n", + sps->temporal_layer[i].num_reorder_pics); + if (avctx->err_recognition & AV_EF_EXPLODE || + sps->temporal_layer[i].num_reorder_pics > HEVC_MAX_DPB_SIZE - 1) { + return AVERROR_INVALIDDATA; + } + sps->temporal_layer[i].max_dec_pic_buffering = sps->temporal_layer[i].num_reorder_pics + 1; + } } - } - if (!sps->sublayer_ordering_info) { - for (i = 0; i < start; i++) { - sps->temporal_layer[i].max_dec_pic_buffering = sps->temporal_layer[start].max_dec_pic_buffering; - sps->temporal_layer[i].num_reorder_pics = sps->temporal_layer[start].num_reorder_pics; - sps->temporal_layer[i].max_latency_increase = sps->temporal_layer[start].max_latency_increase; + if (!sps->sublayer_ordering_info) { + for (i = 0; i < start; i++) { + sps->temporal_layer[i].max_dec_pic_buffering = sps->temporal_layer[start].max_dec_pic_buffering; + sps->temporal_layer[i].num_reorder_pics = sps->temporal_layer[start].num_reorder_pics; + sps->temporal_layer[i].max_latency_increase = sps->temporal_layer[start].max_latency_increase; + } } - } } else { for (int i = 0; i < sps->max_sub_layers; i++) { sps->temporal_layer[i].max_dec_pic_buffering = sps->vps->dpb_size.max_dec_pic_buffering; -- 2.43.0
Otherwise parameter sets from extradata with nuh_layer_id>0 would be ignored. Needed for upcoming MV-HEVC support. --- libavcodec/hevc/parse.c | 3 --- 1 file changed, 3 deletions(-) diff --git a/libavcodec/hevc/parse.c b/libavcodec/hevc/parse.c index ad84b7b152..29f21a5966 100644 --- a/libavcodec/hevc/parse.c +++ b/libavcodec/hevc/parse.c @@ -38,9 +38,6 @@ static int hevc_decode_nal_units(const uint8_t *buf, int buf_size, HEVCParamSets for (i = 0; i < pkt.nb_nals; i++) { H2645NAL *nal = &pkt.nals[i]; - if (nal->nuh_layer_id > 0) - continue; - /* ignore everything except parameter sets and VCL NALUs */ switch (nal->type) { case HEVC_NAL_VPS: -- 2.43.0
A packet should contain a full access unit, which for multilayer video should contain all the layers. --- libavcodec/hevc/parser.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/libavcodec/hevc/parser.c b/libavcodec/hevc/parser.c index a10f38941b..05b92b21ee 100644 --- a/libavcodec/hevc/parser.c +++ b/libavcodec/hevc/parser.c @@ -262,7 +262,7 @@ static int hevc_find_frame_end(AVCodecParserContext *s, const uint8_t *buf, int i; for (i = 0; i < buf_size; i++) { - int nut; + int nut, layer_id; pc->state64 = (pc->state64 << 8) | buf[i]; @@ -270,6 +270,11 @@ static int hevc_find_frame_end(AVCodecParserContext *s, const uint8_t *buf, continue; nut = (pc->state64 >> 2 * 8 + 1) & 0x3F; + + layer_id = (pc->state64 >> 11) & 0x3F; + if (layer_id > 0) + continue; + // Beginning of access unit if ((nut >= HEVC_NAL_VPS && nut <= HEVC_NAL_EOB_NUT) || nut == HEVC_NAL_SEI_PREFIX || (nut >= 41 && nut <= 44) || (nut >= 48 && nut <= 55)) { -- 2.43.0
Cf. F.7.3.6.1 "General slice segment header syntax" --- libavcodec/hevc/hevcdec.c | 36 ++++++++++++++++++++++++++++++++---- libavcodec/hevc/hevcdec.h | 1 + 2 files changed, 33 insertions(+), 4 deletions(-) diff --git a/libavcodec/hevc/hevcdec.c b/libavcodec/hevc/hevcdec.c index 260b9abef0..d9a12279cd 100644 --- a/libavcodec/hevc/hevcdec.c +++ b/libavcodec/hevc/hevcdec.c @@ -584,7 +584,8 @@ static int hls_slice_header(SliceHeader *sh, const HEVCContext *s, GetBitContext { const HEVCPPS *pps; const HEVCSPS *sps; - unsigned pps_id; + const HEVCVPS *vps; + unsigned pps_id, layer_idx; int i, ret; // Coded parameters @@ -607,6 +608,8 @@ static int hls_slice_header(SliceHeader *sh, const HEVCContext *s, GetBitContext pps = s->ps.pps_list[pps_id]; sps = pps->sps; + vps = sps->vps; + layer_idx = vps->layer_idx[s->nuh_layer_id]; if (s->nal_unit_type == HEVC_NAL_CRA_NUT && s->last_eos == 1) sh->no_output_of_prior_pics_flag = 1; @@ -652,7 +655,8 @@ static int hls_slice_header(SliceHeader *sh, const HEVCContext *s, GetBitContext return AVERROR_INVALIDDATA; } if (IS_IRAP(s) && sh->slice_type != HEVC_SLICE_I && - !pps->pps_curr_pic_ref_enabled_flag) { + !pps->pps_curr_pic_ref_enabled_flag && + s->nuh_layer_id == 0) { av_log(s->avctx, AV_LOG_ERROR, "Inter slices in an IRAP frame.\n"); return AVERROR_INVALIDDATA; } @@ -665,8 +669,10 @@ static int hls_slice_header(SliceHeader *sh, const HEVCContext *s, GetBitContext if (sps->separate_colour_plane) sh->colour_plane_id = get_bits(gb, 2); - if (!IS_IDR(s)) { - int poc, pos; + if (!IS_IDR(s) || + (s->nuh_layer_id > 0 && + !(vps->poc_lsb_not_present & (1 << layer_idx)))) { + int poc; sh->pic_order_cnt_lsb = get_bits(gb, sps->log2_max_poc_lsb); poc = ff_hevc_compute_poc(sps, s->poc_tid0, sh->pic_order_cnt_lsb, s->nal_unit_type); @@ -678,6 +684,10 @@ static int hls_slice_header(SliceHeader *sh, const HEVCContext *s, GetBitContext poc = sh->poc; } sh->poc = poc; + } + + if (!IS_IDR(s)) { + int pos; sh->short_term_ref_pic_set_sps_flag = get_bits1(gb); pos = get_bits_left(gb); @@ -724,6 +734,23 @@ static int hls_slice_header(SliceHeader *sh, const HEVCContext *s, GetBitContext sh->slice_temporal_mvp_enabled_flag = 0; } + sh->inter_layer_pred = 0; + if (s->nuh_layer_id > 0) { + int num_direct_ref_layers = vps->num_direct_ref_layers[layer_idx]; + + if (vps->default_ref_layers_active) + sh->inter_layer_pred = !!num_direct_ref_layers; + else if (num_direct_ref_layers) { + sh->inter_layer_pred = get_bits1(gb); + + if (sh->inter_layer_pred && num_direct_ref_layers > 1) { + av_log(s->avctx, AV_LOG_ERROR, + "NumDirectRefLayers>1 not supported\n"); + return AVERROR_PATCHWELCOME; + } + } + } + if (sps->sao_enabled) { sh->slice_sample_adaptive_offset_flag[0] = get_bits1(gb); if (sps->chroma_format_idc) { @@ -3234,6 +3261,7 @@ static int decode_nal_unit(HEVCContext *s, const H2645NAL *nal) int ret; s->nal_unit_type = nal->type; + s->nuh_layer_id = nal->nuh_layer_id; s->temporal_id = nal->temporal_id; if (FF_HW_HAS_CB(s->avctx, decode_params) && diff --git a/libavcodec/hevc/hevcdec.h b/libavcodec/hevc/hevcdec.h index e43f2d0201..7072eee077 100644 --- a/libavcodec/hevc/hevcdec.h +++ b/libavcodec/hevc/hevcdec.h @@ -215,6 +215,7 @@ typedef struct SliceHeader { uint8_t dependent_slice_segment_flag; uint8_t pic_output_flag; uint8_t colour_plane_id; + uint8_t inter_layer_pred; ///< RPS coded in the slice header itself is stored here int short_term_ref_pic_set_sps_flag; -- 2.43.0
The per-frame reference picture set contains two more lists - INTER_LAYER[01]. Assuming at most two layers, INTER_LAYER1 is always empty, but is added anyway for completeness. When inter-layer prediction is enabled, INTER_LAYER0 for the second-layer frame will contain the base-layer frame from the same access unit, if it exists. The new lists are then used in per-slice reference picture set construction as per F.8.3.4 "Decoding process for reference picture lists construction". --- libavcodec/hevc/hevcdec.c | 7 ++++- libavcodec/hevc/hevcdec.h | 8 ++++-- libavcodec/hevc/refs.c | 58 ++++++++++++++++++++++++++++++--------- libavcodec/nvdec_hevc.c | 2 +- libavcodec/vdpau_hevc.c | 2 +- 5 files changed, 59 insertions(+), 18 deletions(-) diff --git a/libavcodec/hevc/hevcdec.c b/libavcodec/hevc/hevcdec.c index d9a12279cd..9a549bb0d2 100644 --- a/libavcodec/hevc/hevcdec.c +++ b/libavcodec/hevc/hevcdec.c @@ -784,7 +784,7 @@ static int hls_slice_header(SliceHeader *sh, const HEVCContext *s, GetBitContext sh->rpl_modification_flag[0] = 0; sh->rpl_modification_flag[1] = 0; - nb_refs = ff_hevc_frame_nb_refs(sh, pps); + nb_refs = ff_hevc_frame_nb_refs(sh, pps, layer_idx); if (!nb_refs) { av_log(s->avctx, AV_LOG_ERROR, "Zero refs for a frame with P or B slices.\n"); return AVERROR_INVALIDDATA; @@ -3352,6 +3352,11 @@ static int decode_nal_units(HEVCContext *s, const uint8_t *buf, int length) s->eos = 0; s->slice_initialized = 0; + for (int i = 0; i < FF_ARRAY_ELEMS(s->layers); i++) { + HEVCLayerContext *l = &s->layers[i]; + l->cur_frame = NULL; + } + /* split the input packet into NAL units, so we know the upper bound on the * number of slices in the frame */ ret = ff_h2645_packet_split(&s->pkt, buf, length, s->avctx, diff --git a/libavcodec/hevc/hevcdec.h b/libavcodec/hevc/hevcdec.h index 7072eee077..eba7cca1a6 100644 --- a/libavcodec/hevc/hevcdec.h +++ b/libavcodec/hevc/hevcdec.h @@ -84,6 +84,8 @@ enum RPSType { ST_FOLL, LT_CURR, LT_FOLL, + INTER_LAYER0, + INTER_LAYER1, NB_RPS_TYPE, }; @@ -442,6 +444,7 @@ typedef struct HEVCLocalContext { typedef struct HEVCLayerContext { HEVCFrame DPB[32]; + HEVCFrame *cur_frame; const HEVCSPS *sps; // RefStruct reference @@ -498,7 +501,7 @@ typedef struct HEVCContext { struct AVMD5 *md5_ctx; ///< candidate references for the current frame - RefPicList rps[5]; + RefPicList rps[NB_RPS_TYPE]; const HEVCVPS *vps; ///< RefStruct reference const HEVCPPS *pps; ///< RefStruct reference @@ -612,7 +615,8 @@ int ff_hevc_res_scale_sign_flag(HEVCLocalContext *lc, int idx); /** * Get the number of candidate references for the current frame. */ -int ff_hevc_frame_nb_refs(const SliceHeader *sh, const HEVCPPS *pps); +int ff_hevc_frame_nb_refs(const SliceHeader *sh, const HEVCPPS *pps, + unsigned layer_idx); int ff_hevc_set_new_ref(HEVCContext *s, HEVCLayerContext *l, int poc); diff --git a/libavcodec/hevc/refs.c b/libavcodec/hevc/refs.c index 09d759f936..f93c5893c6 100644 --- a/libavcodec/hevc/refs.c +++ b/libavcodec/hevc/refs.c @@ -149,6 +149,7 @@ int ff_hevc_set_new_ref(HEVCContext *s, HEVCLayerContext *l, int poc) return AVERROR(ENOMEM); s->cur_frame = ref; + l->cur_frame = ref; s->collocated_ref = NULL; if (s->sh.pic_output_flag) @@ -246,7 +247,9 @@ int ff_hevc_slice_rpl(HEVCContext *s) return ret; if (!(s->rps[ST_CURR_BEF].nb_refs + s->rps[ST_CURR_AFT].nb_refs + - s->rps[LT_CURR].nb_refs) && !s->pps->pps_curr_pic_ref_enabled_flag) { + s->rps[LT_CURR].nb_refs + + s->rps[INTER_LAYER0].nb_refs + s->rps[INTER_LAYER1].nb_refs) && + !s->pps->pps_curr_pic_ref_enabled_flag) { av_log(s->avctx, AV_LOG_ERROR, "Zero refs in the frame RPS.\n"); return AVERROR_INVALIDDATA; } @@ -256,11 +259,14 @@ int ff_hevc_slice_rpl(HEVCContext *s) RefPicList *rpl = &s->cur_frame->refPicList[list_idx]; /* The order of the elements is - * ST_CURR_BEF - ST_CURR_AFT - LT_CURR for the L0 and - * ST_CURR_AFT - ST_CURR_BEF - LT_CURR for the L1 */ - int cand_lists[3] = { list_idx ? ST_CURR_AFT : ST_CURR_BEF, - list_idx ? ST_CURR_BEF : ST_CURR_AFT, - LT_CURR }; + * ST_CURR_BEF - INTER_LAYER0 - ST_CURR_AFT - LT_CURR - INTER_LAYER1 for the L0 and + * ST_CURR_AFT - INTER_LAYER1 - ST_CURR_BEF - LT_CURR - INTER_LAYER0 for the L1 */ + int cand_lists[] = { list_idx ? ST_CURR_AFT : ST_CURR_BEF, + list_idx ? INTER_LAYER1 : INTER_LAYER0, + list_idx ? ST_CURR_BEF : ST_CURR_AFT, + LT_CURR, + list_idx ? INTER_LAYER0 : INTER_LAYER1 + }; /* concatenate the candidate lists for the current frame */ while (rpl_tmp.nb_refs < sh->nb_refs[list_idx]) { @@ -269,7 +275,11 @@ int ff_hevc_slice_rpl(HEVCContext *s) for (j = 0; j < rps->nb_refs && rpl_tmp.nb_refs < HEVC_MAX_REFS; j++) { rpl_tmp.list[rpl_tmp.nb_refs] = rps->list[j]; rpl_tmp.ref[rpl_tmp.nb_refs] = rps->ref[j]; - rpl_tmp.isLongTerm[rpl_tmp.nb_refs] = i == 2; + // multiview inter-layer refs are treated as long-term here, + // cf. G.8.1.3 + rpl_tmp.isLongTerm[rpl_tmp.nb_refs] = cand_lists[i] == LT_CURR || + cand_lists[i] == INTER_LAYER0 || + cand_lists[i] == INTER_LAYER1; rpl_tmp.nb_refs++; } } @@ -408,11 +418,6 @@ int ff_hevc_frame_rps(HEVCContext *s, HEVCLayerContext *l) RefPicList *rps = s->rps; int i, ret = 0; - if (!short_rps) { - rps[0].nb_refs = rps[1].nb_refs = 0; - return 0; - } - unref_missing_refs(l); /* clear the reference flags on all frames except the current one */ @@ -428,6 +433,9 @@ int ff_hevc_frame_rps(HEVCContext *s, HEVCLayerContext *l) for (i = 0; i < NB_RPS_TYPE; i++) rps[i].nb_refs = 0; + if (!short_rps) + goto inter_layer; + /* add the short refs */ for (i = 0; i < short_rps->num_delta_pocs; i++) { int poc = s->poc + short_rps->delta_poc[i]; @@ -457,6 +465,24 @@ int ff_hevc_frame_rps(HEVCContext *s, HEVCLayerContext *l) goto fail; } +inter_layer: + /* add inter-layer refs */ + if (s->sh.inter_layer_pred) { + HEVCLayerContext *l0 = &s->layers[0]; + + av_assert0(l != l0); + + /* Given the assumption of at most two layers, refPicSet0Flag is + * always 1, so only RefPicSetInterLayer0 can ever contain a frame. */ + if (l0->cur_frame) { + // inter-layer refs are treated as short-term here, cf. F.8.1.6 + ret = add_candidate_ref(s, l0, &rps[INTER_LAYER0], l0->cur_frame->poc, + HEVC_FRAME_FLAG_SHORT_REF, 1); + if (ret < 0) + goto fail; + } + } + fail: /* release any frames that are now unused */ for (i = 0; i < FF_ARRAY_ELEMS(l->DPB); i++) @@ -465,7 +491,8 @@ fail: return ret; } -int ff_hevc_frame_nb_refs(const SliceHeader *sh, const HEVCPPS *pps) +int ff_hevc_frame_nb_refs(const SliceHeader *sh, const HEVCPPS *pps, + unsigned layer_idx) { int ret = 0; int i; @@ -484,6 +511,11 @@ int ff_hevc_frame_nb_refs(const SliceHeader *sh, const HEVCPPS *pps) ret += !!long_rps->used[i]; } + if (sh->inter_layer_pred) { + av_assert0(pps->sps->vps->num_direct_ref_layers[layer_idx] < 2); + ret++; + } + if (pps->pps_curr_pic_ref_enabled_flag) ret++; diff --git a/libavcodec/nvdec_hevc.c b/libavcodec/nvdec_hevc.c index 6888507535..e01ce4c782 100644 --- a/libavcodec/nvdec_hevc.c +++ b/libavcodec/nvdec_hevc.c @@ -188,7 +188,7 @@ static int nvdec_hevc_start_frame(AVCodecContext *avctx, .NumBitsForShortTermRPSInSlice = s->sh.short_term_rps ? s->sh.short_term_ref_pic_set_size : 0, .NumDeltaPocsOfRefRpsIdx = s->sh.short_term_rps ? s->sh.short_term_rps->rps_idx_num_delta_pocs : 0, - .NumPocTotalCurr = ff_hevc_frame_nb_refs(&s->sh, pps), + .NumPocTotalCurr = ff_hevc_frame_nb_refs(&s->sh, pps, s->cur_layer), .NumPocStCurrBefore = s->rps[ST_CURR_BEF].nb_refs, .NumPocStCurrAfter = s->rps[ST_CURR_AFT].nb_refs, .NumPocLtCurr = s->rps[LT_CURR].nb_refs, diff --git a/libavcodec/vdpau_hevc.c b/libavcodec/vdpau_hevc.c index affb7e7f5a..0ddcafd897 100644 --- a/libavcodec/vdpau_hevc.c +++ b/libavcodec/vdpau_hevc.c @@ -206,7 +206,7 @@ static int vdpau_hevc_start_frame(AVCodecContext *avctx, } } /* See section 7.4.7.2 of the specification. */ - info->NumPocTotalCurr = ff_hevc_frame_nb_refs(&h->sh, pps); + info->NumPocTotalCurr = ff_hevc_frame_nb_refs(&h->sh, pps, h->cur_layer); if (sh->short_term_ref_pic_set_sps_flag == 0 && sh->short_term_rps) { /* Corresponds to specification field, NumDeltaPocs[RefRpsIdx]. Only applicable when short_term_ref_pic_set_sps_flag == 0. -- 2.43.0
At most two layers are supported. --- doc/decoders.texi | 68 +++++++ libavcodec/hevc/hevcdec.c | 375 ++++++++++++++++++++++++++++++++++---- libavcodec/hevc/hevcdec.h | 41 ++++- libavcodec/hevc/refs.c | 64 +++++-- libavcodec/version.h | 2 +- 5 files changed, 493 insertions(+), 57 deletions(-) diff --git a/doc/decoders.texi b/doc/decoders.texi index 2fcc761d2f..6fc585477c 100644 --- a/doc/decoders.texi +++ b/doc/decoders.texi @@ -38,6 +38,74 @@ Select an operating point of a scalable AV1 bitstream (0 - 31). Default is 0. @end table +@section hevc +HEVC (AKA ITU-T H.265 or ISO/IEC 23008-2) decoder. + +The decoder supports MV-HEVC multiview streams with at most two views. Views to +be output may be selected either as the index of an output layer set defined in +the VPS (the @option{output_layer_set} option), or as a list of view IDs (the +@option{view_ids} option). These options are mutually exclusive - only one may +be specified at a time. These option may be set either statically before decoder +init, or from the @code{get_format()} callback - useful for the case when the +view count or IDs change dynamically during decoding. + +Only the base layer is decoded by default. + +Note that if you are using the @code{ffmpeg} CLI tool, you should be using view +specifiers as documented in its manual, rather than the options documented here. + +@subsection Options + +@table @option + +@item output_layer_set (MV-HEVC) +Select the index of the output layer set that should be output. Available layer +sets may be read by the user from the @option{output_layer_set_available} +option. + +@item output_layer_set_available (MV-HEVC) +This option may be read by the caller to retrieve an array of output layer sets +available in the active VPS. The array is empty for single-layer video. + +Each array element is a bitmask defining layers present in the set. I.e. when +i-th bit is equal to 1, the layer with view ID @option{view_ids[i]} is present +in the set. + +The value of this option is guaranteed to be accurate when read from the +@code{get_format()} callback. It may also be set at other times (e.g. after +opening the decoder), but the value is informational only and may be incorrect +(e.g. when the stream contains multiple distinct VPS NALUs). + +@item view_ids (MV-HEVC) +Specify a list of view IDs that should be output. This option can also be set to +a single '-1', which will cause all views defined in the VPS to be decoded and +output. + +@item view_ids_available (MV-HEVC) +This option may be read by the caller to retrieve an array of view IDs available +in the active VPS. The array is empty for single-layer video. + +Same validity restrictions as for @option{output_layer_set_available} apply to +this option. + +@item view_pos_available (MV-HEVC) +This option may be read by the caller to retrieve an array of view positions +(left, right, or unspecified) available in the active VPS, as AVStereo3DView +values. When the array is available, its elements describe view IDs from +@option{view_ids_available}. + +Same validity restrictions as for @option{output_layer_set_available} apply to +this option. + +@item view_id_cur (MV-HEVC) +This option may be read from within the @code{get_buffer2()} callback to obtain +the view ID of the frame for which the callback is being called. + +It should not be accessed at other times and its contents are then not +specified. + +@end table + @section rawvideo Raw video decoder. diff --git a/libavcodec/hevc/hevcdec.c b/libavcodec/hevc/hevcdec.c index 9a549bb0d2..f0e284bdf6 100644 --- a/libavcodec/hevc/hevcdec.c +++ b/libavcodec/hevc/hevcdec.c @@ -34,6 +34,7 @@ #include "libavutil/mem.h" #include "libavutil/opt.h" #include "libavutil/pixdesc.h" +#include "libavutil/stereo3d.h" #include "libavutil/timecode.h" #include "aom_film_grain.h" @@ -54,6 +55,8 @@ #include "refstruct.h" #include "thread.h" +#define HEVC_OLS_AUTO INT_MAX + static const uint8_t hevc_pel_weight[65] = { [2] = 0, [4] = 1, [6] = 2, [8] = 3, [12] = 4, [16] = 5, [24] = 6, [32] = 7, [48] = 8, [64] = 9 }; /** @@ -417,6 +420,137 @@ static int export_stream_params_from_sei(HEVCContext *s) return 0; } +static int export_multilayer(HEVCContext *s, const HEVCVPS *vps) +{ + const HEVCSEITDRDI *tdrdi = &s->sei.tdrdi; + + av_freep(&s->view_ids_available); + s->nb_view_ids_available = 0; + av_freep(&s->view_pos_available); + s->nb_view_pos_available = 0; + av_freep(&s->output_layer_set_available); + s->nb_output_layer_set_available = 0; + + // don't export anything in the trivial case (1 layer, view id=0) + if (vps->nb_layers < 2 && !vps->view_id[0]) + return 0; + + s->view_ids_available = av_calloc(vps->nb_layers, sizeof(*s->view_ids_available)); + if (!s->view_ids_available) + return AVERROR(ENOMEM); + + if (tdrdi->num_ref_displays) { + s->view_pos_available = av_calloc(vps->nb_layers, sizeof(*s->view_pos_available)); + if (!s->view_pos_available) + return AVERROR(ENOMEM); + } + + for (int i = 0; i < vps->nb_layers; i++) { + s->view_ids_available[i] = vps->view_id[i]; + + if (s->view_pos_available) { + s->view_pos_available[i] = vps->view_id[i] == tdrdi->left_view_id[0] ? + AV_STEREO3D_VIEW_LEFT : + vps->view_id[i] == tdrdi->right_view_id[0] ? + AV_STEREO3D_VIEW_RIGHT : AV_STEREO3D_VIEW_UNSPEC; + } + } + s->nb_view_ids_available = vps->nb_layers; + s->nb_view_pos_available = s->view_pos_available ? vps->nb_layers : 0; + + if (vps->num_output_layer_sets) { + s->output_layer_set_available = av_calloc(vps->num_output_layer_sets, + sizeof(*s->output_layer_set_available)); + if (!s->output_layer_set_available) + return AVERROR(ENOMEM); + + for (int i = 0; i < vps->num_output_layer_sets; i++) + s->output_layer_set_available[i] = vps->ols[i]; + s->nb_output_layer_set_available = vps->num_output_layer_sets; + } + + return 0; +} + +static int setup_multilayer(HEVCContext *s, const HEVCVPS *vps) +{ + unsigned layers_active_output = 0, highest_layer; + + s->layers_active_output = 1; + s->layers_active_decode = 1; + + // nothing requested - decode base layer only + if (!s->nb_view_ids && s->output_layer_set == HEVC_OLS_AUTO) + return 0; + + if (s->nb_view_ids && s->output_layer_set != HEVC_OLS_AUTO) { + av_log(s->avctx, AV_LOG_ERROR, "Both output_layer_set and view_ids " + "were specified, only one of those may be used.\n"); + return AVERROR(EINVAL); + } + + if (s->nb_view_ids == 1 && s->view_ids[0] == -1) { + layers_active_output = (1 << vps->nb_layers) - 1; + } else if (s->nb_view_ids) { + for (int i = 0; i < s->nb_view_ids; i++) { + int view_id = s->view_ids[i]; + int layer_idx = -1; + + if (view_id < 0) { + av_log(s->avctx, AV_LOG_ERROR, + "Invalid view ID requested: %d\n", view_id); + return AVERROR(EINVAL); + } + + for (int j = 0; j < vps->nb_layers; j++) { + if (vps->view_id[j] == view_id) { + layer_idx = j; + break; + } + } + if (layer_idx < 0) { + av_log(s->avctx, AV_LOG_ERROR, + "View ID %d not present in VPS\n", view_id); + return AVERROR(EINVAL); + } + layers_active_output |= 1 << layer_idx; + } + } else { + if (s->output_layer_set < 0 || + s->output_layer_set >= vps->num_output_layer_sets) { + av_log(s->avctx, AV_LOG_ERROR, + "Invalid output layer set index requested: %d\n", s->output_layer_set); + return AVERROR(EINVAL); + } + + layers_active_output = vps->ols[s->output_layer_set]; + } + + if (!layers_active_output) { + av_log(s->avctx, AV_LOG_ERROR, "No layers selected\n"); + return AVERROR_BUG; + } + + highest_layer = ff_log2(layers_active_output); + if (highest_layer >= FF_ARRAY_ELEMS(s->layers)) { + av_log(s->avctx, AV_LOG_ERROR, + "Too many layers requested: %u\n", layers_active_output); + return AVERROR(EINVAL); + } + + /* Assume a higher layer depends on all the lower ones. + * This is enforced in VPS parsing currently, this logic will need + * to be changed if we want to support more complex dependency structures. + */ + s->layers_active_decode = (1 << (highest_layer + 1)) - 1; + s->layers_active_output = layers_active_output; + + av_log(s->avctx, AV_LOG_DEBUG, "decode/output layers: %x/%x\n", + s->layers_active_decode, s->layers_active_output); + + return 0; +} + static enum AVPixelFormat get_format(HEVCContext *s, const HEVCSPS *sps) { #define HWACCEL_MAX (CONFIG_HEVC_DXVA2_HWACCEL + \ @@ -428,6 +562,7 @@ static enum AVPixelFormat get_format(HEVCContext *s, const HEVCSPS *sps) CONFIG_HEVC_VDPAU_HWACCEL + \ CONFIG_HEVC_VULKAN_HWACCEL) enum AVPixelFormat pix_fmts[HWACCEL_MAX + 2], *fmt = pix_fmts; + int ret; switch (sps->pix_fmt) { case AV_PIX_FMT_YUV420P: @@ -547,7 +682,23 @@ static enum AVPixelFormat get_format(HEVCContext *s, const HEVCSPS *sps) *fmt++ = sps->pix_fmt; *fmt = AV_PIX_FMT_NONE; - return ff_get_format(s->avctx, pix_fmts); + // export multilayer information from active VPS to the caller, + // so it is available in get_format() + ret = export_multilayer(s, sps->vps); + if (ret < 0) + return ret; + + ret = ff_get_format(s->avctx, pix_fmts); + if (ret < 0) + return ret; + s->avctx->pix_fmt = ret; + + // set up multilayer decoding, if requested by caller + ret = setup_multilayer(s, sps->vps); + if (ret < 0) + return ret; + + return 0; } static int set_sps(HEVCContext *s, HEVCLayerContext *l, const HEVCSPS *sps) @@ -2944,13 +3095,59 @@ static int set_side_data(HEVCContext *s) return 0; } -static int hevc_frame_start(HEVCContext *s, HEVCLayerContext *l) +static int find_finish_setup_nal(const HEVCContext *s) +{ + int nal_idx = 0; + + for (int i = nal_idx; i < s->pkt.nb_nals; i++) { + const H2645NAL *nal = &s->pkt.nals[i]; + const int layer_id = nal->nuh_layer_id; + GetBitContext gb = nal->gb; + + if (layer_id > HEVC_MAX_NUH_LAYER_ID || s->vps->layer_idx[layer_id] < 0 || + !(s->layers_active_decode & (1 << s->vps->layer_idx[layer_id]))) + continue; + + switch (nal->type) { + case HEVC_NAL_TRAIL_R: + case HEVC_NAL_TRAIL_N: + case HEVC_NAL_TSA_N: + case HEVC_NAL_TSA_R: + case HEVC_NAL_STSA_N: + case HEVC_NAL_STSA_R: + case HEVC_NAL_BLA_W_LP: + case HEVC_NAL_BLA_W_RADL: + case HEVC_NAL_BLA_N_LP: + case HEVC_NAL_IDR_W_RADL: + case HEVC_NAL_IDR_N_LP: + case HEVC_NAL_CRA_NUT: + case HEVC_NAL_RADL_N: + case HEVC_NAL_RADL_R: + case HEVC_NAL_RASL_N: + case HEVC_NAL_RASL_R: + if (!get_bits1(&gb)) // first_slice_segment_in_pic_flag + continue; + case HEVC_NAL_VPS: + case HEVC_NAL_SPS: + case HEVC_NAL_PPS: + nal_idx = i; + break; + } + } + + return nal_idx; +} + +static int hevc_frame_start(HEVCContext *s, HEVCLayerContext *l, + unsigned nal_idx) { const HEVCPPS *const pps = s->ps.pps_list[s->sh.pps_id]; const HEVCSPS *const sps = pps->sps; int pic_size_in_ctb = ((sps->width >> sps->log2_min_cb_size) + 1) * ((sps->height >> sps->log2_min_cb_size) + 1); - int new_sequence = IS_IDR(s) || IS_BLA(s) || s->last_eos; + int new_sequence = (l == &s->layers[0]) && + (IS_IDR(s) || IS_BLA(s) || s->last_eos); + int prev_layers_active = s->layers_active_decode; int ret; if (sps->vps != s->vps && l != &s->layers[0]) { @@ -2961,7 +3158,32 @@ static int hevc_frame_start(HEVCContext *s, HEVCLayerContext *l) ff_refstruct_replace(&s->pps, pps); if (l->sps != sps) { - enum AVPixelFormat pix_fmt; + const HEVCSPS *sps_base = s->layers[0].sps; + enum AVPixelFormat pix_fmt = sps->pix_fmt; + + if (l != &s->layers[0]) { + if (!sps_base) { + av_log(s->avctx, AV_LOG_ERROR, + "Access unit starts with a non-base layer frame\n"); + return AVERROR_INVALIDDATA; + } + + // Files produced by Vision Pro lack VPS extension VUI, + // so the secondary layer has no range information. + // This check avoids failing in such a case. + if (sps_base->pix_fmt == AV_PIX_FMT_YUVJ420P && + sps->pix_fmt == AV_PIX_FMT_YUV420P && + !sps->vui.common.video_signal_type_present_flag) + pix_fmt = sps_base->pix_fmt; + + if (pix_fmt != sps_base->pix_fmt || + sps->width != sps_base->width || + sps->height != sps_base->height) { + av_log(s->avctx, AV_LOG_ERROR, + "Base/non-base layer SPS have unsupported parameter combination\n"); + return AVERROR(ENOSYS); + } + } ff_hevc_clear_refs(l); @@ -2969,14 +3191,17 @@ static int hevc_frame_start(HEVCContext *s, HEVCLayerContext *l) if (ret < 0) return ret; - export_stream_params(s, sps); + if (l == &s->layers[0]) { + export_stream_params(s, sps); - pix_fmt = get_format(s, sps); - if (pix_fmt < 0) - return pix_fmt; - s->avctx->pix_fmt = pix_fmt; + ret = get_format(s, sps); + if (ret < 0) { + set_sps(s, l, NULL); + return ret; + } - new_sequence = 1; + new_sequence = 1; + } } memset(l->horizontal_bs, 0, l->bs_width * l->bs_height); @@ -3011,7 +3236,8 @@ static int hevc_frame_start(HEVCContext *s, HEVCLayerContext *l) s->local_ctx[0].end_of_tiles_x = pps->column_width[0] << sps->log2_ctb_size; if (new_sequence) { - ret = ff_hevc_output_frames(s, l, 0, 0, s->sh.no_output_of_prior_pics_flag); + ret = ff_hevc_output_frames(s, prev_layers_active, + 0, 0, s->sh.no_output_of_prior_pics_flag); if (ret < 0) return ret; } @@ -3068,7 +3294,8 @@ static int hevc_frame_start(HEVCContext *s, HEVCLayerContext *l) s->cur_frame->f->pict_type = 3 - s->sh.slice_type; - ret = ff_hevc_output_frames(s, l, sps->temporal_layer[sps->max_sub_layers - 1].num_reorder_pics, + ret = ff_hevc_output_frames(s, s->layers_active_decode, + sps->temporal_layer[sps->max_sub_layers - 1].num_reorder_pics, sps->temporal_layer[sps->max_sub_layers - 1].max_dec_pic_buffering, 0); if (ret < 0) goto fail; @@ -3079,13 +3306,21 @@ static int hevc_frame_start(HEVCContext *s, HEVCLayerContext *l) goto fail; } - ff_thread_finish_setup(s->avctx); + // after starting the base-layer frame we know which layers will be decoded, + // so we can now figure out which NALUs to wait for before we can call + // ff_thread_finish_setup() + if (l == &s->layers[0]) + s->finish_setup_nal_idx = find_finish_setup_nal(s); + + if (nal_idx >= s->finish_setup_nal_idx) + ff_thread_finish_setup(s->avctx); return 0; fail: - if (s->cur_frame) - ff_hevc_unref_frame(s->cur_frame, ~0); + if (l->cur_frame) + ff_hevc_unref_frame(l->cur_frame, ~0); + l->cur_frame = NULL; s->cur_frame = s->collocated_ref = NULL; s->slice_initialized = 0; return ret; @@ -3160,9 +3395,9 @@ static int verify_md5(HEVCContext *s, AVFrame *frame) return err; } -static int hevc_frame_end(HEVCContext *s) +static int hevc_frame_end(HEVCContext *s, HEVCLayerContext *l) { - HEVCFrame *out = s->cur_frame; + HEVCFrame *out = l->cur_frame; const AVFilmGrainParams *fgp; av_unused int ret; @@ -3194,23 +3429,32 @@ static int hevc_frame_end(HEVCContext *s) } else { if (s->avctx->err_recognition & AV_EF_CRCCHECK && s->sei.picture_hash.is_md5) { - ret = verify_md5(s, s->cur_frame->f); + ret = verify_md5(s, out->f); if (ret < 0 && s->avctx->err_recognition & AV_EF_EXPLODE) return ret; } } s->sei.picture_hash.is_md5 = 0; - av_log(s->avctx, AV_LOG_DEBUG, "Decoded frame with POC %d.\n", s->poc); + av_log(s->avctx, AV_LOG_DEBUG, "Decoded frame with POC %zu/%d.\n", + l - s->layers, s->poc); return 0; } -static int decode_slice(HEVCContext *s, HEVCLayerContext *l, - const H2645NAL *nal, GetBitContext *gb) +static int decode_slice(HEVCContext *s, unsigned nal_idx, GetBitContext *gb) { + const int layer_idx = s->vps ? s->vps->layer_idx[s->nuh_layer_id] : 0; + HEVCLayerContext *l; int ret; + // skip layers not requested to be decoded + // layers_active_decode can only change while decoding a base-layer frame, + // so we can check it for non-base layers + if (layer_idx < 0 || + (s->nuh_layer_id > 0 && !(s->layers_active_decode & (1 << layer_idx)))) + return 0; + ret = hls_slice_header(&s->sh, s, gb); if (ret < 0) { // hls_slice_header() does not cleanup on failure thus the state now is inconsistant so we cannot use it on depandant slices @@ -3226,16 +3470,25 @@ static int decode_slice(HEVCContext *s, HEVCLayerContext *l, return 0; } + // switching to a new layer, mark previous layer's frame (if any) as done + if (s->cur_layer != layer_idx && + s->layers[s->cur_layer].cur_frame && + s->avctx->active_thread_type == FF_THREAD_FRAME) + ff_progress_frame_report(&s->layers[s->cur_layer].cur_frame->tf, INT_MAX); + + s->cur_layer = layer_idx; + l = &s->layers[s->cur_layer]; + if (s->sh.first_slice_in_pic_flag) { - if (s->cur_frame) { + if (l->cur_frame) { av_log(s->avctx, AV_LOG_ERROR, "Two slices reporting being the first in the same frame.\n"); return AVERROR_INVALIDDATA; } - ret = hevc_frame_start(s, l); + ret = hevc_frame_start(s, l, nal_idx); if (ret < 0) return ret; - } else if (!s->cur_frame) { + } else if (!l->cur_frame) { av_log(s->avctx, AV_LOG_ERROR, "First slice in a frame missing.\n"); return AVERROR_INVALIDDATA; } @@ -3247,16 +3500,16 @@ static int decode_slice(HEVCContext *s, HEVCLayerContext *l, return AVERROR_INVALIDDATA; } - ret = decode_slice_data(s, l, nal, gb); + ret = decode_slice_data(s, l, &s->pkt.nals[nal_idx], gb); if (ret < 0) return ret; return 0; } -static int decode_nal_unit(HEVCContext *s, const H2645NAL *nal) +static int decode_nal_unit(HEVCContext *s, unsigned nal_idx) { - HEVCLayerContext *l = &s->layers[0]; + H2645NAL *nal = &s->pkt.nals[nal_idx]; GetBitContext gb = nal->gb; int ret; @@ -3315,7 +3568,7 @@ static int decode_nal_unit(HEVCContext *s, const H2645NAL *nal) case HEVC_NAL_RADL_R: case HEVC_NAL_RASL_N: case HEVC_NAL_RASL_R: - ret = decode_slice(s, l, nal, &gb); + ret = decode_slice(s, nal_idx, &gb); if (ret < 0) goto fail; break; @@ -3416,11 +3669,10 @@ static int decode_nal_units(HEVCContext *s, const uint8_t *buf, int length) H2645NAL *nal = &s->pkt.nals[i]; if (s->avctx->skip_frame >= AVDISCARD_ALL || - (s->avctx->skip_frame >= AVDISCARD_NONREF - && ff_hevc_nal_is_nonref(nal->type)) || nal->nuh_layer_id > 0) + (s->avctx->skip_frame >= AVDISCARD_NONREF && ff_hevc_nal_is_nonref(nal->type))) continue; - ret = decode_nal_unit(s, nal); + ret = decode_nal_unit(s, i); if (ret < 0) { av_log(s->avctx, AV_LOG_WARNING, "Error parsing NAL unit #%d.\n", i); @@ -3429,12 +3681,17 @@ static int decode_nal_units(HEVCContext *s, const uint8_t *buf, int length) } fail: - if (s->cur_frame) { + for (int i = 0; i < FF_ARRAY_ELEMS(s->layers); i++) { + HEVCLayerContext *l = &s->layers[i]; + + if (!l->cur_frame) + continue; + if (ret >= 0) - ret = hevc_frame_end(s); + ret = hevc_frame_end(s, l); if (s->avctx->active_thread_type == FF_THREAD_FRAME) - ff_progress_frame_report(&s->cur_frame->tf, INT_MAX); + ff_progress_frame_report(&l->cur_frame->tf, INT_MAX); } return ret; @@ -3455,6 +3712,11 @@ static int hevc_decode_extradata(HEVCContext *s, uint8_t *buf, int length, int f if (first && s->ps.sps_list[i]) { const HEVCSPS *sps = s->ps.sps_list[i]; export_stream_params(s, sps); + + ret = export_multilayer(s, sps->vps); + if (ret < 0) + return ret; + break; } } @@ -3483,7 +3745,7 @@ static int hevc_receive_frame(AVCodecContext *avctx, AVFrame *frame) av_packet_unref(avpkt); ret = ff_decode_get_packet(avctx, avpkt); if (ret == AVERROR_EOF) { - ret = ff_hevc_output_frames(s, &s->layers[0], 0, 0, 0); + ret = ff_hevc_output_frames(s, s->layers_active_decode, 0, 0, 0); if (ret < 0) return ret; goto do_output; @@ -3547,6 +3809,8 @@ static int hevc_ref_frame(HEVCFrame *dst, const HEVCFrame *src) dst->ctb_count = src->ctb_count; dst->flags = src->flags; + dst->base_layer_frame = src->base_layer_frame; + ff_refstruct_replace(&dst->hwaccel_picture_private, src->hwaccel_picture_private); @@ -3682,9 +3946,25 @@ static int hevc_update_thread_context(AVCodecContext *dst, s->is_nalff = s0->is_nalff; s->nal_length_size = s0->nal_length_size; + s->output_layer_set = s0->output_layer_set; + s->layers_active_decode = s0->layers_active_decode; + s->layers_active_output = s0->layers_active_output; s->film_grain_warning_shown = s0->film_grain_warning_shown; + if (s->nb_view_ids != s0->nb_view_ids || + memcmp(s->view_ids, s0->view_ids, sizeof(*s->view_ids) * s->nb_view_ids)) { + av_freep(&s->view_ids); + s->nb_view_ids = 0; + + if (s0->nb_view_ids) { + s->view_ids = av_memdup(s0->view_ids, s0->nb_view_ids * sizeof(*s0->view_ids)); + if (!s->view_ids) + return AVERROR(ENOMEM); + s->nb_view_ids = s0->nb_view_ids; + } + } + ret = ff_h2645_sei_ctx_replace(&s->sei.common, &s0->sei.common); if (ret < 0) return ret; @@ -3779,6 +4059,31 @@ static const AVOption options[] = { AV_OPT_TYPE_BOOL, {.i64 = 0}, 0, 1, PAR }, { "strict-displaywin", "stricly apply default display window size", OFFSET(apply_defdispwin), AV_OPT_TYPE_BOOL, {.i64 = 0}, 0, 1, PAR }, + { "output_layer_set", "Index of the multilayer output layer set to output", + .offset = OFFSET(output_layer_set), .type = AV_OPT_TYPE_INT, + .default_val.i64 = HEVC_OLS_AUTO, .max = INT_MAX, .flags = PAR, .unit = "output_layer_set", }, + { "auto", "select layer set automatically, or use the view_ids value", + .type = AV_OPT_TYPE_CONST, { .i64 = HEVC_OLS_AUTO }, + .flags = PAR, .unit = "output_layer_set" }, + { "output_layer_set_available", "Array of available output layer sets is exported here", + .offset = OFFSET(output_layer_set_available), .type = AV_OPT_TYPE_UINT64 | AV_OPT_TYPE_FLAG_ARRAY, + .flags = PAR | AV_OPT_FLAG_EXPORT | AV_OPT_FLAG_READONLY }, + { "view_ids", "Array of view IDs that should be decoded and output; a single -1 to decode all views", + .offset = OFFSET(view_ids), .type = AV_OPT_TYPE_INT | AV_OPT_TYPE_FLAG_ARRAY, + .min = -1, .max = INT_MAX, .flags = PAR }, + { "view_ids_available", "Array of available view IDs is exported here", + .offset = OFFSET(view_ids_available), .type = AV_OPT_TYPE_UINT | AV_OPT_TYPE_FLAG_ARRAY, + .flags = PAR | AV_OPT_FLAG_EXPORT | AV_OPT_FLAG_READONLY }, + { "view_pos_available", "Array of view positions for view_ids_available is exported here, as AVStereo3DView", + .offset = OFFSET(view_pos_available), .type = AV_OPT_TYPE_UINT | AV_OPT_TYPE_FLAG_ARRAY, + .flags = PAR | AV_OPT_FLAG_EXPORT | AV_OPT_FLAG_READONLY, .unit = "view_pos" }, + { "unspecified", .type = AV_OPT_TYPE_CONST, .default_val = { .i64 = AV_STEREO3D_VIEW_UNSPEC }, .unit = "view_pos" }, + { "left", .type = AV_OPT_TYPE_CONST, .default_val = { .i64 = AV_STEREO3D_VIEW_LEFT }, .unit = "view_pos" }, + { "right", .type = AV_OPT_TYPE_CONST, .default_val = { .i64 = AV_STEREO3D_VIEW_RIGHT }, .unit = "view_pos" }, + { "view_id_cur", "View ID of the view currently being decoded. Only to be read from within get_buffer2()", + .offset = OFFSET(view_id_cur), .type = AV_OPT_TYPE_UINT, + .flags = PAR | AV_OPT_FLAG_EXPORT | AV_OPT_FLAG_READONLY }, + { NULL }, }; diff --git a/libavcodec/hevc/hevcdec.h b/libavcodec/hevc/hevcdec.h index eba7cca1a6..bfb50b13ff 100644 --- a/libavcodec/hevc/hevcdec.h +++ b/libavcodec/hevc/hevcdec.h @@ -375,6 +375,10 @@ typedef struct HEVCFrame { void *hwaccel_picture_private; ///< RefStruct reference + // for secondary-layer frames, this is the DPB index of the base-layer frame + // from the same AU, if it exists, otherwise -1 + int base_layer_frame; + /** * A combination of HEVC_FRAME_FLAG_* */ @@ -487,9 +491,13 @@ typedef struct HEVCContext { HEVCLocalContext *local_ctx; unsigned nb_local_ctx; - HEVCLayerContext layers[1]; - // index in layers of the layer currently being decoded + // per-layer decoding state, addressed by VPS layer indices + HEVCLayerContext layers[HEVC_VPS_MAX_LAYERS]; + // VPS index of the layer currently being decoded unsigned cur_layer; + // bitmask of layer indices that are active for decoding/output + unsigned layers_active_decode; + unsigned layers_active_output; /** 1 if the independent slice segment header was successfully parsed */ uint8_t slice_initialized; @@ -539,11 +547,31 @@ typedef struct HEVCContext { H2645Packet pkt; // type of the first VCL NAL of the current frame enum HEVCNALUnitType first_nal_type; + // index in pkt.nals of the NAL unit after which we can call + // ff_thread_finish_setup() + unsigned finish_setup_nal_idx; int is_nalff; ///< this flag is != 0 if bitstream is encapsulated ///< as a format defined in 14496-15 int apply_defdispwin; + // multi-layer AVOptions + int output_layer_set; + + uint64_t *output_layer_set_available; + unsigned nb_output_layer_set_available; + + int *view_ids; + unsigned nb_view_ids; + + unsigned *view_ids_available; + unsigned nb_view_ids_available; + + unsigned *view_pos_available; + unsigned nb_view_pos_available; + + unsigned view_id_cur; + int nal_length_size; ///< Number of bytes used for nal length (1, 2 or 4) int nuh_layer_id; @@ -641,12 +669,13 @@ static av_always_inline int ff_hevc_nal_is_nonref(enum HEVCNALUnitType type) * Find frames in the DPB that are ready for output and either write them to the * output FIFO or drop their output flag, depending on the value of discard. * - * @param max_output maximum number of output-pending frames that can be - * present in the DPB before output is triggered + * @param max_output maximum number of AUs with an output-pending frame in at + * least one layer that can be present in the DPB before output + * is triggered * @param max_dpb maximum number of any frames that can be present in the DPB - * before output is triggered + * for any layer before output is triggered */ -int ff_hevc_output_frames(HEVCContext *s, HEVCLayerContext *l, +int ff_hevc_output_frames(HEVCContext *s, unsigned layers_active, unsigned max_output, unsigned max_dpb, int discard); void ff_hevc_unref_frame(HEVCFrame *frame, int flags); diff --git a/libavcodec/hevc/refs.c b/libavcodec/hevc/refs.c index f93c5893c6..dbedd69aa1 100644 --- a/libavcodec/hevc/refs.c +++ b/libavcodec/hevc/refs.c @@ -130,6 +130,8 @@ fail: int ff_hevc_set_new_ref(HEVCContext *s, HEVCLayerContext *l, int poc) { + const HEVCVPS *vps = l->sps->vps; + const int view_id = vps->view_id[s->cur_layer]; HEVCFrame *ref; int i; @@ -144,6 +146,8 @@ int ff_hevc_set_new_ref(HEVCContext *s, HEVCLayerContext *l, int poc) } } + s->view_id_cur = view_id; + ref = alloc_frame(s, l); if (!ref) return AVERROR(ENOMEM); @@ -152,6 +156,9 @@ int ff_hevc_set_new_ref(HEVCContext *s, HEVCLayerContext *l, int poc) l->cur_frame = ref; s->collocated_ref = NULL; + ref->base_layer_frame = (l != &s->layers[0] && s->layers[0].cur_frame) ? + s->layers[0].cur_frame - s->layers[0].DPB : -1; + if (s->sh.pic_output_flag) ref->flags = HEVC_FRAME_FLAG_OUTPUT | HEVC_FRAME_FLAG_SHORT_REF; else @@ -163,6 +170,19 @@ int ff_hevc_set_new_ref(HEVCContext *s, HEVCLayerContext *l, int poc) ref->f->crop_top = l->sps->output_window.top_offset; ref->f->crop_bottom = l->sps->output_window.bottom_offset; + // add view ID side data if it's nontrivial + if (vps->nb_layers > 1 || view_id) { + AVFrameSideData *sd = av_frame_side_data_new(&ref->f->side_data, &ref->f->nb_side_data, + AV_FRAME_DATA_VIEW_ID, sizeof(int), + AV_FRAME_SIDE_DATA_FLAG_REPLACE); + if (!sd) + return AVERROR(ENOMEM); + *(int*)sd->data = view_id; + } + + if (!(s->layers_active_output & (1 << s->cur_layer))) + ref->f->flags |= AV_FRAME_FLAG_DISCARD; + return 0; } @@ -176,30 +196,44 @@ static void unref_missing_refs(HEVCLayerContext *l) } } -int ff_hevc_output_frames(HEVCContext *s, HEVCLayerContext *l, +int ff_hevc_output_frames(HEVCContext *s, unsigned layers_active, unsigned max_output, unsigned max_dpb, int discard) { while (1) { - int nb_dpb = 0; + int nb_dpb[HEVC_VPS_MAX_LAYERS] = { 0 }; int nb_output = 0; int min_poc = INT_MAX; - int i, min_idx, ret; + int min_layer = -1; + int min_idx, ret; - for (i = 0; i < FF_ARRAY_ELEMS(l->DPB); i++) { - HEVCFrame *frame = &l->DPB[i]; - if (frame->flags & HEVC_FRAME_FLAG_OUTPUT) { - nb_output++; - if (frame->poc < min_poc || nb_output == 1) { - min_poc = frame->poc; - min_idx = i; + for (int layer = 0; layer < FF_ARRAY_ELEMS(s->layers); layer++) { + HEVCLayerContext *l = &s->layers[layer]; + + if (!(layers_active & (1 << layer))) + continue; + + for (int i = 0; i < FF_ARRAY_ELEMS(l->DPB); i++) { + HEVCFrame *frame = &l->DPB[i]; + if (frame->flags & HEVC_FRAME_FLAG_OUTPUT) { + // nb_output counts AUs with an output-pending frame + // in at least one layer + if (!(frame->base_layer_frame >= 0 && + (s->layers[0].DPB[frame->base_layer_frame].flags & HEVC_FRAME_FLAG_OUTPUT))) + nb_output++; + if (min_layer < 0 || frame->poc < min_poc) { + min_poc = frame->poc; + min_idx = i; + min_layer = layer; + } } + nb_dpb[layer] += !!frame->flags; } - nb_dpb += !!frame->flags; } if (nb_output > max_output || - (nb_output && nb_dpb > max_dpb)) { - HEVCFrame *frame = &l->DPB[min_idx]; + (nb_output && + (nb_dpb[0] > max_dpb || nb_dpb[1] > max_dpb))) { + HEVCFrame *frame = &s->layers[min_layer].DPB[min_idx]; ret = discard ? 0 : ff_container_fifo_write(s->output_fifo, @@ -208,8 +242,8 @@ int ff_hevc_output_frames(HEVCContext *s, HEVCLayerContext *l, if (ret < 0) return ret; - av_log(s->avctx, AV_LOG_DEBUG, "%s frame with POC %d.\n", - discard ? "Discarded" : "Output", frame->poc); + av_log(s->avctx, AV_LOG_DEBUG, "%s frame with POC %d/%d.\n", + discard ? "Discarded" : "Output", min_layer, frame->poc); continue; } return 0; diff --git a/libavcodec/version.h b/libavcodec/version.h index da2264a097..755c90bbc1 100644 --- a/libavcodec/version.h +++ b/libavcodec/version.h @@ -29,7 +29,7 @@ #include "version_major.h" -#define LIBAVCODEC_VERSION_MINOR 12 +#define LIBAVCODEC_VERSION_MINOR 13 #define LIBAVCODEC_VERSION_MICRO 100 #define LIBAVCODEC_VERSION_INT AV_VERSION_INT(LIBAVCODEC_VERSION_MAJOR, \ -- 2.43.0
From: James Almer <jamrial@gmail.com> Use the 3D Reference Displays Info SEI message to link a view_id with an eye. Signed-off-by: James Almer <jamrial@gmail.com> --- libavcodec/hevc/hevcdec.c | 1 + libavcodec/hevc/refs.c | 19 +++++++++++++++++++ 2 files changed, 20 insertions(+) diff --git a/libavcodec/hevc/hevcdec.c b/libavcodec/hevc/hevcdec.c index f0e284bdf6..0b3c56b526 100644 --- a/libavcodec/hevc/hevcdec.c +++ b/libavcodec/hevc/hevcdec.c @@ -3991,6 +3991,7 @@ static int hevc_update_thread_context(AVCodecContext *dst, s->sei.common.mastering_display = s0->sei.common.mastering_display; s->sei.common.content_light = s0->sei.common.content_light; s->sei.common.aom_film_grain = s0->sei.common.aom_film_grain; + s->sei.tdrdi = s0->sei.tdrdi; return 0; } diff --git a/libavcodec/hevc/refs.c b/libavcodec/hevc/refs.c index dbedd69aa1..a6a7d7b06c 100644 --- a/libavcodec/hevc/refs.c +++ b/libavcodec/hevc/refs.c @@ -22,6 +22,7 @@ */ #include "libavutil/mem.h" +#include "libavutil/stereo3d.h" #include "container_fifo.h" #include "decode.h" @@ -172,12 +173,30 @@ int ff_hevc_set_new_ref(HEVCContext *s, HEVCLayerContext *l, int poc) // add view ID side data if it's nontrivial if (vps->nb_layers > 1 || view_id) { + HEVCSEITDRDI *tdrdi = &s->sei.tdrdi; AVFrameSideData *sd = av_frame_side_data_new(&ref->f->side_data, &ref->f->nb_side_data, AV_FRAME_DATA_VIEW_ID, sizeof(int), AV_FRAME_SIDE_DATA_FLAG_REPLACE); if (!sd) return AVERROR(ENOMEM); *(int*)sd->data = view_id; + + if (tdrdi->num_ref_displays) { + AVStereo3D *stereo_3d; + + av_frame_remove_side_data(ref->f, AV_FRAME_DATA_STEREO3D); + stereo_3d = av_stereo3d_create_side_data(ref->f); + if (!stereo_3d) + return AVERROR(ENOMEM); + + stereo_3d->type = AV_STEREO3D_FRAMESEQUENCE; + if (tdrdi->left_view_id[0] == view_id) + stereo_3d->view = AV_STEREO3D_VIEW_LEFT; + else if (tdrdi->right_view_id[0] == view_id) + stereo_3d->view = AV_STEREO3D_VIEW_RIGHT; + else + stereo_3d->view = AV_STEREO3D_VIEW_UNSPEC; + } } if (!(s->layers_active_output & (1 << s->cur_layer))) -- 2.43.0
Require the character indicating media type to be followed by a non-alphanumeric character (or end of string). Needed by future syntax extensions. --- fftools/cmdutils.c | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/fftools/cmdutils.c b/fftools/cmdutils.c index f725c31c12..8909572ea3 100644 --- a/fftools/cmdutils.c +++ b/fftools/cmdutils.c @@ -988,6 +988,12 @@ FILE *get_preset_file(char *filename, size_t filename_size, return f; } +static int cmdutils_isalnum(char c) +{ + return (c >= '0' && c <= '9') || + (c >= 'A' && c <= 'Z') || + (c >= 'a' && c <= 'z'); +} void stream_specifier_uninit(StreamSpecifier *ss) { @@ -1024,8 +1030,9 @@ int stream_specifier_parse(StreamSpecifier *ss, const char *spec, // this terminates the specifier break; - } else if (*spec == 'v' || *spec == 'a' || *spec == 's' || *spec == 'd' || - *spec == 't' || *spec == 'V') { /* opt:[vasdtV] */ + } else if ((*spec == 'v' || *spec == 'a' || *spec == 's' || + *spec == 'd' || *spec == 't' || *spec == 'V') && + !cmdutils_isalnum(*(spec + 1))) { /* opt:[vasdtV] */ if (ss->media_type != AVMEDIA_TYPE_UNKNOWN) { av_log(logctx, AV_LOG_ERROR, "Stream type specified multiple times\n"); ret = AVERROR(EINVAL); -- 2.43.0
Will be useful for multilayer video. --- fftools/ffmpeg_dec.c | 10 ++-- fftools/ffmpeg_demux.c | 2 +- fftools/ffmpeg_filter.c | 4 +- fftools/ffmpeg_mux_init.c | 2 +- fftools/ffmpeg_sched.c | 102 ++++++++++++++++++++++++++++---------- fftools/ffmpeg_sched.h | 21 ++++++-- 6 files changed, 100 insertions(+), 41 deletions(-) diff --git a/fftools/ffmpeg_dec.c b/fftools/ffmpeg_dec.c index c2bcf784b0..54f7223f0f 100644 --- a/fftools/ffmpeg_dec.c +++ b/fftools/ffmpeg_dec.c @@ -578,7 +578,7 @@ static int process_subtitle(DecoderPriv *dp, AVFrame *frame) if (!subtitle) return 0; - ret = sch_dec_send(dp->sch, dp->sch_idx, frame); + ret = sch_dec_send(dp->sch, dp->sch_idx, 0, frame); if (ret < 0) av_frame_unref(frame); @@ -620,7 +620,7 @@ static int transcode_subtitles(DecoderPriv *dp, const AVPacket *pkt, frame->time_base = pkt->time_base; frame->opaque = (void*)(intptr_t)FRAME_OPAQUE_SUB_HEARTBEAT; - ret = sch_dec_send(dp->sch, dp->sch_idx, frame); + ret = sch_dec_send(dp->sch, dp->sch_idx, 0, frame); return ret == AVERROR_EOF ? AVERROR_EXIT : ret; } else if (pkt && (intptr_t)pkt->opaque == PKT_OPAQUE_FIX_SUB_DURATION) { return fix_sub_duration_heartbeat(dp, av_rescale_q(pkt->pts, pkt->time_base, @@ -773,7 +773,7 @@ static int packet_decode(DecoderPriv *dp, AVPacket *pkt, AVFrame *frame) dp->dec.frames_decoded++; - ret = sch_dec_send(dp->sch, dp->sch_idx, frame); + ret = sch_dec_send(dp->sch, dp->sch_idx, 0, frame); if (ret < 0) { av_frame_unref(frame); return ret == AVERROR_EOF ? AVERROR_EXIT : ret; @@ -951,7 +951,7 @@ static int decoder_thread(void *arg) dp->last_frame_pts + dp->last_frame_duration_est; dt.frame->time_base = dp->last_frame_tb; - ret = sch_dec_send(dp->sch, dp->sch_idx, dt.frame); + ret = sch_dec_send(dp->sch, dp->sch_idx, 0, dt.frame); if (ret < 0 && ret != AVERROR_EOF) { av_log(dp, AV_LOG_FATAL, "Error signalling EOF timestamp: %s\n", av_err2str(ret)); @@ -1355,7 +1355,7 @@ int dec_create(const OptionsContext *o, const char *arg, Scheduler *sch) return ret; enc_idx = ret; - ret = sch_connect(sch, SCH_ENC(enc_idx), SCH_DEC(dp->sch_idx)); + ret = sch_connect(sch, SCH_ENC(enc_idx), SCH_DEC_IN(dp->sch_idx)); if (ret < 0) return ret; diff --git a/fftools/ffmpeg_demux.c b/fftools/ffmpeg_demux.c index 039ee0c785..476efff127 100644 --- a/fftools/ffmpeg_demux.c +++ b/fftools/ffmpeg_demux.c @@ -954,7 +954,7 @@ static int ist_use(InputStream *ist, int decoding_needed) ds->sch_idx_dec = ret; ret = sch_connect(d->sch, SCH_DSTREAM(d->f.index, ds->sch_idx_stream), - SCH_DEC(ds->sch_idx_dec)); + SCH_DEC_IN(ds->sch_idx_dec)); if (ret < 0) return ret; diff --git a/fftools/ffmpeg_filter.c b/fftools/ffmpeg_filter.c index b562e8417c..fb2b1a5b32 100644 --- a/fftools/ffmpeg_filter.c +++ b/fftools/ffmpeg_filter.c @@ -703,7 +703,7 @@ static int ifilter_bind_ist(InputFilter *ifilter, InputStream *ist) if (dec_idx < 0) return dec_idx; - ret = sch_connect(fgp->sch, SCH_DEC(dec_idx), + ret = sch_connect(fgp->sch, SCH_DEC_OUT(dec_idx, 0), SCH_FILTER_IN(fgp->sch_idx, ifp->index)); if (ret < 0) return ret; @@ -749,7 +749,7 @@ static int ifilter_bind_dec(InputFilterPriv *ifp, Decoder *dec) if (dec_idx < 0) return dec_idx; - ret = sch_connect(fgp->sch, SCH_DEC(dec_idx), + ret = sch_connect(fgp->sch, SCH_DEC_OUT(dec_idx, 0), SCH_FILTER_IN(fgp->sch_idx, ifp->index)); if (ret < 0) return ret; diff --git a/fftools/ffmpeg_mux_init.c b/fftools/ffmpeg_mux_init.c index e84fa9719f..5ee2a9685b 100644 --- a/fftools/ffmpeg_mux_init.c +++ b/fftools/ffmpeg_mux_init.c @@ -1458,7 +1458,7 @@ static int ost_add(Muxer *mux, const OptionsContext *o, enum AVMediaType type, ms->sch_idx_src = sched_idx; if (ost->enc) { - ret = sch_connect(mux->sch, SCH_DEC(sched_idx), + ret = sch_connect(mux->sch, SCH_DEC_OUT(sched_idx, 0), SCH_ENC(ms->sch_idx_enc)); if (ret < 0) goto fail; diff --git a/fftools/ffmpeg_sched.c b/fftools/ffmpeg_sched.c index cff824340b..ef0b6e2897 100644 --- a/fftools/ffmpeg_sched.c +++ b/fftools/ffmpeg_sched.c @@ -71,13 +71,19 @@ typedef struct SchTask { int thread_running; } SchTask; +typedef struct SchDecOutput { + SchedulerNode *dst; + uint8_t *dst_finished; + unsigned nb_dst; +} SchDecOutput; + typedef struct SchDec { const AVClass *class; SchedulerNode src; - SchedulerNode *dst; - uint8_t *dst_finished; - unsigned nb_dst; + + SchDecOutput *outputs; + unsigned nb_outputs; SchTask task; // Queue for receiving input packets, one stream. @@ -513,8 +519,14 @@ void sch_free(Scheduler **psch) av_thread_message_queue_free(&dec->queue_end_ts); - av_freep(&dec->dst); - av_freep(&dec->dst_finished); + for (unsigned j = 0; j < dec->nb_outputs; j++) { + SchDecOutput *o = &dec->outputs[j]; + + av_freep(&o->dst); + av_freep(&o->dst_finished); + } + + av_freep(&dec->outputs); av_frame_free(&dec->send_frame); } @@ -712,14 +724,28 @@ int sch_add_demux_stream(Scheduler *sch, unsigned demux_idx) return ret < 0 ? ret : d->nb_streams - 1; } +int sch_add_dec_output(Scheduler *sch, unsigned dec_idx) +{ + SchDec *dec; + int ret; + + av_assert0(dec_idx < sch->nb_dec); + dec = &sch->dec[dec_idx]; + + ret = GROW_ARRAY(dec->outputs, dec->nb_outputs); + if (ret < 0) + return ret; + + return dec->nb_outputs - 1; +} + static const AVClass sch_dec_class = { .class_name = "SchDec", .version = LIBAVUTIL_VERSION_INT, .parent_log_context_offset = offsetof(SchDec, task.func_arg), }; -int sch_add_dec(Scheduler *sch, SchThreadFunc func, void *ctx, - int send_end_ts) +int sch_add_dec(Scheduler *sch, SchThreadFunc func, void *ctx, int send_end_ts) { const unsigned idx = sch->nb_dec; @@ -739,6 +765,10 @@ int sch_add_dec(Scheduler *sch, SchThreadFunc func, void *ctx, if (!dec->send_frame) return AVERROR(ENOMEM); + ret = sch_add_dec_output(sch, idx); + if (ret < 0) + return ret; + ret = queue_alloc(&dec->queue, 1, 0, QUEUE_PACKETS); if (ret < 0) return ret; @@ -943,15 +973,19 @@ int sch_connect(Scheduler *sch, SchedulerNode src, SchedulerNode dst) } case SCH_NODE_TYPE_DEC: { SchDec *dec; + SchDecOutput *o; av_assert0(src.idx < sch->nb_dec); dec = &sch->dec[src.idx]; - ret = GROW_ARRAY(dec->dst, dec->nb_dst); + av_assert0(src.idx_stream < dec->nb_outputs); + o = &dec->outputs[src.idx_stream]; + + ret = GROW_ARRAY(o->dst, o->nb_dst); if (ret < 0) return ret; - dec->dst[dec->nb_dst - 1] = dst; + o->dst[o->nb_dst - 1] = dst; // decoded frames go to filters or encoding switch (dst.type) { @@ -1417,15 +1451,20 @@ static int start_prepare(Scheduler *sch) "Decoder not connected to a source\n"); return AVERROR(EINVAL); } - if (!dec->nb_dst) { - av_log(dec, AV_LOG_ERROR, - "Decoder not connected to any sink\n"); - return AVERROR(EINVAL); - } - dec->dst_finished = av_calloc(dec->nb_dst, sizeof(*dec->dst_finished)); - if (!dec->dst_finished) - return AVERROR(ENOMEM); + for (unsigned j = 0; j < dec->nb_outputs; j++) { + SchDecOutput *o = &dec->outputs[j]; + + if (!o->nb_dst) { + av_log(dec, AV_LOG_ERROR, + "Decoder output %u not connected to any sink\n", j); + return AVERROR(EINVAL); + } + + o->dst_finished = av_calloc(o->nb_dst, sizeof(*o->dst_finished)); + if (!o->dst_finished) + return AVERROR(ENOMEM); + } } for (unsigned i = 0; i < sch->nb_enc; i++) { @@ -2171,21 +2210,26 @@ finish: return AVERROR_EOF; } -int sch_dec_send(Scheduler *sch, unsigned dec_idx, AVFrame *frame) +int sch_dec_send(Scheduler *sch, unsigned dec_idx, + unsigned out_idx, AVFrame *frame) { SchDec *dec; + SchDecOutput *o; int ret; unsigned nb_done = 0; av_assert0(dec_idx < sch->nb_dec); dec = &sch->dec[dec_idx]; - for (unsigned i = 0; i < dec->nb_dst; i++) { - uint8_t *finished = &dec->dst_finished[i]; + av_assert0(out_idx < dec->nb_outputs); + o = &dec->outputs[out_idx]; + + for (unsigned i = 0; i < o->nb_dst; i++) { + uint8_t *finished = &o->dst_finished[i]; AVFrame *to_send = frame; // sending a frame consumes it, so make a temporary reference if needed - if (i < dec->nb_dst - 1) { + if (i < o->nb_dst - 1) { to_send = dec->send_frame; // frame may sometimes contain props only, @@ -2196,7 +2240,7 @@ int sch_dec_send(Scheduler *sch, unsigned dec_idx, AVFrame *frame) return ret; } - ret = dec_send_to_dst(sch, dec->dst[i], finished, to_send); + ret = dec_send_to_dst(sch, o->dst[i], finished, to_send); if (ret < 0) { av_frame_unref(to_send); if (ret == AVERROR_EOF) { @@ -2207,7 +2251,7 @@ int sch_dec_send(Scheduler *sch, unsigned dec_idx, AVFrame *frame) } } - return (nb_done == dec->nb_dst) ? AVERROR_EOF : 0; + return (nb_done == o->nb_dst) ? AVERROR_EOF : 0; } static int dec_done(Scheduler *sch, unsigned dec_idx) @@ -2222,10 +2266,14 @@ static int dec_done(Scheduler *sch, unsigned dec_idx) if (dec->queue_end_ts) av_thread_message_queue_set_err_recv(dec->queue_end_ts, AVERROR_EOF); - for (unsigned i = 0; i < dec->nb_dst; i++) { - int err = dec_send_to_dst(sch, dec->dst[i], &dec->dst_finished[i], NULL); - if (err < 0 && err != AVERROR_EOF) - ret = err_merge(ret, err); + for (unsigned i = 0; i < dec->nb_outputs; i++) { + SchDecOutput *o = &dec->outputs[i]; + + for (unsigned j = 0; j < o->nb_dst; j++) { + int err = dec_send_to_dst(sch, o->dst[j], &o->dst_finished[j], NULL); + if (err < 0 && err != AVERROR_EOF) + ret = err_merge(ret, err); + } } return ret; diff --git a/fftools/ffmpeg_sched.h b/fftools/ffmpeg_sched.h index 7cd839016c..3062c4a6ec 100644 --- a/fftools/ffmpeg_sched.h +++ b/fftools/ffmpeg_sched.h @@ -114,9 +114,12 @@ typedef int (*SchThreadFunc)(void *arg); #define SCH_MSTREAM(file, stream) \ (SchedulerNode){ .type = SCH_NODE_TYPE_MUX, \ .idx = file, .idx_stream = stream } -#define SCH_DEC(decoder) \ +#define SCH_DEC_IN(decoder) \ (SchedulerNode){ .type = SCH_NODE_TYPE_DEC, \ - .idx = decoder } + .idx = decoder } +#define SCH_DEC_OUT(decoder, out_idx) \ + (SchedulerNode){ .type = SCH_NODE_TYPE_DEC, \ + .idx = decoder, .idx_stream = out_idx } #define SCH_ENC(encoder) \ (SchedulerNode){ .type = SCH_NODE_TYPE_ENC, \ .idx = encoder } @@ -178,8 +181,15 @@ int sch_add_demux_stream(Scheduler *sch, unsigned demux_idx); * @retval ">=0" Index of the newly-created decoder. * @retval "<0" Error code. */ -int sch_add_dec(Scheduler *sch, SchThreadFunc func, void *ctx, - int send_end_ts); +int sch_add_dec(Scheduler *sch, SchThreadFunc func, void *ctx, int send_end_ts); + +/** + * Add another output to decoder (e.g. for multiview video). + * + * @retval ">=0" Index of the newly-added decoder output. + * @retval "<0" Error code. + */ +int sch_add_dec_output(Scheduler *sch, unsigned dec_idx); /** * Add a filtergraph to the scheduler. @@ -379,7 +389,8 @@ int sch_dec_receive(Scheduler *sch, unsigned dec_idx, struct AVPacket *pkt); * @retval AVERROR_EOF all consumers are done, should terminate decoding * @retval "another negative error code" other failure */ -int sch_dec_send(Scheduler *sch, unsigned dec_idx, struct AVFrame *frame); +int sch_dec_send(Scheduler *sch, unsigned dec_idx, + unsigned out_idx, struct AVFrame *frame); /** * Called by filtergraph tasks to obtain frames for filtering. Will wait for a -- 2.43.0
This extends the syntax for specifying input streams in -map and complex filtergraph labels, to allow selecting a view by view ID, index, or position. The corresponding decoder is then set up to decode the appropriate view and send frames for that view to the correct filtergraph input(s). --- doc/ffmpeg.texi | 30 +++- fftools/cmdutils.c | 2 +- fftools/cmdutils.h | 2 + fftools/ffmpeg.h | 45 ++++- fftools/ffmpeg_dec.c | 355 +++++++++++++++++++++++++++++++++++++- fftools/ffmpeg_demux.c | 24 ++- fftools/ffmpeg_filter.c | 71 +++++--- fftools/ffmpeg_mux_init.c | 29 +++- fftools/ffmpeg_opt.c | 70 +++++++- 9 files changed, 575 insertions(+), 53 deletions(-) diff --git a/doc/ffmpeg.texi b/doc/ffmpeg.texi index 842e92ad1a..34007f7ea2 100644 --- a/doc/ffmpeg.texi +++ b/doc/ffmpeg.texi @@ -1799,7 +1799,7 @@ Set the size of the canvas used to render subtitles. @section Advanced options @table @option -@item -map [-]@var{input_file_id}[:@var{stream_specifier}][?] | @var{[linklabel]} (@emph{output}) +@item -map [-]@var{input_file_id}[:@var{stream_specifier}][:@var{view_specifier}][?] | @var{[linklabel]} (@emph{output}) Create one or more streams in the output file. This option has two forms for specifying the data source(s): the first selects one or more streams from some @@ -1814,6 +1814,26 @@ only those streams that match the specifier are used (see the A @code{-} character before the stream identifier creates a "negative" mapping. It disables matching streams from already created mappings. +An optional @var{view_specifier} may be given after the stream specifier, which +for multiview video specifies the view to be used. The view specifier may have +one of the following formats: +@table @option +@item view:@var{view_id} +select a view by its ID; @var{view_id} may be set to 'all' to use all the views +interleaved into one stream; + +@item vidx:@var{view_idx} +select a view by its index; i.e. 0 is the base view, 1 is the first non-base +view, etc. + +@item vpos:@var{position} +select a view by its display position; @var{position} may be @code{left} or +@code{right} +@end table +The default for transcoding is to only use the base view, i.e. the equivalent of +@code{vidx:0}. For streamcopy, view specifiers are not supported and all views +are always copied. + A trailing @code{?} after the stream index will allow the map to be optional: if the map matches no streams the map will be ignored instead of failing. Note the map will still fail if an invalid input file index @@ -2206,11 +2226,15 @@ distinguished by the format of the corresponding link label: @item To connect an input stream, use @code{[file_index:stream_specifier]} (i.e. the same syntax as @option{-map}). If @var{stream_specifier} matches multiple -streams, the first one will be used. +streams, the first one will be used. For multiview video, the stream specifier +may be followed by the view specifier, see documentation for the @option{-map} +option for its syntax. @item To connect a loopback decoder use [dec:@var{dec_idx}], where @var{dec_idx} is -the index of the loopback decoder to be connected to given input. +the index of the loopback decoder to be connected to given input. For multiview +video, the decoder index may be followed by the view specifier, see +documentation for the @option{-map} option for its syntax. @item To connect an output from another complex filtergraph, use its link label. E.g diff --git a/fftools/cmdutils.c b/fftools/cmdutils.c index 8909572ea3..d7a78ede12 100644 --- a/fftools/cmdutils.c +++ b/fftools/cmdutils.c @@ -988,7 +988,7 @@ FILE *get_preset_file(char *filename, size_t filename_size, return f; } -static int cmdutils_isalnum(char c) +int cmdutils_isalnum(char c) { return (c >= '0' && c <= '9') || (c >= 'A' && c <= 'Z') || diff --git a/fftools/cmdutils.h b/fftools/cmdutils.h index 9609c6c739..722ec05b0c 100644 --- a/fftools/cmdutils.h +++ b/fftools/cmdutils.h @@ -554,4 +554,6 @@ void remove_avoptions(AVDictionary **a, AVDictionary *b); /* Check if any keys exist in dictionary m */ int check_avoptions(AVDictionary *m); +int cmdutils_isalnum(char c); + #endif /* FFTOOLS_CMDUTILS_H */ diff --git a/fftools/ffmpeg.h b/fftools/ffmpeg.h index 3c5d933e17..ddb4cf47ef 100644 --- a/fftools/ffmpeg.h +++ b/fftools/ffmpeg.h @@ -112,12 +112,32 @@ typedef struct HWDevice { AVBufferRef *device_ref; } HWDevice; +enum ViewSpecifierType { + // no specifier given + VIEW_SPECIFIER_TYPE_NONE = 0, + // val is view index + VIEW_SPECIFIER_TYPE_IDX, + // val is view ID + VIEW_SPECIFIER_TYPE_ID, + // specify view by its position, val is AV_STEREO3D_VIEW_LEFT/RIGHT + VIEW_SPECIFIER_TYPE_POS, + // use all views, val is ignored + VIEW_SPECIFIER_TYPE_ALL, +}; + +typedef struct ViewSpecifier { + enum ViewSpecifierType type; + unsigned val; +} ViewSpecifier; + /* select an input stream for an output stream */ typedef struct StreamMap { int disabled; /* 1 is this mapping is disabled by a negative map */ int file_index; int stream_index; char *linklabel; /* name of an output link, for mapping lavfi outputs */ + + ViewSpecifier vs; } StreamMap; typedef struct OptionsContext { @@ -311,6 +331,10 @@ typedef struct OutputFilterOptions { int sample_rate; AVChannelLayout ch_layout; + + // for simple filtergraphs only, view specifier passed + // along to the decoder + const ViewSpecifier *vs; } OutputFilterOptions; typedef struct InputFilter { @@ -810,7 +834,21 @@ void dec_free(Decoder **pdec); * * @param opts filtergraph input options, to be filled by this function */ -int dec_filter_add(Decoder *dec, InputFilter *ifilter, InputFilterOptions *opts); +int dec_filter_add(Decoder *dec, InputFilter *ifilter, InputFilterOptions *opts, + const ViewSpecifier *vs, SchedulerNode *src); + +/* + * For multiview video, request output of the view(s) determined by vs. + * May be called multiple times. + * + * If this function is never called, only the base view is output. If it is + * called at least once, only the views requested are output. + * + * @param src scheduler node from which the frames corresponding vs + * will originate + */ +int dec_request_view(Decoder *dec, const ViewSpecifier *vs, + SchedulerNode *src); int enc_alloc(Encoder **penc, const AVCodec *codec, Scheduler *sch, unsigned sch_idx); @@ -840,7 +878,8 @@ void ifile_close(InputFile **f); int ist_output_add(InputStream *ist, OutputStream *ost); int ist_filter_add(InputStream *ist, InputFilter *ifilter, int is_simple, - InputFilterOptions *opts); + const ViewSpecifier *vs, InputFilterOptions *opts, + SchedulerNode *src); /** * Find an unused input stream of given type. @@ -868,6 +907,8 @@ void opt_match_per_stream_int64(void *logctx, const SpecifierOptList *sol, void opt_match_per_stream_dbl(void *logctx, const SpecifierOptList *sol, AVFormatContext *fc, AVStream *st, double *out); +int view_specifier_parse(const char **pspec, ViewSpecifier *vs); + int muxer_thread(void *arg); int encoder_thread(void *arg); diff --git a/fftools/ffmpeg_dec.c b/fftools/ffmpeg_dec.c index 54f7223f0f..a173d9bdba 100644 --- a/fftools/ffmpeg_dec.c +++ b/fftools/ffmpeg_dec.c @@ -16,6 +16,8 @@ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA */ +#include <strings.h> + #include "libavutil/avassert.h" #include "libavutil/avstring.h" #include "libavutil/dict.h" @@ -25,6 +27,7 @@ #include "libavutil/opt.h" #include "libavutil/pixdesc.h" #include "libavutil/pixfmt.h" +#include "libavutil/stereo3d.h" #include "libavutil/time.h" #include "libavutil/timestamp.h" @@ -39,6 +42,7 @@ typedef struct DecoderPriv { AVCodecContext *dec_ctx; AVFrame *frame; + AVFrame *frame_tmp_ref; AVPacket *pkt; // override output video sample aspect ratio with this value @@ -77,6 +81,23 @@ typedef struct DecoderPriv { char log_name[32]; char *parent_name; + // user specified decoder multiview options manually + int multiview_user_config; + + struct { + ViewSpecifier vs; + unsigned out_idx; + } *views_requested; + int nb_views_requested; + + /* A map of view ID to decoder outputs. + * MUST NOT be accessed outside of get_format()/get_buffer() */ + struct { + unsigned id; + unsigned out_mask; + } *view_map; + int nb_view_map; + struct { AVDictionary *opts; const AVCodec *codec; @@ -106,6 +127,7 @@ void dec_free(Decoder **pdec) avcodec_free_context(&dp->dec_ctx); av_frame_free(&dp->frame); + av_frame_free(&dp->frame_tmp_ref); av_packet_free(&dp->pkt); av_dict_free(&dp->standalone_init.opts); @@ -116,6 +138,9 @@ void dec_free(Decoder **pdec) av_freep(&dp->parent_name); + av_freep(&dp->views_requested); + av_freep(&dp->view_map); + av_freep(pdec); } @@ -357,7 +382,8 @@ fail: return err; } -static int video_frame_process(DecoderPriv *dp, AVFrame *frame) +static int video_frame_process(DecoderPriv *dp, AVFrame *frame, + unsigned *outputs_mask) { #if FFMPEG_OPT_TOP if (dp->flags & DECODER_FLAG_TOP_FIELD_FIRST) { @@ -419,6 +445,9 @@ static int video_frame_process(DecoderPriv *dp, AVFrame *frame) } } + if (dp->nb_view_map) + *outputs_mask = (uintptr_t)frame->opaque; + return 0; } @@ -715,6 +744,7 @@ static int packet_decode(DecoderPriv *dp, AVPacket *pkt, AVFrame *frame) while (1) { FrameData *fd; + unsigned outputs_mask = 1; av_frame_unref(frame); @@ -763,7 +793,7 @@ static int packet_decode(DecoderPriv *dp, AVPacket *pkt, AVFrame *frame) audio_ts_process(dp, frame); } else { - ret = video_frame_process(dp, frame); + ret = video_frame_process(dp, frame, &outputs_mask); if (ret < 0) { av_log(dp, AV_LOG_FATAL, "Error while processing the decoded data\n"); @@ -773,10 +803,28 @@ static int packet_decode(DecoderPriv *dp, AVPacket *pkt, AVFrame *frame) dp->dec.frames_decoded++; - ret = sch_dec_send(dp->sch, dp->sch_idx, 0, frame); - if (ret < 0) { - av_frame_unref(frame); - return ret == AVERROR_EOF ? AVERROR_EXIT : ret; + for (int i = 0; i < av_popcount(outputs_mask); i++) { + AVFrame *to_send = frame; + int pos; + + av_assert0(outputs_mask); + pos = ffs(outputs_mask) - 1; + outputs_mask &= ~(1U << pos); + + // this is not the last output and sch_dec_send() consumes the frame + // given to it, so make a temporary reference + if (outputs_mask) { + to_send = dp->frame_tmp_ref; + ret = av_frame_ref(to_send, frame); + if (ret < 0) + return ret; + } + + ret = sch_dec_send(dp->sch, dp->sch_idx, pos, to_send); + if (ret < 0) { + av_frame_unref(to_send); + return ret == AVERROR_EOF ? AVERROR_EXIT : ret; + } } } } @@ -975,10 +1023,270 @@ finish: return ret; } +int dec_request_view(Decoder *d, const ViewSpecifier *vs, + SchedulerNode *src) +{ + DecoderPriv *dp = dp_from_dec(d); + unsigned out_idx = 0; + int ret; + + if (dp->multiview_user_config) { + if (!vs || vs->type == VIEW_SPECIFIER_TYPE_NONE) { + *src = SCH_DEC_OUT(dp->sch_idx, 0); + return 0; + } + + av_log(dp, AV_LOG_ERROR, "Manually selecting views with -view_ids " + "or -output_layer_set cannot be combined with view selection " + "via stream specifiers. It is strongly recommended you always " + "use stream specifiers only.\n"); + return AVERROR(EINVAL); + } + + // when multiview_user_config is not set, NONE specifier is treated + // as requesting the base view + vs = (vs && vs->type != VIEW_SPECIFIER_TYPE_NONE) ? vs : + &(ViewSpecifier){ .type = VIEW_SPECIFIER_TYPE_IDX, .val = 0 }; + + // check if the specifier matches an already-existing one + for (int i = 0; i < dp->nb_views_requested; i++) { + const ViewSpecifier *vs1 = &dp->views_requested[i].vs; + + if (vs->type == vs1->type && + (vs->type == VIEW_SPECIFIER_TYPE_ALL || vs->val == vs1->val)) { + *src = SCH_DEC_OUT(dp->sch_idx, dp->views_requested[i].out_idx); + return 0; + } + } + + ret = GROW_ARRAY(dp->views_requested, dp->nb_views_requested); + if (ret < 0) + return ret; + + if (dp->nb_views_requested > 1) { + ret = sch_add_dec_output(dp->sch, dp->sch_idx); + if (ret < 0) + return ret; + out_idx = ret; + } + + dp->views_requested[dp->nb_views_requested - 1].out_idx = out_idx; + dp->views_requested[dp->nb_views_requested - 1].vs = *vs; + + *src = SCH_DEC_OUT(dp->sch_idx, + dp->views_requested[dp->nb_views_requested - 1].out_idx); + + return 0; +} + +static int setup_multiview(DecoderPriv *dp, AVCodecContext *dec_ctx) +{ + unsigned views_wanted = 0; + + unsigned nb_view_ids_av, nb_view_ids; + unsigned *view_ids_av = NULL, *view_pos_av = NULL; + int *view_ids = NULL; + int ret; + + // no views/only base view were requested - do nothing + if (!dp->nb_views_requested || + (dp->nb_views_requested == 1 && + dp->views_requested[0].vs.type == VIEW_SPECIFIER_TYPE_IDX && + dp->views_requested[0].vs.val == 0)) + return 0; + + av_freep(&dp->view_map); + dp->nb_view_map = 0; + + // retrieve views available in current CVS + ret = av_opt_get_array_size(dec_ctx, "view_ids_available", + AV_OPT_SEARCH_CHILDREN, &nb_view_ids_av); + if (ret < 0) { + av_log(dp, AV_LOG_ERROR, + "Multiview decoding requested, but decoder '%s' does not " + "support it\n", dec_ctx->codec->name); + return AVERROR(ENOSYS); + } + + if (nb_view_ids_av) { + unsigned nb_view_pos_av; + + if (nb_view_ids_av >= sizeof(views_wanted) * 8) { + av_log(dp, AV_LOG_ERROR, "Too many views in video: %u\n", nb_view_ids_av); + ret = AVERROR(ENOSYS); + goto fail; + } + + view_ids_av = av_calloc(nb_view_ids_av, sizeof(*view_ids_av)); + if (!view_ids_av) { + ret = AVERROR(ENOMEM); + goto fail; + } + + ret = av_opt_get_array(dec_ctx, "view_ids_available", + AV_OPT_SEARCH_CHILDREN, 0, nb_view_ids_av, + AV_OPT_TYPE_UINT, view_ids_av); + if (ret < 0) + goto fail; + + ret = av_opt_get_array_size(dec_ctx, "view_pos_available", + AV_OPT_SEARCH_CHILDREN, &nb_view_pos_av); + if (ret >= 0 && nb_view_pos_av == nb_view_ids_av) { + view_pos_av = av_calloc(nb_view_ids_av, sizeof(*view_pos_av)); + if (!view_pos_av) { + ret = AVERROR(ENOMEM); + goto fail; + } + + ret = av_opt_get_array(dec_ctx, "view_pos_available", + AV_OPT_SEARCH_CHILDREN, 0, nb_view_ids_av, + AV_OPT_TYPE_UINT, view_pos_av); + if (ret < 0) + goto fail; + } + } else { + // assume there is a single view with ID=0 + nb_view_ids_av = 1; + view_ids_av = av_calloc(nb_view_ids_av, sizeof(*view_ids_av)); + view_pos_av = av_calloc(nb_view_ids_av, sizeof(*view_pos_av)); + if (!view_ids_av || !view_pos_av) { + ret = AVERROR(ENOMEM); + goto fail; + } + view_pos_av[0] = AV_STEREO3D_VIEW_UNSPEC; + } + + dp->view_map = av_calloc(nb_view_ids_av, sizeof(*dp->view_map)); + if (!dp->view_map) { + ret = AVERROR(ENOMEM); + goto fail; + } + dp->nb_view_map = nb_view_ids_av; + + for (int i = 0; i < dp->nb_view_map; i++) + dp->view_map[i].id = view_ids_av[i]; + + // figure out which views should go to which output + for (int i = 0; i < dp->nb_views_requested; i++) { + const ViewSpecifier *vs = &dp->views_requested[i].vs; + + switch (vs->type) { + case VIEW_SPECIFIER_TYPE_IDX: + if (vs->val >= nb_view_ids_av) { + av_log(dp, AV_LOG_WARNING, + "View with index %u requested, but only %u views available " + "in current video sequence (more views may or may not be " + "available in later sequences).\n", + vs->val, nb_view_ids_av); + continue; + } + views_wanted |= 1U << vs->val; + dp->view_map[vs->val].out_mask |= 1U << i; + + break; + case VIEW_SPECIFIER_TYPE_ID: { + int view_idx = -1; + + for (unsigned j = 0; j < nb_view_ids_av; j++) { + if (view_ids_av[j] == vs->val) { + view_idx = j; + break; + } + } + if (view_idx < 0) { + av_log(dp, AV_LOG_WARNING, "View with ID %u requested, " + "but is not available in the video sequence\n", vs->val); + continue; + } + views_wanted |= 1U << view_idx; + dp->view_map[view_idx].out_mask |= 1U << i; + + break; + } + case VIEW_SPECIFIER_TYPE_POS: { + int view_idx = -1; + + for (unsigned j = 0; view_pos_av && j < nb_view_ids_av; j++) { + if (view_pos_av[j] == vs->val) { + view_idx = j; + break; + } + } + if (view_idx < 0) { + av_log(dp, AV_LOG_WARNING, "View position %s requested, " + "but is not available in the video sequence\n", av_stereo3d_view_name(vs->val)); + continue; + } + views_wanted |= 1U << view_idx; + dp->view_map[view_idx].out_mask |= 1U << i; + + break; + } + case VIEW_SPECIFIER_TYPE_ALL: + views_wanted |= (1U << nb_view_ids_av) - 1; + + for (int j = 0; j < dp->nb_view_map; j++) + dp->view_map[j].out_mask |= 1U << i; + + break; + } + } + av_assert0(views_wanted); + + // signal to decoder which views we want + nb_view_ids = av_popcount(views_wanted); + view_ids = av_malloc_array(nb_view_ids, sizeof(*view_ids)); + if (!view_ids) { + ret = AVERROR(ENOMEM); + goto fail; + } + + for (unsigned i = 0; i < nb_view_ids; i++) { + int pos; + + av_assert0(views_wanted); + pos = ffs(views_wanted) - 1; + views_wanted &= ~(1U << pos); + + view_ids[i] = view_ids_av[pos]; + } + + // unset view_ids in case we set it earlier + av_opt_set(dec_ctx, "view_ids", NULL, AV_OPT_SEARCH_CHILDREN); + + ret = av_opt_set_array(dec_ctx, "view_ids", AV_OPT_SEARCH_CHILDREN, + 0, nb_view_ids, AV_OPT_TYPE_INT, view_ids); + if (ret < 0) + goto fail; + + if (!dp->frame_tmp_ref) { + dp->frame_tmp_ref = av_frame_alloc(); + if (!dp->frame_tmp_ref) { + ret = AVERROR(ENOMEM); + goto fail; + } + } + +fail: + av_freep(&view_ids_av); + av_freep(&view_pos_av); + av_freep(&view_ids); + + return ret; +} + static enum AVPixelFormat get_format(AVCodecContext *s, const enum AVPixelFormat *pix_fmts) { DecoderPriv *dp = s->opaque; const enum AVPixelFormat *p; + int ret; + + ret = setup_multiview(dp, s); + if (ret < 0) { + av_log(dp, AV_LOG_ERROR, "Error setting up multiview decoding: %s\n", + av_err2str(ret)); + return AV_PIX_FMT_NONE; + } for (p = pix_fmts; *p != AV_PIX_FMT_NONE; p++) { const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(*p); @@ -1009,6 +1317,27 @@ static enum AVPixelFormat get_format(AVCodecContext *s, const enum AVPixelFormat return *p; } +static int get_buffer(AVCodecContext *dec_ctx, AVFrame *frame, int flags) +{ + DecoderPriv *dp = dec_ctx->opaque; + + // for multiview video, store the output mask in frame opaque + if (dp->nb_view_map) { + int64_t view_id = 0; + + av_opt_get_int(dec_ctx, "view_id_cur", AV_OPT_SEARCH_CHILDREN, &view_id); + + for (int i = 0; i < dp->nb_view_map; i++) { + if (dp->view_map[i].id == view_id) { + frame->opaque = (void*)(uintptr_t)dp->view_map[i].out_mask; + break; + } + } + } + + return avcodec_default_get_buffer2(dec_ctx, frame, flags); +} + static HWDevice *hw_device_match_by_codec(const AVCodec *codec) { const AVCodecHWConfig *config; @@ -1202,6 +1531,7 @@ static int dec_open(DecoderPriv *dp, AVDictionary **dec_opts, dp->dec_ctx->opaque = dp; dp->dec_ctx->get_format = get_format; + dp->dec_ctx->get_buffer2 = get_buffer; dp->dec_ctx->pkt_timebase = o->time_base; if (!av_dict_get(*dec_opts, "threads", NULL, 0)) @@ -1291,6 +1621,10 @@ int dec_init(Decoder **pdec, Scheduler *sch, if (ret < 0) return ret; + if (av_dict_get(*dec_opts, "view_ids", NULL, 0) || + av_dict_get(*dec_opts, "output_layer_set", NULL, 0)) + dp->multiview_user_config = 1; + ret = dec_open(dp, dec_opts, o, param_out); if (ret < 0) goto fail; @@ -1363,6 +1697,10 @@ int dec_create(const OptionsContext *o, const char *arg, Scheduler *sch) if (ret < 0) return ret; + if (av_dict_get(dp->standalone_init.opts, "view_ids", NULL, 0) || + av_dict_get(dp->standalone_init.opts, "output_layer_set", NULL, 0)) + dp->multiview_user_config = 1; + if (o->codec_names.nb_opt) { const char *name = o->codec_names.opt[o->codec_names.nb_opt - 1].u.str; dp->standalone_init.codec = avcodec_find_decoder_by_name(name); @@ -1375,7 +1713,8 @@ int dec_create(const OptionsContext *o, const char *arg, Scheduler *sch) return 0; } -int dec_filter_add(Decoder *d, InputFilter *ifilter, InputFilterOptions *opts) +int dec_filter_add(Decoder *d, InputFilter *ifilter, InputFilterOptions *opts, + const ViewSpecifier *vs, SchedulerNode *src) { DecoderPriv *dp = dp_from_dec(d); char name[16]; @@ -1385,5 +1724,5 @@ int dec_filter_add(Decoder *d, InputFilter *ifilter, InputFilterOptions *opts) if (!opts->name) return AVERROR(ENOMEM); - return dp->sch_idx; + return dec_request_view(d, vs, src); } diff --git a/fftools/ffmpeg_demux.c b/fftools/ffmpeg_demux.c index 476efff127..7eb8ecdd89 100644 --- a/fftools/ffmpeg_demux.c +++ b/fftools/ffmpeg_demux.c @@ -874,7 +874,8 @@ void ifile_close(InputFile **pf) av_freep(pf); } -static int ist_use(InputStream *ist, int decoding_needed) +static int ist_use(InputStream *ist, int decoding_needed, + const ViewSpecifier *vs, SchedulerNode *src) { Demuxer *d = demuxer_from_ifile(ist->file); DemuxStream *ds = ds_from_ist(ist); @@ -961,15 +962,26 @@ static int ist_use(InputStream *ist, int decoding_needed) d->have_audio_dec |= is_audio; } + if (decoding_needed && ist->par->codec_type == AVMEDIA_TYPE_VIDEO) { + ret = dec_request_view(ist->decoder, vs, src); + if (ret < 0) + return ret; + } else { + *src = decoding_needed ? + SCH_DEC_OUT(ds->sch_idx_dec, 0) : + SCH_DSTREAM(d->f.index, ds->sch_idx_stream); + } + return 0; } int ist_output_add(InputStream *ist, OutputStream *ost) { DemuxStream *ds = ds_from_ist(ist); + SchedulerNode src; int ret; - ret = ist_use(ist, ost->enc ? DECODING_FOR_OST : 0); + ret = ist_use(ist, ost->enc ? DECODING_FOR_OST : 0, NULL, &src); if (ret < 0) return ret; @@ -983,14 +995,16 @@ int ist_output_add(InputStream *ist, OutputStream *ost) } int ist_filter_add(InputStream *ist, InputFilter *ifilter, int is_simple, - InputFilterOptions *opts) + const ViewSpecifier *vs, InputFilterOptions *opts, + SchedulerNode *src) { Demuxer *d = demuxer_from_ifile(ist->file); DemuxStream *ds = ds_from_ist(ist); int64_t tsoffset = 0; int ret; - ret = ist_use(ist, is_simple ? DECODING_FOR_OST : DECODING_FOR_FILTER); + ret = ist_use(ist, is_simple ? DECODING_FOR_OST : DECODING_FOR_FILTER, + vs, src); if (ret < 0) return ret; @@ -1074,7 +1088,7 @@ int ist_filter_add(InputStream *ist, InputFilter *ifilter, int is_simple, opts->flags |= IFILTER_FLAG_AUTOROTATE * !!(ds->autorotate) | IFILTER_FLAG_REINIT * !!(ds->reinit_filters); - return ds->sch_idx_dec; + return 0; } static int choose_decoder(const OptionsContext *o, void *logctx, diff --git a/fftools/ffmpeg_filter.c b/fftools/ffmpeg_filter.c index fb2b1a5b32..aa5ebb89ed 100644 --- a/fftools/ffmpeg_filter.c +++ b/fftools/ffmpeg_filter.c @@ -676,11 +676,13 @@ static OutputFilter *ofilter_alloc(FilterGraph *fg, enum AVMediaType type) return ofilter; } -static int ifilter_bind_ist(InputFilter *ifilter, InputStream *ist) +static int ifilter_bind_ist(InputFilter *ifilter, InputStream *ist, + const ViewSpecifier *vs) { InputFilterPriv *ifp = ifp_from_ifilter(ifilter); FilterGraphPriv *fgp = fgp_from_fg(ifilter->graph); - int ret, dec_idx; + SchedulerNode src; + int ret; av_assert0(!ifp->bound); ifp->bound = 1; @@ -698,13 +700,13 @@ static int ifilter_bind_ist(InputFilter *ifilter, InputStream *ist) if (!ifp->opts.fallback) return AVERROR(ENOMEM); - dec_idx = ist_filter_add(ist, ifilter, filtergraph_is_simple(ifilter->graph), - &ifp->opts); - if (dec_idx < 0) - return dec_idx; + ret = ist_filter_add(ist, ifilter, filtergraph_is_simple(ifilter->graph), + vs, &ifp->opts, &src); + if (ret < 0) + return ret; - ret = sch_connect(fgp->sch, SCH_DEC_OUT(dec_idx, 0), - SCH_FILTER_IN(fgp->sch_idx, ifp->index)); + ret = sch_connect(fgp->sch, + src, SCH_FILTER_IN(fgp->sch_idx, ifp->index)); if (ret < 0) return ret; @@ -729,10 +731,12 @@ static int ifilter_bind_ist(InputFilter *ifilter, InputStream *ist) return 0; } -static int ifilter_bind_dec(InputFilterPriv *ifp, Decoder *dec) +static int ifilter_bind_dec(InputFilterPriv *ifp, Decoder *dec, + const ViewSpecifier *vs) { FilterGraphPriv *fgp = fgp_from_fg(ifp->ifilter.graph); - int ret, dec_idx; + SchedulerNode src; + int ret; av_assert0(!ifp->bound); ifp->bound = 1; @@ -745,12 +749,11 @@ static int ifilter_bind_dec(InputFilterPriv *ifp, Decoder *dec) ifp->type_src = ifp->type; - dec_idx = dec_filter_add(dec, &ifp->ifilter, &ifp->opts); - if (dec_idx < 0) - return dec_idx; + ret = dec_filter_add(dec, &ifp->ifilter, &ifp->opts, vs, &src); + if (ret < 0) + return ret; - ret = sch_connect(fgp->sch, SCH_DEC_OUT(dec_idx, 0), - SCH_FILTER_IN(fgp->sch_idx, ifp->index)); + ret = sch_connect(fgp->sch, src, SCH_FILTER_IN(fgp->sch_idx, ifp->index)); if (ret < 0) return ret; @@ -1227,7 +1230,7 @@ int init_simple_filtergraph(InputStream *ist, OutputStream *ost, ost->filter = fg->outputs[0]; - ret = ifilter_bind_ist(fg->inputs[0], ist); + ret = ifilter_bind_ist(fg->inputs[0], ist, opts->vs); if (ret < 0) return ret; @@ -1251,28 +1254,38 @@ static int fg_complex_bind_input(FilterGraph *fg, InputFilter *ifilter) InputFilterPriv *ifp = ifp_from_ifilter(ifilter); InputStream *ist = NULL; enum AVMediaType type = ifp->type; + ViewSpecifier vs = { .type = VIEW_SPECIFIER_TYPE_NONE }; + const char *spec; + char *p; int i, ret; if (ifp->linklabel && !strncmp(ifp->linklabel, "dec:", 4)) { // bind to a standalone decoder int dec_idx; - dec_idx = strtol(ifp->linklabel + 4, NULL, 0); + dec_idx = strtol(ifp->linklabel + 4, &p, 0); if (dec_idx < 0 || dec_idx >= nb_decoders) { av_log(fg, AV_LOG_ERROR, "Invalid decoder index %d in filtergraph description %s\n", dec_idx, fgp->graph_desc); return AVERROR(EINVAL); } - ret = ifilter_bind_dec(ifp, decoders[dec_idx]); + if (type == AVMEDIA_TYPE_VIDEO) { + spec = *p == ':' ? p + 1 : p; + ret = view_specifier_parse(&spec, &vs); + if (ret < 0) + return ret; + } + + ret = ifilter_bind_dec(ifp, decoders[dec_idx], &vs); if (ret < 0) av_log(fg, AV_LOG_ERROR, "Error binding a decoder to filtergraph input %s\n", ifilter->name); return ret; } else if (ifp->linklabel) { + StreamSpecifier ss; AVFormatContext *s; AVStream *st = NULL; - char *p; int file_idx; // try finding an unbound filtergraph output with this label @@ -1309,17 +1322,33 @@ static int fg_complex_bind_input(FilterGraph *fg, InputFilter *ifilter) } s = input_files[file_idx]->ctx; + ret = stream_specifier_parse(&ss, *p == ':' ? p + 1 : p, 1, fg); + if (ret < 0) { + av_log(fg, AV_LOG_ERROR, "Invalid stream specifier: %s\n", p); + return ret; + } + + if (type == AVMEDIA_TYPE_VIDEO) { + spec = ss.remainder ? ss.remainder : ""; + ret = view_specifier_parse(&spec, &vs); + if (ret < 0) { + stream_specifier_uninit(&ss); + return ret; + } + } + for (i = 0; i < s->nb_streams; i++) { enum AVMediaType stream_type = s->streams[i]->codecpar->codec_type; if (stream_type != type && !(stream_type == AVMEDIA_TYPE_SUBTITLE && type == AVMEDIA_TYPE_VIDEO /* sub2video hack */)) continue; - if (check_stream_specifier(s, s->streams[i], *p == ':' ? p + 1 : p) == 1) { + if (stream_specifier_match(&ss, s, s->streams[i], fg)) { st = s->streams[i]; break; } } + stream_specifier_uninit(&ss); if (!st) { av_log(fg, AV_LOG_FATAL, "Stream specifier '%s' in filtergraph description %s " "matches no streams.\n", p, fgp->graph_desc); @@ -1344,7 +1373,7 @@ static int fg_complex_bind_input(FilterGraph *fg, InputFilter *ifilter) } av_assert0(ist); - ret = ifilter_bind_ist(ifilter, ist); + ret = ifilter_bind_ist(ifilter, ist, &vs); if (ret < 0) { av_log(fg, AV_LOG_ERROR, "Error binding an input stream to complex filtergraph input %s.\n", diff --git a/fftools/ffmpeg_mux_init.c b/fftools/ffmpeg_mux_init.c index 5ee2a9685b..c874ccb113 100644 --- a/fftools/ffmpeg_mux_init.c +++ b/fftools/ffmpeg_mux_init.c @@ -1025,7 +1025,7 @@ fail: } static int ost_add(Muxer *mux, const OptionsContext *o, enum AVMediaType type, - InputStream *ist, OutputFilter *ofilter, + InputStream *ist, OutputFilter *ofilter, const ViewSpecifier *vs, OutputStream **post) { AVFormatContext *oc = mux->fc; @@ -1399,6 +1399,7 @@ static int ost_add(Muxer *mux, const OptionsContext *o, enum AVMediaType type, .ch_layout = ost->enc_ctx->ch_layout, .sws_opts = o->g->sws_dict, .swr_opts = o->g->swr_opts, + .vs = vs, .output_tb = enc_tb, .trim_start_us = mux->of.start_time, .trim_duration_us = mux->of.recording_time, @@ -1546,7 +1547,7 @@ static int map_auto_video(Muxer *mux, const OptionsContext *o) } } if (best_ist) - return ost_add(mux, o, AVMEDIA_TYPE_VIDEO, best_ist, NULL, NULL); + return ost_add(mux, o, AVMEDIA_TYPE_VIDEO, best_ist, NULL, NULL, NULL); return 0; } @@ -1590,7 +1591,7 @@ static int map_auto_audio(Muxer *mux, const OptionsContext *o) } } if (best_ist) - return ost_add(mux, o, AVMEDIA_TYPE_AUDIO, best_ist, NULL, NULL); + return ost_add(mux, o, AVMEDIA_TYPE_AUDIO, best_ist, NULL, NULL, NULL); return 0; } @@ -1627,7 +1628,7 @@ static int map_auto_subtitle(Muxer *mux, const OptionsContext *o) input_descriptor && output_descriptor && (!input_descriptor->props || !output_descriptor->props)) { - return ost_add(mux, o, AVMEDIA_TYPE_SUBTITLE, ist, NULL, NULL); + return ost_add(mux, o, AVMEDIA_TYPE_SUBTITLE, ist, NULL, NULL, NULL); } } @@ -1648,7 +1649,7 @@ static int map_auto_data(Muxer *mux, const OptionsContext *o) continue; if (ist->st->codecpar->codec_type == AVMEDIA_TYPE_DATA && ist->st->codecpar->codec_id == codec_id) { - int ret = ost_add(mux, o, AVMEDIA_TYPE_DATA, ist, NULL, NULL); + int ret = ost_add(mux, o, AVMEDIA_TYPE_DATA, ist, NULL, NULL, NULL); if (ret < 0) return ret; } @@ -1690,10 +1691,13 @@ loop_end: av_log(mux, AV_LOG_VERBOSE, "Creating output stream from an explicitly " "mapped complex filtergraph %d, output [%s]\n", fg->index, map->linklabel); - ret = ost_add(mux, o, ofilter->type, NULL, ofilter, NULL); + ret = ost_add(mux, o, ofilter->type, NULL, ofilter, NULL, NULL); if (ret < 0) return ret; } else { + const ViewSpecifier *vs = map->vs.type == VIEW_SPECIFIER_TYPE_NONE ? + NULL : &map->vs; + ist = input_files[map->file_index]->streams[map->stream_index]; if (ist->user_set_discard == AVDISCARD_ALL) { av_log(mux, AV_LOG_FATAL, "Stream #%d:%d is disabled and cannot be mapped.\n", @@ -1724,7 +1728,14 @@ loop_end: return 0; } - ret = ost_add(mux, o, ist->st->codecpar->codec_type, ist, NULL, NULL); + if (vs && ist->st->codecpar->codec_type != AVMEDIA_TYPE_VIDEO) { + av_log(mux, AV_LOG_ERROR, + "View specifier given for mapping a %s input stream\n", + av_get_media_type_string(ist->st->codecpar->codec_type)); + return AVERROR(EINVAL); + } + + ret = ost_add(mux, o, ist->st->codecpar->codec_type, ist, NULL, vs, NULL); if (ret < 0) return ret; } @@ -1794,7 +1805,7 @@ read_fail: return AVERROR(ENOMEM); } - err = ost_add(mux, o, AVMEDIA_TYPE_ATTACHMENT, NULL, NULL, &ost); + err = ost_add(mux, o, AVMEDIA_TYPE_ATTACHMENT, NULL, NULL, NULL, &ost); if (err < 0) { av_free(attachment_filename); av_freep(&attachment); @@ -1849,7 +1860,7 @@ static int create_streams(Muxer *mux, const OptionsContext *o) av_get_media_type_string(ofilter->type)); av_log(mux, AV_LOG_VERBOSE, "\n"); - ret = ost_add(mux, o, ofilter->type, NULL, ofilter, NULL); + ret = ost_add(mux, o, ofilter->type, NULL, ofilter, NULL, NULL); if (ret < 0) return ret; } diff --git a/fftools/ffmpeg_opt.c b/fftools/ffmpeg_opt.c index 3cbf0795ac..972035e176 100644 --- a/fftools/ffmpeg_opt.c +++ b/fftools/ffmpeg_opt.c @@ -46,6 +46,7 @@ #include "libavutil/mem.h" #include "libavutil/opt.h" #include "libavutil/parseutils.h" +#include "libavutil/stereo3d.h" HWDevice *filter_hw_device; @@ -230,6 +231,59 @@ OPT_MATCH_PER_STREAM(int, int, OPT_TYPE_INT, i); OPT_MATCH_PER_STREAM(int64, int64_t, OPT_TYPE_INT64, i64); OPT_MATCH_PER_STREAM(dbl, double, OPT_TYPE_DOUBLE, dbl); +int view_specifier_parse(const char **pspec, ViewSpecifier *vs) +{ + const char *spec = *pspec; + char *endptr; + + vs->type = VIEW_SPECIFIER_TYPE_NONE; + + if (!strncmp(spec, "view:", 5)) { + spec += 5; + + if (!strncmp(spec, "all", 3)) { + spec += 3; + vs->type = VIEW_SPECIFIER_TYPE_ALL; + } else { + vs->type = VIEW_SPECIFIER_TYPE_ID; + vs->val = strtoul(spec, &endptr, 0); + if (endptr == spec) { + av_log(NULL, AV_LOG_ERROR, "Invalid view ID: %s\n", spec); + return AVERROR(EINVAL); + } + spec = endptr; + } + } else if (!strncmp(spec, "vidx:", 5)) { + spec += 5; + vs->type = VIEW_SPECIFIER_TYPE_IDX; + vs->val = strtoul(spec, &endptr, 0); + if (endptr == spec) { + av_log(NULL, AV_LOG_ERROR, "Invalid view index: %s\n", spec); + return AVERROR(EINVAL); + } + spec = endptr; + } else if (!strncmp(spec, "vpos:", 5)) { + spec += 5; + vs->type = VIEW_SPECIFIER_TYPE_POS; + + if (!strncmp(spec, "left", 4) && !cmdutils_isalnum(spec[4])) { + spec += 4; + vs->val = AV_STEREO3D_VIEW_LEFT; + } else if (!strncmp(spec, "right", 5) && !cmdutils_isalnum(spec[5])) { + spec += 5; + vs->val = AV_STEREO3D_VIEW_RIGHT; + } else { + av_log(NULL, AV_LOG_ERROR, "Invalid view position: %s\n", spec); + return AVERROR(EINVAL); + } + } else + return 0; + + *pspec = spec; + + return 0; +} + int parse_and_set_vsync(const char *arg, int *vsync_var, int file_idx, int st_idx, int is_global) { if (!av_strcasecmp(arg, "cfr")) *vsync_var = VSYNC_CFR; @@ -454,6 +508,7 @@ static int opt_map(void *optctx, const char *opt, const char *arg) goto fail; } } else { + ViewSpecifier vs; char *endptr; file_idx = strtol(arg, &endptr, 0); @@ -470,12 +525,18 @@ static int opt_map(void *optctx, const char *opt, const char *arg) goto fail; } - if (ss.remainder) { - if (!strcmp(ss.remainder, "?")) + arg = ss.remainder ? ss.remainder : ""; + + ret = view_specifier_parse(&arg, &vs); + if (ret < 0) + goto fail; + + if (*arg) { + if (!strcmp(arg, "?")) allow_unused = 1; else { - av_log(NULL, AV_LOG_ERROR, "Trailing garbage after stream specifier: %s\n", - ss.remainder); + av_log(NULL, AV_LOG_ERROR, + "Trailing garbage after stream specifier: %s\n", arg); ret = AVERROR(EINVAL); goto fail; } @@ -511,6 +572,7 @@ static int opt_map(void *optctx, const char *opt, const char *arg) m->file_index = file_idx; m->stream_index = i; + m->vs = vs; } } -- 2.43.0
On Tue, Aug 27, 2024 at 05:05:18PM +0200, Anton Khirnov wrote:
This extends the syntax for specifying input streams in -map and complex filtergraph labels, to allow selecting a view by view ID, index, or position. The corresponding decoder is then set up to decode the appropriate view and send frames for that view to the correct filtergraph input(s). --- doc/ffmpeg.texi | 30 +++- fftools/cmdutils.c | 2 +- fftools/cmdutils.h | 2 + fftools/ffmpeg.h | 45 ++++- fftools/ffmpeg_dec.c | 355 +++++++++++++++++++++++++++++++++++++- fftools/ffmpeg_demux.c | 24 ++- fftools/ffmpeg_filter.c | 71 +++++--- fftools/ffmpeg_mux_init.c | 29 +++- fftools/ffmpeg_opt.c | 70 +++++++- 9 files changed, 575 insertions(+), 53 deletions(-)
breaks build on mingw64 src/fftools/ffmpeg_dec.c: In function ‘packet_decode’: src/fftools/ffmpeg_dec.c:811:19: error: implicit declaration of function ‘ffs’ [-Werror=implicit-function-declaration] 811 | pos = ffs(outputs_mask) - 1; | ^~~ cc1: some warnings being treated as errors make: *** [src/ffbuild/common.mak:81: fftools/ffmpeg_dec.o] Error 1 make: *** Waiting for unfinished jobs.... thx [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB Democracy is the form of government in which you can choose your dictator
On 8/29/2024 8:26 PM, Michael Niedermayer wrote:
On Tue, Aug 27, 2024 at 05:05:18PM +0200, Anton Khirnov wrote:
This extends the syntax for specifying input streams in -map and complex filtergraph labels, to allow selecting a view by view ID, index, or position. The corresponding decoder is then set up to decode the appropriate view and send frames for that view to the correct filtergraph input(s). --- doc/ffmpeg.texi | 30 +++- fftools/cmdutils.c | 2 +- fftools/cmdutils.h | 2 + fftools/ffmpeg.h | 45 ++++- fftools/ffmpeg_dec.c | 355 +++++++++++++++++++++++++++++++++++++- fftools/ffmpeg_demux.c | 24 ++- fftools/ffmpeg_filter.c | 71 +++++--- fftools/ffmpeg_mux_init.c | 29 +++- fftools/ffmpeg_opt.c | 70 +++++++- 9 files changed, 575 insertions(+), 53 deletions(-)
breaks build on mingw64
src/fftools/ffmpeg_dec.c: In function ‘packet_decode’: src/fftools/ffmpeg_dec.c:811:19: error: implicit declaration of function ‘ffs’ [-Werror=implicit-function-declaration] 811 | pos = ffs(outputs_mask) - 1; | ^~~ cc1: some warnings being treated as errors make: *** [src/ffbuild/common.mak:81: fftools/ffmpeg_dec.o] Error 1 make: *** Waiting for unfinished jobs....
thx
Would ff_clz(outputs_mask) work here?
Quoting James Almer (2024-08-30 02:40:49)
On 8/29/2024 8:26 PM, Michael Niedermayer wrote:
On Tue, Aug 27, 2024 at 05:05:18PM +0200, Anton Khirnov wrote:
This extends the syntax for specifying input streams in -map and complex filtergraph labels, to allow selecting a view by view ID, index, or position. The corresponding decoder is then set up to decode the appropriate view and send frames for that view to the correct filtergraph input(s). --- doc/ffmpeg.texi | 30 +++- fftools/cmdutils.c | 2 +- fftools/cmdutils.h | 2 + fftools/ffmpeg.h | 45 ++++- fftools/ffmpeg_dec.c | 355 +++++++++++++++++++++++++++++++++++++- fftools/ffmpeg_demux.c | 24 ++- fftools/ffmpeg_filter.c | 71 +++++--- fftools/ffmpeg_mux_init.c | 29 +++- fftools/ffmpeg_opt.c | 70 +++++++- 9 files changed, 575 insertions(+), 53 deletions(-)
breaks build on mingw64
src/fftools/ffmpeg_dec.c: In function ‘packet_decode’: src/fftools/ffmpeg_dec.c:811:19: error: implicit declaration of function ‘ffs’ [-Werror=implicit-function-declaration] 811 | pos = ffs(outputs_mask) - 1; | ^~~ cc1: some warnings being treated as errors make: *** [src/ffbuild/common.mak:81: fftools/ffmpeg_dec.o] Error 1 make: *** Waiting for unfinished jobs....
thx
Would ff_clz(outputs_mask) work here?
In principle yes, but it's private and I'd rather not include a private header in ffmpeg CLI. We could either make it public, or I could add a ffs() implementation for Windows to compat/. Anyone has a preference? -- Anton Khirnov
On 8/31/2024 4:44 AM, Anton Khirnov wrote:
Quoting James Almer (2024-08-30 02:40:49)
On 8/29/2024 8:26 PM, Michael Niedermayer wrote:
On Tue, Aug 27, 2024 at 05:05:18PM +0200, Anton Khirnov wrote:
This extends the syntax for specifying input streams in -map and complex filtergraph labels, to allow selecting a view by view ID, index, or position. The corresponding decoder is then set up to decode the appropriate view and send frames for that view to the correct filtergraph input(s). --- doc/ffmpeg.texi | 30 +++- fftools/cmdutils.c | 2 +- fftools/cmdutils.h | 2 + fftools/ffmpeg.h | 45 ++++- fftools/ffmpeg_dec.c | 355 +++++++++++++++++++++++++++++++++++++- fftools/ffmpeg_demux.c | 24 ++- fftools/ffmpeg_filter.c | 71 +++++--- fftools/ffmpeg_mux_init.c | 29 +++- fftools/ffmpeg_opt.c | 70 +++++++- 9 files changed, 575 insertions(+), 53 deletions(-)
breaks build on mingw64
src/fftools/ffmpeg_dec.c: In function ‘packet_decode’: src/fftools/ffmpeg_dec.c:811:19: error: implicit declaration of function ‘ffs’ [-Werror=implicit-function-declaration] 811 | pos = ffs(outputs_mask) - 1; | ^~~ cc1: some warnings being treated as errors make: *** [src/ffbuild/common.mak:81: fftools/ffmpeg_dec.o] Error 1 make: *** Waiting for unfinished jobs....
thx
Would ff_clz(outputs_mask) work here?
In principle yes, but it's private and I'd rather not include a private header in ffmpeg CLI.
We could either make it public, or I could add a ffs() implementation for Windows to compat/. Anyone has a preference?
I think i prefer making ff_clz public in common.h. It has an optimized version not exclusive to Windows using a gcc/clang builtin, and even one specific for riscv.
Quoting James Almer (2024-08-31 15:56:24)
On 8/31/2024 4:44 AM, Anton Khirnov wrote:
Quoting James Almer (2024-08-30 02:40:49)
On 8/29/2024 8:26 PM, Michael Niedermayer wrote:
On Tue, Aug 27, 2024 at 05:05:18PM +0200, Anton Khirnov wrote:
This extends the syntax for specifying input streams in -map and complex filtergraph labels, to allow selecting a view by view ID, index, or position. The corresponding decoder is then set up to decode the appropriate view and send frames for that view to the correct filtergraph input(s). --- doc/ffmpeg.texi | 30 +++- fftools/cmdutils.c | 2 +- fftools/cmdutils.h | 2 + fftools/ffmpeg.h | 45 ++++- fftools/ffmpeg_dec.c | 355 +++++++++++++++++++++++++++++++++++++- fftools/ffmpeg_demux.c | 24 ++- fftools/ffmpeg_filter.c | 71 +++++--- fftools/ffmpeg_mux_init.c | 29 +++- fftools/ffmpeg_opt.c | 70 +++++++- 9 files changed, 575 insertions(+), 53 deletions(-)
breaks build on mingw64
src/fftools/ffmpeg_dec.c: In function ‘packet_decode’: src/fftools/ffmpeg_dec.c:811:19: error: implicit declaration of function ‘ffs’ [-Werror=implicit-function-declaration] 811 | pos = ffs(outputs_mask) - 1; | ^~~ cc1: some warnings being treated as errors make: *** [src/ffbuild/common.mak:81: fftools/ffmpeg_dec.o] Error 1 make: *** Waiting for unfinished jobs....
thx
Would ff_clz(outputs_mask) work here?
In principle yes, but it's private and I'd rather not include a private header in ffmpeg CLI.
We could either make it public, or I could add a ffs() implementation for Windows to compat/. Anyone has a preference?
I think i prefer making ff_clz public in common.h.
common.h is the wrong place for anything
It has an optimized version not exclusive to Windows using a gcc/clang builtin, and even one specific for riscv.
Which would not be visible to ffmpeg CLI anyway. -- Anton Khirnov
On 9/3/2024 6:47 AM, Anton Khirnov wrote:
Quoting James Almer (2024-08-31 15:56:24)
On 8/31/2024 4:44 AM, Anton Khirnov wrote:
Quoting James Almer (2024-08-30 02:40:49)
On 8/29/2024 8:26 PM, Michael Niedermayer wrote:
On Tue, Aug 27, 2024 at 05:05:18PM +0200, Anton Khirnov wrote:
This extends the syntax for specifying input streams in -map and complex filtergraph labels, to allow selecting a view by view ID, index, or position. The corresponding decoder is then set up to decode the appropriate view and send frames for that view to the correct filtergraph input(s). --- doc/ffmpeg.texi | 30 +++- fftools/cmdutils.c | 2 +- fftools/cmdutils.h | 2 + fftools/ffmpeg.h | 45 ++++- fftools/ffmpeg_dec.c | 355 +++++++++++++++++++++++++++++++++++++- fftools/ffmpeg_demux.c | 24 ++- fftools/ffmpeg_filter.c | 71 +++++--- fftools/ffmpeg_mux_init.c | 29 +++- fftools/ffmpeg_opt.c | 70 +++++++- 9 files changed, 575 insertions(+), 53 deletions(-)
breaks build on mingw64
src/fftools/ffmpeg_dec.c: In function ‘packet_decode’: src/fftools/ffmpeg_dec.c:811:19: error: implicit declaration of function ‘ffs’ [-Werror=implicit-function-declaration] 811 | pos = ffs(outputs_mask) - 1; | ^~~ cc1: some warnings being treated as errors make: *** [src/ffbuild/common.mak:81: fftools/ffmpeg_dec.o] Error 1 make: *** Waiting for unfinished jobs....
thx
Would ff_clz(outputs_mask) work here?
In principle yes, but it's private and I'd rather not include a private header in ffmpeg CLI.
We could either make it public, or I could add a ffs() implementation for Windows to compat/. Anyone has a preference?
Also, ffs() is posix. Is Windows the only target that lacks it?
I think i prefer making ff_clz public in common.h.
common.h is the wrong place for anything
It has an optimized version not exclusive to Windows using a gcc/clang builtin, and even one specific for riscv.
Which would not be visible to ffmpeg CLI anyway.
True. We could make it aware of it by setting HAVE_AV_CONFIG_H during compilation, but that's kinda dirty.
Quoting James Almer (2024-09-03 14:26:21)
On 9/3/2024 6:47 AM, Anton Khirnov wrote:
Quoting James Almer (2024-08-31 15:56:24)
On 8/31/2024 4:44 AM, Anton Khirnov wrote:
Quoting James Almer (2024-08-30 02:40:49)
On 8/29/2024 8:26 PM, Michael Niedermayer wrote:
On Tue, Aug 27, 2024 at 05:05:18PM +0200, Anton Khirnov wrote: > This extends the syntax for specifying input streams in -map and complex > filtergraph labels, to allow selecting a view by view ID, index, or > position. The corresponding decoder is then set up to decode the > appropriate view and send frames for that view to the correct > filtergraph input(s). > --- > doc/ffmpeg.texi | 30 +++- > fftools/cmdutils.c | 2 +- > fftools/cmdutils.h | 2 + > fftools/ffmpeg.h | 45 ++++- > fftools/ffmpeg_dec.c | 355 +++++++++++++++++++++++++++++++++++++- > fftools/ffmpeg_demux.c | 24 ++- > fftools/ffmpeg_filter.c | 71 +++++--- > fftools/ffmpeg_mux_init.c | 29 +++- > fftools/ffmpeg_opt.c | 70 +++++++- > 9 files changed, 575 insertions(+), 53 deletions(-)
breaks build on mingw64
src/fftools/ffmpeg_dec.c: In function ‘packet_decode’: src/fftools/ffmpeg_dec.c:811:19: error: implicit declaration of function ‘ffs’ [-Werror=implicit-function-declaration] 811 | pos = ffs(outputs_mask) - 1; | ^~~ cc1: some warnings being treated as errors make: *** [src/ffbuild/common.mak:81: fftools/ffmpeg_dec.o] Error 1 make: *** Waiting for unfinished jobs....
thx
Would ff_clz(outputs_mask) work here?
In principle yes, but it's private and I'd rather not include a private header in ffmpeg CLI.
We could either make it public, or I could add a ffs() implementation for Windows to compat/. Anyone has a preference?
Also, ffs() is posix. Is Windows the only target that lacks it?
I think so, though I didn't test all that extensively.
I think i prefer making ff_clz public in common.h.
common.h is the wrong place for anything
It has an optimized version not exclusive to Windows using a gcc/clang builtin, and even one specific for riscv.
Which would not be visible to ffmpeg CLI anyway.
True. We could make it aware of it by setting HAVE_AV_CONFIG_H during compilation, but that's kinda dirty.
Yuck. If we wanted to be fancy, we could add stdc_first_trailing_zero() from C23. Possibly write/steal compatibility wrappers like we did for atomics. -- Anton Khirnov
On 9/3/2024 9:44 AM, Anton Khirnov wrote:
Quoting James Almer (2024-09-03 14:26:21)
On 9/3/2024 6:47 AM, Anton Khirnov wrote:
Quoting James Almer (2024-08-31 15:56:24)
On 8/31/2024 4:44 AM, Anton Khirnov wrote:
Quoting James Almer (2024-08-30 02:40:49)
On 8/29/2024 8:26 PM, Michael Niedermayer wrote: > On Tue, Aug 27, 2024 at 05:05:18PM +0200, Anton Khirnov wrote: >> This extends the syntax for specifying input streams in -map and complex >> filtergraph labels, to allow selecting a view by view ID, index, or >> position. The corresponding decoder is then set up to decode the >> appropriate view and send frames for that view to the correct >> filtergraph input(s). >> --- >> doc/ffmpeg.texi | 30 +++- >> fftools/cmdutils.c | 2 +- >> fftools/cmdutils.h | 2 + >> fftools/ffmpeg.h | 45 ++++- >> fftools/ffmpeg_dec.c | 355 +++++++++++++++++++++++++++++++++++++- >> fftools/ffmpeg_demux.c | 24 ++- >> fftools/ffmpeg_filter.c | 71 +++++--- >> fftools/ffmpeg_mux_init.c | 29 +++- >> fftools/ffmpeg_opt.c | 70 +++++++- >> 9 files changed, 575 insertions(+), 53 deletions(-) > > breaks build on mingw64 > > src/fftools/ffmpeg_dec.c: In function ‘packet_decode’: > src/fftools/ffmpeg_dec.c:811:19: error: implicit declaration of function ‘ffs’ [-Werror=implicit-function-declaration] > 811 | pos = ffs(outputs_mask) - 1; > | ^~~ > cc1: some warnings being treated as errors > make: *** [src/ffbuild/common.mak:81: fftools/ffmpeg_dec.o] Error 1 > make: *** Waiting for unfinished jobs.... > > thx
Would ff_clz(outputs_mask) work here?
In principle yes, but it's private and I'd rather not include a private header in ffmpeg CLI.
We could either make it public, or I could add a ffs() implementation for Windows to compat/. Anyone has a preference?
Also, ffs() is posix. Is Windows the only target that lacks it?
I think so, though I didn't test all that extensively.
I think i prefer making ff_clz public in common.h.
common.h is the wrong place for anything
It has an optimized version not exclusive to Windows using a gcc/clang builtin, and even one specific for riscv.
Which would not be visible to ffmpeg CLI anyway.
True. We could make it aware of it by setting HAVE_AV_CONFIG_H during compilation, but that's kinda dirty.
Yuck.
If we wanted to be fancy, we could add stdc_first_trailing_zero() from C23. Possibly write/steal compatibility wrappers like we did for atomics.
I prefer this over ffs() in compat/windows, yes. Good idea.
Restore vertical alignment to OutputFilterOptions initializer. The flags field is kept as-is, since the corresponding lines are already long, aligning them would make them even longer, and breaking them would look worse. --- fftools/ffmpeg_mux_init.c | 31 ++++++++++++++++--------------- 1 file changed, 16 insertions(+), 15 deletions(-) diff --git a/fftools/ffmpeg_mux_init.c b/fftools/ffmpeg_mux_init.c index c874ccb113..c054765be7 100644 --- a/fftools/ffmpeg_mux_init.c +++ b/fftools/ffmpeg_mux_init.c @@ -1388,23 +1388,24 @@ static int ost_add(Muxer *mux, const OptionsContext *o, enum AVMediaType type, (type == AVMEDIA_TYPE_VIDEO || type == AVMEDIA_TYPE_AUDIO)) { char name[16]; OutputFilterOptions opts = { - .enc = enc, - .name = name, - .format = (type == AVMEDIA_TYPE_VIDEO) ? - ost->enc_ctx->pix_fmt : ost->enc_ctx->sample_fmt, - .width = ost->enc_ctx->width, - .height = ost->enc_ctx->height, - .vsync_method = vsync_method, - .sample_rate = ost->enc_ctx->sample_rate, - .ch_layout = ost->enc_ctx->ch_layout, - .sws_opts = o->g->sws_dict, - .swr_opts = o->g->swr_opts, - .vs = vs, - .output_tb = enc_tb, + .enc = enc, + .name = name, + .format = (type == AVMEDIA_TYPE_VIDEO) ? + ost->enc_ctx->pix_fmt : ost->enc_ctx->sample_fmt, + .width = ost->enc_ctx->width, + .height = ost->enc_ctx->height, + .vsync_method = vsync_method, + .sample_rate = ost->enc_ctx->sample_rate, + .ch_layout = ost->enc_ctx->ch_layout, + .sws_opts = o->g->sws_dict, + .swr_opts = o->g->swr_opts, + .vs = vs, + .output_tb = enc_tb, .trim_start_us = mux->of.start_time, .trim_duration_us = mux->of.recording_time, - .ts_offset = mux->of.start_time == AV_NOPTS_VALUE ? - 0 : mux->of.start_time, + .ts_offset = mux->of.start_time == AV_NOPTS_VALUE ? + 0 : mux->of.start_time, + .flags = OFILTER_FLAG_DISABLE_CONVERT * !!keep_pix_fmt | OFILTER_FLAG_AUTOSCALE * !!autoscale | OFILTER_FLAG_AUDIO_24BIT * !!(av_get_exact_bits_per_sample(ost->enc_ctx->codec_id) == 24), -- 2.43.0
Only those that can be decoded with our implementation, so excluding * C and D - independent layers * G, H, I - more than 2 layers Frame hashes verified against the reference implementation from https://hevc.hhi.fraunhofer.de/svn/svn_3DVCSoftware/ --- tests/fate/hevc.mak | 13 +- tests/ref/fate/hevc-conformance-MVHEVCS_A | 106 +++++++++++++++++ tests/ref/fate/hevc-conformance-MVHEVCS_B | 138 ++++++++++++++++++++++ tests/ref/fate/hevc-conformance-MVHEVCS_E | 106 +++++++++++++++++ tests/ref/fate/hevc-conformance-MVHEVCS_F | 106 +++++++++++++++++ 5 files changed, 468 insertions(+), 1 deletion(-) create mode 100644 tests/ref/fate/hevc-conformance-MVHEVCS_A create mode 100644 tests/ref/fate/hevc-conformance-MVHEVCS_B create mode 100644 tests/ref/fate/hevc-conformance-MVHEVCS_E create mode 100644 tests/ref/fate/hevc-conformance-MVHEVCS_F diff --git a/tests/fate/hevc.mak b/tests/fate/hevc.mak index 720603c112..e7dad6c898 100644 --- a/tests/fate/hevc.mak +++ b/tests/fate/hevc.mak @@ -178,7 +178,14 @@ HEVC_SAMPLES_444_12BIT = \ PERSIST_RPARAM_A_RExt_Sony_3\ $(if $(CONFIG_LARGE_TESTS), $(HEVC_SAMPLES_444_12BIT_LARGE)) -FATE_HEVC_VARS := 8BIT 10BIT 422_10BIT 422_10BIN 444_8BIT 444_12BIT +HEVC_SAMPLES_MULTIVIEW = \ + MVHEVCS_A \ + MVHEVCS_B \ + MVHEVCS_E \ + MVHEVCS_F \ + + +FATE_HEVC_VARS := 8BIT 10BIT 422_10BIT 422_10BIN 444_8BIT 444_12BIT MULTIVIEW $(foreach VAR,$(FATE_HEVC_VARS), $(eval HEVC_TESTS_$(VAR) := $(addprefix fate-hevc-conformance-, $(HEVC_SAMPLES_$(VAR))))) # equivalent bitstreams @@ -202,6 +209,8 @@ $(HEVC_TESTS_422_10BIT) $(HEVC_TESTS_422_10BIN): SCALE_OPTS := -pix_fmt yuv422p1 $(HEVC_TESTS_444_12BIT): SCALE_OPTS := -pix_fmt yuv444p12le -vf scale fate-hevc-conformance-%: CMD = framecrc -i $(TARGET_SAMPLES)/hevc-conformance/$(subst fate-hevc-conformance-,,$(@)).bit $(SCALE_OPTS) $(HEVC_TESTS_422_10BIN): CMD = framecrc -i $(TARGET_SAMPLES)/hevc-conformance/$(subst fate-hevc-conformance-,,$(@)).bin $(SCALE_OPTS) +$(HEVC_TESTS_MULTIVIEW): CMD = framecrc -i $(TARGET_SAMPLES)/hevc-conformance/$(subst fate-hevc-conformance-,,$(@)).bit \ + -pix_fmt yuv420p -map "0:view:0" -map "0:view:1" -vf setpts=N FATE_HEVC-$(call FRAMECRC, HEVC, HEVC, HEVC_PARSER) += $(HEVC_TESTS_8BIT) $(HEVC_TESTS_444_8BIT) FATE_HEVC-$(call FRAMECRC, HEVC, HEVC, HEVC_PARSER SCALE_FILTER) += \ @@ -210,6 +219,8 @@ FATE_HEVC-$(call FRAMECRC, HEVC, HEVC, HEVC_PARSER SCALE_FILTER) += \ $(HEVC_TESTS_422_10BIN) \ $(HEVC_TESTS_444_12BIT) \ +FATE_HEVC-$(call FRAMECRC, HEVC, HEVC, HEVC_PARSER SCALE_FILTER) += $(HEVC_TESTS_MULTIVIEW) + fate-hevc-paramchange-yuv420p-yuv420p10: CMD = framecrc -i $(TARGET_SAMPLES)/hevc/paramchange_yuv420p_yuv420p10.hevc -fps_mode passthrough -sws_flags area+accurate_rnd+bitexact FATE_HEVC-$(call FRAMECRC, HEVC, HEVC, HEVC_PARSER SCALE_FILTER LARGE_TESTS) += fate-hevc-paramchange-yuv420p-yuv420p10 diff --git a/tests/ref/fate/hevc-conformance-MVHEVCS_A b/tests/ref/fate/hevc-conformance-MVHEVCS_A new file mode 100644 index 0000000000..c919889207 --- /dev/null +++ b/tests/ref/fate/hevc-conformance-MVHEVCS_A @@ -0,0 +1,106 @@ +#tb 0: 1/1200000 +#media_type 0: video +#codec_id 0: rawvideo +#dimensions 0: 1024x768 +#sar 0: 0/1 +#tb 1: 1/1200000 +#media_type 1: video +#codec_id 1: rawvideo +#dimensions 1: 1024x768 +#sar 1: 0/1 +0, 0, 0, 0, 1179648, 0x4a15620e +1, 0, 0, 0, 1179648, 0x5b752453 +0, 1, 1, 0, 1179648, 0xb33277fe +1, 1, 1, 0, 1179648, 0xacf4376d +0, 2, 2, 0, 1179648, 0x97b969bd +1, 2, 2, 0, 1179648, 0xa9d6e75d +0, 3, 3, 0, 1179648, 0x3c6b7b81 +1, 3, 3, 0, 1179648, 0x2142cc59 +0, 4, 4, 0, 1179648, 0xdf83fe63 +1, 4, 4, 0, 1179648, 0x838d4f0a +0, 5, 5, 0, 1179648, 0x93fc3bdf +1, 5, 5, 0, 1179648, 0x29bc71c2 +0, 6, 6, 0, 1179648, 0x18f90aa3 +1, 6, 6, 0, 1179648, 0x3b25ee18 +0, 7, 7, 0, 1179648, 0x84c730c5 +1, 7, 7, 0, 1179648, 0x60ecae5b +0, 8, 8, 0, 1179648, 0x7b49f2b1 +1, 8, 8, 0, 1179648, 0x8c3c635e +0, 9, 9, 0, 1179648, 0x71f1b862 +1, 9, 9, 0, 1179648, 0x16c2a6fc +0, 10, 10, 0, 1179648, 0x92be991a +1, 10, 10, 0, 1179648, 0x092d6378 +0, 11, 11, 0, 1179648, 0xa948322d +1, 11, 11, 0, 1179648, 0x5d977eec +0, 12, 12, 0, 1179648, 0x54838c22 +1, 12, 12, 0, 1179648, 0x66fc891e +0, 13, 13, 0, 1179648, 0xc2682668 +1, 13, 13, 0, 1179648, 0x0b60bd06 +0, 14, 14, 0, 1179648, 0x1fa22020 +1, 14, 14, 0, 1179648, 0x201706e2 +0, 15, 15, 0, 1179648, 0x3a686683 +1, 15, 15, 0, 1179648, 0x9c59849c +0, 16, 16, 0, 1179648, 0x991431ab +1, 16, 16, 0, 1179648, 0xb495ba09 +0, 17, 17, 0, 1179648, 0xd394e067 +1, 17, 17, 0, 1179648, 0x1d1db0c6 +0, 18, 18, 0, 1179648, 0xbe694632 +1, 18, 18, 0, 1179648, 0x22beec67 +0, 19, 19, 0, 1179648, 0x8a48c7a7 +1, 19, 19, 0, 1179648, 0x3515ac62 +0, 20, 20, 0, 1179648, 0x8d7e9e6d +1, 20, 20, 0, 1179648, 0x3b73015b +0, 21, 21, 0, 1179648, 0x14f28d47 +1, 21, 21, 0, 1179648, 0x4e01fdb7 +0, 22, 22, 0, 1179648, 0x1885631b +1, 22, 22, 0, 1179648, 0xca33f8de +0, 23, 23, 0, 1179648, 0x29c795f1 +1, 23, 23, 0, 1179648, 0x925517c1 +0, 24, 24, 0, 1179648, 0xee23fd91 +1, 24, 24, 0, 1179648, 0x0894bf6a +0, 25, 25, 0, 1179648, 0xa57dce94 +1, 25, 25, 0, 1179648, 0xebfe296d +0, 26, 26, 0, 1179648, 0x6c3b16d8 +1, 26, 26, 0, 1179648, 0x62acb3e0 +0, 27, 27, 0, 1179648, 0xbf5a112f +1, 27, 27, 0, 1179648, 0x72f6ae90 +0, 28, 28, 0, 1179648, 0x5e7ce2a7 +1, 28, 28, 0, 1179648, 0xb4c6bcbc +0, 29, 29, 0, 1179648, 0x0529ad59 +1, 29, 29, 0, 1179648, 0x438c8160 +0, 30, 30, 0, 1179648, 0xc94d561d +1, 30, 30, 0, 1179648, 0x1b3a02af +0, 31, 31, 0, 1179648, 0x7079bbc0 +1, 31, 31, 0, 1179648, 0x6034d275 +0, 32, 32, 0, 1179648, 0x6c8e1265 +1, 32, 32, 0, 1179648, 0x9e688ee2 +0, 33, 33, 0, 1179648, 0x5e1494ee +1, 33, 33, 0, 1179648, 0xe39cab5d +0, 34, 34, 0, 1179648, 0xe3654994 +1, 34, 34, 0, 1179648, 0xeefe533b +0, 35, 35, 0, 1179648, 0x329510fa +1, 35, 35, 0, 1179648, 0x48ed2162 +0, 36, 36, 0, 1179648, 0x92120a9f +1, 36, 36, 0, 1179648, 0xb0fb26b3 +0, 37, 37, 0, 1179648, 0x6f279a56 +1, 37, 37, 0, 1179648, 0xc4185523 +0, 38, 38, 0, 1179648, 0xd2ad5c1d +1, 38, 38, 0, 1179648, 0x8124687d +0, 39, 39, 0, 1179648, 0xd7e5bbaa +1, 39, 39, 0, 1179648, 0x39f1b702 +0, 40, 40, 0, 1179648, 0xb5628743 +1, 40, 40, 0, 1179648, 0xee5802d7 +0, 41, 41, 0, 1179648, 0x2760515e +1, 41, 41, 0, 1179648, 0xdeac4fbd +0, 42, 42, 0, 1179648, 0x11a2f57a +1, 42, 42, 0, 1179648, 0xb1c7977f +0, 43, 43, 0, 1179648, 0x2c47dd8f +1, 43, 43, 0, 1179648, 0xaaaddace +0, 44, 44, 0, 1179648, 0x890fe8c2 +1, 44, 44, 0, 1179648, 0x7ff8e81e +0, 45, 45, 0, 1179648, 0x65e9f9ac +1, 45, 45, 0, 1179648, 0xf9b1b99c +0, 46, 46, 0, 1179648, 0x3e4fa8d9 +1, 46, 46, 0, 1179648, 0xfc3659e6 +0, 47, 47, 0, 1179648, 0x356e13ee +1, 47, 47, 0, 1179648, 0xff8bfa3f diff --git a/tests/ref/fate/hevc-conformance-MVHEVCS_B b/tests/ref/fate/hevc-conformance-MVHEVCS_B new file mode 100644 index 0000000000..07155c50e1 --- /dev/null +++ b/tests/ref/fate/hevc-conformance-MVHEVCS_B @@ -0,0 +1,138 @@ +#tb 0: 1/1200000 +#media_type 0: video +#codec_id 0: rawvideo +#dimensions 0: 1024x768 +#sar 0: 0/1 +#tb 1: 1/1200000 +#media_type 1: video +#codec_id 1: rawvideo +#dimensions 1: 1024x768 +#sar 1: 0/1 +0, 0, 0, 0, 1179648, 0x4b39cdfe +1, 0, 0, 0, 1179648, 0x1afcd2a3 +0, 1, 1, 0, 1179648, 0xef0da64e +1, 1, 1, 0, 1179648, 0x2bb1d07b +0, 2, 2, 0, 1179648, 0x9dd9e5a7 +1, 2, 2, 0, 1179648, 0xcc0547f2 +0, 3, 3, 0, 1179648, 0xee948285 +1, 3, 3, 0, 1179648, 0x3375f14b +0, 4, 4, 0, 1179648, 0x75117927 +1, 4, 4, 0, 1179648, 0x1e059655 +0, 5, 5, 0, 1179648, 0xad9fec31 +1, 5, 5, 0, 1179648, 0xfed7ef37 +0, 6, 6, 0, 1179648, 0x53cd0f53 +1, 6, 6, 0, 1179648, 0xa725f316 +0, 7, 7, 0, 1179648, 0xcbda04b5 +1, 7, 7, 0, 1179648, 0xd13b99db +0, 8, 8, 0, 1179648, 0x71861302 +1, 8, 8, 0, 1179648, 0x911d2c79 +0, 9, 9, 0, 1179648, 0x5d2564dd +1, 9, 9, 0, 1179648, 0x70fa087a +0, 10, 10, 0, 1179648, 0xcfa6d394 +1, 10, 10, 0, 1179648, 0x69a27b84 +0, 11, 11, 0, 1179648, 0x18fe7393 +1, 11, 11, 0, 1179648, 0xcc665fd1 +0, 12, 12, 0, 1179648, 0xb329d489 +1, 12, 12, 0, 1179648, 0xf8d93bbf +0, 13, 13, 0, 1179648, 0x3b029004 +1, 13, 13, 0, 1179648, 0x39655020 +0, 14, 14, 0, 1179648, 0x6c81bf8e +1, 14, 14, 0, 1179648, 0x13da6f67 +0, 15, 15, 0, 1179648, 0x25e3e0a9 +1, 15, 15, 0, 1179648, 0xdc372a0f +0, 16, 16, 0, 1179648, 0x8f2c789f +1, 16, 16, 0, 1179648, 0xba8a3e23 +0, 17, 17, 0, 1179648, 0x372b866f +1, 17, 17, 0, 1179648, 0x27d31b48 +0, 18, 18, 0, 1179648, 0x5cfc8119 +1, 18, 18, 0, 1179648, 0x50aca559 +0, 19, 19, 0, 1179648, 0x593af76c +1, 19, 19, 0, 1179648, 0x27c0e522 +0, 20, 20, 0, 1179648, 0x852864b7 +1, 20, 20, 0, 1179648, 0xa8739c77 +0, 21, 21, 0, 1179648, 0x71f4b961 +1, 21, 21, 0, 1179648, 0x27c16037 +0, 22, 22, 0, 1179648, 0x6c5ebb84 +1, 22, 22, 0, 1179648, 0x74a6ede9 +0, 23, 23, 0, 1179648, 0xb5467da9 +1, 23, 23, 0, 1179648, 0x1ea455b9 +0, 24, 24, 0, 1179648, 0x92d58478 +1, 24, 24, 0, 1179648, 0x9e464f3a +0, 25, 25, 0, 1179648, 0xa6181655 +1, 25, 25, 0, 1179648, 0x8af4fd5b +0, 26, 26, 0, 1179648, 0x15e9ee9a +1, 26, 26, 0, 1179648, 0xa6ed4580 +0, 27, 27, 0, 1179648, 0x4782abf9 +1, 27, 27, 0, 1179648, 0x73fc62e6 +0, 28, 28, 0, 1179648, 0x8c06f337 +1, 28, 28, 0, 1179648, 0xb2d3dfbf +0, 29, 29, 0, 1179648, 0x2e6b1a88 +1, 29, 29, 0, 1179648, 0x2066f08e +0, 30, 30, 0, 1179648, 0x584eac3e +1, 30, 30, 0, 1179648, 0xa0bfb75b +0, 31, 31, 0, 1179648, 0x9a2dae49 +1, 31, 31, 0, 1179648, 0x89445161 +0, 32, 32, 0, 1179648, 0x3a50c6f4 +1, 32, 32, 0, 1179648, 0x9befaa17 +0, 33, 33, 0, 1179648, 0x1afcbfd6 +1, 33, 33, 0, 1179648, 0xb243de65 +0, 34, 34, 0, 1179648, 0xfdda4e8c +1, 34, 34, 0, 1179648, 0xf945451a +0, 35, 35, 0, 1179648, 0x8fffa546 +1, 35, 35, 0, 1179648, 0x06b20109 +0, 36, 36, 0, 1179648, 0xbc2c9dfc +1, 36, 36, 0, 1179648, 0x344c1ed8 +0, 37, 37, 0, 1179648, 0xb2d08414 +1, 37, 37, 0, 1179648, 0xccd35195 +0, 38, 38, 0, 1179648, 0x54886d31 +1, 38, 38, 0, 1179648, 0x653ed9df +0, 39, 39, 0, 1179648, 0x07c3b249 +1, 39, 39, 0, 1179648, 0xf92b9e88 +0, 40, 40, 0, 1179648, 0x073b9491 +1, 40, 40, 0, 1179648, 0x4ed7ee12 +0, 41, 41, 0, 1179648, 0xcd669799 +1, 41, 41, 0, 1179648, 0xa7faed20 +0, 42, 42, 0, 1179648, 0xd21b0a28 +1, 42, 42, 0, 1179648, 0xb34d0f7f +0, 43, 43, 0, 1179648, 0xb4b820c9 +1, 43, 43, 0, 1179648, 0xf496d13d +0, 44, 44, 0, 1179648, 0x2d2940d0 +1, 44, 44, 0, 1179648, 0x985d67f5 +0, 45, 45, 0, 1179648, 0x919c554a +1, 45, 45, 0, 1179648, 0x697d0b66 +0, 46, 46, 0, 1179648, 0x7dd28f61 +1, 46, 46, 0, 1179648, 0x504d6971 +0, 47, 47, 0, 1179648, 0x51af9760 +1, 47, 47, 0, 1179648, 0x956d7fb1 +0, 48, 48, 0, 1179648, 0x8801beba +1, 48, 48, 0, 1179648, 0xec62570e +0, 49, 49, 0, 1179648, 0x80b37318 +1, 49, 49, 0, 1179648, 0x48284486 +0, 50, 50, 0, 1179648, 0x91c52522 +1, 50, 50, 0, 1179648, 0xe3a72477 +0, 51, 51, 0, 1179648, 0x6721373b +1, 51, 51, 0, 1179648, 0x9fb05dfa +0, 52, 52, 0, 1179648, 0xd2a250ca +1, 52, 52, 0, 1179648, 0xbd294daf +0, 53, 53, 0, 1179648, 0x6005c976 +1, 53, 53, 0, 1179648, 0x7f311bbb +0, 54, 54, 0, 1179648, 0x2d42f334 +1, 54, 54, 0, 1179648, 0x3bea1741 +0, 55, 55, 0, 1179648, 0xd696879d +1, 55, 55, 0, 1179648, 0x9b3259fd +0, 56, 56, 0, 1179648, 0x3324c463 +1, 56, 56, 0, 1179648, 0xc90f475d +0, 57, 57, 0, 1179648, 0x824c47df +1, 57, 57, 0, 1179648, 0x0effe63f +0, 58, 58, 0, 1179648, 0x06abf15f +1, 58, 58, 0, 1179648, 0x646ee426 +0, 59, 59, 0, 1179648, 0x34ef0c34 +1, 59, 59, 0, 1179648, 0x0fe901c7 +0, 60, 60, 0, 1179648, 0x54a6ef58 +1, 60, 60, 0, 1179648, 0xb726f458 +0, 61, 61, 0, 1179648, 0x4e9c7db9 +1, 61, 61, 0, 1179648, 0xf28adc7e +0, 62, 62, 0, 1179648, 0x70887af9 +1, 62, 62, 0, 1179648, 0x00258e2f +0, 63, 63, 0, 1179648, 0x661779d7 +1, 63, 63, 0, 1179648, 0x4598b8f7 diff --git a/tests/ref/fate/hevc-conformance-MVHEVCS_E b/tests/ref/fate/hevc-conformance-MVHEVCS_E new file mode 100644 index 0000000000..19a4a3e863 --- /dev/null +++ b/tests/ref/fate/hevc-conformance-MVHEVCS_E @@ -0,0 +1,106 @@ +#tb 0: 1/1200000 +#media_type 0: video +#codec_id 0: rawvideo +#dimensions 0: 1024x768 +#sar 0: 0/1 +#tb 1: 1/1200000 +#media_type 1: video +#codec_id 1: rawvideo +#dimensions 1: 1024x768 +#sar 1: 0/1 +0, 0, 0, 0, 1179648, 0x4a15620e +1, 0, 0, 0, 1179648, 0x5b752453 +0, 1, 1, 0, 1179648, 0xb33277fe +1, 1, 1, 0, 1179648, 0xacf4376d +0, 2, 2, 0, 1179648, 0x97b969bd +1, 2, 2, 0, 1179648, 0xa9d6e75d +0, 3, 3, 0, 1179648, 0x3c6b7b81 +1, 3, 3, 0, 1179648, 0x2142cc59 +0, 4, 4, 0, 1179648, 0xdf83fe63 +1, 4, 4, 0, 1179648, 0x838d4f0a +0, 5, 5, 0, 1179648, 0x93fc3bdf +1, 5, 5, 0, 1179648, 0x29bc71c2 +0, 6, 6, 0, 1179648, 0x18f90aa3 +1, 6, 6, 0, 1179648, 0x3b25ee18 +0, 7, 7, 0, 1179648, 0x84c730c5 +1, 7, 7, 0, 1179648, 0x60ecae5b +0, 8, 8, 0, 1179648, 0x7b49f2b1 +1, 8, 8, 0, 1179648, 0x8c3c635e +0, 9, 9, 0, 1179648, 0x60956482 +1, 9, 9, 0, 1179648, 0x8f4cb379 +0, 10, 10, 0, 1179648, 0x7af31124 +1, 10, 10, 0, 1179648, 0x398fec3c +0, 11, 11, 0, 1179648, 0xf56c7994 +1, 11, 11, 0, 1179648, 0xced70ad2 +0, 12, 12, 0, 1179648, 0x1fbd6730 +1, 12, 12, 0, 1179648, 0x4d85d789 +0, 13, 13, 0, 1179648, 0xf986b387 +1, 13, 13, 0, 1179648, 0xca3a70ea +0, 14, 14, 0, 1179648, 0xe3d5f7c5 +1, 14, 14, 0, 1179648, 0xcda13236 +0, 15, 15, 0, 1179648, 0xc646c7d1 +1, 15, 15, 0, 1179648, 0xe64a93c3 +0, 16, 16, 0, 1179648, 0xed69e4bc +1, 16, 16, 0, 1179648, 0x4426cbae +0, 17, 17, 0, 1179648, 0xaea6e7da +1, 17, 17, 0, 1179648, 0x544d045f +0, 18, 18, 0, 1179648, 0xbde3196f +1, 18, 18, 0, 1179648, 0x62b07e2a +0, 19, 19, 0, 1179648, 0x86beaf83 +1, 19, 19, 0, 1179648, 0x4ae8ac0e +0, 20, 20, 0, 1179648, 0xe162f75d +1, 20, 20, 0, 1179648, 0x092cad19 +0, 21, 21, 0, 1179648, 0xee24e674 +1, 21, 21, 0, 1179648, 0x8eab2b13 +0, 22, 22, 0, 1179648, 0xe887594a +1, 22, 22, 0, 1179648, 0x71fa0330 +0, 23, 23, 0, 1179648, 0x34a281d4 +1, 23, 23, 0, 1179648, 0xb0924dbb +0, 24, 24, 0, 1179648, 0x7ec4e840 +1, 24, 24, 0, 1179648, 0x895d98be +0, 25, 25, 0, 1179648, 0xd89577ca +1, 25, 25, 0, 1179648, 0xd91c1ea6 +0, 26, 26, 0, 1179648, 0xdc6e9c96 +1, 26, 26, 0, 1179648, 0x1b4c5dee +0, 27, 27, 0, 1179648, 0x51add2ea +1, 27, 27, 0, 1179648, 0x88c1a214 +0, 28, 28, 0, 1179648, 0x2fc19963 +1, 28, 28, 0, 1179648, 0x5861006d +0, 29, 29, 0, 1179648, 0x32243191 +1, 29, 29, 0, 1179648, 0xd4ed787c +0, 30, 30, 0, 1179648, 0xb7e2a2d8 +1, 30, 30, 0, 1179648, 0x1911a0e0 +0, 31, 31, 0, 1179648, 0x81662c81 +1, 31, 31, 0, 1179648, 0x5a37d3db +0, 32, 32, 0, 1179648, 0x4e12d459 +1, 32, 32, 0, 1179648, 0x05d5a5a9 +0, 33, 33, 0, 1179648, 0x1dc706da +1, 33, 33, 0, 1179648, 0x59f898f8 +0, 34, 34, 0, 1179648, 0x585dd375 +1, 34, 34, 0, 1179648, 0x7aa2fa66 +0, 35, 35, 0, 1179648, 0x24f7efb1 +1, 35, 35, 0, 1179648, 0x868149ad +0, 36, 36, 0, 1179648, 0xf56aa128 +1, 36, 36, 0, 1179648, 0x4c0a3019 +0, 37, 37, 0, 1179648, 0x1b87470b +1, 37, 37, 0, 1179648, 0x94bea701 +0, 38, 38, 0, 1179648, 0x368f745f +1, 38, 38, 0, 1179648, 0xf6c2b923 +0, 39, 39, 0, 1179648, 0xf0e62ce7 +1, 39, 39, 0, 1179648, 0xcc7c08c4 +0, 40, 40, 0, 1179648, 0xfc66a8a2 +1, 40, 40, 0, 1179648, 0x21afffac +0, 41, 41, 0, 1179648, 0xd136f02f +1, 41, 41, 0, 1179648, 0x0339f774 +0, 42, 42, 0, 1179648, 0x370dec8e +1, 42, 42, 0, 1179648, 0x6a66793e +0, 43, 43, 0, 1179648, 0x5e4de669 +1, 43, 43, 0, 1179648, 0x17f5ae53 +0, 44, 44, 0, 1179648, 0xc0f0e971 +1, 44, 44, 0, 1179648, 0xa741e3bf +0, 45, 45, 0, 1179648, 0x88315d92 +1, 45, 45, 0, 1179648, 0xa7895cc1 +0, 46, 46, 0, 1179648, 0x1c4eb46c +1, 46, 46, 0, 1179648, 0x0228cf31 +0, 47, 47, 0, 1179648, 0x30384288 +1, 47, 47, 0, 1179648, 0x870f2d06 diff --git a/tests/ref/fate/hevc-conformance-MVHEVCS_F b/tests/ref/fate/hevc-conformance-MVHEVCS_F new file mode 100644 index 0000000000..3770354413 --- /dev/null +++ b/tests/ref/fate/hevc-conformance-MVHEVCS_F @@ -0,0 +1,106 @@ +#tb 0: 1/1200000 +#media_type 0: video +#codec_id 0: rawvideo +#dimensions 0: 1024x768 +#sar 0: 0/1 +#tb 1: 1/1200000 +#media_type 1: video +#codec_id 1: rawvideo +#dimensions 1: 1024x768 +#sar 1: 0/1 +0, 0, 0, 0, 1179648, 0x4a15620e +1, 0, 0, 0, 1179648, 0x5b752453 +0, 1, 1, 0, 1179648, 0x568f99cf +1, 1, 1, 0, 1179648, 0xacf4376d +0, 2, 2, 0, 1179648, 0xea67491c +1, 2, 2, 0, 1179648, 0xa9d6e75d +0, 3, 3, 0, 1179648, 0x57ed0a7f +1, 3, 3, 0, 1179648, 0x2142cc59 +0, 4, 4, 0, 1179648, 0x5b218d68 +1, 4, 4, 0, 1179648, 0x838d4f0a +0, 5, 5, 0, 1179648, 0x8075c0c0 +1, 5, 5, 0, 1179648, 0x29bc71c2 +0, 6, 6, 0, 1179648, 0x75e0bb9f +1, 6, 6, 0, 1179648, 0x3b25ee18 +0, 7, 7, 0, 1179648, 0xaef237d0 +1, 7, 7, 0, 1179648, 0x60ecae5b +0, 8, 8, 0, 1179648, 0x9943126a +1, 8, 8, 0, 1179648, 0x8c3c635e +0, 9, 9, 0, 1179648, 0x3c8bbac8 +1, 9, 9, 0, 1179648, 0x8f4cb379 +0, 10, 10, 0, 1179648, 0xad824cae +1, 10, 10, 0, 1179648, 0x398fec3c +0, 11, 11, 0, 1179648, 0x3c156c58 +1, 11, 11, 0, 1179648, 0xced70ad2 +0, 12, 12, 0, 1179648, 0x18261259 +1, 12, 12, 0, 1179648, 0x4d85d789 +0, 13, 13, 0, 1179648, 0xd481dafa +1, 13, 13, 0, 1179648, 0xca3a70ea +0, 14, 14, 0, 1179648, 0x6a0d3af8 +1, 14, 14, 0, 1179648, 0xcda13236 +0, 15, 15, 0, 1179648, 0xaa355e8f +1, 15, 15, 0, 1179648, 0xe64a93c3 +0, 16, 16, 0, 1179648, 0xed69e4bc +1, 16, 16, 0, 1179648, 0x4426cbae +0, 17, 17, 0, 1179648, 0x340d6c4a +1, 17, 17, 0, 1179648, 0x544d045f +0, 18, 18, 0, 1179648, 0xd52398d0 +1, 18, 18, 0, 1179648, 0x62b07e2a +0, 19, 19, 0, 1179648, 0x39d92b93 +1, 19, 19, 0, 1179648, 0x4ae8ac0e +0, 20, 20, 0, 1179648, 0x364fd94f +1, 20, 20, 0, 1179648, 0x092cad19 +0, 21, 21, 0, 1179648, 0x977cabf5 +1, 21, 21, 0, 1179648, 0x8eab2b13 +0, 22, 22, 0, 1179648, 0x23febfaf +1, 22, 22, 0, 1179648, 0x71fa0330 +0, 23, 23, 0, 1179648, 0xd7e5afa0 +1, 23, 23, 0, 1179648, 0xb0924dbb +0, 24, 24, 0, 1179648, 0x83799fed +1, 24, 24, 0, 1179648, 0x895d98be +0, 25, 25, 0, 1179648, 0xaec7466c +1, 25, 25, 0, 1179648, 0xd91c1ea6 +0, 26, 26, 0, 1179648, 0x4c4de340 +1, 26, 26, 0, 1179648, 0x1b4c5dee +0, 27, 27, 0, 1179648, 0x1c8c5bf6 +1, 27, 27, 0, 1179648, 0x88c1a214 +0, 28, 28, 0, 1179648, 0xc980311c +1, 28, 28, 0, 1179648, 0x5861006d +0, 29, 29, 0, 1179648, 0xb55dbf43 +1, 29, 29, 0, 1179648, 0xd4ed787c +0, 30, 30, 0, 1179648, 0xbc29868a +1, 30, 30, 0, 1179648, 0x1911a0e0 +0, 31, 31, 0, 1179648, 0x3d1dfc87 +1, 31, 31, 0, 1179648, 0x5a37d3db +0, 32, 32, 0, 1179648, 0x4e12d459 +1, 32, 32, 0, 1179648, 0x05d5a5a9 +0, 33, 33, 0, 1179648, 0xb4eb0d99 +1, 33, 33, 0, 1179648, 0x59f898f8 +0, 34, 34, 0, 1179648, 0x1d91ae23 +1, 34, 34, 0, 1179648, 0x7aa2fa66 +0, 35, 35, 0, 1179648, 0x0adf2e05 +1, 35, 35, 0, 1179648, 0x868149ad +0, 36, 36, 0, 1179648, 0x947e48dd +1, 36, 36, 0, 1179648, 0x4c0a3019 +0, 37, 37, 0, 1179648, 0xb7dddffd +1, 37, 37, 0, 1179648, 0x94bea701 +0, 38, 38, 0, 1179648, 0x027f1113 +1, 38, 38, 0, 1179648, 0xf6c2b923 +0, 39, 39, 0, 1179648, 0xc2a0afa2 +1, 39, 39, 0, 1179648, 0xcc7c08c4 +0, 40, 40, 0, 1179648, 0xd12a6bc0 +1, 40, 40, 0, 1179648, 0x21afffac +0, 41, 41, 0, 1179648, 0xf8f5d7ee +1, 41, 41, 0, 1179648, 0x0339f774 +0, 42, 42, 0, 1179648, 0xa7e5fd01 +1, 42, 42, 0, 1179648, 0x6a66793e +0, 43, 43, 0, 1179648, 0xe9aee64d +1, 43, 43, 0, 1179648, 0x17f5ae53 +0, 44, 44, 0, 1179648, 0x989fc660 +1, 44, 44, 0, 1179648, 0xa741e3bf +0, 45, 45, 0, 1179648, 0x43e1d4d0 +1, 45, 45, 0, 1179648, 0xa7895cc1 +0, 46, 46, 0, 1179648, 0x38be7f17 +1, 46, 46, 0, 1179648, 0x0228cf31 +0, 47, 47, 0, 1179648, 0xd867c854 +1, 47, 47, 0, 1179648, 0x870f2d06 -- 2.43.0
Typical files use 0 for the base layer and 1 for the secondary one, but any value for the secondary layer should be supported. --- tests/fate/hevc.mak | 4 ++++ tests/ref/fate/hevc-mv-nuh-layer-id | 15 +++++++++++++++ 2 files changed, 19 insertions(+) create mode 100644 tests/ref/fate/hevc-mv-nuh-layer-id diff --git a/tests/fate/hevc.mak b/tests/fate/hevc.mak index e7dad6c898..e75fbd3c16 100644 --- a/tests/fate/hevc.mak +++ b/tests/fate/hevc.mak @@ -269,6 +269,10 @@ FATE_HEVC-$(call FRAMECRC, HEVC, HEVC) += fate-hevc-cabac-tudepth fate-hevc-small422chroma: CMD = framecrc -i $(TARGET_SAMPLES)/hevc/food.hevc -pix_fmt yuv422p10le -vf scale FATE_HEVC-$(call FRAMECRC, HEVC, HEVC, HEVC_PARSER SCALE_FILTER) += fate-hevc-small422chroma +# multiview stream, where the secondary layer has a nontrivial nuh_layer_id=6 +fate-hevc-mv-nuh-layer-id: CMD = framecrc -i $(TARGET_SAMPLES)/hevc/mv_nuh_layer_id.bit -map 0:view:all +FATE_HEVC-$(call FRAMECRC, HEVC, HEVC) += fate-hevc-mv-nuh-layer-id + FATE_SAMPLES_AVCONV += $(FATE_HEVC-yes) FATE_SAMPLES_FFPROBE += $(FATE_HEVC_FFPROBE-yes) diff --git a/tests/ref/fate/hevc-mv-nuh-layer-id b/tests/ref/fate/hevc-mv-nuh-layer-id new file mode 100644 index 0000000000..3cbefe17f6 --- /dev/null +++ b/tests/ref/fate/hevc-mv-nuh-layer-id @@ -0,0 +1,15 @@ +#tb 0: 1/25 +#media_type 0: video +#codec_id 0: rawvideo +#dimensions 0: 128x128 +#sar 0: 0/1 +0, 0, 0, 1, 24576, 0xdfd350a6 +0, 1, 1, 1, 24576, 0xf8f638da +0, 2, 2, 1, 24576, 0x8ac574d5 +0, 3, 3, 1, 24576, 0xd22675a4 +0, 4, 4, 1, 24576, 0xdd0f4704 +0, 5, 5, 1, 24576, 0x60da42e6 +0, 6, 6, 1, 24576, 0x8bf28fdd +0, 7, 7, 1, 24576, 0xe0577f6e +0, 8, 8, 1, 24576, 0x8b3e3c29 +0, 9, 9, 1, 24576, 0x8d9944bd -- 2.43.0
--- tests/fate/hevc.mak | 10 ++ tests/ref/fate/hevc-mv-switch | 172 ++++++++++++++++++++++++++++++++++ 2 files changed, 182 insertions(+) create mode 100644 tests/ref/fate/hevc-mv-switch diff --git a/tests/fate/hevc.mak b/tests/fate/hevc.mak index e75fbd3c16..5631ded860 100644 --- a/tests/fate/hevc.mak +++ b/tests/fate/hevc.mak @@ -273,6 +273,16 @@ FATE_HEVC-$(call FRAMECRC, HEVC, HEVC, HEVC_PARSER SCALE_FILTER) += fate-hevc-sm fate-hevc-mv-nuh-layer-id: CMD = framecrc -i $(TARGET_SAMPLES)/hevc/mv_nuh_layer_id.bit -map 0:view:all FATE_HEVC-$(call FRAMECRC, HEVC, HEVC) += fate-hevc-mv-nuh-layer-id +# NB: $\ at the end of line joins lines without adding whitespace; +# this trick is recommended by GNU make manual +fate-hevc-mv-switch: INPUT = \ +$(TARGET_SAMPLES)/hevc-conformance/LS_A_Orange_2.bit|$\ +$(TARGET_SAMPLES)/hevc/mv_nuh_layer_id.bit|$\ +$(TARGET_SAMPLES)/hevc-conformance/NoOutPrior_B_Qualcomm_1.bit|$\ +$(TARGET_SAMPLES)/hevc-conformance/MVHEVCS_A.bit +fate-hevc-mv-switch: CMD = framecrc -i "concat:$(INPUT)" -fps_mode passthrough -map 0:vidx:0 -map 0:vidx:1 +FATE_HEVC-$(call FRAMECRC, HEVC, HEVC, CONCAT_PROTOCOL) += fate-hevc-mv-switch + FATE_SAMPLES_AVCONV += $(FATE_HEVC-yes) FATE_SAMPLES_FFPROBE += $(FATE_HEVC_FFPROBE-yes) diff --git a/tests/ref/fate/hevc-mv-switch b/tests/ref/fate/hevc-mv-switch new file mode 100644 index 0000000000..0fc3630637 --- /dev/null +++ b/tests/ref/fate/hevc-mv-switch @@ -0,0 +1,172 @@ +#tb 0: 1/25 +#media_type 0: video +#codec_id 0: rawvideo +#dimensions 0: 416x240 +#sar 0: 0/1 +#tb 1: 1/25 +#media_type 1: video +#codec_id 1: rawvideo +#dimensions 1: 128x128 +#sar 1: 0/1 +0, 0, 0, 1, 149760, 0x94a51701 +0, 1, 1, 1, 149760, 0x67c71885 +0, 2, 2, 1, 149760, 0x218f1751 +0, 3, 3, 1, 149760, 0x56951bef +0, 4, 4, 1, 149760, 0x76aec81e +0, 5, 5, 1, 149760, 0x20df61ac +0, 6, 6, 1, 149760, 0x2eacf616 +0, 7, 7, 1, 149760, 0x06322ce2 +0, 8, 8, 1, 149760, 0xf14aa104 +0, 9, 9, 1, 149760, 0xc948dcba +1, 10, 10, 1, 24576, 0xf8f638da +0, 11, 11, 1, 149760, 0x674e34b1 +1, 12, 12, 1, 24576, 0xd22675a4 +0, 13, 13, 1, 149760, 0x41d3acd6 +1, 14, 14, 1, 24576, 0x60da42e6 +0, 15, 15, 1, 149760, 0x55a5b835 +1, 16, 16, 1, 24576, 0xe0577f6e +0, 17, 17, 1, 149760, 0xc6958049 +1, 18, 18, 1, 24576, 0x8d9944bd +0, 19, 19, 1, 149760, 0x94b37050 +0, 20, 20, 1, 149760, 0xed72a560 +0, 21, 21, 1, 149760, 0xd0ccac61 +0, 22, 22, 1, 149760, 0x6cc2d7fa +0, 23, 23, 1, 149760, 0x3a02b5ba +0, 24, 24, 1, 149760, 0xce7ef09c +0, 25, 25, 1, 149760, 0xa518fc05 +0, 26, 26, 1, 149760, 0x01d238fe +0, 27, 27, 1, 149760, 0x5f5012fa +0, 28, 28, 1, 149760, 0x5b8e7405 +0, 29, 29, 1, 149760, 0xcc2e5b33 +0, 30, 30, 1, 149760, 0x590a6890 +0, 31, 31, 1, 149760, 0x9c7b189f +0, 32, 32, 1, 149760, 0xd0752ef4 +0, 33, 33, 1, 149760, 0x367513ce +0, 34, 34, 1, 149760, 0xb64c209d +0, 35, 35, 1, 149760, 0x6e50994c +0, 36, 36, 1, 149760, 0x8276cce4 +0, 37, 37, 1, 149760, 0xb292ac8f +0, 38, 38, 1, 149760, 0x57de9d2c +0, 39, 39, 1, 149760, 0xe8533f38 +0, 40, 40, 1, 149760, 0xde9b536d +0, 41, 41, 1, 149760, 0x83173b1d +0, 42, 42, 1, 149760, 0x853a83a4 +0, 43, 43, 1, 149760, 0x481af1bf +0, 44, 44, 1, 149760, 0x27221abb +0, 45, 45, 1, 149760, 0x094eac00 +0, 46, 46, 1, 149760, 0x3f3a27c8 +0, 47, 47, 1, 149760, 0x8f19b2af +0, 48, 48, 1, 149760, 0x93e7e591 +0, 49, 49, 1, 149760, 0x0c531ab8 +0, 50, 50, 1, 149760, 0x3456ef8a +0, 51, 51, 1, 149760, 0xfee2ec1e +0, 52, 52, 1, 149760, 0x76b4b750 +0, 53, 53, 1, 149760, 0xa48bb670 +0, 54, 54, 1, 149760, 0x3dee7cac +0, 55, 55, 1, 149760, 0x2b20561d +0, 56, 56, 1, 149760, 0xd3c5bf9f +0, 57, 57, 1, 149760, 0x2e87d747 +0, 58, 58, 1, 149760, 0x9952760b +0, 59, 59, 1, 149760, 0xa3f5cbda +0, 60, 60, 1, 149760, 0x56e3f94d +0, 61, 61, 1, 149760, 0x902f743f +0, 62, 62, 1, 149760, 0xeffcfd85 +0, 63, 63, 1, 149760, 0xe7fc31b2 +0, 64, 64, 1, 149760, 0x2e876286 +0, 65, 65, 1, 149760, 0x6358b0b2 +0, 66, 66, 1, 149760, 0x3e569a4d +1, 67, 67, 1, 24576, 0xa18c481f +0, 68, 68, 1, 149760, 0x641cbce0 +1, 69, 69, 1, 24576, 0x9ee94df2 +0, 70, 70, 1, 149760, 0x0357d35c +1, 71, 71, 1, 24576, 0x60c55365 +0, 72, 72, 1, 149760, 0xe9b5b077 +1, 73, 73, 1, 24576, 0x8951536a +0, 74, 74, 1, 149760, 0x3b9c605f +1, 75, 75, 1, 24576, 0xfea04ba5 +0, 76, 76, 1, 149760, 0x16d7028b +1, 77, 77, 1, 24576, 0x78094226 +0, 78, 78, 1, 149760, 0xf4896eff +1, 79, 79, 1, 24576, 0xacf9314e +0, 80, 80, 1, 149760, 0xc90de4b9 +1, 81, 81, 1, 24576, 0x6a6b1c87 +0, 82, 82, 1, 149760, 0x2ec258ad +1, 83, 83, 1, 24576, 0x9f360661 +0, 84, 84, 1, 149760, 0x3bf720ef +1, 85, 85, 1, 24576, 0x52e80514 +0, 86, 86, 1, 149760, 0x67eab73f +1, 87, 87, 1, 24576, 0xe10df48c +0, 88, 88, 1, 149760, 0x020e17ce +1, 89, 89, 1, 24576, 0x71fbdd03 +0, 90, 90, 1, 149760, 0xcd677640 +1, 91, 91, 1, 24576, 0x1d08c3f6 +0, 92, 92, 1, 149760, 0x194cfbc1 +1, 93, 93, 1, 24576, 0x4b93b6ca +0, 94, 94, 1, 149760, 0x740f6261 +1, 95, 95, 1, 24576, 0x0dea9c12 +0, 96, 96, 1, 149760, 0x104b3175 +1, 97, 97, 1, 24576, 0x195a92da +0, 98, 98, 1, 149760, 0xc8fd17d1 +1, 99, 99, 1, 24576, 0x55dd870b +0, 100, 100, 1, 149760, 0x6f4e7787 +1, 101, 101, 1, 24576, 0x7e6b905b +0, 102, 102, 1, 149760, 0xe853418e +1, 103, 103, 1, 24576, 0x0f3fad1d +0, 104, 104, 1, 149760, 0x3dba3e4e +1, 105, 105, 1, 24576, 0xe724d536 +0, 106, 106, 1, 149760, 0xba0b4c38 +1, 107, 107, 1, 24576, 0xb212fed4 +0, 108, 108, 1, 149760, 0x28f8ef1d +1, 109, 109, 1, 24576, 0x3bb423ce +0, 110, 110, 1, 149760, 0xb5d92e4e +1, 111, 111, 1, 24576, 0x20f432db +0, 112, 112, 1, 149760, 0xbaa0d0a2 +1, 113, 113, 1, 24576, 0x27412981 +0, 114, 114, 1, 149760, 0x0acde200 +1, 115, 115, 1, 24576, 0xb7f70143 +0, 116, 116, 1, 149760, 0x7388aad3 +1, 117, 117, 1, 24576, 0xf31acffb +0, 118, 118, 1, 149760, 0xbac95723 +1, 119, 119, 1, 24576, 0x283d8c2c +0, 120, 120, 1, 149760, 0x7633d4af +1, 121, 121, 1, 24576, 0x9fb08184 +0, 122, 122, 1, 149760, 0x836e54fc +1, 123, 123, 1, 24576, 0x09696ad1 +0, 124, 124, 1, 149760, 0x49096e63 +1, 125, 125, 1, 24576, 0x034e69c2 +0, 126, 126, 1, 149760, 0x98e14158 +1, 127, 127, 1, 24576, 0x1ce4882c +0, 128, 128, 1, 149760, 0x955a43e4 +1, 129, 129, 1, 24576, 0x490fda13 +0, 130, 130, 1, 149760, 0x3a76c087 +1, 131, 131, 1, 24576, 0x4c2e1c09 +0, 132, 132, 1, 149760, 0xa221e763 +1, 133, 133, 1, 24576, 0x82dd5f8b +0, 134, 134, 1, 149760, 0x1eb33f17 +1, 135, 135, 1, 24576, 0xcfb07d6b +0, 136, 136, 1, 149760, 0x13ef3914 +1, 137, 137, 1, 24576, 0xef468129 +0, 138, 138, 1, 149760, 0x2521b880 +1, 139, 139, 1, 24576, 0x24c970fa +0, 140, 140, 1, 149760, 0x78a4faf0 +1, 141, 141, 1, 24576, 0x6df14e99 +0, 142, 142, 1, 149760, 0xc5f71d65 +1, 143, 143, 1, 24576, 0x78f44854 +0, 144, 144, 1, 149760, 0x5dbc0a9f +1, 145, 145, 1, 24576, 0x5a7269fd +0, 146, 146, 1, 149760, 0x8ebfd7c3 +1, 147, 147, 1, 24576, 0xb1fd8924 +0, 148, 148, 1, 149760, 0xb45f0581 +1, 149, 149, 1, 24576, 0xa165b829 +0, 150, 150, 1, 149760, 0x3b84540b +1, 151, 151, 1, 24576, 0x694cf0e5 +0, 152, 152, 1, 149760, 0xec921f35 +1, 153, 153, 1, 24576, 0x2fef157b +0, 154, 154, 1, 149760, 0x7b23826a +1, 155, 155, 1, 24576, 0x62ea25dd +0, 156, 156, 1, 149760, 0x518d1f11 +1, 157, 157, 1, 24576, 0xfc02365e +0, 158, 158, 1, 149760, 0xc5e694a4 +1, 159, 159, 1, 24576, 0x073b3ebd +0, 160, 160, 1, 149760, 0xeb79c961 +1, 161, 161, 1, 24576, 0x5ee13be2 -- 2.43.0
-----Original Message----- From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Anton Khirnov Sent: 2024年8月27日 23:05 To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH 42/42] tests/fate/hevc: add a test for switching between single and multi-view
--- tests/fate/hevc.mak | 10 ++ tests/ref/fate/hevc-mv-switch | 172 ++++++++++++++++++++++++++++++++++ 2 files changed, 182 insertions(+) create mode 100644 tests/ref/fate/hevc-mv-switch
diff --git a/tests/fate/hevc.mak b/tests/fate/hevc.mak index e75fbd3c16..5631ded860 100644 --- a/tests/fate/hevc.mak +++ b/tests/fate/hevc.mak @@ -273,6 +273,16 @@ FATE_HEVC-$(call FRAMECRC, HEVC, HEVC, HEVC_PARSER SCALE_FILTER) += fate-hevc-sm fate-hevc-mv-nuh-layer-id: CMD = framecrc -i $(TARGET_SAMPLES)/hevc/mv_nuh_layer_id.bit -map 0:view:all FATE_HEVC-$(call FRAMECRC, HEVC, HEVC) += fate-hevc-mv-nuh-layer-id
+# NB: $\ at the end of line joins lines without adding whitespace; +# this trick is recommended by GNU make manual +fate-hevc-mv-switch: INPUT = \ +$(TARGET_SAMPLES)/hevc-conformance/LS_A_Orange_2.bit|$\ +$(TARGET_SAMPLES)/hevc/mv_nuh_layer_id.bit|$\ +$(TARGET_SAMPLES)/hevc-conformance/NoOutPrior_B_Qualcomm_1.bit|$\ +$(TARGET_SAMPLES)/hevc-conformance/MVHEVCS_A.bit +fate-hevc-mv-switch: CMD = framecrc -i "concat:$(INPUT)" -fps_mode passthrough -map 0:vidx:0 -map 0:vidx:1 +FATE_HEVC-$(call FRAMECRC, HEVC, HEVC, CONCAT_PROTOCOL) += fate-hevc-mv-switch
The test failed when run with msys2, because msys2 only convert the first sample path to Windows style path: [in#0 @ 0000028FF3412EC0] Error opening input: No such file or directory Error opening input file concat;D:\bin\msys\home\quink\work\ffmpeg-fate-sample\hevc-conformance\LS_A_Orange_2.bit|\home\quink\work\ffmpeg-fate-sample\hevc\mv_nuh_layer_id.bit|\home\quink\work\ffmpeg-fate-sample\hevc-conformance\NoOutPrior_B_Qualcomm_1.bit|\home\quink\work\ffmpeg-fate-sample\hevc-conformance\MVHEVCS_A.bit. Error opening input files: No such file or directory
+ FATE_SAMPLES_AVCONV += $(FATE_HEVC-yes) FATE_SAMPLES_FFPROBE += $(FATE_HEVC_FFPROBE-yes)
diff --git a/tests/ref/fate/hevc-mv-switch b/tests/ref/fate/hevc-mv-switch new file mode 100644 index 0000000000..0fc3630637 --- /dev/null +++ b/tests/ref/fate/hevc-mv-switch @@ -0,0 +1,172 @@ +#tb 0: 1/25 +#media_type 0: video +#codec_id 0: rawvideo +#dimensions 0: 416x240 +#sar 0: 0/1 +#tb 1: 1/25 +#media_type 1: video +#codec_id 1: rawvideo +#dimensions 1: 128x128 +#sar 1: 0/1 +0, 0, 0, 1, 149760, 0x94a51701 +0, 1, 1, 1, 149760, 0x67c71885 +0, 2, 2, 1, 149760, 0x218f1751 +0, 3, 3, 1, 149760, 0x56951bef +0, 4, 4, 1, 149760, 0x76aec81e +0, 5, 5, 1, 149760, 0x20df61ac +0, 6, 6, 1, 149760, 0x2eacf616 +0, 7, 7, 1, 149760, 0x06322ce2 +0, 8, 8, 1, 149760, 0xf14aa104 +0, 9, 9, 1, 149760, 0xc948dcba +1, 10, 10, 1, 24576, 0xf8f638da +0, 11, 11, 1, 149760, 0x674e34b1 +1, 12, 12, 1, 24576, 0xd22675a4 +0, 13, 13, 1, 149760, 0x41d3acd6 +1, 14, 14, 1, 24576, 0x60da42e6 +0, 15, 15, 1, 149760, 0x55a5b835 +1, 16, 16, 1, 24576, 0xe0577f6e +0, 17, 17, 1, 149760, 0xc6958049 +1, 18, 18, 1, 24576, 0x8d9944bd +0, 19, 19, 1, 149760, 0x94b37050 +0, 20, 20, 1, 149760, 0xed72a560 +0, 21, 21, 1, 149760, 0xd0ccac61 +0, 22, 22, 1, 149760, 0x6cc2d7fa +0, 23, 23, 1, 149760, 0x3a02b5ba +0, 24, 24, 1, 149760, 0xce7ef09c +0, 25, 25, 1, 149760, 0xa518fc05 +0, 26, 26, 1, 149760, 0x01d238fe +0, 27, 27, 1, 149760, 0x5f5012fa +0, 28, 28, 1, 149760, 0x5b8e7405 +0, 29, 29, 1, 149760, 0xcc2e5b33 +0, 30, 30, 1, 149760, 0x590a6890 +0, 31, 31, 1, 149760, 0x9c7b189f +0, 32, 32, 1, 149760, 0xd0752ef4 +0, 33, 33, 1, 149760, 0x367513ce +0, 34, 34, 1, 149760, 0xb64c209d +0, 35, 35, 1, 149760, 0x6e50994c +0, 36, 36, 1, 149760, 0x8276cce4 +0, 37, 37, 1, 149760, 0xb292ac8f +0, 38, 38, 1, 149760, 0x57de9d2c +0, 39, 39, 1, 149760, 0xe8533f38 +0, 40, 40, 1, 149760, 0xde9b536d +0, 41, 41, 1, 149760, 0x83173b1d +0, 42, 42, 1, 149760, 0x853a83a4 +0, 43, 43, 1, 149760, 0x481af1bf +0, 44, 44, 1, 149760, 0x27221abb +0, 45, 45, 1, 149760, 0x094eac00 +0, 46, 46, 1, 149760, 0x3f3a27c8 +0, 47, 47, 1, 149760, 0x8f19b2af +0, 48, 48, 1, 149760, 0x93e7e591 +0, 49, 49, 1, 149760, 0x0c531ab8 +0, 50, 50, 1, 149760, 0x3456ef8a +0, 51, 51, 1, 149760, 0xfee2ec1e +0, 52, 52, 1, 149760, 0x76b4b750 +0, 53, 53, 1, 149760, 0xa48bb670 +0, 54, 54, 1, 149760, 0x3dee7cac +0, 55, 55, 1, 149760, 0x2b20561d +0, 56, 56, 1, 149760, 0xd3c5bf9f +0, 57, 57, 1, 149760, 0x2e87d747 +0, 58, 58, 1, 149760, 0x9952760b +0, 59, 59, 1, 149760, 0xa3f5cbda +0, 60, 60, 1, 149760, 0x56e3f94d +0, 61, 61, 1, 149760, 0x902f743f +0, 62, 62, 1, 149760, 0xeffcfd85 +0, 63, 63, 1, 149760, 0xe7fc31b2 +0, 64, 64, 1, 149760, 0x2e876286 +0, 65, 65, 1, 149760, 0x6358b0b2 +0, 66, 66, 1, 149760, 0x3e569a4d +1, 67, 67, 1, 24576, 0xa18c481f +0, 68, 68, 1, 149760, 0x641cbce0 +1, 69, 69, 1, 24576, 0x9ee94df2 +0, 70, 70, 1, 149760, 0x0357d35c +1, 71, 71, 1, 24576, 0x60c55365 +0, 72, 72, 1, 149760, 0xe9b5b077 +1, 73, 73, 1, 24576, 0x8951536a +0, 74, 74, 1, 149760, 0x3b9c605f +1, 75, 75, 1, 24576, 0xfea04ba5 +0, 76, 76, 1, 149760, 0x16d7028b +1, 77, 77, 1, 24576, 0x78094226 +0, 78, 78, 1, 149760, 0xf4896eff +1, 79, 79, 1, 24576, 0xacf9314e +0, 80, 80, 1, 149760, 0xc90de4b9 +1, 81, 81, 1, 24576, 0x6a6b1c87 +0, 82, 82, 1, 149760, 0x2ec258ad +1, 83, 83, 1, 24576, 0x9f360661 +0, 84, 84, 1, 149760, 0x3bf720ef +1, 85, 85, 1, 24576, 0x52e80514 +0, 86, 86, 1, 149760, 0x67eab73f +1, 87, 87, 1, 24576, 0xe10df48c +0, 88, 88, 1, 149760, 0x020e17ce +1, 89, 89, 1, 24576, 0x71fbdd03 +0, 90, 90, 1, 149760, 0xcd677640 +1, 91, 91, 1, 24576, 0x1d08c3f6 +0, 92, 92, 1, 149760, 0x194cfbc1 +1, 93, 93, 1, 24576, 0x4b93b6ca +0, 94, 94, 1, 149760, 0x740f6261 +1, 95, 95, 1, 24576, 0x0dea9c12 +0, 96, 96, 1, 149760, 0x104b3175 +1, 97, 97, 1, 24576, 0x195a92da +0, 98, 98, 1, 149760, 0xc8fd17d1 +1, 99, 99, 1, 24576, 0x55dd870b +0, 100, 100, 1, 149760, 0x6f4e7787 +1, 101, 101, 1, 24576, 0x7e6b905b +0, 102, 102, 1, 149760, 0xe853418e +1, 103, 103, 1, 24576, 0x0f3fad1d +0, 104, 104, 1, 149760, 0x3dba3e4e +1, 105, 105, 1, 24576, 0xe724d536 +0, 106, 106, 1, 149760, 0xba0b4c38 +1, 107, 107, 1, 24576, 0xb212fed4 +0, 108, 108, 1, 149760, 0x28f8ef1d +1, 109, 109, 1, 24576, 0x3bb423ce +0, 110, 110, 1, 149760, 0xb5d92e4e +1, 111, 111, 1, 24576, 0x20f432db +0, 112, 112, 1, 149760, 0xbaa0d0a2 +1, 113, 113, 1, 24576, 0x27412981 +0, 114, 114, 1, 149760, 0x0acde200 +1, 115, 115, 1, 24576, 0xb7f70143 +0, 116, 116, 1, 149760, 0x7388aad3 +1, 117, 117, 1, 24576, 0xf31acffb +0, 118, 118, 1, 149760, 0xbac95723 +1, 119, 119, 1, 24576, 0x283d8c2c +0, 120, 120, 1, 149760, 0x7633d4af +1, 121, 121, 1, 24576, 0x9fb08184 +0, 122, 122, 1, 149760, 0x836e54fc +1, 123, 123, 1, 24576, 0x09696ad1 +0, 124, 124, 1, 149760, 0x49096e63 +1, 125, 125, 1, 24576, 0x034e69c2 +0, 126, 126, 1, 149760, 0x98e14158 +1, 127, 127, 1, 24576, 0x1ce4882c +0, 128, 128, 1, 149760, 0x955a43e4 +1, 129, 129, 1, 24576, 0x490fda13 +0, 130, 130, 1, 149760, 0x3a76c087 +1, 131, 131, 1, 24576, 0x4c2e1c09 +0, 132, 132, 1, 149760, 0xa221e763 +1, 133, 133, 1, 24576, 0x82dd5f8b +0, 134, 134, 1, 149760, 0x1eb33f17 +1, 135, 135, 1, 24576, 0xcfb07d6b +0, 136, 136, 1, 149760, 0x13ef3914 +1, 137, 137, 1, 24576, 0xef468129 +0, 138, 138, 1, 149760, 0x2521b880 +1, 139, 139, 1, 24576, 0x24c970fa +0, 140, 140, 1, 149760, 0x78a4faf0 +1, 141, 141, 1, 24576, 0x6df14e99 +0, 142, 142, 1, 149760, 0xc5f71d65 +1, 143, 143, 1, 24576, 0x78f44854 +0, 144, 144, 1, 149760, 0x5dbc0a9f +1, 145, 145, 1, 24576, 0x5a7269fd +0, 146, 146, 1, 149760, 0x8ebfd7c3 +1, 147, 147, 1, 24576, 0xb1fd8924 +0, 148, 148, 1, 149760, 0xb45f0581 +1, 149, 149, 1, 24576, 0xa165b829 +0, 150, 150, 1, 149760, 0x3b84540b +1, 151, 151, 1, 24576, 0x694cf0e5 +0, 152, 152, 1, 149760, 0xec921f35 +1, 153, 153, 1, 24576, 0x2fef157b +0, 154, 154, 1, 149760, 0x7b23826a +1, 155, 155, 1, 24576, 0x62ea25dd +0, 156, 156, 1, 149760, 0x518d1f11 +1, 157, 157, 1, 24576, 0xfc02365e +0, 158, 158, 1, 149760, 0xc5e694a4 +1, 159, 159, 1, 24576, 0x073b3ebd +0, 160, 160, 1, 149760, 0xeb79c961 +1, 161, 161, 1, 24576, 0x5ee13be2 -- 2.43.0
_______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
On 11/30/2024 1:38 PM, Zhao Zhili wrote:
-----Original Message----- From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Anton Khirnov Sent: 2024年8月27日 23:05 To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH 42/42] tests/fate/hevc: add a test for switching between single and multi-view
--- tests/fate/hevc.mak | 10 ++ tests/ref/fate/hevc-mv-switch | 172 ++++++++++++++++++++++++++++++++++ 2 files changed, 182 insertions(+) create mode 100644 tests/ref/fate/hevc-mv-switch
diff --git a/tests/fate/hevc.mak b/tests/fate/hevc.mak index e75fbd3c16..5631ded860 100644 --- a/tests/fate/hevc.mak +++ b/tests/fate/hevc.mak @@ -273,6 +273,16 @@ FATE_HEVC-$(call FRAMECRC, HEVC, HEVC, HEVC_PARSER SCALE_FILTER) += fate-hevc-sm fate-hevc-mv-nuh-layer-id: CMD = framecrc -i $(TARGET_SAMPLES)/hevc/mv_nuh_layer_id.bit -map 0:view:all FATE_HEVC-$(call FRAMECRC, HEVC, HEVC) += fate-hevc-mv-nuh-layer-id
+# NB: $\ at the end of line joins lines without adding whitespace; +# this trick is recommended by GNU make manual +fate-hevc-mv-switch: INPUT = \ +$(TARGET_SAMPLES)/hevc-conformance/LS_A_Orange_2.bit|$\ +$(TARGET_SAMPLES)/hevc/mv_nuh_layer_id.bit|$\ +$(TARGET_SAMPLES)/hevc-conformance/NoOutPrior_B_Qualcomm_1.bit|$\ +$(TARGET_SAMPLES)/hevc-conformance/MVHEVCS_A.bit +fate-hevc-mv-switch: CMD = framecrc -i "concat:$(INPUT)" -fps_mode passthrough -map 0:vidx:0 -map 0:vidx:1 +FATE_HEVC-$(call FRAMECRC, HEVC, HEVC, CONCAT_PROTOCOL) += fate-hevc-mv-switch
The test failed when run with msys2, because msys2 only convert the first sample path to Windows style path:
It shouldn't convert any at all.
[in#0 @ 0000028FF3412EC0] Error opening input: No such file or directory Error opening input file concat;D:\bin\msys\home\quink\work\ffmpeg-fate-sample\hevc-conformance\LS_A_Orange_2.bit|\home\quink\work\ffmpeg-fate-sample\hevc\mv_nuh_layer_id.bit|\home\quink\work\ffmpeg-fate-sample\hevc-conformance\NoOutPrior_B_Qualcomm_1.bit|\home\quink\work\ffmpeg-fate-sample\hevc-conformance\MVHEVCS_A.bit.
I can't reproduce this on msys2 locally. Have you tried using a relative path for SAMPLES?
-----Original Message----- From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of James Almer Sent: 2024年12月1日 0:51 To: ffmpeg-devel@ffmpeg.org Subject: Re: [FFmpeg-devel] [PATCH 42/42] tests/fate/hevc: add a test for switching between single and multi-view
On 11/30/2024 1:38 PM, Zhao Zhili wrote:
-----Original Message----- From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Anton Khirnov Sent: 2024年8月27日 23:05 To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH 42/42] tests/fate/hevc: add a test for switching between single and multi-view
--- tests/fate/hevc.mak | 10 ++ tests/ref/fate/hevc-mv-switch | 172 ++++++++++++++++++++++++++++++++++ 2 files changed, 182 insertions(+) create mode 100644 tests/ref/fate/hevc-mv-switch
diff --git a/tests/fate/hevc.mak b/tests/fate/hevc.mak index e75fbd3c16..5631ded860 100644 --- a/tests/fate/hevc.mak +++ b/tests/fate/hevc.mak @@ -273,6 +273,16 @@ FATE_HEVC-$(call FRAMECRC, HEVC, HEVC, HEVC_PARSER SCALE_FILTER) += fate-hevc-sm fate-hevc-mv-nuh-layer-id: CMD = framecrc -i $(TARGET_SAMPLES)/hevc/mv_nuh_layer_id.bit -map 0:view:all FATE_HEVC-$(call FRAMECRC, HEVC, HEVC) += fate-hevc-mv-nuh-layer-id
+# NB: $\ at the end of line joins lines without adding whitespace; +# this trick is recommended by GNU make manual +fate-hevc-mv-switch: INPUT = \ +$(TARGET_SAMPLES)/hevc-conformance/LS_A_Orange_2.bit|$\ +$(TARGET_SAMPLES)/hevc/mv_nuh_layer_id.bit|$\ +$(TARGET_SAMPLES)/hevc-conformance/NoOutPrior_B_Qualcomm_1.bit|$\ +$(TARGET_SAMPLES)/hevc-conformance/MVHEVCS_A.bit +fate-hevc-mv-switch: CMD = framecrc -i "concat:$(INPUT)" -fps_mode passthrough -map 0:vidx:0 -map 0:vidx:1 +FATE_HEVC-$(call FRAMECRC, HEVC, HEVC, CONCAT_PROTOCOL) += fate-hevc-mv-switch
The test failed when run with msys2, because msys2 only convert the first sample path to Windows style path:
It shouldn't convert any at all.
[in#0 @ 0000028FF3412EC0] Error opening input: No such file or directory Error opening input file
concat;D:\bin\msys\home\quink\work\ffmpeg-fate-sample\hevc-conformance\LS_A_Orange_2.bit|\home\quink\work\ffmpeg-fate-sam ple\hevc\mv_nuh_layer_id.bit|\home\quink\work\ffmpeg-fate-sample\hevc-conformance\NoOutPrior_B_Qualcomm_1.bit|\home\quin k\work\ffmpeg-fate-sample\hevc-conformance\MVHEVCS_A.bit.
I can't reproduce this on msys2 locally. Have you tried using a relative path for SAMPLES?
Doesn't work. [in#0 @ 000001354318E3C0] Error opening input: No such file or directory Error opening input file concat;..\..\ffmpeg-fate-sample\hevc-conformance\LS_A_Orange_2.bit|..\..\ffmpeg-fate-sample\hevc\mv_nuh_layer_id.bit|..\..\ffmpeg-fate-sample\hevc-conformance\NoOutPrior_B_Qualcomm_1.bit|..\..\ffmpeg-fate-sample\hevc-conformance\MVHEVCS_A.bit. Error opening input files: No such file or directory threads=1 ffmpeg source code is at $HOME/work/ffmpeg. Build directory is $HOME/work/ffmpe/build. Fate sample is at $HOME/work/ffmpeg-fate-sample. The only workaround I found is copy fate sample to $HOME/work/ffmpe/build/ffmpeg-fate-sample, without absolute path or relative path prefix.
On 11/30/2024 2:19 PM, Zhao Zhili wrote:
-----Original Message----- From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of James Almer Sent: 2024年12月1日 0:51 To: ffmpeg-devel@ffmpeg.org Subject: Re: [FFmpeg-devel] [PATCH 42/42] tests/fate/hevc: add a test for switching between single and multi-view
On 11/30/2024 1:38 PM, Zhao Zhili wrote:
-----Original Message----- From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Anton Khirnov Sent: 2024年8月27日 23:05 To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH 42/42] tests/fate/hevc: add a test for switching between single and multi-view
--- tests/fate/hevc.mak | 10 ++ tests/ref/fate/hevc-mv-switch | 172 ++++++++++++++++++++++++++++++++++ 2 files changed, 182 insertions(+) create mode 100644 tests/ref/fate/hevc-mv-switch
diff --git a/tests/fate/hevc.mak b/tests/fate/hevc.mak index e75fbd3c16..5631ded860 100644 --- a/tests/fate/hevc.mak +++ b/tests/fate/hevc.mak @@ -273,6 +273,16 @@ FATE_HEVC-$(call FRAMECRC, HEVC, HEVC, HEVC_PARSER SCALE_FILTER) += fate-hevc-sm fate-hevc-mv-nuh-layer-id: CMD = framecrc -i $(TARGET_SAMPLES)/hevc/mv_nuh_layer_id.bit -map 0:view:all FATE_HEVC-$(call FRAMECRC, HEVC, HEVC) += fate-hevc-mv-nuh-layer-id
+# NB: $\ at the end of line joins lines without adding whitespace; +# this trick is recommended by GNU make manual +fate-hevc-mv-switch: INPUT = \ +$(TARGET_SAMPLES)/hevc-conformance/LS_A_Orange_2.bit|$\ +$(TARGET_SAMPLES)/hevc/mv_nuh_layer_id.bit|$\ +$(TARGET_SAMPLES)/hevc-conformance/NoOutPrior_B_Qualcomm_1.bit|$\ +$(TARGET_SAMPLES)/hevc-conformance/MVHEVCS_A.bit +fate-hevc-mv-switch: CMD = framecrc -i "concat:$(INPUT)" -fps_mode passthrough -map 0:vidx:0 -map 0:vidx:1 +FATE_HEVC-$(call FRAMECRC, HEVC, HEVC, CONCAT_PROTOCOL) += fate-hevc-mv-switch
The test failed when run with msys2, because msys2 only convert the first sample path to Windows style path:
It shouldn't convert any at all.
[in#0 @ 0000028FF3412EC0] Error opening input: No such file or directory Error opening input file
concat;D:\bin\msys\home\quink\work\ffmpeg-fate-sample\hevc-conformance\LS_A_Orange_2.bit|\home\quink\work\ffmpeg-fate-sam ple\hevc\mv_nuh_layer_id.bit|\home\quink\work\ffmpeg-fate-sample\hevc-conformance\NoOutPrior_B_Qualcomm_1.bit|\home\quin k\work\ffmpeg-fate-sample\hevc-conformance\MVHEVCS_A.bit.
I can't reproduce this on msys2 locally. Have you tried using a relative path for SAMPLES?
Doesn't work.
[in#0 @ 000001354318E3C0] Error opening input: No such file or directory Error opening input file concat;..\..\ffmpeg-fate-sample\hevc-conformance\LS_A_Orange_2.bit|..\..\ffmpeg-fate-sample\hevc\mv_nuh_layer_id.bit|..\..\ffmpeg-fate-sample\hevc-conformance\NoOutPrior_B_Qualcomm_1.bit|..\..\ffmpeg-fate-sample\hevc-conformance\MVHEVCS_A.bit. Error opening input files: No such file or directory threads=1
ffmpeg source code is at $HOME/work/ffmpeg. Build directory is $HOME/work/ffmpe/build. Fate sample is at $HOME/work/ffmpeg-fate-sample.
The only workaround I found is copy fate sample to $HOME/work/ffmpe/build/ffmpeg-fate-sample, without absolute path or relative path prefix.
Are you compiling using the mingw64, mingw32, clang64 or clang32 environment (And the corresponding gcc/clang package), or directly using the msys2 environment and its gcc/clang package? Because you should not use the latter.
_______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
在 2024年12月1日，上午1:22，James Almer <jamrial@gmail.com> 写道：
On 11/30/2024 2:19 PM, Zhao Zhili wrote:
-----Original Message----- From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of James Almer Sent: 2024年12月1日 0:51 To: ffmpeg-devel@ffmpeg.org Subject: Re: [FFmpeg-devel] [PATCH 42/42] tests/fate/hevc: add a test for switching between single and multi-view
On 11/30/2024 1:38 PM, Zhao Zhili wrote:
-----Original Message----- From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Anton Khirnov Sent: 2024年8月27日 23:05 To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH 42/42] tests/fate/hevc: add a test for switching between single and multi-view
--- tests/fate/hevc.mak | 10 ++ tests/ref/fate/hevc-mv-switch | 172 ++++++++++++++++++++++++++++++++++ 2 files changed, 182 insertions(+) create mode 100644 tests/ref/fate/hevc-mv-switch
diff --git a/tests/fate/hevc.mak b/tests/fate/hevc.mak index e75fbd3c16..5631ded860 100644 --- a/tests/fate/hevc.mak +++ b/tests/fate/hevc.mak @@ -273,6 +273,16 @@ FATE_HEVC-$(call FRAMECRC, HEVC, HEVC, HEVC_PARSER SCALE_FILTER) += fate-hevc-sm fate-hevc-mv-nuh-layer-id: CMD = framecrc -i $(TARGET_SAMPLES)/hevc/mv_nuh_layer_id.bit -map 0:view:all FATE_HEVC-$(call FRAMECRC, HEVC, HEVC) += fate-hevc-mv-nuh-layer-id
+# NB: $\ at the end of line joins lines without adding whitespace; +# this trick is recommended by GNU make manual +fate-hevc-mv-switch: INPUT = \ +$(TARGET_SAMPLES)/hevc-conformance/LS_A_Orange_2.bit|$\ +$(TARGET_SAMPLES)/hevc/mv_nuh_layer_id.bit|$\ +$(TARGET_SAMPLES)/hevc-conformance/NoOutPrior_B_Qualcomm_1.bit|$\ +$(TARGET_SAMPLES)/hevc-conformance/MVHEVCS_A.bit +fate-hevc-mv-switch: CMD = framecrc -i "concat:$(INPUT)" -fps_mode passthrough -map 0:vidx:0 -map 0:vidx:1 +FATE_HEVC-$(call FRAMECRC, HEVC, HEVC, CONCAT_PROTOCOL) += fate-hevc-mv-switch
The test failed when run with msys2, because msys2 only convert the first sample path to Windows style path:
It shouldn't convert any at all.
[in#0 @ 0000028FF3412EC0] Error opening input: No such file or directory Error opening input file
concat;D:\bin\msys\home\quink\work\ffmpeg-fate-sample\hevc-conformance\LS_A_Orange_2.bit|\home\quink\work\ffmpeg-fate-sam ple\hevc\mv_nuh_layer_id.bit|\home\quink\work\ffmpeg-fate-sample\hevc-conformance\NoOutPrior_B_Qualcomm_1.bit|\home\quin k\work\ffmpeg-fate-sample\hevc-conformance\MVHEVCS_A.bit.
I can't reproduce this on msys2 locally. Have you tried using a relative path for SAMPLES? Doesn't work. [in#0 @ 000001354318E3C0] Error opening input: No such file or directory Error opening input file concat;..\..\ffmpeg-fate-sample\hevc-conformance\LS_A_Orange_2.bit|..\..\ffmpeg-fate-sample\hevc\mv_nuh_layer_id.bit|..\..\ffmpeg-fate-sample\hevc-conformance\NoOutPrior_B_Qualcomm_1.bit|..\..\ffmpeg-fate-sample\hevc-conformance\MVHEVCS_A.bit. Error opening input files: No such file or directory threads=1 ffmpeg source code is at $HOME/work/ffmpeg. Build directory is $HOME/work/ffmpe/build. Fate sample is at $HOME/work/ffmpeg-fate-sample. The only workaround I found is copy fate sample to $HOME/work/ffmpe/build/ffmpeg-fate-sample, without absolute path or relative path prefix.
Are you compiling using the mingw64, mingw32, clang64 or clang32 environment (And the corresponding gcc/clang package), or directly using the msys2 environment and its gcc/clang package? Because you should not use the latter.
I’m using ucrt environment with MSVC compiler, started from visual studio developer command line promote.
_______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
_______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
<OpenPGP_signature.asc>
-----Original Message----- From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Zhao Zhili Sent: Samstag, 30. November 2024 19:07 To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org> Subject: Re: [FFmpeg-devel] [PATCH 42/42] tests/fate/hevc: add a test for switching between single and multi-view
在 2024年12月1日，上午1:22，James Almer <jamrial@gmail.com> 写道：
-----Original Message----- From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of James Almer Sent: 2024年12月1日 0:51 To: ffmpeg-devel@ffmpeg.org Subject: Re: [FFmpeg-devel] [PATCH 42/42] tests/fate/hevc: add a test for switching between single and multi-view
On 11/30/2024 1:38 PM, Zhao Zhili wrote:
> -----Original Message----- > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Anton
Khirnov
> Sent: 2024年8月27日 23:05 > To: ffmpeg-devel@ffmpeg.org > Subject: [FFmpeg-devel] [PATCH 42/42] tests/fate/hevc: add a test for switching between single and multi-view > > --- > tests/fate/hevc.mak | 10 ++ > tests/ref/fate/hevc-mv-switch | 172 ++++++++++++++++++++++++++++++++++ > 2 files changed, 182 insertions(+) > create mode 100644 tests/ref/fate/hevc-mv-switch > > diff --git a/tests/fate/hevc.mak b/tests/fate/hevc.mak > index e75fbd3c16..5631ded860 100644 > --- a/tests/fate/hevc.mak > +++ b/tests/fate/hevc.mak > @@ -273,6 +273,16 @@ FATE_HEVC-$(call FRAMECRC, HEVC, HEVC, HEVC_PARSER SCALE_FILTER) += fate-hevc-sm > fate-hevc-mv-nuh-layer-id: CMD = framecrc -i $(TARGET_SAMPLES)/hevc/mv_nuh_layer_id.bit -map 0:view:all > FATE_HEVC-$(call FRAMECRC, HEVC, HEVC) += fate-hevc-mv-nuh-layer-id > > +# NB: $\ at the end of line joins lines without adding whitespace; > +# this trick is recommended by GNU make manual > +fate-hevc-mv-switch: INPUT = \ > +$(TARGET_SAMPLES)/hevc-conformance/LS_A_Orange_2.bit|$\ > +$(TARGET_SAMPLES)/hevc/mv_nuh_layer_id.bit|$\ > +$(TARGET_SAMPLES)/hevc-conformance/NoOutPrior_B_Qualcomm_1.bit|$\ > +$(TARGET_SAMPLES)/hevc-conformance/MVHEVCS_A.bit > +fate-hevc-mv-switch: CMD = framecrc -i "concat:$(INPUT)" -fps_mode
> +FATE_HEVC-$(call FRAMECRC, HEVC, HEVC, CONCAT_PROTOCOL) += fate-hevc- mv-switch
The test failed when run with msys2, because msys2 only convert the first sample path to Windows style path:
It shouldn't convert any at all.
[in#0 @ 0000028FF3412EC0] Error opening input: No such file or directory Error opening input file
concat;D:\bin\msys\home\quink\work\ffmpeg-fate-sample\hevc- conformance\LS_A_Orange_2.bit|\home\quink\work\ffmpeg-fate-sam ple\hevc\mv_nuh_layer_id.bit|\home\quink\work\ffmpeg-fate-sample\hevc- conformance\NoOutPrior_B_Qualcomm_1.bit|\home\quin k\work\ffmpeg-fate-sample\hevc-conformance\MVHEVCS_A.bit.
I can't reproduce this on msys2 locally. Have you tried using a relative path for SAMPLES? Doesn't work. [in#0 @ 000001354318E3C0] Error opening input: No such file or directory Error opening input file concat;..\..\ffmpeg-fate-sample\hevc- conformance\LS_A_Orange_2.bit|..\..\ffmpeg-fate- sample\hevc\mv_nuh_layer_id.bit|..\..\ffmpeg-fate-sample\hevc- conformance\NoOutPrior_B_Qualcomm_1.bit|..\..\ffmpeg-fate-sample\hevc- conformance\MVHEVCS_A.bit. Error opening input files: No such file or directory threads=1 ffmpeg source code is at $HOME/work/ffmpeg. Build directory is $HOME/work/ffmpe/build. Fate sample is at $HOME/work/ffmpeg-fate-sample. The only workaround I found is copy fate sample to $HOME/work/ffmpe/build/ffmpeg-fate-sample, without absolute path or relative
On 11/30/2024 2:19 PM, Zhao Zhili wrote: passthrough -map 0:vidx:0 -map 0:vidx:1 path prefix.
Are you compiling using the mingw64, mingw32, clang64 or clang32 environment (And the corresponding gcc/clang package), or directly using the msys2 environment and its gcc/clang package? Because you should not use the latter.
I’m using ucrt environment with MSVC compiler, started from visual studio developer command line promote.
Hi Zhao and James, I encountered the same issue, specifically when working in MSYS2 shell directly. My patch "[PATCH 3/3] tests/hevc: Fix concat input when running in MSYS2 shell" from today fixes this. Best, sw
On 8/27/2024 12:04 PM, Anton Khirnov wrote:
One potential point of discussion is that the caller may want to know the view ID in its get_buffer2() callback (this is actually used in the ffmpeg CLI code). One potential option is attaching the view ID side data before calling get_buffer2(), but that is quite unusual and would require changes to the progress frame API. For now, I'm using another read-only decoder-private AVOption for that. Opinions welcome.
I just sent a patch for this. I need the same thing for LC-EVC support.
If nobody objects or wants more time for review, I'd like to push patches up to 24 (excluding 04 and 05) in a few days. That includes mainly * av_opt_set_array(), which we want anyway, whatever happens to the rest of the set; * adding HEVC multiview, which is trivial and independent of actual decoding; * moving per-layer HEVC decoder data into a per-layer context - the patches are all straightforward but constitute a lot of churn, so better get them out of the way sooner rather than later. -- Anton Khirnov
Quoting Anton Khirnov (2024-09-03 11:57:48)
If nobody objects or wants more time for review, I'd like to push patches up to 24 (excluding 04 and 05) in a few days. That includes mainly * av_opt_set_array(), which we want anyway, whatever happens to the rest of the set; * adding HEVC multiview, which is trivial and independent of actual decoding; * moving per-layer HEVC decoder data into a per-layer context - the patches are all straightforward but constitute a lot of churn, so better get them out of the way sooner rather than later.
Plan to push the patches mentioned (01-24 excluding 04 and 05) tomorrow if nobody objects. -- Anton Khirnov
Thanks for the patch set! Ran some initial tests and here are some findings: 1. Tested using the HTM ref SW encoded bitstreams; the HTM encoded 2 layer MV-HEVC elementary streams are decoded by both the HTM decoder and ffmpeg, and both resulted in the same decoded pixels for both views. 2. Tested with the iPhone captured sample videos (some samples are available from https://blog.frame.io/2024/02/01/how-to-capture-and-view-vision-pro-spatial-...). To decode using the HTM decoder, converted the .mov file to .hevc elementary stream (using "ffmpeg -view_ids -1 -i sample.mov -vcodec copy sample.hevc"). The iPhone captured MV-HEVC seems to be non-conformant as for the very first AU we have the following NAL units: prefix SEI (layer ID 0), CRA (layer ID 0), prefix SEI (layer ID 0), CRA (layer ID 1). Based on Section 7.4.2.4.4 of the HEVC spec, a prefix SEI with layer ID 0 indicates a new AU, so the bitstream is not conformant for the 2 layer case. The reference HTM decoder seems to be tolerant to this and correctly decodes the "non-conformant" elementary stream, but ffmpeg does not. However, ffmpeg does decode the .mov container file correctly as each AU is correctly passed to the decoder as a video sample. Aside from this con-conformance issue, both the HTM decoder and ffmpeg seem to produce pixel exact views. For the view by view comparison, I had to manually skip some initial frames output by the HTM decoder, as ffmpeg did not output some of the first few views/frames. 3. When ffmpeg skips some of the decoded views for output, I think it makes sense to skip both views of an AU (when both views are being output)? Currently one view from an AU can be skipped without skipping its corresponding other view. To avoid some of the views from being skipped, I had to use the "-vsync 0" option for my tests. When I use this option, I see the following warning: "Application provided invalid, non monotonically increasing dts to muxer in stream 0: 1 >= 1". This seems due to both views of an AU having the same DTS, which is expected. 4. Ran ffmpeg with the "-vf showinfo" option and the side data seems to indicate correct info for the views. I tested both cases where the base layer corresponds to either left or right view, and both cases seem to result in correct side data. BR, Danny
Quoting Danny Hong (2024-09-09 19:45:57)
Thanks for the patch set! Ran some initial tests and here are some findings:
1. Tested using the HTM ref SW encoded bitstreams; the HTM encoded 2 layer MV-HEVC elementary streams are decoded by both the HTM decoder and ffmpeg, and both resulted in the same decoded pixels for both views.
2. Tested with the iPhone captured sample videos (some samples are available from https://blog.frame.io/2024/02/01/how-to-capture-and-view-vision-pro-spatial-...). To decode using the HTM decoder, converted the .mov file to .hevc elementary stream (using "ffmpeg -view_ids -1 -i sample.mov -vcodec copy sample.hevc"). The iPhone captured MV-HEVC seems to be non-conformant as
Note that -view_ids is a decoder option and does nothing with streamcopy. Also, that option is intended for API users, with ffmpeg CLI you should be using view specifiers instead.
for the very first AU we have the following NAL units: prefix SEI (layer ID 0), CRA (layer ID 0), prefix SEI (layer ID 0), CRA (layer ID 1). Based on Section 7.4.2.4.4 of the HEVC spec, a prefix SEI with layer ID 0 indicates a new AU, so the bitstream is not conformant for the 2 layer case. The reference HTM decoder seems to be tolerant to this and correctly decodes the "non-conformant" elementary stream, but ffmpeg does not. However, ffmpeg does decode the .mov container file correctly as each AU is correctly passed to the decoder as a video sample. Aside from this con-conformance issue, both the HTM decoder and ffmpeg seem to produce pixel exact views. For the view by view comparison, I had to manually skip some initial frames output by the HTM decoder, as ffmpeg did not output some of the first few views/frames.
3. When ffmpeg skips some of the decoded views for output, I think it makes sense to skip both views of an AU (when both views are being output)? Currently one view from an AU can be skipped without skipping its corresponding other view. To avoid some of the views from being skipped, I had to use the "-vsync 0" option for my tests. When I use this option, I see the following warning: "Application provided invalid, non monotonically increasing dts to muxer in stream 0: 1 >= 1". This seems due to both views of an AU having the same DTS, which is expected.
Use view specifiers to split the views into their own streams. I do not want too much multiview-speficic logic into ffmpeg CLI, as it is a rather obscure feature. -- Anton Khirnov
Quoting Anton Khirnov (2024-09-13 12:44:55)
Note that -view_ids is a decoder option and does nothing with streamcopy. Also, that option is intended for API users, with ffmpeg CLI you should be using view specifiers instead.
Thanks! Tried with "-map 0:v:view:all" and the result is the same. The resulting elementary stream cannot be decoded by ffmpeg as the stream is not conformant.
Use view specifiers to split the views into their own streams. I do not want too much multiview-speficic logic into ffmpeg CLI, as it is a rather obscure feature.
But the "-map 0:v:view:all" option is still available to output both views as a single stream with alternating views, no? Using "-map 0:v:vidx:0 -f rawvideo out0.yuv420p -map 0:v:vidx:1 -f rawvideo out1.yuv420p" to split the views into separate streams seems to work fine without the issues mentioned before.
On 9/13/2024 2:58 PM, Danny Hong wrote:
Quoting Anton Khirnov (2024-09-13 12:44:55)
Note that -view_ids is a decoder option and does nothing with streamcopy. Also, that option is intended for API users, with ffmpeg CLI you should be using view specifiers instead.
Thanks! Tried with "-map 0:v:view:all" and the result is the same. The resulting elementary stream cannot be decoded by ffmpeg as the stream is not conformant.
Are you talking about decoding the resulting raw bitstream created from doing stream copy from the source mov? If so, what's happening is probably that the parser is splitting AUs when the second SEI with layer_id == 0 shows up in those non-conformant samples. This doesn't happen if you try to decode directly from the mov as the parser does not attempt to do any packetization then.
Use view specifiers to split the views into their own streams. I do not want too much multiview-speficic logic into ffmpeg CLI, as it is a rather obscure feature.
But the "-map 0:v:view:all" option is still available to output both views as a single stream with alternating views, no? Using "-map 0:v:vidx:0 -f rawvideo out0.yuv420p -map 0:v:vidx:1 -f rawvideo out1.yuv420p" to split the views into separate streams seems to work fine without the issues mentioned before. _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
Quoting James Almer (2024-09-13 22:19:35)
Are you talking about decoding the resulting raw bitstream created from doing stream copy from the source mov? If so, what's happening is probably that the parser is splitting AUs when the second SEI with layer_id == 0 shows up in those non-conformant samples. This doesn't happen if you try to decode directly from the mov as the parser does not attempt to do any packetization then.
Yes, this is due to hevc_find_frame_end (in libavcodec/hevc/parser.c) correctly marking a new AU when prefix NAL with layer ID 0 is detected. I think Apple is incorrectly placing the prefix NAL to precede the VCL NAL with layer ID 1.
On 9/13/2024 5:51 PM, Danny Hong wrote:
Quoting James Almer (2024-09-13 22:19:35)
Are you talking about decoding the resulting raw bitstream created from doing stream copy from the source mov? If so, what's happening is probably that the parser is splitting AUs when the second SEI with layer_id == 0 shows up in those non-conformant samples. This doesn't happen if you try to decode directly from the mov as the parser does not attempt to do any packetization then.
Yes, this is due to hevc_find_frame_end (in libavcodec/hevc/parser.c) correctly marking a new AU when prefix NAL with layer ID 0 is detected. I think Apple is incorrectly placing the prefix NAL to precede the VCL NAL with layer ID 1.
It's probably not misplaced but mistagged as nuh_layer_id == 0 when it should be 1 (Can SEI NALUs be in anything other than the base layer?). The SEI is User Data Unregistered, and its contents are different than the one preceding the base layer VCL NALU. so i guess it's some Apple specific metadata that applies to each corresponding view that they didn't bother to document. Can this issue be raised to them?
Quoting James Almer (2024-09-13 22:57:44)
It's probably not misplaced but mistagged as nuh_layer_id == 0 when it should be 1 (Can SEI NALUs be in anything other than the base layer?). The SEI is User Data Unregistered, and its contents are different than the one preceding the base layer VCL NALU. so i guess it's some Apple specific metadata that applies to each corresponding view that they didn't bother to document.
Yes, I believe SEI NALUs can have nuh_layer_id > 0, and if the SEI message is meant only for the view with nuh_layer_id = 1, then its nuh_layer_id should also be set to 1.
Quoting Danny Hong (2024-09-13 19:58:53)
Quoting Anton Khirnov (2024-09-13 12:44:55)
Note that -view_ids is a decoder option and does nothing with streamcopy. Also, that option is intended for API users, with ffmpeg CLI you should be using view specifiers instead.
Thanks! Tried with "-map 0:v:view:all" and the result is the same. The resulting elementary stream cannot be decoded by ffmpeg as the stream is not conformant.
Use view specifiers to split the views into their own streams. I do not want too much multiview-speficic logic into ffmpeg CLI, as it is a rather obscure feature.
But the "-map 0:v:view:all" option is still available to output both views as a single stream with alternating views, no?
It is, but mainly as a testing/development aid, I do not expect it to be generally useful. Or is there a particular reason you need to interleave the views into one stream? -- Anton Khirnov
Quoting Anton Khirnov (2024-09-14 11:01:27)
It is, but mainly as a testing/development aid, I do not expect it to be generally useful. Or is there a particular reason you need to interleave the views into one stream?
Understood. Since we can extract both views synchronized with the view specifier, I guess we don't need the above option. I was just testing it because the option was available. Thanks!
participants (6)
-
Anton Khirnov -
Danny Hong -
James Almer -
Michael Niedermayer -
softworkz . -
Zhao Zhili