【x265】预测模块的简单分析—帧间预测

x265相关：
【x265】x265编码器参数配置
【x265】预测模块的简单分析—帧内预测

1. 帧间预测概述

1.1 编码块结构

帧间预测是编码器中降低编码耗时和编码码率的最有效工具之一，通过时域上的相邻参考，能够大幅度降低编码码率，从而节省网络带宽。在x265当中，帧间预测（Inter Prediction，下文简称Inter模式）是基于PU实现和操作的，它能够将一个CU划分成为若干个子区域，分别实现预测功能，与帧内预测（Intra Prediction，下文简称Intra模式）不同，Inter模式能够将CU分成不规则的PU尺寸，如下所示，一共8种

//    2Nx2N               2NxN 				   Nx2N				   NxN
+---+---+---+---+ 	+---+---+---+---+ 	+---+---+---+---+ 	+---+---+---+---+ 
|               |	|   			|	|       |       |	|       |       |1. 			+	+				+	+		+		+	+		+		+
|   			|   |   			|   |    	|       |   |       |       |2. 			+	+---+---+---+---+	+       +		+	+---+---+---+---+
|   			|	|   			|	|       |       |	|       |       |3. 			+	+				+	+		+		+	+		+		+
|   			|	|   			|	|       |       |	|       |       |
+---+---+---+---+	+---+---+---+---+	+---+---+---+---+	+---+---+---+---+
//    2NxnU				  2NxnD				  nLx2N              nRx2N
+---+---+---+---+ 	+---+---+---+---+ 	+---+---+---+---+ 	+---+---+---+---+ 
|               |	|   			|	|   |   	    |	|           |   |
+---+---+---+---+	+				+	+	+			+	+		    +	+
|   			|   |   			|   |   |       	|   |       	|   |4. 			+	+				+	+   +			+	+			+	+
|   			|	|   			|	|   |       	|	|       	|   |5. 			+	+---+---+---+---+	+	+			+	+			+	+
|   			|	|   			|	|   |       	|	|       	|   |
+---+---+---+---+	+---+---+---+---+	+---+---+---+---+	+---+---+---+---+

1.2 运动估计

通常在视频播放时，前后帧具有比较强的关联性，一个比较好的思考是，在前后图像中找到两个很相似的块，并利用一个运动偏移量来描述这两个块之间的位置差异，前面的帧编码图像块，后续的帧只对这个位置偏移量进行编码，就能够节省编码比特。找到这个运动偏移量的过程叫做运动估计（Motion Estimation，ME），为了找到这两个很相似的块，需要考虑两个问题：

1. 如何描述这两个块的差异程度
2. 如何高效的去找到这两个块

1.2.1 运动估计准则

在Inter模式中，描述参考块（下称refBlock）和当前块（下称curBlock）的方式主要为SAD和SATD，另外需要加上对应MV使用的比特开销，即率失真优化公式 J = D + lambda * R

1.2.2 运动搜索

在x265中使用的运动搜索（Motion Search，MS）分为几个步骤：

1. 整像素搜索
   (1) 菱形搜索（X265_DIA_SEARCH）
   (1) 六边形搜索（X265_HEX_SEARCH）
   这两种搜索方式和x264当中的类似，可以参考雷博的文章：x264源代码简单分析：宏块分析（Analysis）部分-帧间宏块（Inter）。不同之处在于，如果使用HEX搜索，在x265中还会多循环几次，使用半HEX快速搜索，扩大搜索范围，因为x265当中CU的尺寸要更大一些
2. 1/2像素搜索
3. 1/4像素搜索

PS：整像素搜索使用的是SAD来描述损失大小，1/2和1/4像素搜索使用的是SATD来描述损失大小。另外，不使用1/8像素搜索的原因是带来的性能增益不明显

1.3 MV预测技术

在Inter模式中，使用了Merge和AMVP两项技术，辅助实现更好的Inter编码。其中，Merge技术可以看作成一种编码模式，在x265中有专门的宏定义这种模式，并且在实际编码时也会将merge相关信息写入码流（例如m_entropyCoder.codeMergeIndex(cu, 0)），不存在MVD（MV Difference）；而AMVP技术可以看成一种MV预测技术，编码器只需要对实际MV和预测MV的差值进行编码，因此是存在MVD的

1.3.1 Merge模式

Merge模式为当前PU构建一个MV候选列表，这个候选列表存在5个候选MV。通过遍历这个列表，从5个候选MV中选择一个最佳的MV作为Merge模式的MV，merge mv会在后续的帧间预测流程中提供有力指导。

Merge列表的构建分为空域候选列表和时域候选列表两个部分：

1. 空域候选列表的构建
   空域候选列表的构建顺序 = { A1, B1, B0, A0, B2 }，列表从左到右进行顺序构建，空域候选列表至多包含4个候选MV
   
   对于下列使用矩形划分方式中的PU 2，其候选模式需要做额外处理。下图（a）中的情形，PU2的候选列表中不能存在A1的运动信息，因为如果PU2使用了A1（即PU1）中的信息，则PU1和PU2的MV会一样，这与2NxN的划分方式就没有区别了。同理，对于图（b）中的情形，PU2的列表中不能存在B1的运动信息
   
2. 时域候选列表的建立
   时域MV候选列表的建立利用了当前PU在邻近已编码图像中对应位置PU（同位PU）的运动信息，但不是直接使用，而是根据当前帧与参考帧的相对位置做对应的比例伸缩调整。图示如下，其中cur_PU为当前预测PU，col_PU为相邻已编码帧的同位PU，cur_ref为当前帧的参考帧，col_ref为相邻已编码帧的参考帧
   
   当前PU的时域候选MV的计算公式为
   $$curMV=\frac{td}{tb}colMV$$
   时域候选列表中同位块的位置位于右下角H块，如果H块不存在，则使用C3来代替。时域候选列表最多只提供1个候选MV
   

PS：如果merge模式前面两步的操作之后，候选列表不足5个，就填充(0, 0)

1.3.2 AMVP技术

AMVP技术与merge有类似之处，同样使用了空域和时域上运动向量的相关性。

1. 空域候选列表的建立
   沿用merge模式使用的相邻块编号，AMVP空域候选列表分别从左侧和上方各产生一个候选预测MV，左侧选择的顺序 = { A0, A1, scaled A0, scaled A1 }，上方选择的顺序 = { B0, B1, B2, (scaled B0, scaled B1, scaled B2) }，这里的scaled和merge中利用同位块计算当前块MV的方式相同。对于上方选择的顺序而言，MV的比例伸缩只有在左侧两个PU都不可用或者都是Intra模式时才会进行。同时，只有当相邻块候选MV指向的参考帧与当前PU相同时，才可以直接使用相邻MV，否则需要对其进行scale

   另外，AMVP技术中空域候选列表至多包含2个候选MV（merge模式至多包含4个）

2. 时域候选列表的建立
   与merge模式构建的方式一致

PS：如果AMVP技术经过前面两个步骤之后，候选列表中不足2个候选MV，就填充(0, 0)。另外，AMVP技术在实际编码时，会对MV进行差分编码，即只编码MVD

2. 帧间预测入口函数（compressInterCU_rd0_4）

在x265的帧间预测入口函数中，仅简单分析compressInterCU_rd0_4()，函数中的0和4表示如果rdLevel位于0~4之间则使用这个函数，因为默认的配置中rdLevel=3，所以默认会使用这个函数进行帧间预测

函数的定义位于encoder\analysis.cpp中，其主要的工作流程为
（1）评估使用merge和skip模式带来的损失（checkMerge2Nx2N_rd0_4）
（2）评估划分成为4个子块带来的损失（递归调用compressInterCU_rd0_4）
（3）评估当前深度各种划分模式和Intra模式带来的损失（checkInter_rd0_4，checkIntraInInter）

SplitData Analysis::compressInterCU_rd0_4(const CUData& parentCTU, const CUGeom& cuGeom, int32_t qp)
{if (parentCTU.m_vbvAffected && calculateQpforCuSize(parentCTU, cuGeom, 1))return compressInterCU_rd5_6(parentCTU, cuGeom, qp);uint32_t depth = cuGeom.depth;uint32_t cuAddr = parentCTU.m_cuAddr;ModeDepth& md = m_modeDepth[depth];// searchMethod默认为X265_HEX_SEARCHif (m_param->searchMethod == X265_SEA){int numPredDir = m_slice->isInterP() ? 1 : 2;int offset = (int)(m_frame->m_reconPic->m_cuOffsetY[parentCTU.m_cuAddr] + m_frame->m_reconPic->m_buOffsetY[cuGeom.absPartIdx]);for (int list = 0; list < numPredDir; list++)for (int i = 0; i < m_frame->m_encData->m_slice->m_numRefIdx[list]; i++)for (int planes = 0; planes < INTEGRAL_PLANE_NUM; planes++)m_modeDepth[depth].fencYuv.m_integral[list][i][planes] = m_frame->m_encData->m_slice->m_refFrameList[list][i]->m_encData->m_meIntegral[planes] + offset;}PicYuv& reconPic = *m_frame->m_reconPic;SplitData splitCUData;// 是否进行hevc的分析（x265似乎对AVC做了兼容）bool bHEVCBlockAnalysis = (m_param->bAnalysisType == AVC_INFO && cuGeom.numPartitions > 16);// 是否进行avc分析的refinebool bRefineAVCAnalysis = (m_param->analysisLoadReuseLevel == 7 && (m_modeFlag[0] || m_modeFlag[1]));// no-off loading，如果为true，表示不会将CPU当中的任务移动到其他处理器（如GPU等）上面进行bool bNooffloading = !(m_param->bAnalysisType == AVC_INFO);if (bHEVCBlockAnalysis || bRefineAVCAnalysis || bNooffloading){md.bestMode = NULL;bool mightSplit = !(cuGeom.flags & CUGeom::LEAF);bool mightNotSplit = !(cuGeom.flags & CUGeom::SPLIT_MANDATORY);uint32_t minDepth = topSkipMinDepth(parentCTU, cuGeom);bool bDecidedDepth = parentCTU.m_cuDepth[cuGeom.absPartIdx] == depth;bool skipModes = false; /* Skip any remaining mode analyses at current depth */bool skipRecursion = false; /* Skip recursion */bool splitIntra = true;bool skipRectAmp = false;bool chooseMerge = false;bool bCtuInfoCheck = false;int sameContentRef = 0;if (m_evaluateInter){if (m_refineLevel == 2){if (parentCTU.m_predMode[cuGeom.absPartIdx] == MODE_SKIP)skipModes = true;if (parentCTU.m_partSize[cuGeom.absPartIdx] == SIZE_2Nx2N)skipRectAmp = true;}mightSplit &= false;minDepth = depth;}if ((m_limitTU & X265_TU_LIMIT_NEIGH) && cuGeom.log2CUSize >= 4)m_maxTUDepth = loadTUDepth(cuGeom, parentCTU);SplitData splitData[4];splitData[0].initSplitCUData();splitData[1].initSplitCUData();splitData[2].initSplitCUData();splitData[3].initSplitCUData();// avoid uninitialize value in below referenceif (m_param->limitModes){md.pred[PRED_2Nx2N].bestME[0][0].mvCost = 0; // L0md.pred[PRED_2Nx2N].bestME[0][1].mvCost = 0; // L1md.pred[PRED_2Nx2N].sa8dCost = 0;}if (m_param->bCTUInfo && depth <= parentCTU.m_cuDepth[cuGeom.absPartIdx]){if (bDecidedDepth && m_additionalCtuInfo[cuGeom.absPartIdx])sameContentRef = findSameContentRefCount(parentCTU, cuGeom);if (depth < parentCTU.m_cuDepth[cuGeom.absPartIdx]){mightNotSplit &= bDecidedDepth;bCtuInfoCheck = skipRecursion = false;skipModes = true;}else if (mightNotSplit && bDecidedDepth){if (m_additionalCtuInfo[cuGeom.absPartIdx]){bCtuInfoCheck = skipRecursion = true;md.pred[PRED_MERGE].cu.initSubCU(parentCTU, cuGeom, qp);md.pred[PRED_SKIP].cu.initSubCU(parentCTU, cuGeom, qp);checkMerge2Nx2N_rd0_4(md.pred[PRED_SKIP], md.pred[PRED_MERGE], cuGeom);if (!sameContentRef){if ((m_param->bCTUInfo & 2) && (m_slice->m_pps->bUseDQP && depth <= m_slice->m_pps->maxCuDQPDepth)){qp -= int32_t(0.04 * qp);setLambdaFromQP(parentCTU, qp);}if (m_param->bCTUInfo & 4)skipModes = false;}if (sameContentRef || (!sameContentRef && !(m_param->bCTUInfo & 4))){if (m_param->rdLevel)skipModes = m_param->bEnableEarlySkip && md.bestMode && md.bestMode->cu.isSkipped(0);if ((m_param->bCTUInfo & 4) && sameContentRef)skipModes = md.bestMode && true;}}else{md.pred[PRED_MERGE].cu.initSubCU(parentCTU, cuGeom, qp);md.pred[PRED_SKIP].cu.initSubCU(parentCTU, cuGeom, qp);checkMerge2Nx2N_rd0_4(md.pred[PRED_SKIP], md.pred[PRED_MERGE], cuGeom);if (m_param->rdLevel)skipModes = m_param->bEnableEarlySkip && md.bestMode && md.bestMode->cu.isSkipped(0);}mightSplit &= !bDecidedDepth;}}if ((m_param->analysisLoadReuseLevel > 1 && m_param->analysisLoadReuseLevel != 10)){if (mightNotSplit && depth == m_reuseDepth[cuGeom.absPartIdx]){if (m_reuseModes[cuGeom.absPartIdx] == MODE_SKIP){md.pred[PRED_MERGE].cu.initSubCU(parentCTU, cuGeom, qp);md.pred[PRED_SKIP].cu.initSubCU(parentCTU, cuGeom, qp);checkMerge2Nx2N_rd0_4(md.pred[PRED_SKIP], md.pred[PRED_MERGE], cuGeom);skipRecursion = !!m_param->recursionSkipMode && md.bestMode;if (m_param->rdLevel)skipModes = m_param->bEnableEarlySkip && md.bestMode;}if (m_param->analysisLoadReuseLevel > 4 && m_reusePartSize[cuGeom.absPartIdx] == SIZE_2Nx2N){if (m_reuseModes[cuGeom.absPartIdx] != MODE_INTRA  && m_reuseModes[cuGeom.absPartIdx] != 4){skipRectAmp = true && !!md.bestMode;chooseMerge = !!m_reuseMergeFlag[cuGeom.absPartIdx] && !!md.bestMode;}}}}if (m_param->analysisMultiPassRefine && m_param->rc.bStatRead && m_reuseInterDataCTU) {if (mightNotSplit && depth == m_reuseDepth[cuGeom.absPartIdx]){if (m_reuseModes[cuGeom.absPartIdx] == MODE_SKIP){md.pred[PRED_MERGE].cu.initSubCU(parentCTU, cuGeom, qp);md.pred[PRED_SKIP].cu.initSubCU(parentCTU, cuGeom, qp);checkMerge2Nx2N_rd0_4(md.pred[PRED_SKIP], md.pred[PRED_MERGE], cuGeom);skipRecursion = !!m_param->recursionSkipMode && md.bestMode;if (m_param->rdLevel)skipModes = m_param->bEnableEarlySkip && md.bestMode;}}}/* Step 1. Evaluate Merge/Skip candidates for likely early-outs, if skip mode was not set above */// 1. 对Merge、Skip候选模式进行评估以确定是否可以提前终止某些计算过程（如果skip模式在前面没有配置）if ((mightNotSplit && depth >= minDepth && !md.bestMode && !bCtuInfoCheck) || (m_param->bAnalysisType == AVC_INFO && m_param->analysisLoadReuseLevel == 7 && (m_modeFlag[0] || m_modeFlag[1])))/* TODO: Re-evaluate if analysis load/save still works */{/* Compute Merge Cost */// 初始化merge和skip模式的CUmd.pred[PRED_MERGE].cu.initSubCU(parentCTU, cuGeom, qp);md.pred[PRED_SKIP].cu.initSubCU(parentCTU, cuGeom, qp);// 进行merge模式和skip模式的帧间预测checkMerge2Nx2N_rd0_4(md.pred[PRED_SKIP], md.pred[PRED_MERGE], cuGeom);if (m_param->rdLevel)skipModes = (m_param->bEnableEarlySkip || m_refineLevel == 2)&& md.bestMode && md.bestMode->cu.isSkipped(0); // TODO: sa8d threshold per depth}if (md.bestMode && m_param->recursionSkipMode && !bCtuInfoCheck && !(m_param->bAnalysisType == AVC_INFO && m_param->analysisLoadReuseLevel == 7 && (m_modeFlag[0] || m_modeFlag[1]))){skipRecursion = md.bestMode->cu.isSkipped(0);if (mightSplit && !skipRecursion){if (depth >= minDepth && m_param->recursionSkipMode == RDCOST_BASED_RSKIP){if (depth)skipRecursion = recursionDepthCheck(parentCTU, cuGeom, *md.bestMode);if (m_bHD && !skipRecursion && m_param->rdLevel == 2 && md.fencYuv.m_size != MAX_CU_SIZE)skipRecursion = complexityCheckCU(*md.bestMode);}else if (cuGeom.log2CUSize >= MAX_LOG2_CU_SIZE - 1 && m_param->recursionSkipMode == EDGE_BASED_RSKIP){skipRecursion = complexityCheckCU(*md.bestMode);}}}// 检查是否需要跳过递归划分if (m_param->bAnalysisType == AVC_INFO && md.bestMode && cuGeom.numPartitions <= 16 && m_param->analysisLoadReuseLevel == 7)skipRecursion = true;/* Step 2. Evaluate each of the 4 split sub-blocks in series */// 评估4个子块的Inter模式if (mightSplit && !skipRecursion){if (bCtuInfoCheck && m_param->bCTUInfo & 2)qp = int((1 / 0.96) * qp + 0.5);Mode* splitPred = &md.pred[PRED_SPLIT];splitPred->initCosts();CUData* splitCU = &splitPred->cu;splitCU->initSubCU(parentCTU, cuGeom, qp);uint32_t nextDepth = depth + 1;ModeDepth& nd = m_modeDepth[nextDepth];invalidateContexts(nextDepth);Entropy* nextContext = &m_rqt[depth].cur;int nextQP = qp;splitIntra = false;for (uint32_t subPartIdx = 0; subPartIdx < 4; subPartIdx++){const CUGeom& childGeom = *(&cuGeom + cuGeom.childOffset + subPartIdx);if (childGeom.flags & CUGeom::PRESENT){m_modeDepth[0].fencYuv.copyPartToYuv(nd.fencYuv, childGeom.absPartIdx);m_rqt[nextDepth].cur.load(*nextContext);if (m_slice->m_pps->bUseDQP && nextDepth <= m_slice->m_pps->maxCuDQPDepth)nextQP = setLambdaFromQP(parentCTU, calculateQpforCuSize(parentCTU, childGeom));// 进行4个子块的帧间预测splitData[subPartIdx] = compressInterCU_rd0_4(parentCTU, childGeom, nextQP);// Save best CU and pred data for this sub CUsplitIntra |= nd.bestMode->cu.isIntra(0);splitCU->copyPartFrom(nd.bestMode->cu, childGeom, subPartIdx);splitPred->addSubCosts(*nd.bestMode);if (m_param->rdLevel)nd.bestMode->reconYuv.copyToPartYuv(splitPred->reconYuv, childGeom.numPartitions * subPartIdx);elsend.bestMode->predYuv.copyToPartYuv(splitPred->predYuv, childGeom.numPartitions * subPartIdx);if (m_param->rdLevel > 1)nextContext = &nd.bestMode->contexts;}elsesplitCU->setEmptyPart(childGeom, subPartIdx);}nextContext->store(splitPred->contexts);if (mightNotSplit)addSplitFlagCost(*splitPred, cuGeom.depth);else if (m_param->rdLevel > 1)updateModeCost(*splitPred);elsesplitPred->sa8dCost = m_rdCost.calcRdSADCost((uint32_t)splitPred->distortion, splitPred->sa8dBits);}/* If analysis mode is simple do not Evaluate other modes */if (m_param->bAnalysisType == AVC_INFO && m_param->analysisLoadReuseLevel == 7){if (m_slice->m_sliceType == P_SLICE){if (m_checkMergeAndSkipOnly[0])skipModes = true;}else{if (m_checkMergeAndSkipOnly[0] && m_checkMergeAndSkipOnly[1])skipModes = true;}}/* Split CUs*   0  1*   2  3 */uint32_t allSplitRefs = splitData[0].splitRefs | splitData[1].splitRefs | splitData[2].splitRefs | splitData[3].splitRefs;/* Step 3. Evaluate ME (2Nx2N, rect, amp) and intra modes at current depth */// 评估当前深度的ME和intra模式if (mightNotSplit && (depth >= minDepth || (m_param->bCTUInfo && !md.bestMode))){if (m_slice->m_pps->bUseDQP && depth <= m_slice->m_pps->maxCuDQPDepth && m_slice->m_pps->maxCuDQPDepth != 0)setLambdaFromQP(parentCTU, qp);// /*检查是否是skip模式（1）如果是skip模式，跳过当前深度的inter prediction（2）如果不是skip模式，进入下面的inter prediction，会按照顺序去检查各种划分方式（a）2Nx2N（b）矩形划分（i）  2NxN, Nx2N（ii） 2NxnD, 2NxnU（iii）nRx2N, nLx2N*/if (!skipModes){uint32_t refMasks[2];refMasks[0] = allSplitRefs;md.pred[PRED_2Nx2N].cu.initSubCU(parentCTU, cuGeom, qp);// 2Nx2N的帧间预测checkInter_rd0_4(md.pred[PRED_2Nx2N], cuGeom, SIZE_2Nx2N, refMasks);if (m_param->limitReferences & X265_REF_LIMIT_CU){CUData& cu = md.pred[PRED_2Nx2N].cu;uint32_t refMask = cu.getBestRefIdx(0);allSplitRefs = splitData[0].splitRefs = splitData[1].splitRefs = splitData[2].splitRefs = splitData[3].splitRefs = refMask;}// B帧的2Nx2N帧间预测（没有研究）if (m_slice->m_sliceType == B_SLICE){md.pred[PRED_BIDIR].cu.initSubCU(parentCTU, cuGeom, qp);checkBidir2Nx2N(md.pred[PRED_2Nx2N], md.pred[PRED_BIDIR], cuGeom);}Mode *bestInter = &md.pred[PRED_2Nx2N];// 检查是否进行rect模式预测，即矩形划分方式if (!skipRectAmp){/*2NxN划分		 Nx2N划分+---+---+		+---+---+|       |		|   |   |+---+---+		+   +   +|       |		|	|	|+---+---+		+---+---+*/// 检查是否允许进行矩形分割（非正方形）if (m_param->bEnableRectInter){// 计算划分成为4个子块的总损失uint64_t splitCost = splitData[0].sa8dCost + splitData[1].sa8dCost + splitData[2].sa8dCost + splitData[3].sa8dCost;uint32_t threshold_2NxN, threshold_Nx2N;/*（1）如果是P帧，取出前向cost（2）如果是B帧，求前后向的平均cost*/if (m_slice->m_sliceType == P_SLICE){threshold_2NxN = splitData[0].mvCost[0] + splitData[1].mvCost[0];threshold_Nx2N = splitData[0].mvCost[0] + splitData[2].mvCost[0];}else{threshold_2NxN = (splitData[0].mvCost[0] + splitData[1].mvCost[0]+ splitData[0].mvCost[1] + splitData[1].mvCost[1] + 1) >> 1;threshold_Nx2N = (splitData[0].mvCost[0] + splitData[2].mvCost[0]+ splitData[0].mvCost[1] + splitData[2].mvCost[1] + 1) >> 1;}/*下面代码的逻辑（1）如果try_2NxN_first = true，则按照检查顺序的1和2执行（2）如果try_Nx2N_first = true, 则按照检查顺序的2和3执行*/int try_2NxN_first = threshold_2NxN < threshold_Nx2N;/*检查顺序1splitCost:划分成为4个子块的损失md.pred[PRED_2Nx2N].sa8dCost:按照2Nx2N模式进行预测的损失threshold_2NxN:划分成2NxN的阈值如果满足下面的不等式关系，表示使用2NxN有可能损失更小*/if (try_2NxN_first && splitCost < md.pred[PRED_2Nx2N].sa8dCost + threshold_2NxN){// 上半部分refMasks[0] = splitData[0].splitRefs | splitData[1].splitRefs; /* top */// 下半部分refMasks[1] = splitData[2].splitRefs | splitData[3].splitRefs; /* bot */md.pred[PRED_2NxN].cu.initSubCU(parentCTU, cuGeom, qp);// 检查2NxN帧间预测损失checkInter_rd0_4(md.pred[PRED_2NxN], cuGeom, SIZE_2NxN, refMasks);if (md.pred[PRED_2NxN].sa8dCost < bestInter->sa8dCost)bestInter = &md.pred[PRED_2NxN];}/*检查顺序2splitCost:划分成为4个子块的损失md.pred[PRED_2Nx2N].sa8dCost:按照2Nx2N模式进行预测的损失threshold_Nx2N:划分成Nx2N的阈值如果满足下面的不等式关系，表示使用Nx2N有可能损失更小*/if (splitCost < md.pred[PRED_2Nx2N].sa8dCost + threshold_Nx2N){refMasks[0] = splitData[0].splitRefs | splitData[2].splitRefs; /* left */refMasks[1] = splitData[1].splitRefs | splitData[3].splitRefs; /* right */md.pred[PRED_Nx2N].cu.initSubCU(parentCTU, cuGeom, qp);checkInter_rd0_4(md.pred[PRED_Nx2N], cuGeom, SIZE_Nx2N, refMasks);if (md.pred[PRED_Nx2N].sa8dCost < bestInter->sa8dCost)bestInter = &md.pred[PRED_Nx2N];}// 检查顺序3if (!try_2NxN_first && splitCost < md.pred[PRED_2Nx2N].sa8dCost + threshold_2NxN){refMasks[0] = splitData[0].splitRefs | splitData[1].splitRefs; /* top */refMasks[1] = splitData[2].splitRefs | splitData[3].splitRefs; /* bot */md.pred[PRED_2NxN].cu.initSubCU(parentCTU, cuGeom, qp);checkInter_rd0_4(md.pred[PRED_2NxN], cuGeom, SIZE_2NxN, refMasks);if (md.pred[PRED_2NxN].sa8dCost < bestInter->sa8dCost)bestInter = &md.pred[PRED_2NxN];}}// 检查(SIZE_2NxnU, SIZE_2NxnD, SIZE_nLx2N, SIZE_nRx2N)if (m_slice->m_sps->maxAMPDepth > depth){uint64_t splitCost = splitData[0].sa8dCost + splitData[1].sa8dCost + splitData[2].sa8dCost + splitData[3].sa8dCost;uint32_t threshold_2NxnU, threshold_2NxnD, threshold_nLx2N, threshold_nRx2N;// 根据帧类型获取thresholdif (m_slice->m_sliceType == P_SLICE){threshold_2NxnU = splitData[0].mvCost[0] + splitData[1].mvCost[0];threshold_2NxnD = splitData[2].mvCost[0] + splitData[3].mvCost[0];threshold_nLx2N = splitData[0].mvCost[0] + splitData[2].mvCost[0];threshold_nRx2N = splitData[1].mvCost[0] + splitData[3].mvCost[0];}else{threshold_2NxnU = (splitData[0].mvCost[0] + splitData[1].mvCost[0]+ splitData[0].mvCost[1] + splitData[1].mvCost[1] + 1) >> 1;threshold_2NxnD = (splitData[2].mvCost[0] + splitData[3].mvCost[0]+ splitData[2].mvCost[1] + splitData[3].mvCost[1] + 1) >> 1;threshold_nLx2N = (splitData[0].mvCost[0] + splitData[2].mvCost[0]+ splitData[0].mvCost[1] + splitData[2].mvCost[1] + 1) >> 1;threshold_nRx2N = (splitData[1].mvCost[0] + splitData[3].mvCost[0]+ splitData[1].mvCost[1] + splitData[3].mvCost[1] + 1) >> 1;}/*检查是否进行水平或者垂直的划分（1）如果partSize = 2Nx2N，则进行水平划分尝试（2）如果partSize = Nx2N，则进行垂直划分尝试（3）如果partSize = 2Nx2N，并且四叉树根节点有非零系数，则同时采用水平和垂直划分尝试*/bool bHor = false, bVer = false;if (bestInter->cu.m_partSize[0] == SIZE_2NxN)bHor = true;else if (bestInter->cu.m_partSize[0] == SIZE_Nx2N)bVer = true;else if (bestInter->cu.m_partSize[0] == SIZE_2Nx2N &&md.bestMode && md.bestMode->cu.getQtRootCbf(0)){bHor = true;bVer = true;}// 尝试水平划分if (bHor){// 检查2NxnD是否优先，确定检查顺序/*2NxnD						2NxnU+--+--+--+--+				+--+--+--+--+|			|				|			|	25% top+			+				+--+--+--+--+|			|	75% top		|			|+			+				+			+|			|				|			|	75% bottom+--+--+--+--+				+			+|			|	25% bottom	|			|+--+--+--+--+				+--+--+--+--+*/int try_2NxnD_first = threshold_2NxnD < threshold_2NxnU;if (try_2NxnD_first && splitCost < md.pred[PRED_2Nx2N].sa8dCost + threshold_2NxnD){refMasks[0] = allSplitRefs;                                    /* 75% top */refMasks[1] = splitData[2].splitRefs | splitData[3].splitRefs; /* 25% bot */md.pred[PRED_2NxnD].cu.initSubCU(parentCTU, cuGeom, qp);// 检查2NxnDcheckInter_rd0_4(md.pred[PRED_2NxnD], cuGeom, SIZE_2NxnD, refMasks);if (md.pred[PRED_2NxnD].sa8dCost < bestInter->sa8dCost)bestInter = &md.pred[PRED_2NxnD];}if (splitCost < md.pred[PRED_2Nx2N].sa8dCost + threshold_2NxnU){refMasks[0] = splitData[0].splitRefs | splitData[1].splitRefs; /* 25% top */refMasks[1] = allSplitRefs;                                    /* 75% bot */md.pred[PRED_2NxnU].cu.initSubCU(parentCTU, cuGeom, qp);// 检查2NxnUcheckInter_rd0_4(md.pred[PRED_2NxnU], cuGeom, SIZE_2NxnU, refMasks);if (md.pred[PRED_2NxnU].sa8dCost < bestInter->sa8dCost)bestInter = &md.pred[PRED_2NxnU];}if (!try_2NxnD_first && splitCost < md.pred[PRED_2Nx2N].sa8dCost + threshold_2NxnD){refMasks[0] = allSplitRefs;                                    /* 75% top */refMasks[1] = splitData[2].splitRefs | splitData[3].splitRefs; /* 25% bot */md.pred[PRED_2NxnD].cu.initSubCU(parentCTU, cuGeom, qp);checkInter_rd0_4(md.pred[PRED_2NxnD], cuGeom, SIZE_2NxnD, refMasks);if (md.pred[PRED_2NxnD].sa8dCost < bestInter->sa8dCost)bestInter = &md.pred[PRED_2NxnD];}}// 尝试垂直划分if (bVer){	/*nRx2N75% left       25% left+--+--+--+--+    +--+--+--+--+|		 |  |	 |  |        |+        +  +	 +  +        +|        |  |	 |  |        |+        +  +	 +  +        +|        |  |	 |  |        |+        +  +    +  +        +|        |  |    |  |        |+--+--+--+--+	 +--+--+--+--+25% right     75% right*/int try_nRx2N_first = threshold_nRx2N < threshold_nLx2N;if (try_nRx2N_first && splitCost < md.pred[PRED_2Nx2N].sa8dCost + threshold_nRx2N){refMasks[0] = allSplitRefs;                                    /* 75% left  */refMasks[1] = splitData[1].splitRefs | splitData[3].splitRefs; /* 25% right */md.pred[PRED_nRx2N].cu.initSubCU(parentCTU, cuGeom, qp);checkInter_rd0_4(md.pred[PRED_nRx2N], cuGeom, SIZE_nRx2N, refMasks);if (md.pred[PRED_nRx2N].sa8dCost < bestInter->sa8dCost)bestInter = &md.pred[PRED_nRx2N];}if (splitCost < md.pred[PRED_2Nx2N].sa8dCost + threshold_nLx2N){refMasks[0] = splitData[0].splitRefs | splitData[2].splitRefs; /* 25% left  */refMasks[1] = allSplitRefs;                                    /* 75% right */md.pred[PRED_nLx2N].cu.initSubCU(parentCTU, cuGeom, qp);checkInter_rd0_4(md.pred[PRED_nLx2N], cuGeom, SIZE_nLx2N, refMasks);if (md.pred[PRED_nLx2N].sa8dCost < bestInter->sa8dCost)bestInter = &md.pred[PRED_nLx2N];}if (!try_nRx2N_first && splitCost < md.pred[PRED_2Nx2N].sa8dCost + threshold_nRx2N){refMasks[0] = allSplitRefs;                                    /* 75% left  */refMasks[1] = splitData[1].splitRefs | splitData[3].splitRefs; /* 25% right */md.pred[PRED_nRx2N].cu.initSubCU(parentCTU, cuGeom, qp);checkInter_rd0_4(md.pred[PRED_nRx2N], cuGeom, SIZE_nRx2N, refMasks);if (md.pred[PRED_nRx2N].sa8dCost < bestInter->sa8dCost)bestInter = &md.pred[PRED_nRx2N];}}}}/*检查是否需要进行intra模式的尝试，需要满足的条件为（1）sliceType不为B帧，或者允许B帧中使用intra模式（2）CUSize不能为64（3）bCTUInfo第三位不能为1（没研究过，但bCTUInfo默认为0）（4）bCtuInfoCheck表示是否启用基于CTU内容信息的编码策略调整*/bool bTryIntra = (m_slice->m_sliceType != B_SLICE || m_param->bIntraInBFrames) && cuGeom.log2CUSize != MAX_LOG2_CU_SIZE && !((m_param->bCTUInfo & 4) && bCtuInfoCheck);// rdLevel默认为3if (m_param->rdLevel >= 3){/* Calculate RD cost of best inter option */if ((!m_bChromaSa8d && (m_csp != X265_CSP_I400)) || (m_frame->m_fencPic->m_picCsp == X265_CSP_I400 && m_csp != X265_CSP_I400)) /* When m_bChromaSa8d is enabled, chroma MC has already been done */{uint32_t numPU = bestInter->cu.getNumPartInter(0);for (uint32_t puIdx = 0; puIdx < numPU; puIdx++){PredictionUnit pu(bestInter->cu, cuGeom, puIdx);motionCompensation(bestInter->cu, pu, bestInter->predYuv, false, true);}}// 不使用merge模式if (!chooseMerge){// 将前面确定的模式进行编码并计算RdCostencodeResAndCalcRdInterCU(*bestInter, cuGeom);checkBestMode(*bestInter, depth);/* If BIDIR is available and within 17/16 of best inter option, choose by RDO */// 如果BIDIR的损失小于等于最佳模式的17/16倍（应该是经验性参数）if (m_slice->m_sliceType == B_SLICE && md.pred[PRED_BIDIR].sa8dCost != MAX_INT64 &&md.pred[PRED_BIDIR].sa8dCost * 16 <= bestInter->sa8dCost * 17){uint32_t numPU = md.pred[PRED_BIDIR].cu.getNumPartInter(0);if (m_frame->m_fencPic->m_picCsp == X265_CSP_I400 && m_csp != X265_CSP_I400)for (uint32_t puIdx = 0; puIdx < numPU; puIdx++){PredictionUnit pu(md.pred[PRED_BIDIR].cu, cuGeom, puIdx);// BIDIR模式的运动补偿motionCompensation(md.pred[PRED_BIDIR].cu, pu, md.pred[PRED_BIDIR].predYuv, true, true);}// 计算BIDIR模式的损失encodeResAndCalcRdInterCU(md.pred[PRED_BIDIR], cuGeom);checkBestMode(md.pred[PRED_BIDIR], depth);}}// 尝试intra模式if ((bTryIntra && md.bestMode->cu.getQtRootCbf(0)) ||md.bestMode->sa8dCost == MAX_INT64){if (!m_param->limitReferences || splitIntra){ProfileCounter(parentCTU, totalIntraCU[cuGeom.depth]);md.pred[PRED_INTRA].cu.initSubCU(parentCTU, cuGeom, qp);checkIntraInInter(md.pred[PRED_INTRA], cuGeom);encodeIntraInInter(md.pred[PRED_INTRA], cuGeom);checkBestMode(md.pred[PRED_INTRA], depth);}else{ProfileCounter(parentCTU, skippedIntraCU[cuGeom.depth]);}}}else{/* SA8D choice between merge/skip, inter, bidir, and intra */if (!md.bestMode || bestInter->sa8dCost < md.bestMode->sa8dCost)md.bestMode = bestInter;if (m_slice->m_sliceType == B_SLICE &&md.pred[PRED_BIDIR].sa8dCost < md.bestMode->sa8dCost)md.bestMode = &md.pred[PRED_BIDIR];if (bTryIntra || md.bestMode->sa8dCost == MAX_INT64){if (!m_param->limitReferences || splitIntra){ProfileCounter(parentCTU, totalIntraCU[cuGeom.depth]);md.pred[PRED_INTRA].cu.initSubCU(parentCTU, cuGeom, qp);checkIntraInInter(md.pred[PRED_INTRA], cuGeom);if (md.pred[PRED_INTRA].sa8dCost < md.bestMode->sa8dCost)md.bestMode = &md.pred[PRED_INTRA];}else{ProfileCounter(parentCTU, skippedIntraCU[cuGeom.depth]);}}/* finally code the best mode selected by SA8D costs:* RD level 2 - fully encode the best mode* RD level 1 - generate recon pixels* RD level 0 - generate chroma prediction */if (md.bestMode->cu.m_mergeFlag[0] && md.bestMode->cu.m_partSize[0] == SIZE_2Nx2N){/* prediction already generated for this CU, and if rd level* is not 0, it is already fully encoded */}else if (md.bestMode->cu.isInter(0)){uint32_t numPU = md.bestMode->cu.getNumPartInter(0);if (m_csp != X265_CSP_I400){for (uint32_t puIdx = 0; puIdx < numPU; puIdx++){PredictionUnit pu(md.bestMode->cu, cuGeom, puIdx);motionCompensation(md.bestMode->cu, pu, md.bestMode->predYuv, false, true);}}if (m_param->rdLevel == 2)encodeResAndCalcRdInterCU(*md.bestMode, cuGeom);else if (m_param->rdLevel == 1){/* generate recon pixels with no rate distortion considerations */CUData& cu = md.bestMode->cu;uint32_t tuDepthRange[2];cu.getInterTUQtDepthRange(tuDepthRange, 0);m_rqt[cuGeom.depth].tmpResiYuv.subtract(*md.bestMode->fencYuv, md.bestMode->predYuv, cuGeom.log2CUSize, m_frame->m_fencPic->m_picCsp);residualTransformQuantInter(*md.bestMode, cuGeom, 0, 0, tuDepthRange);if (cu.getQtRootCbf(0))md.bestMode->reconYuv.addClip(md.bestMode->predYuv, m_rqt[cuGeom.depth].tmpResiYuv, cu.m_log2CUSize[0], m_frame->m_fencPic->m_picCsp);else{md.bestMode->reconYuv.copyFromYuv(md.bestMode->predYuv);if (cu.m_mergeFlag[0] && cu.m_partSize[0] == SIZE_2Nx2N)cu.setPredModeSubParts(MODE_SKIP);}}}else{if (m_param->rdLevel == 2)encodeIntraInInter(*md.bestMode, cuGeom);else if (m_param->rdLevel == 1){/* generate recon pixels with no rate distortion considerations */CUData& cu = md.bestMode->cu;uint32_t tuDepthRange[2];cu.getIntraTUQtDepthRange(tuDepthRange, 0);residualTransformQuantIntra(*md.bestMode, cuGeom, 0, 0, tuDepthRange);if (m_csp != X265_CSP_I400){getBestIntraModeChroma(*md.bestMode, cuGeom);residualQTIntraChroma(*md.bestMode, cuGeom, 0, 0);}md.bestMode->reconYuv.copyFromPicYuv(reconPic, cu.m_cuAddr, cuGeom.absPartIdx); // TODO:}}}} // !earlyskipif (m_bTryLossless)tryLossless(cuGeom);if (mightSplit)addSplitFlagCost(*md.bestMode, cuGeom.depth);}if (mightSplit && !skipRecursion){Mode* splitPred = &md.pred[PRED_SPLIT];if (!md.bestMode)md.bestMode = splitPred;else if (m_param->rdLevel > 1)checkBestMode(*splitPred, cuGeom.depth);else if (splitPred->sa8dCost < md.bestMode->sa8dCost)md.bestMode = splitPred;checkDQPForSplitPred(*md.bestMode, cuGeom);}/* determine which motion references the parent CU should search */splitCUData.initSplitCUData();if (m_param->limitReferences & X265_REF_LIMIT_DEPTH){if (md.bestMode == &md.pred[PRED_SPLIT])splitCUData.splitRefs = allSplitRefs;else{/* use best merge/inter mode, in case of intra use 2Nx2N inter references */CUData& cu = md.bestMode->cu.isIntra(0) ? md.pred[PRED_2Nx2N].cu : md.bestMode->cu;uint32_t numPU = cu.getNumPartInter(0);for (uint32_t puIdx = 0, subPartIdx = 0; puIdx < numPU; puIdx++, subPartIdx += cu.getPUOffset(puIdx, 0))splitCUData.splitRefs |= cu.getBestRefIdx(subPartIdx);}}if (m_param->limitModes){splitCUData.mvCost[0] = md.pred[PRED_2Nx2N].bestME[0][0].mvCost; // L0splitCUData.mvCost[1] = md.pred[PRED_2Nx2N].bestME[0][1].mvCost; // L1splitCUData.sa8dCost = md.pred[PRED_2Nx2N].sa8dCost;}// 最佳模式是skip模式，更新cu统计信息if (mightNotSplit && md.bestMode->cu.isSkipped(0)){FrameData& curEncData = *m_frame->m_encData;FrameData::RCStatCU& cuStat = curEncData.m_cuStat[parentCTU.m_cuAddr];uint64_t temp = cuStat.avgCost[depth] * cuStat.count[depth];cuStat.count[depth] += 1;cuStat.avgCost[depth] = (temp + md.bestMode->rdCost) / cuStat.count[depth];}/* Copy best data to encData CTU and recon */// 拷贝最新的data到recon缓冲区中md.bestMode->cu.copyToPic(depth);if (m_param->rdLevel)md.bestMode->reconYuv.copyToPicYuv(reconPic, cuAddr, cuGeom.absPartIdx);if ((m_limitTU & X265_TU_LIMIT_NEIGH) && cuGeom.log2CUSize >= 4){if (mightNotSplit){CUData* ctu = md.bestMode->cu.m_encData->getPicCTU(parentCTU.m_cuAddr);int8_t maxTUDepth = -1;for (uint32_t i = 0; i < cuGeom.numPartitions; i++)maxTUDepth = X265_MAX(maxTUDepth, md.bestMode->cu.m_tuDepth[i]);ctu->m_refTuDepth[cuGeom.geomRecurId] = maxTUDepth;}}}else{// ...}return splitCUData;
}

2.1 检查Merge/Skip模式（checkMerge2Nx2N_rd0_4）

函数的主要作用是检查merge模式和skip模式对应的损失，主要的工作流程为
（1）获取merge候选列表（getInterMergeCandidates）
（2）检查merge候选模式列表，确认最佳merge模式（使用了运动补偿motionCompensation，基于SAD）
（3）基于最佳merge模式，计算不编码残差的损失（encodeResAndCalcRdSkipCU，基于SSE）
（4）基于最佳merge模式，计算编码残差的损失（encodeResAndCalcRdInterCU，基于SSE）

PS：需要注意的是，这里说的Skip模式指的是基于最佳Merge模式，不对最佳Merge模式的残差进行编码的操作

/* sets md.bestMode if a valid merge candidate is found, else leaves it NULL */
void Analysis::checkMerge2Nx2N_rd0_4(Mode& skip, Mode& merge, const CUGeom& cuGeom)
{uint32_t depth = cuGeom.depth;ModeDepth& md = m_modeDepth[depth];Yuv *fencYuv = &md.fencYuv;/* Note that these two Mode instances are named MERGE and SKIP but they may* hold the reverse when the function returns. We toggle between the two modes */Mode* tempPred = &merge;Mode* bestPred = &skip;X265_CHECK(m_slice->m_sliceType != I_SLICE, "Evaluating merge in I slice\n");tempPred->initCosts();tempPred->cu.setPartSizeSubParts(SIZE_2Nx2N);tempPred->cu.setPredModeSubParts(MODE_INTER);tempPred->cu.m_mergeFlag[0] = true;bestPred->initCosts();bestPred->cu.setPartSizeSubParts(SIZE_2Nx2N);bestPred->cu.setPredModeSubParts(MODE_INTER);bestPred->cu.m_mergeFlag[0] = true;MVField candMvField[MRG_MAX_NUM_CANDS][2]; // double length for mv of both lists，存储MV列表uint8_t candDir[MRG_MAX_NUM_CANDS];	// 存储前后向// 1. 获取merge候选列表，MRG_MAX_NUM_CANDS = 5，实际使用时可能为3，与参数配置有关系uint32_t numMergeCand = tempPred->cu.getInterMergeCandidates(0, 0, candMvField, candDir);PredictionUnit pu(merge.cu, cuGeom, 0);bestPred->sa8dCost = MAX_INT64;int bestSadCand = -1;int sizeIdx = cuGeom.log2CUSize - 2;int safeX, maxSafeMv;if (m_param->bIntraRefresh && m_slice->m_sliceType == P_SLICE){safeX = m_slice->m_refFrameList[0][0]->m_encData->m_pir.pirEndCol * m_param->maxCUSize - 3;maxSafeMv = (safeX - tempPred->cu.m_cuPelX) * 4;}// 2. 检查merge候选模式for (uint32_t i = 0; i < numMergeCand; ++i){// 是否启用帧级并行处理if (m_bFrameParallel){// Parallel slices bound checkif (m_param->maxSlices > 1){// NOTE: First row in slice can't negativeif (X265_MIN(candMvField[i][0].mv.y, candMvField[i][1].mv.y) < m_sliceMinY)continue;// Last row in slice can't reference beyond bound since it is another slice area// TODO: we may beyond bound in future since these area have a chance to finish because we use parallel slices. Necessary prepare research on load balanceif (X265_MAX(candMvField[i][0].mv.y, candMvField[i][1].mv.y) > m_sliceMaxY)continue;}if (candMvField[i][0].mv.y >= (m_param->searchRange + 1) * 4 ||candMvField[i][1].mv.y >= (m_param->searchRange + 1) * 4)continue;}if (m_param->bIntraRefresh && m_slice->m_sliceType == P_SLICE &&tempPred->cu.m_cuPelX / m_param->maxCUSize < m_frame->m_encData->m_pir.pirEndCol &&candMvField[i][0].mv.x > maxSafeMv)// skip merge candidates which reference beyond safe reference areacontinue;// merge候选模式存储在L0中tempPred->cu.m_mvpIdx[0][0] = (uint8_t)i; // merge candidate ID is stored in L0 MVP idxX265_CHECK(m_slice->m_sliceType == B_SLICE || !(candDir[i] & 0x10), " invalid merge for P slice\n");tempPred->cu.m_interDir[0] = candDir[i]; // 候选列表信息tempPred->cu.m_mv[0][0] = candMvField[i][0].mv;	// 前向mv（第二个维度0表示前向，1表示后向）tempPred->cu.m_mv[1][0] = candMvField[i][1].mv;	// 后向mvtempPred->cu.m_refIdx[0][0] = (int8_t)candMvField[i][0].refIdx;	// 前向参考帧索引tempPred->cu.m_refIdx[1][0] = (int8_t)candMvField[i][1].refIdx;	// 后向参考帧索引// 运动补偿（根据MV来获取预测块）motionCompensation(tempPred->cu, pu, tempPred->predYuv, true, m_bChromaSa8d && (m_csp != X265_CSP_I400 && m_frame->m_fencPic->m_picCsp != X265_CSP_I400));tempPred->sa8dBits = getTUBits(i, numMergeCand);// 根据运动补偿MC获取的预测块，来计算sadtempPred->distortion = primitives.cu[sizeIdx].sa8d(fencYuv->m_buf[0], fencYuv->m_size, tempPred->predYuv.m_buf[0], tempPred->predYuv.m_size);if (m_bChromaSa8d && (m_csp != X265_CSP_I400 && m_frame->m_fencPic->m_picCsp != X265_CSP_I400)){tempPred->distortion += primitives.chroma[m_csp].cu[sizeIdx].sa8d(fencYuv->m_buf[1], fencYuv->m_csize, tempPred->predYuv.m_buf[1], tempPred->predYuv.m_csize);tempPred->distortion += primitives.chroma[m_csp].cu[sizeIdx].sa8d(fencYuv->m_buf[2], fencYuv->m_csize, tempPred->predYuv.m_buf[2], tempPred->predYuv.m_csize);}// 计算rdCosttempPred->sa8dCost = m_rdCost.calcRdSADCost((uint32_t)tempPred->distortion, tempPred->sa8dBits);// 检查当前模式的rdCost是否是最佳的if (tempPred->sa8dCost < bestPred->sa8dCost){bestSadCand = i;std::swap(tempPred, bestPred);}}/* force mode decision to take inter or intra */if (bestSadCand < 0)return;/* calculate the motion compensation for chroma for the best mode selected */// 检查chroma分量if ((!m_bChromaSa8d && (m_csp != X265_CSP_I400)) || (m_frame->m_fencPic->m_picCsp == X265_CSP_I400 && m_csp != X265_CSP_I400)) /* Chroma MC was done above */motionCompensation(bestPred->cu, pu, bestPred->predYuv, false, true);if (m_param->rdLevel){if (m_param->bLossless)bestPred->rdCost = MAX_INT64;else // 3. 基于最佳merge模式，计算直接skip的损失（基于SSE），skip模式不会实际编码残差encodeResAndCalcRdSkipCU(*bestPred);/* Encode with residual */tempPred->cu.m_mvpIdx[0][0] = (uint8_t)bestSadCand;tempPred->cu.setPUInterDir(candDir[bestSadCand], 0, 0);tempPred->cu.setPUMv(0, candMvField[bestSadCand][0].mv, 0, 0);tempPred->cu.setPUMv(1, candMvField[bestSadCand][1].mv, 0, 0);tempPred->cu.setPURefIdx(0, (int8_t)candMvField[bestSadCand][0].refIdx, 0, 0);tempPred->cu.setPURefIdx(1, (int8_t)candMvField[bestSadCand][1].refIdx, 0, 0);tempPred->sa8dCost = bestPred->sa8dCost;tempPred->sa8dBits = bestPred->sa8dBits;tempPred->predYuv.copyFromYuv(bestPred->predYuv);// 4. 将bestSadCand使用SSE再计算一遍，获取基于SSE的损失，会实际编码残差encodeResAndCalcRdInterCU(*tempPred, cuGeom);/*从两者中取出最佳的模式（1）bestPred指向的是skip模式（2）tempPred指向的是从merge候选列表中得到bestSadCand模式（基于SSE重新计算之后）*/md.bestMode = tempPred->rdCost < bestPred->rdCost ? tempPred : bestPred;}elsemd.bestMode = bestPred;/* broadcast sets of MV field data */// 存储最佳模式md.bestMode->cu.setPUInterDir(candDir[bestSadCand], 0, 0);md.bestMode->cu.setPUMv(0, candMvField[bestSadCand][0].mv, 0, 0);md.bestMode->cu.setPUMv(1, candMvField[bestSadCand][1].mv, 0, 0);md.bestMode->cu.setPURefIdx(0, (int8_t)candMvField[bestSadCand][0].refIdx, 0, 0);md.bestMode->cu.setPURefIdx(1, (int8_t)candMvField[bestSadCand][1].refIdx, 0, 0);checkDQP(*md.bestMode, cuGeom);
}

2.1.1 获取Merge候选列表（getInterMergeCandidates）

/* Construct list of merging candidates, returns count */
uint32_t CUData::getInterMergeCandidates(uint32_t absPartIdx, uint32_t puIdx, MVField(*candMvField)[2], uint8_t* candDir) const
{uint32_t absPartAddr = m_absIdxInCTU + absPartIdx;const bool isInterB = m_slice->isInterB();const uint32_t maxNumMergeCand = m_slice->m_maxNumMergeCand;for (uint32_t i = 0; i < maxNumMergeCand; ++i){candMvField[i][0].mv = 0;candMvField[i][1].mv = 0;candMvField[i][0].refIdx = REF_NOT_VALID;candMvField[i][1].refIdx = REF_NOT_VALID;}/* calculate the location of upper-left corner pixel and size of the current PU */int xP, yP, nPSW, nPSH;int cuSize = 1 << m_log2CUSize[0];int partMode = m_partSize[0];/*// Partition table.总共有3个维度：（1）第1维表示划分的方式，例如SIZE_2Nx2N，长度为9（2）第2维表示划分之后的索引号，即第几个块，长度为4（3）第3维长度为2，其中第一个表示划分的尺寸，第二个表示划分偏移量举例如下（1）partTable[0][0][0] = 0x44，表示这是一个4x4的块，partTable[0][0][1] = 0x00，表示不存在偏移量（2）partTable[3][0][0] = 0x22，表示这是一个划分成为4个子块的情况，并且标识的是第一个2x2的块；partTable[3][1][1] = 0x20，表示第二个2x2块，水平偏移量为2，垂直偏移量为0const uint32_t partTable[8][4][2] ={//        XY{ { 0x44, 0x00 }, { 0x00, 0x00 }, { 0x00, 0x00 }, { 0x00, 0x00 } }, // SIZE_2Nx2N.{ { 0x42, 0x00 }, { 0x42, 0x02 }, { 0x00, 0x00 }, { 0x00, 0x00 } }, // SIZE_2NxN.{ { 0x24, 0x00 }, { 0x24, 0x20 }, { 0x00, 0x00 }, { 0x00, 0x00 } }, // SIZE_Nx2N.{ { 0x22, 0x00 }, { 0x22, 0x20 }, { 0x22, 0x02 }, { 0x22, 0x22 } }, // SIZE_NxN.{ { 0x41, 0x00 }, { 0x43, 0x01 }, { 0x00, 0x00 }, { 0x00, 0x00 } }, // SIZE_2NxnU.{ { 0x43, 0x00 }, { 0x41, 0x03 }, { 0x00, 0x00 }, { 0x00, 0x00 } }, // SIZE_2NxnD.{ { 0x14, 0x00 }, { 0x34, 0x10 }, { 0x00, 0x00 }, { 0x00, 0x00 } }, // SIZE_nLx2N.{ { 0x34, 0x00 }, { 0x14, 0x30 }, { 0x00, 0x00 }, { 0x00, 0x00 } }  // SIZE_nRx2N.};*/int tmp = partTable[partMode][puIdx][0]; // 尺寸nPSW = ((tmp >> 4) * cuSize) >> 2;  // 宽nPSH = ((tmp & 0xF) * cuSize) >> 2; // 高tmp = partTable[partMode][puIdx][1]; // 偏移量xP = ((tmp >> 4) * cuSize) >> 2;	 // x偏移量（或者说相对位置）yP = ((tmp & 0xF) * cuSize) >> 2;	 // y偏移量uint32_t count = 0;// 根据partSize计算left-bottom位置的idxuint32_t partIdxLT, partIdxRT, partIdxLB = deriveLeftBottomIdx(puIdx);PartSize curPS = (PartSize)m_partSize[absPartIdx];/*Merge候选列表的建立（下图中PU尺寸不固定，只表明相对位置），merge列表最多5个（1）空域候选列表 = { A1, B1, B0, A0, B2 }，按照顺序至多选择4个+--+           +--+--+|B2|		   |B1|B0|+--+--+--+--+--+--+--+|           |+           +|  Current  |+    PU     +|           |+--+           +|A1|           |+--+--+--+--+--+|A0|+--+需要特殊处理的情况，针对下面的PU 2情况1: 不能存在A1的运动信息				情况2: 不能存在B1的运动信息+--+        +--+--+            +--+--+--+--+|B2|        |B1|B0|		       |           |+--+--+--+--+--+--+--+--+--+         +--+    PU 1   +--+|           |           |            |B2|           |B0|+           +           +			+--+--+--+--+--+--+|           |           |               |           |+    PU 1   +    PU 2   +			+--+    PU 2   +|           |           |			|A1|           |+           +           +			+--+--+--+--+--+|           |           |            |A0|+--+--+--+--+--+--+--+--+            +--+|A0|+--+（2）时域候选列表 = { H or C3(if H not exist) }Current PU+----+----+----+----+|         |         |+         +         +|         |         |+----+----+----+----+|         | C3 |    |+         +----+    +|         |         |+----+----+----+----+----+|  H |+----+*/// leftuint32_t leftPartIdx = 0;const CUData* cuLeft = getPULeft(leftPartIdx, partIdxLB);// 检查A1是否存在bool isAvailableA1 = cuLeft &&/* isDiffMER()用于检查当前PU和空域需要参考的PU是否位于同一merge区域（1）相邻块x = xP - 1，相邻块y = yP + nPSH - 1（2）当前块x = xP，当前块y = yP我理解这里相邻块x = xP - 1不是表示相邻块左上角的x，而是相邻区域（不然似乎对应不上）*/cuLeft->isDiffMER(xP - 1, yP + nPSH - 1, xP, yP) && // 检查是否为情况1，如果是情况1，则不存在A1信息!(puIdx == 1 && (curPS == SIZE_Nx2N || curPS == SIZE_nLx2N || curPS == SIZE_nRx2N)) &&cuLeft->isInter(leftPartIdx);// 如果A1块存在，则取出dir和mvif (isAvailableA1){// get Inter DircandDir[count] = cuLeft->m_interDir[leftPartIdx];// get Mv from LeftcuLeft->getMvField(cuLeft, leftPartIdx, 0, candMvField[count][0]);if (isInterB)cuLeft->getMvField(cuLeft, leftPartIdx, 1, candMvField[count][1]);if (++count == maxNumMergeCand)return maxNumMergeCand;}// 更新partIdxLT和partIdxRTderiveLeftRightTopIdx(puIdx, partIdxLT, partIdxRT);// aboveuint32_t abovePartIdx = 0;const CUData* cuAbove = getPUAbove(abovePartIdx, partIdxRT);// 检查B1是否存在bool isAvailableB1 = cuAbove &&/* 检查当前PU和空域需要参考的PU是否位于同一merge区域（1）相邻块x = xP + nPSW - 1，相邻块y = yP - 1（2）当前块x = xP，当前块y = yP*/ cuAbove->isDiffMER(xP + nPSW - 1, yP - 1, xP, yP) &&!(puIdx == 1 && (curPS == SIZE_2NxN || curPS == SIZE_2NxnU || curPS == SIZE_2NxnD)) &&cuAbove->isInter(abovePartIdx);if (isAvailableB1 && (!isAvailableA1 || !cuLeft->hasEqualMotion(leftPartIdx, *cuAbove, abovePartIdx))){// get Inter DircandDir[count] = cuAbove->m_interDir[abovePartIdx];// get Mv from LeftcuAbove->getMvField(cuAbove, abovePartIdx, 0, candMvField[count][0]);if (isInterB)cuAbove->getMvField(cuAbove, abovePartIdx, 1, candMvField[count][1]);if (++count == maxNumMergeCand)return maxNumMergeCand;}// above rightuint32_t aboveRightPartIdx = 0;const CUData* cuAboveRight = getPUAboveRight(aboveRightPartIdx, partIdxRT);// 检查B0是否存在bool isAvailableB0 = cuAboveRight &&/* 检查当前PU和空域需要参考的PU是否位于同一merge区域（1）相邻块x = xP + nPSW，相邻块y = yP - 1（2）当前块x = xP，当前块y = yP*/ cuAboveRight->isDiffMER(xP + nPSW, yP - 1, xP, yP) &&cuAboveRight->isInter(aboveRightPartIdx);if (isAvailableB0 && (!isAvailableB1 || !cuAbove->hasEqualMotion(abovePartIdx, *cuAboveRight, aboveRightPartIdx))){// get Inter DircandDir[count] = cuAboveRight->m_interDir[aboveRightPartIdx];// get Mv from LeftcuAboveRight->getMvField(cuAboveRight, aboveRightPartIdx, 0, candMvField[count][0]);if (isInterB)cuAboveRight->getMvField(cuAboveRight, aboveRightPartIdx, 1, candMvField[count][1]);if (++count == maxNumMergeCand)return maxNumMergeCand;}// left bottomuint32_t leftBottomPartIdx = 0;const CUData* cuLeftBottom = this->getPUBelowLeft(leftBottomPartIdx, partIdxLB);// 检查A0是否存在bool isAvailableA0 = cuLeftBottom &&/* 检查当前PU和空域需要参考的PU是否位于同一merge区域（1）相邻块x = xP - 1，相邻块y = yP + nPSH（2）当前块x = xP，当前块y = yP*/ cuLeftBottom->isDiffMER(xP - 1, yP + nPSH, xP, yP) &&cuLeftBottom->isInter(leftBottomPartIdx);if (isAvailableA0 && (!isAvailableA1 || !cuLeft->hasEqualMotion(leftPartIdx, *cuLeftBottom, leftBottomPartIdx))){// get Inter DircandDir[count] = cuLeftBottom->m_interDir[leftBottomPartIdx];// get Mv from LeftcuLeftBottom->getMvField(cuLeftBottom, leftBottomPartIdx, 0, candMvField[count][0]);if (isInterB)cuLeftBottom->getMvField(cuLeftBottom, leftBottomPartIdx, 1, candMvField[count][1]);if (++count == maxNumMergeCand)return maxNumMergeCand;}// above left// 如果前面获取的merge cand小于4个，还会检查左上角的块，即B2if (count < 4){uint32_t aboveLeftPartIdx = 0;const CUData* cuAboveLeft = getPUAboveLeft(aboveLeftPartIdx, absPartAddr);// 检查B2是否可用bool isAvailableB2 = cuAboveLeft &&cuAboveLeft->isDiffMER(xP - 1, yP - 1, xP, yP) &&cuAboveLeft->isInter(aboveLeftPartIdx);if (isAvailableB2 && (!isAvailableA1 || !cuLeft->hasEqualMotion(leftPartIdx, *cuAboveLeft, aboveLeftPartIdx))&& (!isAvailableB1 || !cuAbove->hasEqualMotion(abovePartIdx, *cuAboveLeft, aboveLeftPartIdx))){// get Inter DircandDir[count] = cuAboveLeft->m_interDir[aboveLeftPartIdx];// get Mv from LeftcuAboveLeft->getMvField(cuAboveLeft, aboveLeftPartIdx, 0, candMvField[count][0]);if (isInterB)cuAboveLeft->getMvField(cuAboveLeft, aboveLeftPartIdx, 1, candMvField[count][1]);if (++count == maxNumMergeCand)return maxNumMergeCand;}}/*检查TemporalMVP是否可用，如果可用则去获取时域上的参考列表*/if (m_slice->m_sps->bTemporalMVPEnabled){// 获取右下角pu idxuint32_t partIdxRB = deriveRightBottomIdx(puIdx);MV colmv;int ctuIdx = -1;// image boundary checkif (m_encData->getPicCTU(m_cuAddr)->m_cuPelX + g_zscanToPelX[partIdxRB] + UNIT_SIZE < m_slice->m_sps->picWidthInLumaSamples &&m_encData->getPicCTU(m_cuAddr)->m_cuPelY + g_zscanToPelY[partIdxRB] + UNIT_SIZE < m_slice->m_sps->picHeightInLumaSamples){uint32_t absPartIdxRB = g_zscanToRaster[partIdxRB];uint32_t numUnits = s_numPartInCUSize;// 检查absPartIdxRB是否是最后一列或者最后一行bool bNotLastCol = lessThanCol(absPartIdxRB, numUnits - 1); // is not at the last column of CTUbool bNotLastRow = lessThanRow(absPartIdxRB, numUnits - 1); // is not at the last row    of CTU// 确定时域候选列表同位PU的位置if (bNotLastCol && bNotLastRow){absPartAddr = g_rasterToZscan[absPartIdxRB + RASTER_SIZE + 1];ctuIdx = m_cuAddr;}else if (bNotLastCol)absPartAddr = g_rasterToZscan[(absPartIdxRB + 1) & (numUnits - 1)];else if (bNotLastRow){absPartAddr = g_rasterToZscan[absPartIdxRB + RASTER_SIZE - numUnits + 1];ctuIdx = m_cuAddr + 1;}else // is the right bottom corner of CTUabsPartAddr = 0;}// B帧具有两个时域候选模式，P帧只有一个int maxList = isInterB ? 2 : 1;int dir = 0, refIdx = 0;for (int list = 0; list < maxList; list++){// 获取colocated-mvbool bExistMV = ctuIdx >= 0 && getColMVP(colmv, refIdx, list, ctuIdx, absPartAddr);if (!bExistMV){// 如果右下角的PU没有可用MV，则从C3位置获取mv，作为可用的mvuint32_t partIdxCenter = deriveCenterIdx(puIdx);bExistMV = getColMVP(colmv, refIdx, list, m_cuAddr, partIdxCenter);}// 如果找到可用MV，则加入到队列中if (bExistMV){dir |= (1 << list);candMvField[count][list].mv = colmv;candMvField[count][list].refIdx = refIdx;}}if (dir != 0){candDir[count] = (uint8_t)dir;if (++count == maxNumMergeCand)return maxNumMergeCand;}}// B帧处理组合列表（没研究过）if (isInterB){const uint32_t cutoff = count * (count - 1);uint32_t priorityList0 = 0xEDC984; // { 0, 1, 0, 2, 1, 2, 0, 3, 1, 3, 2, 3 }uint32_t priorityList1 = 0xB73621; // { 1, 0, 2, 0, 2, 1, 3, 0, 3, 1, 3, 2 }for (uint32_t idx = 0; idx < cutoff; idx++, priorityList0 >>= 2, priorityList1 >>= 2){int i = priorityList0 & 3;int j = priorityList1 & 3;if ((candDir[i] & 0x1) && (candDir[j] & 0x2)){// get Mv from cand[i] and cand[j]int refIdxL0 = candMvField[i][0].refIdx;int refIdxL1 = candMvField[j][1].refIdx;int refPOCL0 = m_slice->m_refPOCList[0][refIdxL0];int refPOCL1 = m_slice->m_refPOCList[1][refIdxL1];if (!(refPOCL0 == refPOCL1 && candMvField[i][0].mv == candMvField[j][1].mv)){candMvField[count][0].mv = candMvField[i][0].mv;candMvField[count][0].refIdx = refIdxL0;candMvField[count][1].mv = candMvField[j][1].mv;candMvField[count][1].refIdx = refIdxL1;candDir[count] = 3;if (++count == maxNumMergeCand)return maxNumMergeCand;}}}}int numRefIdx = (isInterB) ? X265_MIN(m_slice->m_numRefIdx[0], m_slice->m_numRefIdx[1]) : m_slice->m_numRefIdx[0];int r = 0;int refcnt = 0;// 如果当前MV候选列表长度不足5个，需要填充(0,0)while (count < maxNumMergeCand){candDir[count] = 1;candMvField[count][0].mv.word = 0;candMvField[count][0].refIdx = r;if (isInterB){candDir[count] = 3;candMvField[count][1].mv.word = 0;candMvField[count][1].refIdx = r;}count++;if (refcnt == numRefIdx - 1)r = 0;else{++r;++refcnt;}}return count;
}

2.1.2 运动补偿（motionCompensation）

运动补偿会根据前面提取到的MV，进行预测，获取到参考帧中的参考块。在x265中，主要调用了predInterLumaPixel()进行帧间的运动补偿

void Predict::motionCompensation(const CUData& cu, const PredictionUnit& pu, Yuv& predYuv, bool bLuma, bool bChroma)
{int refIdx0 = cu.m_refIdx[0][pu.puAbsPartIdx];int refIdx1 = cu.m_refIdx[1][pu.puAbsPartIdx];// 是否是P帧if (cu.m_slice->isInterP()){/* P Slice */WeightValues wv0[3];X265_CHECK(refIdx0 >= 0, "invalid P refidx\n");X265_CHECK(refIdx0 < cu.m_slice->m_numRefIdx[0], "P refidx out of range\n");const WeightParam *wp0 = cu.m_slice->m_weightPredTable[0][refIdx0]; // 加权预测相关，没有研究过MV mv0 = cu.m_mv[0][pu.puAbsPartIdx];cu.clipMv(mv0);if (cu.m_slice->m_pps->bUseWeightPred && wp0->wtPresent){for (int plane = 0; plane < (bChroma ? 3 : 1); plane++){wv0[plane].w      = wp0[plane].inputWeight;wv0[plane].offset = wp0[plane].inputOffset * (1 << (X265_DEPTH - 8));wv0[plane].shift  = wp0[plane].log2WeightDenom;wv0[plane].round  = wp0[plane].log2WeightDenom >= 1 ? 1 << (wp0[plane].log2WeightDenom - 1) : 0;}ShortYuv& shortYuv = m_predShortYuv[0];if (bLuma)predInterLumaShort(pu, shortYuv, *cu.m_slice->m_refReconPicList[0][refIdx0], mv0);if (bChroma)predInterChromaShort(pu, shortYuv, *cu.m_slice->m_refReconPicList[0][refIdx0], mv0);addWeightUni(pu, predYuv, shortYuv, wv0, bLuma, bChroma);}else{	// 亮度模式运动补偿if (bLuma)predInterLumaPixel(pu, predYuv, *cu.m_slice->m_refReconPicList[0][refIdx0], mv0);// 色度模式运动补偿if (bChroma)predInterChromaPixel(pu, predYuv, *cu.m_slice->m_refReconPicList[0][refIdx0], mv0);}}else // B帧（没有研究）{	// ...}
}

2.1.2.1 获取预测块（predInterLumaPixel）

从参考帧中获取对应的参考块

void Predict::predInterLumaPixel(const PredictionUnit& pu, Yuv& dstYuv, const PicYuv& refPic, const MV& mv) const
{pixel* dst = dstYuv.getLumaAddr(pu.puAbsPartIdx);intptr_t dstStride = dstYuv.m_size;intptr_t srcStride = refPic.m_stride;intptr_t srcOffset = (mv.x >> 2) + (mv.y >> 2) * srcStride;int partEnum = partitionFromSizes(pu.width, pu.height);const pixel* src = refPic.getLumaAddr(pu.ctuAddr, pu.cuAbsPartIdx + pu.puAbsPartIdx) + srcOffset;int xFrac = mv.x & 3; // 水平方向偏移量int yFrac = mv.y & 3; // 垂直方向偏移量/*下面根据mv的值确定偏移量（1）如果x和y的偏移量都为0，直接copy，使用copy_pp()如果有偏移量，还会进行像素插值，用于后续的亚像素搜索，下面的8tap表示8抽头（2）如果有x方向的偏移量，使用luma_hpp()进行水平方向的亚像素插值（3）如果有y方向的偏移量，使用luma_vpp()进行垂直方向的亚像素插值（4）如果x和y方向的偏移量都不为0，使用luma_hvpp()进行两个方向的亚像素插值*/if (!(yFrac | xFrac))/*调试过程中发现会使用到的copy函数（非正方形也有对应的处理函数，例如blockcopy_pp_32x16_avx）p.pu[LUMA_64x64].copy_pp  = PFX(blockcopy_pp_64x64_avx);p.pu[LUMA_32x32].copy_pp  = PFX(blockcopy_pp_32x32_avx);p.pu[LUMA_16x16].copy_pp  = x265_blockcopy_pp_16x16_sse2; p.pu[LUMA_8x8].copy_pp  = x265_blockcopy_pp_8x8_sse2;*/primitives.pu[partEnum].copy_pp(dst, dstStride, src, srcStride);else if (!yFrac)/*调试过程中发现会使用到的hpp函数p.pu[LUMA_8x8].luma_hpp = PFX(interp_8tap_horiz_pp_8x8_avx2);p.pu[LUMA_16x16].luma_hpp = PFX(interp_8tap_horiz_pp_16x16_avx2);p.pu[LUMA_32x32].luma_hpp = PFX(interp_8tap_horiz_pp_32x32_avx2); */primitives.pu[partEnum].luma_hpp(src, srcStride, dst, dstStride, xFrac);else if (!xFrac)/*调试过程中发现会使用到的vpp函数p.pu[LUMA_8x8].luma_vpp = PFX(interp_8tap_vert_pp_8x8_avx2);p.pu[LUMA_16x16].luma_vpp = PFX(interp_8tap_vert_pp_16x16_avx2);p.pu[LUMA_32x32].luma_vpp = PFX(interp_8tap_vert_pp_32x32_avx2); */primitives.pu[partEnum].luma_vpp(src, srcStride, dst, dstStride, yFrac);else/*调试过程中发现可以使用的vpp函数ALL_LUMA_PU_T(luma_hvpp, interp_8tap_hv_pp_cpu);interp_8tap_hv_pp_cpu<size>是一个模板函数，模板变量为size当size = 1时，表示对8x8块进行处理当size = 2时，表示对16x16块进行处理当size = 3时，表示对32x32块进行处理*/primitives.pu[partEnum].luma_hvpp(src, srcStride, dst, dstStride, xFrac, yFrac);
}

2.1.3 计算不编码残差的损失（encodeResAndCalcRdSkipCU）

前面已经获取了基于SAD的最佳merge模式，这里计算如果不对残差进行编码，直接进行skip带来的损失，检查是否可以直接使用skip。这里计算distortion使用的是SSE而不是SAD

/* Note: this function overwrites the RD cost variables of interMode, but leaves the sa8d cost unharmed */
// 该函数会覆盖interMode中的RDCost，但不会改动sa8d开销
void Search::encodeResAndCalcRdSkipCU(Mode& interMode)
{CUData& cu = interMode.cu;Yuv* reconYuv = &interMode.reconYuv;const Yuv* fencYuv = interMode.fencYuv;Yuv* predYuv = &interMode.predYuv;X265_CHECK(!cu.isIntra(0), "intra CU not expected\n");uint32_t depth  = cu.m_cuDepth[0];// No residual coding : SKIP mode// skip模式不去编码残差cu.setPredModeSubParts(MODE_SKIP);cu.clearCbf();cu.setTUDepthSubParts(0, 0, depth);reconYuv->copyFromYuv(interMode.predYuv);// 计算基于SSE的Rdcost// Lumaint part = partitionFromLog2Size(cu.m_log2CUSize[0]);// 计算sse损失，需要注意的是计算的双方是orig block和recon blockinterMode.lumaDistortion = primitives.cu[part].sse_pp(fencYuv->m_buf[0], fencYuv->m_size, reconYuv->m_buf[0], reconYuv->m_size);interMode.distortion = interMode.lumaDistortion;// Chromaif (m_csp != X265_CSP_I400 && m_frame->m_fencPic->m_picCsp != X265_CSP_I400){interMode.chromaDistortion = m_rdCost.scaleChromaDist(1, primitives.chroma[m_csp].cu[part].sse_pp(fencYuv->m_buf[1], fencYuv->m_csize, reconYuv->m_buf[1], reconYuv->m_csize));interMode.chromaDistortion += m_rdCost.scaleChromaDist(2, primitives.chroma[m_csp].cu[part].sse_pp(fencYuv->m_buf[2], fencYuv->m_csize, reconYuv->m_buf[2], reconYuv->m_csize));interMode.distortion += interMode.chromaDistortion;}cu.m_distortion[0] = interMode.distortion;m_entropyCoder.load(m_rqt[depth].cur); // 将当前CU的信息输入到熵编码器中，为后续的编码做准备m_entropyCoder.resetBits(); // 重置比特缓冲区if (m_slice->m_pps->bTransquantBypassEnabled)m_entropyCoder.codeCUTransquantBypassFlag(cu.m_tqBypass[0]);m_entropyCoder.codeSkipFlag(cu, 0); // 编码skip flagint skipFlagBits = m_entropyCoder.getNumberOfWrittenBits();m_entropyCoder.codeMergeIndex(cu, 0); // 编码merge idxinterMode.mvBits = m_entropyCoder.getNumberOfWrittenBits() - skipFlagBits;interMode.coeffBits = 0;interMode.totalBits = interMode.mvBits + skipFlagBits;if (m_rdCost.m_psyRd)interMode.psyEnergy = m_rdCost.psyCost(part, fencYuv->m_buf[0], fencYuv->m_size, reconYuv->m_buf[0], reconYuv->m_size);else if(m_rdCost.m_ssimRd)interMode.ssimEnergy = m_quant.ssimDistortion(cu, fencYuv->m_buf[0], fencYuv->m_size, reconYuv->m_buf[0], reconYuv->m_size, cu.m_log2CUSize[0], TEXT_LUMA, 0);interMode.resEnergy = primitives.cu[part].sse_pp(fencYuv->m_buf[0], fencYuv->m_size, predYuv->m_buf[0], predYuv->m_size);// 更新该模式的损失updateModeCost(interMode);// 存储已编码信息m_entropyCoder.store(interMode.contexts);
}

2.1.4 计算编码残差的损失（encodeResAndCalcRdInterCU）

沿用前面的最佳merge模式，依据这个模式，在本函数中进行SSE的distortion的计算，使用estimateResidualQT()计算损失。此外，还考虑了cbf为0的情况，即不编码和不传输残差的方式，评估是否有可能使用这种编码方式

/* encode residual and calculate rate-distortion for a CU block.* Note: this function overwrites the RD cost variables of interMode, but leaves the sa8d cost unharmed */
void Search::encodeResAndCalcRdInterCU(Mode& interMode, const CUGeom& cuGeom)
{ProfileCUScope(interMode.cu, interRDOElapsedTime[cuGeom.depth], countInterRDO[cuGeom.depth]);CUData& cu = interMode.cu;Yuv* reconYuv = &interMode.reconYuv;Yuv* predYuv = &interMode.predYuv;uint32_t depth = cuGeom.depth;ShortYuv* resiYuv = &m_rqt[depth].tmpResiYuv;const Yuv* fencYuv = interMode.fencYuv;X265_CHECK(!cu.isIntra(0), "intra CU not expected\n");uint32_t log2CUSize = cuGeom.log2CUSize;int sizeIdx = log2CUSize - 2;// 将预测块pred和编码块enc做差值，获得残差resiYuv->subtract(*fencYuv, *predYuv, log2CUSize, m_frame->m_fencPic->m_picCsp);uint32_t tuDepthRange[2];cu.getInterTUQtDepthRange(tuDepthRange, 0);m_entropyCoder.load(m_rqt[depth].cur);if ((m_limitTU & X265_TU_LIMIT_DFS) && !(m_limitTU & X265_TU_LIMIT_NEIGH))m_maxTUDepth = -1;else if (m_limitTU & X265_TU_LIMIT_BFS)memset(&m_cacheTU, 0, sizeof(TUInfoCache));Cost costs;if (m_limitTU & X265_TU_LIMIT_NEIGH){/* Save and reload maxTUDepth to avoid changing of maxTUDepth between modes */int32_t tempDepth = m_maxTUDepth;if (m_maxTUDepth != -1){uint32_t splitFlag = interMode.cu.m_partSize[0] != SIZE_2Nx2N;uint32_t minSize = tuDepthRange[0];uint32_t maxSize = tuDepthRange[1];maxSize = X265_MIN(maxSize, cuGeom.log2CUSize - splitFlag);m_maxTUDepth = x265_clip3(cuGeom.log2CUSize - maxSize, cuGeom.log2CUSize - minSize, (uint32_t)m_maxTUDepth);}estimateResidualQT(interMode, cuGeom, 0, 0, *resiYuv, costs, tuDepthRange);m_maxTUDepth = tempDepth;}else // 估计编码残差，并计算对应的rdcostestimateResidualQT(interMode, cuGeom, 0, 0, *resiYuv, costs, tuDepthRange);/*检查是否使用bypass（旁路）模式进行编码（1）对于那些概率接近均匀分布的符号，使用bypass编码可以减少编码开销（2）这些符号的概率大致相同，不适合使用普通的上下文自适应二进制算术编码*/uint32_t tqBypass = cu.m_tqBypass[0];if (!tqBypass){// 计算Cbf为0情况下的损失，随后与当前模式的costs进行对比，Cbf为0表示不编码残差，也不传输残差sse_t cbf0Dist = primitives.cu[sizeIdx].sse_pp(fencYuv->m_buf[0], fencYuv->m_size, predYuv->m_buf[0], predYuv->m_size);if (m_csp != X265_CSP_I400 && m_frame->m_fencPic->m_picCsp != X265_CSP_I400){cbf0Dist += m_rdCost.scaleChromaDist(1, primitives.chroma[m_csp].cu[sizeIdx].sse_pp(fencYuv->m_buf[1], predYuv->m_csize, predYuv->m_buf[1], predYuv->m_csize));cbf0Dist += m_rdCost.scaleChromaDist(2, primitives.chroma[m_csp].cu[sizeIdx].sse_pp(fencYuv->m_buf[2], predYuv->m_csize, predYuv->m_buf[2], predYuv->m_csize));}/* Consider the RD cost of not signaling any residual */m_entropyCoder.load(m_rqt[depth].cur);m_entropyCoder.resetBits();m_entropyCoder.codeQtRootCbfZero();uint32_t cbf0Bits = m_entropyCoder.getNumberOfWrittenBits();uint32_t cbf0Energy; uint64_t cbf0Cost;if (m_rdCost.m_psyRd){cbf0Energy = m_rdCost.psyCost(log2CUSize - 2, fencYuv->m_buf[0], fencYuv->m_size, predYuv->m_buf[0], predYuv->m_size);cbf0Cost = m_rdCost.calcPsyRdCost(cbf0Dist, cbf0Bits, cbf0Energy);}else if(m_rdCost.m_ssimRd){cbf0Energy = m_quant.ssimDistortion(cu, fencYuv->m_buf[0], fencYuv->m_size, predYuv->m_buf[0], predYuv->m_size, log2CUSize, TEXT_LUMA, 0);cbf0Cost = m_rdCost.calcSsimRdCost(cbf0Dist, cbf0Bits, cbf0Energy);}elsecbf0Cost = m_rdCost.calcRdCost(cbf0Dist, cbf0Bits);// 对比cbf为0的cost和当前模式的costif (cbf0Cost < costs.rdcost) // {cu.clearCbf();cu.setTUDepthSubParts(0, 0, depth);}}if (cu.getQtRootCbf(0))saveResidualQTData(cu, *resiYuv, 0, 0);/* calculate signal bits for inter/merge/skip coded CU */m_entropyCoder.load(m_rqt[depth].cur);m_entropyCoder.resetBits();if (m_slice->m_pps->bTransquantBypassEnabled)m_entropyCoder.codeCUTransquantBypassFlag(tqBypass);uint32_t coeffBits, bits, mvBits;// 启用merge && size = 2Nx2N && 根节点Cbf为0if (cu.m_mergeFlag[0] && cu.m_partSize[0] == SIZE_2Nx2N && !cu.getQtRootCbf(0)){// 根节点的Cbf为0，说明子块不再需要继续预测，直接skipcu.setPredModeSubParts(MODE_SKIP);/* Merge/Skip */coeffBits = mvBits = 0;m_entropyCoder.codeSkipFlag(cu, 0); // 编码skip Flagint skipFlagBits = m_entropyCoder.getNumberOfWrittenBits();m_entropyCoder.codeMergeIndex(cu, 0); // 编码merge idxmvBits = m_entropyCoder.getNumberOfWrittenBits() - skipFlagBits;bits = mvBits + skipFlagBits;}else{m_entropyCoder.codeSkipFlag(cu, 0);int skipFlagBits = m_entropyCoder.getNumberOfWrittenBits();m_entropyCoder.codePredMode(cu.m_predMode[0]);m_entropyCoder.codePartSize(cu, 0, cuGeom.depth);m_entropyCoder.codePredInfo(cu, 0);mvBits = m_entropyCoder.getNumberOfWrittenBits() - skipFlagBits;bool bCodeDQP = m_slice->m_pps->bUseDQP;m_entropyCoder.codeCoeff(cu, 0, bCodeDQP, tuDepthRange);bits = m_entropyCoder.getNumberOfWrittenBits();coeffBits = bits - mvBits - skipFlagBits;}m_entropyCoder.store(interMode.contexts);if (cu.getQtRootCbf(0))reconYuv->addClip(*predYuv, *resiYuv, log2CUSize, m_frame->m_fencPic->m_picCsp);elsereconYuv->copyFromYuv(*predYuv);// update with clipped distortion and cost (qp estimation loop uses unclipped values)// 计算最佳的SSEsse_t bestLumaDist = primitives.cu[sizeIdx].sse_pp(fencYuv->m_buf[0], fencYuv->m_size, reconYuv->m_buf[0], reconYuv->m_size);interMode.distortion = bestLumaDist;// 计算chroma分量if (m_csp != X265_CSP_I400 && m_frame->m_fencPic->m_picCsp != X265_CSP_I400){sse_t bestChromaDist = m_rdCost.scaleChromaDist(1, primitives.chroma[m_csp].cu[sizeIdx].sse_pp(fencYuv->m_buf[1], fencYuv->m_csize, reconYuv->m_buf[1], reconYuv->m_csize));bestChromaDist += m_rdCost.scaleChromaDist(2, primitives.chroma[m_csp].cu[sizeIdx].sse_pp(fencYuv->m_buf[2], fencYuv->m_csize, reconYuv->m_buf[2], reconYuv->m_csize));interMode.chromaDistortion = bestChromaDist;interMode.distortion += bestChromaDist;}if (m_rdCost.m_psyRd) // 计算心理视觉的rdcostinterMode.psyEnergy = m_rdCost.psyCost(sizeIdx, fencYuv->m_buf[0], fencYuv->m_size, reconYuv->m_buf[0], reconYuv->m_size);else if(m_rdCost.m_ssimRd)interMode.ssimEnergy = m_quant.ssimDistortion(cu, fencYuv->m_buf[0], fencYuv->m_size, reconYuv->m_buf[0], reconYuv->m_size, cu.m_log2CUSize[0], TEXT_LUMA, 0);interMode.resEnergy = primitives.cu[sizeIdx].sse_pp(fencYuv->m_buf[0], fencYuv->m_size, predYuv->m_buf[0], predYuv->m_size);interMode.totalBits = bits;interMode.lumaDistortion = bestLumaDist;interMode.coeffBits = coeffBits;interMode.mvBits = mvBits;cu.m_distortion[0] = interMode.distortion;// 更新costupdateModeCost(interMode);checkDQP(interMode, cuGeom);
}