参加2018之江杯全球人工智能大赛 ：视频识别问答（三）

　　今天处理一下问答部分。首先将文本处理一下，代码如下：

import os
import io
import numpy as np

def dealline(line):
    lineArr = line.split(',')
    name = lineArr[0]
    questionslist = []
    for index in range(1,len(lineArr)-2,3):
        questiondic = {}
        questionlist = []
        question = lineArr[index]
        answer1 = lineArr[index+1]
        answer2 = lineArr[index+2]
        answer3 = lineArr[index+3]
        questionlist.append(answer1)
        questionlist.append(answer2)
        questionlist.append(answer3)
        questiondic[question] = questionlist
        questionslist.append(questiondic)
    return name,questionslist
        
videodic = {}
rootdir = r"D:\ai\AIE04\VQADatasetA_20180815\train.txt"
f = open(rootdir,'r',encoding="utf-8")
for line in f:
    name,questionlist = dealline(line)
    videodic[name] = questionlist
    print(name)
np.savez("npz/question.npz",question=videodic)
print('finish')

处理成结构化数据之后，后边要对问题切分，例如：what is是一组，in front of是一组，the person是一组，in video是一组。分组的思路是从高到底逐步加词统计出现的次数，次数比较多的为一组词；或者已经有成熟的英文分组算法，也要查资料看看。