目录

1、简介

2、Human Memories as Repositories of Events: Event Graph Knowledge Acquisition

背景

2022 CVPR CLIP-Event: Connecting Text and Images with Event Structures

2021 EMNLP The Future is not One-dimensional: Complex Event Schema Induction via Graph Modeling for Event Prediction

3、参考

1、简介

报告主题

Human Memories as Repositories of Events: Event Graph Knowledge Acquisition

报嘉宾

李曼玲（UIUC）

报告摘要

Human memories can be regarded as repositories of historical events. Event structures encapsulate the fundamental questions of Who, What, Where, When, and Why that humans discuss on a daily basis. The enormous volume of data, however, necessitates that machines have the capability of automatically obtaining events and their arguments (i.e., participants) from vast amounts of unstructured data, such as text, images, videos, etc. It is critical to enable computers to extract local event structures (i.e., who, what, where, and when) from unstructured data, and also to perform global reasoning across events (i.e., what is likely to occur, and why). 

This talk focuses on constructing event graphs to deal with real-world events that are multimedia, interconnected, probabilistic, and span a long time period. We leverage historical events to discover global event schema knowledge, such as the interaction patterns between events and the evolution patterns along a long period, which can be used as constraints for reasoning about future events. Our structural event graph modeling is able to represent the global inter-dependencies of events and long-distance interactions via arguments, leading to a comprehensive understanding of events and effective forecasting of future events.

报告人简介

Manling Li is a final-year Ph.D. student at the Computer Science Department of University of Illinois Urbana-Champaign. Her work on multimedia knowledge extraction won the ACL'20 Best Demo Paper Award and NAACL'21 Best Demo Paper Award. She is a recipient of Microsoft Research PhD Fellowship. She was selected as a EE CS Rising Star in 2022. She was awarded C.L. Dave and Jane W.S. Liu Award, and has been selected as Mavis Future Faculty Fellow. She has more than 30 publications on multimedia knowledge extraction and reasoning, and gave tutorials about multimedia IE at ACL'21, AAAI'21 and NAACL'22. She serves as a senior PC member at IJCAI 2021 and PC members for AAAI, ARR, ACL, EMNLP, NAACL, COLING, etc.

2、Human Memories as Repositories of Events: Event Graph Knowledge Acquisition

背景

 关注的是event（语义）信息

 例子 ：不仅仅关注实体

还需要关注动作等信息 

 真实世界中大家关注的其实都是事件的知识图谱

 人们对于事件的发生，首先关注的是发生了什么

 然后去找与事件相关的事件节点

也会想知道与事件相关的其他信息，如地点、时间等

 因此需要全面了解多媒体数据

具有 时序关系（前后 

 事件之间有丰富语义信息

 下游任务，预测

 时序图结构

 事件抽取（从历史的模式中挖掘知识）

 应用

对事件理解-单个事件

事件抽取-图结构-时间轴生成等

 事件生成

 文本上的事件抽取例子

找其他实体 

 挖掘实体与died实体间边上的信息

  挖掘实体与fired实体间边上的信息

 图像上事件抽取例子

 一般是图像（理解为一个事件）与图像object间关系

 目标（任务）

2022 CVPR CLIP-Event: Connecting Text and Images with Event Structures

 伪标签+对比学习

用图结构方式表示

 还需要变成text形式放进Transformer编码器进行编码

 ​​​​​

 对齐文本和图像信息（图结构对齐）

放在同一个映射空间计算距离，通过optimal transport计算

 实验（新闻网站）

 希望编码器具有很好的事件抽取和理解能力

 问答实验结果

 case实验结果

其他下游任务实验结果 

2021 EMNLP The Future is not One-dimensional: Complex Event Schema Induction via Graph Modeling for Event Prediction

多事件学习

 现有模型大多是线性结构

因此，当成事件语言模型来学习，no，他们之间的关系不是线性的，是图结构

思路

common pattern 

 学习过程

 之前工作的实验？

 作者的实验结果

 根据状态的改变来更好的理解事件的发生

 根据本文的推理能力应用在视觉上的预测

2022 NAACL  Event Schema Induction with Double Graph Autoencoders 

3、参考

录播链接 || Mila实验室/蒙特利尔大学朱兆成：基于图神经网络的知识图谱推理

Mila实验室/蒙特利尔大学朱兆成：基于图神经网络的知识图谱推理_哔哩哔哩_bilibili

参考文献

1. CLIP-Event: Connecting Text and Images with Event Structures. CVPR 2022

2. Event Schema Induction with Double Graph Autoencoders. NAACL 2022

3. The Future is not One-dimensional: Complex Event Schema Induction via Graph Modeling for Event Prediction. EMNLP 2021