Employing Coarse-grained Task to Improve Fine-grained Dialogue Topic Shift Detection

The goal of dialogue topic shift detection is to identify whether the current topic in a dialogue has shifted or not. Previous work has focused on detecting whether a topic has shifted, without delving into the finer-grained topic situations of the dialogue. To address these issues, we further explore fine-grained topic shift detection. Based on different categories of topic semantics, a multi-task learning framework is constructed by treating the labels of both coarse and fine granularity as different tasks. The topic semantics of the two granularities reinforce each other and enhance the robustness of the model. Finally, semantic coherence learning as well as weight adaptation learning are applied to alleviate the sample imbalance problem in the dataset, so that the model can distinguish different topic shift situations more effectively. Experimental results on the Chinese dataset CNTD show that the proposed model outperforms several baseline models.