With the rapid development and application of autonomous vehicles, human-autonomous vehicle interaction (HAI) has recently gained importance. Simulation is an effective and efficient approach for the research and testing in the HAI domain, and scenarios are crucial to the validity and experience of HAI simulators. However, research on systematically building scenarios is still lacking. This paper proposes the concept of the narrative scenario in the HAI simulation, and a three-stage framework for building narrative scenarios is established. First, key-plot scenarios are collected from the whole journey analysis and deconstructed into elements. After extracted and categorized into a library, the physical and narrative attributes of scenario elements are defined. Finally, scenarios in simulators can be rebuilt in a narrative sequence driven by task goals.