The development of maritime autonomous surface ships (MASS) is advancing globally, supported by the IMO’s forthcoming non-mandatory MASS Code and the upcoming experience-building phase. Simulation-based verification of autonomous navigation systems is essential for ensuring safety, especially in congested waters. This study focuses on the Singapore Strait and constructs realistic scenarios reflecting traffic density, navigational risks and operational constraints. The ANS is evaluated using a full-mission simulator with expert feedback from experienced captains. The research aims to establish a practical ANS evaluation methodology and provide insights that enhance system reliability and support future validation protocols.