Walk before you run: using heuristic evaluation to assess a training tool prototype

Context: One of the chief challenges of Global Software Development (GSD) is for globally, culturally and linguistically diverse team members to communicate effectively. Failing to meet this challenge can lead to misunderstandings that impede project success. To prepare practitioners for remote and diverse forms of communication we have developed VENTURE: a simulationbased training platform that aims to increase cultural awareness. Aim of study: Having developed a prototype training platform, we now assess whether VENTURE can deliver real learning and increase cultural awareness. Method: A survey was designed based on educational theory. Using a heuristic evaluation approach, a group of potential users trialed the system and completed a proof of learning survey before and after using the platform. Survey results, and VENTURE’s own automated assessment scores, were analysed and compared. Results: Results indicate that the platform has the potential to provide useful and meaningful training for cultural awareness in GSD. A secondary result is that the assessment scheme proved useful in providing both evidence of learning and highlighting areas where VENTURE could be improved. Conclusion: Researchers and practitioners can gain an understanding of how to evaluate, train and measure soft skills, such as communication, important to all forms of software development. The methodology applied served to adapt the environment to the needs of potential users and the actual problems of GSD.