A Conceptual Model for Integrating Mixed Reality and Artificial Intelligence in Cultural Heritage Application

Alifeony Mutiara Diarsya; Ari Widyanti; Tara Taufiqur Rahman

doi:10.61221/jriem.v4i1.74

Penulis

Alifeony Mutiara Diarsya Magister Teknik dan Manajemen Industri, Fakultas Teknologi Industri, Institut Teknologi Bandung
Ari Widyanti Department of Industrial Engineering, Bandung Institute of Technology, Indonesia2
Tara Taufiqur Rahman Indonesia AI Institute, Indonesia

DOI:

https://doi.org/10.61221/jriem.v4i1.74

Kata Kunci:

Mixed Reality, Artificial Intelligence, cultural heritage application

Abstrak

This study conducts a systematic literature review (SLR) to justify the integration of Mixed Reality (MR) and Artificial Intelligence (AI) in cultural heritage applications, aiming to enhance user engagement through a conceptual model. The review identifies four key dimensions namely, immersion, adaptivity/personalization, embodiment, and interactivity, as critical factors influencing presence and engagement. Through the synthesis of 21 studies, the proposed conceptual model demonstrates how embodiment, immersion, interactivity, adaptive and personalization collectively strengthen user presence and engagement. Immersion establishes the psychological foundation for presence, reinforced by dynamic AI-driven content and contextualized experiences. Adaptivity and personalization enable systems to respond to user preferences, fostering repeated interaction and experience sharing. Embodiment, through body ownership and perspective-taking, transforms users into active narrative agents, deepening emotional and cognitive involvement. Interactivity, supported by multimodal inputs and gesture-based controls, acts as a direct driver of engagement by enabling responsive and participatory storytelling. Collectively, this conceptual model provides demonstrate that AI-MR integration strengthens the presence and engagement relationship for cultural heritage preservation, interpretation, and public participation.

Referensi

1. Aloqaily, M., Bouachir, O., & Karray, F. (2023). Digital twin for healthcare immersive services: fundamentals, architectures, and open issues. In Digital Twin for Healthcare (pp. 39–71). Elsevier. https://doi.org/10.1016/B978-0-32-399163-6.00008-1

2. Ariya, P., Wongwan, N., Worragin, P., Intawong, K., & Puritat, K. (2025). Immersive realities in museums: evaluating the impact of VR, VR360, and MR on visitor presence, engagement and motivation. Virtual Reality, 29(3), 1–20. https://doi.org/10.1007/s10055-025-01201-5

3. Belenioti, Z. C., & Kypri, C. (2025). A Cutting-Edge Exploration: A Literature Review on Optimizing Museum Experiences Through Digital Transformation and VR Tour Software with 3DVista. Springer Proceedings in Business and Economics, 247–259. https://doi.org/10.1007/978-3-031-87019-4_17

4. Bertocci, S., Bigongiari, M., & Pasquali, A. (2025). Digitisation of the wooden maquette at Casa Buonarroti: A project never realised for the facade of San Lorenzo in Firenze. 1–8. https://doi.org/10.1145/3719236.3719260

5. Bronzin, T., Prole, B., Stipić, A., & Pap, K. (2021). Artificial Intelligence (AI) Brings Enhanced Personalized User Experience. 1070–1075. https://doi.org/10.23919/MIPRO52101.2021.9596938

6. Bronzin, T., Prole, B., Stipić, A., & Pap, K. (2022). The Proposed Method of Measuring How Mixed Reality Can Affect the Enhancement of the User Experience. 894–899. https://doi.org/10.23919/MIPRO55190.2022.9803734

7. Chateauvert, J., Jégo, J.-F., & Tsampounaris, G. (2025). Remixing the Flying Words Project. 1–3. https://doi.org/10.1145/3721244.3742448

8. Constantinides, N., Constantinides, A., Koukopoulos, D., Fidas, C., & Belk, M. (2024). CulturAI: Exploring Mixed Reality Art Exhibitions with Large Language Models for Personalized Immersive Experiences. 102–105. https://doi.org/10.1145/3631700.3664874

9. Dana, T., & Yevgeniya, D. (2025). Approaches to real-time XR visualization. Procedia Computer Science, 265, 616–619. https://doi.org/10.1016/j.procs.2025.07.228

10. De Paolis, L. T., Gatto, C., Corchia, L., & De Luca, V. (2023). Usability, user experience and mental workload in a mobile Augmented Reality application for digital storytelling in cultural heritage. Virtual Reality, 27(2), 1117–1143. https://doi.org/10.1007/s10055-022-00712-9

11. Dima, M., Daylamani-Zad, D., & Vr, E. (2024). Mixed Reality Heritage Performance As a Decolonising Tool for Heritage Sites Vangelis Lympouridis. ArXiv Preprint ArXiv:2404.07348, 1–5. https://doi.org/https://doi.org/10.48550/arXiv.2404.07348

12. Dubovi, I. (2022). Cognitive and emotional engagement while learning with VR: The perspective of multimodal methodology. Computers and Education, 183. https://doi.org/10.1016/j.compedu.2022.104495

13. Ejjami, R. (2024). The Adaptive Personalization Theory of Learning: Revolutionizing Education with AI. Journal of Next-Generation Research 5.0, 5, 1–18. https://doi.org/10.70792/jngr5.0.v1i1.8

14. Fan, X., Jiang, X., & Deng, N. (2022). Immersive technology: A meta-analysis of augmented/virtual reality applications and their impact on tourism experience. Tourism Management, 91, 1–17. https://doi.org/10.1016/j.tourman.2022.104534

15. Forster, P. P., Karimpur, H., & Fiehler, K. (2022). Why we Should Rethink Our Approach to Embodiment and Presence. Frontiers in Virtual Reality, 3, 1–14. https://doi.org/10.3389/frvir.2022.838369

16. Gruber, A., & Kaplan-Rakowski, R. (2024). A Comparative Study on the Impact of High-Immersion Virtual Reality and Video Narratives on the Development of Empathy. 1–35. https://doi.org/10.2139/ssrn.4906309

17. Guo, W., Zhao, W., Sun, G., Li, Y., Ye, Y., & Wang, S. (2024). Enhancing the Digital Inheritance and Embodied Experience of Zen based on Multimodal Mixed Reality System. 1–3. https://doi.org/10.1145/3641234.3671076

18. Hammady, R., Ma, M., Strathern, C., & Mohamad, M. (2020). Design and development of a spatial mixed reality touring guide to the Egyptian museum. Multimedia Tools and Applications, 79(5–6), 3465–3494. https://doi.org/10.1007/s11042-019-08026-w

19. Huang, D., Huang, K., Wang, R., & Hu, B. A. (2025). Body Oracle: Exploring Somatic Hieroglyphs for Collective Bodily Awareness. 117–124. https://doi.org/10.1145/3749893.3749912

20. International Organization for Standardization. (2010). ISO Standard No. 9241-210:2010.

21. Kazlauskaitė, R. (2022). Embodying ressentimentful victimhood: virtual reality re-enactment of the Warsaw uprising in the Second World War Museum in Gdańsk. International Journal of Heritage Studies, 28(6), 699–713. https://doi.org/10.1080/13527258.2022.2064897

22. Kourtesis, P. (2024). A Comprehensive Review of Multimodal XR Applications, Risks, and Ethical Challenges in the Metaverse. Multimodal Technologies and Interaction, 8(11), 1–46. https://doi.org/10.3390/mti8110098

23. Lee, H. (2025). A unified conceptual model of immersive experience in extended reality. Computers in Human Behavior Reports, 18, 1–13. https://doi.org/10.1016/j.chbr.2025.100663

24. Li, Y., Wang, S., Sun, X., Yang, L., Zhu, T., Chen, Y., Zhao, K., Zhao, Y., Li, M., & LC, R. (2025). Reality as Imagined: Design and Evaluation of a TeleAbsence-Driven Extended Reality Experience for (Re) Interpreting Urban Cultural Heritage Narratives Across Time. International Journal of Human-Computer Interaction, 1–31. https://doi.org/10.1080/10447318.2025.2554296

25. Makransky, G., & Petersen, G. B. (2021). The Cognitive Affective Model of Immersive Learning (CAMIL): a Theoretical Research-Based Model of Learning in Immersive Virtual Reality. Educational Psychology Review, 33(3), 937–958. https://doi.org/10.1007/s10648-020-09586-2

26. O’Brien, H. L., Roll, I., Kampen, A., & Davoudi, N. (2022). Rethinking (Dis)engagement in human-computer interaction. Computers in Human Behavior, 128, 1–11. https://doi.org/10.1016/j.chb.2021.107109

27. Papadopoulos, T., Evangelidis, K., Kaskalis, T. H., Evangelidis, G., & Sylaiou, S. (2021). Interactions in augmented and mixed reality: An overview. Applied Sciences (Switzerland), 11(18), 1–24. https://doi.org/10.3390/app11188752

28. Pietroni, E. (2025). Multisensory Museums, Hybrid Realities, Narration, and Technological Innovation: A Discussion Around New Perspectives in Experience Design and Sense of Authenticity. Heritage, 8(4), 1–61. https://doi.org/10.3390/heritage8040130

29. Rokhsaritalemi, S., Sadeghi-Niaraki, A., & Choi, S. M. (2020). A review on mixed reality: Current trends, challenges and prospects. Applied Sciences (Switzerland), 10(2), 1–26. https://doi.org/10.3390/app10020636

30. Sariman, M. S., Othman, M., Akir, R. M., Mahamad, A. K., & Ab Rahman, M. A. (2024). A Review: Current Trend of Immersive Technologies for Indoor Navigation and the Algorithms. Pertanika Journal of Science and Technology, 32(2), 955–977. https://doi.org/10.47836/pjst.32.2.25

31. Škola, F., Rizvić, S., Cozza, M., Barbieri, L., Bruno, F., Skarlatos, D., & Liarokapis, F. (2020). Virtual reality with 360-video storytelling in cultural heritage: Study of presence, engagement, and immersion. Sensors (Switzerland), 20(20), 1–17. https://doi.org/10.3390/s20205851

32. Sung, E. C., Han, D.-I., Bae, S., & Kwon, O. (2022). What drives technology-enhanced storytelling immersion? The role of digital humans. Computers in Human Behavior, 132, 1–10. https://doi.org/10.1016/j.chb.2022.107246

33. Teruggi, S., Grilli, E., Fassi, F., & Remondino, F. (2021). 3D SURVEYING, SEMANTIC ENRICHMENT and VIRTUAL ACCESS of LARGE CULTURAL HERITAGE. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 8(M-1–2021), 155–162. https://doi.org/10.5194/isprs-annals-VIII-M-1-2021-155-2021

34. Wang, B. (2024). The Application of Interactive Design Technology in Digital Intelligent Exhibition Display. Computer-Aided Design and Applications, 21(S28), 296–308. https://doi.org/10.14733/cadaps.2024.S28.296-308

35. Xiong, H., Nie, K., Zeng, J., Shen, S., & Guo, M. (2025). Animating the Ephemeral: Transforming Edible Cultural Heritage into Dynamic Digital Heritage through AI and Mixed Reality. 1–8. https://doi.org/10.1145/3743093.3771655

36. Yang, S. (2023). Sentiment and Storytelling: What Affect User Experience and Communication Effectiveness in Virtual Environments? Journalism and Media, 4(1), 1–12. https://doi.org/10.3390/journalmedia4010003

37. Yannier, N., Crowley, K., Do, Y., Hudson, S. E., & Koedinger, K. R. (2022). Intelligent science exhibits: Transforming hands-on exhibits into mixed-reality learning experiences. Journal of the Learning Sciences, 31(3), 335–368. https://doi.org/10.1080/10508406.2022.2032071

38. Yu, T.-C., Huang, M.-X., Liu, Y.-H., Wu, H.-Y., Wang, B.-S., Tsai, H.-C., & Chien, S.-Y. (2025). A Systematic Review of Integrating Mixed Reality and Artificial Intelligence in Museums: Enhancing Visitor Experiences and Innovating Exhibit Design. Proceedings of the Annual Hawaii International Conference on System Sciences, 1533–1542. https://doi.org/10.24251/hicss.2025.185

39. Zhou, D., & Globa, A. (2025). Social Connectedness in Older Adults: The Role of MR and Haptic Interaction in Remote Digital Storytelling. 807–818. https://doi.org/10.1145/3764687.3769932