Projects per year
Abstract / Description of output
In situ evaluations of novel authentication systems, where the system is evaluated in its intended usage context, are often infeasible due to ethical and legal constraints.
Consequently, researchers evaluate their authentication systems in the lab, which questions the ecological validity. In this work, we explore how VR can overcome the shortcomings of authentication studies conducted in the lab and contribute towards more realistic authentication research. We built a highly realistic automated teller machine (ATM) and a VR replica to investigate through a user study (N=20) the impact of in situ evaluations on an authentication system's usability results. We evaluated and compared: Lab studies in the real world, lab studies in VR, in situ studies in the real world, and in situ studies in VR. Our findings highlight 1) VR`s great potential to circumvent potential restrictions researchers experience when evaluating authentication schemes and 2) the impact of the context on an authentication system`s usability evaluation results. In situ ATM authentications took longer (+24.71% in the real world, +14.17% in VR) than authentications in a traditional (VR) lab environment and elicited a higher sense of being part of an ATM authentication scenario %in both the real world
compared to a real-world and VR-based evaluation in the lab. Our quantitative findings, along with participants` qualitative feedback, provide first evidence of increased authentication realism when using VR for in situ authentication research. We provide researchers with a novel research approach to conduct (simulated) in situ authentication research, discuss our findings in the light of prior works, and conclude with three key lessons to support researchers in deciding when to use VR for in situ authentication research.
Consequently, researchers evaluate their authentication systems in the lab, which questions the ecological validity. In this work, we explore how VR can overcome the shortcomings of authentication studies conducted in the lab and contribute towards more realistic authentication research. We built a highly realistic automated teller machine (ATM) and a VR replica to investigate through a user study (N=20) the impact of in situ evaluations on an authentication system's usability results. We evaluated and compared: Lab studies in the real world, lab studies in VR, in situ studies in the real world, and in situ studies in VR. Our findings highlight 1) VR`s great potential to circumvent potential restrictions researchers experience when evaluating authentication schemes and 2) the impact of the context on an authentication system`s usability evaluation results. In situ ATM authentications took longer (+24.71% in the real world, +14.17% in VR) than authentications in a traditional (VR) lab environment and elicited a higher sense of being part of an ATM authentication scenario %in both the real world
compared to a real-world and VR-based evaluation in the lab. Our quantitative findings, along with participants` qualitative feedback, provide first evidence of increased authentication realism when using VR for in situ authentication research. We provide researchers with a novel research approach to conduct (simulated) in situ authentication research, discuss our findings in the light of prior works, and conclude with three key lessons to support researchers in deciding when to use VR for in situ authentication research.
| Original language | English |
|---|---|
| Title of host publication | Proceedings of the 2022 IEEE Conference on Virtual Reality and 3D User Interfaces - VR 2022 |
| Publisher | Institute of Electrical and Electronics Engineers |
| Pages | 301-310 |
| Number of pages | 10 |
| ISBN (Electronic) | 978-1-6654-9617-9 |
| ISBN (Print) | 978-1-6654-9618-6 |
| DOIs | |
| Publication status | Published - 20 Apr 2022 |
| Event | IEEE Conference on Virtual Reality and 3D User Interfaces (IEEE VR) - Virtual Conference Duration: 12 Mar 2022 → 16 Mar 2022 https://ieeevr.org/2022/ |
Publication series
| Name | IEEE Annual International Symposium Virtual Reality |
|---|---|
| Publisher | IEEE |
| ISSN (Print) | 2642-5246 |
| ISSN (Electronic) | 2642-5254 |
Conference
| Conference | IEEE Conference on Virtual Reality and 3D User Interfaces (IEEE VR) |
|---|---|
| Abbreviated title | IEEE VR 2022 |
| Period | 12/03/22 → 16/03/22 |
| Internet address |
Keywords / Materials (for Non-textual outputs)
- Virtual Reality
- Authentication
- In Situ Research