More than 9 million people live in shelters globally, often in extremely hot climates. The thermal performance of shelters is often overlooked in the design process, despite being a consideration second only to safety in surveys of camp dwellers. Indeed, indoor temperatures exceeding 40 °C have been recorded. To aid in improving conditions, we examine the roles building simulation and prototyping could play in forecasting shelter thermal performance as part of a new shelter design process. The thermal performance of seven prototypes, built in the refugee camp of Azraq, was monitored during the hot season to test the suitability of four design approaches: (1) “blind” (uncalibrated) models, (2) calibrated models, (3) on-site prototypes and (4) off-site prototypes. These included the original shelter and six design alternatives implementing different overheating countermeasures. The results demonstrate that blind models are sensitive to the judgement of uncertainties but were still qualitatively useful. Model calibration vastly improves the agreement and significantly enhances forecasts of performance for the design alternatives, which remained similar across examined climates. We therefore conclude that simulation and prototyping, either on-site or off-site, should be adopted within the shelter design process before mass deployment, to create better living conditions for their dwellers.