The susceptibility of Field Programmable Gate Arrays (FPGAs) to power analysis attacks has gained more attention lately. While this was predicted a long time ago, it was not until recently that successful attacks were reported. Xilinx FPGA families before the UltraScale architecture offer little when it comes to protecting the encryption keys from being retrieved via power analysis attacks. In this paper, we propose a scheme for securing Xilinx FPGAs against these attacks by using the on-chip Xilinx Analog-to-Digital Converter (XADC) to monitor the power consumption of the chip during an encrypted partial bitstream configuration. Our proposed solution to power analysis attacks involves activating or deactivating power consumer circuits to level out the power consumption of the FPGA with the aim of masking the power signature of the encryption key. We present here our implementation strategy and preliminary power consumption measurements of FPGA primitives that can be used to achieve this objective.