I went back to the book and it says “It had long been the custom amongst the dragonlords of Valyria to wed brother to sister, to keep the bloodlines pure”.

This is in the context of Aegon the Conqueror, so, yes it was already a practice before the Doom.