Stir together all the ingredients in a small saucepan. Cook over medium-high heat while stirring frequently until all of the sugar is dissolved, and bring to a boil.
Reduce the heat to low and let simmer for 5 minutes. Do not stir.
Remove from stovetop and pour into a heatproof jar. Let cool.
Meanwhile, preheat the oven to 177 °C and grease a 9x5" loaf pan.
Using an electric mixer, cream together the butter and sugar in a large bowl. Mix in the eggs, milk and vanilla.
Stir in the flour, baking powder and baking soda until almost combined. Fold in the bananas and 1/2 cup caramel until just combined.
Spoon into prepared loaf pan and bake for 55-60 minutes or until a toothpick inserted into the center comes out clean.
Let cool and drizzle with more caramel sauce before serving.