Heat some oil in a large heavy bottomed pan with a lid. Brown the fennel on both sides- do this in batches if your pan is not large enough. Remove from the pan and season with salt and pepper.
Pour more oil into the pan and add the garlic cloves. Fry for about five minutes to brown slightly. Add the lemon juice and vinegar and reduce the mixture to half its volume over a medium heat.
Add the diced tomato, half the water, the capers, olives, thyme and sugar plus a pinch of salt and pepper. Return the fennel to the pan with the remaining water and cook for 12 minutes.
Add the lemon zest, chilli flakes and a drizzle of good olive oil. Remove the garlic cloves and set aside. Grate your Gruyere and weigh out the ricotta then set these aside too.
Whisk together the custard ingredients in a bowl then strain through a fine sieve into a jug.
In the prepared baking dish layer the bread, braised fennel, half the Gruyere then dot with the ricotta. Pour two thirds of the custard over the top, then sprinkle with salt, pepper, nutmeg and chilli flakes.
Let the pudding rest for 30-60 minutes to absorb the custard. Meanwhile, preheat the oven to 180 C, 160 Fan, Gas 4.
When ready cover the pudding with the remaining Gruyere and pour over the rest of the custard. Bake for 1 hour until set and golden. Cut into portions and serve warm.