Preheat your oven to 175°C (350°F). Generously grease and flour a 9×5 inch loaf pan, or line it with parchment paper for easy removal.
In a large mixing bowl, whisk together the all-purpose flour, granulated sugar, baking powder, baking soda, salt, ground cinnamon, and ground nutmeg. Whisk thoroughly to distribute all the leavening and spices evenly.
In a separate medium bowl, lightly whisk the eggs. Gradually add the milk, vegetable oil, and vanilla extract, whisking until well combined. Gently fold in the diced apples, ensuring they are coated.
Pour the wet ingredient mixture into the dry ingredients. Using a spatula or wooden spoon, mix until just combined. Be careful not to overmix; a few lumps are perfectly fine and will lead to a tender bread.
In a small bowl, combine the melted butter, brown sugar, and cinnamon for the fritter swirl. Stir until it forms a thick, paste-like consistency.
Pour half of the bread batter into the prepared loaf pan, spreading it evenly. Spoon dollops of the fritter swirl mixture over the batter. Cover with the remaining bread batter, gently spreading to cover the swirl.
Bake for 50-60 minutes, or until a wooden skewer inserted into the center comes out clean. If the top browns too quickly, tent it loosely with aluminum foil. Once baked, remove the loaf from the oven and let it cool in the pan for 10-15 minutes on a wire rack before carefully removing it to cool completely.
While the bread cools, whisk together the powdered sugar, milk (or cream), and vanilla extract in a small bowl until smooth and free of lumps. Adjust the consistency by adding more powdered sugar for a thicker glaze or more milk for a thinner glaze. Once the bread is completely cool, generously drizzle the glaze over the top.