Preheat the oven to 350°F. Grease a 9x5-inch loaf pan and line it with parchment paper if desired.
In a medium bowl, whisk the flour, baking soda, baking powder, salt, cinnamon, nutmeg, and cloves together.
In a large bowl, whisk the granulated sugar, brown sugar, oil, eggs, pumpkin puree, milk, and vanilla until smooth.
Gradually stir the dry ingredients into the pumpkin mixture just until combined. Do not overmix.
Pour the batter into the prepared pan, smooth the top, and bake for 50–55 minutes, or until a toothpick inserted in the center comes out clean.
Cool the bread in the pan for 10 minutes, then transfer it to a wire rack to cool completely.
Beat the softened cream cheese and butter together until smooth and creamy.
Beat in the powdered sugar and vanilla. Add milk a little at a time until the frosting is thick and spreadable, then spread it over the cooled loaf.