Preheat oven to 350°F (175°C). Line two 12-cup muffin pans with paper liners.
In a large bowl, whisk together flour, sugar, baking powder, and salt until well combined.
In a separate medium bowl, combine milk, oil, eggs, and vanilla extract. Whisk until smooth.
Pour wet ingredients into dry ingredients. Mix on medium speed for 2 minutes until well combined.
Slowly add hot water while mixing on low. Batter will be thin. This is normal.
Fill each cupcake liner two-thirds full (about 1/4 cup batter per cupcake).
Bake 18-20 minutes until toothpick inserted in center comes out clean.
Cool in pan 5 minutes, then transfer to wire rack to cool completely before frosting.
For frosting: Beat butter until fluffy, add powdered sugar gradually, then cream and vanilla. Whip until light and spreadable.