In a large bowl, sift together flour, baking powder, and salt. Set aside.
In a standing mixer with the paddle attachment on medium speed, or with an electric mixer, beat butter and sugar until pale and fluffy. Beat in egg yolks and vanilla.
Reduce mixer speed to low and gradually add flour mixture and milk until just incorporated. Scrape dough on to a floured work surface, shape into a ball, and wrap in plastic wrap. Chill at least 2 hours or up to 2 days in advance.
To bake the cookies:
Preheat oven to 350 degrees. Line 2 baking sheets with parchment paper or silicone mats.
On a floured surface, roll chilled dough between parchment paper or wax paper to 1/8-inch thickness. Cut with heart-shaped cookie cutters. Transfer to prepared baking sheets with an offset spatula.
Repeat with the remaining dough. Scrape all remnants together and re-roll. If the dough becomes too soft or sticky, refrigerate an additional 10 minutes.
Bake until the edges are lightly browned, about 8 to 10 minutes (if thicker than 1/8 inch, they may take 10 to 15 minutes). Transfer to wire racks and cool completely before decorating with frosting (recipe follows).
To make the royal icing:
In a standing mixer fit with the whisk attachment on medium-high speed, add the egg whites and vanilla and beat until foamy.
Reduce mixer speed to low and add the powdered sugar until just incorporated and shiny. Beat on high for 5 to 7 minutes or until the mixture is thick and glossy with stiff peaks.
Add food coloring if desired, then transfer to piping bags or spread with a spatula. If using sprinkles or sugar, add quickly before the frosting hardens.
To make in advance, store in an airtight container in the refrigerator for up to 3 days.
Eggs: You can use the 2 leftover egg whites for the royal icing instead of the pasteurized egg whites (you'll need 3 egg whites in total). There is some risk of food-borne illness whenever you consume raw eggs.
Royal icing with meringue powder (instead of egg whites): In a mixer on high speed, beat together 4 cups powdered sugar, 3 tablespoons meringue powder, 1 teaspoon vanilla extract, and 9 tablespoons water for 1 to 2 minutes. Lift the whisk attachment out of the icing. The icing should drizzle evenly off the attachment within a few seconds. If not, add another tablespoon water. Mix again and repeat the process. If the icing is too thin, add a little more powdered sugar.
Toppings: If you’re planning on using sprinkles, dragees, or colored sugar on the icing, work quickly because the surface of royal icing dries pretty fast.
Spread it on: Instead of a piping bag, you can use a spatula or a pastry brush to apply the icing.
Stiff frosting: If the frosting begins to feel stiff, loosen it up with a few drops of hot water at a time. Stir until well-blended.
Taking a break: A damp paper towel on the surface of the icing helps keep the icing from hardening if you need to step away.
Make ahead: Both the cookie dough and the royal icing can be made up to 3 days in advance (store covered in the refrigerator).
Freezer: Cookie dough and royal icing can be frozen for up to 2 months.