Adjust an oven rack to the top middle position and preheat oven to 350 degrees.
In a standing mixer fit with the paddle attachment, or with an electric mixer by hand, cream the butter on medium-high speed for 30 seconds. Add sugar, baking powder, baking soda, pumpkin pie spice, and salt. Continue mixing until all the ingredients have been incorporated, scraping down the sides as necessary.
With the mixer still running, add the eggs, vanilla, and pumpkin. Mix until the ingredients are uniformly combined. Add 1 cup of flour at a time, mixing well after each addition and scraping down the sides of the bowl. If you cannot fit all the flour in your standing mixer bowl, stir in any remaining flour by hand.
Drop rounded teaspoonfuls of batter onto ungreased cookie sheets about 2 inches apart. Bake until the tops have set but the cookies are still soft, about 10 to 12 minutes per sheet. Transfer to a cooling rack and cool completely before frosting.
To make the frosting:
Melt the butter in a small skillet over medium heat until the foaming subsides. Continue to heat the butter until it turns light brown in color, about 3 minutes. Remove from heat and pour into a bowl.
Add brown sugar, milk, and vanilla and powdered sugar. Whisk until smooth. Frost the cooled cookies with 1 to 2 teaspoons of frosting each. Garnish with a sprinkling of cinnamon and sugar if desired.
Pumpkin pie spice: If you don't have the blend on hand already, make your own by whisking together 1 teaspoon ground cinnamon + 1/4 teaspoon EACH ground cloves, ground allspice, ground nutmeg, and ground ginger.
Pumpkin purée: Canned pumpkin pureé (not pumpkin pie filling) is convenient for this recipe. If you want to make your own pureé, preheat the oven to 375 degrees. Cut your pumpkin in half and remove all seeds and fibrous strings. Cut each half into 2 or 3 pieces, then roast until tender, 60 to 90 minutes. Cool completely, then scoop out the pumpkin flesh from the skin. Blend in a food processor blender until smooth. Finally, drain over cheesecloth or paper towels to remove excess water.
Yield: This recipe makes about 60 (2-inch) cookies.
Storage: Store extra cookies in an airtight container for up to 2 weeks at room temperature or 2 months in the refrigerator. If you live in a cold climate, store them on a patio or in a chilly garage for an even longer shelf-life.
Make ahead: The cookie dough can be made in advance and stored in the refrigerator or freezer.
Freezer: Cooled cookies can be stored in freeze-safe bags, labeled, dated, and frozen for up to 6 months. Thaw at room temperature.