In a large bowl using a hand mixer or stand mixer fitted with the paddle attachment, beat the butter and both sugars together on medium speed until combined, about 2 minutes. Beat in the egg, milk, and vanilla extract until combined. Scrape down the sides and bottom of the bowl as needed.
In a separate bowl, whisk the flour, cocoa powder, hot cocoa mix, baking soda, and salt together until combined. Add the dry ingredients to the wet ingredients and mix on low speed until just combined. The dough will be very soft and sticky.
Cover tightly and chill the dough in the refrigerator for at least 2 hours and up to 3 days. Chilling is required for this dough.
Preheat oven to 350°F (177°C). Line two large baking sheets with parchment paper or silicone baking mats.
Roll the dough into 1.5 Tablespoon-sized balls. Place 9 balls onto each prepared baking sheet, leaving room in between for slight spreading. Bake for 10-11 minutes, or until the edges are set and the centers are slightly soft.
Remove from the oven and immediately place a marshmallow half, cut-side down, onto each warm cookie. Return the baking sheets to the oven and bake for 2 minutes, or until the marshmallows are soft and puffy.
Remove from the oven and let cool on the baking sheets for 5 minutes. Meanwhile, melt the chocolate in a double boiler or in the microwave in 30-second intervals, stirring in between, until smooth. Use a spoon to drizzle or dollop the melted chocolate over each marshmallow. You can also dip each cookie into the melted chocolate, if desired.
Let the chocolate set completely at room temperature or in the refrigerator. Cookies stay soft for days! Cover and store leftover cookies at room temperature for up to 1 week.