Put the freezer bowl of ice cream maker in the freezer at least 24 hours before.
In a medium mixing bowl, use a hand mixer on low speed to combine the peanut butter and sugar until smooth. Add the milk and mix on low speed until the sugar is dissolved, about 1 to 2 minutes. Stir in the heavy cream and vanilla. Cover and refrigerate 1 to 2 hours, or overnight.
Turn on the Cuisinart® ice cream maker; pour the mixture into freezer bowl and let mix until thickened, about 20-25 minutes. Five minutes before mixing is completed, add the chopped candy through the top and let mix in completely. The ice cream will have a soft, creamy texture. If a firmer consistency is desired, transfer the ice cream to an airtight container and place in freezer for about 2 hours.
Remove from freezer about 15 minutes before. Scoop the ice cream in a bowl or in waffle cone.
Add Reese's pieces and/or chopped Reese's mini cups on top of the ice cream bowl. this is optional. Enjoy!