Make your own homemade cocoa mix to suit your needs. This is an easy to make cocoa mix that can be made dairy free.
If you don't have powdered milk on hand you can leave it out of the mix. But I think it does make the mix more creamy. For the dairy free option, I use Better Than Milk Powder Milk Alternative or Vance's DariFree for the powdered milk.