Margaret of Anjou was a woman. Obvious I know, but critical to the understanding of her place in 15th centry society. The right to bear arms and thus the accompanying heraldry was exclusively reserved for the males. A woman's only entitlement to arms was therefore from her father or husband.
Margaret of Anjou was the second eldest daughter of René of Anjou. Until her marriage to Henry VI, her arms (strictly speaking, arms belonged to a family rather than a surname) would be those of Anjou
From the time of her marriage to Henry, and thus becoming Queen of England, her arms would be those of Henry link
Livery was often (but not always) associated with the main colours of the coat of arms. We have very little information about livery colours (as opposed to coats of arms) so often liveries are "best guess". Given the predominant colours are red and blue then you could probably use those for any Lancastrian household troops.
Bear in mind also that the wearing of livery was part of the "livery and maintenance" scheme whereby a man would contract to serve a lord in return for the right to wear the lord's livery and the provision of food and accommodation (maintenance). As a woman, Margaret of Anjou was not empowered to issue such contracts, so she would not have had a retinue of her own. Any troops she might have indirect authority over would be those loyal to Henry VI or one of the noble lords loyal to him.