The Governor shared to Andrea in the “Killer Within” episode of The Walking Dead that his real name is Philip and that he lost his wife in a car accident before the zombie apocalypse began. He also mentioned that he lives with his daughter.
In the comic books, the “daughter” is named Penny who became a walker when she was killed by marauders during an attack to their group during the apocalypse.
His father Philip Blake could not bear to kill her so she was put on a leash and fed human remains. After Philip’s death, his brother Brian assumed his identity and established a settlement in Woodbury and became known as The Governor.
Being the good uncle that he is, he continued to feed her niece and when he amputated Rick’s hand, he gave the body part to her. At one point, he pulled her teeth out with pliers so he can kiss her.
In the TV series, The Governor is keeping Penny in a cell and treats her as a normal person. He always tell her that “Daddy still loves you.” When Michonne found out about Penny and eventually killed her with her katana, the Governor got devastated.
Watch how Michonne killed Penny in S03E08: “Made to Suffer”: