Judging by our ancient, sacred tradition, it seems possible that Joseph and his brothers and their families all wound up in Egypt evolving into a slave population for one reason: Joseph had grown up motherless.
Joseph was deprived of appropriate female influence. His mother Rachel died right after she gave birth to his little brother Benjamin.
As a result, Joseph grew up without proper female guidance. We all know what a mess that motherless boy made of his relationships to his brothers, a mess leading to generations of slavery.