Oh for Pete’s sake. First the slave was a gift to Grants wife by her slave owning father, Grant treated the man well, worked side by side with him in trying to build Hard Scrabble for which the locals were appalled that he did manual labor side by side with the man and then less than two years he freed the man when he could of sold him for $1500 which at that time was worth more than twice the annual median household income and Grant was dead broke at the time.
Nor does it take into account that, with time and experience, a person can change and grow as Grant did on the issue of slavery and Civil Rights. Grant was deeply moved by the dedication, loyalty, courage and fighting skills of the Black Soldiers he enlisted. Let’s not forget Grant was the first major Union Officer to utilize freed slaves and created the Freedman’s Bureau during the war. He also fought for and succeeded getting Black soldiers equal pay. Not to mention Grant de facto permanently won Black folks freedom by defeating the Confederacy on the Battlefield.
I would also suggest that you read what the Contemporary Black Intellectuals of that time, such as, Frederick Douglass, felt about Grants service and commitment to Freed Black Folk.
I mean if all you have is he inherited a slave from his wife that he shortly there after freed when he was dead broke all I can say is “Ward, don’t you think you’re being a little hard on the Beaver?”.