Juneteenth — Emancipation Day, 1865 — was supposed to start a new era of black wealth creation. After 12 generations of being subject to slavery’s institutionalised theft, four million African-Americans were now free to earn incomes and degrees, hold property, weather hard times and pass down wealth to the next generation. They would surely scramble up the economic ladder, if not in one generation then in a few.

Eight generations later, the racial wealth gap is both yawning and growing. The typical black family has just one tenth the wealth of the typical white one. In 1863, black Americans owned one half of 1 per cent of the national wealth. Today it’s just over 1.5 per cent for roughly the same percentage of the overall population. The cause of that stagnation has largely been invisible, hidden by the assumption of progress after the end of slavery and the achievements of civil rights. But for every gain black Americans made, people in power created new bundles of discrimination, largely hidden from sight, that thwarted, again and again, the economic promise of emancipation.

It is a common misperception that the racial wealth gap is an unfortunate legacy of a bygone era. The myth goes like this: slavery and Jim Crow bred black poverty. In the 20th century, millions of black families moved out of the South chasing high wages in urban industry. But after a few decades the factories closed, inner cities decayed and a “complex tangle of pathology” emerged in single-parent households and soaring incarceration rates.

If those were the causes, then the solutions seemed evident: end job, housing and school discrimination, enforce civil rights and sprinkle the market with affirmative action. If those things failed to close the gap, then the problem was one of follow-through. If black people would just move to the suburbs, marry, finish school, train up and play by the rules, the gap would vanish.

But that is a myth, concealed in what Ta-Nehisi Coates terms “the quiet plunder”. In the grand narrative of freedom and civil rights, the disadvantages that persist are invisible precisely because people in power continuously innovated new forms of discrimination.

The nation industrialised between the 1870s and the 1910s, but instead of vanishing, the disadvantages confronting black Americans simply morphed. Slavery’s violent theft was replaced by convict leasing, sharecropping, and — after a heroic civil rights struggle between 1863 and 1873 — disenfranchisement and legal discrimination, or Jim Crow.

When African-descended people began to move into industrial cities during and after the First World War, they faced surcharges on opportunities — high rents, for instance — and new forms of prejudice. “The Negro is up against the white man’s standard, without the white man’s opportunity,” argued black mathematician and sociologist Kelly Miller in 1930. African-American migrants who fled the South crammed into marginal neighbourhoods where landlords gouged them thanks to their lack of access to alternatives. Isabella Wilkerson, who chronicled the journey of black Southerners north, noted that migrants to Chicago “were living in virtual slave cabins stacked on top of one another”.

As the Great Migration took shape, escalators into the middle class began moving for black families, but they moved slowly. Black workers joined unions, trained up, doubled up in housing, scrimped and saved, and played by the rules. But Jim Crow followed black people out of the South, re-emerging in federal housing and social policies. Southern Democrats were gatekeepers of New Deal legislation, so when the federal government created economic entitlements, it built in white privilege.

African-Americans seeking home loans found themselves redlined — ineligible for credit — because the Government would not guarantee the loans. Housing costs rose without giving black residents a stake in the value of their homes, while neighbourhoods decayed from lack of investment. Real estate professionals and developers acted hand-in-glove with racist lending policies, hemming black families into neighbourhoods where disadvantages compounded one another. Low-quality schools and services in poor neighbourhoods were a drag on upward mobility. Since housing equity makes up about two thirds of median household wealth, excluding black Americans from establishing equity during a time of unprecedented rises in home values locked in and exacerbated wealth disparities.

Social security was unavailable to farm and domestic workers, including most black workers. And after fighting for a double victory against fascism abroad and racism at home, black Second World War veterans returned to find their Government Issue benefits were a sham. As one historian argues, “the GI Bill did create a more middle-class society, but almost exclusively for whites ... [it] was deliberately designed to accommodate Jim Crow”. Local officials directing benefits such as job training steered black applicants to unskilled and white applicants to skilled and semi-skilled jobs while denying home and business loans to black applicants. When accepted at all, black veterans typically attended historically black colleges, many of which were underfunded, while whites steamed into flagship universities.

By 1947, veterans made up half the national college enrolments. Veterans who attended college tended to have children and grandchildren who attended college, creating a pipeline to white-collar careers. And the disadvantages of missing out on GI benefits was like negative interest compounding in each generation.

It did not need to be that way.

The racial wealth gap persisted in spite of an unprecedented nose-dive in economic inequality more broadly. In the middle of the 20th century, workers took home a greater share of the nation’s income than at any time before or since. And missing out on that enormous expansion of new middle-class wealth meant not benefiting from investments in home and education that became a lever on opportunity for the next generation.

Even worse, the signal victories of the civil rights movement, which convinced many white Americans that discrimination was a thing of the past, came just as economic structures were disadvantaging those who had missed out. Workers’ gains petered out after 1974. Unionised African-American workers were disproportionately vulnerable to deindustrialisation, and policy responses to economic challenges of the 1970s weakened unions, promoting income inequality.

Rural black Americans, still the largest rural minority, fared no better. Those who remained in the locales in which their ancestors were enslaved live in precisely those places with the least intergenerational upward mobility in the 21st century. Since the 1940s, African-American farmers have been dispossessed of their land with the help of government agencies. Rural African-Americans have among the worst health outcomes in the nation.

Amid all this, the politics of white entitlement and resentment led to mass incarceration. Early in the 20th century, black men were four times as likely to do time. By the 1980s, it was 11 times. While, on their face, criminal-justice policies have little to do with economics or discrimination, in reality, this was yet another example of the economic hurdles facing African-Americans morphing and changing through policy actions. Incarceration is often an economic life sentence for families of those convicted. Missing from school, family, voting polls and work, the economic toll struck black families, even as some of the earlier hurdles, such as housing discrimination, were being abolished by law.

This was like an insidious game of Whac-A-Mole: as soon as African-Americans fought to remove one barrier, another new one sprang up based on false perceptions that black people were benefiting unfairly. Over the past 35 years, the racial wealth gap has grown substantially. Typical black household incomes halved between 1983 and 2016 while median white household incomes have risen by one third.

The 2008 housing crisis and its aftermath show the tragedy of the process, as African-Americans who finally attained the American dream of home ownership were precisely those most likely to find themselves underwater. Before the crisis, black customers, regardless of income, were steered to subprime loans and consequently faced a higher foreclosure rate than white borrowers. Since most started with less wealth and saw a greater proportion of it disappear as home values crashed, black homeowners came out of the crisis in much worse shape than white borrowers. Since the Great Recession, black workers saw the lowest wage gains of any group.

Lower incomes and correspondingly low savings rates prevented blacks from transferring wealth to the next generation as whites in cities and suburbs do. In fact, the costs of elder care are often like a negative inheritance. One fifth of African-American families have a net worth of $0 or below; 75 per cent have less than $10,000 for retirement.

The enduring barriers to black economic equality are structural rather than individual. Instead of working to alleviate them, the party in power is rolling back civil rights protections, reviving the War on Drugs and expanding private prisons. Black aspirants to the American dream continue to face lending and real estate discrimination, and “predatory inclusion” in higher education. Escalators into the middle class have slowed and stalled, and the rung of the economic ladder one starts on is most likely where one will end up.

Unless present economics change, black families will be poorer on the 175th anniversary of emancipation than they were in 1980.

• Calvin Schermerhorn is professor of history in Arizona State University’s School of Historical, Philosophical and Religious Studies and author of The Business of Slavery and the Rise of American Capitalism, 1815-1860