When Bernie Sanders criticised Barack Obama’s accepting of $400,000 to address a Wall Street conference as “unfortunate”, many liberals leapt to the former president’s defence. Obama is a private citizen now, they reminded us, and hence he is free to enter into any contractual agreement he pleases. Further, he is far from the first former government official to profit from well-remunerated public speaking gigs. Why should Obama alone, they asked, be expected to forego a completely standard source of post-presidential income?

Increasingly, Obama’s defenders began to insist that the reason he was being treated differently in this case was the same reason that he had been treated differently so many times before: his race.

At The Root magazine, Michael Harriot wrote: “Maybe asking a man to work for eight years cleaning up the sloppy mess some white guy made, and then telling him he shouldn’t ask for money, just feels racist.”

Daily Show host Trevor Noah, meanwhile, expressed exasperation that “the first black president must also be the first president to not take money after office”?

As the first black president, Obama was the subject of an unprecedented smear campaign, most notably the “birther” conspiracy theory alleging that he was not a citizen of the United States. He was treated with a level of personal disrespect that previously would have been unthinkable, such as in 2009 when Joe Wilson, a Republican representative from South Carolina, screamed “You lie!” as Obama addressed a joint session of Congress. His political opponents denied him a crucial part of his legacy when they flatly refused to consider his final Supreme Court nominee, and they ritually repealed his signature healthcare law dozens of times after reclaiming the House. As they have now failed to formulate a plausible alternative, it becomes increasingly clear that their real complaint about Obamacare is those first three syllables. All of this adds up to an attempt to write the first black president out of the historical record.

Racism was arguably the single greatest obstacle Obama faced as president. Invoking it in this case, however, is cynical and dangerous.

Obama is not the first politician to be criticised for making paid speeches to Wall Street; there is no evidence he is being treated any differently here because of his race. Using accusations of racism to shame and silence good-faith critics instrumentalises one of the most pervasive forms of oppression in contemporary America, turning it into a cheap rhetorical tool. Conservatives are eager to dismiss any discussion of race as playing the “race card”, and this kind of shortsighted defence plays right into their hands.

Even worse, it normalises a practice that undermines confidence in the American political system. This is far from a pure hypothetical. Those claiming that Obama is being held to an unfair standard seem to forget how much Hillary Clinton’s choice to cash in on the Wall Street lecture circuit damaged voters’ trust in her ability to crack down on banks.

It wasn’t just Hillary: former President Bill Clinton was also scrutinised during his wife’s campaign for his own paid speeches to big banks. A few years before that, voters were similarly ill at ease with Republican candidate Mitt Romney’s significant ties to the finance world. As far back as Gerald Ford, presidents working the Wall Street speaking circuit after their terms have ended have gotten, in Ford’s words, “guff”. The idea that taking money from powerful bankers so soon after leaving office creates the appearance of impropriety is not being invented on the spot to use against Obama.

Nor is it arbitrary to apply that standard to Obama, in particular: there is a widespread perception that he was too easy on the bankers who contributed to the global financial crisis, virtually none of whom were prosecuted for their contributions. In that context, an exorbitant fee for a single speech could almost appear to be deferred compensation. The implication that such concerns are a distraction from racial issues is reductive and misleading.

The subprime mortgage bubble disproportionately victimised black communities. Black borrowers who qualified for prime mortgages were frequently given more expensive subprime loans in an attempt to raise profits. When the bubble burst, it destroyed the hard-won housing wealth of many black families and blighted already struggling communities. That Obama was so solicitous of financial markets but declined to spend all the money that Congress had appropriated for helping individual mortgage borrowers is particularly unforgivable from this perspective.

There is, admittedly, a kernel of truth in the idea that racism contributes to our notion that Obama should be held to a higher ethical standard than other presidents.

He was acutely aware of the symbolic burden of serving as our first black president. He responded to that pressure by running arguably the least corrupt, most scandal-free administration in American history. Critics of his speaking fees are not holding him to an artificial standard; they are holding him to the standard that he set, to the great benefit of our public life.

That may be “unfair” in so far as people are inclined to give a pass to other presidents for much worse behaviour, but is it really “unfair” to expect a public servant to avoid even the appearance of corruption? Should that not be the baseline expectation? That such questionable behaviour is “normal” should not be a pass to engage in it, as though it is one of the perks of office.

Most absurd of all are the claims that people are more outraged at Obama’s speech than Trump’s open corruption. This is clearly not the case: Trump is one of the most reviled presidents in American history, owing in no small part to the perception that he is cashing in on his office.

His inauguration sparked the single-greatest worldwide protest in human history, reaching even as far as Antarctica. And I would argue that Americans are outraged by Trump’s constant violations of ethical norms partly because his predecessor showed that such behaviour does not need to be normal.

In the age of Trump, we need Obama’s model of ethical probity now more than ever. Liberals should not be so willing to throw that away, especially not at the cost of cheapening his continual and very real struggle against racism.

• Adam Kotsko, assistant professor of humanities at Shimer College, is the author of The Prince of This World