[T]he Constitution explicitly assumes that felons may be barred from voting. The 14th Amendment — which, like the 15th, was passed during Reconstruction to ensure equal treatment of African Americans — acknowledges that states can disenfranchise people for "participation in rebellion, or other crime." So an interpretation of the Voting Rights Act to bar felon disenfranchisement would not only be inconsistent with the intent of that statute, it would exceed Congress' constitutional authority.Or look at it this way: When someone is kept from voting because he has been convicted of a felony, this does not "result in a denial or abridgement of the right … to vote on account of race or color" (to quote the law); it results in the denial of the right to vote because that person has chosen to commit a serious crime against a fellow citizen.
And they conclude:
Today's laws may have a disproportionate impact on some racial groups, because at any point in time there are always going be some groups that commit more crimes than others, but that doesn't make the laws racist — just as the fact that more crimes are committed by men doesn't make criminal laws sexist.
And the people whose voting rights will be diluted the most if felons are allowed to vote are the law-abiding people in high-crime areas, who are themselves disproportionately black and Latino.
