It's illegal in many states because of the U.S.'s past, and currently a minority think it should be illegal, and it's rarely if ever charged in the states that have it on the books. In fact, in trying to look this up, it's so limited that I'm having trouble finding cases for many of the states that have laws, and keep getting articles that ran in various states 6 months ago about a New York case that was only the 13th charge of adultery in the state's history. In Michigan, the last conviction was in 1971. Maryland, supposedly hasn't prosecuted it in over half a century. Frankly, I've only found two cases in the last 5 years of states charging people for adultery.