Some are. Notable threats, that is. People like Mace Windu or Shaak Ti, high end Jedi Masters who could possibly stand against Sidious in a duel. Others are no more threatening that many non Force users. Which doesn't mean they are not threats of course, since NFUs are themselves often very threatening. Something that Sidious the elitist disregarded at his peril. In fact, if Vader hadn't killed him when he did, Lando Calrissian, Nien Nunb and Wedge Antilles, all of whom are NFUs, would have killed him about five minutes later.