After a few moments, Lex realized that it wasn't that he was in danger, but that the people in this area were mostly at a higher cultivation level. At the academy he learned that for humans, on average, reaching the Foundation realm in your early life was considered normal, and reaching the Golden core realm before you die fell in the realm of moderately talented. This was keeping in mind that, for a Foundation cultivator with a life span of 150 years, even up to the age of 50 was considered early in his or her life.
But these people would only be very average cultivators and carry no or limited danger to someone like Lex. That was why he walked comfortably through the town without feeling much danger. Yes, there were other races as well, but not all races carried the overwhelming advantages that the Crystal race or Kraven boasted.