After his thirty year 'seclusion,' however, things had changed a little in Di Tou's mind. Although the Void Qi was still scary, having been in the state of Void himself, Di Tou was far less… scared of it.
Humans, more then anything else, are scared of the unknown. The Void, Chaos, whatever you wanted to call it, was, by principle, something unknown and unknowable. It was like that question back at the Selfless Pavilion where the question asker was curious about what true death was like. Even mighty cultivators who could live thousands of years and had ways to come back to life from death were curious and fearful of actual death.
For Di Tou, the Void Qi should have remained an enigma for his entire life. Even if he got better and better at using the 'Void state' while sweeping, or having an empty mind, the Void Qi itself, and the Void behind it would remain scary and unknown.