With multiple events simultaneously happening in the Yamanashi Prefecture, Kazuichi and his group decided to leave for now and regroup for what was about to come later on.
Right now, although they were relatively aware of what was going on, (specifically the bad blood between the three Yamanashi Strongholds) Kazuichi knew that they couldn't just go in without a plan.
Not only would that be incredibly risky, but it might also endanger the newly formed stronghold under his name.
He may not have ties with the Kazuichi Stronghold right now, but it was named after him. Of course, it would fall under preying eyes if he were to take action without thinking twice.
As for Hinokami, he knew that Kazuichi made the right decision. It was totally the opposite of how he acted and it struck a chord in his heart. Not only was Kazuichi pretty self-composed and didn't make any rash decisions, but he also was resolute about his decision.