Shinigami was a small assassination group which resided under the forest of Wansai, hiding away from everyone's reach. At that time, it was a small assassination group and they lived like normal villagers in the forest. Even if people find out their whereabouts, they would never doubt they were all assassins.
War, disputes among clans, civil fights, extortions, unfair justice, classification evrything made this group.
The people who weren't part of the superior part of the society, they still had some part in lower class society but those who had nowhere else to go, were all welcomed by Shinigami. The people in this group were driven by hatred and anger so it was easier to direct them to any kind of shady businesses...even if it wasn't related to them.