Yumi archers are effective against Desert Raiders because Desert Raiders lack pierce armor.

Castle Age Yumi archers with the veteran upgrade can one-shot Desert Raiders.

Samurai barely do any damage to Desert Raiders because Desert Raiders have 5 melee armor; Samurai are just a meat shield.

The Yari Cavalry with bonus damage against cavalry does affect Desert Raiders as well naturally. So it just made it so difficult for you to actually get maximum value from the dervishes due to you having to deal with even better than regular horsemen. Combine that with potentially the movement speed once a Daimyo Estate is up and suddenly you're feeling very scared to leave the confines of your base.

It's such an important thing for Sengoku to focus on to ensure that your yatai come out instead of you just going out and going for fishing, because the yatai get to tick for longer without getting contested, getting that bonus on your matsuri so much quicker.