No, your species does not affect how much damage you can deal. Staff of Righteous Fury is a variable icon weapon, though (7.5-12 icons), so maybe you're just not getting as many full hits lately.
As for damage, some species have resistances to certain icon types (for example, pteri has a 40% air resistance, and
check out the kau!) and some have weaknesses (jubjub has a 20% weakness to fire), but bori and kacheek both have neither.
Oh, and also, are you fighting the same enemy over and over? As their strength boosts go up, they will do more damage to you with their same weapons.