A minor clarification to what ppr wrote...
...would be that "off-weapon" includes not only ED from your other equipment pieces, but also ED from str/dex and skills/auras.
Example:
Lets say you have a sword with a base average damage of 34 (A = 34)
with +200% ED on it (B = 1+200/100 = 3)
and are using war travellers, with its 15-25 damage (C = 20).
You are a paladin running fanaticism (+373%) attacking with zeal (+336%).
You have 100 strength and are wearing fortitude (+300%)
for a total of +1109% off-weapon ED (D = 12.09).
Your average damage would be
( (34*3) + 20 ) * 12.09 = 1475.
If you add a 40% ED jewel to your hat, that makes your off-weapon ED +1149% (D = 12.49)
for an average of
( (34*3) + 20 ) * 12.49 = 1524
which is only a 3% improvement.