Which Spirits are you struggling with? The different types have very varied abilities.
Wraiths, Shades, and Spectres are pretty weak so shouldn't be much trouble.
Banshees, and Swart Banshees can be troublesome. They have magic attacks so try and close the distance between you without getting hit, or manoeuvre yourself to get the first shot if you're using ranged. Walls are a great tools for this.
Ghosts main strength is their damage type. If one of them is giving you more trouble than the others then concentrate on getting a resistance to that damage type. Resistances are common in the game so that shouldn't be to much trouble to engineer.
Will-o'-wisps are very hard to beat. Run for the exit.
Hack, Slash, Loot, Chat