Ouch! These guys hurt. They are this area's version of the Black Skeletons or Red Eye Knights. If you go head to head with one, expect a club to the face followed by imminent death. They move fairly fast as well. If you can fight them in an open area you can dodge their swings. Otherwise, the best bet is to use ranged attacks, and take full advantage of the terrain available. These guys can't climb, so backtrack to a place where you have to pull yourself up to get away from them. Barring that, try a Thief's Ring and run away when they approach.
To clear the island with four of them (!) in the poison swamp of 5-2, drop the ramp that leads back to the beginning of the area, then snipe them from this area of raised walkway. With the Thief's Ring equipped they'll walk back to the island after you stop shooting for a bit. Since you're safe on the shortcut walkway, you might try buffing your attacks with a Clever Rat's Ring to save time (Magic Sword "Makoto" helps since it carries a curse that drops 0.5% of your maximum hp/sec).
Whatever you do, don't fight them while standing in the swamp. You'll probably get poisoned and they can move quickly through it whereas you can not. Always fight from dry land! Prepare to die if you don't because you cannot and I repeat cannot dodge at all in the swamp. At least, not while you're locked on to anyone. The key thing is to go out of lock-on mode! Giant Depraved Ones attack slowly enough that as they wind up for a swing, you can run out of their range as long as you're not locked on to them, and thus avoid death by clubbing.
Obviously you'll want to be running towards dry land, preferably somewhere with plenty of space to work with. In the meantime, keep an eye on how close the Giant Depraved One is to you, and make sure that you've got enough stamina for a quick dash (not a roll) away, as the Giant Depraved One comes up behind you and stops to swing. Even at a full dash, the Giant Depraved One is likely to catch up to you, and if that moment is when you run out of stamina, you get a club to the head.
Eventually, the Giant Depraved One may give up its chase as it gets too far away from its origin point or you get far enough away from it that it doesn't see you any longer, or you'll get to dry land where you can take up the fight.
Do not try to melee a Giant Depraved One in the swamp unless you're 100 percent certain that your attack is going to kill it. If you get a comfortable distance away from it, you can shoot arrows or magic. However, considering that you're probably at the very least poisoned, it's usually best to use that little respite to pop a quick antidote and/or herb instead, and keep on moving.
Lock-on and dodge to the left when he attacks. Locking-on and strafing clockwise works if you are really near each other because, more often than not, his arm (during his swing attack) won't do any damage.
Lava Bow strategy
They're weak to fire and dangerous when close, so ranged fire drops them pretty easily. Note that they tend to move very quickly once they spot you, and if they get to you when you've got a bow out, you'll usually be in trouble. Use a Thief's Ring or Cloak to maximize your shooting time. Since no rolling is required, this works in the swamp as well.