Go to options, Put the game on Novice, Then beat the hell out of him.

That's the most pathetic way of beating him that I've ever heard. Seriously.
I got a Flawless Victory on him with the difficulty on the standard setting.
All you have to do is start the fight as Dairou, pull out your sword, and begin repeatedly cheesing him with the SQUARE, SQUARE, FORWARD + X combo over and over again. He almost never blocks the final hit, even if he doesn't let the first two land.
Knock him down with it, rush in. As he pops up, SIDESTEP, since he usually comes up swinging. Once you've dodged him, do the combo again, rush in, wash/rinse/repeat.
Worked like a charm for me.