After the cutscene, run through the large room into the wide hallway on the left (south).
This leads to the raised gate with secret #2.
Shoot an explosive arrow in the general direction of the angled block with the chains in order to open the gate. (There are more details about this in the section on obtaining the secret. Here you want to use explosives, since you won't have much time before the horseman catches up to you.)
Grab the edge of the opening with the secret and traverse into the left corner, near the angled block below the chains.
You may need to pull up and then dangle like this a few times, but eventually the horseman should get stuck in the corner where the angled block meets the wall.
Once he's stuck, pull up into the opening and turn around. Target the blue gem with the crossbow and laser sight, and shoot it with normal arrows (or the pistol with the laser sight bug).
After a bit of this, the horseman will fall off his horse. You can then hop down and fight him as you did the others