I can see the problem with bringing over a full army. I'm more concerned about a few good climbers coming over and opening the gates from inside. I guess a loosely manned fortification along the crest of one of the cliffs would discourage any scouts coming from that side, along with some spikes dug into the rock.
As for wells and water supply, the thought was that an invading force has easy access to the river - the water runs from the north - but that wells are within the walls, or in the case of the keep, in the basement. That makes it very hard for an invasion to poison or tamper with the wells but easier to do with a river. Also, rivers cause natural weak points in walls, so the river stays outside the earlier fortifications, but gets incorporated into the later ones.