The parallel channels are caused by near-flat areas of the terrain, which can result from too much precipiton erosion or from fill basins without adding enough noise back on. The incise flow operator just gives you more of what your surface already has.

If you want to get rid of those channels:
1) do a bit of precipiton erosion to fill in the existing channels
2) fill basins to get rid of hollows (that one spot in the second circled area where a river just stops is caused by a hollow)
3) add some noise (about 10% of the overall height of the terrain)
4) fill basins again to connect up the river system
5) do incise flow to your desired result.

If you want to ensure that the terrain exactly matches those coastlines, select everything above sea level before doing step 1.

If you think that there's too much noise after you've finished the above processing, do a Filter>>Morphological>>Erode to get rid of the spiky bits.