@Monkey1984,
you should have the first isolator between the PV array and the inverter,on the DC side.All you have to worry about is the maximum amps the array is going to pump in the inverter,so you multiply that by 1.5 and this will give you the size of the isolator.
For the other 32A one, near the fuse board,you are right,its too big,as its on the AC and having a 16A MCB determines the size of the isolator.