Hi,even though the builder in question,might be a friend of mine...i would suggest jacking,with immediate effect.
The other issue,you may want to consider,is the "wild-fire" syndrome....where fellow builders,like to chat,concerning the subbies that they've straightened out...
If you get known,for being pliable - you're done
Ring him up,and tell him you cannot continue,as just like him...you are short-handed. Then,99/100 folk will reply "...What do you mean? Short handed? ...I'm not short handed!...."
Then,without resorting to rudeness,tell him your arms cannot reach your tool-box,in the same way,his arms,cannot reach his pocket....then bid him good-day
I genuinely look forward,to one of my R-C builder mates,telling me this story