Yeah,it is a common problem in the project execution. Originally, I want the supervisor to be the representative of the buyers.Their responsibilities are to supervise and control the deeds of construction company. I think you need to tell thim the ponderance of his duties.And at the same time, you should check the quality of the construction at any moment.If anythiny wrong happened,you should collect some materials to prove that it was his defult.