Section 16. A town shall, for each refusal or neglect of its officers to comply with the requirements of the preceding section, forfeit not more than five hundred dollars. The director may appoint one or more inspectors for such town, and may remove an inspector who refuses or neglects to be sworn or who, in the opinion of the director, does not properly perform the duties of his office and may appoint another inspector for the residue of his term.