A cladding system that is used to form a building envelope is disclosed. The system includes pre-fabricated modular cladding panels, each including a composite layer mounted onto metal or alloy sub-frame. The composite layer provides an outer insulation to the system, and a generally flat outer surface to which a veneer may be mounted. The composite layer further acts as an air and vapour barrier for the building envelope. The sub-frames may connect to one another by a tongue and groove interconnect.