Originally, the upper storeys of the "Palas" (great hall) consisted of a timber frame construction. In 1504, during the war of the Landshut (aka Bavarian) succession the Nuremberg besiegers burned the "Palas" down to the base walls. The free city of Nuremberg took over the sovereignty over the ruins. In the following time, ownership had been given to different patrician families.

In 1530, a first reconstruction had been carried out by the Behaim family. However, around 1555 again a partial destruction occurred as part of the war of the Markgräfler succession. Eventually, in 1561 Endres Oertel could complete the reconstruction to the state of today: Rhaetian sandstone arranged in a renaissance style.