Materials with reduced dimensionalities, such as 2D and quasi-1D systems, are associated with a variety of instabilities accompanied by a wealth of phase transitions. This thesis treats four systems, each consisting of a metal layer evaporated on a silicon substrate, which reversely undergoes a metal-insulator transition depending on temperature. It is often caused by the interplay between electronic correlation effects and spin-orbit coupling in these systems. Besides, spatial confinement is able to tune both SOC strength and the phase transitiontemperature or it even leads to a suppression of the insulating phase.For the (4 × 1)-In reconstruction on Si(111), it is found that the critical temperature of its Peierls-like phase transition is significantly affected by confinement due to surface steps. Transport experiments point towards destabilization of the low-temperature (8 × 2) phase in the vicinity of step edges, which lowers the phase transition temperature upon heating. During cooling the (8 × 2) phase nucleates inside the (4 × 1) domains, thus the critical temperature decreases only for a sufficiently high density of steps, i.e., with merely three (4 × 1) chains per terrace.The sqrt(3)-Sn reconstruction on Si(111) is a prototype system for a two-dimensional Mott phase. By means of spin-resolved photoemission experiments, the spin-structure of the electronically correlated insulating surface state was explored in detail. The analysis of the spin-integrated bands, as well as the spin-texture of the surface state along different crystallographic directions, provide clear evidence for the formation of collinear antiferromagnetic (2sqrt(3)×sqrt(3)) domains, while the Sn reconstruction structurally reveals a (sqrt(3)×sqrt(3)) symmetry.The Rashba-splitting of the highest occupied Mott state was found to be Delta k = 0.05 Å^−1, i.e., the alpha-Sn phase should rather be termed a weakly spin-orbit coupled Mott system.In order to investigate finite-size effects such an electronically correlated system, a detailed feasibility study was done on the growth of Sn nanowires on Si(557). Depending on the Sn submonolayer coverage, various Sn-nanowires were identified. For Sn-coverages above 0.5 ML, (sqrt(3)×sqrt(3)) and (2sqrt(3)×2sqrt(3)) reconstructions were found. In particular, these phases cover extended (111)-areas, thus leading to an inhomogeneous refacetting of the Si(557) surface. The (223) facets between the (111) terraces reveal structures, which resemble a ×2reconstruction along the edges. The initial step structure of the Si(557) surface is maintained for Sn-coverages below 0.5 ML, showing the alpha-Sn phase on 3 nm wide (111)-terraces. In contrast to the 2D Mott state of alpha-Sn/Si(111), this confinement quenches the correlated electronic phase yielding metallic surface states at 40 K, as seen by photoemission.Furthermore, the low-temperature spin-orbit density phase of Pb on Si(557) was investigated, where the surface orientation is well-known to change to local (223) facets separated by (111) terraces yielding in a nesting condition of the Fermi wavevector in the direction across the nanowires. Structural models are proposed based on high-resolution scanning tunneling microscopy images. Concerning the nanowires, spectroscopic data allowed the determination of the insulating gap size induced by the formation of a spin-orbit density wave below 78 K. Evaporation of surplus Pb atoms up to 0.2 monolayers adds an extra row of Pb atoms on the mini-(111) terraces in addition to the filling up of the step edge adsorption sites. These atoms induce a metallic state located at the step edge at low temperature. The insulating phase completely collapses as a second Pb layer is added on the nanowires.