Regular inspection and maintenance of infrastructure facilities are crucial to ensure their functionality and safety for users. However, current inspection methods are labor-intensive and can vary depending on the inspector. To improve this process, modern sensor systems and machine learning algorithms can be deployed to detect defects based on rapidly acquired data, resulting in lower downtime. A quality-controlled processing chain allows to provide hence informed uncertainty assessments to inspection operators. In this study, we present several Deeplab V3+ models optimized to predict corroded segments of the quay wall at JadeWeserPort, Germany, which is a dataset from the 3D HydroMapper research project. Our models achieve generally high accuracy in detecting this damage type. Therefore, we examine the use of a Region Growing-based weakly supervised approach to efficiently extend our model to other common types in the future. This approach achieves about 90 % of the results compared to corresponding fully supervised networks, of which a ResNet-50 variant peaks at 55.6 % Intersection-over-Union regarding the test set's corrosion class.