In this thesis, novel analytical and computational approaches are proposed for addressing several topics in the field of random vibration. The first topic pertains to the stochastic response determination of systems with singular parameter matrices. Such systems appear, indicatively, when a redundant coordinate modeling scheme is adopted. This is often associated with computational cost-efficient solution frameworks and modeling flexibility for treating complex systems.Further, structures are subject to environmental excitations, such as ground motions, that typically exhibit non-stationary characteristics. In this regard, aiming at a joint time-frequency analysis of the system response a recently developed generalized harmonic wavelet (GHW)-based solution framework is employed in conjunction with tools originated form the generalized matrix inverse theory. This leads to a generalization of earlier excitation-response relationships of random vibration theory to account for systems with singular matrices. Harmonic wavelet-based statistical linearization techniques are also extended to nonlinear multi-degree-of-freedom (MDOF) systems with singular matrices.The accuracy of the herein proposed framework is further improved by circumventing previous “local stationarity” assumptions about the response. Furthermore, the applicability of the method is extended beyond redundant coordinate modeling applications. This is achieved by a formulation which accounts for generally constrained equations of motion pertaining to diverse engineering applications. These include, indicatively, energy harvesters with coupled electromechanical equations and oscillators subject to non-white excitations modeled via auxiliary filter equations.The second topic relates to the probabilistic modeling of excitation processes in the presence of missing data. In this regard, a compressive sampling methodology is developed for incomplete wind time-histories reconstruction and extrapolation in a single spatial dimension, as well as for related stochastic field statistics estimation. An alternative methodology based on low rank matrices and nuclear norm minimization is also developed for wind field extrapolation in two spatial dimensions. The proposed framework can be employed for monitoring of wind turbine systems utilizing information from a few measured locations as well as in the context of performance-based design optimization of structural systems.Lastly, the problem of with data-driven sparse identification methods of nonlinear dynamics is considered. In particular, utilizing measured responses a Bayesian compressive sampling technique is developed for determining the governing equations of stochastically excited structural systems exhibiting diverse nonlinear behaviors and also endowed with fractional derivative elements. Compared to alternative state-of-the-art schemes that yield deterministic estimates for the identified model, the herein developed methodology exhibits additional sparsity promoting features and is capable of quantifying the uncertainty associated with the model estimates. This provides a quantifiable degree of confidence when employing the proposed framework as a predictive tool.