In hydrodynamic descriptions of lipid bilayers, the membrane is often approximated as being impermeable to the surrounding, solute-containing fluid. However, biological and in vitro lipid membranes are influenced by their permeability and the resultant osmotic forces -- whose effects remain poorly understood. Here, we study the dynamics of a fluctuating, planar lipid membrane that is ideally selective: fluid can pass through it, while the electrically-neutral solutes cannot. We find that the canonical membrane relaxation mode, in which internal membrane forces are balanced by fluid drag, no longer exists over all wavenumbers. Rather, this mode only exists when it is slower than solute diffusion -- corresponding to a finite range of wavenumbers. The well-known equipartition result quantifying the size of membrane undulations due to thermal perturbations is consequently limited in its validity to the aforementioned range. Moreover, this range shrinks as the membrane surface tension is increased, and above a critical tension the membrane mode vanishes. Our findings are relevant when interpreting experimental measurements of membrane fluctuations, especially in vesicles at moderate to high tensions.