It would obviously stop growing at some point.There is its own weight that would have a bearing on how high the wave could get. There is the speed of the wind, the radius of the planet ,the depth of the surface liquid,its viscosity,the density of the atmosphere ....
If the wave circled the planet (as it would) it would run into other waves and their heights would either add or subtract.(think that may be called constructive versus destructive interference)
The discussion you alluded to on the physics subforum (the one started by Yonathon Yeager?)seems a nice one . Hopefully we will have a few more if the forum maintains its rehabilitation from its latterday doldrums.