Was browsing through the old maps of Portobello today in the library - demmed interesting, old bean!
Anyway, it appears from one of them that a greenhouse/glasshouse for Mount Charles was there
after your house was built .... maybe this is a clue?

Don't know date, but looked like OS map and was issued before Ramsay Place was built - in fact, Ramsay Lodge was still there then.