The institute was targeted by Nazis as early as 1920.
From about the early 1920s onward, Hirschfeld became a target of the far-right in Germany, including the Nazi Party. He was physically attacked during multiple incidents, including an incident in Munich on 4 October 1920 in which he was badly injured… By 1929, frequent targeting by Nazis made it difficult for Hirschfeld to continue with his appearances in public.
If you really want to discuss technicalities:
The institute was targeted by Nazis as early as 1920.