I can't speak to lightwave 11.x, because I'm still running 10.1, or I was. I updated my OS (finally) to High Sierra (10.13.6) on my Mac Pro mid 2010, and so far Modeler will open - most of the time. I can save (maybe) or perform one modeler function before the app crashes without so much as the superfluous unexpected error message. Oddly enough, LW9 seems unaffected.
I've tried running the combo installer but get an error that the developer needs to update the software, which Newtek has obviously done with some success. I'll have to see if there's a strictly 64 bit version installer on disk somewhere.
I'm sorry that this reply is late, and may not help with your issue Commander_Chaos, except to maybe bump up the thread a bit. Did you ever find an answer?
So while exploring earlier builds of LW I had on another drive, I found that 10.1 on that drive worked fine under High Sierra. The one I had installed on the main system drive appears to have been borked by the update from El Capitan to High Sierra. Crisis averted.