Page 1 of 6 123 ... LastLast
Results 1 to 15 of 86

Thread: If you use & care about your (lw) investment this is esesential reading at last!

  1. #1
    Registered User ianr's Avatar
    Join Date
    Oct 2006
    Chiltern Riviera

    If you use & care about your (lw) investment this is esesential reading at last!

    A simple question from me results in a many faceted ,insightful reply about the health of our
    cherished program LightWave now & maybe onwards, as answered on the LightWiki Channel..
    After a corrosive drought of non-Information on LightWave comes an extensively thoughtful
    & lovingly crafted essay by one of LightWave’s top flag- bearers & Emmy award winner,Mr Kelly.Kat Myers Esq.
    That which follows is with Kat’s permission with a small edit here & there,
    making for an invigorating and inspiring read I hope for all of you, Devs & Users alike:

    The Full Enchilada (part I)

    ianR: Kel, in your consideration can the NEW (LW 2018) SDK do advection?

    Kat: Kelly Myers I'm not really the one to ask about this. A new particle system for LightWave was sacrificed in this release in order to deal with the flop that was "Hydra" (the geo engine from Chronosculpt which couldn't be wedged into Layout or Modeler as initially thought it would be - you can blame that one on the management of the time, but what is done is done).

    [This assertion is incorrect. The implementation of the new Hydra geometry engine in Layout was done successfully for LightWave 2018; yes, this is exactly the engine that was discussed on the blog; yes it is responsible for a number of performance and capability improvements in LightWave 2018 and offers loads of opportunity for future enhancements. We confirmed that on the day of the December announcement of the LightWave 2018 release and we've repeated it a number of times since then.]

    An update to the particle system would be required before this would be possible of course, however, with Hurley Works LWUP I am wondering if particles from that system could be made to be aware of TurbulenceFD somehow?

    You never know. There is so much that has changed "under the hood" that we (the average LW users) don't really understand to its extent just yet. Now that this release is out and initial patches quickly following suit, it may be that attention to a new particle system will come. But come from inside the LWG or from 3rd party developers? Well, I feel that a particle system should come from inside the LWG but more importantly, the work to the SDK should be extended so that a particle system in LW can be aware of the "world" outside of itself.
    No more black box development. Keep in mind that the reason why PFX, HardFX, SoftFX, ClothFX all work together is that they all essentially came from the same developer to form a complete physics solution. But that solution cannot "talk" much, if anything, beyond that which it has been applied to and in rather limited ways. I say limited because it hasn't really advanced much, if at all in almost 20 years. But at the time LW's physics system was considered to be incredibly powerful and rather robust for a package of its kind. But ultimately, it's a plug-in, based on the SDK. But that is what LightWave is. A platform for running plug-ins. And you know what? At the end of the day, it's actually really good at this! In some peoples opinion, the SDK for LightWave as far as a developer is much concerned, is much more friendly than say another 'plug-in' platform that came up around the same - 3DS Max.

    Clear proof of this comes from the work that Steve Worley and Denis P. have done. Steve gave us tools that allowed for us to pretty much destroy any other package out there for what we could do with them in LightWave giving it capabilities that even still to this day you can clearly see in terms of their influence on the development of other products, LightWave included. Frpime & Sasquatch most notably but also the Polk and Taft collections.

    Kelly Myers Later when the LWG introduced their spline deformer I was excited... but... well, it's nice and all, it doesn't give me what we can get out of DSpline or Hoser. The point here that I am trying make is that all 3 of these solutions are essentially plug-ins for LightWave made possible by the SDK and in the case of Taft/Hoser - based on the LightWave 5.6 SDK! From 20 years ago! Right around the time I first used Taft in 1999 or so (if memory serves me right) and pretty much the height of LW's popularity in the industry in large part because of the massive assortment of 3rd Party solutions available to users that were not present in the package. No coincidence there. During the cycles of development from 5.6 to 8.0 we saw huge changes to the SDK before largely tapering off in 9.6. That's where things went off the rails really for LightWave combined with a number of other factors both internally and externally of NewTek.

    It is for this reason that while certain things should be done by the LWG in terms of feature development, it is more important that the SDK be made as robust as possible to start with when making these fundamental improvements to the architecture itself.If you only have the resources to do certain things, you that stuff and work with your 3rd party developers on it. You make it better for you and you make it better for them and everyone along the way generally stays happy. Besides, you dont' get any of this stuff anyway until someone needs to make it possible and that almost never comes a developer exclusively. All features found in a product like LightWave are the result of the need for them by users. All 3rd party developers are users. So, fight for the users! WIN!

    Priorties of course.... that's the challenge. All the time, because of limited resources. What gets priority comes down to a lot of factors but with this release I think we are seeing the future of what is to come for 3rd party developers. I'm ok with it, so long as is done right. I believe the perceived delay for LW 2018 release was in part to ensure it got stuff right.

    If you dont' you f**k your future plans or potential plans for product features and you set your third party developers up for a cluster screw as well and you need them as much as they need you, because ultimately at the end of the day, LightWave3D is a platform for running plugins. Making them talk to each other is all well and good, but if its through a two tin cans tied together with a piece of string it's not a great thing to have. Some stuff needs to be done first regardless or there is no point to even that.

    The proof comes from the particle system example It being sidelined in favour of work on the new geometry engine, which HAS to be done by the development team internally makes absolute sense. A new geometry engine is kind of needed anyway because any new particle system built is going to come up against the performance limits of that geometry system. Want further proof? Consider bullet prior to LW 2018 and the performance of it now and I'm not even sure if Bullet has been optimized specifically for the advancements in 2018. One "level" affects the next is what it comes down to. Improve this and you get benefits to that which it is built on top of. Hell, it may be with some optimization work that even the "old' physics system, PFX included could see some major gains due to the advancement in the geometry engine performance? It is after all a plug-in. And believe it or not, sometimes it's performance has nothing to do with the architecture or the SDK, but by some developer long gone who cut off the tool at the knees because at the time failure to do so would have allowed for its use in ways that would have made it unstable due to ram consumption or simply impossible for the product as a whole to deal with.

    The particle system is a perfect example of this again, it had a hard coded limit for a certain amount of particles and I hit that on Iron Sky when I had never hit it before. Even on BSG...Why? Well, we had never the ability to bake the motions of all the emitters in a scene down to single motion and then apply it back to a simple null before. That and much of the particle work I had done on BSG was still in a 32bit environment (and thus maximum memory space for 32bit apps being somewhere around 3GB leaving the rest to the OS). The plug-in was produced at a time when 128MB of ram was, considered....f***ing huge Kelly Myers James Willmott thankfully came to the rescue on that one. ( ianR: After a forum discussion here quickly noted by J.W.) Did it require architecture change? No. Because its a plug-in. He removed the limit and we got a new build. Done! Thank you James. its stuff like that which many users hit and then say 'LW sucks' & come to this conclusion that is has to get a re-write in order to be better. Well, which part? The plug-in you are dealing with or the system it runs on top of or a portion of that system the plug-in deals with? Or all of it?

    I certainly wanted a new particle system and hopefully one that was GPU enabled so I could get the benefits of the promises GPUs offer in terms of speed. A faster geometry core was needed to. Everyone needed that, but some- times you can't do both and what happened with the particle system and why is very important for people to understand. Advancements to the base level architecture and the SDK, have to take priority over some things even when you are being pushed to provide new features or fixes to features.

    Think about it. If the internal resources available to the development team are limited in anyway, you are setting yourself up to be in a better position down the road by choosing the smoothest road that is the quickest to take to get you to where you want to be, faster. Sometimes you need to pave road to do it, even if to be a passenger on the drive down it. The goal is to get to the destination and share the cost of gas. An improved and robust SDK that is made available to 3rd Party developers while the main product developers continue work on pillar areas of the product including its main architecture, and feature set for which the SDK is based; ensures that new features resulting from the effort are the best that they can be. When you don't do this, it usually results in half tools which are historically produced internally due to a lack of resources compounded by the desire to sell copies of the product in order to maintain or grow the resources you have from which to draw upon and thus make a better product and make a profit. The NT developers hit the same stuff 3rd party developers hit when making features at the end of the day. Some stuff requires changes to the architecture and thus the SDK and some stuff doesn't.The only difference is that the LWG development team can directly do something about it, where as 3rd party developers can only request certain things. And it must pain them at the LWG for the exact same reasons 3rd party developers have, yet even more so when priorities don't align with what they want to do or are being asked to do when they have it sitting right in front of them. We always assume its easy because it is sitting right in front of them. And you know, sometimes some stuff is such as in the example I gave with the particle FX hard limit

    (ianR See PART ll)
    Last edited by Chuck; 02-07-2018 at 10:02 AM. Reason: try to make this more readable

  2. #2
    Registered User ianr's Avatar
    Join Date
    Oct 2006
    Chiltern Riviera
    The Full Enchilada (part Il)

    Kelly Myers(con't) But they don't know unless there is a need to and that need comes from feature requests from the product users and from 3rd party developers looking to make products in the form of plugins for third party users. I'm positive that Jasha has made this request as a third party user when it comes to LW particles being able to respond to TurbulenceFD via advection channels.( ianR: Matt Gorner did contact Jasha on this last year I believe)

    A new particle system would certainly have addressed that. It may be possible for PFX to do it, but I could only see that with changes to how LW works with plugins and allowing them to share data between each other. That data is probably available and supplied by TFD itself, possibly with PFX being made to understand it without too much trouble, but the transport of that data doesn't exist right now. PFX certainly supplies data that PFX can read.. but that's just one tool talking tool to another for a specific purpose and doesn't really do much for any other developer unless it's documented in the SDK. I think the LWG has learned from its past on this issue finally.

    But over all, the best thing to do is make the changes to the architecture needed to make such things possible and during that time produce a tool that makes use of it in the plug-in environment in ways that are optimized from the get go to transport and handle that information. The developers internally know how to do that best, but sometimes its 3rd party developers who really know "what to do with it".

    (ianR; cite Thinking-Particles plug in other apps, does what it says on the can!)

    And that is the approach I think that's culminated in this release and I hope it continues if that is the case. It's only been out for 4 weeks and "delay" I think was warranted because its given many developers the time to adjust.

    It's pretty clear, do that and include fully, the changes made in the SDK so 3rd party has the same access to what internal developers have and you solve a lot of things than just one problem with a 'feature' that typically comes out half baked.. thus the half-tool. This is a much better approach. Not only that but you get a solution that rocks if done right, and a 3rd party developer can do so as well if you miss the mark on a new feature. But really if the internal developers spent less time on features on some stuff, 3rd party developers will step in and that's the way to go. If not to simply pick up the slack leaving you to focus on other things that need to get dealt with next to make a better product everyone can enjoy regardless of their need for a certain feature natively available or not It's really simple. Doing it right means staying out of feature creep land.

    A new particle system would have likely particle advection data provided this, but if we are not going to get a full replacement for it and the new geometry system at least improves performance on some level when working with particles as we often do, then perhaps a bridge can be done now. It certainly doesn't require a change to the architecture and with the new geometry engine out the door I could see it at least being seriously looked at again. I certainly wouldn't consider it to be a full-fledged feature, but simply a modification of a native tool to provide and understand another data channel so as to keep the product competitive with other products on the market, while working with a third party solution to a feature for which it does not have natively (computational fluid dynamics) including one of its primary competitors (C4D). Why? Because Jascha is working on a Modo port and you want to keep Modo at bay in some way and this is an easy one to do. Easier than losing users to another product for some-thing like this, that's for sure. (IanR: bookmark this thought LW3DG please) And it will have immediate benefits to Turbulence users and spill over I'm sure to other 3rd party plug-ins like LWUP and DeepRisingFX. Why? because of the new rendering and volumetrics system. That's why. So now is the time even if it doesn't come in the form of a completely new particle system.

    Make no mistake we got a lot of features in this new version but almost all of them are fundamental "pillars" of the product advancements and that's important. This round of improvements meant many architectural changes along the way and its been done while maintaining the SDK every step. It's important to understand how big of a deal that is. In the past, the product has advanced in big ways, but the SDK wasn't well maintained. And yes still 3rd party developers were able to do so much amazing stuff with it for what was documented. This latest round of changes I think is going to do some well needed good for everyone. Based on some of the developments coming out now however along with some that have been in progress for the last year (at least) - it is my belief and I do fully believe this; that LightWave3D and its users about to encounter a massive explosion in tools development coming internally as well as externally. Oliver Hotz (OD)tools are a clear indication of this. Mambo Banda II (Deep Rising FX) has been making solid strides in advancing his tools.

    On the subject of physics, I'm in direct contact with another developer as well (who is not as well known to the LW community) who has been working on a Flex based solution for some time now. So far, to my knowledge, only myself and Mike Green have been provided early versions of this toolset and it seemed to have stalled for a while, but I was contacted immediately after the release announcement date came in mid-December by that developer asking if I could help reconnect them to the LWG.Who would also benefit from having access to new particle data channels and being able to read and write them and it accessible to Turbulence via the same method potentially implemented for PFX to get it all rolling.

    Consider now that so much is heading towards the GPU. Well that has it's own set of problems and I think the LWG can avoid those pitfalls here by using the GPU where it should be used and the CPU for what it should be, leveraging both at what they are good at, at the same time. ( ianR. The GPUs ability to enable the use of System Ram when VRam [on your GFX card] is maxed out & not choke, as in RedShift, is essential to the LW ethos).The changes to the architecture needed to make that possible are probably similar to those needed in order make it so that plug-ins to talk to each other 'better' than they do currently. Maybe since this release is out, in the course of doing some research and development while also benefiting TFD users working in LightWave and the TurbulenceFD developer (Jascha @ Jawset) - now would be a good time to investigate it using this as a "test". Everyone wins on that one I think.

    As a side note, the boost in 3rd party activity and renewed interest in the product is very interesting as well as the timing. You can read into that part whichever way you want, but the point is, that within the last 16 months, let's say... there has been a flurry of development activity that much of the community has not been aware of beyond those of Oliver, Mambo, Hurley, and the efforts of Ryan Roye and Chilton Webb over at and there are others including Craig Moins (Rebel Hill) and 3rd Powers Japan (ianR:soon to be Lino Grandi) so more will come and more will return and many are hard at work.

    Support the release by buying your copy now and then support your 3rd party developers.

    We are all in this together and we can't do it right by standing apart.

    This is a massive opportunity for LightWave3D and its users.

    ( IanR: Drop a comment & get this thread rolling, or just flag it post you have read it! ).
    Last edited by Chuck; 02-06-2018 at 11:29 AM. Reason: try to make this more readable

  3. #3
    Registered User Schwyhart's Avatar
    Join Date
    Mar 2013
    Tulsa OK
    Last edited by Schwyhart; 02-04-2018 at 08:06 AM.
    Mac Pro
    3.5GHz 6-core with 12MB of L3 cache
    32GB (4x8GB) of 1866MHz DDR3 ECC
    1TB PCIe-based SSD
    Dual AMD FirePro D700 GPUs with 6GB of GDDR5 VRAM each

  4. #4
    RETROGRADER prometheus's Avatar
    Join Date
    Aug 2003
    sweden stockholm
    Huh..I have copied this and will make a paragraph break for every four lines.
    May get back after reading it.

  5. #5
    Registered User
    Join Date
    Sep 2015
    Pretty interesting. I hope that he is right about what the sdk changes will bring in the next few months. The last few weeks have seen lots of activity, I hope it holds out. We haven’t heard anything new from 3rd powers for example. I for one would be curious if they have been cooking up something with the new sdk.

  6. #6

    way too much to read (?) probably

    re-written >
    ...the boost in 3rd party activity and renewed interest in the product is very interesting
    LW vidz   DPont donate   LightWiki   RHiggit   IKBooster   My vidz

  7. #7
    Man of many cells. shrox's Avatar
    Join Date
    Aug 2006
    Eureka, CA
    I'm scared.
    Heavy Metal Landing

    I build the best spaceships, the biggest spaceships, they're great, you'll love them.

  8. #8
    Registered User
    Join Date
    Jan 2016
    Hmm.. paragraphs.. but I read through it all.

    I've said before that I would look into developing for LW.. and I will. I have a physics solution that like UP is flex. I posted a few videos about it earlier.
    But I dropped dev.. when I kept hearing a deafening sound of silence from LWG. Now it's different. But I will probably begin from scratch again.. as I forgot most of it by now.
    I hope that there are sufficient access to the internals via C/C++ now because if I have to write a python layer to reach everything.. I'll probably go frantic

  9. #9
    Holy wall of text, Batman!

    Like the others will copy it all out and reformat it to easier readable chunks.

    Thanks for sharing nonetheless.
    Humans are pretty much monkeys with car keys.

  10. #10
    Quote Originally Posted by MichaelT View Post
    Hmm.. paragraphs.. but I read through it all.

    I've said before that I would look into developing for LW.. and I will. I have a physics solution that like UP is flex. I posted a few videos about it earlier.
    But I dropped dev.. when I kept hearing a deafening sound of silence from LWG. Now it's different. But I will probably begin from scratch again.. as I forgot most of it by now.
    I hope that there are sufficient access to the internals via C/C++ now because if I have to write a python layer to reach everything.. I'll probably go frantic
    Please do.

    All of this sounds the same as it has. The issue is as it always has been: tools made by artists for artists. If the artists are gone ya gotta find artists that do it like that. That being programming, artistry and the need of them.

    Devvers make the app. Companies make ecosystem opportunities. Starbucks saw that clearly. I digress.

    Long live the 3rd party devvers. Wallet in hand...

  11. #11
    Registered User KurtF's Avatar
    Join Date
    Jul 2008
    Northern California
    Interesting read. And yes, customers do like hearing about the challenges and possibilities while developing the product. Why developers remain silent for years - like the ElectricImage and Lightwave teams, instead of keeping users posted on progress, I have no idea. I do know I can post stuff about ElectricImage here and it stays. When I post over there, it gets deleted or edited.

  12. #12
    One of the biggest impacts to Python scripting in Lightwave regards the load/saveserver commands implemented in LW 2018.

    These commands allow the script writer to access almost any bit of information from Lightwave's various tools and plugins, meaning that the plugins that people write are no longer inaccessible via scripting. In other words, I can now collect data from ANY plugin in Lightwave that has a persistent presence and have my script make changes and decisions based off of that. This includes 3rd powers, Syflex, motion mixer, and yes, even IKBooster. Even plugins made decades ago can benefit from this function; it works the same across the board.

    So... it'll take some time for people to realize the potential this brings to the table, but it really opens some doors for plugin developers.
    Professional-level 3d training: Ryan's Lightwave Learning
    Plugin Developer: RR Tools for Lightwave

  13. #13
    Registered User
    Join Date
    Mar 2016
    Oxford, UK
    Ryan, I am reminded of that fine still used on your Python course intro, "You're here 2 learn !? LOL !!", but beyond your course was it a very deep curve to cross the bridge and write your own LW plugins ?

  14. #14
    Registered User
    Join Date
    Jan 2005
    Colorado Springs
    I read the whole thing. That's great that LWG spent the time to modify the underlying LW architecture aimed at connectivity and "the pipeline" being exposed to plug-in developers as much as possible. One of the things Microsoft (among others) figured out early: make the development tools readily available and the users start building stuff in their basements.

    So next: Bought my 2018 license, downloaded kit & content, and hopefully will install it soon and start playing when I find / make time. Downloaded the new SDK documentation, will look into porting some of my old (even pre-V9) plugins that might be useful, see if some of the limitations I ran into are removed, and hopefully write some new ones and look into GPU-based development. Try to save my pennies to support Denis (DPKit), LWCAD, Advanced Placement, and other 3rd party plug-ins I've bought over the years, if possible.

    And, as someone who *likes* LW since 1996, crossing my fingers!


    P.S. everyone probably knows this, but a trick for reading the "Wall o' Text" is to reduce the width of the browser to minimum, causing it to wrap the text into a much narrower format. Easier to read if it's more like a newspaper column, or at least not the full screen width.
    Last edited by MonroePoteet; 02-04-2018 at 04:12 PM. Reason: Newspaper column trick

  15. #15 rustythe1's Avatar
    Join Date
    Feb 2006
    the thing I'm a bit confused about is his very first comment? so is the new geo engine not the one that was talked about in Robs original blog, i.e. it is a more advanced variation of the hydra engine, or did something happen and that was the point it was scrapped, or is it referring to Core being the Hydra flop, or do we now have a totally new engine altogether to what the original blog said?
    Intel i9 7980xe, Asus Rampage Vi extreme, 2x NVIDIA GTX1070ti, 64GB DDR4 3200 corsair vengeance,

Page 1 of 6 123 ... LastLast


Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts