PDA

View Full Version : LW NEXT Alembic importer overhaul?



m.d.
02-22-2016, 02:41 PM
Just pulling my hair getting liquid sims from Houdini to LW...

Realflow mesh export doesnt really work (kinda does...but crashes after a few seconds)....OBJ sequence isnt an option as each frame is 10's of millions of poly's (and no MB)....and Alembic leaves behind points and poly's...

The only option right now is export particles using the Realflow houdini exporter and let realflow renderkit mesh it on the fly....but Houdini has so much more advanced meshing control compared to realflow, and there is no IPR/VPR preview


What would be nice, and idiot proof...would be proper Alembic import.

I can even import the mesh into Fusion...but for some reason not lightwave.

Is this planned to be fixed for a new version? LW group always talks interchange-centric workflows, but in practice with FBX, Alembic and others....it is the least friendly...even compared to compositing apps

Tranimatronic
02-22-2016, 03:37 PM
We had /have huge issues getting alembic from houdini to maya at work.
It took me a good week to work out what is going on and honestly you dont get 'idiot proof' alembic from Houdini.

Make sure in Houdini you DONT have animated vertex connections, or animated UV's (these kill Maya stone dead, I'm guessing LW too)

m.d.
02-22-2016, 09:55 PM
just to get some interchange workflow...I did a simple fluid sim, rather than trying to send a real fluid sim over...classic ball of liquid dropping in a tank. No UV's ect...nothing to complicated

I dont have every app available at my disposal, but C4D...no issues...Fusion...no issues.

Lightwave...frame 1 works fine...then as the fluid ball drops, half of it remains static...random points and poly's freeze in time

So this is as simple a test as it can get...I am thinking topology likely changed and that killed it.

Good tips on the other points...I'll keep that in mind, but I think even the most basic stuff with animated topology won't make it over

Tranimatronic
02-22-2016, 11:13 PM
even the most basic stuff with animated topology won't make it over

exactly.

I can't remember the exact setting in Houdini that fixed it, but will ask the Houdini guys at work (they now have to do it for every export).
I think there is a setting where you pre-generate each point so the number remains constant.
If you download and install the python alembic module there is an easy check you can run on alembic files to see if they are OK for maya (and I'm guessing Lightwave)
Again, I'll dig that up tomorrow if youre interested

Ztreem
02-22-2016, 11:54 PM
Alembic in LW doesn't support changes of topology/vertex number. If that is the case( usually with liquids) you have to use obj sequence or other solutions( if any?).

m.d.
02-23-2016, 12:19 AM
exactly.

I can't remember the exact setting in Houdini that fixed it, but will ask the Houdini guys at work (they now have to do it for every export).
I think there is a setting where you pre-generate each point so the number remains constant.
If you download and install the python alembic module there is an easy check you can run on alembic files to see if they are OK for maya (and I'm guessing Lightwave)
Again, I'll dig that up tomorrow if youre interested


That makes sense, sure please ask....but at this point I'm probably going to render in Houdini, so don't make a big deal of it if it's not convenient for you.
Pre generating all the points on a liquid sim would result in massive amount of points, technically I could see it working, but practically could be a nightmare.

m.d.
02-23-2016, 12:23 AM
Alembic in LW doesn't support changes of topology/vertex number. If that is the case( usually with liquids) you have to use obj sequence or other solutions( if any?).

Ya had assumed as much...obj sequence probably wouldn't allow motion blur though. You can export Houdini liquid particles with the real flow ROP exporter, and mesh in lightwave....but that requires the render kit, which technically means a real flow license for commercial work and the meshing is much more advanced in Houdini interns of filtering etc

The title of this thread kinda says it all.....just wondering if future versions of lightwave are looking at correcting this.

Ztreem
02-23-2016, 01:11 AM
Ya had assumed as much...obj sequence probably wouldn't allow motion blur though. You can export Houdini liquid particles with the real flow ROP exporter, and mesh in lightwave....but that requires the render kit, which technically means a real flow license for commercial work and the meshing is much more advanced in Houdini interns of filtering etc

The title of this thread kinda says it all.....just wondering if future versions of lightwave are looking at correcting this.

Hopefully with the new geometry Engine this will be possible in the future...

m.d.
02-23-2016, 09:36 AM
fingers crossed :goodluck:

Tranimatronic
02-23-2016, 02:31 PM
Download & install PyAlembic.
Call:
verfiyABC("/path/to/alembic/File.abc")
It will tell you if the abc will render in maya (or current Lightwave)


import imath
import alembic

def childIterator(childnode):
if childnode.getMetaData().get("schema").count("AbcGeom_PolyMesh"):
mesh=alembic.AbcGeom.IPolyMesh(childnode, alembic.Abc.WrapExistingFlag.kWrapExisting)
meshSchema = mesh.getSchema()
unanimated=""; hasAnimatedVerts=""
animatedVerts = meshSchema.getTopologyVariance()
hasAnimatedVerts = "FAIL" if str(animatedVerts).strip() == "kHeterogenousTopology" else "OK"
uvs=meshSchema.getUVsParam()
unanimated= "OK" if uvs.isConstant() else "FAIL"
normals=meshSchema.getNormalsParam()
if "FAIL" in [unanimated, hasAnimatedVerts]:
print "THIS MESH HAS PROBLEMS"
for child in childnode.children:
childIterator(child)
return

def verifyABC(alembicFilePath):
alembicArchive = alembic.Abc.IArchive(alembicFilePath)
ABC=alembicArchive.getTop()
for child in ABC.children:
childIterator(child)

erikals
02-23-2016, 02:46 PM
...Make sure in Houdini you DON'T have animated vertex connections, or animated UV's (these kill Maya stone dead, I'm guessing LW too)

like in, slowing it down?

Tranimatronic
02-23-2016, 03:05 PM
no crashed on load.... killed stone dead

The python above will tell you if you have a problem abc file.

After talking to our Houdini lead, it appears I oversimplified the solution.
Allegedly there is a whole whack of packing veloicity values into RGB channels and reading those as velocity for mentalRay (I believe this is for motion blur only)
The geo-only answer is setting houdini vertices to exist in every frame

erikals
02-23-2016, 03:20 PM
i'm wondering if this is the problem eetu had...
http://forums.odforce.net/topic/7260-houdini-to-lightwave-pipeline/page-2#entry141299

edit:  ah, reading again, seems i misunderstood, there might not be a "geometry limit"
guess i should test this one more time... (!)

m.d.
02-23-2016, 05:12 PM
Download & install PyAlembic.
Call:
verfiyABC("/path/to/alembic/File.abc")
It will tell you if the abc will render in maya (or current Lightwave)


import imath
import alembic

def childIterator(childnode):
if childnode.getMetaData().get("schema").count("AbcGeom_PolyMesh"):
mesh=alembic.AbcGeom.IPolyMesh(childnode, alembic.Abc.WrapExistingFlag.kWrapExisting)
meshSchema = mesh.getSchema()
unanimated=""; hasAnimatedVerts=""
animatedVerts = meshSchema.getTopologyVariance()
hasAnimatedVerts = "FAIL" if str(animatedVerts).strip() == "kHeterogenousTopology" else "OK"
uvs=meshSchema.getUVsParam()
unanimated= "OK" if uvs.isConstant() else "FAIL"
normals=meshSchema.getNormalsParam()
if "FAIL" in [unanimated, hasAnimatedVerts]:
print "THIS MESH HAS PROBLEMS"
for child in childnode.children:
childIterator(child)
return

def verifyABC(alembicFilePath):
alembicArchive = alembic.Abc.IArchive(alembicFilePath)
ABC=alembicArchive.getTop()
for child in ABC.children:
childIterator(child)

Thanks for digging into it....will put this to use. :thumbsup:

m.d.
02-23-2016, 05:25 PM
i'm wondering if this is the problem eetu had...
http://forums.odforce.net/topic/7260-houdini-to-lightwave-pipeline/page-2#entry141299

edit:  ah, reading again, seems i misunderstood, there might not be a "geometry limit"
guess i should test this one more time... (!)

The only way at this time to import animated meshes with changing topology from houdini is through the realflow plugins.
You could do it through obj sequences....but that would be very old school and no MB with huge filesizes...but it would work in a pinch.

There is a realflow mesh export ROP which comes with the free plugins, but I have found some issues with the workflows......first off there is zero documentation, so through a lot of trial and error I have imported fluid meshes into LW....but these often (like every time) will crash layout. You may be able to scrub them a few times, but they will eventually crash.... usually with a line 108 saying something about unexpected format

I think the problem is I am exporting a houdini mesh which doesn't specifically conform to what the realflow plugin is expecting. I have tried all the VDB convert settings in the fluid SOP, and only polygons work (AFAIK) polygon soup, VDB mesh and others don't seem to import at all. And I have removed all the extra attributes except velocity, as realflow will only accept velocity and texture in it's mesh.

I am somewhat close to getting a working import....I am sure some others have figured it out...but in Houdini there is a lot of SOP's you could point the Realflow MeshExport ROP to....and being totally undocumented there are dozens of variables and geo types complicating the issue further.

If I get it down, I definitely will share...hopefully newtek has better alembic or VDB (for liquids) support in the near future.

m.d.
02-23-2016, 05:30 PM
no crashed on load.... killed stone dead

The python above will tell you if you have a problem abc file.

After talking to our Houdini lead, it appears I oversimplified the solution.
Allegedly there is a whole whack of packing veloicity values into RGB channels and reading those as velocity for mentalRay (I believe this is for motion blur only)
The geo-only answer is setting houdini vertices to exist in every frame

curious as to how they pre-generate the vertices.....that sounds complicated, basically would have to be done in reverse as you wouldn't know the final number until the end of the simulation.
The RGB velocity is clever....but have no clue how you would wire that so LW render respects it. Likely not possible

erikals
02-23-2016, 05:44 PM
The only way at this time to import animated meshes with changing topology from houdini is through the realflow plugins.
You could do it through obj sequences....but that would be very old school and no MB with huge filesizes...but it would work in a pinch.

Prometheus and I did get MoBlur to work with the 32bit plugin...
http://forums.newtek.com/showthread.php?143243-The-amazing-Houdini-reel-of-Spencer-Lueders-%28-fluids-fire-and-smoke%29&p=1397776&viewfull=1#post1397776

132555

m.d.
02-23-2016, 06:11 PM
Prometheus and I did get MoBlur to work with the 32bit plugin...
http://forums.newtek.com/showthread.php?143243-The-amazing-Houdini-reel-of-Spencer-Lueders-%28-fluids-fire-and-smoke%29&p=1397776&viewfull=1#post1397776

132555

is that just spheres/particles?

it is fairly easy (again with the RF plugins...which are free) to get in Houdini particles...motion blur as well....and millions if need be into LW....
It's the real mesh which is the hard part...

good experiment though....could use particles and spheres to generate low rez motion vectors to apply to an obj sequence in post.....
mind you Fusion has alembic import, so you could do it there as well...

erikals
02-23-2016, 06:15 PM
is that just spheres/particles?
yes, for some reason outputting geometry didn't work (first time around)


It's the real mesh which is the hard part...
not sure i understand, is it problematic to export an obj sequence from Houdini ?

m.d.
02-23-2016, 06:22 PM
if you want a 10-20 million poly mesh/per frame then its no issue....
alembic is way lighter and includes points and point movement/topology...kinda similar to mdd on top of an animated object vs an OBJ sequence way lighter..

the sim I am doing is an ocean sim for a yacht....so pretty heavy scene as is

erikals
02-23-2016, 06:25 PM
ah, like that.

yes, hope for some decent Houdini Alembic support inside LightWave in the near future.

m.d.
02-23-2016, 06:31 PM
for smaller sims, its probably a pretty good solution (obj export)

the realflow plugins are a great holdover for now (if we can get the import/export workflow right) until proper alembic or VDB support

the plugins are free, and although I like realflow, for any commercial work Houdini Indie is the way to go, (price wise) but wow....did realflow ever make a dead simple interchange workflow for its products.

erikals
02-23-2016, 06:49 PM
one idea, for waves it is possible to render out a hightmap image sequence in Houdini, (top view camera)
then add the image sequence to a SubD plane inside LightWave

(z depth)

would look something like this >
http://www.creativecrash.com/system/photos/000/060/878/60878/big/waterdispl2.jpg?1253505010

foam output could also be done this way

...could work

m.d.
02-23-2016, 08:55 PM
I've done something like that before....but a better way similar to what you describe is an obj with MDD....

the reason being height map is strictly 2d (up/down) displacement...where MDD is 3d displacement....so you can have some curling and sliding (some)

But ya I have done that out of necessity once before to try to incorporate a hand painted boat wake (actually an extremely thin 2d is TFD sim) on top of ocean waves displacement. Had to convert everything to an image map so I could control the composite of the 2 deformations.

Actually just dug through some old files and found a test using that method

https://www.youtube.com/watch?v=PQXnJbqyeMM

This was from a personnel project a few years ago....I eventually abandoned realwave as it introduced a ton of high frequency noise along the edges of the boat that could not be removed. My solution was to paint out the noise in fusion after converting to an height-field as you suggested. The resulting bulge was fairly unrealistic though.


I would not recommend realwave for moving boats because of this....I think you have to do a real fluid flip sim to do this properly....or fake it entirely with HOT ocean or likewise....realwave (2013 version at least) was very unpredictable interacting with collisions.

Anyways this sim I am working on is too complicated for deformations only...crashing waves ect

m.d.
02-23-2016, 08:59 PM
here is a little render from this....personnel project from about 2013/2014 octane beta v1 I believe



https://www.youtube.com/watch?v=Scp4yZQgGvc

not very good, but more of a workflow test

seghier
02-25-2016, 12:05 AM
Just pulling my hair getting liquid sims from Houdini to LW...

Realflow mesh export doesnt really work (kinda does...but crashes after a few seconds)....OBJ sequence isnt an option as each frame is 10's of millions of poly's (and no MB)....and Alembic leaves behind points and poly's...

The only option right now is export particles using the Realflow houdini exporter and let realflow renderkit mesh it on the fly....but Houdini has so much more advanced meshing control compared to realflow, and there is no IPR/VPR preview


What would be nice, and idiot proof...would be proper Alembic import.

I can even import the mesh into Fusion...but for some reason not lightwave.

Is this planned to be fixed for a new version? LW group always talks interchange-centric workflows, but in practice with FBX, Alembic and others....it is the least friendly...even compared to compositing apps

are you tried to load bin file using Nextlimit RFRK plugin manager ; than you can see the mesh when you render the scene and you don't need vpr
you can see the mesh in the viewport and you can change level of details

http://i.imgbox.com/dMSr6Upx.jpg

m.d.
02-25-2016, 12:24 AM
are you tried to load bin file using Nextlimit RFRK plugin manager ; than you can see the mesh when you render the scene and you don't need vpr
you can see the mesh in the viewport and you can change level of details

http://i.imgbox.com/dMSr6Upx.jpg

Good tip....I didn't know about that one.(used the render kit before but didn't realize there was a mesh display) I would have definitely used that in the past had I known.

My problem I think is non standard .bin files from Houdini, I have yet to have files that don't cause the RF import or RenderKit to crash after a short time.

The other issue is render kit requires a commercial license, and the RF import plugins do not.

seghier
02-25-2016, 12:43 AM
i didn't tried houdini
if you mean nextlimit render kit ; it's free and didn't requires a commercial license
you can also export sd files ; or use realflow plugin for houdini

m.d.
02-25-2016, 10:20 AM
i didn't tried houdini
if you mean nextlimit render kit ; it's free and didn't requires a commercial license
you can also export sd files ; or use realflow plugin for houdini

the renderkit is a free download but requires a license from the realflow license manager. I have an educational license from 2013 so it still works, but for commercial jobs it's a no go.

Unless things have changed, because now I see the connectivity plugins and the renderkit are combined in a single download.....but I definitely remember wasting about an hour of my time trying to install a license for the renderkit.

I am using the realflow plugin for houdini...still having issues with the mesh. And as far as SD....I think that is just scene transfer, you need the .bin files for mesh or particles.
It may be my system or lightwave install....as I was able to bring in a fluid mesh yesterday and it worked until i tried going back 1 frame on the timeline...then it hangs indefinitely and im forced to close lightwave.

erikals
02-25-2016, 10:25 AM
the renderkit is a free download but requires a license from the realflow license manager.
I have an educational license from 2013 so it still works, but for commercial jobs it's a no go.

yup.

seghier
02-25-2016, 02:06 PM
i don't understand why renderkit didn't work ? are you mean when use network rendering ? or you can't render anything ?
i used demo version of realflow 2015 and renderkit didn't require license ; why you tried to install a license ?

http://i.imgbox.com/MPg4Oj0i.jpg

m.d.
02-25-2016, 02:17 PM
i don't understand why renderkit didn't work ? are you mean when use network rendering ? or you can't render anything ?
i used demo version of realflow 2015 and renderkit didn't require license ; why you tried to install a license ?

http://i.imgbox.com/MPg4Oj0i.jpg

Didn't say it wouldn't work (although still some incompatibilities with Houdini) saying it isnt legal

From the realflow licensing FAQ's

What is the RealFlow RenderKit?
The RealFlow RenderKit is a set of tools that have been designed to facilitate the complex task of rendering RealFlow fluids. The RFRK enables you to generate procedural geometry at render time, and also render individual fluid particles. As a result, the RFRK will dramatically simplify and accelerate your RealFlow workflow. You will receive one copy of the RFRK with each standard license purchase. You can also purchase additional licenses separately. You will need one license for each machine you wish to use with the RFRK and your render engine of choice.

Do I have to buy the RealFlow RenderKit to use RealFlow?
No. To get the full benefit of the new features of RealFlow, you will need the RFRK - this is why we have included a free license of the RFRK with each standard license purchase. You have the option to purchase additional licenses to facilitate the rendering process. It is not obligatory but highly recommended!

I have installed my license, but the RFRK is still running in demo mode. What should I do?
Make sure you have deleted the demo license from the software.

seghier
02-25-2016, 02:24 PM
Didn't say it wouldn't work (although still some incompatibilities with Houdini) saying it isnt legal

From the realflow licensing FAQ's

What is the RealFlow RenderKit?
The RealFlow RenderKit is a set of tools that have been designed to facilitate the complex task of rendering RealFlow fluids. The RFRK enables you to generate procedural geometry at render time, and also render individual fluid particles. As a result, the RFRK will dramatically simplify and accelerate your RealFlow workflow. You will receive one copy of the RFRK with each standard license purchase. You can also purchase additional licenses separately. You will need one license for each machine you wish to use with the RFRK and your render engine of choice.

Do I have to buy the RealFlow RenderKit to use RealFlow?
No. To get the full benefit of the new features of RealFlow, you will need the RFRK - this is why we have included a free license of the RFRK with each standard license purchase. You have the option to purchase additional licenses to facilitate the rendering process. It is not obligatory but highly recommended!

I have installed my license, but the RFRK is still running in demo mode. What should I do?
Make sure you have deleted the demo license from the software.

ok ; the solution is to buy :)
but how you use bin files exported from realflow ; you are in the same case with license

m.d.
02-25-2016, 02:24 PM
no the plugins never had a license associated...only the renderkit, they were always free

the houdini exporters are also license free


The plugins are only import and export, where the renderkit is Realflow's entire meshing technology inside a plugin

seghier
02-25-2016, 02:33 PM
yes but you need realflow license to use bin files for commercial projects ; than you can use renderkit without problem

m.d.
02-25-2016, 02:51 PM
yes but you need realflow license to use bin files for commercial projects ; than you can use renderkit without problem

where do you read that? Do I need a Maya license to use OBJ sequence as well?

The bin files are not generated by Realflow, but by Houdini....the free and license free importers and exporters are the only thing used

EDIT: It appears looking at it now the render kit may now free as well...must have changed in 2015