@Fifth here's another build to try, I think it fixes the things I broke in the last one: http://quakespasm.ericwa.com/job/tyrutils-ericw-spike/9/artifact/dist/tyrutils-ericw-v0.15.3-10-g896b821-win32.zip

There is still room for improvement; I still get a few fringe artifacts on some terrain that was rotated 15 degrees, but it's pretty minor and way better than the last build.

@Spike: I switched this build to use SSE; for some reason x87 vs SSE was making a difference in the barycentric coordinates code, with SSE having less artifacts when interpolating points outside the triangles. I will investigate it better at some point :-) 

I will give it a whirl after work. These compilers give me leaks compared to the ones I normally use 

I normally use bjp's compilers. 

Yeah, these are less forgiving than bjp's. Raising the -epsilon parameter to qbsp can help, default is -epsilon 0.0001, try 0.001, 0.01 

seems to crash for me in this build. No idea what's causing it. 

Here's another try...
http://quakespasm.ericwa.com/job/tyrutils-ericw-spike/10/artifact/dist/tyrutils-ericw-v0.15.3-11-g78ca193-win32.zip

I notice extra/extra4 + soft crashes this one. Sorry about all the buggy builds but it'll take a bit to get all of this stuff stabilized and merged. I gotta go map first! 

This one appears to compile but I'm getting some black bars across surfaces even worse than the first build.
It's very likely this is due what I am testing it on (bsp pipe work) and is caused by micro-leaks and things along those lines. I suspect there might not be an easy fix for this other than to reduce detail for smoothed light maps, have smoothed textures use whole number values instead of decimals when doing its calculations? (I'm pretending I know what I'm talking about by this point, from my tests it appears that the closer to a whole number you get the better the smoothing appears to be across a surface). 

In some cases, the position where two vertices of two different brushes meet can become different for either brush due to differences in precision between them during the compiling. This will make the hidden orthogonal face of one of them to pop out, and this will mess up with the smoothing algorithm indeed.

It's one of the extreme cases I had predicted, but it shouldn't happen too often. 

ERROR: Too many vertices ( 65541 > 65535 )

Map also leaks in some mysterious way I haven't deduced yet because there is no .bsp generated due to the error... not sure if they are related...

Map only has 7901 brushes, so it shouldn't be hitting vertex limits I would think?

Any reason why this limit couldn't be raised? 

nm found the leak, and is what causes the vertex overflow... 

Cool - if you need >65k verts, I think you just have to switch to bsp2 with the -bsp2 flag. I will add that to the error message. 

Which is actually a good working philosophy even if you don't want to ship a BSP2. It allows you to work on an unsealed map for much longer. 

Bad working practice IMO. The longer you leave something unsealed the bigger the final task of sealing it.

I prefer keeping everything sealed as I go along, even if it's with giant bricks I'll eventually delete. At least then I know that everything else is without b0rken3ss. 

Well, this only leaked because I'm working on this map with Lun and I merged in some of his stuff, and one of the brushes was apparently corrupt after import. Think even with some more to do, we'll still be well under and won't need to bsp2. 

Depends on the map. I've had times when it would have been a fuckton of work to seal the map and it just wasn't time ... 

The longer you leave something unsealed the bigger the final task of sealing it.  

and whatnot. 

I've confirmed that the problem with not all water surfaces being lit is a tools bug.

The light tool will only generate lightmaps for the liquid surfaces that gets hit by a light entity's beam. If no beams hit a liquid surface, no lightmaps will be generated for it. I couldn't figure out a simple way to fix this yet.

I'm compiling the maps with qbsp -splitspecial enabled. 

I see, normally face->lightofs == -1 means the lightmap is black, but for liquids you need to treat -1 as "not compiled with lightmapped water, so render fullbright", is that right?

It should be easy enough to always save lightmaps for liquid faces, even if solid black. But I don't think I added lightmapped liquids support to tyrutils-ericw, do you have the code for it already? I am happy to add a command line flag for it, just don't want to waste time reimplementing it if you already coded it. 

that would be my fault. I added -lightwater etc args in that smoothnstuff branch, but didn't actually get around to testing it at all.
(release early, release often, right?)
it should be simple enough to tweak WriteLightmaps to just write a lightofs of 0 instead of -1 when texinfo->flags&TEX_SPECIAL, with all styles still set to 255, I guess.
I assume engines will be able to cope with that, but I've not checked. 

Wasn't there a concern due to the way that most engines render water by using a deforming mesh? 

@Spike ah, cool. Still meaning to merge in your smoothnstuff branch, just lazy right now :)
@Fifth hm good point, this will need to be tested. Also thanks for the idea about using "smooth" prefix on texture names to enable phong shading, that could be good. 

re: unmerged branch
I wouldn't worry too much, I've been too lazy to fix any niggles since I uploaded it. :s

re: smooth prefix
I disagree in the long term on account of that making multiple smoothing groups too awkward to use effectively.
supporting q2 .map files for their surface flags would be good. tyr's qbsp can already parse them, so we just need editors that support q2 .map with wad files, and to actually implement smoothing groups with it.

re: lit water
yes, lit water would generally imply that the lit water surfaces should also be subdivided by the qbsp in the same way that regular walls are (this means the qbsp needs to be aware of it too, and not just the light tool).
engine-based turb subdivision isn't really a concern, as it doesn't affect the lightmap. there are already occasions where surfaces are overly pointy. if anything, having the qbsp subdivide turb surfaces too would reduce this
the 'real' concern is that very few engines actually support lit turb surfaces

re: nstuff
I kinda want to build func_illusionaries and brushes with alpha-masked textures into the world too. conceptually this would be easy (assuming it doesn't have any fields set), they can just be treated like some water contents with water vising stuff, and be converted to the appropriate contents type just before qbsp outputs the unvised bsp.
this should improve engine batching and stuff (and be consistent with what hlbsp support already requires from engines).
there's some other things that need tweeking, but I forgot much of it. :s 

Yes, I've fixed something else in the light tool for the lights to display properly on liquids, but by default you just need to compile the BSP with qbsp -splitspecial. This will make water surfaces be subdivided, and with the TEX_SPECIAL flag not set. In theory, this should have made them be treated exactly like regular surfaces by the light compiler, so this bug with fully dark liquids is puzzling me� 

I've managed to fix it tonight. It should be bug-free, but it would be better to attribute a dummy lightstyle for fully dark liquids, to avoid potential "too many light styles on face" warnings.

Posted at InsideQC:
http://forums.insideqc.com/viewtopic.php?f=12&t=5769