That seems to work actually. Very cool. Have you altered the code at all? It seems slightly less smooth than before (but still a lot better than ordinary lighting). 

Seems like whole number integers on vertices or on texture alignment place a huge role in getting the lightmap to look right. Otherwise you get huge black bars or just errors in the lightmap. 

I see what you mean about less smooth / more banding, I messed up something with that last build; trying to fix it.. 

@Fifth here's another build to try, I think it fixes the things I broke in the last one: http://quakespasm.ericwa.com/job/tyrutils-ericw-spike/9/artifact/dist/tyrutils-ericw-v0.15.3-10-g896b821-win32.zip

There is still room for improvement; I still get a few fringe artifacts on some terrain that was rotated 15 degrees, but it's pretty minor and way better than the last build.

@Spike: I switched this build to use SSE; for some reason x87 vs SSE was making a difference in the barycentric coordinates code, with SSE having less artifacts when interpolating points outside the triangles. I will investigate it better at some point :-) 

I will give it a whirl after work. These compilers give me leaks compared to the ones I normally use 

I normally use bjp's compilers. 

Yeah, these are less forgiving than bjp's. Raising the -epsilon parameter to qbsp can help, default is -epsilon 0.0001, try 0.001, 0.01 

seems to crash for me in this build. No idea what's causing it. 

Here's another try...
http://quakespasm.ericwa.com/job/tyrutils-ericw-spike/10/artifact/dist/tyrutils-ericw-v0.15.3-11-g78ca193-win32.zip

I notice extra/extra4 + soft crashes this one. Sorry about all the buggy builds but it'll take a bit to get all of this stuff stabilized and merged. I gotta go map first! 

This one appears to compile but I'm getting some black bars across surfaces even worse than the first build.
It's very likely this is due what I am testing it on (bsp pipe work) and is caused by micro-leaks and things along those lines. I suspect there might not be an easy fix for this other than to reduce detail for smoothed light maps, have smoothed textures use whole number values instead of decimals when doing its calculations? (I'm pretending I know what I'm talking about by this point, from my tests it appears that the closer to a whole number you get the better the smoothing appears to be across a surface). 

In some cases, the position where two vertices of two different brushes meet can become different for either brush due to differences in precision between them during the compiling. This will make the hidden orthogonal face of one of them to pop out, and this will mess up with the smoothing algorithm indeed.

It's one of the extreme cases I had predicted, but it shouldn't happen too often. 

ERROR: Too many vertices ( 65541 > 65535 )

Map also leaks in some mysterious way I haven't deduced yet because there is no .bsp generated due to the error... not sure if they are related...

Map only has 7901 brushes, so it shouldn't be hitting vertex limits I would think?

Any reason why this limit couldn't be raised? 

nm found the leak, and is what causes the vertex overflow... 

Cool - if you need >65k verts, I think you just have to switch to bsp2 with the -bsp2 flag. I will add that to the error message. 

Which is actually a good working philosophy even if you don't want to ship a BSP2. It allows you to work on an unsealed map for much longer. 

Bad working practice IMO. The longer you leave something unsealed the bigger the final task of sealing it.

I prefer keeping everything sealed as I go along, even if it's with giant bricks I'll eventually delete. At least then I know that everything else is without b0rken3ss. 

Well, this only leaked because I'm working on this map with Lun and I merged in some of his stuff, and one of the brushes was apparently corrupt after import. Think even with some more to do, we'll still be well under and won't need to bsp2. 

Depends on the map. I've had times when it would have been a fuckton of work to seal the map and it just wasn't time ... 

The longer you leave something unsealed the bigger the final task of sealing it.  

and whatnot. 

I've confirmed that the problem with not all water surfaces being lit is a tools bug.

The light tool will only generate lightmaps for the liquid surfaces that gets hit by a light entity's beam. If no beams hit a liquid surface, no lightmaps will be generated for it. I couldn't figure out a simple way to fix this yet.

I'm compiling the maps with qbsp -splitspecial enabled. 

I see, normally face->lightofs == -1 means the lightmap is black, but for liquids you need to treat -1 as "not compiled with lightmapped water, so render fullbright", is that right?

It should be easy enough to always save lightmaps for liquid faces, even if solid black. But I don't think I added lightmapped liquids support to tyrutils-ericw, do you have the code for it already? I am happy to add a command line flag for it, just don't want to waste time reimplementing it if you already coded it. 

that would be my fault. I added -lightwater etc args in that smoothnstuff branch, but didn't actually get around to testing it at all.
(release early, release often, right?)
it should be simple enough to tweak WriteLightmaps to just write a lightofs of 0 instead of -1 when texinfo->flags&TEX_SPECIAL, with all styles still set to 255, I guess.
I assume engines will be able to cope with that, but I've not checked. 

Wasn't there a concern due to the way that most engines render water by using a deforming mesh? 

@Spike ah, cool. Still meaning to merge in your smoothnstuff branch, just lazy right now :)
@Fifth hm good point, this will need to be tested. Also thanks for the idea about using "smooth" prefix on texture names to enable phong shading, that could be good.