Is there a simple reason why the light tools can't be extended to produce lightmaps of essentially arbitrary resolution?

More generallly, -extra 4 is still producing aliased shadows for me (e.g. here: https://ubiquitousgame.files.wordpress.com/2016/09/01.jpg). Is there anything I could try to get better looking lightmap resolution? 

It would require new file formats and engine support I would imagine.

This was thoroughly investigated a while back and actually got working, and from what I remember the feedback went something like:

"Oh this is cool but a bit too sharp actually, can the shadows be made softer?" (makes it softer) "Hmmm, even softer?" (makes it even softer) "...softer still?...ok actually on reflection I think quake's default lightmap resolution actually looks better but thanks anyway ^_^" 

I imagine the tools to generate the lightmaps weren't as good then? From what I can tell the quality of lighting improved quite a bit in the last N years. 

This is a limitation of the vanilla BSP format. Lit2 solves it, but only a few engines supports it. 

I think it was only something like one year ago when this was explored. Might be wrong - my temporal awareness is somewhat shoddy to say the least. 

what about "-soft 2" or even more? 

higher-res lightmaps have their perks: http://triptohell.info/moodles/junk/fte-20150311190731-0.jpg

the biggest issue is that lightmap changes (both flashing lights and dlights) become abusive, which essentually requires multithreading and/or rtlights in order to prevent it being unplayable, which greatly limits the number of engines willing to implement it.

crank it up to max and the bsp files become insanely huge, but that's only to be expected from poor-man's megatexture. :)

either way, the stepping issue doesn't really go away. the only way to fix that is to use cubic filtering instead of linear filtering.
currently this requires custom glsl. I really ought to investigate it some time. :s 

I think there is also some room to improve the filtering within light.exe.. it's using a box filter for -soft and -extra, which is the worst possible filter, I want to try lancosz or others.

Anyway to get the best quality currently you should combine -soft and -extra4.
-extra4 alone will be sharper but have more aliasing 

Thanks for the explanation all 

the biggest issue is that lightmap changes (both flashing lights and dlights) become abusive, which essentually requires multithreading and/or rtlights in order to prevent it being unplayable, which greatly limits the number of engines willing to implement it.

As an aside, I've always thought the ideal handling for light styles would be to upload the four styles as 4 separate textures and combine them using multitexture (or multiple passses) when rendering. Are there any engines that do this? 

rmqe does, iirc.
my engine that tries to draw the entire worldmodel in a single draw call also does.

the fragment shader is a bit faster if you do all 4 lightmaps with a single texture, at least when there's no lit support. you can then do the style->value lookups in the vertex shader and your fragment shader basically gains only a dot4product compared to a luma texture. 

but couldn't the engine be modified to interpolate hard edged shadows? 

As an aside, I've always thought the ideal handling for light styles would be to upload the four styles as 4 separate textures and combine them using multitexture (or multiple passses) when rendering. Are there any engines that do this?

I also have working, but unreleased, Q2 code that does this. It also needs 2X modulate support, but in practice anything that's not of 3DFX vintage has that. It's possible with basic GL_ARB_texture_env_combine but much easier (and nicer) with shaders.

The basic formula is texture * light0 * style0 + texture * light1 * style1 + texture * light2 * style2 + texture * light3 * style3, or texture * (light0 * style0 + light1 * style1 + light2 * style2 + light3 * style3).

You can do it with multitexture and a single shader, substituting 0 for the style value (and using an all-black texture) for styles that a surface doesn't have. Pros is code simplicity, everything goes through the one code path, fewer state changes; con is extra texture accesses.

You can do it with multitexture and 4 separate shaders, trading off texture lookups versus state changes.

You can do it with multipass and a single shader; similar tradeoff as above but with added overhead of extra draw calls too.

Fastest way is if you've monochrome lighting so it becomes a single texture access and a dotproduct: texture * dot (lightmap, styles).

Performance is similar to stock GLQuake for scenes without many lightstyle animations; it's one of those optimizations where you shouldn't expect it to double your framerate in DM3, but if you play a map with lots of lightstyles it works great.

What's nice is that you can then add lightstyle interpolation to the engine and get it for free.

For dynamic lights you do something similar to RT lights which works much nicer for MDL files because you've now got proper directional shading and attenuation on them instead of them just being uniformly lit. 

Pros is code simplicity, everything goes through the one code path, fewer state changes; con is extra texture accesses.

Would the texture accesses problem be solved by making sure all 4 styles end up on the same texture atlas? Or is it just the literal lookup into 4 different pixel locations? And is that anywhere near the cost of uploading new textures? 

Or is it just the literal lookup into 4 different pixel locations? And is that anywhere near the cost of uploading new textures?

It's 4 different locations, one for each style.

It can be mitigated by replacing texcoords for unused styles in a surface with {0,0} (so that the lookup will be in the GPU's texture cache) or by using a 1x1 black texture for lightmaps of unused styles (same reason), but again we're getting into state change tradeoffs.

Comparison with cost of uploading new textures is an "it depends" answer. If you've a switchable lightstyle like the ramp in e1m1 then uploading is going to be faster. If you've a room with hundreds of surfaces each of which has multiple torch flicker animations on it, then uploading is going to be slower.

On balance I think that irrespective of which method you use to handle animation, it's the right kind of optimization. What's already fast either remains fast or drops a little, whereas performance of what's slow comes up and best case is levelled. 

I forgot to mention this.

Rather than using 4 textures, one for each style, with 3 of the RGBA channels used and the 4th unused, an alternative is to use 3 textures, one for each of R, G, B with the styles packed into the 4 channels. That both saves memory and reduces those 4 lookups to 3. 

Seeing as we're talking about lightstyles etc. right now I thought this would be a good time to ask: had there ever been an attempt to raise the number of lightmaps past 4?

I ask because as a mapper i've run into the issue a few times, and its pretty annoying when it crops up. i understand the need for such a low limit in the past, but is there any particular reason other than a desire for compatibility that we still have it? I can't imagine that there are many computers today that struggle to switch lightmaps frequently... 

It's not a problem for RT lights.

For lightmaps it would need BSP format changes as well as engine and tool changes. I don't know/can't remember why we didn't do it for BSP2. 

One thing that may help, in the next release I made light.exe not blow up when the "too many light styles on a face" warning happens. It will compute all the light styles and then save the brightest 4 per face. 

There'll still be a warning, right? Not knowing why you're not getting the desired results would be pretty frustrating and cause a lot of forum posts! 

Yep the warning is still there 

One of the quirks of Quake, in terms of engine, tools and formats, is that it's stuffed full of hard-coded magic numbers like this that can make extending limits while retaining compatibility sometimes impossible.

Unless you actually knew, you'd think that it would be reasonable to suppose that each face structure would have an int member indicating how many lightstyles it has. I know I'd think that. So you might be surprised to learn that it actually doesn't - instead there's a flat array of 4 styles and that's fixed by the BSP format.

What's even more fun is that the code is littered with cases where sometimes a #define is used but other times the number 4 is hard-coded. 

That does sound like great fun.

I like to daydream about one day just taking quake and gutting it and building something similar that's barely compatible but much more friendly and so on. Really just daydreams though, I don't know C. And I don't want to really deal with fun 

I've been wondering if I could use this feature to get smooth blends of textures for my map, i.e. a gravel path that fades smoothly into grass, but unfortunately all that _project_texture seems to do is basically tint the colour of the light a bit, with some textures that have significant colour differences producing multi-coloured lights

Part of me thinks that this is something that just can't be overcome; an issue with lightmap resolution, or something like that. But I feel like I should still ask; is it possible to get more fidelity out of this? If it's already possible, how do I do it?

It's hardly dealbreaking if it's infeasible, but it'd be nice to have...