Windows OpenGL + WinQuake

(didn't include the DirectX 8 build because because it flags Microsoft's malware detection for some reason ...)

Changes:
1) Fixed "give bug" pointed out by therailmccoy.
2) Fine tuned timedemo, playdemo and start demos transition handling (zzzzzz, but I want it to be "right").
3) Change to how warp textures and console scaling changes take effect. Not very interesting but laying the groundwork for adding scaling to the WinQuake version.
4) Some other fine tuning.

Next update is likely to have:
a) Scaling solution for WinQuake vertical resolution > 900 +/-.
b) The extra keys therailmccoy likes that are in Source engines (ctrl + a, etc.)
c) Rework to give especially for the things that only Mark V lets you give yourself like keys and runes. Now that I have auto-complete cryptic names like Q1 for sigils and KS for silver key need to go. Will have names like "give silverkey" and you can either auto-complete from the 'S' or press CTRL+SPACE to see the valid ones. And somehow letting user know they can remove the silverkey/goldkey/rune1-4 using same command.
d) Attempt to solve PNG screenshot too white with sbar alpha for NightFright.
e) Mark V does play DZip demos. You have to have dzip.exe in your Quake folder. Next update is likely to not require dzip.exe at all and just be built-in. The SpeedDemosArchive has 1000s of speedruns in .dz format.
f) Maybe cut/paste/shift select for all text fields in the game.
g) Quakespasm allocblock lightmap speedmap.
h) Quakespasm lightmap precision refinements, but I want to try to have the problem first. 

Regarding h), it should be easy to reproduce on the Mac build, I was able to with the recent Mac build you posted.

Also, if you want, you should be able to reproduce the issue in Visual Studio as well with the "/arch:sse2" flag. IIRC this will still make a 32-bit exe, but will require a Pentium 4 or better. Info on that flag: https://msdn.microsoft.com/en-us/library/7t5yh4fd.aspx 

I was able to with the recent Mac build you posted Sounds like when I boot up that Mac that'll be easy then (most of past attempts failed because the screenshots of the issue had no name and I looked at a few mapjams and didn't see issue ... was frustrating). The BSP compiler actually does the calcs using doubles so your fix is a natural extension of what Q engines should be doing already.

General note: When I was looking at Quakespasm's render adjustments, something bumped my memory of MH's expression "DrawSequentialPoly must die" (from a few year's back) ... and I find it a bit onerous to have to maintain 2 copies of the drawing code (one for world, one for brush models) when they are essentially the same aside from .alpha.

JoeQuake and FuhQuake (predecessor to ezQuake), for instance have had DrawSequentialPoly gone for a long, long time.

Note #2: I wish I knew an easy way to have WinQuake use vid_vsync. That uses a wgl function on Windows. I could make WinQuake use OpenGL only for buffer swapping, but then exposes WinQuake on Windows to bad OpenGL driver issues. 

ditch mgl so that you can use directdraw directly.
use directdraw's vsync stuff, then you won't need to copy textures twice nor suffer from missing gl drivers. 

I wasn't aware DirectDraw had vsync somewhere in the API, although I'm not using mgl nor DirectDraw but rather just BitBlt/StretchBlt.

[I copy the buffer using BitBlt on Windows derived from what initially started as MH's pure API WinQuake and still has some of that left. The Mac WinQuake renders to texture in OpenGL via proxy texture and draws the texture to the entire window.] 

The link above for the updated package should be http://quakeone.com/proquake/interims/mark_v_20150419_windows.zip 

The impetus for getting rid of DrawSequentialPoly in qs was ijed's RRP map telefragged.bsp... The first main room, after you leave the entrance room and go through a corridor, has several large brush models in the PVS (a train outside, and moving bridges, inside). I was getting something like 17 fps on my laptop (while not a gaming laptop, high-end enough that it should breeze through anything quake. i.e. i7 quad, geforce 650gt).

Profiling showed 90% of rendering time in that room was in DrawSequentialPoly, and most of that is doing gl state changes, because the brush model surfaces aren't sorted by texture. So if the brush model uses several textures, and has a couple hundred faces, you end up doing: "bind texture A, draw surf, bind B, draw surf, bind A, draw, bind B,.. Etc.."

In fitzquake's renderer it's a little more awkward to merge brush model and world drawing than in plain GLQuake, because of how fitz sorts the world surfaces by texture when the PVS changes, but that sorting persists across several frames. So initially I tried to have each brush model clear the texture chains, then chain each surface in the brush model, but this broke the world drawing, because the world wasn't re-chained every frame.

What I ended up doing is making two sets of texture chains, one for the world that can persist across several frames, and one for the brush models that's cleared every time a brush model is drawn. It's a bit convoluted, but it works well, and is simple than maintaining two copies of the drawing code (and faster!). (Side note: I've thought of reverting QS to use a GLQuake style R_DrawRecursiveWorld that regenerates the world texture chains every frame. Based on some comments from mh it may be a bit faster because surfaces are sorted front-to-back, and it would remove the complexity of two sets of texture chains. However the current method works fine so it's not a high priority.)

The other nice thing is, that refactoring paved the way to using a VBO to store the world and brush model (which is isolated to just one drawing function), but it's certainly not necessary to use VBO's to get a good performance boost from eliminating DrawSequentialPoly, at least in cases like ijed's map that make heavy use of brush models. 

Unfortunately, Mk V currently fails at the very same spot as JoeQuake does when playing Nehahra:

In "Nehahra's Den", after you kill Nehahra and all the gibs start flying with the exit lift descending, game crashes with following error:

1032 byte packet exceeds standard limit of 1024
host_error: ed_alloc: no free edicts (max_edicts: 512)

Here are some savegames to check (no autodemo this time, sorry). The quicksave will bring you to the time right before Nehahra goes down, just keep shooting at it and see what happens. 

I'll take a look at that and resolve. 

Pleased to hear the additional text editing shortcuts might make it in soon. =)

It's an excellent idea to have more intuitive names for give-able items along with autocomplete; might it be worth adding every possible item, so Biosuit, Ring of Shadows etc? Possibly Nehahra/Quoth items too if running those mods.

Btw, the 'give' issue is indeed fixed for commands chained with semi-colons, but giving runes still has the problems I described before.

I also noticed that 'pak help' and 'zip help' commands will result in a crash. 

Gamma range

Extended to the limits that Windows hardware gamma will accept for the next version so your higher level gamma will work. I had limited them to attempt to keep them in the valid range, but the high range for gamma corresponded to the menu limit not the hardware limit.

Give - more items, quoth, nehahra, etc.

Not sure. I've already made the names user-friendly. Many of those things are likely to be impulses which are mod specific behaviors outside the engine. One example, the "give" command cannot give you "Quad".

Extra editing keys

Those are in the next one.

I also noticed that 'pak help' and 'zip help' commands

Hehe, 'pak' and 'zip' exist just to expose those for testing. I don't expect them to be there in the end, and didn't expect them to be noticed ;-) 

Items like quad are given by impulse commands, but Quoth does implement named aliases for these commands, for example "quadcheat" in the console performs impulse 255 and gives you the quad. These should autocomplete when the engine supports it - as fitz085 does I'm sure Mark V would as well... 

I am actually wondering why that Nehahra crash happened. Didn't you raise the limits for RRP already, including edicts? That should have avoided this issue. Was using the April 19 snapshot after all. Anyway, maybe Nehahra is handled differently.

I am pretty sure it's the only problem with the mod, though. Until that point, everything worked flawlessly (and I don't expect it to screw up in the last level when you got to fight Max).

Side note: Is there actually any chance you are able to add the missing fog effects or is that something Mk V simply cannot pull off? 

Not Nehahra's fault, engine's fault. Reason is long-winded and technical. I'll try to explain.

Mark V guesses the number of server edicts needed. ne_ruins and end of nehahra, because they spawn lots of entities tricks this and over-runs the limit.

I wanted to avoid dynamic allocation of entities because then it requires a client to dynamically allocate them. I'm not interested in Mark V as a server being incompatible with FitzQuake 0.85 or Quakespasm as a client, I view that unacceptable incompatibility.

Hence, the guessing (mostly to avoid the max_edicts cvar)--- and guessing wrong in this case.

But I think I have an alternative, but need to think more.

For now, setting host_max_edicts to, say, 8192 should make anything run. Mark V will honor the cvar if it isn't set to 0.

fog

I can't remember the specifics. I know I had the fog enabled at one point. Then I had cause to disable it and I believe it was for a good reason, but I don't remember it right now.

I like fog so it had to be a good reason, but I can't recall --- it could have been something like ...

A) Perhaps FitzQuake fog and Nehahra don't mix right. Maybe it totally obscured the skyboxes. Maybe it was something else.
B) Maybe the fog keys in the maps were wildly inconsistent?
C) Maybe I compared DPNehahra vs. JoeQuake there was a serious difference?

Like I said, there was a reason. Now, I need to remember why ... 

Having the client dynamically allocate entities is easy. Having the server dynamically allocate them is hard, because the QC interpreter does a lot of pointer offset trickery to access edict fields. It's even valid for the interpreter to try to access a field in a different edict to the current one. I'm not sure why you'd even want to, but I've tripped up on that assumption in the past.

Static entities and efrags have absolutely no reason to have hard-coded limits and can be just pulled from Hunk memory as required. At a guess I'd say that the hard-coded limits in the original were there because Quake had to run on an 8mb MS-DOS machine (a lot of the weirdness in Quake's memory system comes from that and the fact that DOS had no virtual memory, and it's notable that id threw it all out when they moved away from DOS). There's no reason to preserve that in a modern source port. 

@baker:
better the client disconnects than the server crashes.

fog is different in almost every engine. it would be nice if someone actually documented the formulas engines use in an attempt to create some standard that all engines could then use...
https://xkcd.com/927/

@mh
the _only_ reason for weird fields is weird ens - ie: qccx hacks (there's no other way to easily generate those dodgy fields). imho, nuke them from orbit, but I guess baker doesn't have that luxuary due to his proquake stuff.
fte+dp use regular indexes for ents instead of offsets (screw qccx), and in doing so, they can dynamically allocate more memory. the catch is that you can't free it again mid-map. 

Fog isn't just different on every engine, it's also potentially different on different hardware, drivers and even APIs.

Fixed pipeline fog (i.e the various glFog commands) is allowed to be evaluated per-vertex or per-fragment. AFAIK OpenGL doesn't specify this, but D3D also allows more explicit control of range-based (versus simple depth-based) fog and even table-based fog.

You're completely at the mercy of what your 3D card and driver gives you for this, so it's pointless specifying a standard. The best anyone can do is implement something that looks good enough to pass, but even so it may look completely different (even with the very same code) on somebody else's PC.

Fixed pipeline fog doesn't even work at all under D3D with Shader Model 3 or better hardware. You must write your own fog in a shader.

Fixed pipeline fog quality in OpenGL is controlled by a hint and the documentation for glHint explicitly states that "the interpretation of the hints depends on the implementation". It's perfectly legal for you to request GL_NICEST but for the driver to give you low-quality instead.

Shader-based fog can use any formula at all, and can likewise be calculated per-vertex or per-fragment. The end result can look much higher quality than fixed pipeline fog (you can even experiment with crazy things like moving fog) but the higher quality may be interpreted as different or non-standard by comparison to what a player is otherwise used to.

Quake 3 and Doom 3 avoid all of this mess by doing their own fog as an extra pass over all surfaces, which obviously comes at the cost of extra geometry submission and processing. Depending on the scene, this could cut framerate in half.

It's also possible to implement fog by drawing the depth buffer as a full-screen quad over the scene and deriving a fog formula from the values stored in it, but that of course will fail for anything that's not written to the depth buffer (particles, translucent water, etc won't be fogged). 

Well, putting max_edicts "8192" into autoexec.cfg enabled me to get past "Nehahra's Den" with existing savegames. The problem is that while Nehahra's gibs are flying around, the exit elevator breaks through the ceiling, and with all the glass shards and debris raining down while gibs are still visible, it's just too much with low edicts settings.

I noticed a small detail with episode ending text screens. Looks like the last line(s) of text are cut off. Thought it may be because I use scr_menuscale 3, but reducing the value didn't reveal the missing lines (they were complete when checking it on console, though).

Summary of Nehahra playthrough:
- "Nehahra's Den" crash fix needed
- Possible issue with ending texts (may be general issue or not)
- Adding missing fog would be nice, but optional
- Otherwise, flawless gameplay. Music plays (if set up properly), cutscenes work, all maps completable etc

Keeping the "resurrection crash issue" with ne_ruins in mind, I will continue my Mk V testing queue with:
- Quake mission packs (SoA almost done already)
- Travail
- Honey
- Rubicon Rumble Pack

Possibly more after that, provided my free time allows it. 

@nightfright .. I appreciate the testing. I can't work on this and play a ton of mods at the same time ;-) 

Did you say that earlier you had max_edicts at 512? Because that's lower than dosquake which allowed 600 edicts, which might explain why it caused crashing. There's an argument here that allowing people to reduce the memory footprint below what was considered acceptable 19 years ago is unnecessary today. This argument would continue that engines should prevent people shooting themselves in the foot by enforcing a minimum max_edicts of 600. 

Edict size in ID1 is 876 bytes. This can be variable because mods can add their own edict fields which are tagged on at the end of the struct, accessible normally through QC but only accessible through C code by means of some pointer jiggery-pokery.

Say 1k as a good working size and because it makes subsequent calculations easier.

So if max_edicts is 8192 that's 8mb of memory. If it's 32768 it's 32mb of memory. Neither is a huge figure today, nor would it be considered huge on a PC from 10 years ago.

Most of that memory is going to be unused though, and there's also the fact that a lot of engine developers like to have their engines run on the same (or at least similar) class of machine that the original ran on.

One solution under Windows would be to VirtualAlloc a largeish pool of memory - say 64mb (for headroom) - with MEM_RESERVE. This won't count towards memory used, it just marks pages as reserved for subsequent use. Then as edicts are assigned from that pool, another VirtualAlloc with MEM_COMMIT for each new edict will actually use the memory, and only the memory, required for however many edicts a map might need.

I'm sure Linux and other POSIX systems have APIs that allow a similar scheme, but I'm not familiar with them. Either way, this setup allows the max_edicts cvar to go away.

IMO this is a good thing. I get that max_edicts is well-meant, but a major weakness of it is that it puts the obligation on the player to do the right thing. And the player must be aware that it exists, know that it may be needed, know what the appropriate value to set it to is, and know to set it before loading the map.

That just seems appallingly player-unfriendly to me. OK, I also understand that newer mods come with their own quake.rc that sets it accordingly, but it's still broken under older mods and it's something that with a bit of thought doesn't even need to exist.

In the end I just don't see an acceptable middle-ground for players. Either set a hard max_edicts and design mods around it, or make there be no practical maximum. But putting the burden on the player to do the right thing is eventually just going to blow up in your face. 

512 sounds low, but it was a machine picking the limit because the map probably had like 150 entities.

Dynamic ents will fix.

Dynamic ents doesn't solve any issues except this one. By the time you hit 5000 server edicts you are going to have all kinds of problems with the networking limits for stock Quake.

If it isn't one thing, its another. 

Windows OpenGL | WinQuake

1) PNG screenshots might be fixed. Need NightFright to check since I don't experience issue. Changed download from GL_RGBA to old style GL_RGB before saving to PNG.

2) WinQuake large resolutions should be fine, but scaled (because WinQuake renderer can only do so many vertical lines). I don't have a 1080p monitor, so need verification as well.

3) Added Ctrl+Left+Arrow, CTRL-A, CTRL-Z (undo), CTRL-SHIFT-Z (redo) to console.

[I was going to do one level of undo. Then I thought, I'm not going to bother with undo. So of course, it has an unlimited undo buffer that compacts to words.]

4) Fixed drawing chat text if the console width was too narrow to display the whole line.

I have another batch of changes, but wanted to hit a check a "checkpoint" because I made heavy revisions to the console for auto-complete, console drawing and for undo. The auto-complete function in previous versions worked fine, but the code was embarassing cobbled together spaghetti caked on layer after layer that happened to work. 

Windows Open GL | WinQuake 

I dunno what you did between this release and the last one but music suddenly works!

Weeeeeeee.....