sizeof(entity_t) = 336 bytes on 32-bit (pretty small size). 336 bytes x 8192 statics = 2.7 MB.

You are correct in engine design and philosophy.

But if I rewired it to do as you suggest, Mr. end user won't know even know he saved around 2.7 MB in RAM on all but the most unusual maps.

I mean, I raised the default zone allocation to 8 MB. That's probably 6 MB more than the most rare situation would actually need.

And both Mark V and Quakespasm default to 256 MB -heapsize. Which is about double the memory any map actually requires.

/At least those are my thoughts 

What's the drive to optimize for memory usage in Quake engines these days?

Is it programmer pride, or is there a solid tech reason? 

MH wrote DirectQ and RMQ engine (first BSP2 engine, MH designed BSP2) --- it has some slick memory management. FTE and DarkPlaces do as well, but I've only ever really looked at MH's stuff in relation to the memory management.

(Because MH writes really great code. Time permitting, I try to code in his clean style.)

Almost acquired the entirety of RMQ's dynamic memory management into Mark V in 2014.

I have unusually ambitious backwards compatibility goals because of another engine called ProQuake and realized that if I implemented full dynamic that non-dynamic memory engines (ProQuake, Qrack, Quakespasm, WinQuake, FitzQuake, etc.) wouldn't reliably be able to either connect (coop or over the internet) or playback demos with certainty.

To address that, would have to have a mechanism to set a hard limit. Ends up being a lot of work to be back where you started.

[So both Mark V and Quakespasm use 8192 entities as the limit by default.] 

'Why make maps better than id1?' :>

I think the other reason is the megamap format. Some things just don't work very well, like flickering lights. 

What's the drive to optimize for memory usage

"so modders/mappers can push the engine more" would be my initial thought, although I don't know much about engine coding. 

"'Why make maps better than id1?' :> "

No, no ... it means, "My machine has a kojillian gigs of RAM in it. Why stress about a few hundred bytes?"

:) 

Why stress about a few hundred bytes?

Because it's not. 

I think it would be daft to take a bloatware approach to engine mods to a 1996 game, even if computers in 2016 can handle it. A lot of people will want to run this on older PCs. The original was coded efficiently and there's no reason not continue in the same philosophy. 

What's the drive to optimize for memory usage
Dynamic limits for stuff means that it'll not crash due to the limit being too low (note: you typically still need a sanity limit).
Fixed limits are a constant battle between sanity and actual use. Any time anyone makes a slightly bigger map, the engine needs twice as much memory. Essentually, the program eventually ends up allocating memory right up to the edge of sanity. Which then causes it to crash because its BSS section clogs its virtual memory pool.
Okay, maybe I'm exagurating a little, while 1 fixed limit can be ignored, by the time you have enough such limits things do start to hit the fan. Its just good to make all limits dynamic early, before it gets too problematic.


Baker, so you want MarkV to have limited static entity counts because someone *might* record a demo and then play that demo in an even more crippled engine?
Really you're just ensuring that maps won't work in either engine, spreading the crippleware. 

"The original was coded efficiently and there's no reason not continue in the same philosophy."

Well, it would eliminate the now routine conversation that takes place when any new mod or large map gets released about which engine it works in, which limits need to be raised, etc.

And there's nothing inefficient about a dynamic array. Software around the world uses them every day. :) 

Well, it'd be nice if Quake was more accessible to people who don't know what the console is, or simply don't want to mess around with heapsize and the like.

But the onus there is on the mappers to set up their rc or autoexec properly.

Which engine is more difficult, although Quakespasm has become the standardised engine of choice for many. 

Well, it'd be nice if Quake was more accessible to people who don't know what the console is

Absolutely. I dream of an engine where a noob can just start it, and load a mod/map through an ingame menu and literally not need to worry about anything else. 

I think the ideal situation is dynamic arrays when possible, with some sanity check hard limit, and also "xxx exceeds standard limit of yyy" messages when "developer 1" is set.

- still runs as well on old computers / vanilla maps as an engine with vanilla limits would
- mappers can check how high over vanilla limits they are.
- eliminates frustration for players running huge maps


In practice there can be challenges doing this for each different limit.

Looks like QS and MarkV both MAX_STATIC_ENTITIES at 512. Hunk_Allocing the static entities as they arrive sounds like a good solution, as long you can be sure things added to the hunk during CL_ParseServerMessage will stay around until the client is restarted. 

map viewer

Mark V has a map browser already... maybe it could do with a mod browser. To be honest is it necessary? Most maps come with a readme txt with instructions. 

To be honest is it necessary? Most maps come with a readme txt with instructions

I once tried getting some people at work into Quake. The short answer? Yes. 

Doesn't something like QuakeInjector solve the noob issue? 

I didn't know about that thing. I guess I'm the noob :} 

The original was coded efficiently and there's no reason not continue in the same philosophy.

It actually wasn't. The original was coded for an 8 MB MS-DOS machine with no multitasking and no virtual memory, so (1) it could freely grab all of the memory in a machine without worrying about having to play nice with other programs, and (2) if it ran out of physical memory it would explode.

There's vestiges of this all through the original memory management system. Zone memory is one example; the whole LRU/MRU/flush/evict thing in cache memory is another.

When doing any analysis of Quake's memory subsystem you absolutely have to bear in mind that it was coded for an OS and for constraints that no longer exist.

And of course, with specific reference to static entities (and a few other things), there is always this: https://github.com/id-Software/Quake/blob/master/WinQuake/cl_main.c#L46 

The quake engine was coded for efficiency in the use of memory and stability. For one, the so-called modern engines typically have bugs which are not detectable because of the slack memory handling in a modern OS, whereas the popular DOS extenders were strict. This enables detection of memory related bugs and stable code.

The majority of modified winquake engines seem to be enthusiast projects to show off the latest visual feature. This is worthwhile for working on experimental features, but much less so for the mappers and gamers. They should have to debug the maps and OS issues and not have a common source of bugs in the engine, too. This is why quakespasm is favored, even though it is missing the original software engine. 

Perhaps what you have to say would be more interesting if you provided some specific examples with some specific engine names.

Everyone likes a good rant, but that's muddy as hell with no specifics. 

They should have to debug the maps and OS issues and not have a common source of bugs in the engine, too. This is why quakespasm is favored, even though it is missing the original software engine.

It's not that simple. Quakespasm, FitzQuake and so on are also favored because they operate in 32-bit color. They have 32-bit colored lights, 32-bit colored fog, 32-bit alpha blending, 32-bit color skyboxes and so on.

Software renderers can't compete with that. Even less so if 32-bit color filtering for 8-bit textures is also taken into account. There's no way to make a 24-bit software renderer fast enough to do all those true color effects at a playable speed. No way.

I was happy enough to figure out how to do color-corrected 8-bit alpha blending in real time, because the alternative color system I was developing would be slower and overcomplicated. It's hard to create precomputed 16-bit or 24-bit color transformation tables that fits in the CPU caches and are fast (using a single access for each RGB color instead of doing the R, G & B transformations separately).

The smooth, smooth colors of hardware rendering are something that really sets both worlds apart, even if texture filtering is disabled. Software rendering feels just poor in comparison.

That's why there's no market for it. Texture filtering gets disabled because it distorts the texture resolution, and software rendering is not used because it distorts the texture's colors.

Any software renderer with 8-bit colored lights, 8-bit colored fog, 8-bit alpha blending, 8-bit color skyboxes and so on is doomed to be just a trifle, just a novelty. There's no reason for anyone to put them above what the hardware renderers can do.

All this may sound weird coming from me, but I'm just thinking objectively. 

Looks like QS and MarkV both MAX_STATIC_ENTITIES at 512. Hunk_Allocing the static entities as they arrive sounds like a good solution, as long you can be sure things added to the hunk during CL_ParseServerMessage will stay around until the client is restarted.

We can be absolutely certain this is the case for static entities.

The client-side code (where the Hunk_Alloc would be done) adds efrags as the last part of setting up a static entity, and efrags depend on the world existing. The world is loaded into the hunk. Therefore the hunk must be valid otherwise the world itself wouldn't be valid.

Server-side static entities depend on the progs.dat to have been loaded first, and the progs.dat is also loaded into the hunk. So the same applies - if the hunk were somehow invalid then the progs.dat would also be invalid.

So both client-side and server-side we can conclusively prove that there are absolutely no circumstances in which it's not valid to use the hunk by the time static entities are loaded. 

Quake 1 and its artwork was designed around 8-bit color and the software renderer. By comparison, the opengl renderer has a "washed out" appearance as described earlier by the Winquake community and in a summary of Quake engine differences at Quaddicted. I agree that the opengl engine is superior in maps specifically designed for that engine, however, the remakes of Quake maps (ExMx) look better to me in the software renderer. These maps aren't designed around high resolution skyboxes and transparent textures.

I agree that the software renderer can't compete with the opengl renderer where it performs best. However, the software renderer has its strengths, such as portability (and performance) on non-gl systems, suitability to represent the original artwork and derivatives, (and better compatibility with past mods in the case of Winquake projects).

If there was a Winquake client with Quakespasm like software development practices, then it would be easier for developers to offer secondary compatibility to the software renderer, such as Sock and others have done. 

Maybe someone should code a very conservative WinQuake client that doesn't focus on effects and instead is essentially a WinQuake clone of FitzQuake/Quakespasm.

Would you agree with that statement? 

Fine! Next update I'll change static ents to array of pointers. And in cl_parse.c

Change this to Hunk_Alloc size(entity_y) ent = &cl.static_entities[i];

I guess it's a 2 liner anyway.