I want to create a stealth system for quake, but am unsure of how and where to start. I have a general plan on how to create a rudimentary stealth system, which involved taking the location/light level of the player in the level and connecting it to an awareness system for the monsters. The lower the brightness, the less the awareness. I'm looking to implement this using QuakeC but am unsure of how to get the players brightness levels or lightmap locations for this sort of thing.

And yes, I have played In The Shadows. It's cool, but not the sort of stealth I have in mind. 

One of the obstacles is that basic Quake engines don't give you a way to detect light levels. You'll need a custom engine that support the DP_QC_GETLIGHT extension (for example Darkplaces).

Once you have that, you can tell the light at a given coordinate. Worth remembering that Quake doesn't actually have a 3D representation of lighting - a model is lit according to the light level of the floor directly below it. That's what the function will give you, so plan accordingly when placing shadow areas in your map.

What you do with the light level is a whole other matter. In basic Quake, AI awareness is an on-off affair - either the monster is entirely ignorant of you or they are actively attacking and have omniscient awareness of exactly where you are. You'd probably want to invent at least two other states of behaviour:

- Monster is not alerted to you yet, but saw something in the shadows they want to investigate
- Monster was alerted but lost sight of you, and now wants to hunt in the area they last saw you

You'll have to add a field to store the monster's alertness level and make it behave differently at each level. One idea to try out is to create two different sequences of walking functions for e.g. patrol and investigating, which can share the same animation frames but pair them with different AI functions.

Another thing to add to the monster is a vector field to store where they think the player is - crucial to keep this updated when they can see the player (including when they see what they suspect to be the player but aren't sure) and equally important to stop updating it when they can't see you.

Once you have this, the refinements can pile on top. Maybe you give some monsters higher intelligence and have them anticipate where you were heading when they're in hunting mode - offset the last-seen location a short distance in the direction you were moving. Perhaps let even let them take another random guess if they arrive and see you aren't where they expected. 

I'm working on the stealth system as detailed above, and I'm making steady progress. One thing I'm not sure how to do is use the getlight command from Darkplaces in/as a function. Once I get that part figured out, the rest should be smooth sailing. Any and all help is greatly appreciated.

Also, what sort of light values does getlight output, RBG or just a general light level? 

Latter question is a quick answer, it's return a vector with RGB components - you can add them together to get an overall light level.

Basic way to access the function
1) Download the DarkPlaces mod from
https://icculus.org/twilight/darkplaces/download.html
2) Look at the QC that comes with the mod and find dpextensions.qc
3) Open dpextensions.qc and search for the code that defines getlight
4) Copy that code and add it to the bottom of your mod's defs.qc file.

You can repeat steps 3 and 4 for each DP extension you would like to use. Or if you prefer the kitchen sink approach, just add the entire dpextensions.qc file to your mod.