an unvised map would not run quicker -- but it might run at nearly the same speed with the right engine code. The only way unvised would be quicker is if the engine detected this case and used different code to render it compared to a vised map.

But the main point still stands, polygons are not the bottleneck for quake on modern hardware, it's things like draw calls, lightmap uploads, and (perhaps) geometry uploads.

I believe the ideal quake renderer for modern hardware would reduce draw calls down to 1 per texture, put all geometry in vertex buffers (potentially ignoring vis to make this work,) and do lightstyle calculations in hardware. 

It would definitely be a revolution to leave VIS behind ... imagine the iteration times if level designers only had to QBSP and LIGHT. 

Even if you stick the entire map into a single vbo and draw it with a single draw call (which is possible, except for sky+water), you will still suffer from overdraw.
those individual props with more polys than entire quake maps do not have 15 different rooms overlapping each other.

it might be interesting to throw the entire map at the screen in a single draw call, both with and without an early z pass... 

Quakespasm has upgraded world and mdl rendering paths vs Fitzquake, if I disable these with the "-novbo" option my framerate goes from 60 -> high 20s in that map.

However, I think QS would still run faster if this map were vised - it could skip drawing entities, and skip draw calls for the world (there's a draw call for each batch of polys that share a texture and a lightmap.)

IMO all maps should still be vised, if someone did develop an engine that was faster ignoring vis data, that engine could just ignore the visdata.

What would be interesting, I think, is an upgraded / state of the art vis tool that could give less exact results but finish in a reasonable time. I wonder if there's some off the shelf, open source vis algorithm that could be used. I tried to vis the jam version of jam6_ericwtronyn but the progress indicator got stuck at the end after 2 weeks, it didn't finish in the following week so I killed it.
Vis isn't going to do a ton on this map, but if it could just separate the cave and outdoor sections that would help.

it might be interesting to throw the entire map at the screen in a single draw call, both with and without an early z pass...
I'm curious how you'd handle the texture switches when drawing the world in 1 call, is there a better way than a texture atlas? When I read about atlases, they sounded like a big pain because you have to partially tile the textures in the atlas so texture filtering doesn't read into adjacent textures. 

Thanks, some cool info there :)

I tried to vis the jam version of jam6_ericwtronyn but the progress indicator got stuck at the end after 2 weeks

o_O

To what extent were detail brushes used in that map? I figured that kind of solved the vis time problem, even for crazy bonkers tronyn stuff. 

It made good use of func_detail too, had something like 7k clusters, 24k leafs - so vis only had to be computed for the 7k clusters.

Could be it's just the map layout being open. For comparison I've tested ijed's telefragged.bsp and it full vises in about a minute! 

I just realised the map source is provided so I fired it up in radiant, pressed Alt-2 to only show the world brushes, and...yes I see the problem :) 

Yes, overdraw is the thing I forgot... Still need a good solution for that. 

I'm having some trouble understanding the technical bits - so I assume drawing X triangles that are all stacked behind each other (lots of overdraw) is slower than drawing the same number of triangles spread out on a sheet but still all in view (no overdraw) ? 

Carmack himself stated a while ago, you can ultimately throw brute force at every problem once the hardware is fast enough. Vis is a clever solution to a problem, precalculate instead of doing things during runtime.

Afaik the main problem with overdraw is related to shading, ie too many fragment-shader invocations per pixel ( scenes with many overlapping particles tend to show this, makes older GPUs rev up nicely ), but shouldn't early-Z take care of this for opaque surfaces ?

But it probably still overdraws during the z-phase as you end up trying to draw a lot of unnecessary polygons, so it depends on the player hardware.

Any software engine people want to chime in ? I guess it might be quite bad for them at least. 

Everything seems to run well for me, I guess the intel GPUs are finally good with Broadwell :-) 

"but shouldn't early-Z take care of this for opaque surfaces "

It does. If the depth buffer rejects a pixel, you won't eat the rendering.

Overdraw is really only a problem on systems without depth buffers or stacks of non-opaque surfaces - like particle systems or glass. 

The main reason to always use vis, even if an engine is faster by disregarding vis entirely, is serverside culling and networking bandwidth even if the client totally disregards it.
culling realtime lights via vis is also very useful, although I suppose you could also use oclusion queries for that.

@ericw
use GL_ARB_bindless_texture
atlasing or texture arrays are also an option, but more fiddly (but also more likely to be supported by hardware).
water+skys could be done with subroutines.

@Kinn
overdraw is when you draw the same pixel multiple times. the earlier times become redundant and are essentually become a waste of memory bandwidth.
typically, graphics cards utilize an 'early z' optimisation which massively reduces the cost of overdraw, so if you draw the only world's depth first, then draw it normally (with depthfunc gl_equal), then you're not wasting time calculating the colours+textures of geometry which will never be seen.
really the advanttage depends on how expensive your fragment shaders are (including the cost of texture lookups+bandwidth).

Quake's software renderer had a zero-overdraw strategy. vanilla glquake draws triangles as they come from the bsp tree (nearest first). all modern glquake ports instead batch by texture, which can result in excess overdraw.
it'd be nice to return to a single-draw-call nearest-first renderer. best of both worlds - assuming your hardware+drivers are recent enough... 

it'd be nice to return to a single-draw-call nearest-first renderer

I don't know much about current hardware capabilities, but is this even possible, given that there are typically dozens to 100 textures in a bsp, plus a bunch of lightmaps that, even with atlassing, probably can't fit in a single texture? Are there enough texture units on modern cards to accommodate all of this? 

GL_ARB_bindless_texture
no binding = no texture unit limit.
pass the texture via a vertex attribute.

GL_ARB_shader_subroutine
efficient branching, based upon vertex attributes.

both together and you have some serious dependancies on modern hardware... but should be able to draw the entire world in a single draw call - so long as your graphics card has enough memory (probably not an issue with vanilla textures, but will undoubtably be an issue with replacements).

I've not used either, so while I'm sure its possible, I'm not sure on the actual practicalities, but hey... 

Post #38, quoting here (from mh)

I've been thinking more about the idea I mentioned above (using occlusion queries) and it seems to me that something may be possible by using a combination of the world bounds and reducing to the same 16-texel scale as lightmaps.

So what I'm thinking here is to divide the world into a grid of 16x16x16 boxes, then for each box do a Mod_PointInLeaf on the center. If it's in solid don't bother, otherwise run a 6-view-draw with occlusion queries and merge the resulting leafs into visibility for this leaf (which will have been cleared to nothing visible at initialization).

Obviously there are probably edge cases that I haven't fully thought through, but overall this is an interesting enough approach that I might even code something up.

Ignoring the realtime thing (the subject of this thread) - if an offline vis tool was developed that used such a GPU-based occlusion approach to create the vis data - would this theoretically lead to higher quality visdata than the current portal-based method?

It would certainly allow for much more open maps, surely? 

Kinn, it does sound like a tempting/cool idea.
I'm not sure if the vis quality difference would be noticeable.

The biggest advantage, I think, would be vis time being proportional only to the interior volume of the map. Also func_detail would be unnecessary.

The disadvantage is you'd be moving to a system that could, in corner cases, draw less than it should. I'm thinking a hole in the wall where you have to stand just in the right place to see through, and the sample points used by vis never line up with that spot. Probably a 16x16x16 grid would be fine enough that it'd never happen in practice.

The other concern is, how fast will it be? An 8192x8192x8192 box is the worst case. That's 512^3 vis sample points using a 16x16x16 grid, and if each vis sample point can be computed in 1ms (rendering all 6 views and getting the occlusion query results) that gives you 37 hours. 

The problem with open maps is not how vis tests visibility, it's how the world it's testing is split up into a tree. If the splits in the tree don't correspond to occlude-able pockets of geometry, it doesn't matter what method you use to determine which ones can see which ones, you're always going to be 'seeing' geometry you think you shouldn't.

The solution you're looking for is careful construction and planning of your big open map, and hint brushes. 

Hint the lot and trust the player has 256 allocated... 

The biggest advantage, I think, would be vis time being proportional only to the interior volume of the map. Also func_detail would be unnecessary.

vis time was the main thing I was thinking of, and also that vis time would scale linearly with map size meaning that complex "vis breaker" maps would no longer be an issue. 

I'd need to do some research to understand more how hint brushes work in order to apply them to a big open map.

Is there a way to visualise portals in quakespasm or another engine? 

 

The portalization is discarded following the vis process; all that's stored in the BSP file is a list of which leafs are potentially visible from each leaf. 

I have found Darkplaces has r_drawportals 1 command. So what is that visualising? 

 

load up any map in darkplaces, and do r_drawportals 1. Doesn't need a prt file. 

Intersections between leafs or something? 

IIRC DarkPlaces does it's own realtime portalization. That would be what it's visualizing. 

I'm afraid I'm gonna be asking some stupid questions for a while.

The only other portal-based rendering I'm familiar with is Doom 3 and the portalling there is hand-placed and is coarser than quake I think? (it's done per room, more or less).

Is the portalling in quake on a per-leaf basis? i.e. (ignoring detail brushes for now), are visportals created for each bsp leaf? 

There's no such thing as a stupid question; particularly when it comes to something like this, where the knowledge actually isn't anywhere that's publicly accessible.

Anyway - don't know. Somebody else is going to have to chime in with that one; this part of tools work makes my head hurt. 

Is the portalling in quake on a per-leaf basis? i.e. (ignoring detail brushes for now), are visportals created for each bsp leaf?
afaik that's correct, same with what Spirit said, "Intersections between leafs". So, they're super fine-grained. With "r_drawportals 1" in DP you are seeing the portals, but it's also a visualization of the leafs at the same time.

Bringing detail brushes into the picture, the info on which faces/leaves were detail is not stored in the bsp file, so DP's "r_drawportals 1" will be showing portals as if all detail was converted to world first. 

Thanks guyz, so...if I compiled the map without detail brushes (I think the "jury-rigged" bjp compiler lets me do that)...then viewed it in DP with "r_drawportals 1", can I trust that i'd be seeing the actual portals that vis.exe will be using? 

you can load the prt file q3radiant aswell, there is a plugin for that.
Quark also loads and displays the portals of a map.
Comparing those may help you. Idk what you are up to tho. 

Thanks for info

Idk what you are up to tho.

Just wanted some help with visualising what happens when I dick around with hint brushes, because right now I don't have a scooby doo of how I'd go about hint-brushing a massive open map :/ 

i feel you, i have always got advice from a friend to use hint brushes, if i want to optimize vising times/quality.

I never bothered to, actually i am playing around with those atm. Hint brushes can make a big difference, thats all i know by now. 

from what I understand, hint faces force qbsp to split along those planes, before using any other faces as split places. just imagine doing a 3-point clipping on the whole map, putting the clip points on the surface of the hint face. I think qbsp picks the order in which to do these splits, but it will always do the hint faces first.

Also how do you actually use them? I assume a brush with one face textured "hint" and the others "skip", and I assume it gets deleted from the final bsp, otherwise they would be annoying. 

Yep, using "wedge brushes" for hint seems to be the way to go. Hint texture for the desired split, hintskip on the sides that can be discarded.

I have produced some serious HOMs with that already, discarding more protals than it should i assume.
It is tricky, and the tuts that exists for using hint/hintskip brushes all rely upon HL2 compilers.

Hmm.. 

Paraphrasing the best explanation I can recall of hint brush functionality:

Imagine a large room with a high ceiling. It is divided by a wall taller than the player but which only reaches halfway to the ceiling. What you'd want as a level designer is for the space on one side of the wall to be invisible when you're on the other side.

QBSP on its own will take the vertical faces of the divider wall and split the world vertically, leading to two tall leaves corresponding to the two sides of the wall and one narrow one that's on top of the wall. The entire map is visible from everywhere this way, because if you're standing in the bottom of one of the 'wells' you can see the top of the other well's full-height leaf, so the whole leaf is visible all the way to the floor.

If you place a wide horizontal hint plane so that it lines up with the top surface of the divider wall, it splits the room horizontally instead. Now the two leaves on either side of the wall do not extend higher than the wall itself, so the wall fully occludes them from each other until the viewpoint passes the top of the wall, and the entire top half of the room is now one big leaf, so when you're on top of the wall the inside of both wells is still drawn as you'd expect.

Hinting an outside area is really tricky if you've got trisouped rolling terrain, because you've got to find whatever the equivalent of the divider walls is and it's often not as obvious or convenient as a wall. Having this in mind when you make your rolling terrain in the first place makes your life a lot easier.

It's worth remembering that if your big open space is suitably split by big un-see-overable divider walls, you could just have a wall of sky texture stick up out of the top of the divider and make your life easier that way instead. If that's not desirable - maybe the player will at some point be high enough to see over (or will stand on top of) said dividers - then from that vantage point the whole outside area is going to draw anyway, and that is what will set the minimum performance bar for your level. If a given computer can handle that view playably, it doesn't matter if drawing the entire outside area at other times is in principle 'more than necessary.' It's going to become necessary when the player makes his way up there. 

especially Lunaran with that super useful post - essentially what you described is what would in theory solve my map - imagine a town where the player is mainly low with most of the rest of the town occluded by walls and other buildings. Tall spires and towers rise above the walls though and should be visible from a distance.

If I can fudge things with hinting so that a fair amount of the structures below the walls are hidden, then that will probably do the trick.

Of course there are times when the player gets a bit higher and sees a bit more, but at least I'll be able to say I made a bit of an effort :} 

Best of all is that this hint plane can be from a 16x16x16 brush. 

Do it hierarchically, so that there's a second tier of taller buildings that forms a higher barrier for the rooftops of the tier-1 buildings.

And use lots of arched passages and tunnel alleys.

because those are cool. 

Lunaran: it has all those things and more :}

Question:

In the creation of the bsp tree, do hint planes *always* take priority over structural brush planes when building the tree?

I.e. if I make a map with one obscure little hint plane somewhere, does that form the very first split in the entire bsp tree? I'm guessing no, as that could lead to very unbalanced trees? 

Looking through the bsp.exe code it appears that a structural plane won't be chosen for a bsp split if that plane splits a hint face, but otherwise the splitting planes are chosen that produce the least face splits (no matter whether they are hint planes or structural planes).

So...

Best of all is that this hint plane can be from a 16x16x16 brush.

The size of the hint face should matter though surely? If I have read the code correctly, a whopping great hint face vs a small hint face (but both faces on the same plane) should cause the bsp tree to be split in a different way. 

Hint brushes get along really bad with liquids.
And by really bad I mean not at all.

The latest version of the Good Compilers doesn't throw a mixed contents error anymore at least, but as I remember it still clips away liquid surfaces inside hint brushes and gets really confused about leaf contents in general.