Chaos:
#51 posted by metlslime on 2009/08/04 09:47:20
If you made a q3 map with 7000 structural brushes, it might take a while. Detail brushes help a lot.
How About An Editor With On-the-fly Compile In Other Thread(s)?
#52 posted by bear on 2009/08/04 11:27:55
Should be lots of idle cpu-time while mapping although the problem is of course working with a changing data set and figuring out how to recompute as little as possible when the geometry changes...
Necros, Ricky, William
#53 posted by negke on 2009/08/04 12:51:40
That's what I do basically. Kept a copy of the bjp state file and resumed the process with Wvis. Both tools can be used interchangably. We'll see how it turns out in the end, though. I hope that c_chains discrepancy doesn't have any severe consequences.
#54 posted by JneeraZ on 2009/08/04 15:53:28
metlslime
I think having level designers place hand created portal sheets would be interesting. Have a texture called "PORTAL" or something that gets ignored during QBSP and LIGHT but is used to define the portal planes during VIS? I dunno. I don't know nearly enough about the internals of VIS to do this but it IS interesting.
#55 posted by JneeraZ on 2009/08/04 18:24:12
Not to make anyone cry, but I just got a new machine at work. 24GB of RAM and 16 cores. Good god...
But Does The New Eight Cores Make Any Difference?
#56 posted by bear on 2009/08/04 18:35:03
I guess you can always make everyone jealous by compiling three maps at the same time.
#57 posted by JneeraZ on 2009/08/04 19:42:42
Well, it's for work, so yeah it'll make a difference. For Quake VIS'ing, probably not.
16 Cores?
#58 posted by necros on 2009/08/04 19:54:28
you should sell your vising capabilities. :P
What Is It Clocked At?
#59 posted by RickyT33 on 2009/08/04 21:04:31
I mean 4.5Ghz isnt uncommon AFAIK, for your Core 2's and i7's and the new phenoms. I wonder how long it would take to compile CDA, probably like a day or something.........
#60 posted by JneeraZ on 2009/08/04 21:14:16
Intel Xeon
X5560 @ 2.80GHz
2 Processors
So 8 cores per processor...
Hyperthreading
#61 posted by inertia on 2009/08/04 21:41:45
The intercoils indicate that your CPU has 4 real cores per processor, which with hyperthreading gives you a perceived 4 more. So, it's not quite 16 cores, maybe approximately 12, if you want to talk about it like that. I'm still jealous, though.
#62 posted by grahf on 2009/08/05 07:12:43
going further off topic, but there will be 6-core Xeons soon (so 12 threads per core).
And about the hand-placed portals, isn't that what hint brushes do?
Relatedly, as I like to point out every time these topics come up, the codebase exists for adding hint, skip, and detail in Q1:
http://quake.chaoticbox.com/downloads/equakeutils.sit
Grahf:
#63 posted by metlslime on 2009/08/05 08:15:51
i think i've seen you mention it before, but i simply don't believe that detail is possible in the quake1 bsp format. Maybe i should actually follow your link this time and see for myself :)
Also...
#64 posted by metlslime on 2009/08/05 08:18:32
hint brushes supposedly indicate preferred planes to do major cuts in the BSP tree, from what i understand. And "skip" in quake2 is used on the other faces of a hint brush so all 6 sides don't have to be "hint".
"Skip" in quake1 has come to mean the equivalent of "nodraw but structural and casts shadows" in quake3 terminology.
Also: how do i open a .sit file in windows?
Well...
#65 posted by metlslime on 2009/08/05 08:21:03
let me back up and say that perhaps detail brushes would be possible if you were willing to totally break software engines.
Stuff
#66 posted by grahf on 2009/08/05 09:53:29
.sit is Stuffit, a totally lame and outdated mac-only compression format. Here, this should be easier:
http://www.quaketastic.com/upload/files/tools/mac/misc/eUtils_r10.zip
The codebase is basically the Quest compilers, but enhanced and fixed up. Like, the original Quest utils had their own pointfile format, but pOx reverted it to standard quake style. Some higher limits too (but nowhere near bjp-tools' level), better handling of complex geometry, and probably some other features I am forgetting right now.
Truth time: the detail brushes in the eUtils don't work exactly like Q3 engine detail. I have done tests and they split the BSP tree exactly the same as regular brushes. However, and this is the important part for this discussion, they are ignored by VIS.
Both qbsp and vis have to be aware of the detail brushes, but they definitely work in all engines. I'm no programmer, but I had a look at the relevant parts of the source, and it looked to me like the main change was a new contents type (sky, lava, water, slime...and detail), and all brushes in am_detail entities were given that type. Then vis knows to ignore them, I guess.
John Carmack On Parallelization
#67 posted by inertia on 2009/08/06 08:08:54
There are opportunities for network level parallelization with some recoding. Qbsp uses fork level parallelization to utilize up to three processors, and it would be trivial to change to allow it to be run on three computers. Light is embarrassingly parallel, and can be divided arbitrarily. Vis is highly parallel, but it takes some advantage of order of completion in
the serial case, so scaling is not quite linear.
http://www.gamers.org/dEngine/quake/Qbsp/
I had to correct his maddening spelling mistakes.
A Word Of Caution
#68 posted by negke on 2009/11/05 14:25:49
As it turns out, this multithreaded vis isn't safe. The problem lies with the autosave feature:
Each thread processes one portal at a time, that's why the fullvis is sped up on the whole the more threads a machine has. However, as we know, not all portals take the same amount of time - especially towards the end, their processing time increases greatly, some of them can even take days. The autosave option is geared for single-threaded processing, so it doesn't take into account which portals are done and which are still being processed on the other threads when saving the state. Therefore, pausing the process (ctrl+c) will leave these portals (that were being processed but couldn't be finished yet) open, so to speak. This will eventually lead to a situation where vis reaches the end of the processing line, but can't wrap up the entire thing properly because there're still those 'undone' portals in-between. It will then stop with a "portal not done" warning and the state file will be corrupted.
This means the multithreaded vis tool can only be used (more or less) safely if one leaves the process run from start to finish. Maps that take longer and thus have to rely on the autosave function should not be fullvised with this tool.
#69 posted by JneeraZ on 2009/11/05 15:30:35
Oh. OK, that explains why I wasn't able to VIS your map for you then (my resumes kept failing). Damn.
#70 posted by JneeraZ on 2009/11/05 15:33:24
Well, the source code is on Quaketastic so I'll see about making some time to look into it. It shouldn't be too difficult.
The solution, probably, is to turn off the threading when the autosave is ready to go, wait until the last portal finishes, save, and then start them all up again. We'll see...
Thread Saving
#71 posted by Tuna on 2010/01/11 14:00:54
Ahoi,
aroused by Spirit's coding bounties over at quaddicted.com I thought I might take a look this one.
Honestly I haven't really dived into the code much. However judging by the comments I thought I try some blind thread synchronization upon saving.
Here is my first appalling version:
http://user.cs.tu-berlin.de/~tuna/WVis_thread_test.zip
I'm not even sure how to trigger the issue described. I tried ctrl-c-ing and continuing with the unmodified version and I didn't encounter the "warning" message..
If you have precise instructions how to make it fail please let me know.
Also feedback if the issue has changed or improved with my version is highly appreciated.
#72 posted by JneeraZ on 2010/01/11 14:48:42
Hooray! Thanks man. I took a few shots at it but always came up short.
Awesome
#73 posted by Spirit on 2010/01/11 16:52:38
Could somebody explain how to properly test this?
#74 posted by JneeraZ on 2010/01/11 16:53:36
I think just get a map that takes a long time to VIS and stop/start it repeatedly.
#75 posted by Spirit on 2010/01/11 17:40:34
I tried gmsp3tw and it worked very well.
|