I'm certainly going to use surface flags. Q1BSP doesn't support these, but scrolling textures would appear broken in vanilla engines anyway.

Surface flags will also allow for physics scrolling (so conveyor belts will really work), and for entity interaction (bind it to an entity field, so the entity can toggle the scrolling on/off and change direction and speed at will).

And the "{" for alphamasked textures is dumb.

I already have plans for a new map format. Things will eventually happen, and I'm going to do it carefully.

Editor support will also be considered. 

the unofficial map specs are just that - unofficial.

I'm far too lazy to actually test it, but as far as I'm aware the following is a valid-but-sadistic .map file as supported by QBSP (apart from the bit in angle-brackets that denotes the other planes, because I'm too lazy to come up with witty wtfs for those too):

"{" // }
classname worldspawn
sounds 2 "{""(""what" the hell ")"( 1 shoddy parser ) ( hello "2, or 'to' if you'd prefer" you ")" "tex with space"naught//(shift_s) slash(shift_t) there(rotate) 1t(scale_s) 2oo(scale_t) //and a real comment.
<OTHER PLANES>
"}" "wad" "c:\this\is\borked\"
"message" "and somehow\nthat was a newline, at least for the engine" "}""{" classname info_player_start }

thank god nothing actually uses all those quirks... at least not all at the same time...

the various bits of punctuation (especially noteworthy with the ( and ) brackets) MUST be their own token, which is why they normally have extra spaces.
the format is simple and stupid. a token is just a token, once parsed it is irrelevant whether it originally had quotes.
A token is either quoted, or its all chars until the next whitespace where 'whitespace' is any char above 32. Because this check is agained signed chars, any utf-8 chars or non-ascii ansi chars will be considered whitespace but can be quoted to pass them through unmodified.
If a token did have quotes, then whitespace is not needed after the token to determine the end, as this is inferred from the fact that the quotation ended. Additionally, there is no special handling for punctuation (like any kind of brackets).
New lines are disallowed in certain places, but entirely optional otherwise (with the obvious exception at the end of a // comment).
Double-slash/single-line/c++ comments are only recognised as comments when they are at the start of a token. If they directly follow an unquoted token without whitespace beforehand then the token will contain the slashes rather than the parser seeing the comment. Because they are terminated by a newline, they cannot be used anywhere that a newline cannot appear.

*SOME* editors write quotes around texture names (I'm fairly sure I've seen this happen). These editors do not have problems with spaces in texture names, and also avoid the {foo issue when importing in certain other editors (although nothing can be done about those other editors saving and then failing to re-load the file they just wrote themselves, other than fixing the editor).

QuakeEd uses the exact same helper functions as QBSP. It doesn't write quotes itself (and thus will bug out if the file is re-saved thanks to the spaces in the texture name), but it should parse it all just fine (I don't have a machine with actual NEXTSTEP on it).

If you're worried about compatibility with other unofficial tools then just write the quotes only when it has chars that are not azAZ09_*+
If an editor sees "{foo" as literally containing the quotes then at least it'll save it out correctly too, even if it cannot display the texture properly in the editor.
But whatever you do, don't write out a file that the same bit of software cannot then re-load!

So yeah, that's what I mean by 'using quotes properly' - using them in order to stop your importer+qbsp from reading the file wrongly. 

"*SOME* editors write quotes around texture names (I'm fairly sure I've seen this happen)."

I'd like to see an example of that. I've never come across that in all my years of Quake tinkering. 

how does one make a new map format, with everything that modern mappers want without losing the q1 feel?

If we're going to lose the feel of the q1 experience, then why not just use an engine that supports and migrate our mapping HL(2) / D3 / Q4 formats? 

Depends what you mean by "q1 feel". BSP2 already lead to many not very Q1-ish maps. 

Bsp 2 with quake 2 style surface properties would be a dream. Phone shading!!!!! 

The same questions also applies to fence textures: they're an specific kind of data that needs engine support, otherwise they look broken in the game.

And faithfulness to the Q1 feel will depend entirely on the mapper. A new map format isn't needed to lose the original feel of the game, there are several custom Q1 maps that does this already.

It's a matter of using new features where needed, to do things properly instead of using hacks. Alphamasked textures can be faked with sprites, and scrolling textures can be faked with moving entities, but they're much simpler to create when using a map format that supports them natively.

Part of my my purpose is to make the task of map creation easier, to replace hacks with proper implementations.

Each eyecandy I implement is directed at eliminating a visual artifact that Id Software would certainly have eliminated if they had had more development time. Sky horizon fading and hybrid skyboxes are an example � and they are more faithful to the Q1 feel than static skyboxes.

I make sure to follow a minimalistic "if it's not broken, don't fix it" approach. Which is why I'm not going to support replacement textures, parallax mapping, bump mapping and other features that "prettyfies" the game without allowing for more gameplay possibilities or for easier creation of assets.

And all that is because as an engine coder, using engines coded by others would be akin to a mapper using maps bought from an asset store instead of creating his own. I have my own ideas for a game engine, and I want to bring them to life.

My minimalistic improvement approach is a way of ensuring I can do that even if I'm completely on my own. If others would like to follow it, that's a bonus.

I never had much luck with teamwork, so I must make sure I can do everything myself. 

It is clear that you've put a lot of thought into this. I hope you can get it through to completion.

On a side note, why re-write an engine rather than contributing to an existing engine? Your comment seems like you'll be doing it for philosophical rather than practical reasons. It sort of seems like you'll be re-inventing the wheel if that is the case.

Unless you want a bare-bones engine. In which case, yeah, go nuts. 

And faithfulness to the Q1 feel will depend entirely on the mapper.

These are wise words. 

Because I'm really not sure whether other editors can read them back in properly. One option would be to have TB write with quotes if the input map file had them in the first place, and have some option to turn this behavior on and off.

Demonstrating how bad the QBSP parser is doesn't really prove anything other than "QBSP doesn't conform to the agreed upon standard". And from what I have seen so far, the unofficial Quake specs appear to describe the least common denominator that all tools that I have seen so far understand.

I do agree though that having texture names in quotes would be the best option. I just fear for compatibility, and it's quite important as a lot of people seem to use more than one level editing tool. 

The 'agreed upon standard' also doesn't include floating point coords, nor Hammer's texture planes stuff.
Writing out .map files that the same editor cannot later read in is always going to be broken however you look at it.

By all means only use quotes when they're actually needed (yay for smaller file sizes), but do make sure they're used when they ARE needed, because writing out a file that can only be read by your own program is muuuuch better than writing out a file that can be read by nothing.

The times when you NEED quotes are:
empty texture string.
texture name contains spaces / other whitespace.
texture name contains chars that an unofficial parser may consider to be punctuation.
texture name contains non-ascii chars

Also, I wouldn't call QBSP's parser bad, I'd much rather call it simple. And by simple, I mean that its tokenizer is consistent and predictable.
That said, showing exactly what weirdness you can throw at it is fun, as well as a possible way to hide other optional extensions (without needing the qbsp to write out multiple copies of the same texture). 

"why re-write an engine rather than contributing to an existing engine? [�] Unless you want a bare-bones engine. In which case, yeah, go nuts."

I want to learn how the whole engine works, and the best way to do so is by rewriting it.

I will never accept that I can't fully learn how it works. If Carmack created it all almost alone, then it's humanly possible.

All my life I've been taught that I should give up and get an uninspired job where I do nothing but follow orders. I refuse to not study and I refuse to not be creative. I may only switch to another engine when I finish learning this one.

I refuse to give up. 

 

Carmack is kind of an exceptional individual, but more power to you. Doing it yourself is very rewarding. 

If Carmack created it all almost alone, then it's humanly possible.

Way to go assuming Carmack is human. 

For a hobby project. Doesn't mean you can't get a job that pays the bills though. 

Writing a game engine from scratch - nothing stops you from trying to get it greenlit. 

that's awesome.

Do us a favor, if you make a new map format, document and comment the hell out of your code so that others can implement it in their own engine too :) 

Carmack also wasn't nearly alone - Michael Abrash was instrumental. 

SleepwalkR: I do have a full time job. The payment is low, but it's almost stress-free and gives me free time to study.

ijed: There are some plans for a commercial release using it.

Shamblernaut: Sure!

Lunaran: Yes, and I've already beaten some of his contributions. My particle drawing code is faster than his x86 ASM code, and my Z-buffer drawing code in C is as fast as his x86 ASM code. I like the challenge of optimizing the software renderer. 

There's a lot more to a game engine than the renderer. Don't pat yourself on the back too hard JUST yet. :) 

 

Great stuff! I'd recommend posting over at the vogons, there are some super smart and retro affine people there. 

keep a hold on me! 

That thing looks great.

Also, somebody should tell the second Orbb that his RL is backwards.