Preach, that's a cool idea of tacking the lower 8 bits of each coordinate on at the end of the file. Only concern I have is code to find that refinement block could be ugly; you'd have to parse all of the structures in the mdl keeping track of the largest byte offset from the start, then if that offset + 1 is still within bounds, try to parse it as the refinement data.

Perhaps a cleaner way would be just leave the mdl file alone and dump the refinement data into another file (.bsp + .lit style), so you'd have shambler.mdl + shambler.16b (or whatever). Whenever loading a .mdl, the engine could check for a .16b file, and if present, use it to enhance the precision.

It feels a little wrong to create yet another model format, but the beauty of this would be it would work on all quake engines from the beginning, just without the enhanced precision. 

I dig that, sounds like a nice way of keeping the fallback .mdls "pure".

best of both worlds! 

it would also avoid potential problems if the model was loaded into QME. you wouldn't have to worry about the old editor clobbering the new data. 

Actually, qme already supports 16bit mdls. If you use the 'save' option instead of the export option, it'll write out an mdl format with a load of qme-specific data on the end of the file.
figure out the extensions and get some engine authors to support it, as well as a blender exporter.

regarding iqm, there's no reason that an engine couldn't just transform verts on load and then interpolate from there the same as they do with mdl (or just skip interpolation completely, if you really like vanilla quake).
Even geforce2 drivers will emulate support for vertex shaders.

iqm omits skingroups, so cannot be used as a pure superset (ignoring animation differences). luckily these are rarely used, and where they are used, they're often better replaced with a shader instead anyway. assuming your engine supports shaders...

mdl has other issues, like texture coord positions precise only to a texel, a single surface, implied flood fill requirements, onseam stuff is ugly as hell (and unsupported in any tool but qme (which breaks when onseam stuff isn't used)), palette limitations.
but worst, many of the limitations of mdl are not with the format itself, but with the engine.
by the time you have 1000 verts, all of those glVertex calls will be a significant time waster.
frankly, if you're going to rewrite enough of the engine to remove those limitations, you might as well add support for md3 too, which is typically easier to export to anyway.
ultimately, I don't see any point in mdl extensions unless qme can write them. backwards compatiblity is pointless if its going to just look terrible anyway. 

really? i wouldn't really let QME be a deciding factor at all. it's shit...
Best results with models always involves avoiding the use of QME in the workflow. 

backwards compatiblity is pointless if its going to just look terrible anyway

I tend to agree. The only reason to use 16bit or more accuracy is for large models with a large range of movement that *would* look terrible in 8-bit. If most users are going to see an awful mangled .mdl version of it anyway, then I would say the idea has failed. 

things like the vermis come to mind. or monsters that move around their model space a lot. 

It could be that using md3 makes more sense than a MDL + new 16bit refinement format.

I could have a look at how difficult a minimal md3 implementation for fitzquake/qs/glquake would be. 

would be just awesome. 

Pwease ^_^ 

maybe, except with preach's idea, a 16 bit model would be fully compatible with even winquake/glquake. 

If you're gonna have a separate file, you may as well make it a MD3... 

True dat. Add MD3 support to the engine and boo-ya... 

If nothing else, md3 support saves creating a new standard. :P
Plus its already supported by a load of engines.


Software renderers might get something out of: http://sourceforge.net/p/fteqw/code/2002/tree/trunk/engine/sw/sw_model.c#l2908 but its too long ago for me to really remember much about it.

I'm tempted to port fte's mdl/md2/md3/iqm/zym/dpm/psk/md5 loader+renderer to glquake, but I doubt any engines would actually use any resulting patch which kinda makes it pointless, plus I'm feeling lazy. 

I'm tempted to port fte's mdl/md2/md3/iqm/zym/dpm/psk/md5 loader+renderer to glquake, but I doubt any engines would actually use any resulting patch which kinda makes it pointless

I touched on this above, but this is actually a great example of why certain features don't make it to widespread adoption.

You're right in that nobody would use the resulting patch, and the reason why is that the resulting can only be overly complex. 8 model formats and a renderer (I assume it's a single common renderer for all 8) is just too much. It touches too many parts of the code, and adoption would involve too much surgery, particularly if an engine author has already done work on these parts of the code themsleves.

In order to drive adoption features need to be easy to adopt. By way of contrast, an MD2 loader that brutalizes it into the same in-memory format as MDL (and they're not too different so it wouldn't be too difficult) would be just a simple matter of a drop-'n'-go loader, some simple struct modifications and another line or two here and there. That's the kind of feature that gets adopted. 

1) but this is actually a great example of why certain features don't make it to widespread adoption

2) particularly if an engine author has already done work on these parts of the code themselves

In very recent times, I have become increasingly convinced that many programming languages --- especially C -- are broke by design.

At first, this sounds crazy. Because C is incredibly powerful and amazingly well constructed. And the assertion that C is "bad" means virtually every programming language is bad becomes most of them mimic the behaviors.

But I am convinced C is a terrible programming language (and almost all the ones we use):

Let's assume there is problem X and solution Y.

1. An "ideal programming language" (which probably does not exist at this time), the abstract solution does not get reduced to code. The abstract solution becomes a very specific implementation that drifts away from the abstract solution.

This is a characteristic of most programming languages -- and it is terrible.

2. C offers too many ways of doing things. Many of these ways are ridiculous. And coding towards a specific implementation vs. abstract solution is actually rewarded by the language. So you get stuff like this:

g = (byte)((((k & 0x03) << 3) + ((j & 0xE0) >> 5)) << 3);

3. A good example of C encouraging drifting away from the abstract solution is short-circuit evaluation. In a specific implementation, this might make sense but backing up a bit would deny a highly intelligent or advanced compiler from deriving intent. The order of logic in a statement using short-circuit evaluation doesn't obscure the logic --- it actually removes the information permanently -- so a highly advanced compiler could never know if the order was important or the programmers best guess of the right order for speed.

As a result the language doesn't offer:
1) Collective works
2) Additive gain
3) People spend their time reinventing the wheel.

And because the language offers so many coding styles and ways of doing things, you could draw an entire org chart of different styles.

This results in the kind of failures seen in Quake, where engines struggle to adopt features that have been common place in other engines for a decade --- because the abstract solution never gets reduced to code, but an implementation-specific solution.

[Of course, if a language is ever written that does this right, it will have to be written in C ;-)] 

You've more or less restated the problem that OOP was meant to solve.

The nirvana here was supposed to be that you could have components expressed as reusable objects and then all that you need to know is the public interface of that object and you can link to it and use it without having to worry about it's internal implementation.

Likewise if you're writing such a component you specify and document the public interface and the same magic happens for anyone using it.

At a sufficiently high level of abstraction using these objects becomes broadly analogous to putting Lego bricks together. Join a "model" object to a "renderer" object and you have support for a model format. Want to support another model format? Just drop in another object and if it understands the interface to the "renderer" (and vice-versa) you have the awesome.

True, the person writing such an object still has a lot more work to do, but that's a one-time-only job and then everyone gets to join the party.

Of course the end result we have today is somewhat different to this ideal world, but hey-ho, you can't win them all. 

http://www.cs.yale.edu/homes/perlis-alan/quotes.html -> " 93. When someone says "I want a programming language in which I need only say what I wish done," give him a lollipop. " 

And you've been tricked! There may be 99 gems of actual wisdom on that page, but you've fallen for decoy with that one.

If two programmers will code essentially the same solution, the programmers are NOT coding.

Rather the programmers are, in fact, acting as the compiler.

This is a waste of human time. 

I think the point is more that observing that this is a problem does not bring us any closer to a solution. Languages like Haskell are finally getting more widely used these days so maybe there's hope for the world (although I found Haskell pretty hard for just getting things done when I used it, that was 15 years ago now though...). 

The problem is this: A compiler is a transformation program that transforms one model to another. In the case of a C compiler, the input model is source code that conforms to the C spec, and the output is machine code that conforms to the spec of the targeted platform.

The input model is already an abstraction in that it abstracts (somewhat) from the details of the target platform. As you know, abstraction means omission of details, that is, every abstraction entails a (purposeful) loss of information. In the case of C, the missing information (platform details) can be added by the compiler because the information can be derived from the C spec and the platform spec.

What you want is even more abstraction, which again entails more loss of information. The question is, how can we get this information back during compilation? If you want an automated compiler from your more abstract input model, you need to store the missing information somewhere and add it back in during compilation. This is how toolkits work. For example, in a game toolkit like the latest Unreal Engine, you work on a much more abstract level, and the missing information (that is, how to render things, how to handle input, how to handle sound etc.) is added in when the game is assembled by the compiler.

Basically, the missing information is again stored in a platform specification. In this example, the platform specification is the specification of the engine, its toolchains, its file formats, and so on. Of course all of this abstraction comes at a price, and that is less flexibility. You are restricted in what you can do with the engine and how you can do it so that the "compiler" can add in the missing information.

But what you seem to want is having the ultimate flexibility AND a large degree of abstraction. You want to specify a very abstract solution and have the compiler figure out how to generate machine code from that. Again, the compiler needs to add the missing information, but where does it come from? If you formalize it somehow as a model, you must again limit the user / programmer in what she can do with your system.

And therein lies the problem: The more abstract your language, the more restricted it must be. Don't want do deal with memory representation? Use Java, but suffer the restrictions in performance and memory consumption. Don't want to specify types and such? Use Python/Ruby/whatever, but suffer the restrictions in performance and safety.

The challenge of abstraction is to balance it with flexibility, and apparently this is such a hard problem that the entire software engineering discipline has not been able to solve it (generally) and is nowhere near a solution, too. Some abstractions (languages) are "good" in this sense and others are not, but we don't precisely know why.

But! you say - can't the computer guess the missing information somehow? My brain can do it, so the computer should be able to, too! No, it cannot, because it's not creative and needs precise mathematical descriptions. Only our brain can do this.

But! what about Prolog and such? You don't even need to specify a solution, only a problem! Yes, but specifying complex problems so that a system like Prolog can derive a solution is (in general) much harder than inventing a solution in your head and coding that up, even if that's hard too. Logical programming is very useful for a specific set of problems, but it is more targeted at the scientific and engineering communities and are usually applied in AI and computer linguistics. You can't write a 3D game in Prolog (AFAIK), nor would you want to.

So, I for one don't believe that there will ever be a system that will write your code for you. Or if computers ever get "smart" in this way, I doubt we will be around to use them. Maybe our grandchildren will. 

make a trenchbroom for C code!.. 

Imho C is a necessary and best kludge to get the most of out hardware and hence is the underpinning of serious programming. C++ is an even uglier but still great kludge for the reasons sleepwalkr says.

But I for one *love* my main language. Wish/Tcl is a beautiful archaic thing with aspects of functional programming,and great interfaces to a huge (slightly ugly) graphical toolkit, and to C/C++ as well.

wish <<!
pack [button .b -text "Func is.. " -command {.b configure -text Dead ; update ; after 1000 ; destroy .}]
! 

MD3 support for Fitzquake and Quakespasm!? :)
I for one can use this already... I'm using it to replace models for KMquake2 and currently working on a new Shambler model for the Original Quake. But ofcourse it ends up as an MDL in Quakespasm and Fitzquake... My pipeline goes from 3dsmax exported to FBX and then I simply use Neosis for the conversion to MDL right now. Works perfectly but the model formats lack of precision makes me sad. If somebody adds MD3 support I could pretty much instantly take advantage of it. I would still use the quake palette though because I prefer the pixel retro look.

IF you add it I will use it! Regardless

https://www.dropbox.com/s/ckylp58vipdh9e3/Shambler_Wip.jpg?dl=0

Here is a sneak preview for those wondering what it looks like. 

https://dl.dropboxusercontent.com/u/1849053/Shambler_Wip.jpg