What is a reasonable time a page should take to load? And are there some general tricks to reduce it apart from thinking, output buffering and using good queries?

The new Quaddicted listing currently takes about 200ms. The details page is fast, 20-40ms. Is that good? Overall hits for those pages are very low hundreds per day.

I am using sqlite3 and php. 

the listing took ~30ms without the tags. Those are joined from another table. Maybe I should store them as string in the maps table too. But that sounds like more work and not too easy. 

200ms is ok really. If you want to make it faster you could try a caching reverse proxy like varnish, or shoving bits of the page in mamcache. but remember the almost fundamental truth if caching: if you cache, you will sometimes be serving out of date data - and decide if you care (i bet you don't)

Sqlite is pretty fast but doesn't cope with concurrency all that well - if the number of concurrent requests starts climbing, you might want to try postgres or mysql instead. 

200ms is not ok I think, it should be faster. If I understand correctly, the problem arises with the use of tags from another database table. You say you are joining them in - are you sure that you set all indices correctly? What kind of join do you use? Also, you should definitely consider using mysql because since SQLite is serverless, it cannot cache any data between queries. 

just build some simple caching mechanism. Save the output into a file, check if the file is older than 5mins. that will protect you against /.ing as much as possible with almost no implementation overhead. 

I will not use mysql, I like the single files db with sqlite. And with a project this small it should not matter too much. The spmaps.html gets <<10000 hits per month. That's <300 per day, one all >4 minutes. :)

My query:
SELECT *, GROUP_CONCAT(DISTINCT tag) AS tags FROM maps LEFT JOIN tags ON maps.id = tags.mapid GROUP BY maps.id ORDER BY maps.id

maps has a lot of columns. tags only
Currently the maps.id is (oops) TEXT PRIMARY KEY, tags.mapid is TEXT. I will have a INTEGER PRIMARY KEY for the maps.id (and thus use INTEGER for tags.mapid too) in the end, maybe that will help a bit (was a stupid oversight when I started tinkering).

megaman: Cheers, already created that for an old project once:

$filename = 'blah.html';
$lastupdate = filemtime($filename);
$currenttime = time();
$timesincelastupdate = $currenttime - $lastupdate;

if ($timesincelastupdate > 300) { //rerender the file } 

don't forget proper headers and it might be a nice idea to put the caching check into the php start section:
http://paste2.org/p/704653 

Well, you can always try "explain"ing your query, try "explain SELECT *, GROUP_CONCAT(DISTINCT tag) AS tags FROM maps LEFT JOIN tags ON maps.id = tags.mapid GROUP BY maps.id ORDER BY maps.id", you might be able to tell if you're missing an index or something-- but it looks like sqlite gives fairly incomprehensible results for this... 

Screenshot : http://www.simonoc.com/files/maps/q3/moteof_final.jpg

Here is the final release of a map that some of you may have seen over the last 8 months via the mountain of screenshots I have drowned this forum in and wondered 'what is going on here!'. It has been a very long journey for me with countless builds and months of building but thanks to all my friends it has been an amazing experience. :)

As always, constructive comments are welcome and if you have the chance to record any demo's please let me know because I would love to see how you played the map. I do understand that this will not be everyone's cup of tea because it has a very strong puzzle element but take your time and try to resist the urge to noclip around the place. All the source files and a detailed explaination of the map will be released next week. The website does contain some help on how to find things if you are really stuck!

Zip File : http://www.simonoc.com/files/maps/q3/moteof_final.zip (49Mb)
Website : http://www.simonoc.com/pages/design/maps_q3/moteof.htm 

then rendering static HTML files is the way to go. Is tags.mapid indexed? 

now a news thread: http://www.celephais.net/board/view_thread.php?id=60417 

http://www.rockpapershotgun.com/2010/03/10/unlimited-detail-wants-to-kill-3d-cards/

Hmmm? 

Point cloud rendering. It's probably better to just wait until we have enough horse power to do real time ray tracing with nearly limitless triangles. 

I dunno that looks like a very elegant solution, at least as an alternative. I am intrigued. 

For all I know this is a marketing hoax or at least way to vague to trust. Don't you remember sparse voxel trees Carmark was talking about and with that awesome Siggraph paper? 

If it exists and is reliable then great but I'm not convinced.

But its a bit star trek - formulate a complex plan and explain it with a simplistic formula.

'It's 3d rendering but like Google.'

It's obviously a marketing video and I doubt they want to release a technical one. Will be interesting to see if this exists in 2 years - will say alot obout boasting, barriers to entry and 3d technology. 

Once we've got something real that's better than polygons I won't shed a tear at their departure.

But a fuzzy YouTube video isn't exactly conclusive evidence. 

I've seen a lot of points that seem to miss the entire reason the video was made (imo) - it is pretty much a marketing video these guys did to drum up interest. You can't get the funds you need to do the research to finish the project without interest. And besides it was 8 minutes long and tried to cram in a bunch of different things on top of explaining the IDEA behind the technology, not the entire scheme of operations.

Also, this: (unrelated) http://i.imgur.com/lLdEX.gif 

I 'm going to setup a quake 1 DM server probably using proquake and was wondering what the bandwidth usage would be like? my current connection is a 12mbit down 2mbit up. Also what levels would you guys like to see on the server. 

those assholes would've impressed me a lot more if they didn't spend most of that video explaining what polygons are, as if anyone who gives a shit didn't already know all that. Good way to use up time so you don't have to explain your idea though.

Also, they did a good job showing how much polygons suck, using footage from many graphically advanced N64 games.

I also lose faith in a company like that that doesn't have the judgement to hire one fucking artist. Presentation might be kinda important in this case, who knows. 

Go back to making happy demons starbuck








<3 

Wasn't even mad! AT THOSE GOD DAMN PIG FUCKERS 

be sure to read that in Hunter S. Thompson's voice. Thanks. 

actually, i totally agree with starbuck. the whole video was more about why polygons suck than what the new tech will mean beyond 'unlimited detail'.
and if you want to impress people with your graphical engine, it probably helps to have people who know how to create graphical content.
it's such a cop-out to say 'this would look better if we had hired a real artist to do it.' well then, why didn't you? 

It also would have looked better if it were trying to show off the art instead of the technical specs and what it was trying to do ... and also if it were complete and built off of tens of years of research with an entire industry and competition fueling said research ...