Discussion:
Standard APIs for databases, Web-apps
(too old to reply)
Graham Fawcett
2006-02-10 16:29:29 UTC
Permalink
I'd like to open discussion about introducing two formal APIs for use
in the development of future Chicken eggs: a standard API for
accessing relational databases, and a standard API for connecting Web
frameworks/applications to HTTP servers or *CGI gateways.

Adding proper interfaces at these two points would serve multiple
duties, including (a) easier porting of code from one backend
(database or Web server) to another, and (b) the opportunity for
"middleware" components (such as O/RM mappers on the database side,
and mod_* style components on the Web side).

Two popular database APIs in other languages come to mind: JDBC (Java)
and DB-API (Python). Neither is a perfect fit for Scheme, though the
simplicity of DB-API might make it a good source of ideas. I don't
know whether a similar layer has been implemented in some other
Scheme.

On the Web front, I would love to see something like the WSGI proposal
that is current in the Python community. WSGI is somewhat analagous to
CGI (the interface, not the execution model): it's an in-process
contract between server-communication code on on side (scgi, fcgi,
tcp-server) and Web application code on the other. This decoupling
allows the vast sea of Python Web frameworks to share common Web
server bindings, and lets those frameworks share middleware components
(authentication, logging, content-validation, etc.) as well as
higher-level components from other frameworks.

I'm currently working on an SCGI adapter for Chicken; I'd like the
"inside" of this adapter to speak a WSGI-like protocol. Ideally, I'd
like to help rework Spiffy's internals, so that it could speak the
common protocol, and decouple it from http-server at its dedicated
back-end. I'd be happy to discuss implementation details with anyone
who's interested.

I don't follow developments in other Scheme implementations; it's
possible that suitable modules could be borrowed from other projects.

Is there interest in either (both?) of these proposals?

Graham
Brandon J. Van Every
2006-02-10 19:16:30 UTC
Permalink
Post by Graham Fawcett
I'd like to open discussion about introducing two formal APIs for use
in the development of future Chicken eggs: a standard API for
accessing relational databases, and a standard API for connecting Web
frameworks/applications to HTTP servers or *CGI gateways.
I really have no stake in the issue. I'm a 3D / AI / game programmer,
and not generally interested in databases or web stuff. My comments are
only about implementing "standards." In open source, standards become
such only in hindsight. Things have to get implemented and then used.
Different people often provide different implementations and APIs for
the same class of problem; there's a "Survival Of The Fittest" dynamic
at work here. So I would suggest, don't worry about standards. Just
implement whatever API you want, and get other people to help out / use
it. If you and your crew get enough oomph behind it, then maybe someday
it'll be a de facto standard.

Still, I'll be honest, I have a hard time seeing that happening for
*any* Chicken egg. It may be "standard for Chicken" but I'm not
convinced Chicken has a large enough user community to ever create
broader standards. Python has tons more people than Chicken, it is much
much farther along in its tools and community evolution than Chicken
is. Never mind Java. Meanwhile, each Scheme implementation is a right
unto itself, the playing field is very fragmented.

Writing a binding for an extant, bona fide standard might be a more
profitable course to pursue. The downside is that popular programming
standards are never written with a Scheme mindset. So then you ask
yourself, am I going to make a low level binding, or am I going to do a
lot of value add in Scheme? If you do the latter, then you don't really
have a standard anymore. For instance, in my case the tension is
between producing an OpenGL 2.0 binding and a 3D engine. They're
different goals that please different crowds.



Cheers,
Brandon Van Every
Graham Fawcett
2006-02-10 21:00:38 UTC
Permalink
Post by Brandon J. Van Every
Still, I'll be honest, I have a hard time seeing that happening for
*any* Chicken egg. It may be "standard for Chicken" but I'm not
convinced Chicken has a large enough user community to ever create
broader standards. Python has tons more people than Chicken, it is much
much farther along in its tools and community evolution than Chicken
is. Never mind Java. Meanwhile, each Scheme implementation is a right
unto itself, the playing field is very fragmented.'
Points well taken. But it's never to early to start -- if evolution is
a community goal, then a little forthought into common interfaces is
better done sooner, rather than later when the community code base
(and resultant dependencies) may have grown significantly.

Best,
Graham
Brandon J. Van Every
2006-02-11 00:18:36 UTC
Permalink
Post by Graham Fawcett
Post by Brandon J. Van Every
Still, I'll be honest, I have a hard time seeing that happening for
*any* Chicken egg. It may be "standard for Chicken" but I'm not
convinced Chicken has a large enough user community to ever create
broader standards. Python has tons more people than Chicken, it is much
much farther along in its tools and community evolution than Chicken
is. Never mind Java. Meanwhile, each Scheme implementation is a right
unto itself, the playing field is very fragmented.'
Points well taken. But it's never to early to start -- if evolution is
a community goal, then a little forthought into common interfaces is
better done sooner, rather than later when the community code base
(and resultant dependencies) may have grown significantly.
Well, it's just that Chicken is realistically at the stage where 1 guy
does 1 thing. Are there any large team efforts for any Chicken projects
at all? I see no evidence of it in the Eggs libraries, at a glance they
all look like the works of solo authors. Chicken itself is the only
thing that seems to get multiple people working on it, for things like
bug reports and builds on different systems and whatnot. What I'm
saying is, community planning doesn't have much value. At this stage,
if you want it, you build it. For instance there's no reason for me to
ask about OpenGL 2.0 binding details, what people would like to see.
I'm either going to do it or I'm not, and I'll be doing what I want.
Really, at such an early stage, it's unlikely that anyone will ever do
anything useful, and even if they do they're only along for the ride.
Projects pretty much have to be "one person's baby" for awhile, until
they get big enough that other people can be slotted into them.

So actually, yeah, I think it is too early to start. On worrying about
community database standards, that is. I do think Chicken is large
enough to consider issues that affect *all* Chicken users, or at least
Chicken users on specific OS platforms, that sort of thing. If more
infrastructure is created for Chicken in general, then there will be
more Chicken users, and then there will be more manpower for database
standards projects and OpenGL 2.0 bindings and whatnot.

By all means implement whatever you wish to though. I'm not a proponent
of Stop Energy.
http://www.userland.com/whatIsStopEnergy

BTW I did float the idea of a standard OpenGL 2.0 binding for all
Schemes on comp.lang.scheme. Crickets chirped. Nobody cares, nobody's
doing 3D with Scheme. I'm on the frontier. Well, ok, OpenSceneGraph
has got a Scheme binding, so there isn't *zero* 3D done in Scheme. But
really, there are very, very few projects that do any 3D. Maybe you
fare better in databaseland, I dunno. It's just that, this apathy
really hit home how fragmented the Scheme universe is.


Cheers,
Brandon Van Every
Peter Keller
2006-02-11 00:38:40 UTC
Permalink
Post by Brandon J. Van Every
BTW I did float the idea of a standard OpenGL 2.0 binding for all
Schemes on comp.lang.scheme. Crickets chirped. Nobody cares, nobody's
doing 3D with Scheme. I'm on the frontier. Well, ok, OpenSceneGraph
has got a Scheme binding, so there isn't *zero* 3D done in Scheme. But
really, there are very, very few projects that do any 3D. Maybe you
fare better in databaseland, I dunno. It's just that, this apathy
really hit home how fragmented the Scheme universe is.
Maybe some of us should write a pretty fun game using an opengl 2.0
binding for chicken (along with the SDL_* API if it isn't already
done). Make it hackable with lots of small but interesting things in the
codebase (like higher order functions for AI, etc, etc, etc that people
can drop in). Give people a reason to want to use and hack Chicken for 3D.

In short, I've noticed a lot of people like to write scheme interpreters,
compilers, and internal bits like C bindings to other languages, but
not many write true applications. :) <- I'm not causing trouble with that
statement! Well, not much trouble at any rate....

-pete
Brandon J. Van Every
2006-02-11 01:06:48 UTC
Permalink
Post by Peter Keller
Post by Brandon J. Van Every
BTW I did float the idea of a standard OpenGL 2.0 binding for all
Schemes on comp.lang.scheme. Crickets chirped. Nobody cares, nobody's
doing 3D with Scheme. I'm on the frontier. Well, ok, OpenSceneGraph
has got a Scheme binding, so there isn't *zero* 3D done in Scheme. But
really, there are very, very few projects that do any 3D. Maybe you
fare better in databaseland, I dunno. It's just that, this apathy
really hit home how fragmented the Scheme universe is.
Maybe some of us should write a pretty fun game using an opengl 2.0
binding for chicken (along with the SDL_* API if it isn't already
done). Make it hackable with lots of small but interesting things in the
codebase (like higher order functions for AI, etc, etc, etc that people
can drop in). Give people a reason to want to use and hack Chicken for 3D.
"A game you can hack in Scheme" already exists:
http://stratagus.sourceforge.net/index.shtml
It's a Warcraft clone. It is not 3D and it is not Chicken. I played
the Warcraft demo once upon a time and didn't like it, so I haven't
pursued this. Generally speaking I don't like the Real Time Strategy
genre. I'm a Turn Based Strategy guy.

I've been playing an inordinate amount of The Battle For Wesnoth
lately. http://www.wesnoth.org
It has rather high production values for an open source game. It's a
Turn Based Strategy done in a sort of a fantasy RPG way. I guess like
Age Of Wonders or something. You build an army, you level up the guys
in your army. Unfortunately, your guys tend to get killed and then you
start all over again. I am sorta wondering if there's a point to the
leveling up. Anyways it's a SDL / C++ / Python thingy. No 3D. I've
thought about seeing if Chicken's C++ capabilities are up to interfacing
with Wesnoth. However, figuring out how to build Wesnoth with MinGW
looks tedious, if doable. So I'm still hemming and hawing about it.
Maybe I'll take a stab at it; I'm bored with signature gathering and
need to kill some time before a film event this evening.

One problem I have with all these "use someone else's game" projects is
that they're all GPLed. I want to work on stuff that I can use for my
own commercial games. If I work on an AI for some GPLed game, I feel
like I'm wasting my time. Ditto if I do any significant amount of
OpenGL 2.0 work. Really I'm only interested in MIT / BSD licensed
stuff, and almost nobody offers games to hack under those terms. LGPL
is on the borderline of what I'd be willing to deal with.

Actually, "C-Evo" is a TBS kinda like Civ and its license is,
surprisingly, public domain. http://www.c-evo.org . But, the main
codebase is Pascal. Someone wrote a C++ binding for the AI stuff, but
honestly, doing a Pascal / C++ / Scheme shuffle doesn't turn me on.
Again, not a 3D game.

I've thought about binding Chicken to various 3D engines. I'm inclined
to view this as a waste of time, though. Every 3D engine out there will
have its own stupid C++ calls for manipulating the scene graph, and
really, that's something you'd rather be doing with Scheme proper.
Having to ape other languages' baloney high level data structures just
strikes me as really really tacky and counterproductive. I'm thinking
it is better to implement a 3D engine almost from scratch. In practice,
that means from scratch.

As far as giving people a reason to want to hack Chicken for 3D, well,
how many people on this list are already interested in doing that?
Really there has to be some critical mass to get started. Otherwise,
it's just like I was telling the other fellow about database standards.
All I can do is make my own stuff. Nobody else is really going to get
anything done. For my part, getting other people to hack Chicken for 3D
is a decidedly secondary goal. My primary goal is to make something of
use to me for my own commercial games.


Cheers,
Brandon Van Every
Shawn Rutledge
2006-02-11 01:45:44 UTC
Permalink
I've thought about binding Chicken to various 3D engines. I'm inclined to
view this as a waste of time, though. Every 3D engine out there will have
its own stupid C++ calls for manipulating the scene graph, and really,
that's something you'd rather be doing with Scheme proper. Having to ape
other languages' baloney high level data structures just strikes me as
really really tacky and counterproductive. I'm thinking it is better to
implement a 3D engine almost from scratch. In practice, that means from
scratch.
Yes I agree that would be nice, but since you'd want to take as much
advantage as possible of hardware acceleration, how can you avoid the
OpenGL API, for the sake of portability?

My goal is not a game, but to mess around with next-generation GUI
ideas. I don't even need 3D very badly to do that, but I figure the
trend is that 3D hardware acceleration is taking over, and the leading
2D GUI systems (OS X, Vista and xegl) are taking advantage of it, not
to mention the 3D ones like Looking Glass. Unfortunately I don't have
experience with OpenGL, so I'm going to have to learn, I guess.

I got all excited a few months ago when I discovered the openGL egg
but then was disappointed when I saw how incomplete it was for my
purposes, and how much real work all those bindings are still going to
be to finish implementing, and how painful it is to use a 3D API for
doing mostly 2D work.

I am also interested in having my GUI system be portable to PDAs etc;
realistically by the time I have done something useful, PDAs and
phones and such will probably all have 3D hardware, but in the mean
time there are a couple of software-only openGL implementations, like
this one which does have some hardware acceleration on XScale
processors:

http://studierstube.icg.tu-graz.ac.at/klimt/

So I think a 3D binding which is based on OpenGL is probably going to
have a very long life.

I'm not much of a gamer, but have enjoyed Civ and SimCity in the past,
and agree that's probably a very good game genre to try to do in
Scheme.
Peter Keller
2006-02-11 06:10:33 UTC
Permalink
Post by Shawn Rutledge
So I think a 3D binding which is based on OpenGL is probably going to
have a very long life.
Having once did a FULL binding to opengl 1.1 for Bigloo, I know how much
Post by Shawn Rutledge
How stable is Chicken's FFI? <
Because what happened was that I did the bigloo opengl port (my god was
it *sexy*) and then Mr. Serrano changed the FFI a version later causing
my work to crumble and I didn't want to repeat it so it died.

If the FFI is stable enough (felix?) and I occasionally get some help
from the list, I will seriously consider doing a _full_ opengl 2.0 port
for chicken.

And, if you wanted the most bang for your buck, then the SDL_* API should
be completed, a wrapper for libpng should be made, and a wrapper for a
popular .wav file library. Then, all of the pieces are ready for someone
to do game programming in Chicken.

I am fully willing to make my work BSD licensed.

-pete
Shawn Rutledge
2006-02-11 07:23:32 UTC
Permalink
Post by Peter Keller
Because what happened was that I did the bigloo opengl port (my god was
it *sexy*) and then Mr. Serrano changed the FFI a version later causing
my work to crumble and I didn't want to repeat it so it died.
That's terrible!
Post by Peter Keller
And, if you wanted the most bang for your buck, then the SDL_* API should
Forgive my naivety but why do you still need SDL in that case? To
display images you map them as textures onto rectangles, and the
OpenGL implementation takes care of the rest, right?

What do you have to do to get text via OpenGL? Would you need an
interface to freetype or something like that? And if so, would the
output from freetype have to be a bitmap? Or is it too terribly
expensive to translate glyphs into sequences of OpenGL drawing
commands for each line segment?

Another reason I was starting my own 2D graphics library (and
procrastinating dealing with 3D) is that I see OpenGL uses
floating-point arguments to most functions. For now that is not a
good idea on small devices (even the ones that are powerful enough to
do some 3D rendering). But klimt claims to have a fixed-point
implementation. So I think it would be ideal to keep the numeric
types swappable somehow.

For my 2D library I planned to use 24.8 bit fixed point for screen
coordinates, with the units being millimeters, and on most devices
(where exact dimensions are not important) to fudge it by assuming
there are 4 pixels per mm (101.6 dpi is pretty close to typical
resolution of today's monitors). So you'd end up with 6 bits of
sub-pixel resolution. A simple implementation would just round to the
nearest pixel; an integer antialiasing implementation would use the
subpixels to be more accurate; and if OpenGL acceleration is
available, the 2D drawing commands could be translated into OpenGL
calls, and the fixed-point numbers would get translated into
floating-point. But, this approach assumes that the majority of the
work is 2D, because OpenGL is just being used as an implementation.
Wouldn't it be nice to get away from needing completely separate 2D
and 3D methodologies?

My prioritized concerns are first, to get the best 2D performance on
every device; second, to be forward-compatible with 3D hardware;
third, to make the 3D functionality on such hardware available fully
via a "nice" clean API which very closely resembles the 2D API (and
last but not least, slow 3D should even be available on slower
machines, using the same API). Apparently Cairo achieves at least the
first two goals (it can do software 2D rendering, or you can use the
"glitz" backend to render to OpenGL calls) but it's kindof
heavyweight, and also uses floating-point extensively.

When a file containing a model is saved, if you are saving a Scheme
data structure in ASCII form, then you cannot necessarily tell whether
a number with a decimal point in it (like say 167.25) is fixed or
floating point (except when it's obvious that the number can't be
exactly expressed in fixed-point). So what if there was a version of
Chicken for devices without math coprocessors in which flonums are
really fixed-point 24.8 format, and another version in which they are
floating point? For many uses of flonums (for middle-sized values at
least), an approximate implementation would be available on the
fixed-point version, which is better than just not having flonums at
all. And the OpenGL API would appear identical - it takes flonum
coordinates, and only the implementation is different. The PDA
version could be linked with klimt and the PC version to Mesa, or
whatever. And 3D models would be portable across the two; just that
they might not render the same, depending on the scale of the numbers
that are being used. As fp implementations get faster and more
pervasive the fixed-point implementation would fade out of use anyway.

Well sorry if this seems like a half-baked idea, but I'm curious what
you all think of it anyway.
Brandon J. Van Every
2006-02-11 13:52:09 UTC
Permalink
Post by Shawn Rutledge
Post by Peter Keller
And, if you wanted the most bang for your buck, then the SDL_* API should
Forgive my naivety but why do you still need SDL in that case?
You don't. I've avoided learning SDL because I'm not convinced it's a
value add if you're going to require OpenGL anyways. SDL began life as
a 2D portability layer. I don't have a sense of how OpenGL oriented it
is nowadays. I'm kinda jaded. I'm thinking, gee there's probably a lot
of code in the way of the OpenGL stuff, because of SDL's long and sundry
history.
Post by Shawn Rutledge
What do you have to do to get text via OpenGL?
There's a bunch of stuff at www.opengl.org about this.
Post by Shawn Rutledge
Another reason I was starting my own 2D graphics library (and
procrastinating dealing with 3D) is that I see OpenGL uses
floating-point arguments to most functions. For now that is not a
good idea on small devices (even the ones that are powerful enough to
do some 3D rendering). But klimt claims to have a fixed-point
implementation. So I think it would be ideal to keep the numeric
types swappable somehow.
Man this is so early 90s. Back when we all wrote our own 3D APIs from
scratch! Look into OpenGL ES and what devices support it. ES is
supposed to be the stripped down version of OpenGL for such gizmos.
Post by Shawn Rutledge
For my 2D library I planned to use 24.8 bit fixed point for screen
coordinates, with the units being millimeters, and on most devices
(where exact dimensions are not important) to fudge it by assuming
there are 4 pixels per mm (101.6 dpi is pretty close to typical
resolution of today's monitors). So you'd end up with 6 bits of
sub-pixel resolution. A simple implementation would just round to the
nearest pixel; an integer antialiasing implementation would use the
subpixels to be more accurate; and if OpenGL acceleration is
available, the 2D drawing commands could be translated into OpenGL
calls, and the fixed-point numbers would get translated into
floating-point. But, this approach assumes that the majority of the
work is 2D, because OpenGL is just being used as an implementation.
Wouldn't it be nice to get away from needing completely separate 2D
and 3D methodologies?
I stared long and hard at the problem, and concluded that doing 2D via
3D HW is the correct way to do things. You're going to need image
rotation and scaling operations and so forth. There's a reason the rest
of the industry went "3D for 2D." I wouldn't waste time reinventing
last decade's thought processes on the matter.
Post by Shawn Rutledge
My prioritized concerns are first, to get the best 2D performance on
every device;
Unless this is going to get you big $$$$$$, this is a complete waste of
time.
Post by Shawn Rutledge
second, to be forward-compatible with 3D hardware;
third, to make the 3D functionality on such hardware available fully
via a "nice" clean API which very closely resembles the 2D API (and
last but not least, slow 3D should even be available on slower
machines, using the same API). Apparently Cairo achieves at least the
first two goals (it can do software 2D rendering, or you can use the
"glitz" backend to render to OpenGL calls) but it's kindof
heavyweight, and also uses floating-point extensively.
Yeah, so, 2 years from now nobody will care. You'll waste all this time
trying to optimize for obsolescent technology. Reminds me when I was
going to port Free3D from i486 to Pentium. Glad I got lazy, because in
hindsight, it would have been for nothing. The HW changes too fast.
Plan for the future, not the past.
Post by Shawn Rutledge
So what if there was a version of
Chicken for devices without math coprocessors in which flonums are
really fixed-point 24.8 format, and another version in which they are
floating point?
Blow off the device that has poor market share. You know, I wasted my
early career on trying to make 3D APIs portable to everything,
everywhere. I ended up with a really spiffy portable 2D rasterization
layer, and my 3D API barely drew a polygon. I was also an Autoconf and
IMake guru. It's better to pick one platform that's worth *money* and
do a decent job on it.
Post by Shawn Rutledge
Well sorry if this seems like a half-baked idea, but I'm curious what
you all think of it anyway.
I think I've been there, done that. I know everyone has to reinvent the
world anew for themselves, but I hope you heed the warnings of older souls.


Cheers,
Brandon Van Every
Shawn Rutledge
2006-02-11 22:54:28 UTC
Permalink
Post by Brandon J. Van Every
Man this is so early 90s. Back when we all wrote our own 3D APIs from
scratch! Look into OpenGL ES and what devices support it. ES is supposed
to be the stripped down version of OpenGL for such gizmos.
Yes it claims to use fixed-point math, just like it should. So what's
your favorite free implementation then?

klimt claims to be very similar (maybe it's a subset) but not quite
compliant. Are there licensing issues, or having to pay in order to
claim to be compliant?
Post by Brandon J. Van Every
I stared long and hard at the problem, and concluded that doing 2D via 3D
HW is the correct way to do things. You're going to need image rotation and
scaling operations and so forth. There's a reason the rest of the industry
I agree.
Post by Brandon J. Van Every
went "3D for 2D." I wouldn't waste time reinventing last decade's thought
processes on the matter.
I'm plenty lazy enough not to re-do other people's work, as long as
their work is accessible to me.
Post by Brandon J. Van Every
Yeah, so, 2 years from now nobody will care. You'll waste all this time
trying to optimize for obsolescent technology. Reminds me when I was going
I'm talking about phones, the Zaurus, the Nokia 700, etc. 3D games
are starting to show up on phones, presumably using OpenGL ES. Of
course, maybe in a couple generations they will have real 3D
accelerators but I'm not seeing much of it yet.
Post by Brandon J. Van Every
So what if there was a version of
Chicken for devices without math coprocessors in which flonums are
really fixed-point 24.8 format, and another version in which they are
floating point?
Blow off the device that has poor market share. You know, I wasted my
early career on trying to make 3D APIs portable to everything, everywhere.
I ended up with a really spiffy portable 2D rasterization layer, and my 3D
API barely drew a polygon. I was also an Autoconf and IMake guru. It's
better to pick one platform that's worth *money* and do a decent job on it.
Like what kind of platform?
Post by Brandon J. Van Every
I think I've been there, done that. I know everyone has to reinvent the
world anew for themselves, but I hope you heed the warnings of older souls.
I'm probably not as young as you think, just haven't done much 3D, and
try hard not to get too jaded about technology in general, although it
does seem kindof inevitable doesn't it?
Brandon J. Van Every
2006-02-12 03:58:04 UTC
Permalink
Post by Shawn Rutledge
Post by Brandon J. Van Every
Man this is so early 90s. Back when we all wrote our own 3D APIs from
scratch! Look into OpenGL ES and what devices support it. ES is supposed
to be the stripped down version of OpenGL for such gizmos.
Yes it claims to use fixed-point math, just like it should. So what's
your favorite free implementation then?
I don't have one. I think making games on small devices is junk. I'll
worry about OpenGL ES if that's what Playstation3 is using. If I go
that route, I'm certainly not going to be worrying about free
implementations.
Post by Shawn Rutledge
klimt claims to be very similar (maybe it's a subset) but not quite
compliant. Are there licensing issues, or having to pay in order to
claim to be compliant?
Dunno. I'm a game designer. The most basic question is, what game are
you trying to write? Then, to what audience are you trying to sell it?
Do small portable devices fit your vision? They don't for mine.
Cellphone games, at any rate, seems like a really bozo market to try to
succeed in. You can't brand your product, you've got like a 15
character game title in some big list of "free downloadables." There
are articles on Gamasutra about this sort of thing. Cellphones look
like the game industry "writ small." Like if you want to do something
that was cutting edge in the early 80s or whatever. YMMV for
handhelds. Probably much better money there, and I have no idea what
technologies they're using.
Post by Shawn Rutledge
Post by Brandon J. Van Every
Yeah, so, 2 years from now nobody will care. You'll waste all this time
trying to optimize for obsolescent technology. Reminds me when I was going
I'm talking about phones, the Zaurus, the Nokia 700, etc. 3D games
are starting to show up on phones, presumably using OpenGL ES. Of
course, maybe in a couple generations they will have real 3D
accelerators but I'm not seeing much of it yet.
This is only more to the point on cell phones. The standards are
unstable. There's no marketing potential in it. So why bother? Have
you heard about any compelling cell phone games at all? That should
tell you something. It's not like they're brand new. I'm still waiting
for someone to inform me why, as a game designer, I'm supposed to care
about cell phones.
Post by Shawn Rutledge
Post by Brandon J. Van Every
It's
better to pick one platform that's worth *money* and do a decent job on it.
Like what kind of platform?
Either Windows, one of the consoles, or one of the handhelds. You could
even do Macs if you're funky. They have a tiny market, but also few
game developers, so almost no competition. Linux is useless as a
consumer desktop OS. They can't get 3D drivers right because of NVIDIA
/ GPL tiffs, and that issue isn't going away. Forget the cell phones,
it's a waste of time. Keep your eye on whether the market conditions
change for cell phones, though. It just doesn't pay to be the 1st one
pouring blood on the table for stuff like this.
Post by Shawn Rutledge
Post by Brandon J. Van Every
I think I've been there, done that. I know everyone has to reinvent the
world anew for themselves, but I hope you heed the warnings of older souls.
I'm probably not as young as you think, just haven't done much 3D, and
try hard not to get too jaded about technology in general, although it
does seem kindof inevitable doesn't it?
I'd feel a lot better if I had a working Chicken toolchain. I'm
probably going to give up on Eclipse and Schemeway. For some reason,
the Schemeway interpreter doesn't work in recent versions of Eclipse.
Meanwhile, I haven't found Eclipse particularly intuitive to use. It
looks nice and is reasonably documented, but there's still a learning
curve for getting non-Java things to work with it. The thing that
bugged me the other day is trying to search files to find stuff. Seems
like Eclipse wants me to set up all these projects and whatnot before I
can do that. The idea of just picking a directory and saying "search it
for what I want" seems alien. So, I gotta RTFM... and that's the main
thing I didn't like about XEmacs. If I gotta RTFM anyways no matter
what, then I may be better off with XEmacs. I don't have to worry about
the Scheme tools being "bleeding edge" and so forth.


Cheers,
Brandon Van Every
Peter Keller
2006-02-11 21:48:42 UTC
Permalink
Post by Shawn Rutledge
Post by Peter Keller
Because what happened was that I did the bigloo opengl port (my god was
it *sexy*) and then Mr. Serrano changed the FFI a version later causing
my work to crumble and I didn't want to repeat it so it died.
That's terrible!
Yeah, I was sad, which is why I'll only do it if the FFI is stable for
Chicken. I'd probably build on Felix's Egg, depending how it was done.
Post by Shawn Rutledge
Post by Peter Keller
And, if you wanted the most bang for your buck, then the SDL_* API should
Forgive my naivety but why do you still need SDL in that case? To
display images you map them as textures onto rectangles, and the
OpenGL implementation takes care of the rest, right?
While opengl is happy to draw whatever you want into a window, it has no
idea how to acquire that window from the window manager, set it up for
the right color depth, double bufferring, etc, and deal with keyboard,
mouse, window manager events, etc.

That's why you need something like GLUT (which isn't a part of opengl,
but often bundled with it) or SDL. GLUT has it's own internal event
handler loop which you cannot control, and SDL exposes this loop so in
essence you write it. After coding plenty of 3D things in my life, I've
found I very much prefer SDL to GLUT any day. Plus, SDL has an audio
and mixer set to its API which are really nice. There is OpenAL, which
is the opengl equivalent for an audio library, but it had some issues
for a while politically (Loki Games wrote it, went out of business,
released it, it lied fallow for a while, then some people took it over),
I'm not sure on its completness and support at this time.
Post by Shawn Rutledge
What do you have to do to get text via OpenGL? Would you need an
interface to freetype or something like that? And if so, would the
output from freetype have to be a bitmap? Or is it too terribly
expensive to translate glyphs into sequences of OpenGL drawing
commands for each line segment?
You'll probably have to use freetype or something to deal with the
truetype font. I've never used truetype stuff in anything I've done,
usually I just have some big bitmap of a bunch of antialiased characters
and then I select the subtexture which represents one letter out whenever
I need it. The pipeline takes care of bilenear filtering it and mapping
it onto whatever surface it needs to go on.
Post by Shawn Rutledge
Another reason I was starting my own 2D graphics library (and
procrastinating dealing with 3D) is that I see OpenGL uses
floating-point arguments to most functions. For now that is not a
good idea on small devices (even the ones that are powerful enough to
do some 3D rendering). But klimt claims to have a fixed-point
implementation. So I think it would be ideal to keep the numeric
types swappable somehow.
Yikes! Don't bother doing a raw 2D library anymore. It isn't worth
it. Opengl has the ability to ask for pixel perfect placement on the
window in a 2D setting, even though to coordinates are floating point. Use
that instead.
Post by Shawn Rutledge
Wouldn't it be nice to get away from needing completely separate 2D
and 3D methodologies?
If your project deals with 2D components specifically, then a small
abstraction layer would be good to represent it, but have the back end
be opengl.
Post by Shawn Rutledge
My prioritized concerns are first, to get the best 2D performance on
every device; second, to be forward-compatible with 3D hardware;
third, to make the 3D functionality on such hardware available fully
via a "nice" clean API which very closely resembles the 2D API (and
last but not least, slow 3D should even be available on slower
machines, using the same API).
Truth be told, use a 3D interface like opengl and do the mathematics
so things end up 2D. That will be the fastest solution across the most
hardware.
Post by Shawn Rutledge
When a file containing a model is saved, if you are saving a Scheme
data structure in ASCII form, then you cannot necessarily tell whether
a number with a decimal point in it (like say 167.25) is fixed or
floating point (except when it's obvious that the number can't be
exactly expressed in fixed-point). [...]
Surprisingly, you have this problem is every other language too. :) The
main way to deal with it is simply not to care about the slop and assume a
"thickness" around everything which is built into your calculations.

-pete
Shawn Rutledge
2006-02-11 23:05:57 UTC
Permalink
Post by Peter Keller
That's why you need something like GLUT (which isn't a part of opengl,
but often bundled with it) or SDL. GLUT has it's own internal event
handler loop which you cannot control, and SDL exposes this loop so in
essence you write it. After coding plenty of 3D things in my life, I've
found I very much prefer SDL to GLUT any day. Plus, SDL has an audio
OK that makes sense.
Post by Peter Keller
You'll probably have to use freetype or something to deal with the
truetype font. I've never used truetype stuff in anything I've done,
usually I just have some big bitmap of a bunch of antialiased characters
and then I select the subtexture which represents one letter out whenever
I need it. The pipeline takes care of bilenear filtering it and mapping
it onto whatever surface it needs to go on.
I imagine you're just referring to game menus and such? Whereas most
non-game apps will involve much more text, so one needs text rendering
to be as fast as possible, and also to support scalable fonts etc.
And I think soon there will be a lot more attention paid to aesthetics
of the text too - evolving towards the kind of quality on-screen that
you get out of TeX on paper.
Post by Peter Keller
Yikes! Don't bother doing a raw 2D library anymore. It isn't worth
it. Opengl has the ability to ask for pixel perfect placement on the
window in a 2D setting, even though to coordinates are floating point. Use
that instead.
Can you point me to some examples?
Post by Peter Keller
Post by Shawn Rutledge
Wouldn't it be nice to get away from needing completely separate 2D
and 3D methodologies?
If your project deals with 2D components specifically, then a small
abstraction layer would be good to represent it, but have the back end
be opengl.
Well that's what I figured.
Post by Peter Keller
Post by Shawn Rutledge
When a file containing a model is saved, if you are saving a Scheme
data structure in ASCII form, then you cannot necessarily tell whether
a number with a decimal point in it (like say 167.25) is fixed or
floating point (except when it's obvious that the number can't be
exactly expressed in fixed-point). [...]
Surprisingly, you have this problem is every other language too. :) The
main way to deal with it is simply not to care about the slop and assume a
"thickness" around everything which is built into your calculations.
So do you agree that Scheme ought to have portable flonums even when
the hardware doesn't support floating-point, by substituting
fixed-point instead?

What I'm proposing is to do that, and then use OpenGL ES or something
like it on small devices, and regular OpenGL on large devices, and
keep the API the same so that the Scheme OpenGL programs will run
unchanged on either one.
Peter Keller
2006-02-11 23:31:21 UTC
Permalink
Hello,
Post by Shawn Rutledge
I imagine you're just referring to game menus and such? Whereas most
non-game apps will involve much more text, so one needs text rendering
to be as fast as possible, and also to support scalable fonts etc.
And I think soon there will be a lot more attention paid to aesthetics
of the text too - evolving towards the kind of quality on-screen that
you get out of TeX on paper.
Yeah, then definitely mix the truetype library with opengl. I found a link
with a ridiculous amount of knowledge on it:

http://www.opengl.org/resources/features/fontsurvey/
Post by Shawn Rutledge
Post by Peter Keller
Yikes! Don't bother doing a raw 2D library anymore. It isn't worth
it. Opengl has the ability to ask for pixel perfect placement on the
window in a 2D setting, even though to coordinates are floating point. Use
that instead.
Can you point me to some examples?
Sure, here is a transform which will setup up the window as a 2D coordinate
system with the origin in the upper left hand corner:

glViewport(0, 0, w, h);
glMatrixMode(GL_PROJECTION);
glLoadIdentity();
glOrtho(0, w, h, 0, -1, 1);
glMatrixMode(GL_MODELVIEW);
glLoadIdentity();

You can pick where you want the origin to be with glOrtho()....

Then when you want to set the raster position at a pixel position, do this:

glVertex2i(x, y);
glRasterPos2i(x, y);

If you're mixing 2d and 3d, then lookup the implementation of
MESA_window_pos() in raw Opengl and use that so you dn't disturb the
current opengl settings when switching back and forth.
Post by Shawn Rutledge
So do you agree that Scheme ought to have portable flonums even when
the hardware doesn't support floating-point, by substituting
fixed-point instead?
Honestly, I can't make a good statement about it because I never develop
for hardware which doesn't have floating-point hardware.
Post by Shawn Rutledge
What I'm proposing is to do that, and then use OpenGL ES or something
like it on small devices, and regular OpenGL on large devices, and
keep the API the same so that the Scheme OpenGL programs will run
unchanged on either one.
After skimming the OpenGL ES docs, it appears that it can natively
deal with fixed point mathematics. And so you're asking (in essenace)
can the numeric tower be implemented in fixed point?

-pete
Shawn Rutledge
2006-02-12 06:43:18 UTC
Permalink
Post by Peter Keller
Yeah, then definitely mix the truetype library with opengl. I found a link
http://www.opengl.org/resources/features/fontsurvey/
That looks very useful, thanks.
Post by Peter Keller
Sure, here is a transform which will setup up the window as a 2D coordinate
glViewport(0, 0, w, h);
glMatrixMode(GL_PROJECTION);
glLoadIdentity();
glOrtho(0, w, h, 0, -1, 1);
glMatrixMode(GL_MODELVIEW);
glLoadIdentity();
You can pick where you want the origin to be with glOrtho()....
glVertex2i(x, y);
glRasterPos2i(x, y);
If you're mixing 2d and 3d, then lookup the implementation of
MESA_window_pos() in raw Opengl and use that so you dn't disturb the
current opengl settings when switching back and forth.
OK, thanks.
Post by Peter Keller
After skimming the OpenGL ES docs, it appears that it can natively
deal with fixed point mathematics. And so you're asking (in essenace)
can the numeric tower be implemented in fixed point?
Yep, exactly.

There would need to be predicates available to distinguish them, of
course. But hopefully in practice you wouldn't care most of the time.
Brandon J. Van Every
2006-02-12 04:10:08 UTC
Permalink
Post by Shawn Rutledge
So do you agree that Scheme ought to have portable flonums even when
the hardware doesn't support floating-point, by substituting
fixed-point instead?
Uuuh, dunno dude. "Ought to have" is mighty strong language. Oughtn't
you implement it? Besides, for the problems you've been describing, you
just sound like someone who's obsessing about performance for not much
gain. Maybe there are legitimate problems where "I must have fixed
point!" is paramount, but you haven't described one yet. We're way past
that stage of 3D history.
Post by Shawn Rutledge
What I'm proposing is to do that, and then use OpenGL ES or something
like it on small devices, and regular OpenGL on large devices, and
keep the API the same so that the Scheme OpenGL programs will run
unchanged on either one.
What is your basic market motive for wanting things to "run
everywhere?" How do you personally benefit from this, aside from some
kind of programmer aesthetic satisfaction that "things have been
perfected?" How will others benefit from this? The problem is, if it's
just your own whim, then nobody else is going to do the maintenance
upkeep on it. So that portability will never happen. People need a
stronger reason for it to happen than "well I'd like it to be that
way." Sure. I'd like an airplane to always take off when I board it,
to never suffer maintenance delays. But that's not how the airline
industry actually works. My point is, most people in Schemeland live
and die by their implementations. There's likely no value in "run
everywhere." Never mind all the disparate HW we've already commented on.


Cheers,
Brandon Van Every
Shawn Rutledge
2006-02-12 07:48:15 UTC
Permalink
Post by Brandon J. Van Every
Uuuh, dunno dude. "Ought to have" is mighty strong language. Oughtn't
you implement it?
Yeah, I suppose so. We'll see if I get around to it.
Post by Brandon J. Van Every
What is your basic market motive for wanting things to "run
everywhere?" How do you personally benefit from this, aside from some
Like a lot of open source developers I don't know how or if I will
make money. I have a day job for that. I'd rather that my work get
unlimited exposure (assuming I get anything worthwhile done :-) rather
than tie it down with licensing fees or retail sales, and
simultaneously have to turn a fun hobby into a stressful drudgery to
meet commercial expectations. And projects that are commercially
successful have to have a narrow enough scope that they can be done on
time, not be too revolutionary to be saleable, etc. My day-job
projects have mostly been decidedly underwhelming compared to my own
dreams. (There was one exception, but that company was not
commercially successful. If it had been an open-source project, at
least I could keep using the cool code that I wrote there. But alas,
it was not.)
Post by Brandon J. Van Every
kind of programmer aesthetic satisfaction that "things have been
perfected?" How will others benefit from this? The problem is, if it's
just your own whim, then nobody else is going to do the maintenance
upkeep on it. So that portability will never happen. People need a
stronger reason for it to happen than "well I'd like it to be that
It has worked that way for some projects. Just some are more popular
than others. The most popular open-source projects have become quite
portable.
Brandon J. Van Every
2006-02-12 16:30:04 UTC
Permalink
Post by Shawn Rutledge
Post by Brandon J. Van Every
What is your basic market motive for wanting things to "run
everywhere?" How do you personally benefit from this, aside from some
Like a lot of open source developers I don't know how or if I will
make money. I have a day job for that.
Well it's good you like your day job better than mine. :-) I quit DEC
8 years ago for lack of satisfaction. This was at the height of the
boom. Used credit cards to live, ran out of credit, and by the time I
said "oh oh" the bust was in full swing. Didn't find any computer work
during the bust, discovered signature gathering instead. Declared
bankruptcy last October. At least I have a job that I can stand, its
main advantage is I can do as much or as little work as I want, whenever
I want. Also it's exercise instead of sitting in front of a desk. Plus
sometimes I make good money. Not now though. It'll be a drag for the
next 3 weeks, but then I'll have more measures and it'll get better.
Post by Shawn Rutledge
I'd rather that my work get
unlimited exposure (assuming I get anything worthwhile done :-) rather
than tie it down with licensing fees or retail sales, and
simultaneously have to turn a fun hobby into a stressful drudgery to
meet commercial expectations.
Well, I don't consider getting Chicken builds to run well on Windows to
be "fun hobby." Rather it's stressful drudgery. But, I think it's
tractable, and important to my objectives. I'm starting to eliminate
other toolchains that create more stressful drudgery on top of that
though. I'm starting to learn about Dev-C++ because the MinGW packaging
is really good. So instead of worrying about how to build 10 libraries
for The Battle Of Wesnoth, it seems I'm down to 1. Knock on wood, I
haven't proven that those packages are actually reliable enough to
produce a stacked SDL app like Wesnoth. We'll see in another hour or two.
Post by Shawn Rutledge
Post by Brandon J. Van Every
kind of programmer aesthetic satisfaction that "things have been
perfected?" How will others benefit from this? The problem is, if it's
just your own whim, then nobody else is going to do the maintenance
upkeep on it. So that portability will never happen. People need a
stronger reason for it to happen than "well I'd like it to be that
It has worked that way for some projects. Just some are more popular
than others. The most popular open-source projects have become quite
portable.
Yes, but we're talking about the Scheme universe. You simply aren't
going to get masses of people cooperating on *anything* unless they have
a compelling interest to do so. Fixed point for 3D is definitely not
topical to these people. Even 3D itself isn't, frankly. For things 3D,
the correct strategic place to start is 1 implementation, say Chicken,
that does one thing well, like a 3D engine or an OpenGL value-add API.
If you can do something with that that's relevant in the real world,
like ship a game, then you can probably get people to pay attention and
glom onto the project. That's the position you need to be in before a
fragmented community will start doing things in a more standardized way.

For 3D there is also the much, much deeper problem that 99.9999999% of
3D developers use C++. I think it's really going to take a "show me the
money" project in Scheme before much of anyone changes their mind, and
even then they'll drag their feet forever. I do hope that Chicken's C++
interfacing is good enough to provide some conversion potential.


Cheers,
Brandon Van Every
Brandon J. Van Every
2006-02-12 03:34:50 UTC
Permalink
Post by Peter Keller
Post by Shawn Rutledge
Forgive my naivety but why do you still need SDL in that case? To
display images you map them as textures onto rectangles, and the
OpenGL implementation takes care of the rest, right?
While opengl is happy to draw whatever you want into a window, it has no
idea how to acquire that window from the window manager, set it up for
the right color depth, double bufferring, etc,
These things are trivial. OpenGL has platform-specific functions to
handle this sort of stuff, i.e. the WGL functions on Windows.
Post by Peter Keller
and deal with keyboard,
mouse, window manager events, etc.
These are more involved. The proper comparison, however, is SDL vs. any
other cross-platform GUI. Games benefit from their own in-game GUIs.
Various OpenGL-based GUIs exist; I have no idea how worthy they are for
game development. It's such a bog trying to compile all these stacks of
stuff on Windows, that I've never really gotten that far. I'm really
not sold that the effort is worth it vs. just cooking something up
yourself. Especially since foreign libraries tend to pull away from
Scheme development.
Post by Peter Keller
That's why you need something like GLUT (which isn't a part of opengl,
but often bundled with it) or SDL. GLUT has it's own internal event
handler loop which you cannot control, and SDL exposes this loop so in
essence you write it. After coding plenty of 3D things in my life, I've
found I very much prefer SDL to GLUT any day.
Well, yeah. No shocker. But I don't see why I should prefer SDL over
my own code. Especially since, I don't care about cross-platform
support as a primary goal. I need to ship games on Windows. The Mac
game market is marginal, and the Linux game market is worthless. The
real porting money is in consoles and I don't think SDL has anything to
say about that. My thought is, I'll build what I want. If it turns out
to be compelling, and worth bothering to make open source, and others
want to make it work on other platforms, great they can. Not really my
problem as far as getting the ball rolling. 3D engines tend to have
critical mass on one particular platform anyways. They have to survive
an awfully long time and please a lot of people for multiple platforms
to be supported equally well. Honestly I'm not sure there's any open
source 3D engine that is really at that level. Seems like, the story is
usually, it works on Platform X and someone else is dinking on a broken
crufty barely working version for Platform Y.
Post by Peter Keller
Plus, SDL has an audio
and mixer set to its API which are really nice. There is OpenAL, which
is the opengl equivalent for an audio library, but it had some issues
for a while politically (Loki Games wrote it, went out of business,
released it, it lied fallow for a while, then some people took it over),
I'm not sure on its completness and support at this time.
Yeah, dunno. But it's out there, it's not going away, it'll get
improved. Really it's either OpenAL or Ogg Vorbis.
Post by Peter Keller
You'll probably have to use freetype or something to deal with the
truetype font. I've never used truetype stuff in anything I've done,
usually I just have some big bitmap of a bunch of antialiased characters
and then I select the subtexture which represents one letter out whenever
I need it. The pipeline takes care of bilenear filtering it and mapping
it onto whatever surface it needs to go on.
Yeah it might be less of a big deal to do it this way nowadays. Texture
maps used to be a lot smaller.
Post by Peter Keller
Post by Shawn Rutledge
Wouldn't it be nice to get away from needing completely separate 2D
and 3D methodologies?
If your project deals with 2D components specifically, then a small
abstraction layer would be good to represent it, but have the back end
be opengl.
Agreed. The main problem with using OpenGL for 2D is just that it's a
PITA to get started with, before you've actually cooked up your 2D
abstraction layer. Sort of a baroque underlying machine for simple 2D
tasks. It's irritating that the industry never came up with standard
abstractions for 2D stuff. Instead everyone has to be good little
engineers and re-implement 2D abstractions over and over and over
again. Unless you actually like SDL and the LGPL.


Cheers,
Brandon Van Every
felix winkelmann
2006-02-13 07:17:59 UTC
Permalink
[...SDL...OpenGL...Game development...]
Now, I'm not a game developer (and the last game I wrote was a
really bad Donkey Kong clone written in TI Extended Basic, which was
ultimately rejected by a computer magazine, which nearly killed
my computing career... ;-) but I do work in a situation (embedded
application development). where I have to do with the problem of
getting nice snazzy graphics and GUI elements onto constrained
devices.

I really *love* SDL as it handles all the ugly stuff like getting a
window on the screen (even if in fullscreen mode) and letting me
handle events is a simple and sane manner. It is boring drudgery
to get these things to work and it solves portability problems
pretty well. Especially the tight focus on the gaming "infrastructure"
(if one can call it that) like event-handling and window-management
makes it so good: it doesn't try to take over my graphics pipeline,
and it doesn't throw stuff at me that I don't need (and what I need
I can get via SDL_ttf, SDL_image, and so on).
It's portability is excellent and the API is so simple that I seldom use
the SDL egg: easyffi and a few `define-foreign-record's is more
than enough to get going, if one doesn't mind using a procedural
C-like API inside ones Scheme code.
I don't know enough about licensing issues: LGPL *may* be a problem
for some people.
I also think that portability is still important. Brandon seems to have
the opinion "Only Windows pays off - so I don't bother with the rest",
to which he has every right. On the other hand the Mac is an interesting
platform, since there is practically very little competition, and I still
use Linux a lot and somehow still believe in open source... :-)
And don't forget the commercial aspect of Linux: I have heard of at least
one case where our company was approached by a customer who does
game systems (like the one you'll find in gaming arcades) on a Linux basis.
Why shouldn't the gaming console of tomorrow use Linux and standard
APIs and tools?


cheers,
felix
Brandon J. Van Every
2006-02-13 13:03:54 UTC
Permalink
Post by felix winkelmann
Why shouldn't the gaming console of tomorrow use Linux and standard
APIs and tools?
They might; for instance, I believe the Playstation3 is going to use
some kind of Linux and OpenGL. The problem is, people usually need to
worry about making a profit today. Linux isn't going anywhere as a
consumer desktop OS. I think you have it figured correctly, that it
could end up being used as an embedded OS, such as for a console or
arcade machine. Also of course Linux is relevant as a game server.

If you like the SDL universe then you might check out Agar.
http://agar.csoft.org/ It describes itself thusly:

"The Agar project produces a portable and window system independent
graphics toolkit for SDL and OpenGL®. For a complete listing of what is
included in the core Agar distribution, see the Agar libraries page.

"Agar is being used by applications such as emulators, scientific
simulations, visualization tools and SDL/OpenGL games. Some other
actively developed csoft.org projects which use Agar include Agar-SG (3D
scene graph and physics simulation framework), Agar-SC (scientific
computing toolkit which includes a set of Agar widgets), and Agar-EDA
(electronic simulation and design toolkit)."


Cheers,
Brandon Van Every
Peter Keller
2006-02-13 15:54:58 UTC
Permalink
Post by Brandon J. Van Every
"The Agar project produces a portable and window system independent
graphics toolkit for SDL and OpenGL®. For a complete listing of what is
included in the core Agar distribution, see the Agar libraries page.
Honestly, I recommend having a good game design first (theme, hand drawn
screen shots of everything, understanding all menu entries and what they
do, description of play, etc), before doing anything. Once the game is
set and you know it'll be fun and interesting (shop it around to a few of
your friends), THEN start looking at languages/technology/platforms/APIs.

Even writing in C and going for Windows isn't going to mean squat unless
the game is fun to play and somehow distinguishes itself from others of
its kind, or (nonintuitively) is a highly polished pinnacle of what a
genre has to offer. First and formost, it has to be _fun_ and _addicting_.

-pete
Brandon J. Van Every
2006-02-13 19:35:14 UTC
Permalink
Post by Peter Keller
Post by Brandon J. Van Every
"The Agar project produces a portable and window system independent
graphics toolkit for SDL and OpenGL®. For a complete listing of what is
included in the core Agar distribution, see the Agar libraries page.
Honestly, I recommend having a good game design first (theme, hand drawn
screen shots of everything, understanding all menu entries and what they
do, description of play, etc), before doing anything. Once the game is
set and you know it'll be fun and interesting (shop it around to a few of
your friends), THEN start looking at languages/technology/platforms/APIs.
Even writing in C and going for Windows isn't going to mean squat unless
the game is fun to play and somehow distinguishes itself from others of
its kind, or (nonintuitively) is a highly polished pinnacle of what a
genre has to offer. First and formost, it has to be _fun_ and _addicting_.
I halfway agree with you. I'm first and foremost a game designer,
although obviously I have a strong "technologist" streak in me as well
if I'm worrying about AI and Chicken Scheme and so forth. I do believe
in prototyping tools, with pencil and paper being my primary tools, but
3D frameworks are also such tools. The main reason I moved on to
functional programming languages is C++ really failed me in the
prototyping department, it was just too "stiff" for a lot of the
geometry problems I was dealing with.

I will reiterate my earlier post. In the absence of an extant codebase,
if I'm going to be working on things "from scratch," the only games or
game frameworks I'm interested in working on are my own. What I'm
working on, are Turn Based Strategy games played on 2D hex or square
tile maps. I think hex maps are a good way to do a wargame. For square
tiles I'm actually interested in rather large, screen-aligned tiles, not
isometric tiles. The point of such tiles would be to display elaborate
tile artwork, rather like the face cards in a deck of playing cards.
Also I should clarify, the projection of the 2D map doesn't have to be
overhead, it could be looked at "landscape style" or freelooked or
whatever. It would be a 3D game with 3D pieces, it's just that the game
mechanical framework is fundamentally 2D. A typical Real Time Strategy
game is still a quantized 2D game underneath it all. It's just played
really really fast. I don't care about the RTS genre, but the framework
I have in mind could be extended to that purpose. I don't intend to
"fight" someone about RTS, as TBS is my priority. But if we could work
it out amicably then that's fine. I do have a performance orientation
even to TBS: there are often many many pieces to move around, and the
user doesn't want to sit around waiting for that to happen.

So, for me, the "what am I doing?" phase is already decided. If anyone
else is seriously interested in such a framework, and wants to deliver
MIT licensed code, I may be interested in working with them. I'm leery
of such things however because "hey let's do a project!" often just
turns into a source of communications overhead with no actual
productivity. Ergo I remain primarily focused on getting my own crap
together. I've had would-be business partners in the past. They've had
a tendency to go round and round in circles on what to do, for 2..3
months. Then when it's time to actually *do* something, they haven't
delivered. So generally I bring a discussion to the point of, "Ok
great! Here's a task. You do A and I'll do B," and then see if they
actually do something. Typically they don't and then I forget about
them and move on. I'm afraid I have enough difficulty managing my own
productivity; I can't spend time marshalling other people. The most
important quality in a potential business partner, is whether they just
do things that need to be done, without much communications overhead or
prompting.

If I wasn't working from scratch, if I was just refactoring a game that
was at least "Beta" quality, my equation would be different. I'd be a
lot more open to working on a project that isn't mine. The labor value
of something that already basically works, that's ready to be tweaked
for AI and game design, would outweigh my own project plans.
Unfortunately I just surveyed Sourceforge and found nothing suitable.
I'll take another stab at it; I'm wavering in my resistance to SDL,
seeing as how Felix likes it, and Agar might be worth something beyond
games. So SDL based projects might be more acceptable than I first
thought. Still, my expectations on Sourceforge are not very high. Most
of the good stuff is GPLed and I won't work on a GPLed project.

So for those who want to do a group 3D game project, I see 3 options:

1) work on my 2D hex framework thingy

2) suggest an extant, open source, not GPLed, beta-or-better quality
game that can be tweaked

3) "race me" to see if by your own machinations of communication and
prognostication, you come up with something that other people want to
work on and successfully contribute to. :-)

It's really all a question of how firm a Vision you have, and what you
like to work on. I've already got one, so I'm not abandoning it just
for whatever. I fully understand if other people are so turned on by
their own game ideas, that they're not interested in 2D hex frameworks
or stuff on Sourceforge or whatever. I also know that arguing about
"What game are we working on?" takes a long long time, like 3 months
minimum, and doesn't lead to anything. So my list above is designed to
short-circuit the process. Really the list is (1) decide, (2) review
and decide, (3) wake me when it's over. :-)


Cheers,
Brandon Van Every
"The pioneer is the one with the arrows in his back."
- anonymous entrepreneur
felix winkelmann
2006-02-14 06:27:21 UTC
Permalink
Post by Brandon J. Van Every
If you like the SDL universe then you might check out Agar.
Hey, that sounds interesting. Thanks for the link!


cheers,
felix
john
2006-02-11 10:28:18 UTC
Permalink
I think a game would be an excellent group project to test out the all
round capabilities of Chicken. I would be interested to help in any
network protocol design and development.

John.
Post by Peter Keller
Post by Shawn Rutledge
So I think a 3D binding which is based on OpenGL is probably going to
have a very long life.
Having once did a FULL binding to opengl 1.1 for Bigloo, I know how much
Post by Shawn Rutledge
How stable is Chicken's FFI? <
Because what happened was that I did the bigloo opengl port (my god was
it *sexy*) and then Mr. Serrano changed the FFI a version later causing
my work to crumble and I didn't want to repeat it so it died.
If the FFI is stable enough (felix?) and I occasionally get some help
from the list, I will seriously consider doing a _full_ opengl 2.0 port
for chicken.
And, if you wanted the most bang for your buck, then the SDL_* API should
be completed, a wrapper for libpng should be made, and a wrapper for a
popular .wav file library. Then, all of the pieces are ready for someone
to do game programming in Chicken.
I am fully willing to make my work BSD licensed.
-pete
_______________________________________________
Chicken-users mailing list
http://lists.nongnu.org/mailman/listinfo/chicken-users
Peter Keller
2006-02-11 21:58:16 UTC
Permalink
Post by john
I think a game would be an excellent group project to test out the all
round capabilities of Chicken. I would be interested to help in any
network protocol design and development.
That would be pretty fun. I suppose I could do engine design or an asset
management library. I've done that stuff before.

Though, take my advice, no threading unless it is dirt obvious how it
works and has amazing benefit for the time investment. It is too hard
to debug and come to terms with the fact it might not ever be "right"
due to unreproducable race conditions.

Also, I'd say the first group project game should be something simple
so the requirements are plain as day and milestones are achievable in a
few months time each with hobby programming time. Something which didn't
involve flashy texture art would be a plus, unless some of us can draw.
(Most games these days are 5 programmers and 100 artists.)

-pete
Brandon J. Van Every
2006-02-11 13:32:22 UTC
Permalink
Post by Peter Keller
If the FFI is stable enough (felix?) and I occasionally get some help
from the list, I will seriously consider doing a _full_ opengl 2.0 port
for chicken.
I am fully willing to make my work BSD licensed.
Yeah, I didn't think MIT / BSD licensing was in any way problematic for
a mere OpenGL binding. It's full games that people tend to GPL, making
them pretty much unusable. Middleware like 3D engines tend to be either
LGPL or BSD. I prefer the latter.


Cheers,
Brandon Van Every
felix winkelmann
2006-02-13 06:58:15 UTC
Permalink
Post by Shawn Rutledge
How stable is Chicken's FFI? <
Chicken's FFI is probably the most stable part, since so many eggs
already depend on it.
There has been a lot of incompatible changes recently (until 2.2),
but from now on I'd like to have a long period of consolidation, at least
until R6RS comes out (and those changes will be on the surfacem
not the FFI), and even then I don't intend to make changes
that break backwards compatibility, if I can avoid it.


cheers,
felix
John
2006-02-11 17:30:05 UTC
Permalink
I think a 3D framework in Scheme is a pretty rockin' idea. I had a
simple one running in Chicken that worked pretty well, but I ended up
moving it to Bigloo. I was more afraid of the copying collector than
anything else: It'd be easy enough to allocate all big stuff outside of
scheme with the FFI, but I feel like I'd still get bitten by the copying
gc in the end. Which is kind of a shame, because Chicken's gc seems to
handle cases that Bigloo doesn't like too much (lots of garbage in a
short amount of time, which is handled by a minor gc).

The OpenGL interface wasn't too bad to write. I ended up just adding
OpenGL functions as I needed them along the way. And if you don't mind
names that are a little ugly, you can almost just feed gl.h into cigloo.
The OpenGL egg for chicken is a pretty good start, and it's nice that it
has good SDL functionality too.

In the end, I feel a little torn between Chicken and Bigloo. I really
like the completeness of Chicken's REPL, call/cc and the community.
Bigloo compiles faster, handles floating point numerics better, and has
a nice module system. I'd like to give gambit a try, but I'm not sure
it'd be worth it. It's got so little in the way of support libraries, I
don't see a reasonable object system (meroon doesn't quite work with
gsc), and the macro system is somewhat broken (syntax-case messes up DSSSL).

I definitely agree that it's a waste of time to try to make Chicken
bindings for an existing engine. Almost everything is written in C++
for starters -- it feels like interfacing any scheme with any C++
application of size will be a long, painful road. Besides, Scheme will
never achieve world domination if it's always just an extension language :)

-- John
Post by Shawn Rutledge
I've thought about binding Chicken to various 3D engines. I'm inclined to
view this as a waste of time, though. Every 3D engine out there will have
its own stupid C++ calls for manipulating the scene graph, and really,
that's something you'd rather be doing with Scheme proper. Having to ape
other languages' baloney high level data structures just strikes me as
really really tacky and counterproductive. I'm thinking it is better to
implement a 3D engine almost from scratch. In practice, that means from
scratch.
Yes I agree that would be nice, but since you'd want to take as much
advantage as possible of hardware acceleration, how can you avoid the
OpenGL API, for the sake of portability?
My goal is not a game, but to mess around with next-generation GUI
ideas. I don't even need 3D very badly to do that, but I figure the
trend is that 3D hardware acceleration is taking over, and the leading
2D GUI systems (OS X, Vista and xegl) are taking advantage of it, not
to mention the 3D ones like Looking Glass. Unfortunately I don't have
experience with OpenGL, so I'm going to have to learn, I guess.
I got all excited a few months ago when I discovered the openGL egg
but then was disappointed when I saw how incomplete it was for my
purposes, and how much real work all those bindings are still going to
be to finish implementing, and how painful it is to use a 3D API for
doing mostly 2D work.
I am also interested in having my GUI system be portable to PDAs etc;
realistically by the time I have done something useful, PDAs and
phones and such will probably all have 3D hardware, but in the mean
time there are a couple of software-only openGL implementations, like
this one which does have some hardware acceleration on XScale
http://studierstube.icg.tu-graz.ac.at/klimt/
So I think a 3D binding which is based on OpenGL is probably going to
have a very long life.
I'm not much of a gamer, but have enjoyed Civ and SimCity in the past,
and agree that's probably a very good game genre to try to do in
Scheme.
_______________________________________________
Chicken-users mailing list
http://lists.nongnu.org/mailman/listinfo/chicken-users
felix winkelmann
2006-02-13 07:27:13 UTC
Permalink
Post by John
I think a 3D framework in Scheme is a pretty rockin' idea. I had a
simple one running in Chicken that worked pretty well, but I ended up
moving it to Bigloo. I was more afraid of the copying collector than
anything else: It'd be easy enough to allocate all big stuff outside of
scheme with the FFI, but I feel like I'd still get bitten by the copying
gc in the end. Which is kind of a shame, because Chicken's gc seems to
handle cases that Bigloo doesn't like too much (lots of garbage in a
short amount of time, which is handled by a minor gc).
The copying GC is indeed a problem in tight loops, but if you keep
your heap small, the GC times should be minor. Can you say something
about the size and type of the datasets you had?
Post by John
I definitely agree that it's a waste of time to try to make Chicken
bindings for an existing engine. Almost everything is written in C++
for starters -- it feels like interfacing any scheme with any C++
application of size will be a long, painful road. Besides, Scheme will
never achieve world domination if it's always just an extension language :)
Well, it can start infiltrating before it dominates. I think connecting to
an existing engine is the path of least resistance (so to speak).
I have the beginnings of a SWIG-based binding to the Irrlicht 3D
engine, but it's far from being complete.


cheers,
felix
John
2006-02-16 19:58:46 UTC
Permalink
I had about twenty megs of data on the heap, which was around 100 ms for
a major gc. Now that I've read the docs a bit more, it seems that I've
spoken too soon: allocating all of my srfi-4 stuff as NONGC would cut
down on the heap size considerably. Hmmm, one less reason to not use
Chicken.

By the way, has anyone thought of doing something like PyGame for
Scheme? It seems to have popularized Python quite a bit.

-- John
Post by felix winkelmann
Post by John
I think a 3D framework in Scheme is a pretty rockin' idea. I had a
simple one running in Chicken that worked pretty well, but I ended up
moving it to Bigloo. I was more afraid of the copying collector than
anything else: It'd be easy enough to allocate all big stuff outside of
scheme with the FFI, but I feel like I'd still get bitten by the copying
gc in the end. Which is kind of a shame, because Chicken's gc seems to
handle cases that Bigloo doesn't like too much (lots of garbage in a
short amount of time, which is handled by a minor gc).
The copying GC is indeed a problem in tight loops, but if you keep
your heap small, the GC times should be minor. Can you say something
about the size and type of the datasets you had?
Post by John
I definitely agree that it's a waste of time to try to make Chicken
bindings for an existing engine. Almost everything is written in C++
for starters -- it feels like interfacing any scheme with any C++
application of size will be a long, painful road. Besides, Scheme will
never achieve world domination if it's always just an extension language :)
Well, it can start infiltrating before it dominates. I think connecting to
an existing engine is the path of least resistance (so to speak).
I have the beginnings of a SWIG-based binding to the Irrlicht 3D
engine, but it's far from being complete.
cheers,
felix
Brandon J. Van Every
2006-02-17 06:00:44 UTC
Permalink
Post by John
By the way, has anyone thought of doing something like PyGame for
Scheme? It seems to have popularized Python quite a bit.
Sure. In the long term that's what my 2D hex engine is about, at least
for a specific class of games. Rather than what people have "thought
about," I'd be much more interested in someone else reviewing the
Sourceforge game filters I posted, or saying "I am doing this particular
project, take a look," or otherwise taking action.

BTW the Chicken MinGW CMake build looks like it will be conquered soon.
Within a week if no further issues arise. I think the heavy lifting is
over, knock on wood.


Cheers,
Brandon Van Every
felix winkelmann
2006-02-17 07:17:57 UTC
Permalink
Post by John
I had about twenty megs of data on the heap, which was around 100 ms for
a major gc. Now that I've read the docs a bit more, it seems that I've
spoken too soon: allocating all of my srfi-4 stuff as NONGC would cut
down on the heap size considerably. Hmmm, one less reason to not use
Chicken.
Yes, that's what one should do - as Chicken does GC relatively frequently,
a heap as big as that will definitely be noticable.


cheers,
felix

Matthew David Parker
2006-02-12 02:47:24 UTC
Permalink
I don't know if it's too early to start suggesting a game to make, but I
have an idea for a pretty simple (artless) game that would actually be
quite useful to me and a lot of people (and fun).

Right now I'm working for my dad who is a computer science professor and I
changed Xpilot (www.xpilot.org) so that you can control your ship using
scheme (chicken). So we use that to try out neural networks and genetic
algorithms and other artificial intelligence ideas. It's the perfect game
for it because the controls are so simple (turn, thrust, shoot), yet the
behavior of the ship must be very advanced to avoid walls, dodge bullets,
kill the enemy ship, or capture their flag. He's going to use it to teach
his artificial intelligence class, and we've already gotten a couple of
papers accepted at conferences using it.

I think we should make a 3D version of xpilot from the ground up in
Chicken. If you want, install xpilot from ftp.xpilot.org and take a look
at it, and imagine a full 3d version.

If you don't want to play the game, here's the gist of what the 3d version
would be like:

-space simulator with realistic physics, accelleration, momentum, etc.
-multi-player client/server, server calculates physics, client displays
relevent game-world data.
-full 3-axis movement. mouse points ship, another button thrusts
-large cave-like maps made of simple blue wireframe blocks and diagonal
half-blocks.
-little white blocks for bullets

The game would look very retro, mostly black with blue wireframe walls and
white enemy ships and bullets. Conrolling the ship would be very unique.
In all other space games I have ever seen, the thrust button doesn't
actually make you accelerate, but it makes you go a certain velocity. In
xpilot it really makes you accelerate, and because there is no friction in
space, you just go faster and faster. The ship also has a mass and
momentum, so getting it to change directions means that you have to deal
with the momentum of your ship. It's fun to control the ship like this in
2d, and I get excited when I imagine skimming around the map like this in
3d. Not to forget that the combat would be entirely unique, as you can
point your ship to shoot in any direction regardless of your ship's
movement.

Also we would make it so the user could control the ship with AI, and get
relevant game-world information for input to the AI. I am mostly excited
to play this game, but I do think it would be very useful for doing AI that could be used for controlling real spacecraft.

So that's my idea for a game, and I hope it gets made sometime by someone.
With a BSD license, of course (Wo! to the GPL...)

Matt
Shawn Rutledge
2006-02-12 07:12:55 UTC
Permalink
Post by Matthew David Parker
Right now I'm working for my dad who is a computer science professor and I
changed Xpilot (www.xpilot.org) so that you can control your ship using
scheme (chicken). So we use that to try out neural networks and genetic
algorithms and other artificial intelligence ideas. It's the perfect game
for it because the controls are so simple (turn, thrust, shoot), yet the
behavior of the ship must be very advanced to avoid walls, dodge bullets,
kill the enemy ship, or capture their flag. He's going to use it to teach
his artificial intelligence class, and we've already gotten a couple of
papers accepted at conferences using it.
I think we should make a 3D version of xpilot from the ground up in
Chicken. If you want, install xpilot from ftp.xpilot.org and take a look
at it, and imagine a full 3d version.
There is a gentoo ebuild for it so I just tried it. It looks like an
advanced version of Space War. :-) Folklore has it being implemented
at MIT in the 60's using an oscilloscope as the display. I had a DOS
version and used to play with a friend on my first PC in about 1990 or
so. Anyway, it still requires some reflexes to play, and I can
imagine a 3D version would be really tough, maybe something like
Descent. It was best played with a fancy joystick. But on a 2D map
you can see all your adversaries at once (if they are nearby) whereas
in 3D, you see only forward, and might have to use keys to switch
between alternate views to look behind you etc. Seems like maybe I
could use the hat on my joystick to pan the view, if I remember right.

Anyway it definitely seems like a good platform for AI development. I
had an AI class at the university too, and it wasn't nearly that much
fun, but at least I got introduced to Lisp a little.
Brandon J. Van Every
2006-02-12 20:36:53 UTC
Permalink
Post by Matthew David Parker
I think we should make a 3D version of xpilot from the ground up in
Chicken. If you want, install xpilot from ftp.xpilot.org and take a look
at it, and imagine a full 3d version.
So that's my idea for a game, and I hope it gets made sometime by someone.
With a BSD license, of course (Wo! to the GPL...)
It's too bad that xpilot is GPLed. That makes it completely useless as
a starter codebase, for those of us like myself that are only going to
do MIT / BSD licensed stuff. It may be easier to refactor an existing
game than to write one from scratch, particularly if it's a simple
game. Also if the point is to do AI, I would strongly suggest blowing
off any multiplayer games. It's just extra work and it won't showcase
any particular advantages of Scheme or Chicken Scheme. I do think you
had the right idea about a game with simple graphics though. Either
that or it's a refactored game that already has its own art assets.

I am currently searching through sourceforge.net to see if there are any
suitable candidates. If I come up with anything I'll let you know, and
those interested might try this exercise themselves. sourceforge.net
has an annoying filter interface in that you can't specify OR functions,
so I've been using pretty minimal filters to start with: REQUIRE OpenGL,
TRASH GPL. Then using my brain for the rest. Annoying is that I'd like
to filter for (MIT or BSD), (C or C++), etc. I don't really want C++
but the reality is that's what a lot of people do. Good test of
Chicken's C++ interfacing abilities at any rate. I don't really want
LGPL but again the reality is that's what a lot of projects do.

So far I've had no luck. I think I may have to drop the OpenGL
requirement and just look for games that could be OpenGL-ified.


Cheers,
Brandon Van Every
Brandon J. Van Every
2006-02-12 22:59:42 UTC
Permalink
Post by Brandon J. Van Every
I am currently searching through sourceforge.net to see if there are any
suitable candidates.
So far I've had no luck.
I've completed my search. No luck. There is nothing worth refactoring
on Sourceforge under my criteria, which is:
REQUIRE C
TRASH GPL
REQUIRE Beta | Prouduction/Stable | Mature
REQUIRE some kind of Windows support

I also tried
REQUIRE Scheme
TRASH GPL
and predictably didn't get anywhere.

I'll be honest: if I have to implement stuff from scratch, I'm only
going to work on my own game project ideas. I'm a Turn Based Strategy
kind of guy. The framework I'm hoping to implement is something 2D with
hexes or large square tiles (to show off the tile artwork in that case),
OpenGL for the rendering, single player only, lotsa AI. Is anyone else
interested in that sort of game?


Cheers,
Brandon Van Every
Matthew David Parker
2006-02-13 19:13:37 UTC
Permalink
Yeah, I hate that Xpilot is GPL'd, though it was the reason we switched to
chicken from Petite Chez scheme. It's ok for us because we plan to offer
the software for free and then sell books or modules for xpilot-ai that
help a professor teach a particular topic.

The multiplayer aspect of Xpilot has actually been essential for our AI
stuff. We have for one of our projects about 130 xpilot clients connected
to one server, each client has it's own chromosome, and they fight it out
on the server, swapping genes as they kill one another, evolving to get
better in that way. Also, we can take our AI client out on a real life
server to play against real players, and maybe one day learn against them.
It just makes it much more versatile for what you can do.

Your RTS game is a good idea though, and would be much easier than a 3d
Xpilot. Though it seems like just making a simple little space game in
chicken would be a good test of chickens abilities. All you'd really have
to do is get it to load up the little cave map and have some real physics
to make the ship fly around. I mean, I don't think anyone should plan on
making a whole xpilot 3d game, but just seeing if chicken can do the 3d
engine would be good, and then you could use that base to make other sorts
of games.

Matt
Peter Keller
2006-02-13 19:50:55 UTC
Permalink
Post by Matthew David Parker
I mean, I don't think anyone should plan on
making a whole xpilot 3d game, but just seeing if chicken can do the 3d
engine would be good, and then you could use that base to make other sorts
of games.
Just because opengl is a 3d api doesn't mean you must write 3d games
with it. :) Just set the z coord to zero and be done with it. :)

-pete
Brandon J. Van Every
2006-02-13 20:15:28 UTC
Permalink
Post by Matthew David Parker
Your RTS game is a good idea though,
TBS! TBS! Do not be confused. I Am Not Now, Nor Have I Ever Been, A
Real Time Strategy Player.
Post by Matthew David Parker
and would be much easier than a 3d
Xpilot. Though it seems like just making a simple little space game in
chicken would be a good test of chickens abilities.
I have faith in Chicken's abilities - and Felix's - or I would never
have become the Windows CMake buildmaster. So for me the point is more
about what is worth doing as a matter of game design. Simple little
space games don't matter to me. Frameworks that other people can use
for more than one game, are higher up my list of priorities. That's
what my 2D hex framework is about. Complicated games are more
interesting to me, which is why I'm into TBS.

Physics is complicated and of great interest to many game programmers.
It doesn't turn me on though, so that functionality would have to be
done already for me to bother with it. I'm not looking for a
from-scratch physics project. That sort of thing is a lot of work for
no clear gain in terms of game design. On the other hand, if the
physics framework is already done, then I can put my Game Designer hat
on and just use it. Not sure how I feel about physics from an AI
standpoint. I don't object to *geometric* AI processing, i.e. the
tactical requirements of 3D spaceships fighting each other, but physics
is going to complicate the game state, probably to the point of
intractability. Then again all the TBS problems I contemplate are
intractable anyways, so I dunno. The intersection between physics and
my TBS penchant is probably discrete simulation and finite element
analysis. At what point are these things of value to the player, in
terms of game design? I don't have an answer. It is all too easy to do
physics for the sake of simulationism; that's not game design.
Post by Matthew David Parker
All you'd really have
to do is get it to load up the little cave map and have some real physics
to make the ship fly around. I mean, I don't think anyone should plan on
making a whole xpilot 3d game, but just seeing if chicken can do the 3d
engine would be good,
Chicken *can* do it. The question is not "seeing," the question is
doing it, and whether someone wants to do it. I'm saying, skip the
proof-of-concept project. Aim higher, towards whatever it is you
actually want. (Which admittedly could be 3D XPilot.)
Post by Matthew David Parker
and then you could use that base to make other sorts of games.
Someone else might want to do a networking engine, or a First Person
Shooter engine, or a RPG engine, or a MUD engine, or whatever. There
are all sorts of underlying "middleware" engines one could pursue. I've
told you the one I'm pursuing, the 2D hex engine. The question is, do
you really, really, really want to pursue a "3D space physics" engine?
It sounds like a set of problems you're already experienced with. I'm
just asking if your Vision and committment is so strong that you know
for sure you want to pursue this. The reason I pose it as a question,
is because if you really *were* totally committed to it, you'd just be
telling us so. :-) As I am with the 2D hex engine, minus some caveats
about "if you show me another project that's already working...."


Cheers,
Brandon Van Every
Toby Butzon
2006-02-10 19:37:45 UTC
Permalink
I don't mean to start a flamewar. With respect, Graham and everyone
else, I'm throwing this out there to see what you guys think.
Post by Graham Fawcett
Adding proper interfaces at these two points would serve multiple
duties, including (a) easier porting of code from one backend
(database or Web server) to another, and (b) the opportunity for
"middleware" components (such as O/RM mappers on the database side,
and mod_* style components on the Web side).
I find APIs designed around porting and middleware are too often
cumbersome or counterintuitive, while the benefits gained in porting and
installing "middleware" are minimal.
Post by Graham Fawcett
Two popular database APIs in other languages come to mind: JDBC (Java)
and DB-API (Python). Neither is a perfect fit for Scheme, though the
simplicity of DB-API might make it a good source of ideas. I don't
know whether a similar layer has been implemented in some other
Scheme.
I don't know much about DB-API, but JDBC is dumb. Here's the gist, for
those who haven't used it:

You use JDBC's standard API to locate/instantiate the vendor-specific
(i.e., MSSQL, Oracle, etc.) driver. (The driver pretty much has to be on
the classpath for this, or in a similar system-standard place. You
generally *don't* point at the specific driver in your code, e.g.
"C:\drivers\oracle.jar"; instead, you say it's "com.oracle.sql.Driver"
or something like that. It's been a while, I'm just trying to give the
idea.) Then that object is essentially a factory for other objects,
etc., for querying the database and accessing the results.

The problem I have with all this is, unless you *really* focus on being
non-database-specific, and make try to force all the database
interaction into its own "box", you'll end up with SQL statements
throughout your application (along with the associated references to
the JDBC API). That's bad because SQL *isn't* non-database-specific
(even though that was the original intent), so you'll have to chase down
those points if you change databases anyway.

The only way to maintain portability in this case is to make the database
code a well-organized, compact "layer" between the DB and the rest of your
app. Then if the DB changes, the layer is all that has to change. But
once you start thinking like this, the app-dependant "layer" takes the
place of JDBC's API. The API is at that point useless, and only confuses
matters because its notions don't always map well to the DB vendor's
notions of queries, responses, etc. It's clearer to use the DB vendor's
library/API directly, and forget JDBC even exists.
Post by Graham Fawcett
On the Web front, I would love to see something like the WSGI proposal
that is current in the Python community. WSGI is somewhat analagous to
CGI (the interface, not the execution model): it's an in-process
contract between server-communication code on on side (scgi, fcgi,
tcp-server) and Web application code on the other. This decoupling
allows the vast sea of Python Web frameworks to share common Web
server bindings, and lets those frameworks share middleware components
(authentication, logging, content-validation, etc.) as well as
higher-level components from other frameworks.
Hrm... so the vast number of frameworks out there don't actually do
different jobs? They're copies of each other with maybe a few innovative
ideas each?

This is one complaint I hear about all the software out there on the
web. There's so much of it, and lots of it does the same thing, and in
many cases, there's no discernable quality difference.

Another point is: One thing I like about Spiffy is that you can control
just about any behavior you want. You don't need to go figure out how
some mod_* works (only to find out in some cases what you want isn't
supported). You're better off, IMHO, knowing how HTTP works, and being
able to do whatever crazy stuff you want to in the web server. There's
no good reason (other than there only being one port 80 on each server)
to not run each web app in its own web-server process, that can do
whatever funky, custom stuff it wants to do.

The mod_*s are the right idea, as far as conceptualizing things and
giving a standard way to do something, but in Spiffy that might as well
be just a separate .scm file that hooks in. You want mod_mysql_auth?
Just drop in the .scm file. No reason to have a sysadmin load a module
into Apache for you, when you control the entire web-serving process for
your application.
Post by Graham Fawcett
I'm currently working on an SCGI adapter for Chicken; I'd like the
"inside" of this adapter to speak a WSGI-like protocol. Ideally, I'd
like to help rework Spiffy's internals, so that it could speak the
common protocol, and decouple it from http-server at its dedicated
back-end. I'd be happy to discuss implementation details with anyone
who's interested.
I'm only confused about why you would do this in the first place. What
alternatives are there to spiffy? And if there are none, but "may" be in
the future, why would there be? Why not continue to improve spiffy?

What I'm hearing is that you want to refactor Spiffy. What I'm asking
is, why? It sounds like refactoring for the sake of it.

Again, I'm not trying to be personal, Graham. I'm just trying out ideas.

-- TB
Graham Fawcett
2006-02-10 20:53:25 UTC
Permalink
Post by Toby Butzon
The problem I have with all this is, unless you *really* focus on being
non-database-specific, and make try to force all the database
interaction into its own "box", you'll end up with SQL statements
throughout your application (along with the associated references to
the JDBC API). That's bad because SQL *isn't* non-database-specific
(even though that was the original intent), so you'll have to chase down
those points if you change databases anyway.
I think we're on the same page. I'm not suggesting that the proposed
API should try to obscure the differences between SQL *dialects*. But
I would like to see a standard set of functions to handle
query-responses. For example, the current postgresql egg does not
provide a means for accesing the metadata of a result set; I think a
consistent metadata-accessor is a reasonable goal for all db bindings.
Also, wouldn't it be nice to have a standard set of row accessors:
fetch-all, fetch-one, foldr, for-each, etc., common to each database
wrapper?

I appreciate that the eggs are the work of volunteers, and not
suggesting that any of the eggs is deficient; rather, they are
idiosyncratic, and I think that users (me!) could benefit from a more
consistent API.
Post by Toby Butzon
The only way to maintain portability in this case is to make the database
code a well-organized, compact "layer" between the DB and the rest of your
app. Then if the DB changes, the layer is all that has to change. But
once you start thinking like this, the app-dependant "layer" takes the
place of JDBC's API. The API is at that point useless, and only confuses
matters because its notions don't always map well to the DB vendor's
notions of queries, responses, etc. It's clearer to use the DB vendor's
library/API directly, and forget JDBC even exists.
I agree about separating database concerns into a different layer. But
are vendors' notions of queries and responses really so different?
Maybe you've used databases that I haven't. It's possible that one
vendor might provide a feature set not available in other databases,
but a designer who has portability as a goal would generally avoid
those features. (I know that portability isn't always an objective.)
Post by Toby Butzon
Post by Graham Fawcett
This decoupling
allows the vast sea of Python Web frameworks to share common Web
server bindings, and lets those frameworks share middleware components
Hrm... so the vast number of frameworks out there don't actually do
different jobs? They're copies of each other with maybe a few innovative
ideas each?
Definitely true; a lot of NIH going on there. I don't see the
profileration of Python frameworks being played out again in the the
Chicken/Scheme community; as Brandon mentioned, the two communities
are very different.

I didn't intend to bash Spiffy in any way. It would be helpful,
though, if there were a clear set of hooks for introducing new
server-drivers. At work, we use SCGI extensively, and I'd like to plug
in a Chicken app the same way we plug in everything else. But it's not
clear to me how to write an SCGI driver that works with Spiffy out of
the box. I don't want to write a new framework, but I do want a
framework that talks SCGI in a clean (and easily monitored/debugged)
fashion. Making the Web API explicit would be a benefit here.
Post by Toby Butzon
I'm only confused about why you would do this in the first place. What
alternatives are there to spiffy? And if there are none, but "may" be in
the future, why would there be? Why not continue to improve spiffy?
Honestly, that's my game plan. I really just want a clear point of
demarcation between my SCGI driver and the app server. If that point
already exists, I'm having a little trouble finding it. :-) I don't
want to dilute the audience by introducing a different app server,
it's the connectivity side that I'm concerned with. I mentioned
middleware because it's a possibility, but it's not a personal
motivator.
Post by Toby Butzon
Again, I'm not trying to be personal, Graham. I'm just trying out ideas.
Me too. I don't mind if my ideas get shot down; every shot fired is a
learning opportunity. ;-)

Graham
Dominique Boucher
2006-02-10 21:13:39 UTC
Permalink
Graham,
Post by Graham Fawcett
I didn't intend to bash Spiffy in any way. It would be helpful,
though, if there were a clear set of hooks for introducing new
server-drivers. At work, we use SCGI extensively, and I'd like to plug
in a Chicken app the same way we plug in everything else. But it's not
clear to me how to write an SCGI driver that works with Spiffy out of
the box. I don't want to write a new framework, but I do want a
framework that talks SCGI in a clean (and easily monitored/debugged)
fashion. Making the Web API explicit would be a benefit here.
Have you looked at mod_lisp? Writing a Scheme "driver" for it is a matter
of minutes. I have used it myself with great success.

The comments for this blog entry

http://theschemeway.blogspot.com/2005/09/web-app-servers-and-modlisp.html

will help you appreciate the pros of mod_lisp and the cons of SCGI.

Dominique

--
The SchemeWay project
http://schemeway.sourceforge.net
Graham Fawcett
2006-02-10 21:26:22 UTC
Permalink
Post by Dominique Boucher
Have you looked at mod_lisp? Writing a Scheme "driver" for it is a matter
of minutes. I have used it myself with great success.
The comments for this blog entry
http://theschemeway.blogspot.com/2005/09/web-app-servers-and-modlisp.html
Thanks for the link! Mod_lisp was on my reading list; I'll take a look.

Graham
Reed Sheridan
2006-02-11 03:45:12 UTC
Permalink
Subject: [Chicken-users] Standard APIs for databases, Web-apps
I'd like to open discussion about introducing two formal APIs for use
in the development of future Chicken eggs: a standard API for
accessing relational databases
A DBI would be nice, though I wouldn't consider it a top priority, and it
might not be worth the effort given the small size of our community and the
other things we lack which may be more important. I'm ok with being
married to Postgresql, at least for now. But if you want to do this, you
might want to take a look at Gauche's DBI, new as of a few months ago:
http://www.kahua.org/cgi-bin/kahua.fcgi/kahua-web/show/dev/DBI

(The Kahua project also has an ORM in a CLOS-like object system too, in case
you're still interested in that).


On the Web front, I would love to see something like the WSGI proposal
that is current in the Python community. WSGI is somewhat analagous to
CGI (the interface, not the execution model): it's an in-process
contract between server-communication code on on side (scgi, fcgi,
tcp-server) and Web application code on the other. This decoupling
allows the vast sea of Python Web frameworks to share common Web
server bindings, and lets those frameworks share middleware components
(authentication, logging, content-validation, etc.) as well as
higher-level components from other frameworks.
Now you've got my attention. I'm already working on something like this.
My idea is to have each backend (CGI, FastCGI, SCGI, mod_lisp, you typing in
environment variables at a terminal, etc) return a record with an object
representing what would be environment variables if it were a CGI process
and an input, output, and error port. Output to the server would be done
similarly to the WSGI way. A more friendly interface could then be built on
top of this. I've nearly completed this for CGI and FastCGI (my
implementation of which unfortunately has some serious problems..).

I don't claim that this is a 100% solution, but I lack the time and probably
the judgement to come up with something much better. Hopefully it's at
least a subset of a 90% solution.

I'm currently working on an SCGI adapter for Chicken; I'd like the
"inside" of this adapter to speak a WSGI-like protocol.
I was planning to start on an SCGI adapter myself. How is that going? We
don't need two of them.

Ideally, I'd
like to help rework Spiffy's internals, so that it could speak the
common protocol, and decouple it from http-server at its dedicated
back-end. I'd be happy to discuss implementation details with anyone
who's interested.
Why would you want to use this with Spiffy? This protocol would be usable
with faster and more mature servers like Apache and lighttpd without any
extra work. Spiffy still has the advantage that you have greater control
over it, but if you're using the protocol, aren't you giving that up anyway?

Reed Sheridan
Graham Fawcett
2006-02-14 20:28:34 UTC
Permalink
I'm replying off-list for now, Reed, since my SCGI code may not be of
general interest.
Post by Reed Sheridan
Post by Graham Fawcett
I'm currently working on an SCGI adapter for Chicken; I'd like the
"inside" of this adapter to speak a WSGI-like protocol.
I was planning to start on an SCGI adapter myself. How is that going? We
don't need two of them.
Well, it's working. ;-) I've got an SCGI adapter that talks a
WSGI-like protocol on the inside. I've attached my code. I'm still a
rather new Schemer, so hopefully the code isn't too opaque. Really
there's not much to implementing SCGI; my (parse-scgi-request-environ)
function does most of the negotation.
Post by Reed Sheridan
Ideally, I'd
Post by Graham Fawcett
like to help rework Spiffy's internals, so that it could speak the
common protocol, and decouple it from http-server at its dedicated
back-end. I'd be happy to discuss implementation details with anyone
who's interested.
Why would you want to use this with Spiffy? This protocol would be usable
with faster and more mature servers like Apache and lighttpd without any
extra work. Spiffy still has the advantage that you have greater control
over it, but if you're using the protocol, aren't you giving that up anyway?
Really, I just didn't want to fracture an already-small community. If
there were existing Spiffy users, I'd rather have given them a new
tool than introduced them to a new framework altogether. But that's a
political decision, not a pragmatic one; I don't use Spiffy myself,
and would be just as happy with an alternate framework that was
multi-protocol friendly.

If you can glean anything from my SCGI code, feel free to rewrite or
incorporate it as you wish. I'd love to see your other adapters; do
you have a repository online?

Graham
Graham Fawcett
2006-02-14 20:29:19 UTC
Permalink
Post by Graham Fawcett
I'm replying off-list for now, Reed, since my SCGI code may not be of
general interest.
...except that I didn't reply offline after all. :-) Sorry about that, folks.

Graham
Loading...