Discussion:
Working on format-compiler egg, question about macros, eggs as wiki pages
(too old to reply)
Alejandro Forero Cuervo
2010-06-13 20:36:25 UTC
Permalink
As part of my migration from Chicken 3 to Chicken 4 I'm rewritting the
format-modular egg into what I'm calling 'format-compiler'. I'm
reverting many changes from it and I'm cleaning up some of the
interfaces, to hopefully make it easier to extend (ie. to define
control characters that do interesting things). I'm also fixing some
bugs I've found and removing a lot of unnecessary complexity that has
been gradually added to it with little purpose. I have a draft of
this code here:

http://azul.freaks-unidos.net/format-compiler

I'm doing a pretty large change to it: I'm separating the process of
parsing the format string rom the process of actually generating the
output. My thinking is that most of the time format strings are
specified statically and never change during the execution of a
program, as in:

(format (current-error-port) "Evaluating ~A form: ~A~%" adj name)

Instead of parsing the string every time, I want to parse it once and
then use the result of that parsing on the arguments passed.

Interestingly, this is actually allowing me to simplify the
implementation significantly (or maybe I'm a slightly better
programmer now than I was when I wrote format-modular?).

The new interface will be that you define a spec mapping bindings to
formatting functions (in pretty much the same way you do with
format-modular, although I'm making a few changes in the interface to
the procedures that you register). If you call that spec 'bindings',
you can use it as such:

(let ((case-sensitive #t) (escape-char #\~))
(make-format-compiler case-sensitive escape-char bindings))

This returns you a format compiler —call it «fmt-compiler»—, to which
you pass a format string:

(define my-format-proc (fmt-compiler "Evaluating ~A form: ~A~%"))

This, on its turn, returns a function that receives the output port
(which, as usual with format procedures, may be #t or #f for output to
current output port or to a string) and the arguments, for example:

(my-format-proc output adj name)

Of course, you may find this more readable:

((fmt-compiler "Evaluating ~A form: ~A~%") output adj name)

And, indeed, you may trivially define a format procedure with the
traditional interface:

(define (format output str . args)
(apply (fmt-compiler str) output args))

The catch is, of course, that one may memoize fmt-compiler —perhaps
using “eq?” on the format string (for which I'll probably extend the
memoize egg)— which means that a given statically specified format
string would be parsed at most once and the results of that parsing
reused through-out.

In order to make this happen, I figured I would ask: is there a way
for, in a macro, check if a given symbol is actually a hard-coded (ie.
static) string? In other words, if my macro is called “format”, I
would like “(format x "static" ...)” and “(format x foo ...)” to have
different expansions, where the expansion for the former caches the
results of the parsing forever and the expansion of the later only
caches them for some time, with some heuristics. Is this possible at
all?

I will add this code as an egg for both Chicken 3 and Chicken 4 once
it's ready. I still have to add support for a few control characters.

This is also interesting for me in that it is an experiment of storing
an egg entirely in a wiki page. For example, the page has the
following tag:

<schemelibrarydefinition
author="Alejandro Forero Cuervo"
authoremail="***@freaks-unidos.net"
category="io"
synopsis="Creates format procedures to generate output based on format strings."
license="Public Domain"
name="format-compiler"
uses="posix embedded-test"
exports="*formatter-params* ..."/>

This is enough for Svnwiki to generate the setup, meta and Scheme
source code files (with the blocks embedded in the wiki page) for me
to just copy into the eggs repository. In that sense, the wiki page
itself becomes the authoritative location for the egg. See, for
example:

http://wiki.freaks-unidos.net/weblogs/azul/xsvnwiki-enscript/format-compiler/format-compiler.scm
http://wiki.freaks-unidos.net/weblogs/azul/xsvnwiki-enscript/format-compiler/format-compiler.meta
http://wiki.freaks-unidos.net/weblogs/azul/xsvnwiki-enscript/format-compiler/format-compiler.setup

Cheers!

Alejo.
http://azul.freaks-unidos.net/
Felix
2010-06-15 13:52:41 UTC
Permalink
From: Felix <***@call-with-current-continuation.org>
Subject: Re: [Chicken-users] Working on format-compiler egg, question about macros, eggs as wiki pages
Date: Tue, 15 Jun 2010 15:29:30 +0200 (CEST)
Subject: [Chicken-users] Working on format-compiler egg, question about macros, eggs as wiki pages
Date: Sun, 13 Jun 2010 22:36:25 +0200
Post by Alejandro Forero Cuervo
As part of my migration from Chicken 3 to Chicken 4 I'm rewritting the
format-modular egg into what I'm calling 'format-compiler'. I'm
reverting many changes from it and I'm cleaning up some of the
interfaces, to hopefully make it easier to extend (ie. to define
control characters that do interesting things). I'm also fixing some
bugs I've found and removing a lot of unnecessary complexity that has
been gradually added to it with little purpose. I have a draft of
http://azul.freaks-unidos.net/format-compiler
Very good! A solid implementation of CL-style formatting is dearly
needed, since the `format' egg is buggy and the basic `[sf]printf'
stuff is not particularly powerful. There is Shinn's formidable `fmt',
which I like to get used to (but haven't used it intensively enough).
In the end I always use `print' and `display' and normally that's
enough.
The complexity of the current `format-modular' code isn't so much the
problem - the large number of dependencies is. It would be good to
remove i18n support and for example provide hooks or another sort of
interface to extend it in a clean manner without increasing the size
and the number of base-dependencies of a bread-and-butter extension
like formatted text output.
Post by Alejandro Forero Cuervo
I'm doing a pretty large change to it: I'm separating the process of
parsing the format string rom the process of actually generating the
output. My thinking is that most of the time format strings are
specified statically and never change during the execution of a
(format (current-error-port) "Evaluating ~A form: ~A~%" adj name)
Instead of parsing the string every time, I want to parse it once and
then use the result of that parsing on the arguments passed.
Cool - this reminds me of Common Lisp's `formatter' macro. By
seperating the format-compiler into a module, it would be possible
to use it at compile time and provide a similar facility by using
compiler-syntax.
Well, expanding the complete format-string into code might lead to
code-explosion (this isn't so much an issue for chicken's builtin
formatting procedures, since the expansion is relatively simple).
So you probably should ignore my previous suggestion. If you compile
the format-string into a closure-tree you could at least cache/memoize
it somehow, just as you wrote initially.


cheers,
felix
Alejandro Forero Cuervo
2010-06-15 15:12:38 UTC
Permalink
Post by Felix
Subject: Re: [Chicken-users] Working on format-compiler egg, question about macros, eggs as wiki pages
Date: Tue, 15 Jun 2010 15:29:30 +0200 (CEST)
Subject: [Chicken-users] Working on format-compiler egg, question about macros, eggs as wiki pages
Date: Sun, 13 Jun 2010 22:36:25 +0200
Post by Alejandro Forero Cuervo
As part of my migration from Chicken 3 to Chicken 4 I'm rewritting the
format-modular egg into what I'm calling 'format-compiler'. I'm
reverting many changes from it and I'm cleaning up some of the
interfaces, to hopefully make it easier to extend (ie. to define
control characters that do interesting things). I'm also fixing some
bugs I've found and removing a lot of unnecessary complexity that has
been gradually added to it with little purpose. I have a draft of
http://azul.freaks-unidos.net/format-compiler
Very good! A solid implementation of CL-style formatting is dearly
needed, since the `format' egg is buggy and the basic `[sf]printf'
stuff is not particularly powerful. There is Shinn's formidable `fmt',
which I like to get used to (but haven't used it intensively enough).
In the end I always use `print' and `display' and normally that's
enough.
The complexity of the current `format-modular' code isn't so much the
problem - the large number of dependencies is. It would be good to
remove i18n support and for example provide hooks or another sort of
interface to extend it in a clean manner without increasing the size
and the number of base-dependencies of a bread-and-butter extension
like formatted text output.
Post by Alejandro Forero Cuervo
I'm doing a pretty large change to it: I'm separating the process of
parsing the format string rom the process of actually generating the
output. My thinking is that most of the time format strings are
specified statically and never change during the execution of a
(format (current-error-port) "Evaluating ~A form: ~A~%" adj name)
Instead of parsing the string every time, I want to parse it once and
then use the result of that parsing on the arguments passed.
Cool - this reminds me of Common Lisp's `formatter' macro. By
seperating the format-compiler into a module, it would be possible
to use it at compile time and provide a similar facility by using
compiler-syntax.
Well, expanding the complete format-string into code might lead to
code-explosion (this isn't so much an issue for chicken's builtin
formatting procedures, since the expansion is relatively simple).
So you probably should ignore my previous suggestion. If you compile
the format-string into a closure-tree you could at least cache/memoize
it somehow, just as you wrote initially.
I believe the size of the expanded code would be proportional linearly
to the length of the format string, so I think it wouldn't be very
bad. However, I will, still, probably take the approach of compiling
the strings at runtime, the first time they are used, and then just
caching them (forever, in the case of static strings).

Alejo.
http://azul.freaks-unidos.net/
Alejandro Forero Cuervo
2010-07-03 13:16:39 UTC
Permalink
How does one compile a file that requires an extension which is not
currently installed? I couldn't find the answer looking at The
Chicken Manual. Obviously, the file being compiled does not depend on
macros from the module, it just depends on it at runtime.
Syntax error (import): cannot import from undefined module
I know one can do something like:

(load (string-append (repository-path) module-name ".so"))

But I'm thinking about something cleaner than that (that, because it
uses the normal code that loads eggs at runtime, will, for instance,
not load the module if it had already been loaded by another
extension).

This is a problem for me because (1) I have files that want to execute
different codepaths depending on which eggs are available (loading
those that are) and (2) I have mutually dependent eggs (which now I
can't install in Chicken 4).

Is there a way to do this?

Thanks for your help!

Alejo.
http://azul.freaks-unidos.net/
Alejandro Forero Cuervo
2010-07-03 17:07:42 UTC
Permalink
Post by Alejandro Forero Cuervo
How does one compile a file that requires an extension which is not
currently installed? I couldn't find the answer looking at The
Chicken Manual. Obviously, the file being compiled does not depend on
macros from the module, it just depends on it at runtime.
Syntax error (import): cannot import from undefined module
(load (string-append (repository-path) module-name ".so"))
But I'm thinking about something cleaner than that (that, because it
uses the normal code that loads eggs at runtime, will, for instance,
not load the module if it had already been loaded by another
extension).
This is a problem for me because (1) I have files that want to execute
different codepaths depending on which eggs are available (loading
those that are) and (2) I have mutually dependent eggs (which now I
can't install in Chicken 4).
(The particular eggs are the format-compiler egg, which has embedded
tests using my embedded-test framework, and the embedded-test, which
uses format when tests fail.)

Alejo.
http://azul.freaks-unidos.net/
Thomas Chust
2010-07-03 22:14:18 UTC
Permalink
Post by Alejandro Forero Cuervo
How does one compile a file that requires an extension which is not
currently installed?  I couldn't find the answer looking at The
Chicken Manual.  Obviously, the file being compiled does not depend on
macros from the module, it just depends on it at runtime.
Syntax error (import): cannot import from undefined module
[...]
Hello,

the syntactic form (import ...) needs the relevant import libraries to
be present at compile time, the actual "implementation libraries" do
not have to be installed during compilation and can be loaded at
runtime using the procedure (require ...) from the library unit, for
example.

Ciao,
Thomas
--
When C++ is your hammer, every problem looks like your thumb.
Thomas Chust
2010-07-03 22:19:19 UTC
Permalink
[...] and can be loaded at runtime using the procedure (require ...)
from the library unit, [...]
Sorry, it's in the eval unit, not the library unit.

Ciao,
Thomas
--
When C++ is your hammer, every problem looks like your thumb.
Felix
2010-07-03 22:43:31 UTC
Permalink
From: Alejandro Forero Cuervo <***@freaks-unidos.net>
Subject: Compiling files that 'import' eggs that aren't installed?
Date: Sat, 3 Jul 2010 15:16:39 +0200
Post by Alejandro Forero Cuervo
How does one compile a file that requires an extension which is not
currently installed? I couldn't find the answer looking at The
Chicken Manual. Obviously, the file being compiled does not depend on
macros from the module, it just depends on it at runtime.
Syntax error (import): cannot import from undefined module
(load (string-append (repository-path) module-name ".so"))
But I'm thinking about something cleaner than that (that, because it
uses the normal code that loads eggs at runtime, will, for instance,
not load the module if it had already been loaded by another
extension).
This is a problem for me because (1) I have files that want to execute
different codepaths depending on which eggs are available (loading
those that are) and (2) I have mutually dependent eggs (which now I
can't install in Chicken 4).
Is there a way to do this?
You can use

(require-library NAME)

which is just like `require-extension' but doesn't do an `import'.

Note that if the extension is compiled as a module, you will not be
able to access its exported bindings, which doesn't help you a lot, I
guess.

Otherwise this is not possible. The compiler doesn't know that the
require'd extension doesn't export any syntax. What I would do for
(1) is to write a macro that expands into the right thing,
depending on the availability of an extension:

(define-syntax (maybe-require-extension x r c)
(let ((name (strip-syntax (cadr x))))
(if (extension-information name)
`(,(r 'require-extension) ,name)
`(,(r 'begin)))))

(2) is problematic: it actually indicates a design flaw in your module
structure (if you allow me to say so). Couldn't you extract the
mutually-recursive part into a third module to break up the
circularity? It should be possible to handle this by using some sort
of intermediate module, but it's too damn hot in this room to think of
an example (it may also because I'm too dim, but taking the heat as an
excuse looks better, of course).


cheers,
felix
Alejandro Forero Cuervo
2010-07-04 17:00:33 UTC
Permalink
Post by Felix
Note that if the extension is compiled as a module, you will not be
able to access its exported bindings, which doesn't help you a lot, I
guess.
Ah, dang, you're right: not very useful...
Post by Felix
Couldn't you extract the mutually-recursive part into a third module
to break up the circularity? It should be possible to handle this by
using some sort of intermediate module, but it's too damn hot in
this room to think of an example (it may also because I'm too dim,
but taking the heat as an excuse looks better, of course).
Yeah, I'll probably do this. Good idea. :-)

Thanks for your help.

Alejo.
http://azul.freaks-unidos.net/
Alejandro Forero Cuervo
2010-07-04 17:19:20 UTC
Permalink
If an extension called, say, 'hello' has been installed, what's the
canonical way, with Chicken 4, to load it?

I'm confused with require-extension, require-library, require, import,
use and declare uses. I'm looking for an answer that only has two
words, the second being "hello", and two parenthesis. I would hate to
find out that one must use the words like "depends" to answer this
question.

Alejo.
http://azul.freaks-unidos.net/

Ps: I'm purposefully not specifying if 'hello' has macros, uses
'(module ...)', etc..

PPs: http://chicken.wiki.br/eggs%20tutorial seems confusing, it talks
about mpeg.import.scm without specifying that it's an autogenerated
file and it assumes that mpeg.scm uses "(module ...)", which may not
be the case. I'd recommend fixing that or erasing that page
(replacing it with a link to the proper documentation that egg authors
should use).
Alejandro Forero Cuervo
2010-07-04 22:33:58 UTC
Permalink
I'm experimenting with my format-compiler extension. I got past the
limitation preventing the module system from dealing with recursively
dependent modules by moving all the functionality of my format
compiler to format-compiler-base, the unit tests (using the
embedded-test framework) to format-compiler and making embedded-test
depend in format-compiler-base.

Now I'm stuck in another problem, it seems to be caused by a bug in
let-optionals. In Chicken 3.4.0, the following form evaluates to #f,
as expected:

(let-optionals '(#f) ((rest #f)) #f)

In Chicken 4.5.0, both csi and in compiled code, it yields an error:

Error: too many optional arguments: #f

The following, however, works:

(let-optionals '(#f) ((rest2 #f)) #f)

Does anybody use let-optionals? Is this a known issue fixed in newer
versions? It strikes me as surprising that such simple errors could go
undetected. Am I supposed to use some form other than let-optionals
for the kind of thing it does?

I guess I'll workaround this by making sure to never name any
variables “rest” —adding yet another to the list of workarounds I've
had to make to keep my Chicken code working—, but I figured I'd report
the problem, I think it'd be best if it was fixed in Chicken, to save
others from wasting hours like I just did.

Thanks!

Alejo.
http://azul.freaks-unidos.net/
Alejandro Forero Cuervo
2010-07-05 11:04:02 UTC
Permalink
I have a module embedded-test that exports a procedure called
'register' and a macro called 'test' that transforms a form into a
call to 'register'. It looks like this (I'm simplifying a lot):

(module embedded-test (test register)
(import chicken scheme)

(define (register) #f)
(define-syntax test (syntax-rules () ((test) (register))))
)

I install the extension and can use it from csi without problems but
when I try the following compiled program,

(use embedded-test)
(test)

I get:

Error: unbound variable: embedded-test#register

Call history:
embedded-test#register <--

Am I doing something wrong? I've tried various combinations of 'use',
'import', 'require-library', 'import-for-syntax' and
'require-extension' to no avail. How does one make a macro transform
a form into a call to a procedure from its own module (that the module
exports)?

In case the answer is of the form "Chicken does not support this, what
are you trying to do?", what I'm trying to do is let the programmer
say things like (test (+ 2 3) 5) and have that transformed into
something like (register-test '(+ 2 3) (lambda () (+ 2 3)) (lambda ()
5) equal?). If there are better ways to do this, I'm all ears.

For my format-compiler, I workarounded the let-optionals bug by
renaming 'rest' to 'rest-xrenamed' throughout my code but now I'm
stuck with this problem. Any help would be appreciated.

Thanks!

Alejo.
http://azul.freaks-unidos.net/
Peter Bex
2010-07-05 11:25:51 UTC
Permalink
Post by Alejandro Forero Cuervo
Am I doing something wrong? I've tried various combinations of 'use',
'import', 'require-library', 'import-for-syntax' and
'require-extension' to no avail. How does one make a macro transform
a form into a call to a procedure from its own module (that the module
exports)?
I didn't get the error you're getting, but I think what you need is this
section from the manual:

"EXPORT may be a symbol or a list of the form
(IDENTIFIER1 IDENTIFIER2 ...). In the former case the identifier given
is exported from the module and can be imported at the toplevel or in
other modules. The latter case exports IDENTIFIER1 (which should name
a macro) and also arranges for the remaining identifiers in the list
to be visible in the expansion of the macro (this is a hint to the
module expander to export bindings referenced by syntax-definitions
which make use of them, but which would normally be internal to the
module - which gives more opportunities for optimization)."
--- http://chicken.wiki.br/man/4/Modules%20and%20macros

In other words, do (module embedded-test ((test register)) ...)

I think it worked because the code you pasted in the mail also exported
REGISTER, which your real code perhaps does not?

Cheers,
Peter
--
http://sjamaan.ath.cx
--
"The process of preparing programs for a digital computer
is especially attractive, not only because it can be economically
and scientifically rewarding, but also because it can be an aesthetic
experience much like composing poetry or music."
-- Donald Knuth
Alejandro Forero Cuervo
2010-07-05 11:59:13 UTC
Permalink
Post by Peter Bex
I didn't get the error you're getting, but I think what you need is this
[snip]
In other words, do (module embedded-test ((test register)) ...)
I think it worked because the code you pasted in the mail also exported
REGISTER, which your real code perhaps does not?
Hm, very strange.

I'm literally testing with this (after your suggestion):

(module embedded-test ((test register) register)
(import chicken scheme)

(define (register) #f)
(define-syntax test (syntax-rules () ((test) (register))))
)

The setup file does this:

/opt/chicken-4.5.0/bin/csi -bnq -setup-mode -e "(require-library setup-api)" -e "(import setup-api)" -e "(extension-name-and-version '(\"embedded-test\" \"\"))" embedded-test.setup
/opt/chicken-4.5.0/bin/csc -feature compiling-extension -setup-mode -O2 -d1 -s embedded-test.scm -j embedded-test
/opt/chicken-4.5.0/bin/csc -feature compiling-extension -setup-mode -O2 -d1 -c embedded-test.scm -unit embedded-test
/opt/chicken-4.5.0/bin/csc -feature compiling-extension -setup-mode -O2 -d0 -s embedded-test.import.scm
cp -r embedded-test.o /opt/chicken-4.5.0/lib/chicken/5/embedded-test.o
chmod a+r /opt/chicken-4.5.0/lib/chicken/5/embedded-test.o
rm -fr /opt/chicken-4.5.0/lib/chicken/5/embedded-test.import.so
cp -r embedded-test.import.so /opt/chicken-4.5.0/lib/chicken/5/embedded-test.import.so
chmod a+r /opt/chicken-4.5.0/lib/chicken/5/embedded-test.import.so
rm -fr /opt/chicken-4.5.0/lib/chicken/5/embedded-test.so
cp -r embedded-test.so /opt/chicken-4.5.0/lib/chicken/5/embedded-test.so
chmod a+r /opt/chicken-4.5.0/lib/chicken/5/embedded-test.so
chmod a+r /opt/chicken-4.5.0/lib/chicken/5/embedded-test.setup-info

The generated embedded-test.import.scm has this:

(eval '(import chicken scheme))
(##sys#register-compiled-module
'embedded-test
(list '(register . embedded-test#register))
'((register . embedded-test#register))
(list (cons 'test (syntax-rules () ((test) (register)))))
(list))

And yet, /opt/chicken-4.5.0/bin/csc test.scm && ./test yields the error I
quoted. :-/ Are you using 4.5.0?

Thanks a lot for your help!

Alejo.
http://azul.freaks-unidos.net/
Peter Bex
2010-07-05 12:08:10 UTC
Permalink
Post by Alejandro Forero Cuervo
(eval '(import chicken scheme))
(##sys#register-compiled-module
'embedded-test
(list '(register . embedded-test#register))
'((register . embedded-test#register))
(list (cons 'test (syntax-rules () ((test) (register)))))
(list))
And yet, /opt/chicken-4.5.0/bin/csc test.scm && ./test yields the error I
quoted. :-/ Are you using 4.5.0?
Yes, I'm using 4.5.0, but I didn't install the extension. I just ran
(use embedded-test) from the directory where I compiled it.

Could you send me a tarball containing the full egg as-is, including the
setup and meta files?

Cheers,
Peter
--
http://sjamaan.ath.cx
--
"The process of preparing programs for a digital computer
is especially attractive, not only because it can be economically
and scientifically rewarding, but also because it can be an aesthetic
experience much like composing poetry or music."
-- Donald Knuth
Alejandro Forero Cuervo
2010-07-05 13:42:46 UTC
Permalink
Post by Peter Bex
Yes, I'm using 4.5.0, but I didn't install the extension. I just ran
(use embedded-test) from the directory where I compiled it.
Could you send me a tarball containing the full egg as-is, including the
setup and meta files?
I'm attaching it to this message. Here is how I use it:

tar xvfz embedded-test.tar.gz
cd embedded-test
/opt/chicken-4.5.0/bin/chicken-install
/opt/chicken-4.5.0/bin/csc test.scm
./test

Errors! :-(

Thanks a lot for your help! :-)

Alejo.
http://azul.freaks-unidos.net/
Alejandro Forero Cuervo
2010-07-05 13:45:03 UTC
Permalink
Oh, BTW, the meta file has a "needs" entry that you'll probably want
to remove while testing this.

Alejo.
http://azul.freaks-unidos.net/
Stephen Eilert
2010-07-05 13:55:47 UTC
Permalink
Try

(define-for-syntax (register) #f)


--Stephen

Sent from my Emacs



On Mon, Jul 5, 2010 at 10:45 AM, Alejandro Forero Cuervo
Post by Alejandro Forero Cuervo
Oh, BTW, the meta file has a "needs" entry that you'll probably want
to remove while testing this.
Alejo.
http://azul.freaks-unidos.net/
_______________________________________________
Chicken-users mailing list
http://lists.nongnu.org/mailman/listinfo/chicken-users
Felix
2010-07-05 13:57:33 UTC
Permalink
From: Alejandro Forero Cuervo <***@freaks-unidos.net>
Subject: Re: [Chicken-users] Exporting proc and macro that references it?
Date: Mon, 5 Jul 2010 13:59:13 +0200
Post by Alejandro Forero Cuervo
Post by Peter Bex
I didn't get the error you're getting, but I think what you need is this
[snip]
In other words, do (module embedded-test ((test register)) ...)
I think it worked because the code you pasted in the mail also exported
REGISTER, which your real code perhaps does not?
Hm, very strange.
(module embedded-test ((test register) register)
(import chicken scheme)
(define (register) #f)
(define-syntax test (syntax-rules () ((test) (register))))
)
Note: just using (test register) as export-list is fine, since
you are exporting register, anyway. You only need that if register
shouldn't be exported.
Post by Alejandro Forero Cuervo
And yet, /opt/chicken-4.5.0/bin/csc test.scm && ./test yields the error I
quoted. :-/ Are you using 4.5.0?
Your .setup file installs embedded-test as `syntax' (see property-list
in the invocation if `install-extension'), which means the extension
has no runtime part (and thus nothing is loaded at runtime to
provide the necessary runtime code). Remove the `(syntax)' and it
works.


cheers,
felix
Alejandro Forero Cuervo
2010-07-05 21:12:09 UTC
Permalink
Post by Felix
Post by Alejandro Forero Cuervo
Post by Peter Bex
I didn't get the error you're getting, but I think what you need is this
[snip]
In other words, do (module embedded-test ((test register)) ...)
I think it worked because the code you pasted in the mail also exported
REGISTER, which your real code perhaps does not?
Hm, very strange.
(module embedded-test ((test register) register)
(import chicken scheme)
(define (register) #f)
(define-syntax test (syntax-rules () ((test) (register))))
)
Note: just using (test register) as export-list is fine, since
you are exporting register, anyway. You only need that if register
shouldn't be exported.
OK. Makes perfect sense.
Post by Felix
Post by Alejandro Forero Cuervo
And yet, /opt/chicken-4.5.0/bin/csc test.scm && ./test yields the error I
quoted. :-/ Are you using 4.5.0?
Your .setup file installs embedded-test as `syntax' (see property-list
in the invocation if `install-extension'), which means the extension
has no runtime part (and thus nothing is loaded at runtime to
provide the necessary runtime code). Remove the `(syntax)' and it
works.
Ah! So that's what was breaking it, I see. Thanks for your help, it
works now!

Alejo.
http://azul.freaks-unidos.net/
Felix
2010-07-05 12:34:41 UTC
Permalink
From: Alejandro Forero Cuervo <***@freaks-unidos.net>
Subject: [Chicken-users] bug in let-optionals in 4.5.0?
Date: Mon, 5 Jul 2010 00:33:58 +0200
Post by Alejandro Forero Cuervo
Now I'm stuck in another problem, it seems to be caused by a bug in
let-optionals. In Chicken 3.4.0, the following form evaluates to #f,
(let-optionals '(#f) ((rest #f)) #f)
Error: too many optional arguments: #f
(let-optionals '(#f) ((rest2 #f)) #f)
Does anybody use let-optionals? Is this a known issue fixed in newer
versions? It strikes me as surprising that such simple errors could go
undetected. Am I supposed to use some form other than let-optionals
for the kind of thing it does?
This is caused by a bug in the implementation of `let-optionals'
(I inadvertently made two different temporaries have the same name).
A patch is attached - thanks for reporting this!


cheers,
felix
Alejandro Forero Cuervo
2010-07-05 21:17:29 UTC
Permalink
Thanks for the quick fix!

Can I assume that this will be fixed in the next Chicken release? Do
you have a rough estimate of when the next release will be made?

Thanks!

Alejo.
http://azul.freaks-unidos.net/
Felix
2010-07-06 11:02:50 UTC
Permalink
From: Alejandro Forero Cuervo <***@freaks-unidos.net>
Subject: Re: [Chicken-users] bug in let-optionals in 4.5.0?
Date: Mon, 5 Jul 2010 23:17:29 +0200
Post by Alejandro Forero Cuervo
Thanks for the quick fix!
Can I assume that this will be fixed in the next Chicken release? Do
you have a rough estimate of when the next release will be made?
The patch is already in the "master" branch and will be included
in the next release. A release isn't planned in the moment, but
development snapshots are made frequently (it just needs a tag
in the repository). Do you need a tarball to build from?


cheers,
felix
F. Wittenberger
2010-07-05 11:09:34 UTC
Permalink
Post by Alejandro Forero Cuervo
I'm experimenting with my format-compiler extension. I got past the
limitation preventing the module system from dealing with recursively
dependent modules
I would not call this a limitation. IMHO a module system, which allows
mutual depending modules is a pointless as a compiler with doesn't check
syntax. It will take you quite far, until you realize that you code has
been broken all the time.
Post by Alejandro Forero Cuervo
by moving...
but you've got *the* solution already.
Post by Alejandro Forero Cuervo
Now I'm stuck in another problem, it seems to be caused by a bug in
let-optionals. In Chicken 3.4.0, the following form evaluates to #f,
(let-optionals '(#f) ((rest #f)) #f)
Apparently nobody uses let-optionals with a "rest"-Variable.

in chicken-syntax.scm around line 641 you find:

;; A private var, bound to the value auf the ARG-LIST expression
(rest-var (r '%rest))

Change this to

(rest-var (r '%%rest))

and you are done.

/Jerry
Felix
2010-07-05 13:11:46 UTC
Permalink
From: Alejandro Forero Cuervo <***@freaks-unidos.net>
Subject: [Chicken-users] How to load an extension?
Date: Sun, 4 Jul 2010 19:19:20 +0200
Post by Alejandro Forero Cuervo
If an extension called, say, 'hello' has been installed, what's the
canonical way, with Chicken 4, to load it?
(require-extension hello)
Post by Alejandro Forero Cuervo
I'm confused with require-extension, require-library, require, import,
use and declare uses. I'm looking for an answer that only has two
words, the second being "hello", and two parenthesis. I would hate to
find out that one must use the words like "depends" to answer this
question.
(require-extension hello)

"hello" in this case should be a normal, canonical, standard extension
library for the CHICKEN Scheme system. It's always `require-extension'
for these. When in doubt, use `require-extension'.

If the extension does not provide a module (which implies it also
doesn't provide an import library), which means it is *NOT* a normal,
canonical, standard extension library for the CHICKEN Scheme system,
then of course, it depends.
Post by Alejandro Forero Cuervo
PPs: http://chicken.wiki.br/eggs%20tutorial seems confusing, it talks
about mpeg.import.scm without specifying that it's an autogenerated
file and it assumes that mpeg.scm uses "(module ...)", which may not
be the case. I'd recommend fixing that or erasing that page
(replacing it with a link to the proper documentation that egg authors
should use).
I will try to improve this page. All existing documentation is indeed
confusing, and everybody (including myself) gets regularly confused
about the correct way to use extensions, but that is caused by
some things that, AFAICT, can not be easily circumvented:

1. There exists an alias to `require-extension' called `use', which is
totally equivalent, but short. I wouldn't want to remove this,
because, lazy as I am, I try to reduce keypresses here and there,
particularly for boilerplate forms like this.

2. CHICKEN supports static linking of extensions in a relatively
convenient manner, something very few language implementations
provide. Since this influences translation, compilation,
linking/loading, extension-building and extension-installation, it
necessarily requires some machinery (even if controlled by obscure
command-line-options or non-standard loading/linking forms). Note
that you don't have to worry about this, if you are just trying to
put together normal extensions.

3. It is also possible to link library units into a single shared
library (that's what libchicken.so is, in the end), this and static
linking requires all this "unit" stuff that appears in the manual.

So things are actually not that bad:

(require-extension X)/(use X) loads and imports an extension module.

(require-library X) loads an extension, but doesn't import.

(import X) imports a module, which must have been loaded somehow.

Just ignore the rest. Don't worry about static linking until you need
it. Write extensions that use `(module ...)' and all is well.


cheers,
felix
Alejandro Forero Cuervo
2010-07-05 21:20:15 UTC
Permalink
This makes perfect sense. Thanks for the explanation.
Post by Felix
Post by Alejandro Forero Cuervo
If an extension called, say, 'hello' has been installed, what's the
canonical way, with Chicken 4, to load it?
(require-extension hello)
Post by Alejandro Forero Cuervo
I'm confused with require-extension, require-library, require, import,
use and declare uses. I'm looking for an answer that only has two
words, the second being "hello", and two parenthesis. I would hate to
find out that one must use the words like "depends" to answer this
question.
(require-extension hello)
"hello" in this case should be a normal, canonical, standard extension
library for the CHICKEN Scheme system. It's always `require-extension'
for these. When in doubt, use `require-extension'.
If the extension does not provide a module (which implies it also
doesn't provide an import library), which means it is *NOT* a normal,
canonical, standard extension library for the CHICKEN Scheme system,
then of course, it depends.
Post by Alejandro Forero Cuervo
PPs: http://chicken.wiki.br/eggs%20tutorial seems confusing, it talks
about mpeg.import.scm without specifying that it's an autogenerated
file and it assumes that mpeg.scm uses "(module ...)", which may not
be the case. I'd recommend fixing that or erasing that page
(replacing it with a link to the proper documentation that egg authors
should use).
I will try to improve this page. All existing documentation is indeed
confusing, and everybody (including myself) gets regularly confused
about the correct way to use extensions, but that is caused by
1. There exists an alias to `require-extension' called `use', which is
totally equivalent, but short. I wouldn't want to remove this,
because, lazy as I am, I try to reduce keypresses here and there,
particularly for boilerplate forms like this.
2. CHICKEN supports static linking of extensions in a relatively
convenient manner, something very few language implementations
provide. Since this influences translation, compilation,
linking/loading, extension-building and extension-installation, it
necessarily requires some machinery (even if controlled by obscure
command-line-options or non-standard loading/linking forms). Note
that you don't have to worry about this, if you are just trying to
put together normal extensions.
3. It is also possible to link library units into a single shared
library (that's what libchicken.so is, in the end), this and static
linking requires all this "unit" stuff that appears in the manual.
(require-extension X)/(use X) loads and imports an extension module.
(require-library X) loads an extension, but doesn't import.
(import X) imports a module, which must have been loaded somehow.
Just ignore the rest. Don't worry about static linking until you need
it. Write extensions that use `(module ...)' and all is well.
Alejo.
http://azul.freaks-unidos.net/
Alejandro Forero Cuervo
2010-07-06 09:55:25 UTC
Permalink
OK, with great help from many of you, —for which I'm very grateful—, I
managed to get the format-compiler and the embedded-test eggs to work
in Chicken 4.5.0. chicken-install can now correctly download and
install them.

The embedded-test egg provides a simple interface to define unit tests
for procedures directly before the procedures themselves are defined.
There's more information about it here:

http://wiki.freaks-unidos.net/embedded-test

The format-compiler egg, based somewhat on the format-modular egg,
compiles format strings according to specifications of supported
escape sequences. It provides a format function, implementing Common
Lisp's format interface. format-compiler is not yet complete, there
is a non-trivial amount of escape sequences that I have yet to get
supported (I would estimate about 25% of the work remains). I also
have yet to modify it's interface to make it easier to extend it.
There's more information about it here:

http://wiki.freaks-unidos.net/weblogs/azul/format-compiler

The authoritative source for both eggs are the aforementioned URLs.
The entire code is embedded in those pages and all the files making up
the egg are automatically generated from them by Svnwiki. As I've
mentioned in the past, this is my experiment for a wiki-based open
source literate programming framework for Scheme. We'll see how it
goes.

In the course of the next few weeks I expect to continue porting my
eggs from Chicken 3 to this framework and to Chicken 4. We'll see how
it goes.

Thanks for all your help. Note that the meta and setup files are
entirely autogenerated: if you have any suggestions to improve them,
feel free to send them my way.

Alejo.
http://azul.freaks-unidos.net/
Felix
2010-07-06 13:35:56 UTC
Permalink
From: Alejandro Forero Cuervo <***@freaks-unidos.net>
Subject: [Chicken-users] embedded-test and format-compiler ported to chicken 4
Date: Tue, 6 Jul 2010 11:55:25 +0200
Post by Alejandro Forero Cuervo
OK, with great help from many of you, —for which I'm very grateful—, I
managed to get the format-compiler and the embedded-test eggs to work
in Chicken 4.5.0. chicken-install can now correctly download and
install them.
Excellent! I will take a look indeed.
Post by Alejandro Forero Cuervo
Thanks for all your help. Note that the meta and setup files are
entirely autogenerated: if you have any suggestions to improve them,
feel free to send them my way.
The `dynld-name' stuff shouldn't be needed. Dynamically loadable code
has the ".so" extension o
Alejandro Forero Cuervo
2010-07-08 09:12:26 UTC
Permalink
Post by Felix
The `dynld-name' stuff shouldn't be needed. Dynamically loadable code
has the ".so" extension on all platforms.
Ahh, cool, I'll get rid of that now.

Alejo.
http://azul.freaks-unidos.net/
Alejandro Forero Cuervo
2010-07-08 09:35:56 UTC
Permalink
I've created another egg for Chicken 4, simple-logging, which provides
a relatively simple interface to log events during the lifespan of a
process. Events have different severities, with a separate log
generated for each. The system only keeps a certain size of logs,
with the most recent entries, for each severity (by default it keeps
between 2 and 3MB). There's more information about it here:

http://azul.freaks-unidos.net/simple-logging

I've recently started using it for Svnwiki (I'm using the analogous
egg under Chicken 3) and it has greatly helped me understand specific
production issues. You may want to consider using it for your
applications.

I implemented it based on the main ideas of Logging section that I
added on March to my Principles of Software article (which I have yet
to update, to list this egg):

http://azul.freaks-unidos.net/principles-of-software#logging

Thanks.

Alejo.
http://azul.freaks-unidos.net/

Felix
2010-06-15 13:29:30 UTC
Permalink
From: Alejandro Forero Cuervo <***@freaks-unidos.net>
Subject: [Chicken-users] Working on format-compiler egg, question about macros, eggs as wiki pages
Date: Sun, 13 Jun 2010 22:36:25 +0200
Post by Alejandro Forero Cuervo
As part of my migration from Chicken 3 to Chicken 4 I'm rewritting the
format-modular egg into what I'm calling 'format-compiler'. I'm
reverting many changes from it and I'm cleaning up some of the
interfaces, to hopefully make it easier to extend (ie. to define
control characters that do interesting things). I'm also fixing some
bugs I've found and removing a lot of unnecessary complexity that has
been gradually added to it with little purpose. I have a draft of
http://azul.freaks-unidos.net/format-compiler
Very good! A solid implementation of CL-style formatting is dearly
needed, since the `format' egg is buggy and the basic `[sf]printf'
stuff is not particularly powerful. There is Shinn's formidable `fmt',
which I like to get used to (but haven't used it intensively enough).
In the end I always use `print' and `display' and normally that's
enough.

The complexity of the current `format-modular' code isn't so much the
problem - the large number of dependencies is. It would be good to
remove i18n support and for example provide hooks or another sort of
interface to extend it in a clean manner without increasing the size
and the number of base-dependencies of a bread-and-butter extension
like formatted text output.
Post by Alejandro Forero Cuervo
I'm doing a pretty large change to it: I'm separating the process of
parsing the format string rom the process of actually generating the
output. My thinking is that most of the time format strings are
specified statically and never change during the execution of a
(format (current-error-port) "Evaluating ~A form: ~A~%" adj name)
Instead of parsing the string every time, I want to parse it once and
then use the result of that parsing on the arguments passed.
Cool - this reminds me of Common Lisp's `formatter' macro. By
seperating the format-compiler into a module, it would be possible
to use it at compile time and provide a similar facility by using
compiler-syntax.
Post by Alejandro Forero Cuervo
Interestingly, this is actually allowing me to simplify the
implementation significantly (or maybe I'm a slightly better
programmer now than I was when I wrote format-modular?).
Well, I don't know about now, but I always envied the clean
coding style you use. :-)
Post by Alejandro Forero Cuervo
The catch is, of course, that one may memoize fmt-compiler —perhaps
using “eq?” on the format string (for which I'll probably extend the
memoize egg)— which means that a given statically specified format
string would be parsed at most once and the results of that parsing
reused through-out.
In order to make this happen, I figured I would ask: is there a way
for, in a macro, check if a given symbol is actually a hard-coded (ie.
static) string? In other words, if my macro is called “format”, I
would like “(format x "static" ...)” and “(format x foo ...)” to have
different expansions, where the expansion for the former caches the
results of the parsing forever and the expansion of the later only
caches them for some time, with some heuristics. Is this possible at
all?
Sure:

(define-syntax (a-string? x r c) (string? (cadr x)))

(a-string? "abc") ~~ expands to ~~> #t

To allow using `format' like a nomrmal function (if in non-operator
position), I recommend using compiler-syntax
(`define-compiler-syntax') for `format' and the mentioned separate
module that allows using the format-compiler at
compile-time. Exporting compiler-syntax cleanly from a module is a bit
tricky, though - I will try to figure this out.
Post by Alejandro Forero Cuervo
I will add this code as an egg for both Chicken 3 and Chicken 4 once
it's ready. I still have to add support for a few control characters.
Great!
Post by Alejandro Forero Cuervo
This is also interesting for me in that it is an experiment of storing
an egg entirely in a wiki page. For example, the page has the
<schemelibrarydefinition
author="Alejandro Forero Cuervo"
category="io"
synopsis="Creates format procedures to generate output based on format strings."
license="Public Domain"
name="format-compiler"
uses="posix embedded-test"
exports="*formatter-params* ..."/>
This is enough for Svnwiki to generate the setup, meta and Scheme
source code files (with the blocks embedded in the wiki page) for me
to just copy into the eggs repository. In that sense, the wiki page
itself becomes the authoritative location for the egg. See, for
http://wiki.freaks-unidos.net/weblogs/azul/xsvnwiki-enscript/format-compiler/format-compiler.scm
http://wiki.freaks-unidos.net/weblogs/azul/xsvnwiki-enscript/format-compiler/format-compiler.meta
http://wiki.freaks-unidos.net/weblogs/azul/xsvnwiki-enscript/format-compiler/format-compiler.setup
So this is effectively like a literate progr
Alejandro Forero Cuervo
2010-06-15 15:10:33 UTC
Permalink
Post by Alejandro Forero Cuervo
As part of my migration from Chicken 3 to Chicken 4 I'm rewritting the
format-modular egg into what I'm calling 'format-compiler'. I'm
reverting many changes from it and I'm cleaning up some of the
interfaces, to hopefully make it easier to extend (ie. to define
control characters that do interesting things). I'm also fixing some
bugs I've found and removing a lot of unnecessary complexity that has
been gradually added to it with little purpose. I have a draft of
http://azul.freaks-unidos.net/format-compiler
Very good! A solid implementation of CL-style formatting is dearly
needed, since the `format' egg is buggy and the basic `[sf]printf'
stuff is not particularly powerful. There is Shinn's formidable `fmt',
which I like to get used to (but haven't used it intensively enough).
In the end I always use `print' and `display' and normally that's
enough.
The complexity of the current `format-modular' code isn't so much the
problem - the large number of dependencies is. It would be good to
remove i18n support and for example provide hooks or another sort of
interface to extend it in a clean manner without increasing the size
and the number of base-dependencies of a bread-and-butter extension
like formatted text output.
Yes! I have removed i18n support from it, something that I think
should never have been added to it. It had been added to output
numbers in different languages, but this was clearly broken: I wrote a
bit about it here:

http://wiki.freaks-unidos.net/weblogs/azul/format-compiler#english-output

I want to come up with an interface for people to define other
format-compiler-* eggs that extend the functionality of the default
format function provided by format-compiler if they are installed
(akin to how the svnwiki extensions (since I completely redefined the
interface for extensions), if installed (and enabled), extend the
logic of different hooks in svnwiki).
Post by Alejandro Forero Cuervo
I'm doing a pretty large change to it: I'm separating the process of
parsing the format string rom the process of actually generating the
output. My thinking is that most of the time format strings are
specified statically and never change during the execution of a
(format (current-error-port) "Evaluating ~A form: ~A~%" adj name)
Instead of parsing the string every time, I want to parse it once and
then use the result of that parsing on the arguments passed.
Cool - this reminds me of Common Lisp's `formatter' macro. By
seperating the format-compiler into a module, it would be possible
to use it at compile time and provide a similar facility by using
compiler-syntax.
Post by Alejandro Forero Cuervo
Interestingly, this is actually allowing me to simplify the
implementation significantly (or maybe I'm a slightly better
programmer now than I was when I wrote format-modular?).
Well, I don't know about now, but I always envied the clean
coding style you use. :-)
Thanks, Felix!
Post by Alejandro Forero Cuervo
The catch is, of course, that one may memoize fmt-compiler —perhaps
using “eq?” on the format string (for which I'll probably extend the
memoize egg)— which means that a given statically specified format
string would be parsed at most once and the results of that parsing
reused through-out.
In order to make this happen, I figured I would ask: is there a way
for, in a macro, check if a given symbol is actually a hard-coded (ie.
static) string? In other words, if my macro is called “format”, I
would like “(format x "static" ...)” and “(format x foo ...)” to have
different expansions, where the expansion for the former caches the
results of the parsing forever and the expansion of the later only
caches them for some time, with some heuristics. Is this possible at
all?
(define-syntax (a-string? x r c) (string? (cadr x)))
(a-string? "abc") ~~ expands to ~~> #t
To allow using `format' like a nomrmal function (if in non-operator
position), I recommend using compiler-syntax
(`define-compiler-syntax') for `format' and the mentioned separate
module that allows using the format-compiler at
compile-time. Exporting compiler-syntax cleanly from a module is a bit
tricky, though - I will try to figure this out.
I will read about this. I'm still a bit confused about the semantics
for macros in Chicken 3 and Chicken 4. Hopefully I'll be able to
figure out how it works. May I abuse you by asking you what the
canonical location for the documentation about macros in Chicken 4 is?
So this is effectively like a literate programming tool? An
interesting idea.
That's exactly what it is. I like literate programming. This is my
attempt to create a framework for development of Scheme code combining
some principles from literate programming, wikis as a collaborative
medium and unit testing (the tests are embedded there in the wiki
page, actually). I have some tools to assist this apart from the
logic directly in Svnwiki.

Thanks for your feedback!

Alejo.
http://azul.freaks-unidos.net/
Felix
2010-06-19 22:01:19 UTC
Permalink
From: Alejandro Forero Cuervo <***@freaks-unidos.net>
Subject: Re: [Chicken-users] Working on format-compiler egg, question about macros, eggs as wiki pages
Date: Tue, 15 Jun 2010 17:10:33 +0200
Post by Alejandro Forero Cuervo
I will read about this. I'm still a bit confused about the semantics
for macros in Chicken 3 and Chicken 4. Hopefully I'll be able to
figure out how it works. May I abuse you by asking you what the
canonical location for the documentation about macros in Chicken 4 is?
See

http://chicken.wiki.br/man/4/Modules and macros.html

If you have any questions, please feel free to ask.


cheers,
felix
Loading...