[Chicken-users] http-client: distinguishing between responses 301 and 302

[Norman Gray]

Greetings.

In http-client, is there any way of distinguishing between server
responses 301 and 302?

In test code, I want to confirm that a server is responding with the
correct redirection code, but can't quite get there.

If I make a request using call-with-input-request and the server
produces a redirection, then the client code follows the redirection.

If I parameterise that with (parameterize ((max-redirect-depth 0)) ...),
then the client library doesn't follow the redirection but instead
raises (exn http redirect-depth-exceeded). That's almost there, except
that the exception doesn't contain information about _which_
redirection, 301 or 302, prompted it. Using (condition->list), I get

redirect-depth-exceeded: e=#<condition: (exn http
redirect-depth-exceeded)>
list=(
(exn (arguments ("http://localhost:8081/host/"))
(message "Maximum number of redirects exceeded")
(location send-request))
(http)
(redirect-depth-exceeded
(request #<request>)
(new-uri #<URI-common: scheme=http port=8081 host="localhost" path=(/
"top" "host") query=() fragment=#f>)
(uri #<URI-common: scheme=http port=8081 host="localhost" path=(/
"host" "") query=() fragment=#f>)))

The documentation for call-with-input-request* doesn't suggest that it
behaves differently from call-with-input-request in this respect. And
although call-with-response mentions the possibility of redirects, and
multiple calls to 'reader', it doesn't suggest that the reader procedure
has any access to the response headers.

Is there any other way of getting this status code, or do I feel an
enhancement request coming on?

Possibly better than this handling of redirect-depth-exceeded would be a
parameter follow-redirects? which defaults to #t, or having
call-with-input-request return normally (ie, without the exception) in
the particular case that max-redirect-depth is 0.

I'm using Chicken 4, but as far as I can see, the Chicken 5 http-client
is identical in this behaviour.

Best wishes,

Norman

[John Cowan]

The difference between 301 and 302 is primarily relevant to crawlers and
caches. I agree that it needs to exist, but not clear that a
general-purpose client needs to expose it. Can you explain your use case
more clearly? Thanks.

Post by Norman Gray
Greetings.
In http-client, is there any way of distinguishing between server
responses 301 and 302?
In test code, I want to confirm that a server is responding with the
correct redirection code, but can't quite get there.
If I make a request using call-with-input-request and the server
produces a redirection, then the client code follows the redirection.
If I parameterise that with (parameterize ((max-redirect-depth 0)) ...),
then the client library doesn't follow the redirection but instead
raises (exn http redirect-depth-exceeded). That's almost there, except
that the exception doesn't contain information about _which_
redirection, 301 or 302, prompted it. Using (condition->list), I get
redirect-depth-exceeded: e=#<condition: (exn http
redirect-depth-exceeded)>
list=(
(exn (arguments ("http://localhost:8081/host/"))
(message "Maximum number of redirects exceeded")
(location send-request))
(http)
(redirect-depth-exceeded
(request #<request>)
(new-uri #<URI-common: scheme=http port=8081 host="localhost" path=(/
"top" "host") query=() fragment=#f>)
(uri #<URI-common: scheme=http port=8081 host="localhost" path=(/
"host" "") query=() fragment=#f>)))
The documentation for call-with-input-request* doesn't suggest that it
behaves differently from call-with-input-request in this respect. And
although call-with-response mentions the possibility of redirects, and
multiple calls to 'reader', it doesn't suggest that the reader procedure
has any access to the response headers.
Is there any other way of getting this status code, or do I feel an
enhancement request coming on?
Possibly better than this handling of redirect-depth-exceeded would be a
parameter follow-redirects? which defaults to #t, or having
call-with-input-request return normally (ie, without the exception) in
the particular case that max-redirect-depth is 0.
I'm using Chicken 4, but as far as I can see, the Chicken 5 http-client
is identical in this behaviour.
Best wishes,
Norman
--
Norman Gray : https://nxg.me.uk
_______________________________________________
Chicken-users mailing list
https://lists.nongnu.org/mailman/listinfo/chicken-users

[Jörg F. Wittenberger]

Post by John Cowan
The difference between 301 and 302 is primarily relevant to crawlers and
caches. I agree that it needs to exist, but not clear that a
general-purpose client needs to expose it. Can you explain your use case
more clearly? Thanks.

John, to me this does not look as if Norman was using the http-client for a
general purpose client. But more like a dev/testing tool.

Anyway. I'm looking into http-client with similar worries. Ages ago I wrote
some http client+server code (using rscheme at that time) which is since
used in a rather unusual setting: It is used to replicate incoming requests
to a network of machines each hosting many state machines (a.k.a.
communication sequential processes, a.k.a. agents) in byzantine fault
tolerant replication. (Things are always implemented in LISP first. Smart
contracts too. ;-)

Now I want to replace that mess I wrote over 15 years ago with an
alternative implementation I don't have to maintain alone. http-client
comes to mind.

Looking closer I don't like to install exception handlers in order to
handle the http status. That might be a matter of taste.

Then, when replicating a request, the same payload is send to several
receivers. I'd like to not repeat the serialization process. However in
order to account for different bandwidth to certain peers, there is the
need to half way synchronize the delivery of the last byte(s) of each
stream until at least the majority of the peers has received all of the
request.

Things I'd like to do with http-client. But it looks complicated.

So maybe the high level interface of http-client is simply not the one
Norman wants to use.

Best

/Jörg

Post by John Cowan

Post by Norman Gray
Greetings.
In http-client, is there any way of distinguishing between server
responses 301 and 302?
In test code, I want to confirm that a server is responding with the
correct redirection code, but can't quite get there.
If I make a request using call-with-input-request and the server
produces a redirection, then the client code follows the redirection.
If I parameterise that with (parameterize ((max-redirect-depth 0)) ...),
then the client library doesn't follow the redirection but instead
raises (exn http redirect-depth-exceeded). That's almost there, except
that the exception doesn't contain information about _which_
redirection, 301 or 302, prompted it. Using (condition->list), I get
redirect-depth-exceeded: e=#<condition: (exn http
redirect-depth-exceeded)>
list=(
(exn (arguments ("http://localhost:8081/host/"))
(message "Maximum number of redirects exceeded")
(location send-request))
(http)
(redirect-depth-exceeded
(request #<request>)
(new-uri #<URI-common: scheme=http port=8081 host="localhost" path=(/
"top" "host") query=() fragment=#f>)
(uri #<URI-common: scheme=http port=8081 host="localhost" path=(/
"host" "") query=() fragment=#f>)))
The documentation for call-with-input-request* doesn't suggest that it
behaves differently from call-with-input-request in this respect. And
although call-with-response mentions the possibility of redirects, and
multiple calls to 'reader', it doesn't suggest that the reader procedure
has any access to the response headers.
Is there any other way of getting this status code, or do I feel an
enhancement request coming on?
Possibly better than this handling of redirect-depth-exceeded would be a
parameter follow-redirects? which defaults to #t, or having
call-with-input-request return normally (ie, without the exception) in
the particular case that max-redirect-depth is 0.
I'm using Chicken 4, but as far as I can see, the Chicken 5 http-client
is identical in this behaviour.
Best wishes,
Norman
--
Norman Gray : https://nxg.me.uk
_______________________________________________
Chicken-users mailing list
https://lists.nongnu.org/mailman/listinfo/chicken-users

[Peter Bex]

Post by Jörg F. Wittenberger
Now I want to replace that mess I wrote over 15 years ago with an
alternative implementation I don't have to maintain alone. http-client comes
to mind.
Looking closer I don't like to install exception handlers in order to handle
the http status. That might be a matter of taste.

http-client was written as a simple to use "90% use cases" tool. That's
why it will handle redirects automatically and it doesn't have an easy
way to interfere with that.

For most people this is sufficient. If you have specific requirements
that can't be done in a clean way with http-client, it might make more
sense to use intarweb directly and perhaps copy and paste some code
from http-client.

Of course, if there's a clean way to extend http-client to support new
use cases, I'm always open to considering making some changes to make
it possible, but it should never interfere with common use cases.

Cheers,
Peter

[Norman Gray]

Peter and all, hello.

Post by Peter Bex
For most people this is sufficient. If you have specific requirements
that can't be done in a clean way with http-client, it might make more
sense to use intarweb directly and perhaps copy and paste some code
from http-client.

...and that's a better suggestion than my 'dumb' mode suggestion from
the previous message.

Best wishes,

Norman

[Norman Gray]

John, hello.

Post by John Cowan
The difference between 301 and 302 is primarily relevant to crawlers and
caches. I agree that it needs to exist, but not clear that a
general-purpose client needs to expose it. Can you explain your use case
more clearly?

It seems to me that the difference is relevant to users other than
crawlers and caches. As you know, 301 is saying 'this is the wrong URI;
don't come here again', but 302 is saying 'this is the right URI; see
over there for the details', and the difference between 'right URI' and
'wrong URI' is significant. A browser typically won't expose the
distinction to its user in those terms, but the browser should be able
to distinguish the cases.

Also, even if the distinction were indeed relevant only to caches and
crawlers, if I want to implement a crawler in Chicken, then I need to
have access to the distinct statuses. It's reasonable that the default
or simple behaviour is to blur the distinction, largely for the reason
that you suggest, but in a general-purpose client, the information needs
to be available somehow.

Also, as Jörg suggests in a message I see this one is crossing, I'm
writing testing code at this point, so need to be able to see what the
server actually said, rather than what the http-client library thinks it
intended.

As Peter says, this was clearly just an oversight in the http-client
library rather than a design decision. Thanks, Peter: I'll be able to
try the updated library shortly.

Though I understand why it was designed that way, it does feel slightly
awkward to me that I'm having to use an exception-handling mechanism to
get at this status distinction. A follow-redirects? parameter would
suit _my_ use-case very nicely, but I appreciate it may well have other
consequences for the interface. Similarly, a 'dumb' mode where the
library treated 2xx, 3xx, 4xx and 5xx statuses the same way, would work
very well for my case.

Best wishes,

Norman

[Peter Bex]

Post by Norman Gray
If I parameterise that with (parameterize ((max-redirect-depth 0)) ...),
then the client library doesn't follow the redirection but instead raises
(exn http redirect-depth-exceeded). That's almost there, except that the
exception doesn't contain information about _which_ redirection, 301 or 302,
prompted it.

Hi Norman,

This is definitely an oversight on my part; a http-client-error condition
object should always contain a response object when there is one.

I've just pushed 0.18 (for C4) and 1.1 (for C5), please give it a try.

Cheers,
Peter

[Norman Gray]

Peter, hello.

Post by Peter Bex
I've just pushed 0.18 (for C4) and 1.1 (for C5), please give it a try.

I can confirm that 0.18 works exactly as expected, on Chicken 4.

Best wishes,

Norman