1/* Part of SWI-Prolog 2 3 Author: Jan Wielemaker 4 E-mail: J.Wielemaker@vu.nl 5 WWW: http://www.swi-prolog.org 6 Copyright (c) 2007-2016, University of Amsterdam 7 VU University Amsterdam 8 All rights reserved. 9 10 Redistribution and use in source and binary forms, with or without 11 modification, are permitted provided that the following conditions 12 are met: 13 14 1. Redistributions of source code must retain the above copyright 15 notice, this list of conditions and the following disclaimer. 16 17 2. Redistributions in binary form must reproduce the above copyright 18 notice, this list of conditions and the following disclaimer in 19 the documentation and/or other materials provided with the 20 distribution. 21 22 THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 23 "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 24 LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS 25 FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE 26 COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, 27 INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, 28 BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; 29 LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER 30 CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 31 LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN 32 ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE 33 POSSIBILITY OF SUCH DAMAGE. 34*/ 35 36:- module(http_stream, 37 [ http_chunked_open/3, % +Stream, -DataStream, +Options 38 http_is_chunked/1, % +DataStream 39 http_chunked_flush/2, % +DataStream, +Extensions 40 http_chunked_add_trailer/3, % +DataStream, +Key, +Value 41 stream_range_open/3, % +Stream, -DataStream, +Options 42 multipart_open/3, % +Stream, +DataStream, +Options) 43 multipart_open_next/1, % +DataStream 44 45 % CGI Stream interaction 46 cgi_open/4, % +Stream, -DataStream, :Hook, +Options 47 cgi_property/2, % +Stream, -Property 48 cgi_set/2, % +Stream, -Property 49 cgi_discard/1, % +Stream 50 is_cgi_stream/1, % +Stream 51 cgi_statistics/1 % ?Statistics 52 ]). 53:- use_foreign_library(foreign(http_stream)). 54:- public http_stream_debug/1. % set debug level 55 56:- meta_predicate 57 stream_range_open(,,). % onclose option is module sensitive
true (default false), the parent stream is closed
if DataStream is closed.set_stream(DataStream, buffer(line)) on the data stream to
get line-buffered output. See set_stream/2 for details.
Switching buffering to false is supported.Here is example code to write a chunked data to a stream
http_chunked_open(Out, S, []),
format(S, 'Hello world~n', []),
close(S).
If a stream is known to contain chunked data, we can extract this data using
http_chunked_open(In, S, []),
read_stream_to_codes(S, Codes),
close(S).
The chunked protocol allows for two types of out of band data. Each chunk may be associated with additional metadata. That is achieved using http_chunked_flush/2. The last chunk may be followed by HTTP header lines. That can be achieved by calling http_chunked_add_trailer/3 before closing the chunked stream.
After http_chunked_open/3, the encoding of DataStream is the same as
the encoding of RawStream, while the encoding of RawStream is
octet, the only value allowed for HTTP chunked streams. Closing
the DataStream restores the old encoding on RawStream.
Key=Value terms. For example, to close a chunked stream with an
error chunk we something like below. First, we flush the last
pending data, next we fill a new chunk and flush it with
extensions.
flush_output(current_output),
format("Sorry, something went wrong!\n"),
http_chunked_flush(current_output, [error=true])
Key: Value\r\n to the output. The
strings for Key and Value need to be compliant with the HTTP
header syntax.
166 /******************************* 167 * RANGES * 168 *******************************/
size(ContentLength). Closing DataStream does not close
RawStream. Options processed:
call(Closure, RawStream, BytesLeft) when DataStream is
closed. BytesLeft is the number of bytes of the range stream
that have not been read, i.e., 0 (zero) if all data has been
read from the stream when the range is closed. This was
introduced for supporting Keep-alive in http_open/3 to
reschedule the original stream for a new request if the data
of the previous request was processed.188 /******************************* 189 * MULTIPART * 190 *******************************/
end_of_file if the
multipart boundary is encountered. The stream can be reset to
read the next part using multipart_open_next/1. Options:
All parts of a multipart input can be read using the following skeleton:
process_multipart(Stream) :-
multipart_open(Stream, DataStream, [boundary(...)]),
process_parts(DataStream).
process_parts(DataStream) :-
process_part(DataStream),
( multipart_open_next(DataStream)
-> process_parts(DataStream)
; close(DataStream)
).
232 /******************************* 233 * CGI SUPPORT * 234 *******************************/
call(Hook, header, Stream), where Stream is a stream holding
the buffered header.call(Hook, data, Stream), where Stream holds the buffered
data.The stream calls Hook, adding the event and CGIStream to the closure. Defined events are:
chunked or when the
CGI stream is closed. Typically it requests the current
header, optionally the content-length and sends the header
to the original (client) stream.The predicates cgi_property/2 and cgi_set/2 can be used to control the stream and store status info. Terms are stored as Prolog records and can thus be transferred between threads.
chunked or none.Keep-Alive or closenone.header, data or discardedsend_header hook to send the reply header to the
client.Keep-Alive or close.chunked or none. Initially set to none. When
switching to chunked from the header hook, it calls the
send_header hook and if there is data queed this is send
as first chunk. Each subsequent write to the CGI stream
emits a chunk. The implementation does not use the
chunked stream filter defined by http_chunked_open/3. It
shares most of the implementation though and CGI streams
do support http_is_chunked/1, http_chunked_flush/2 and
http_chunked_add_trailer/3.discarded, causing close to omit the writing the data. This
must be used for an alternate output (e.g. an error page) if the
page generator fails.339:- multifile 340 http:encoding_filter/3, % +Encoding, +In0, -In 341 http:current_transfer_encoding/1. % ?Encoding 342 343:- public 344 http:encoding_filter/3, 345 http:current_transfer_encoding/1.
chunked encoded messages. Used
by library(http_open).
352httpencoding_filter(chunked, In0, In) :-
353 http_chunked_open(In0, In,
354 [ close_parent(true)
355 ]).library(http_open).
361httpcurrent_transfer_encoding(chunked).373cgi_statistics(requests(Requests)) :- 374 cgi_statistics_(Requests, _). 375cgi_statistics(bytes_sent(Bytes)) :- 376 cgi_statistics_(_, Bytes)
HTTP Streams
This module realises encoding and decoding filters, implemented as Prolog streams that read/write to an underlying stream. This allows for sequences of streams acting as an in-process pipeline.
The predicate http_chunked_open/3 realises encoding and decoding of the HTTP Chunked encoding. This encoding is an obligatory part of the HTTP 1.1 specification. Messages are split into chunks, each preceeded by the length of the chunk. Chunked encoding allows sending messages over a serial link (typically a TCP/IP stream) for which the reader knows when the message is ended. Unlike standard HTTP though, the sender does not need to know the message length in advance. The protocol allows for sending short chunks. This is supported totally transparent using a flush on the output stream.
The predicate stream_range_open/3 handles the Content-length on an input stream for handlers that are designed to process an entire file. The filtering stream claims end-of-file after reading a specified number of bytes, dispite the fact that the underlying stream may be longer.