1 <!-- DOCTYPE part PUBLIC "-//OASIS//DTD DocBook V3.1//EN" -->
2 <!-- =============================================================== -->
6 <!-- Another Tiny HTTPD Server for eCos -->
8 <!-- =============================================================== -->
9 <!-- ####COPYRIGHTBEGIN#### -->
11 <!-- =============================================================== -->
12 <!-- Copyright (C) 2003, 2004, 2007 eCosCentric Ltd. -->
13 <!-- This material may be distributed only subject to the terms -->
14 <!-- and conditions set forth in the Open Publication License, v1.0 -->
15 <!-- or later (the latest version is presently available at -->
16 <!-- http://www.opencontent.org/openpub/) -->
17 <!-- =============================================================== -->
19 <!-- ####COPYRIGHTEND#### -->
20 <!-- =============================================================== -->
21 <!-- #####DESCRIPTIONBEGIN#### -->
23 <!-- ####DESCRIPTIONEND#### -->
24 <!-- =============================================================== -->
30 <title>Another Tiny HTTP Server for <productname>eCos</productname></title>
34 This package provides an extensible, small footprint, full featured HTTP
35 server for <productname>eCos</productname>. Many of these features can be
36 disabled via the configuration tool, thus reducing the footprint of the server.
37 The server has been written for the FreeBSD network stack.
41 <chapter id="net-athttpd">
42 <title>The ATHTTP Server</title>
43 <sect1 id="athttpd-features">
44 <title>Features</title>
45 <para>This ATHTTP implementation provides the following features:</para>
47 <listitem><para>GET, POST and HEAD Methods</para></listitem>
48 <listitem><para>File system Access</para></listitem>
49 <listitem><para>Callbacks to C functions</para></listitem>
50 <listitem><para>MIME type support</para></listitem>
51 <listitem><para>CGI mechanism through the OBJLOADER package or through a
52 simple tcl interpreter</para></listitem>
53 <listitem><para>Basic and Digest (MD5) Authentication</para></listitem>
54 <listitem><para>Directory Listing</para></listitem>
55 <listitem><para>Extendable Internal Resources</para></listitem>
59 Ecos tables are used extensively throught the server to provide a high degree
60 of customization.</para>
63 <sect1 id="athttpd-using">
64 <title>Starting the server</title>
66 In order to start the web server, the user needs to call the function:</para>
68 <programlisting width=72>
72 <para>in the application code. The server initialization code spawns a new
73 thread which calls <command>init_all_network_interfaces()</command> to
74 initialize the TCP/IP stack and then starts the deamon. The function is safe
75 to call multiple times.
79 <sect1 id="athttpd-mime-types">
80 <title>MIME types</title>
82 The server has an internal table with all the recognized mime types. Each time
83 a file or an internal resource is sent out by the server, its extension is
84 searched in this table and if a match is found, the associated MIME type is
85 then sent out in the header.
87 The server already provides entries for the following standard file extensions:
89 'html', 'htm', 'gif', 'jpg', 'css', 'js', 'png'
91 and the user is responsible for adding any further entry. The syntax for
92 adding an entry is the following:</para>
94 <para><programlisting width=72>
95 CYG_HTTPD_MIME_TABLE_ENTRY(entry_label, extension_string, mime_tipe_sting);
97 entry table : an identifier unique to this entry
98 extension string : a string containing the extension for this entry
99 type_string : the mime string. The strings for many more mime types
100 is included in a file in the "doc" directory.
101 </programlisting></para>
104 The following is an example of how to add the Adobe Portable Document Format
105 <command>pdf</command> MIME type to the table:</para>
107 <para><programlisting width=72>
108 CYG_HTTPD_MIME_TABLE_ENTRY(hal_pdf_entry, "pdf", "application/pdf");
109 </programlisting></para>
111 <sect2 id="athttpd-mime-types-chunked">
112 <title>MIME Types for Chunked Frames</title>
114 For chunked frames, which are generally used inside c language callbacks, there
115 is no file name to match an extension to, and thus the extension to be used
116 must be passed in the <command>cyg_httpd_start_chunked()</command> call. The
117 server will then scan the MIME table to find a MIME type to match the extension.
119 For example, to start a chunked transfer of an <command>html</command> file,
120 the following call is used:</para>
122 <para><programlisting width=72>
123 cyg_httpd_start_chunked("html");
124 </programlisting></para>
127 In any event, it is the responsibility of the user to make sure that a match to
128 all used extensions is found in the table search. Failing this,
129 the default MIME type specified in the CYGDAT_NET_ATHTTPD_DEFAULT_MIME_TYPE
130 string is returned.</para>
134 <sect1 id="athttpd-callback">
135 <title>C language callback functions</title>
137 The server allows the association of particular URLs to C language callback
138 functions. eCos tables are used to define the association between a URL and its
139 corresponding callback. The syntax of the macro to add callback entries to
143 <para><programlisting width=72>
144 CYG_HTTPD_HANDLER_TABLE_ENTRY(entry_label, url_string, callback);
146 entry table : an identifier unique to this entry.
147 url_string : a string with the extension url that will be appended to the
149 callback : a function with a prototype:
150 cyg_int32 callback_function(CYG_HTTPD_STATE*);
151 Return value is ignored - just return 0.
152 </programlisting></para>
155 <command>CYG_HTTPD_STATE*</command> is a pointer to a structure that
156 contains, among others, a buffer (outbuffer) that can be used to send data
157 out. The definitions of the structure is in http.h.</para>
160 The following is an example of how to add a callback to a function myForm()
161 whenever the URL /myform.cgi is requested:
164 <programlisting width=72>
165 CYG_HTTPD_HANDLER_TABLE_ENTRY(hal_cb_entry, "/myform.cgi", myForm);
169 and somewhere in the source tree there is a function:</para>
172 cyg_int32 myForm(CYG_HTTPD_STATE* p)
174 cyg_httpd_start_chunked("html");
175 strcpy(p->outbuffer, "eCos Web Server");
176 cyg_httpd_write_chunked(p->outbuffer, strlen(p->outbuffer))
177 cyg_httpd_end_chunked();
181 <para>This function also shows the correct method of using the chunked frames
182 API inside a c language callback and also shows the use of outbuffer to
183 collect data to send out.</para>
185 <para>Chunked frames are useful when the size of the frame is not known upfront.
186 In this case it possible to send a response in chunks of various sizes, and
187 terminate it with a null chunk (See RFC 2616 for details). To use chunked
188 frames, the <command>cyg_httpd_start_chunked()</command> function is used.
189 The prototype is the following:</para>
192 ssize_t cyg_httpd_start_chunked(char *);
195 <para>The only parameter is the <command>extension</command> to use in the
196 search for the MIME type. For most files this will be "html" or "htm" and
197 it will be searched in the MIME table for an approriate MIME type that will
198 be sent along in the header. The function returns the number of bytes sent
201 <para>The chunked frame must be terminated by a call to
202 <command>cyg_httpd_end_chunked()</command>:</para>
205 void cyg_httpd_end_chunked()(void);
208 <para>In between these two calls, the user can call the function
209 <command>cyg_httpd_write_chunked()</command> to send out data any number of
210 times. It is important that <command>cyg_httpd_write_chunked()</command> be
211 the only function used to send data out for chunked frames. This
212 guarantees that proper formatting of the response is respected.
213 The prototype for the function is:</para>
216 ssize_t cyg_httpd_write_chunked(char* p, int len);
219 <para>The 'char*' points to the data to send out, the 'int' is the length of the
222 <para>In the case in which the size of the data is known upfront, the
223 callback can instead create the header with a call to
224 <command>cyg_httpd_create_std_header()</command> with the following
228 void cyg_httpd_create_std_header(char *ext, int len);
230 extension : the extension used in the search of the MIME type
231 len : length of the data to send out
235 <command>cyg_httpd_write()</command> to send data out to the client. The
236 prototype of <command>cyg_httpd_write()</command> is the same as
237 <command>cyg_httpd_write_chunked()</command></para></sect1>
239 <sect1 id="athttpd-cgi">
242 The web server allows writing of pseudo-CGI programs. This is helpful in order
243 to modify the functionality of the server without having to recompile it and
246 <para>One way to implement CGI is, of course, the C language callback mechanism
247 described above: This assumes, of course, that all the callbacks are written
248 by compile time and cannot be modified later on. Another way to perform the
249 same functionality is the use of a library in the form of an object file.
250 These object files reside in the file system and are loaded, executed and
251 unloaded on demand.</para>
253 <para>Yet a third way is the use of a scripting language. Since full fledged
254 implementation of the most popular scripting languages such as Python or Perl
255 are too large for most embedded systems, a slim down implementation of tcl
256 was chosen for this server. Most of the tcl functionality is still there,
257 and makes writing cgi a lot easier.</para>
259 <para>In order to limit the footprint of the operating system support for both
260 the objloader and the tcl script for dealing with cgi files can be
261 independently selected out. Tcl support in particular increases the memory
262 requirements considerably.
265 <sect2 id="athttpd-cgi-objloader">
266 <title>CGI via objloader</title>
268 In order to use the cgi mechanism the CYGPKG_OBJLOADER must be included
269 when building the operating system. This will enable the proper option in the
270 configuration tool and if selected, the necessary code will be compiled
271 in the eCos kernel. The user will then have to compile the necessary libraries
272 and place them in the file system under a directory defined by
273 CYGDAT_NET_ATHTTPD_SERVEROPT_CGIDIR.
274 When a request is made, the web server checks if the root directory of the
275 requested URL is inside the CYGDAT_NET_ATHTTPD_SERVEROPT_CGIDIR directory.
276 If so, the server assumes that the user requested a cgi file and looks into the
277 directory to see if a library by the same name is present, and if so load it
278 and tries to execute a function inside the library with the following prototype:
281 <programlisting width=72>void exec_cgi(CYG_HTTPD_STATE *)
285 The pointer <command>CYG_HTTPD_STATE*</command> gives access to the socket
286 data: The user will use this pointer to access the 'outbuffer' and use it to
287 copy data to send data out.
291 When using the OBJLOADER package within the HTTP server a number of functions
292 are automatically added to the externals table of the OBJLOADER package. These
293 functions are likely to be used inside the library and the relocator need to
294 have a pointer to them. In order to add more functions, see the OBJLOADER
295 documentation. The complete list of the functions automatically added is:
299 <listitem><para>cyg_httpd_start_chunked()</para></listitem>
300 <listitem><para>cyg_httpd_write_chunked()</para></listitem>
301 <listitem><para>cyg_httpd_end_chunked()</para></listitem>
302 <listitem><para>cyg_httpd_write()</para></listitem>
303 <listitem><para>cyg_httpd_find_form_variable()</para></listitem>
304 <listitem><para>cyg_httpd_find_ires()</para></listitem>
305 <listitem><para>cyg_httpd_send_ires()</para></listitem>
306 <listitem><para>diag_printf()</para></listitem>
307 <listitem><para>cyg_httpd_format_header()</para></listitem>
308 <listitem><para>cyg_httpd_find_mime_string()</para></listitem>
311 <para>Every time the web client issues a GET or POST request for a file with an
312 extension of '.o'in the /cgi-bin directory (or whatever path the user chooses
313 to hold the libraries) then the library by that name is loaded, run and
314 when the execution is over, it is dumped from memory.
316 The library must be compiled separately, using the same toolchain used to
317 compile the server and then added to the file system.</para>
319 <para>In order to reduce the footprint of the server, CGI through OBJLOADER
320 can be compiled out by unchecking CYGOPT_NET_ATHTTPD_USE_CGIBIN_OBJLOADER
321 in the configuration tool.</para>
324 <sect2 id="athttpd-cgi-tcl">
325 <title>CGI via the simple tcl interpreter</title>
326 <para>A small tcl interpreter has been added to the web server, and it can
327 be used to write simple cgi scripts. The interpreter is admittedly very
328 minimal, and it is only useful for very simple applications, but it is an
329 excellent starting point for further development.</para>
331 <para>In order for the scripting language to be useful, it has to access
332 the form variables passed on during the GET or POST request. Because of
333 this, all form variables registered with the CYG_HTTPD_FVAR_TABLE_ENTRY()
334 macro are accessible via tcl. For example, if we have registered a
335 form variable called foo, and during the GET request we are defining foo
338 <programlisting width=72>GET /myForm.cgi?foo=1</programlisting>
340 <para>then tcl will be able to access the variable foo as $foo. The data
341 in the body of a POST request is also accessible through the use of the variable
342 $post_data. This is useful if the data is not in "multipart/form-data"
343 and tcl has to perform any type of processing on the data itself.</para>
345 <para>In order to send back a response to the client a few functions have been
346 added to the interpreter. These functions are:</para>
348 <sect3 id="athttpd-start-chunked">
349 <title>start_chunked</title>
350 <programlisting width=72>start_chunked "extension";</programlisting>
351 <para>"extension" is a string used to search the
352 table of the mime types. For example, to send back to the client an HTML file,
353 we can use: start_chunked "html";
357 <sect3 id="athttpd-write-chunked">
358 <title>write_chunked</title>
359 <programlisting width=72>write_chunked content;</programlisting>
360 <para>content is a string to send back to the client.
364 <sect3 id="athttpd-end-chunked">
365 <title>end_chunked</title>
366 <programlisting width=72>end_chunked;</programlisting>
367 <para>No parameters. Send back an end of frame to the client.</para>
370 <sect3 id="athttpd-tcl-hello-world">
371 <title>tcl hello world example</title>
373 The following example demonstrates how to send a log file in the file
374 <filename>/ram/log</filename> to a web client. It replaces
375 newline characters with <literal><br></literal> so that it is formatted on the
377 <programlisting width=72>
378 start_chunked "html";
380 set fp [aio.open "/ram/log" r];
382 set fsize [$fp tell];
385 set data [$fp read $fsize];
387 set data [string map {\n <br>} $data];
390 append datax "<html><body>" $data "</body></html>";
392 write_chunked $datax;
397 The above file should exist on a filesystem
398 on the embedded target within its <filename class=directory>cgi-bin</filename>
399 directory, for example as <filename>/cgi-bin/hello.tcl</filename>. Thereafter
400 it may be accessed at the URL
401 <literal>http://<replaceable>TARGET_NAME</replaceable>/cgi-bin/hello.tcl</literal>.
407 <sect1 id="athttpd-authentication">
408 <title>Authentication</title>
410 The server supports both Basic (base64) and Digest (MD5) authentication,
411 although they have not been tested with all clients. In this implementation,
412 the contents of certain directories of the file system can be protected, such
413 that the user will be required to issue a username/password to access the
414 content of the directory.</para>
416 <para>To protect a directory with a basic authentication, there is a
417 specific macro:</para>
420 CYG_HTTPD_AUTH_TABLE_ENTRY(entry, path, domain, un, pw, mode)
422 entry : an identifier unique to this entry.
423 path : the path to the directory whose content must be
424 authenticated before it is sent out
425 domain : a domain identifier for this directory.
426 un : username for authentication
427 pw : password for authentication
428 mode : CYG_HTTPD_AUTH_BASIC for base64 encoding or
429 CYG_HTTPD_AUTH_DIGEST for MD5 encoding
432 <para>for example, to require basic authentication of the content of directory
433 "/ecos/" with a username of "foo" and password "bar", the following is used:
437 CYG_HTTPD_AUTH_TABLE_ENTRY(hal_domain1_entry, \
438 "/ecos/", "ecos_domain", \
440 CYG_HTTPD_AUTH_BASIC);
443 <para>Any request for a file in the directory /ecos/ will now trigger a
444 credential check. These credentials, once provided, are automatically sent by
445 the client for every request within the particular domain.</para>
447 <para>It must be noticed that the path name set in the macro is relative to the
448 HTML document directory, CYGDAT_NET_HTTPD_SERVEROPT_HTMLDIR and it is the
449 first part of the path provided by the client request (including the leading
452 <para>In order to reduce the footprint of the server, authentication
453 is not enabled by default, and so the option CYGOPT_NET_ATHTTPD_USE_AUTH must
454 be used to enable support for basic and digest authentication.</para>
456 <para>The MD5 digest authentication support is implemented using the RSA
457 Data Security, Inc. MD5 Message-Digest Algorithm. Derivative works with
458 MD5 digest authentication included must be identified as "derived from the
459 RSA Data Security, Inc. MD5 Message-Digest Algorithm" in all material
460 mentioning or referencing the derived work. See the file md5.c within this
461 package for license details.</para>
464 <sect1 id="athttpd-dirlist">
465 <title>Directory Listing</title>
467 <para>If the user issues a "GET" request with a URL terminating in a slash, the
468 server will try to locate one of the following index files in the directory,
469 choosing one in the following order:</para>
472 <listitem><para>index.html</para></listitem>
473 <listitem><para>index.htm</para></listitem>
474 <listitem><para>default.html</para></listitem>
475 <listitem><para>home.html</para></listitem>
478 <para>If any of these files is found, its contents are sent back
479 to the client. If no such file is found the server uses the user-provided
480 index file name (if any is specified with the CYGDAT_NET_ATHTTPD_ALTERNATE_HOME
481 setting. Failing all this a directory listing is sent.</para>
483 <para>Trailing slash redirection for directory names is supported.</para>
485 <para>In order to reduce the footprint of the server, directory listing can
486 be disabled by unchecking CYGOPT_NET_ATHTTPD_USE_DIRLIST. The savings are
487 substantial since directory listing also makes use of a few internal
488 resources (gif files) which are also compiled out.</para>
491 <sect1 id="athttpd-formvars">
492 <title>Form Variables</title>
494 <para>The server will automatically try to parse form variables when a form is
495 submitted in the following cases:
498 <listitem><para>In a GET request, when the URL is followed by a question
499 mark sign</para></listitem>
500 <listitem><para>In a POST request, when the the 'Content-Type' header line
501 is set to 'application/x-www-form-urlencoded'</para></listitem>
504 The variable names to look for during the parsing are held in
505 an eCos table. In order to take advantage of this feature, the user first
506 adds the variable names to the table, which also requires providing a buffer
507 where the parsed value will eventually be stored. The values will then be
508 available in the buffers during the processing of the request, presumably in
509 the body of a c language callback or CGI script.</para>
511 <para>For example, if the user wants two form variables, "foo" and "bar", to
512 be parsed automatically, those variable names must be added to the table
513 with the following macro:</para>
516 CYG_HTTPD_FVAR_TABLE_ENTRY(entry, name, buffp, bufflen)
518 entry : an identifier unique to this entry.
519 name : name of the form variable
520 buffp : a pointer to a buffer of characters where to store the value
521 of the form variable.
522 bufflen : The length of the buffer. Must include a trailing string
526 <para>or, in the specific instance mentioned above:</para>
529 #define HTML_VAR_LEN 20
530 char var_foo[HTML_VAR_LEN];
531 char var_bar[HTML_VAR_LEN];
532 CYG_HTTPD_FVAR_TABLE_ENTRY(hal_form_entry_foo, "foo", var_foo, HTML_VAR_LEN);
533 CYG_HTTPD_FVAR_TABLE_ENTRY(hal_form_entry_bar, "bar", var_bar, HTML_VAR_LEN);
536 <para>and after the GET or POST submissions, the list will contain the value
537 for "foo" and "bar" (if they were found in the form data.) It is the
538 responsability of the user to make sure that the buffer is large enough
539 to hold all the data parsed (including the string terminator). The parser will
540 write only up to the length of the buffer minus one (the last being the
541 terminator) and discard any additional data.</para>
543 <para>The values parsed are likely going to be used in c language callback, or
544 in CGI files. In a c language callback the user can directly access the pointers
545 of individual variables for further processing, keeping in mind that the parsing
546 always result in a string of characters to be produced, and any conversion
547 (e.g. from strings to integer) must be performed within the callback. In
548 a TCL script the user can just access a variable by its name. For example,
549 in the case of the variables 'foo' and 'bar' shown above, it is possible
550 to do something like 'write_chunked "You wrote $foo". The data that was sent in
551 the body of a POST request is accessible in through a variable called
552 'post_data'. In CGI functions
553 implemented using the objloader the pointers to the
554 variables cannot be accessed directly, since the library will likely not
555 know their location in memory. The proper way to access them is by using the
556 cyg_httpd_find_form_variable() function from within the library:</para>
559 char* cyg_httpd_find_form_variable(char* name)
561 name : name of the form variable to look up
563 returns a pointer to the buffer, or 0 if the variable was not found.
566 <para>When using the OBJLOADER package within the web server, an entry
567 for the cyg_httpd_find_form_variable() function is automatically added to the
568 externals table the OBJLOADER for relocation. See the OBLOADER paragraph of
569 the ATHTTP user's guide for the full list of the exported functions.</para>
571 <para>In order to avoid stale data, all the buffers in the table are cleared
572 before running the parser and thus any variable in the list that was not
573 assigned a new value dureing the request will be an empty string.</para>
576 <sect1 id="athttpd-ires">
577 <title>Internal Resources</title>
579 <para>When the server does not use a file system the user must be responsible
580 to provide a C language callback function for each URL that will be
581 requested by the client. This means locating the data and sending it out
582 using either <command>cyg_httpd_write()</command> or
583 <command>cyg_httpd_write_chunked()</command>.</para>
585 <para>In order to simplify this process the server allows registering
586 any number of URLs as internal resources, by providing the URL name, the
587 pointer to the resource data and its size. When a URL is requested the
588 server will look it up among all internal resources, and if found, it
589 will send out the resource.</para>
591 <para>Internal resource can also be used along with a file system. In this
592 case the file system is searched first, and if a file is found, it it
593 sent. If a file is not found, the internal resources are searched and
594 if a match if found it is sent.</para>
596 <para>The drawback of this approach is, of course, that all these
597 resources are going to add to the size of the operating system image, and thus
598 it should be used only when memory is not a major constraint of the
601 <para>As always, to provide this type of customization, ecos tables are used.
602 The format for adding a new resource to the internal table is the following:
606 CYG_HTTPD_IRES_TABLE_ENTRY(entry, name, buffp, len)
608 entry : an identifier unique to this entry.
609 name : name of the URL including leading '/'
610 buffp : a pointer to a buffer of characters where to store the value
611 of the form variable.
612 len : size of the array
615 <para>As an example, if the user wants to provide his own web page by
616 hardcoding it in the application code, here is how he would do it:</para>
619 #define MY_OWN_HOME_PAGE "eCos RTOS"
620 CYG_HTTPD_IRES_TABLE_ENTRY(cyg_httpd_ires_home, \
626 <para>The extension of the file name determines the MIME type to be used for
627 internal resources.</para>
629 <para>When using directory listing you are implicitly making use of internal
630 resources. The small icons that appear to the left of file names and
631 directories are internal resources. Unchecking CYGOPT_NET_HTTP_USE_DIRLIST
632 will prevent the addition of these files.</para>
634 <para>In order to use internal resources, a generic file must first be
635 turned into a c language array, which is then compiled in the application
636 code. To create this array you can use the tcl script that comes with the
637 ecos distribution at packages/fs/rom/current/support/file2.tcl.</para>