______ Strategy for developing a web server (CS 273 (OS), Fall 2019)
Home
>>    




Strategy for developing a web server

CS 273 (OS), Fall 2019

Carry out the following tasks, in an order consistent with the stated prerequisites.
Note: The task labels A, B, etc., below are for convenience in cross-referencing, and do not themselves indicate strict ordering.

Changes:

  • This style indicates any content added since a task was no longer under construction.

  • This style indicates any content deleted since a task was no longer under construction.

Index of tasks

A. Working directories
    (Prereq: submission of proj_team.html and creation of team shared repo by RAB)

B. Git development branches
    (Prereq: Task A)

C. Start server.c and client.c
    (Prereq: Task B)

D. Start Makefile
    (Prereq: Task C; involves server.c, client.c)

E. Start server's work.c
    (Prereq: Task D; involves server.c do_work())

F. Start client's request.c
    (Prereq: Task D; involves client.c do_request())

G. Request IDs and workdat data structure
    (Prereq: Task E; involves server.c work.c work.h do_work())

H. Round-trip client and server
    (Prereq: Task E and Task F; involves work.c request.c do_work() do_request())

I. Parse HTTP request (initial version)
    (Prereq: Task B; involves work.c Makefile)

J. Incorporate parser into server
    (Prereq: Task I; involves work.c Makefile)

K. Add logging (log.c)
    (Prereq: Task G; involves server.c work.c)

L. Error checking in parser
    (Prereq: Task J; involves parse.c work.c client.c)

M. Send file to client
    (Prereq: Task L; involves parse.c work.c Makefile client_tests)

N. Implement 501 (Not Implemented)
    (Prereq: Task L; involves work.c Makefile)

O. Implement 400 (Bad Request)
    (Prereq: Task L; involves work.c parse.c Makefile)

P. Implement 404 (Not Found)
    (Prereq: Task M; involves work.c Makefile)

Q. Worker thread
    (Prereq: Task G; involves work.c do_work.c do_work() workdat Makefile)

R. Multiple worker threads
    (Prereq: Task Q; involves server.c work.c main() do_work() Makefile)

S. Browser test
    (Prereq: Task M)

Under construction:

T. Parallel threads demo (Extra feature)
    (Prereq: Task R and Task H; involves work.c wdp_array.c worker() add_wdp() check_threads())

A. Working directories (Prereq: submission of proj_team.html and creation of team shared repo by RAB)

Start by creating a clone ~/OS/pp-threads in your ~/OS working directory for your team's shared stogit repository. 

cd ~/OS
git clone git@stogit.cs.stolaf.edu:os-f19/repo.git pp-threads
Here, replace repo by your team repository name (created after you submit proj_team.html and receive a repository-creation notice from RAB) .

This will make a new subdirectory ~/OS/pp-threads that its own git structure:

  • Inside the subdirectory ~/OS/pp-threads, your git operations such as pull and push will interact with your team's repo.

  • Inside ~/OS or its subdirectories other than pp-threads, git operations would interact with your usual personal course repository.

Link to index

B. Git development branches (Prereq: Task A)

We recommend that you use git branches to carry out your team project work.

  • push tested working code only to the master branch (e.g., git pull origin master )

  • Create new branches for developing new code features, and merge each development branch with master as those new features whenever a new incremental step in development is completed and tested.

    • By merging frequently, you avoid lengthy and complex integration of large code modifications with other code changes your teammates (or you!) may have made recently.

    • Exception: Do Not Break the master Branch! Instead, thoroughly test that the master branch code works after you merge your code.

Reminder of git branch commands (reference: Git book, sections 3.1 and 3.2

  • git branch mybranch

    Create a new branch mybranch in your local working directory.

  • git checkout mybranch

    Switch to the branch mybranch .

    Note: Perform git checkout operations only when all your changes have already been recorded in commits, so you won't lose any work. You can check for any changes in your code since your last commit using

    git status

  • git checkout -b mybranch

    This is equivalent to 

    git branch mybranch
git checkout mybranch
    Only use -b when mybranch doesn't exist yet.

  • git pull origin mybranch
    git push origin mybranch

    Use mybranch instead of master when you want to pull/push with that branch.

    Note:

    Omit the pull operation the first time only with a new branch, since that branch doesn't exist yet on stogit. Thereafter, use pull then push as usual.

  • git merge mybranch

    Merge the changes in mybranch into the current branch. To merge your branch's changes into master, enter these two commands:

    git checkout master
git merge mybranch


Here is a team exercise with git development branches, for teams of 2 or 3 students referred to as Student 1, Student 2, and (if any) Student 3. Perform these steps in the indicated order.

  1. In Student 1's ~/OS/pp-threads working directory, Student 1

    • creates a simple hello.c program that performs a single printf of a string including Student 1's username, and Student 1 tests that code.

    • Student 1 then creates a commit with the new file hello.c, and performs pull/push to commit to the shared team project respository (master branch).

  2. Then in Student 2's ~/OS/pp-threads working directory, that Student 2 performs

    • performs git pull origin master in order to receive Student 1's hello.c

    • Creates a new branch stu2 for modifying that file hello.c

      git branch stu2
git checkout stu2
      You can check your current branch using 
      git branch
      The asterisk * in the output from git branch indicates your current branch. Alternatively, you can issue
      git status
      which should also indicate the current branch in output.

    • Enter ls, and verify that hello.c exists in the branch

    • Edit the program hello.c to add a second printf() instruction, in order to print a line after the original output line. Student 2 should include their username in their added output line, then should proceed to test the modified program.

    • Create a commit for the change to hello.c, then perform

      git push origin stu2
      Note: This is one time when a pull does not precede a git push operation. Here, a pull operation for the branch stu2 would fail, because that branch doesn't exist on stogit yet.

    • Now, Student 2 perform a merge to integrate their changes into the master branch.

      git checkout master
git merge stu2 
git pull origin master
git push origin master
      Note: The git merge command will create a new merge commit, and will start up an editor for Student 2 to document that merge commit. Exiting that editor immediately will serve our purposes here and enable the merge commit to complete.

  3. In a 3-member team, Student 3 should carry out the same steps as Student 2, except:

    • add a printf() to print one line of output before the original line created by Student 1 (include Student 3's username in the added output);

    • The name of the branch should be stu3.

  4. Now Student 1 should perform

    • Enter the command 

      git pull origin master
      to retrieve the changes to hello.c, and test those changes.

    • Create a development branch stu1, and edit hello.c to add more characters to the end of the original line entered by Student 1 in that branch.

    • Perform a git push origin stu1 operation

    • Merge your branch stu1 into the master branch, as indicated above for Student 2.

You can reuse branches or make new ones according to what makes sense to you.

  • Once you have successfully merged a branch to master, that branch and master have the same HEAD (latest commit), so it's safe to delete that branch if you want to. Or, you could continue using that branch for your next new additions to the code.

  • You can delete a branch using 

    git branch -d branchname
git push origin --delete branchname
    The first of these commands deletes the local branch (on your working directory), and the second command deletes the remote branch (on stogit).

Link to index

C. Start server.c and client.c (Prereq: Task B)

  1. Start a development branch for your work on this task, e.g., 

    git checkout -b mybranch
    where mybranch is a new branch name for your work

  2. Copy sender.c and receiver.c from your hw4 directory, and rename the copies client.c and server.c.

  3. Modify client.c, moving the following code from main() into the definition of a function do_request() with two arguments, a socket descriptor (int) and a program name (string). Then insert a call do_request(sd, prog) in main() where that code used to be. 

     
  char *buff = NULL;  /* message buffer */
  size_t bufflen = 0;  /* current capacity of buff */
  size_t nchars;  /* number of bytes recently read */

  ...

  printf("%d characters sent\n", ret);
  if ((ret = close(sd)) < 0) {
    printf("%s ", prog);
    perror("close()");
    return 1;
  }
    Note: You will also need to define an integer variable ret in your new function do_request() since ret is needed for that code.

  4. Modify server.c, moving the following code from main() into the definition of a function do_work() with two arguments, a socket descriptor (int) and a program name (string). Then insert a call do_work(clientd, prog) in main() where that code used to be. 

     
  char buff[MAXBUFF];  /* message buffer */
  int ret;  /* return value from a call */
  if ((ret = recv(clientd, buff, MAXBUFF-1, 0)) < 0) {
    printf("%s ", prog);
    perror("recv()");
    return 1;
  }

  buff[ret] = '\0';  // add terminating nullbyte to received array of char
  printf("Received message (%d chars):\n%s\n", ret, buff);

  if ((ret = close(clientd)) < 0) {
    printf("%s ", prog);
    perror("close(clientd)");
    return 1;
  }

  5. Compile the two programs server.c and client.c and test/debug them, using the same networked testing as in HW4.

  6. When your code is working, push your work to your branch of the shared repository

    git add client.c server.c
git commit -m "Task C - start server.c and client.c
git push origin mybranch
    (where mybranch should be replaced by the name of your development branch).
    Note: We are omitting pull this time only because mybranch is a new branch.

  7. Now merge your correct code into master.

    git checkout master
git merge mybranch
git pull origin master
git push origin master

Link to index

D. Start Makefile (Prereq: Task C; involves server.c, client.c)

  1. Switch to your development branch

    git checkout mybranch

  2. Add the following Makefile to your pp-threads directory.

    # Makefile for pp-threads project

%.o:  %.c
	gcc -c $*.c

all:  server client

server:  server.o
	gcc -o server server.o

client:  client.o
	gcc -o client client.o

    Explanation:

    • # Makefile ...  is documentation

    • %.o:  %.c
	gcc -c $*.c
      is a default rule for compiling a C program. This could be modified later as needed.

    • Note: The indentations above must be tab characters (which could be followed by spaces).

    • all: server client  is the first target all and its prerequisites server client.
      The first target is the default when you enter 

      make
      without specifying a target file to be made.

    • server:  server.o
	gcc -o server server.o
      is a Makefile rule for creating the executable server:

      • The target is server

      • The prerequisite is server.o

      • The action is gcc -o server server.o

      If you enter the command 

      make server
      then the prerequisite server.o will be created first (using the default rule for compiling a C program) if necessary, then the action gcc -o server server.o will be performed if necessary.

      • "if necessary" operations are only performed if their target is out of date with their prerequisites (or a prerequisite of a prerequsite).
        For example, server.o is generated only if its (implicit) prerequisite server.c is newer than server.o; and server is generated only if its prerequites server.o is newer than server.

  3. Test the Makefile as follows.

    1. Exit any editors that are using your code files client.c and server.c, in order to prevent confusion due to the following operations.

    2. make
      This should recompile the target executables server and client if necessary. Most likely, it will report that they are "up to date" without any recompilation, unless you have made changes since your last compile.

    3. touch server.o
make
      The touch command changes the modified date of a file to the current time, but doesn't change characters or other aspects of that file.
      Here, touch server.o makes server.o newer than server, so the make command should relink to create a new version of server.

    4. touch server.c
      This make should cause one compile (of server.c) and one link (of the resulting server.o), producing a new version of server.

    5. touch client.o
make
      Should cause one link operation, producing a new version of the executable client

    6. touch client.c
      Should cause one compile and one link.

  4. Once your Makefile is tested and debugged, send your changes to stogit. 

    git add Makefile
git commit -m "Task D - start Makefile"
git pull origin mybranch
git push origin mybranch

git checkout master
git merge mybranch
git pull origin master
git push origin master
    Note: You could omit 
    git pull origin mybranch
git push origin mybranch
    but then there the branch changes would only occur in your local working directory, and would not be known to stogit.

    We recommend posting all of your work on branches to stogit, in case someone else on your team needs to examine those changes for a later task.

Link to index

E. Start server's work.c (Prereq: Task D; involves server.c do_work())

We now move do_work() from server.c into a separate code module work.c with header file work.h, so that server.c focuses on obtaining new client connections, whereas work.c handles requests received from such clients.

  1. Switch to your development branch

    git checkout mybranch
    If there may have been updates to master since you last used mybranch (e.g., updates by a team member), then update mybranch as follows.
    git merge origin/master
git push mybranch

    • Note: The merge operation above merges changes on the stogit server's version of master (origin/master) with your development branch. Like all merge operations, we expect a merge commit.

    • Note: Another approach to updating mybranch involves git rebase. Avoid using git rebase for updating your development branch, because it generates an invented commit history that might cause confusion. . (See the second answer to this stackoverflow article for more information.)

  2. Move the definition of the function do_work() from server.c to a new file work.c. Copy the #include directives from the beginning of server.c to the beginning of the new file work.c.

  3. Create a new file work.h with a function declaration for do_work(), and add a new include directive 

    #include "work.h"
    at the end of the existing #include directives in each source file server.c and work.c. Here is example code for work.h:
    /* header file for work.c */

void do_work(int sock, char *progname);
    Modify this file as needed to fit the definition of your do_work() function. For example, if your function do_work() returns a value, use a matching return type in work.h.

  4. Modify Makefile to compile work.c and link the resulting work.o when creating the executable server, and to update dependencies including work.h. The code below shows the needed changes.

    # Makefile for pp-threads project

%.o:  %.c
	gcc -c $*.c

all:  server client

server:  server.o work.o
	gcc -o server server.o work.o

server.o: server.c work.h

work.o: work.c work.h

client:  client.o
	gcc -o client client.o

    Notes:

    • In the rule for the target server, be sure to add work.o as both a prerequisite and as a file to be linked.

    • We added rules for server.o and work.o in order to show the new prerequisite work.h.

      • Before, we used make's default rule, which is equivalent to 

        server.o:  server.c

      • The action for these rules is to compile using the default compilation rule 

        %.o:  %.c
	gcc -c $*.c
        If we sometime need to compile differently for some target, we can add an action to that target's rule in order to override this default.

  5. Now issue the command make, and debug compilation and linking as needed. Then test run the program using the tests of Task C, and debug as needed to obtain working code.

  6. Test your updated Makefile

    1. Exit any editors using server.c, work.c, or work.h, to avoid confusion about modification times.

    2. make
      This will most likely report that your code is "up to date."

    3. touch work.c
make
      This should recompile work.c, then relink to produce a new server program. (Test that program.)

    4. touch server.c
make
      Should recompile server.c then relink to produce a new server.

    5. touch work.h
make
      This should produce two recompiles (both server.c and work.c) then relink (producing another server executable).

  7. Make a commit of your successful changes so far.

    git add work.c work.h server.c Makefile
git commit -m "Task E - move do_work() to work.c, create work.h"

    8. Note: Making an intermediate commit like this enables you to revert to a prior (committed) version of your code at a later time. For example, suppose you inadvertently made changes after the commit above that caused bugs in the program, and that you can't figure out how to fix those bugs. Then you could revert to the commit above as follows:

    • Enter 

      git log
      to find the hash for the commit you want to revert to (you can abbreviate the full 40-character hash to the first few characters, as long as those few characters are different than other hashes).

    • Enter 

      git reset --hard hash
      where hash is the (abbreviated) hash you just found.

    • Use git reset with care! git reset --hard will probably overwrite any changes you may have made since that commit hash. If there's some part of your code that you want to keep, make a copy of those files (e.g., in a new subdirectory ~/OS/pp-threads/tmp) before performing that git reset command.

  8. Remove unneeded #include directives from the beginning of server.c and work.c.

    For example, work.c uses system calls recv() and close(), which only require the header files sys/types.h, sys/socket.h, and unistd.h. work.c also requires stdio.h for printf() and perror(). So, remove the following:

    #include <stdlib.h>
#include <netinet/in.h>

    • Note: All the header file information above was found by googling man recv, etc.

    Verify that your code still makes and runs correctly

  9. Once your changes are tested and debugged, commit and send them to stogit. 

    git add work.c server.c
git commit -m "Task E DONE - Start server's work.c"
git pull origin mybranch
git push origin mybranch

git checkout master
git merge mybranch
git pull origin master
git push origin master

Link to index

F. Start client's request.c (Prereq: Task D; involves client.c do_request())

Similarly to Task E, we move do_request() from client.c into a separate code module request.c with header file request.h, so that client.c focuses on connecting to the server and request.c transmits requests to that server.

  1. Switch to your development branch

    git checkout mybranch
    If there may have been updates to master since you last used mybranch (e.g., updates by a team member), then update mybranch as follows.
    git merge origin/master
git push mybranch

  2. Move the definition of the function do_request() from client.c to a new file request.c. Copy the #include directives from the beginning of client.c to the beginning of the new file request.c.

  3. Create a new file request.h with a function declaration for do_request(), and add a new include directive 

    #include "request.h"
    at the end of the existing #include directives in each source file client.c and request.c.

    • Use an approach similar to Task E.

  4. Modify Makefile to compile request.c and link the resulting request.o when creating the executable client, and to update dependencies including request.h.

    • Use an approach similar to Task E.

  5. Now issue the command make, and debug compilation and linking as needed. Then test run the program using the tests of Task C, and debug as needed to obtain working code.

  6. Test your updated Makefile

    • Use an approach similar to Task E.

  7. Make a commit of your successful changes so far.

    git add request.c request.h client.c Makefile
git commit -m "Task F - move do_request() to request.c, create request.h"

  8. Remove unneeded #include directives from the beginning of server.c and work.c.

    • Use an approach similar to Task E.

  9. Once your changes are tested and debugged, commit and send them to stogit. 

    git add client.c request.c
git commit -m "Task F DONE - Start client's request.c"
git pull origin mybranch
git push origin mybranch

git checkout master
git merge mybranch
git pull origin master
git push origin master

Link to index

G. Request IDs and workdat data structure (Prereq: Task E; involves server.c work.c work.h do_work())

The required logging mechanism will use a unique request ID number. We will store this request ID in a new data structure workdat, which will eventually hold other data associated with a particular request.

  1. Switch to your development branch

    git checkout mybranch
    If there may have been updates to master since you last used mybranch (e.g., updates by a team member), then update mybranch as follows.
    git merge origin/master
git push mybranch

  2. In server.c, define a local int variable request_id in main(), initialized at 1. Add the value of this variable as a new first argument for do_work() (this will require changes in all three files server.c, work.c, work.h).

    Compile and test your modified server with the (unmodified) client as in HW4.

  3. Make a commit of your successful changes so far.

    git add work.c work.h server.c
git commit -m "Task G - Add request_id to main() and do_work()"

  4. In work.c, define a data structure struct workdat with one integer field rid (for Request ID). Then, at the beginning of do_work(), define a local variable wd of that type struct workdat, and assign the value of the new first argument to the field wd.rid. Also add a temporary printf() call to print the value of wd.rid, in order to verify that the request ID value 1 was correctly stored in that field.

    Remake and test your modified server with the (unmodified) client as in HW4.

    You can remove the temporary printf() now or at a later time.

  5. Once your changes are tested and debugged, commit and send them to stogit. 

    git add work.c
git commit -m "Task G DONE - Request IDs and workdat data structure"
git pull origin mybranch
git push origin mybranch

git checkout master
git merge mybranch
git pull origin master
git push origin master

Link to index

H. Round-trip client and server (Prereq: Task E and Task F; involves work.c request.c do_work() do_request())

______

  1. Switch to your development branch

    git checkout mybranch
    If there may have been updates to master since you last used mybranch (e.g., updates by a team member), then update mybranch as follows.
    git merge origin/master
git push mybranch

  2. Modify do_work() in the server's work.c to send a reply to the client, after receiving the client's message and before shutting down the socket. For now, send a 3-character message ACK as the reply.

  3. Modify do_request() in the client's request.c to receive that reply message from the server, after sending it's message and before closing its socket. Also add a printf() to print the content of the message that was received.

  4. Compile and test your modified server and client as in HW4, debugging as necessary.

  5. Once your changes are tested and debugged, commit and send them to stogit. 

    git add work.c request.c
git commit -m "Task H DONE - Round-trip client and server"
git pull origin mybranch
git push origin mybranch

git checkout master
git merge mybranch
git pull origin master
git push origin master

Link to index

I. Parse HTTP request (initial version) (Prereq: Task B; involves work.c Makefile)

Next, we will define two more source modules for parsing an HTTP request and for interacting with files and integrate them into the server code.

  1. Switch to your development branch

    git checkout mybranch
    If there may have been updates to master since you last used mybranch (e.g., updates by a team member), then update mybranch as follows.
    git merge origin/master
git push mybranch

  2. Copy your program copyfile.c from HW8 into your project directory. Add a Makefile target for building the program copyfile from copyfile.o (and recall that copyfile.o is automatically generated from copyfile.c. Compile the program using 

     
make copyfile
    and run that program to verify that it correctly reads an input line and performs a file copy.

  3. Make a commit of your successful changes so far.

    git add copyfile.c Makefile
git commit -m "Task I - import copyfile.c"

  4. Now create a function parse() in copyfile.c that generates a name structure from the input line.

    parse

    1 arg: A string.

    State change: A struct name memory location is dynamically allocated using malloc(), and that struct name is assigned to hold dynamically allocated copies of the tokens in arg1.

    Return: struct name*, the address of that assigned struct name memory location.

    Then modify main() in copyfile.c to call parse() to process the line it reads from standard input.

    • Note: Do not include your getline() call in parse(), which would violate the spec above.

    Issue make copyfile to generate a new version of copyfile, and test/debug. The version of copyfile should behave the same as the prior version did.

  5. Make a commit of your successful changes so far.

    git add copyfile.c
git commit -m "Task I - define parse()"

  6. Define a function delete_name() that deallocates a struct name, satisfying the following spec.

    delete_name

    1 arg: struct name*, the address of a dynamically allocated and assigned struct name memory location.

    State change: All dynamically allocated memory pointed to by arg1 becomes deallocated using free(), including any tok[] elements and the struct name memory *arg1 itself.

    Return: None.

    Also add temporary printf() calls near the beginning and end of the definition of delete_name(), so you can observe when delete_name() becomes called.

    Insert call(s) of delete_name() in the copyfile.c code as needed to insure that the struct name object allocated by parse() is correctly deallocated, in order to avoid memory leaks. Test and debug this program with correct deallocation.

  7. Make a commit of your successful changes so far.

    git add copyfile.c
git commit -m "Task I - add delete_name(), for correct memory management"

  8. Move the definitions of parse() and delete_name to a new file parse.c, and put the definition of struct name plus declarations for those two files in a header file parse.h. Add 

    #include "parse.h"
    to both copyfile.c and parse.c in order to use the struct and the two functions in both code files.

    Modify Makefile to

    • add a rule for target parse.o with prerequisites parse.c and parse.h

    • add a rule for target copyfile.o with prerequisites copyfile.c and parse.h

    • add the object file parse.o to the prerequisites and the action for target copyfile

    Then compile copyfile via the command

    make copyfile
    and verify that the complation occurs correctly. Proceed to test the resulting executable copyfile, checking that it behaves as before.

  9. Once your changes are tested and debugged, commit and send them to stogit. 

    git add copyfile.c parse.c parse.h Makefile
git commit -m "Task I DONE - Parse HTTP request (initial version)"
git pull origin mybranch
git push origin mybranch

git checkout master
git merge mybranch
git pull origin master
git push origin master

Link to index

J. Incorporate parser into server (Prereq: Task I; involves work.c Makefile)

______

  1. Switch to your development branch

    git checkout mybranch
    If there may have been updates to master since you last used mybranch (e.g., updates by a team member), then update mybranch as follows.
    git merge origin/master
git push mybranch

  2. Modify work.c to call parse() in order to extract the tokens from a client's input line. (This will involve modifying either do_work() or worker(), depending on whether you have completed Task Q.) Reply to the client with the second token received from that client. Modify Makefile accordingly (parse.h becomes a prerequisite for work.o, and parse.o becomes a prerequisite and an added object file in the action for target server).

    Procede to make and test/debug the resulting server. The behavior should be the same as before, except that the server responds with the client's second token instead of ACK.

  3. Note that the first line of the HTTP GET protocol should be 

    GET filename HTTP/1.1
    If you enter that line as input for the client, then the server now parses to extract the filename. Include that protocol in a test run of the client and server

  4. Once your changes are tested and debugged, commit and send them to stogit. 

    git add work.c Makefile
git commit -m "Task J DONE - Incorporate parser into server"
git pull origin mybranch
git push origin mybranch

git checkout master
git merge mybranch
git pull origin master
git push origin master

Link to index

K. Add logging (log.c) (Prereq: Task G; involves server.c work.c)

The log feature described in the assignment sheet calls for writing three lines to a text file to record results of receiving and responding to a client request.

  1. Switch to your development branch

    git checkout mybranch
    If there may have been updates to master since you last used mybranch (e.g., updates by a team member), then update mybranch as follows.
    git merge origin/master
git push mybranch

  2. Start a new source file log.c that defines five functions:

    • start_log(), which opens a file server.log for appending and writes a line indicating the beginning of a new series of log entries. 

      START log timestamp
      where timestamp represents the date and time of that call to start_log().

      • The fopen() call will return a FILE* value. Store that value in a static variable log

        static FILE *log;
        defined in the file log.c before the definition of start_log(). Here, the keyword static indicates that log is accessible only within the file log.c.

      • For the timestamp string, allocate a local variable now of type time_t and a (local) array timestamp of 30 chars, and fill that array using the following calls to the library functions: 

          now = time(NULL);
  strftime(timestamp, 30, "%a, %d %b %Y %T %Z", gmtime(&now));
        Note that these library calls could return errors. In particular, strftime() returns 0 if the value that would be assigned to timestamp requires more than 30 characters, including nullbyte. Otherwise, strftime() returns the number of characters stored in timestamp, excluding nullbyte. It is wise to check for this error, since we expect the RFC 1123 format needed for this project to include exactly 29 characters, excluding nullbyte.

        • Look up the #include files needed for those library calls and add them to your file log.c .

      • Do not close the log file within start_log(). We will reuse the static variable log for the four functions below.

      • The project assignment doesn't specify the line printed by start_log() (nor the line printed by end_line() below), but these aren't prohibited, and they serve as delimiters for log output from multiple runs of your server.

    • start_work(), to print the first log line for a client interaction on the open log file.

      • start_work() will need two arguments: the integer request_id; a thread ID (which should be the pthread ID once you implement the thread Task Q, but otherwise can be an arbitrary value like 1).

      • Use the code above involving time() and strftime() to obtain the timestamp string required for the first log line.

    • received_protocol(), to print the second log line for a client interaction on the open log file.

      • received_protocol() should have two arguments, an integer request_id and a string protocol name (GET) extracted from parsing the client's message.

    • sent_protocol(), to print the final log line for a client's interaction.

      • sent_protocol() should have two arguments, an integer request_id and a string response code (e.g., 200 OK) that was sent to the client in response to its request.

    • end_log() with no arguments, which prints a message 

       
END log
      and closes the FILE* variable log.

  3. Also, create a file log.h that contains function declarations for the five functions described above.

  4. Add 

    #include "log.h"
    to server.c, and call start_log() near the beginning of main() within server.c.

  5. Also include log.h in work.c. Add a call of received_protocol() to your function do_work() in an appropriate place (e.g., just after parsing).

    • Note: Add your received_protocol() call to your function worker() instead if you have already implemented the thread Task Q.

  6. Add a call of sent_protocol() to do_work() (or worker()) just after you send your reply to the client. For now, use "ACK" for the second argument; later, replace by the actual protocol you send back to the client (e.g., "200 OK").

  7. Modify the Makefile to incorporate the new source files.

    • Add a new target log.o with prerequisites log.c and log.h (and empty action, since the default compilation is sufficient).

    • The targets sender.o and work.o should both now have log.h as an additional prerequisite.

    • The target sender should now have log.o as an additional prerequisite and log.o should be added to the linking command in the action for that target.

  8. The make command should now generate your updated server. Debug your new code as necessary, and test with the (unmodified) client.

  9. Once your changes are tested and debugged, commit and send them to stogit. 

    git add work.c log.c
git commit -m "Task K DONE - Add logging (log.c)"
git pull origin mybranch
git push origin mybranch

git checkout master
git merge mybranch
git pull origin master
git push origin master

Link to index

L. Error checking in parser (Prereq: Task J; involves parse.c work.c client.c)

The project spec calls for the server to receive two lines of protocol from the client.

GET pagename HTTP/1.1
Host: hostname
After Task J, the server has the capability to receive a one-line message from the client containing at least two tokens, and to extract the second token from that message.

In this step, we will check the first token in that first line of protocol, and take action if that token is not the string "GET". We will also modify the client and server in order to communicate two lines of text, and to verify that those lines match the protocol rules above for correct GET requests.

In fact, we will modify the client to send any number of two-line messages to the server, and develop a test set of messages for verifying that all cases of protocol error checking in the server. In future tasks, we can use those test messages to check that error checking hasn't been affected by some later code modification.

  1. Switch to your development branch

    git checkout mybranch
    If there may have been updates to master since you last used mybranch (e.g., updates by a team member), then update mybranch as follows.
    git merge origin/master
git push mybranch

  2. We will first modify the client to send multiple 2-line messages. Before building any code, we will specify the desired test messages and server responses. Perform these copy operations:

    cp ~rab/os/pp2/protocol_test.in .
cp ~rab/os/pp2/protocol_test.out .
cp protocol_test.in protocol_test.in_all
cp protocol_test.out protocol_test.out_all
chmod -w protocol_test.*_all
    The file protocol_test.in contains lines such as the following.
    GET mypage.html HTTP/1.1
Host: hostname
# 1
GET mypage2.html HTTP/1.1
Host: rns202-1.cs.stolaf.edu
# 2
GIT mypage.html HTTP/1.1
Host: hostname
# 3
Get mypage.html HTTP/1.1
Host: hostname
# 4

Host: hostname
# 5

# 6
GET nopage.html HTTP/1.1
Host: hostname
# 15
...
    Here, each section terminated by a line beginning with the # character represents a protocol message to for the client to send to the server. (The number following # will be ignored in client-server messages, but is included for human reference.)

    The file protocol_test.out contains lines such as the following.

    mypage.html
# 1
mypage2.html
# 2
Unimplemented protocol GIT
# 3
Unimplemented protocol Get
# 4
Unimplemented protocol Host:
# 5
Invalid request
# 6
Cannot open file
# 15
...
    Here, each one-line section terminated by a line beginning with the # character is a response message we want the server to send back to the client when that client sends the corresponding protocol message from protocol_test.in to that server. The numbers following # show the correspondence.

    Finally, the copies protocol_test.in_all and protocol_test.out_all will be used to record the original versions of protocol_test.in and protocol_test.out during testing. We write-protected these with a chmod command in order to avoid inadvertently changing these originals.

  3. As a first use of the protocol test files, delete all but the first two lines of protocol_test.in, and all but the first line of protocol_test.out.

    • protocol_test.in should contain only

      GET pagename HTTP/1.1
Host: hostname

    • protocol_test.out should contain only

      pagename

    Now try your (unmodified) client and server, except invoke the client using this command:

    ./client hostname port < protocol_test.in | diff - protocol_test.out
    Here, hostname and port are values for connecting to your running server. We expect the following results:

    • Presumably your client still reads only one line from standard input. In this case, that line will be the first line of the current protocol_test.in, which is 

      GET pagename HTTP/1.1

    • When the server receives this (one-line) request, it will extract the second token pagename and send that token back to the client.

    • The client will receive that message and print it on standard output (according to Task H).

    • The output will be piped to the diff command, which compares to protocol_test.out.

    • The output from diff should include your client's prompt for input, but no other differences (unless your client prints a label when it prints the response message received from the server).

    If you observe a different behavior, make sure you understand why, and modify your code if necessary to achieve this behavior (e.g., removing a label from client's output of a response message, so the output matches the line in protocol_test.out).

  4. Now insert the printed form of your client's prompt for input to the beginning of your file protocol_test.out. This should cause the diff command in the ./client pipeline above to print nothing, indicating an exact match of your client's output with the expected output in the modified protocol_test.out.

    Record your protocol test files in a commit.

    git add protocol_test.* 
git commit -m "Task J - first version of protocol testing files"
    You can optionally pull/push this commit in order to submit to stogit, or you can wait until a later commit is pull/pushed. Likewise, if you modified your client, you could add the modified code files before creating this commit, or wait to commit these with other code changes in a later commit for this task.

  5. Create a shell script client_tests that performs the client testing pipeline above. This means to create a file client_tests that consists of the following:

    #! /bin/bash

./client $1 $2 < protocol_test.in | 
diff - protocol_test.out
    Perform the following shell command to make client_tests an executable.
     
chmod +x client_tests
    Then verify that the script client_tests behaves the same as the original client pipeline above by entering the following shell command.
    ./client_tests hostname port
    where hostname and port are the hostname and port for your (already started!) server.

    Notes:

    • The first two characters #! of the first line of a file

      #! /bin/bash
      mark that file as a script, assuming that file also has executable mode bit(s) (which the chmod +x command accomplished).

    • This is a shell script because the first line indicates that a shell /bin/bash should be used for interpreting the remaining lines of the file. (You could make scripts for other languages by replacing /bin/bash by a different programming language processor.)

    • The tokens $1 and $2 in the shell script represent two command-line arguments for the script. Note: This shell script doesn't have any error checking, so be sure to supply a host and port number whenever you use ./client_tests .

    • We added a newline after the pipe character | for readability.

  6. Modify your client to read all lines of standard input up to end of input into a single message buffer, including newline characters within that buffer after each input line. Then, send that multiline message to the server. We expect the same response as for a one-line message, since the server currently only examines one line (assuming Task J).

    • Define a character array msg for collecting the input lines, e.g., 

      char msg[MAXMSG]
      where the preprocessor constant MAXMSG is large enough to hold one entire multiline protocol. Or, you can use an existing array that you already use in the client for receiving the server's response. Make sure at the array size (e.g., MAXMSG is large enough - 1000 characters should be plenty for the basic assignment.

    • Initialize msg[] as an empty string (nullbyte in msg[0]). Then, for each line of input, use a library function such as strcat() to append each new line of input to msg[].

    • If your client uses C's getline() library function for input (as did sender.c), note that getline() copies each line's newline character (if there is one) automatically.

    • Send the entire (multi-line) msg string, up to but excluding the terminating nullbyte, to the server in a single message, i.e., in a single send() socket call.

      • Now, start your server and use client_tests to test that server, verifying that the server receives the entire multi-line message then replies with the second token of the first line, as before.

      • If necessary, add a temporary printf() to work() in the server in order to verify that the server received the entire two-line message. However, your unmodified server code may already implement the output of the request message.

  7. Make a commit of your successful changes so far.

    git add client.c client_tests
git commit -m "Task L - multi-line input in client, multi-line message to server; script client_tests"

  8. Your client now has a loop for reading input lines and appending them to msg[] until the end of input. Modify that loop to stop repeating if either end of input is encountered or a line beginning with # is encountered.

    To test that modification, add the four lines 

    # 1
GET mypage.html HTTP/1.1
Host: rns202-1.cs.stolaf.edu
# 2
    at the end of protocol_test.in, and test the server using client_tests. Your server should receive only
     
GET pagename HTTP/1.1
Host: hostname
    and no other lines, and the server's response to the client should be unchanged (so the pipeline should show no differences with the unmodified protocol_test.out). Debug as necessary to obtain this expected behavior.

    Now, add an outer loop in your client to send multiple requests to the server.

    • The body of that outer loop should start with the client's socket() call and end after closing the socket returned by that call (in do_request()).

    • Exit the outer loop only when end of file is encountered (not #).

    • One way to manage the outer loop is to use the do while loop syntax, e.g., 

      do { 
  if ((sd = socket(AF_INET, SOCK_STREAM, 0)) < 0) {
    ... 
  }
  ...
} while (nchars != -1);
      Here, we assume that nchars holds the return value from getline().

      • Note: nchars is a local variable in do_request(). Pass the value of nchars as the return value of do_request(), and assign it (in your do loop) to a variable nchars for the guard evaluation.

      • Also check all the return values from do_request() to return the value of nchars for the most recently call of getline().

    • Consider moving your loop to read multiple lines of input into msg[] before the socket() call, since there's no need to make a socket and try to connect() if there is no message to send. This might lead to an outer loop with this structure:

      do { 
  /* loop to read input and assemble msg[] */
  if (nchars != 1) {
    if ((sd = socket(AF_INET, SOCK_STREAM, 0)) < 0) {
      ... 
    }
    ...
  }
} while (nchars != -1);
      or an equivalent structure of your choice.

      Also, consider moving the prompt for input outside of the outer loop. We only want that prompt for interactive use, and we would probably only enter one protocol section interactively, which would mean there only needs to be one prompt. This would avoid multiple prompts in the client's output, which makes it easier to add tests to protocol_test.out.

    • Add the following line to the end of your protocol_test.out

       
mypage.html
      since we now expect the client to send two requests. (Here, we are omitting the # lines in the original protocol_test.out_all .)

    Now test your server using client_tests. We expect the following:

    • First, the client should read the first protocol section from standard input, which will be 

      GET pagename HTTP/1.1
Host: hostname
      client should not add the third line #1 to msg[], but should proceed to send only that two-line message to the server.

    • The server should receive exactly those two lines, and should respond to the client with a one-line message pagename

    • The client should receive that one-line message and should print it.

    • Then, in the second iteration of its outer loop, the client should read the second protocol section from standard input, namely

      GET mypage.html HTTP/1.1
Host: rns202-1.cs.stolaf.edu
      Again, the client should detect the #2 line and take action to stop its input loop and forward the two-line message to the server.

    • In a second iteration of the loop in server.c's main(), the server should receive that second two-line message, then reply to the client with the message mypage.html .

    • The client should receive that reply and print it.

    • On the third iteration of the client's outer loop, end of input should be encountered by the client. The client should then exit the outer loop and exit, without sending anything more to the server.

    • Finally, the diff in the client's pipeline should report no differences.

  9. Make a commit of your successful changes so far.

    git add client.c protocol_test.in protocol_test.out
git commit -m "Task L - multiple request capability in the client"

  10. Now that we have upgraded the client for testing, we are ready to start modifying the server to check the validity of a request from the client. First, define a char* variable reply at the beginning of the function do_work() in work.c (or in the function worker() if you have already completed Task Q), and assign a dynamically allocated array of 8500 bytes to reply.

    • 8500 bytes is large enough for up to 300 bytes of header and 8K of content in a reply message for the client.

    Also, after parsing the client's request in do_work() (or worker()), copy the second token to reply[] (e.g., with strcpy()), then send() the contents of reply[] to the client. The (dynamically allocated) array reply[] is large enough to hold any response message for the client in the basic assignment, not just the temporary reply message consisting of that second token. Don't forget to deallocate reply using free(), e.g., just after using it to send() the response to the client.

    Recompile, then test your server with client_tests to check that you get the same results as before.

  11. For the basic assignment, the server must not only recognize correct GET requests and act on those, but must also detect when the first token of protocol is not GET, and respond appropriately in that case.

    We will use the library function sscanf() (string scanf) to examine that first token, and postpone full parsing of a request until we know that first token is GET. sscanf() performs input the same as scanf() does, except sscanf() reads from a string instead of standard input. For example, the code 

    char *str = "62 49";
int val;
sscanf(str, "%d", &val);
    assigns the value 62 to the integer variable val.

    1. In the function do_work() (or worker()), use sscanf() with the format string "%s" to read the first string (up to the first whitespace character) from the request message received from the client into an array of characters 

      char first[100];

    2. Check that the return value from sscanf() is 1 which means that one value was assigned (namely, the value of first[]). If that return value is not 1, then there must not be a first token in the message from the client, so copy

       
Invalid request
      to reply[] and send reply to the client. Also call received_protocol() to record that an Empty protocol was received from the client.

    3. Otherwise, a first token exists. Compare that first string to "GET", e.g., using strcmp().

      • If the string in first[] matches "GET", proceed to your code for parsing the client's request and send the second token to the client as before. Call received_protocol() to record that GET was received from the client.

      • If first[] does not match "GET", send the string 

        Unimplemented protocol first
        to the client, where first is the string value contained in first[]. (You could use library functions such as strcpy() and strcat().) Call received_protocol() to record that first was received from the client.

    4. Double-check your code to insure that the dynamically allocated memory for reply is correctly deallocated using free() in all cases. Also check to make sure that received_protocol() is called only once per client request.

    Add the test case #3 to your protocol_test.in and protocol_test.out files.

    • This means to add 3 lines 

      GIT mypage.html HTTP/1.1
Host: hostname
# 3
      to the end of protocol_test.in (these lines can be found in protocol_test.in_all) , and add the two lines 
       
Unimplemented protocol GIT
# 3
      (found in protocol_test.out_all) to the end of protocol_test.out .

    Recompile, and test your server with client_tests.

    • Note: The label #3 in protocol_test.out will cause an extra section of diff output from the script client_tests. To avoid this, add the following line to the script client_tests:

      #! /bin/bash

./client $1 $2 < protocol_test.in | 
grep -v '^#' |
diff - protocol_test.out
      The added line inserts another command (grep) to the pipeline that removes all lines beginning with # from the output stream from ./client .

    • You may optionally add labels #1 and #2 to your protocol_test.out file if you wish.

    Check that you see no unexpected diff output or other issues. Debug as needed.

    Then, add the test cases #4, #5, #6, and #7 to your protocol_test.in and protocol_test.out files.

    • This means to add 9 lines 

      Get mypage.html HTTP/1.1
Host: hostname
# 4

Host: hostname
# 5

# 6
# 7

      • These lines can be found in protocol_test.in_all .

      • The final line #7 causes an empty request to be sent to the server) to the end of protocol_test.in

      Also add the 8 lines 

      Unimplemented protocol Get
# 4
Unimplemented protocol Host:
# 5
Invalid request
# 6
Invalid request
# 7
      (found in protocol_test.out_all) to the end of protocol_test.out .

    Recompile, and test your server with the (modified) client_tests to check that you see no unexpected diff output or other issues. Debug as needed.

    • Note: If you encounter a segmentation fault for one of the test cases #6 and/or #7, be sure to check the return value from sscanf() before using the value of first[], since sscanf() doesn't necessarily assign a value to its variable first when the format string isn't matched.

  12. Make a commit of your successful changes so far.

    git add work.c protocol_test.in protocol_test.out client_tests
git commit -m "Task L - Checking the first token in client's protocol"

  13. The server now detects cases when a client request doesn't begin with the token GET, and takes action in those cases. We will now proceed to take action when the first token is GET.

    Define a function parse_GET() in parse.c for validating the protocol for a GET request.

    parse_GET

    2 args: A char* message received from the client, and an array of char.

    State change: A string is copied into arg2 that indicates whether or not the client message arg1 satisfies the protocol rules for an HTTP GET request.

    Return: An int value 1 if arg1 is a valid HTTP GET request, and 0 otherwise.

    Note: For the remainder of this task, we will use the name request to refer to arg1 in this spec, and the name reply for arg2.

    Rewrite your do_work() to call parse_GET() (or rewrite worker(), if you have already implemented Task Q), and move your code for parsing the client's request message to parse_GET().

    • In do_work()/worker(), pass the 8500-byte string reply as the first argument for the call of parse_GET(), and pass the request message received from the client as the second argument for that call. (Thus, we're using the name reply for both the local variable in do_work()/worker() and for arg1 within parse_GET().)

    Your initial version of parse_GET() should parse the string request, then copy the second token to reply[] (i.e., parse_GET()'s arg2) and return 1. Also, after do_work() (or worker()) calls parse_GET(), do_work()/worker() should send the value in (its local variable) reply to the client as before.

    • Within parse_GET(), you can copy a string into reply[] using a library function such as strcpy().

    • Be sure to add a declaration of parse_GET() to parse.h, so work.c can compile using the new function.

    Recompile, and test your server with client_tests to check that you see no unexpected diff output or other issues. Debug as needed.

  14. Make a commit of your successful changes so far.

    git add work.c parse.c parse.h
git commit -m "Task L - add function parse_GET() in parse.c"

  15. The remainder of this task involves modifying parse_GET() to check for token-count errors represented in protocol_test.in_all and generate the appropriate one-line responses to the client indicated in protocol_test.out_all.

    Insert an if statement in parse_GET() just after your code for parsing the (first line of) the argument request[] into tokens, to check whether the number of tokens equals 3. If the count of tokens is not 3, copy the string "Invalid request" to reply[] and return the value 0 from parse_GET(). Otherwise, proceed with the rest of your code for parse_GET().

    • As before, you can copy that string into reply[] using strcpy().

    • Note: No else is required for this if - you can simply proceed with the remainder of the parse_GET() code, since that code will not be reached by the CPU if you have already returned from parse_GET() during that if statement.

    Add test case #8 to the end of your protocol_test.in, i.e., the three lines 

     
GET try.txt 
Host: hostname
# 8
    and add the corresponding two lines
    Invalid request
# 8
    to the end of your protocol_test.out. Then recompile and test run your server against client_tests, and verify that the diff produces no output, or debug until this test succeeds.

  16. Add test cases #9 and #10 to your files protocol_test.in and protocol_test.out .

    Then recompile and test run your server with client_tests, and verify that the diff produces no output, or debug until this test succeeds.

  17. Make a commit of your successful changes so far.

    git add parse.c protocol_test.in protocol_test.out
git commit -m "Task L - parse_GET() checks number of tokens in first protocol line"

  18. Now add code to parse the second line of protocol in parse_GET()'s argument request[], using a second data structure (in order to preserve the results of parsing the first line of protocol).

    • Add this code after the if for checking the number of tokens in the first protocol line, but before returning the second token.

    • We suggest that you define a char* pointer variable secondp and assign address of the first element of request[] after the first newline to secondp, and parse starting at secondp instead of at the beginning of request. (We will use secondp below in order to refer to the beginning of the second protocol line within request[].)

    • Parse the second line into a separate data structure, so you can still access tokens of the first line.

    Then, add a second if for checking the count of tokens in the second line of protocol. If that count is not 2, copy the string "Invalid request" to reply[] and return the value 0 immediately. Otherwise, proceed with the rest of your code for parse_GET() (i.e., copy the second token of the first line to reply[], deallocate the memory that was dynamically allocated for tokens, and return 1).

    Add test case # 11 to your files protocol_test.in and protocol_test.out . Recompile and test run your server against client_tests, and verify that the diff produces no output, or debug until this test succeeds.

  19. Add test cases #12 through #14 to your files protocol_test.in and protocol_test.out . Recompile and test run your server with client_tests, and verify that the diff produces no output, or debug until these tests succeed.

  20. Once your changes are tested and debugged, commit and send them to stogit. 

    git add parse.c protocol_test.in protocol_test.out
git commit -m "Task L DONE - Error checking in parser"
git pull origin mybranch
git push origin mybranch

git checkout master
git merge mybranch
git pull origin master
git push origin master

Link to index

M. Send file to client (Prereq: Task L; involves parse.c work.c Makefile client_tests)

We are now ready to send a requested file to the client, using the correct HTTP protocol for the response message.

  1. Switch to your development branch

    git checkout mybranch
    If there may have been updates to master since you last used mybranch (e.g., updates by a team member), then update mybranch as follows.
    git merge origin/master
git push mybranch

  2. Copy the files mypage.html, mypage2.html, and protocol_test.out_200 to your working directory.

    cp ~rab/os/pp2/mypage.html .
cp ~rab/os/pp2/mypage2.html .
cp ~rab/os/pp2/protocol_test.out_200 .
    The two .html files are examples for protocol testing, and protocol_test.out_200 has expected replies in HTTP protocol from the server to the client for the first two test requests (only) in protocol_test.in_all.

    Note for Fall 2019: Also make fresh copies of protocol_test.in_all and protocol_test.out_all for use in this task.

    chmod +w protocol_test.in_all protocol_test.out_all
cp ~rab/os/pp2/protocol_test.in protocol_test.in_all
cp ~rab/os/pp2/protocol_test.out protocol_test.out_all
chmod -w protocol_test.in_all protocol_test.out_all

  3. Modify parse.c and parse.h to add a function http_header() that satisfies the following spec.

    http_header

    3 args: An array of characters, a char* string representing a protocol response code (e.g. "200 OK") and an int string representing the length of content to follow the header.

    State change: A valid HTTP header of type arg2 with Content-Length: value arg3 is assigned to the beginning of the array arg1, followed by a nullbyte.

    Return: char *, the (modified) array arg1.

    For example, the call http_header(reply, "200 OK", 425) should assign the following string to the beginning of reply[].

    "HTTP/1.1 200 OK\r
Content-type: text/html; charset=utf-8\r
Content-length: 425\r
\r
"
    (recall that \r represents the CR character, required for the ends of HTTP header lines). This HTTP header would be appropriate for sending a 425-byte file in response to a client's GET request.

    • Note: As this example shows, you may omit the Date: and Connection: fields in HTTP reply headers for this project.

    • Add a function declaration for http_header() to parse.h, and add the function definition to parse.c.

    Then, modify do_work() (or worker()) to check the return value from its call of parse_GET().

    • If parse_GET() returns 0 (indicating that parsing failed), then

      • send reply[] to the client as before,

      • Call sent_protocol() to record that reply[] was sent to the client.

      • deallocate reply[] using free(), and

      • return from do_work() (or worker())

    We can now assume that parse_GET() returns 1 (successful parsing of client's request message). Carry out these steps:

    • copy reply[] (the file name that was the second token of the GET request) to a char array variable filename,

    • call http_header() to assign a   200 OK   header to reply[] with the length of filename as the third argument,

    • append filename to reply[], and

    • send this revised reply[] to the client.

    To test these changes, replace the first two tests (four lines) of your protocol_test.out by the two sections of protocol_test.out_200, so that your updated protocol_test.out begins with 

    HTTP/1.1 200 OKCR
Content-Type:  text/html; charset=utf-8CR
Content-Length:  91CR
CR
<html>
<head><title>My page</title></head>
<body><p>This is mypage.html</p></body>
</html>
# 1
HTTP/1.1 200 OKCR
Content-Type:  text/html; charset=utf-8CR
Content-Length:  107CR
CR
<html>
<head><title>My page 2</title></head>
<body><p>This is <code>mypage2.html</code></p></body>
</html>
# 2
Unimplemented protocol GIT
# 3
Unimplemented protocol Get
# 4
...
    where CR represents a carriage return character (displays as ^M in emacs).

    • The file protocol_test.out_200 contains these revised sections #1 and #2, including those carriage-return characters, so you can copy and paste from that file rather than entering manually.

    Recompile, and test your server with client_tests. This time, we expect client_tests to produce differences: your client should print 

    mypage.html
    instead of 
    <html>
<head><title>My page</title></head>
<body><p>This is mypage.html</p></body>
</html>
    and your client should print 
    mypage2.html
    instead of 
    <html>
<head><title>My page 2</title></head>
<body><p>This is <code>mypage2.html</code></p></body>
</html>
    But there should be no other differences. Debug as needed.

  4. Make a commit of your successful changes so far.

    git add parse.c parse.h work.c protocol_test.out protocol_test.in_all protocol_test.out_all
git commit -m "Task M - add html_header(), revise protocol_test.out, and do preliminary test"

    • You may optionally also commit protocol_test.out_200. However, this file won't be needed for future steps or tasks. Alternatively, you can either delete that file or add that filename to a file .gitignore in your working directory (then commit .gitignore).

  5. Now, modify your do_work() (or worker()) to send the contents of the file filename instead of just the name of that file.

    • First, try opening that file, e.g., 

      FILE *fp = fopen(filename, "r");
      Be sure to examine the value of fp to determine whether there was an error opening that file. If there was an error, then

      • assign the string 

        "Cannot open file"
        to reply[] (replacing whatever was already there),

      • send that message reply[] to the client,

      • call sent_protocol() to record that "Cannot open file" was sent

      • free() the dynamic allocation of reply[], and

      • return

    • We can now assume that the file opened successfully. We will need the file's length in order to call http_header(). The following code finds that length:

      fseek(fp, 0L, SEEK_END);
int sz = ftell(fp);
rewind(fp);
      Here,

      • the fseek() call positions fp to the end of the open file,

      • the ftell() call returns the integer position of fp, which is the number of bytes in that file since we are positioned at the end, and

      • the rewind() call repositions fp to point to the beginning of the open file (which we will need in order to copy that file's contents into reply[] later).

    • Now use your existing http_header() call to insert an HTTP 200 header at the beginning of reply[], except use the file length sz determined above for the third argument of that call.

    • Finally, append the file contents to reply[]. This can be done by a call of the library function fread(). Consulting with the fread() manual page, we use the following arguments for this call.

      • fread()'s first argument is the array to copy into. This can be expressed using pointer arithmetic as 

        reply + strlen(reply)
        since strlen(reply) is the number of bytes in the HTTP 200 header now located at the beginning of reply[].

      • fread()'s second argument should be 1, meaning that we are reading individual bytes at a time (not ints or some other larger unit).

      • fread()'s third argument should be number of byte units to read, namely, sz.

      • fread()'s final argument is the FILE* stream fp.

    Recompile, and test your server with client_tests. After these changes, we expect no output from the tests. Debug as necessary.

  6. Modify your call of sent_protocol() in do_work() (or worker()) to print "200 OK" as the response code for a successful file transfer reply.

  7. Make a commit of your successful changes so far.

    git add work.c
git commit -m "Task M - test sending of files to client, and check for successful fopen call"

  8. To test for Cannot open file, add test case #15 to your files protocol_test.in and protocol_test.out .

    • This request refers to a file nopage.html that is not supposed to exist. Do not create that file (or if there's some reason you need that filename, modify that test request #15 to use a name that doesn't exist as a file.

    Recompile and test run your server with client_tests, and verify that the diff produces no output, or debug until these tests succeed.

  9. Before leaving this task, we will refactor and automate our approach for creating new versions of protocol_test.out that have HTTP protocol instead of one-line informational messages. For this task, we modified protocol_test.out manually for cases #1 and #2, but this will be too laborious and error prone for all the tests we want to make.

    • First, we will retrieve our prior version of protocol_test.out, just before adding HTTP 200 messages. 

      mv protocol_test.out protocol_test.out_new.saved
git log
      The git log command displays a local history of commits that were made, in reverse chronological order. Identify the commit at the beginning of this task, i.e., just before the commit you made with the message
      Task M - add html_header(), revise protocol_test.out, and do preliminary test
      Commits have random 40-character hash names, but you only need to use the first few characters of such a name to refer to a commit in a git command - the first 5 or 6 characters would be enough (e.g., a47xc6).

    • Retrieve the version of protocol_test.out from that commit using this command.

      git checkout hash -- protocol_test.out
      where hash is the abbreviated name of the commit you found. The retrieved file should start with
      mypage.html
# 1
mypage2.html
# 2
Unimplemented protocol GIT
# 3
Unimplemented protocol Get
# 4
...

    • Now copy the script add_http and associated file into your working directory.

      cp ~rab/os/pp2/{add_http,[245]*.{html,http}} .
      This wildcard notation should copy the following files into your working directory:
      add_http 200.http 400.html 400.http 404.html 404.http 501.html 501.http

    • Make a commit of your successful changes so far.

      git add add_http 200.http 400.html 400.http 404.html 404.http 501.html 501.http
git commit -m "Task M - Add script add_http and related files"

    • Modify Makefile to add a new prerequisite protocol_test.out_new to the default target all: (e.g., after client and server) also add the following new rule (e.g., at the end of Makefile).

      protocol_test.out_new: 
	cp protocol_test.out protocol_test.out_new
      Test these changes by issuing make. The targets client and server should be up to date, but we expect the file protocol_test.out_new to be generated as a copy of protocol_test.out. Verify that these files have the same content at this point.

    • Now add another Makefile rule for updating the file protocol_test.out_new.

      protocol_test.out_new: protocol_test.out
	./add_http 200
      Also, enter the following two shell commands.
      chmod +x add_http
touch protocol_test.out
      Explanation:

      • The chmod command ensures that the script add_http will be executable.

      • The touch command changes the "last modified" date to the current time for the file protocol_test.out. This will trigger the Makefile rule you just added for the target protocol_test.out_new.

      Now issue the make command, which should perform the command 
      ./add_http 200
      to update the target protocol_test.out_new.

      We now expect protocol_test.out_new to be the result of substituting HTTP 200 protocol sections for the first two cases #1 and #2. In other words, the command add_http 200 should perform the changes you made manually above. Verify that this is the case using the command 

      diff  protocol_test.out_new  protocol_test.out_new.saved
      There should be no differences.

      • If you do find differences, determine whether this is due to errors in the use of the script add_http or errors that you made when manually substituting HTTP 200 protocol, in order to determine how to proceed.

      • You can retest this step by using 

        touch  protocol_test.out
        to regenerate a new version of protocol_test.out_new.

    • Modify your own script client_tests to perform its diff command using protocol_test.out_new instead of protocol_test.out . Now test run your server with client_tests, and verify that client_tests produces no output, or debug until this test succeeds.

      After this check, we now have an automated way to transform protocol_test.out into a file protocol_test.out_new that incorporates desired HTTP reply protocols, using make and the provided script add_http, and your client_tests script can check against the HTTP reply protocols you have implemented so far.

      • After this check, we no longer need the old manually produced file protocol_test.out_new.saved , so you can either delete that file or add it to .gitignore .

      Note: If you made any changes (e.g., in server source code) in order to make steps above work correctly, commit those changes with an appropriate commit message.

  10. Make a commit of your successful changes so far.

    git add Makefile client_tests
git commit -m "Task M - Integrate add_http into Makefile and client_tests, for automated substitution of HTTP reply protocols"

    Note:

    • We do not commit protocol_test.out_new because that file is generated by Makefile and add_http.
      Add this filename to the file .gitignore in your working directory (creating a new one if necessary) so the git system will know that protocol_test.out_new doesn't need to be committed if it changes.

  11. Finally, add some documentation to your new Makefile rules

    • Add the following boldface line just before the rule for creating a default protocol_test.out_new .

      # default creation of protocol_test.out_new
protocol_test.out_new: 
	cp protocol_test.out protocol_test.out_new

    • Add the following documentation line just before your rule for updating protocol_test.out_new .

      # regenerate protocol_test.out_new with HTTP reply protocols impl to date 

  12. Once your changes are tested and debugged, commit and send them to stogit. 

    git add Makefile .gitignore
git commit -m "Task M DONE - Send file to client"
git pull origin mybranch
git push origin mybranch

git checkout master
git merge mybranch
git pull origin master
git push origin master

Link to index

N. Implement 501 (Not Implemented) (Prereq: Task L; involves work.c Makefile)

Your function do_work() (or worker()) uses a library function such as sscanf() to examine the first token of the client's request message. If that token is GET, your code calls parse_GET() and proceeds to attempt to send a file to the client.

But if that first token is not GET, your code sends an Unimplemented protocol message to the client. We will now implement the correct HTTP 501 reply to that situation.

  1. Switch to your development branch

    git checkout mybranch
    If there may have been updates to master since you last used mybranch (e.g., updates by a team member), then update mybranch as follows.
    git merge origin/master
git push mybranch

  2. For Fall 2019: Make fresh copies of protocol_test.in_all and protocol_test.out_all for use in this task.

    chmod +w protocol_test.in_all protocol_test.out_all
cp ~rab/os/pp2/protocol_test.in protocol_test.in_all
cp ~rab/os/pp2/protocol_test.out protocol_test.out_all
chmod -w protocol_test.in_all protocol_test.out_all

  3. Locate the code in do_work() (or worker()) where the Unimplemented protocol... message is constructed (in reply[]) and sent to the client.

    • This is the case where first[] does not equal "GET".

    Instead of constructing a message 

    Unimplemented protocol first
    (where first is the non-GET first token your code finds), construct a two-part message consisting of a "501 Not Implemented" HTTP header followed by the contents of the file 501.html .

    • Delete or comment out the code for inserting Unimplemented protocol first at the beginning of reply[].

    • Open the file 501.html using fopen().

      • The file 501.html should have been added to your team's repository in Task M. Check whether this file opens correctly using fopen(), and if it fails to open, consult with the person who carried out that task if necessary to determine what happen and add 501.html and other files to your team's stogit repository and your own working directory.

    • Determine the length of 501.html using fseek() and ftell() as above, and call rewind() to reposition the open file back to the beginning.

    • Call http_header() to insert an HTTP 501 header at the beginning of reply[], using the length of 501.html for the third argument of that call.

    • Append the contents of 501.html to reply[] using fread() as before.

    • Send the resulting message reply[] to the client

      Modify your call of sent_protocol() for this case to indicate that "501 (Not implemented)" was sent to the client

      (Retain the free() call for deallocating the memory that was dynamically allocated to reply[].)

  4. Update the file protocol_test.out_new to include the proper HTTP 501 protocol replies.

    • Add a command 

      ./add_http 501
      to the actions for the updating rule for protocol_test.out_new . Thus, that updating rule might now be 
      # regenerate protocol_test.out_new with HTTP reply protocols impl to date
protocol_test.out_new: protocol_test.out
	./add_http 200
	./add_http 501

    • Enter the commands

      touch protocol_test.in
make
      to regenerate protocol_test.out_new .

    • Check the updated contents of protocol_test.out_new. For example, section #3 of protocol_test.out_new should now consist of the following lines:

      HTTP/1.1 501 Not ImplementedCR
Content-Type:  text/html; charset=utf-8CR
Content-Length:  208CR
CR
<html>
<head><title>501 Not Implemented</title></head>
<body><p style='text-align:center; font-size:150%'>501 Not Implemented</p>
<p>Sorry, this server doesn't handle that type of request.</p></body>
</html>
# 3

  5. Recompile, and test run your server with client_tests. Verify that the diff produces no output, or debug until these tests succeed.

  6. Once your changes are tested and debugged, commit and send them to stogit. 

    git add work.c Makefile
git commit -m "Task N DONE - Implement 501 (Not Implemented)"
git pull origin mybranch
git push origin mybranch

git checkout master
git merge mybranch
git pull origin master
git push origin master

Link to index

O. Implement 400 (Bad Request) (Prereq: Task L; involves work.c parse.c Makefile)

After Task L, your server can handle requests that did not begin with the token GET and requests that contained no tokens. For GET requests, your server checks whether the first two lines had the right numbers of tokens. We will now add further checks to insure that those tokens have the expected values, and modify the server to reply with the HTTP 400 protocol message when a client request has a valid request format

  1. Switch to your development branch

    git checkout mybranch
    If there may have been updates to master since you last used mybranch (e.g., updates by a team member), then update mybranch as follows.
    git merge origin/master
git push mybranch

  2. In do_work() (or worker()) we check whether a client's message has no tokens. If there are no tokens, we send the reply[] value

    Invalid request
    to the client.

    Replace that message by an HTTP 400 protocol message.

    • Open the file 400.html and determine its length, as before. Call rewind() to reposition the open file back to the beginning.

    • Call http_header() to insert an HTTP 400 header at the beginning of reply[], using the length of 400.html for the third argument of that call.

    • Append the contents of 400.html to reply[] using fread() as before.

    • Send the resulting message reply[] to the client

      Modify your call of sent_protocol() for this case to indicate that "400 (Bad request)" was sent to the client

      (Retain the free() call for deallocating the memory that was dynamically allocated to reply[].)

  3. Later in do_work() (or worker()), your code sends Invalid request to the client if parse_GET() returns 0. Also replace that message by an HTTP 400 protocol message.

  4. Update the file protocol_test.out_new to include the proper HTTP 400 protocol replies.

    • Add a command 

      ./add_http 400
      to the actions for the updating rule for protocol_test.out_new .

    • Enter the commands

      touch protocol_test.in
make
      to regenerate protocol_test.out_new .

    • Check the updated contents of protocol_test.out_new. For example, section #6 of protocol_test.out_new should now consist of the following lines:

      HTTP/1.1 400 Bad RequestCR
Content-Type:  text/html; charset=utf-8CR
Content-Length:  195CR
CR
<html>
<head><title>400 Bad Request</title></head>
<body><p style='text-align:center; font-size:150%'>400 Bad Request</p>
<p>The server received an incorrectly formed request.</p></body>
</html>
# 6

  5. Recompile, and test run your server with client_tests. Verify that the diff produces no output, or debug until these tests succeed.

  6. Make a commit of your successful changes so far.

    git add work.c Makefile
git commit -m "Task O - HTTP 400 protocol replies"

  7. Now add further tests in parse_GET() to insure that tokens in the client's request have expected values.

    • In parse_GET(), after you have verified that there are three tokens in the first line of protocol received from the client, check that the third token matches the string HTTP/1.1. If the strings don't match, return 0 from parse_GET()

    • To test this case, add sections #16 and #17 to protocol_test.in and protocol_test.out (not protocol_test.out_new).

    • Recompile, test your server with client_tests, and verify that client_tests produces no output, or debug until this test succeeds.

    • Make a commit of your successful changes so far.

      git add parse.c
git commit -m "Task O - complete checking for first line of GET protocol"

    • In parse_GET(), after you have verified that there are two tokens in the second line of protocol received from the client, check that the first of those tokens matches the string Host:. If the strings don't match, return 0 from parse_GET()

    • To test this case, add sections #18 through #21 to protocol_test.in and protocol_test.out (not protocol_test.out_new).

    • Recompile, test your server with client_tests, and verify that client_tests produces no output, or debug until this test succeeds.

  8. Once your changes are tested and debugged, commit and send them to stogit. 

    git add parse.c
git commit -m "Task O DONE - Implement 400 (Bad Request)"
git pull origin mybranch
git push origin mybranch

git checkout master
git merge mybranch
git pull origin master
git push origin master

Link to index

P. Implement 404 (Not Found) (Prereq: Task M; involves work.c Makefile)

  1. Switch to your development branch

    git checkout mybranch
    If there may have been updates to master since you last used mybranch (e.g., updates by a team member), then update mybranch as follows.
    git merge origin/master
git push mybranch

  2. For Fall 2019: Make fresh copies of protocol_test.in_all and protocol_test.out_all for use in this task.

    chmod +w protocol_test.in_all protocol_test.out_all
cp ~rab/os/pp2/protocol_test.in protocol_test.in_all
cp ~rab/os/pp2/protocol_test.out protocol_test.out_all
chmod -w protocol_test.in_all protocol_test.out_all

  3. Locate the code in do_work() (or worker()) where the Cannot open file message is constructed (in reply[]) and sent to the client.

    • This occurs when a GET request has successfully been received, but the indicated file does not open correctly using fopen().

    Instead of copying a message 

    Cannot open file
    into reply[], construct a two-part message consisting of a "404 Not Found" HTTP header followed by the contents of the file 404.html .

    • Delete or comment out the code for inserting Cannot open file at the beginning of reply[].

    • Open the file 404.html using fopen().

    • Determine the length of 404.html using fseek() and ftell() as above, and call rewind() to reposition the open file back to the beginning.

    • Call http_header() to insert an HTTP 404 header at the beginning of reply[], using the length of 404.html for the third argument of that call.

    • Append the contents of 404.html to reply[] using fread() as before.

    • Send the resulting message reply[] to the client

      Modify your call of sent_protocol() for this case to indicate that "404 (Not implemented)" was sent to the client

      (Retain the free() call for deallocating the memory that was dynamically allocated to reply[].)

  4. Update the file protocol_test.out_new to include the proper HTTP 404 protocol replies.

    • Add a command 

      ./add_http 404
      to the actions for the updating rule for protocol_test.out_new .

    • Enter the commands

      touch protocol_test.in
make
      to regenerate protocol_test.out_new .

    • Check the updated contents of protocol_test.out_new. The section #15 of protocol_test.out_new should now consist of the following lines:

      HTTP/1.1 404 Not FoundCR
Content-Type:  text/html; charset=utf-8CR
Content-Length:  189CR
CR
<html>
<head><title>404 Not Found</title></head>
<body><p style='text-align:center; font-size:150%'>404 Not Found</p>
<p>Sorry, that page isn't available on this server.</p></body>
</html>
# 15

  5. Recompile, and test run your server with client_tests. Verify that the diff produces no output, or debug until these tests succeed.

  6. Once your changes are tested and debugged, commit and send them to stogit. 

    git add work.c Makefile
git commit -m "Task P DONE - Implement 404 (Not Found)"
git pull origin mybranch
git push origin mybranch

git checkout master
git merge mybranch
git pull origin master
git push origin master

Link to index

Q. Worker thread (Prereq: Task G; involves work.c do_work.c do_work() workdat Makefile)

This step calls for executing most of the body of do_work() in a pthread, rather than in the main() thread for the server process. For now, the main() thread will simply block until that pthread to finish (by calling pthread_join()), but in Task R we will add a loop in main() and avoid blocking the main() thread, so that multiple worker threads might execute at once (multi-threaded server).

  1. Switch to your development branch

    git checkout mybranch
    If there may have been updates to master since you last used mybranch (e.g., updates by a team member), then update mybranch as follows.
    git merge origin/master
git push mybranch

  2. Move the definition of struct workdat into work.h (so that struct can be shared with server.c).
    Add a field named tid (for Thread ID) of type pthread_t to struct workdat, to be filled in with a new pthread's ID. (This field will be needed for logging when there are multiple threads.)
    Also add

     
#include <pthread.h>
    to both server.c and work.c .

    • Note: It's a good practice to #include system header files (such as pthread.h) before user-defined header files (such as work.h), because a user-defined header file might interfere with some internal naming within a system header file.

  3. Modify the Makefile to use the flag -pthread for compiling both server.c and work.c, and the same flag -pthread when linking to produce the executable server.

    • Note: It's simplest to add -pthread to the default compiling rule (target %.o) and also to the link command for server (i.e., the action for the target server).

      If you use other .c source files for building server, the extra compilation flag -pthread won't interfere with compiling those additional .c files.

    Now, use make to generate a new version of server, and test it with (unchanged) client in the usual way (as in HW4). The client and server should behave as they did before making these changes related to struct workdat.

  4. Make a commit of your successful changes so far.

    git add work.c work.h Makefile
git commit -m "Task Q - add thread ID field tid to struct workdat"

  5. Next, replace the struct workdat variable wd by a pointer variable wdp that is dynamically allocated:

    struct workdat *wdp;
wdp = (struct workdat *) malloc(sizeof(struct workdat));
    Also replace field references such as wd.rid by pointer expressions such as wdp->rid, wherever these appear in do_work().

    Remake server and test with the (unmodified) client, debugging as necessary.

  6. Make a commit of your successful changes so far.

    git add work.c
git commit -m "Task Q - change workdat variable wd to a dynamically allocated pointer wdp"

  7. Now define a function worker() for a new pthread to execute, which performs all the code of do_work() after the definition and allocation of wdp and the assignment of do_work()'s request ID argument to wdp->rid. Since it will become the code for a pthread, your new function worker() should have the following function prototype:

     
void *worker(void *datp) {...
    Notes:

    • You may either define worker() before do_work(), or you may use a function declaration of worker() before the definition of do_work(), and locate the definition of worker() after do_work().

    • Suggestion: Insert temporary printf() calls at the beginning and end of worker() 

      printf("Starting worker()\n");
...
printf("Ending worker()\n");
      These calls could help with debugging this step and the future step when a pthread executes worker().

    Also, insert a call

    worker(wdp);
    into do_work() where the moved code used to be.

    • (Note: It is not necessary to cast non-void pointers to the void pointer type in C, e.g., (void*) is unnecessary in

      worker((void*)wdp);
      )

    Finally, the argument of worker() is a pointer datp of type void*, but we know datp actually refers to a (dynamically) allocated struct workdat* pointer. The code we moved from do_work() to the new function worker() refers to wdp, and replacing all those occurrences of wdp by 

    ((struct workdat*) datp)
    would make the code less readable. Therefore, define a pointer variable 
    struct workdat *wdp = datp;
wdp->tid = pthread_self();
    at the beginning of worker(), and leave the references to wdp unchanged in the remainder of worker().
    The second line here assigns the new pthread's ID to the added field tid of the pthread's struct workdat data structure.

    • (Note: It is not necessary to cast void pointers to a non-void pointer type in C, e.g., (struct workdat*) is unnecessary in

      struct workdat *wdp = (struct workdat*)datp;
      This is unlike C++, where void pointers must be cast when assigned to non-void pointers.)

    Compile and test these changes to the server, and debug as necessary. The tests should behave the same as they did before you implemented this worker() step.

  8. Make a commit of your successful changes so far.

    git add work.c
git commit -m "Task Q - implement worker() function for a pthread, and test as a helper for do_work()"

  9. Finally, carry out worker() in a new pthread, created within do_work. This means replacing the existing call of worker() by 

    pthread_t thread_id;
pthread_create(&thread_id, NULL, worker, wdp);
    Just after creating this thread, perform a pthread_join() operation on thread_id to block do_work() until that thread completes its work.

    Then, after the thread has terminated (i.e., after calling pthread_join()), we should deallocate the memory that was dynamically allocated for the pointer wdp, in order to avoid a "memory leak". 

    free(wdp);

    Compile and test these changes, and debug as necessary. The server/client tests should behave the same as they did before you implemented this step.

  10. Once your changes are tested and debugged, commit and send them to stogit. 

    git add work.c
git commit -m "Task Q DONE - Worker thread"
git pull origin mybranch
git push origin mybranch

git checkout master
git merge mybranch
git pull origin master
git push origin master

Link to index

R. Multiple worker threads (Prereq: Task Q; involves server.c work.c main() do_work() Makefile)

After Task Q, the work of handling a client interaction takes place within a pthread that executes worker() instead of the server's main thread as it calls do_work(). We now want to modify main() (in server.c) to enclose the call of do_work() in a loop, in order to handle multiple client requests, using parallel pthreads.

  1. Switch to your development branch

    git checkout mybranch
    If there may have been updates to master since you last used mybranch (e.g., updates by a team member), then update mybranch as follows.
    git merge origin/master
git push mybranch

  2. The main() in server.c currently performs one accept() of a client connection, then calls do_work() to handle that connection. Enclose those calls of accept() and do_work() (plus the printf() that precedes that call of accept()) in an infinite loop

    while (1) {
  ...
}
    Test that this modification accepts multiple client requests and handles them correctly.

    • Note: You may use CTRL/C in the server window to exit this infinite loop when you want to terminate your server. (It is an extra feature to quit the server more gracefully.)

  3. Make a commit of your successful changes so far.

    git add server.c
git commit -m "Task R - multiple client requests"

  4. The server now executes each client request in a separate pthread, but those pthreads run sequentially (without parallelism) because do_work() (which runs in the server's main thread) blocks by calling pthread_join() on the pthread it creates. To achieve parallel execution of the pthreads, we will move the pthread_join() calls from do_work() to main() and make those join calls non-blocking. This way, the loop in main() can go on to accept() new client connections while prior client requests may still be in progress.

    But this means we need a data structure to hold IDs for pthreads that haven't been joined yet. We also need to free() the struct workdata data structures that were dynamically allocated in do_work() (by the server's main thread), in order to avoid memory leaks (see the pthread creation step above)

    We will define this data structure as an array of pointers to struct workdat, since those structs need to be deallocated, and since they also hold the thread id's that must be joined in their tid field.

    To start on this agenda, we will first build the array of pointers to struct workdat and partially test it within work.c, as follows.

    1. Create a new source file wdp_array.c that defines

      • a preprocessor constant for an array size

        #define MAX_ARRAY 100

      • a static array of pointers to struct workdat

        static struct workdat *wdp_array[MAX_ARRAY];
        Here, the keyword static indicates that wdp_array[] is accessible only within the file wdp_array.c. Since the array wdp_array[] is allocated globally (within the file wdp_array.c), the C language initializes all its elements as 0.

      • A function add_wdp() that satisfies the following spec.

        add_wdp

        1 arg: type struct workdat*, representing the address of the data structure for a created pthread.

        State change: For the lowest index i such that wdp_array[i] is 0, assign arg1 to that element wdp_array[i].
        If all MAX_ARRAY elements of wdp_array are non-zero, then print an error message and call _exit(1) to quit the server immediately.

        Return: none

      • A function check_threads() that satisfies the following spec.

        check_threads

        0 args.

        State change: For all non-zero elements of the array wdp_array[], call pthread_tryjoin_np() on that element's tid field. If that call returns EBUSY, take no action; otherwise, deallocate that element's struct workdat using free(), then assign 0 to that element.

        Return: none

      Add #include directives at the beginning of wdp_array.c for the system include files needed for functions used in your code, plus

      #include "work.h"
      in order to define the data type struct workdat.

      Notes:

      • A straightforward implementation is sufficient for the basic assignment. For example, add_wdp() could use a simple loop to iterate until an element of wdp_array[] containing 0 is discovered (or all MAX_ARRAY elements have been checked).

      • However, implement thread-safe versions of your functions add_wdp() and check_threads(). In the end, main() will call check_threads(), and each pthread will need to call add_wdp() for its own struct workdat (in order to get the timing correct for that call).

        You can use a pthread_mutex_t lock variable as in pthreads.c in order to achieve mutual exclusion among calls of the functions add_wdp() and check_threads(). Be sure to use a single lock variable in both functions -- we want to use the same lock variable, since both functions are interacting with the same shared data structure wdp_array[].

    2. Also create a header file wdp_array.h that consists only of function declarations for add_wdp() and check_threads().

    3. Add a Makefile target for wdp_array.o in order to specify the prerequisites wdp_array.c wdp_array.h work.h. Try compiling your new file with 

      make wdp_array.c
      and debug whatever syntax issues arise.

    4. Now temporarily modify check_threads() to comment out the call of pthreads_tryjoin_np() and replace it by a call of pthreads_join().

    5. Add an #include directive for wdp_array.h to work.c.
      Add the call 

      add_wdp(wdp);
      just after assigning pthread_self() to wdp->tid in worker().
      Replace the calls of pthread_join() and free() within do_work() by
      check_threads();

      Proceed to make the server, then test the server and (unmodified) client, debugging as necessary. The system should behave as before (in part because pthread_join() is being used instead of pthread_tryjoin_np() ).

  5. Make a commit of your successful changes so far.

    git add wdp_array.c wdp_array.h work.c Makefile
git commit -m "Task R - create wdp_array[], with preliminary test within work.c"

  6. Now, move thread checking to main() as follows.

    1. Move the call of check_threads() from do_work() (in work.c) to main() (in server.c), at the end of the infinite loop in main(). This will allow a call of do_work() to return without waiting for its pthread to terminate.

    2. Also modify check_threads() (in wdp_array.c) to remove the temporary call of pthread_join() and restore the original call of pthread_tryjoin_np().

    3. Add 

      #include "wdp_array.h"
      to server.c, and add that dependency to the Makefile target for server.o.
      Also add wdp_array.o as both a dependency for the executable target server and in the link action for that target.

    4. Proceed to make the server and test with the client. This test should behave as before.

  7. Once your changes are tested and debugged, commit and send them to stogit. 

    git add server.c work.c wdp_array.c Makefile
git commit -m "Task R DONE - Multiple worker threads"
git pull origin mybranch
git push origin mybranch

git checkout master
git merge mybranch
git pull origin master
git push origin master

Link to index

S. Browser test (Prereq: Task M)

In this task, use an ordinary web brower (such as Chrome) to request and receive a web page from your server.

This is the final task for the complete basic assignment.

Link to index

Remaining tasks under construction - Fall 2019

T. Parallel threads demo (Extra feature) (Prereq: Task R and Task H; involves work.c wdp_array.c worker() add_wdp() check_threads())

This task is an extra feature. In this task, demonstrate that your worker threads can actually execute in parallel.

  1. Switch to your development branch

    git checkout mybranch
    If there may have been updates to master since you last used mybranch (e.g., updates by a team member), then update mybranch as follows.
    git merge origin/master
git push mybranch

  2. Add an artificial delay in worker(), e.g., 

    sleep(30);
    Be sure to google the sleep() library function and make sure that work.c has the required #include directives in order to make this call.

    Locate this artificial sleep() call at about the point in the code where worker() carries out a client's request.

    • If your team has completed Task K (log.c), insert the sleep() call between the second and third log messages.

    • Otherwise, add temporary printf() calls just before and just after your code in worker() that sends a response message to the client, and add the sleep() call between those temporary printf() calls.

      • These temporary printf() messages should include the request ID wdp->rid, the thread ID wdp->tid (cast as an int), and whether that printf() is before or after sending the message to the client.

      These temporary printf() calls can be removed when your team completes Task K.

  3. Also add temporary printf() calls in the functions add_wdp() and check_threads() in wdp_array.c.

    • ______

    • ______

    • ______

    • ______

    • ______

    • ______

    • ______

    • ______

    • ______

    • ______

      • ____________
    ______

  4. Use make to generate a new version of server with the artificial delay. Test this server with a single client over the network - it should satisfy the request as usual, except for the long delay.

  5. ______

  6. ______

  7. ______

  8. ______

  9. ______

  10. ______

  11. ______

  12. ______

  13. ______

  14. ______

  15. ______

  16. ______

  17. ______

  18. Once your changes are tested and debugged, commit and send them to stogit. 

    git add work.c
git commit -m "Task T DONE - Parallel threads demo (Extra feature)"
git pull origin mybranch
git push origin mybranch

git checkout master
git merge mybranch
git pull origin master
git push origin master

Link to index




rab@stolaf.edu, December 1, 2019