CS-202

ImgFS: Image-oriented File System --- create, delete and help

Introduction

This week's objective is to implement three features for our image management system:

  • the create command, to create a new (empty) file in imgFS format (= a new image database);
  • image deletion (delete);
  • complement the help command, a standard and essential element of any command line interface.

One of the aims of this exercise is to learn how to write data structures to disk using basic I/O operations.

As in previous weeks, you'll be writing your own code, modifying the elements provided.

Provided material

Except new tests, there is no new provided material. You will continue to modify the files used last week: imgfscmd.c and imgfscmd_functions.c.

Tasks

This week's work consists of five modifications, summarized here and detailed below if necessary:

  1. in a new imgfs_create.c file (to be created), implement the do_create() function (prototyped in imgfs.h), the purpose of which is to create a new image database in a (binary) file on disk;

  2. complete the do_create_cmd() function in the imgfscmd_functions.c file in order to call do_create() correctly;

  3. implement the do_delete() function (prototyped in imgfs.h) in a new imgfs_delete.c file; the do_delete() function must "delete" a specified image (we'll see below what this really means);

  4. complete the do_delete_cmd() function in the imgfscmd_functions.c file in order to call do_delete() correctly;

  5. define the help() function, which will print instructions for using the imgfscmd command line interface (CLI).

1. Define do_create().

do_create() must create a new database for the imgfs format. It receives the name of the database file, and a partially filled imgfs_file structure, containing only, in the header, max_files and resized_res.

This function should finish initializing the received imgfs_file structure before writing it to disk, first the header, then the metadata. It must use standard C input/output functions to create the new image base in a binary file on disk. If the file already exists, it is simply overwritten (without message nor error).

It is important to initialize all relevant elements explicitly before writing. And, of course, it's essential to write the right-sized array of metadata in the file.
Note: the database name must be set by do_create() from the provided constant CAT_TXT.

It is also important to handle all possible errors. In the absence of an error, do_create() should return ERR_NONE; in the event of an error, it returns the corresponding value code as defined in error.h.

As the create command is only used once (to create a database) and always from the command line utility imgfscmd (it will never be launched from a Web server, for example), we are exceptionally going to add a side effect in the form of a display indicating the (true) number of objects saved on disk.
For example, with one header then ten metadatas, we'll have the following display:

11 item(s) written

11 because the header and then each of the ten metadatas have been successfully written by fwrite().

2. Complete do_create_cmd().

We have provided you with an incomplete implementation of do_create_cmd(). As part of your solution, you need to create an imgfs_file, initialize the max_files and resized_res fields of its header with the values provided, then call do_create() (which will initialize the other fields).

Parsing create command arguments

The main role of do_create_cmd() is to correctly parse all of its arguments, both mandatory and optional.

Your solution should have the following structure:

  • start by retrieving the mandatory argument (<imgFS_filename>)

  • iterate on argv;

  • at each iteration, first determine whether it's an acceptable optional argument (-max_files, -thumb_res or -small_res; see also the help text below);

  • if so, check if there are still enough parameters for the corresponding values (at least one for -max_files and at least 2 for the other two); if not, return ERR_NOT_ENOUGH_ARGUMENTS;

  • then convert the next parameter(s) to the correct type; check that the value is correct (neither zero nor too large); if not, return either ERR_MAX_FILES (for -max_files), or ERR_RESOLUTIONS; note that util.c, already supplied in the past, offers two tool functions (atouint16() and atouint32()) for converting a character string containing a number into its uint16 or uint32 value; we encourage you to use these two functions to convert character strings in command line arguments; they handle the various error cases in the event of converting an invalid number, or a number too large for the specified type (e.g., trying to convert 1000000 to a 16-bit number); they return 0 in these cases; use them to implement your code correctly;

  • if not an optional argument, return error ERR_INVALID_ARGUMENT.

Please note:

  • optional arguments may be repeated, e.g. -max_files 1000 -max_files 1291; in this case, only the last value is valid;

  • the mandatory argument cannot be repeated.

3. Define do_delete().

We here describe how to implement the functionality for deleting an image. The idea is as follows: we don't actually delete the contents of the image, as this would be too costly (especially in terms of time). In fact, the size of the image base file on disk never decreases, even when you ask to "delete" an image from the base.
Rather, an image is "deleted" by

  1. finding the image reference with the same name in the "metadata";
  2. invalidating the reference by writing the value EMPTY in is_valid;
  3. adjusting the "header" information.

Changes must be made first to the metadata (memory, then disk), then to the header if successful.
Note: for reasons of compatibility between systems, it is preferable to rewrite the entire "struct" to disk, rather than just the modified fields.

The do_delete() function takes the following arguments:

  • an identifier (string, const char *);
  • an imgfs_file structure.

To write the changes to disk, you first need to set the position at the right place in the file, using fseek() (see the course and man fseek) and then fwrite().

Of course, if the reference in the image database does not exist (and there is no invalidation), this must be handled correctly.

Don't forget to update the header if the operation is successful. You also need to increase the version number (imgfs_version) by 1, adjust the number of valid images stored (nb_files) and write the header to disk.

4. Define do_delete_cmd()

Complete the code for do_delete_cmd(). If the received imgID is empty or its length is greater than MAX_IMG_ID, do_delete_cmd() should return the error ERR_INVALID_IMGID (defined in error.h).

5. Define help().

The help command is intended to be used in two different cases (already covered):

  1. when the arguments passed to the utility are invalid;
  2. when the user explicitly requests the list of possibilities by typing imgfscmd help.

The command output must have exactly the following format:

imgfscmd [COMMAND] [ARGUMENTS]
  help: displays this help.
  list <imgFS_filename>: list imgFS content.
  create <imgFS_filename> [options]: create a new imgFS.
      options are:
          -max_files <MAX_FILES>: maximum number of files.
                                  default value is 128
                                  maximum value is 4294967295
          -thumb_res <X_RES> <Y_RES>: resolution for thumbnail images.
                                  default value is 64x64
                                  maximum value is 128x128
          -small_res <X_RES> <Y_RES>: resolution for small images.
                                  default value is 256x256
                                  maximum value is 512x512
  delete <imgFS_filename> <imgID>: delete image imgID from imgFS.

Write the function in imgfscmd_functions.c.

Testing

Testing by hand

It's best to start testing your code on a simple case you're familiar with.

Use a copy of the provided/tests/data/test02.imgfs file from previous weeks (we insist: make a copy!!) to see its contents, delete one or two image(s). Check each time by looking at the result with list.

Also test any edge cases you can think of.

Test your two new commands (use help to find out how to use create;-P ).

To check that the binary file has been correctly written to disk, use last week's list command.

Provided tests

We provide you with a bunch of unit and end-to-end tests, you can run them as usual.

If you're on your own VM, please install libvips-dev, e.g.:

sudo apt install libvips-dev

Personal unit tests

As we move forward with the project, it is important that you can write your own tests, to complete the provided ones. You can find those in provided/tests/unit/. Before adding new tests, don't forget to copy the test/ directory in done/. You will also need to modify the TEST_DIR variable in the Makefile.

We strongly advise you to edit these files to add your own tests, or even to create new ones as you move forward. This can be done quite simply by adding your own values or lines of code to the tests already provided, or by copying this file and drawing inspiration from it (don't forget to update the tests' Makefile accordingly). You don't need to understand everything in this file, at least not initially, but it is important you start to get familiar with its content.

That said, for those who want to go further, the main test functions available in the environment we use (Check) are described over there: https://libcheck.github.io/check/doc/check_html/check_4.html#Convenience-Test-Functions. For example, to test whether two int are equal, use the ck_assert_int_eq macro: ck_assert_int_eq(a, b).

We have also defined the following "functions" in tests.h:

  • ck_assert_err(int actual_error, int expected_error) : assert that actual_error is expected_error ;
  • ck_assert_err_none(int error) : assert that error is ERR_NONE ;
  • ck_assert_invalid_arg(int error) : assert that error is ERR_INVALID_ARGUMENT (i.e. correspond to the return code of a function which received a invalid argument; see error.h) ;
  • ck_assert_ptr_nonnull(void* ptr) : assert that ptr is not NULL ;
  • ck_assert_ptr_null(void* ptr) : assert that ptr is NULL.

Finally, we'd like to remind you that just because 100% of the tests provided here pass doesn't mean you'll get 100% of the points. Firstly, because these tests may not be exhaustive (it's also part of a programmer's job to think about tests), but also and above all (as indicated on the page explaining the project grading scale, because we attach great importance to the quality of your code, which will therefore be evaluated by a human review (and not blindly by a machine).