Distributed Systems Lab/06


Lab 3 / DSGFileMan - A simple versioning client (RMI)

Introduction

In Lab 1 and Lab 2 you had to do some low-level distributed systems programming with sockets. Most distributed systems nowadays are implemented with distributed objects systems, such as Java RMI (Remote Method Invocation), which we will use in this lab. The main idea of the Java RMI is to use remote objects just as "normal" objects. The main difference is that remote objects reside in a different Java virtual machine. When you call such a remote Java object, a stub class handles all the network communication with the server transparently.

The basic steps to create such a remote object (we call it simply, a "server") are as follows:

  1. Create your custom remote interface which extends java.rmi.Remote.
  2. Define the desired methods to implement the functionality of the server.
  3. Each method in your interface has to throw a RemoteException by convention.
  4. Use the Java RMI compiler rmic to create the necessary stubs.
  5. Implement a main method to instantiate your remote object by registering it in an RMI registry.


Short Introduction to Versioning Systems

If you have never used a versioning system or a document management system, it is a good opportunity to learn it in this course.

General

In general, the aim of version control software is to manage a set of files and its changes. All files in its various versions are stored in a so-called repository. Such a repository is typically managed by a dedicated versioning server. Each user can do a checkout of such a repository to his local machine. The locally checked out files are often referred to as working copy. The user can then modify his local files and commit the modified files back to the server to share the new versions with others. Typically, a version control software is used in software development to control and share the source code of a software project. That means, it allows several project developers (which are potentially widely separated) to collaborate. Popular open source version control systems are Concurrent Versions System (CVS), and the more recent Subversion (SVN).

Terminology

The basic architecture of a version control software is shown in the figures below.

Basic Architecture of a Version Control System

Version control software usually provides a repository containing the set of files that should be controlled. The repository is located on a repository server. Clients are able to create a local working copy of the repository by performing a checkout. After modifying the working copy, each client can commit its changes to the server. Furthermore, a client is able to update its working copy with more recent versions from the repository. If two changes are made to the same document by different parties, a conflict occurs. Current version control systems are able to resolve such conflicts.

Detailed Assignment Description

In our lab example, you will implement a simplified versioning system called DSGFileMan. In lab3, you will implement the command-line DSGFileMan client AND a very simple repository monitor, while the DSGFileMan server is implemented in lab 4.
An IMPORTANT NOTE about this lab: You cannot access our RMI servers from outside of the lab environment because of the firewall. Unfortunately, using RMI over firewalls is problematic. Hence, you should work on the pizza or pasta servers to test your client.

Each DSGFileMan client has to implement the following Java interface:

01 package at.ac.tuwien.dslab.rmi.common.interfaces;
02 
03 import at.ac.tuwien.dslab.rmi.common.impl.FileManException;
04 import at.ac.tuwien.dslab.rmi.common.impl.InvalidCommandException;
05 
06 /**
07  * The <code>IFileManClient</code> is the basic interface
08  * which has to be implemented by the client.
09  
10  * If you do not implement this interface TEST 0 will fail
11  * because we try to instantiate your client class via 
12  * reflection.
13  */
14 public interface IFileManClient {
15 
16   /**
17    * Processes a command of the <i>DSGFileMan</i> server.
18    @param cmd The name of the command (without leading or trailing whitespace characters)
19    @param args A list of arguments for each command.
20    @return The response for this command (which will be echoed to the command line).
21    @throws InvalidCommandException This exception has to be thrown whenever a command is 
22    * invalid (arguments are wrong, etc) or the command is generally unknown.
23    @throws FileManException This exception is the base class for user-defined exceptions in
24    * the <i>DSGFileMan</i> system and has to be thrown if any other exception occurs. 
25    * Other exceptions (e.g., IOException) have to be wrapped using this exception.
26    */
27   public String processCommand(String cmd, String... argsthrows InvalidCommandException, FileManException;
28   
29 }

In the following, all the commands a user can enter to the system are specified in EBNF. The issued command string is the input of the processCommand method of the client. This method has to parse the arguments and then invoke the appropriate method or class that actually handles the processing of a command. We suggest that you use the Command design pattern from the GoF (Gang of Four) to implement the command processing logic. This pattern is a very interesting and a useful thing to know, especially if you have a number of commands to implement, and you want to get rid of very long if-then-else cascades (which should be avoided anyway). If you don't know this pattern, take the opportunity to learn it now, it will help you not only when you want to debug a special command, it also makes your code much more readable. Just use it ;-).

Client commands

The commands are specified as follows (in EBNF):

CreateRepository ::= "create-repo" <username> <password> <server url> <repository name> {<allowed users>}
DeleteRepository ::= "del-repo" <local directory>

Checkout ::= "checkout" <username> <password> <server url> <repository name> <local dir>
ImportDocument ::= "import" <filename>
CommitDocument ::= "commit" <filename>

UpdateDocument ::= "update" [<filename> [<version>]]
DeleteDocument ::= "del" <filename>

A little explanation: The text between double quotes is the name of the command, the text between angle brackets (e.g., <username>) is the actual parameter of a command. Square brackets (e.g., [<filename>]) represent optional parameters, while curly brackets (e.g, {<allowed users>}) represent a list of parameters (cardinality 0..n) separated by a blank character.

The processCommand(String cmd, String... arguments) method takes the above command name as the first string argument and the arguments of each command as variable length argument list arguments (check out the reading suggestions for more information about variable arguments in Java).
The processCommand should issue the appropriate command at the DSGFileMan server and inform the client about the outcome. The client is not interactive (i.e., it quits after every command). For example, if you issue a checkout, do what the command is supposed to do, and then quit the client.

The CreateRepository command creates a repository called <repository name> on the server (identified by <server URL>). The user who wants to create the repository (i.e., the owner) is identified by <username> and <password>. Use your dsgfmXXX account which you can find in the dsgfm-account.txt in you home directory for creating repositories. In addition, the command takes a list of users {< allowed users>} that are allowed to access the repository. For testing this feature, we have created 10 test accounts from test001 to test010 (usernames and passwords are equal). If you omit the allowed user names, only the owner has access to it.
Furthermore, the repository name <repository name> has to stick to the following naming convention: A repository owned by <username> has to begin with the string dsgfm_ (for instance, user dslab999 may create a repository named dsgfm999_repo).

The Checkout command is used to check out a repository. This command takes <username> and <password> of the user who wants to perform the checkout. It then tries to connect to the repository server located on <server URL>. If <username> and <password> are correct, and the given user is allowed to access the repository, all repository documents of <repository name> are stored in the local directory <local directory>. In other words, a working copy of <repository name> is created in <local directory>.
Furthermore, the client is responsible for storing all required repository information (e.g., username, password, server URL, repository documents and versions) in the local working copy. The commands DeleteRepository, ImportDocument, CommitDocument, UpdateDocument and DeleteDocument will read this information from the working copy config.

The DeleteRepository command deletes a repository on client and server. The command takes <local directory> (that is the directory where the working copy is located on the client) as argument, and deletes both the working copy on the client, and the corresponding repository on the server.

The ImportDocument command imports the document <filename> to the repository (i.e., the document <filename> is added to the repository for the first time). The command has to be invoked in the directory where the working copy is located, and reads all required information from the local working copy config.

The CommitDocument command commits the document <filename> to the repository (i.e., a new version of document <filename> is added to the repository). If there is a version conflict (i.e., the document version in the repository is higher than the version in the working copy), the document is not committed and a FileManException is thrown by the server. The command has to be invoked in the directory where the working copy is located, and reads all required information from the local working copy config.

The UpdateDocument command updates the repository document <filename> in the working copy. (i.e., the document <filename> from the repository is updated in the client's working copy). If the document does not exist in the repository any more, it also has to be removed in the working copy.
The command takes the version number <version> of the repository document that should be updated. If you omit the version number, the working copy is updated with the most recent version of the repository document. If you also omit <filename>, all repository documents are updated in the working copy. The command has to be invoked in the directory, where the working copy is located, and reads all required information from the local working copy config.

The DeleteDocument command deletes the document <filename> on client and server (i.e, the document <filename> is deleted from the working copy, and all versions of this document are deleted from the repository). The command has to be invoked in the directory where the working copy is located, and reads all required information from the local working copy config.

Some interactive command-line examples

Create repository dsgfm999_repo on the server (user dsgfm998 has access rights):
> create-repo dsgfm999 pw999 rmi://www.dslab.tuwien.ac.at/DSGFileMan dsgfm999_repo dsgfm998

Check out repository dsgfm999_repo from the server to the local directory workingcopy:
> checkout dsgfm999 pw999 rmi://www.dslab.tuwien.ac.at/DSGFileMan dsgfm999_repo workingcopy

Change to the directory where the working copy is located:
> cd workingcopy

Import document test.txt to the repository:
> import test.txt

Commit document test.txt to the repository:
> commit test.txt

Update document test.txt from the repository to version 3:
> update test.txt 3

Update all documents:
> update

Delete document test.txt from working copy and repository:
> del test.txt

Delete the working copy in the local directory . and the corresponding repository on the server:
> del-repo .

DSGFileMan Server Functionality

The functionality of the DSGFileMan server is specified by the IFileManager interface. You should use the functionality offered by the server interface to implement you client. The interface and all the other classes you need can be downloaded in the provided JAR file which you need to include in your class path.

Implementation Details

DSGFileMan Client

Before you start implementing the DSGFileMan client, make sure that you download the JAR file containing the interfaces and the basic classes you will need to solve the lab. You should also read carefully the documentation of all the classes in this JAR file as generated by the JavaDoc utility.

You should start implementing by creating a new class called at.ac.tuwien.dslab.rmi.client.FileManClientImpl, that implements IFileManClient. In the main() method, you should process the command line arguments and pass them to the processCommand method. If the command syntax is wrong, or the parameter of a command is a local file that does not exist (e.g, when using the import command), always throw an InvalidCommandException in the processCommand() method. If any other exception occurs within the processCommand method (e.g., by receiving one from the server), you have to make sure that the exception is thrown correctly to the caller of the processCommand(). Do not ignore any exception inside the processCommand method, otherwise the grading robot cannot determine if you throw the correct exception. You can find the exceptions thrown by the server in the JavaDoc of the IFileManager.
In your main method, you can simple print the exception message to System.out to inform the user what was wrong:

1 try {
2   // process the command and the return value
3 catch (Exception e) {
4   System.out.println(e.getMessage());
5 }

Remember that the client does NOT run in a loop, but executes a command and quits after every command.

Managing your client-side working copy

On the client side you need to store the configuration of the working copy somewhere. Your working copy consists of all the files which you checked out from the remote repository on the server, plus your configuration file where you store the relevant repository information. The relevant information you need to store in your working copy configuration file can be determined if you have a detailed look at the commands and the server API (Hint: have a look at the create-repo and checkout commands where you have to provide the necessary information => use this information wisely ;-)).
How you store and manage your working copy configuration file is up to you. Yet, be aware that this can only be done in the appropriate directory of your working copy, not in the home or in the tmp directory of your system. If you do so, the grading robot might fail testing your solution.

You can also have files in your working copy which are not in the repository (e.g., a file you create locally but do not import to the repository). Moreover, it is very important that each file is stored exactly in the filename retrieved from the server (see the getName() method of the IDocument interface). Therefore, do not modify the content or the name of the document you retrieve from the server.

As you implement a remote versioning system, you do not have to store document backups in the working copy on the client. That means, there is always exactly one version of each document in the working copy. When performing a document update from the repository, you should simply overwrite the document in the working copy.

To keep it simple, we assume a flat directory hierarchy on client and server. That means, you do not have to consider documents in sub-directories. Of course, you can implement this feature, but we don't test it :-)

RepositoryMonitor

The second part of lab3 is the implementation of a simple RepositoryMonitor, which should receive notifications from the server to monitor what is going on in your repository. The server fires three kind of events to all registered listeners. In case of such an event, you should do the following on client:

  • document imported: print the document name on to System.out
  • document committed: print the document name AND the version number to System.out
  • document removed: print the document name to System.out
The functionality has to be implemented in another class called at.ac.tuwien.dslab.rmi.client.RepositoryMonitor which implements the IRepositoryListener interface. Make sure that your RepositoryMonitor is registered as a listener at the server.
The main method of the RepositoryMonitor takes a number of command-line arguments in the following order:
  1. RMI URL of the repository server
  2. repository name
  3. username
  4. password
After starting the RespositoryMonitor it has to run in a loop as long as the user enters "exit" (case-insensitive) on the console.
Before starting to implement this part, make sure that your understand how to consume remote events (e.g., read the information provided in the link section at the end of this document, especially the RMI callback stuff). We use a new feature of Java 1.5, called Dynamic Generation of Stub Classes, therefore, you don't need to generate the stub for your remote object. Make sure that you read this information to understand which method to use to create remote object which uses this feature. To pass all the test, please make sure that you handle all the arguments correctly and that you deregister the listener at the server and unexport your RMI instance correctly.

Printing output to System.out and System.err

In this assignment, we use JUnit to automatically test your application. You are free to print any debug messages you like to System.err. You are also free to format your output to System.out as you like as long as you provide the functionality we require.

System.exit

In this assignment, you should not use System.exit in both of your clients implementations. This might cause unexpected problems during the automated testing. You should use return instead.

How to compile and run you clients?

Compiling your Java client is not difficult on the command-line, nevertheless we provide you with a simple Ant to compile your clients (download here). Since this is a distributed systems lab, we do not go into details of the Ant script. We assume that you have the following directory structure in your $HOME/lab3 folder:
	dslab150@pasta:~/lab3$ ll
	total 12
	-rw------- 1 dslab150 dslab150  620 2006-11-02 12:04 build.xml
	drwx--x--x 2 dslab150 dslab150 4096 2006-10-30 13:28 lib
	drwx--x--x 3 dslab150 dslab150 4096 2006-11-02 11:41 src
	
Simply store the Ant script in a file called build.xml in $HOME/lab3 and execute it by issuing the ant compile on the bash. Delete the compiled classes by issuing a ant clean. Your compiled Java classes will be stored in the build folder.

Due to the fact that most commands of the DSGFileMan client has to be executed in the same directory where the working copy of you repository is stored, it would be very useful if you create a simple bash script for testing purposes. The synopsis of the script is the following (we assume that the script is stored in the root directory of your home):
	#!/bin/bash  
   	java -cp $DIR/lab3/build:`echo $DIR/lab3/lib/*.jar | tr ' ' ':'` 
             -Djava.rmi.server.hostname=pasta.dslab.tuwien.ac.at 
              at.ac.tuwien.dslab.rmi.client.FileManClientImpl $*
	
You can write a simple and very similar ;-) script for the RepositoryMonitor class. Please note that the JVM property -Djava.rmi.server.hostname=pasta.dslab.tuwien.ac.at has to be set to the hostname where your client is executed, otherwise you will run into troubles receiving callbacks in the RepositoryMonitor correctly.

Hints for Solving the Lab

  1. If you are new to Java RMI programming, please check out the reading suggestions below to get familiar with RMI.
  2. We provide you the set of interfaces, classes and exceptions that are needed to use the DSGFileMan server interfaces. You have to download the JAR and add it to your classpath.
  3. In order to test your client, you can use our 2 DSGFileMan servers instances, both running on www.dslab.tuwien.ac.at. In the registry, the first DSGFileMan server is stored with the name DSGFileMan1, the second one is stored with the name DSGFileMan2. Please note that these two instances are only accessible from inside the lab (either in the lab in the Argentinierstr. 8 or from pizza and pasta). Please do NOT use a security manager for your client because our servers do not support it.
  4. If our grading robot tells you that your submission does not pass all the tests, always start to correct the first test that fails because the other error could be consecutive failures.

Deliverables

To submit your solution to us, you need to follow these steps:

  1. Develop your solution with Java (version 1.5), and implement the IFileManClient interface in a file called FileManClientImpl. Furthermore you need to implement the RepositoryMonitor which implements the IRepositoryListener interface to receive notifications from the server. Put all your files in the $HOME/lab3/src/ folder and use the provided package names. Make sure that your Java program compiles and runs in the lab environment. You can implement and use as much classes as you need. The only constraints is that you implement the required interface and use the package name at.ac.tuwien.dslab.rmi.client for all your client implementation classes. It is not allowed to use external libraries (e.g., XML parser), they won't be submitted to the server.
  2. If you have created your files on some other machine, copy them over using scp (secure copy).
  3. In your $HOME/lab3/src directory call submit3 to submit all your files.
  4. Read any error or success messages
  5. It might take up to an hour for our grading robot to check your program. Don't forget to read your e-mail to check the results of the automatic grading robot. If you have errors, correct them and try again!

Reading Suggestions

The following reading suggestions should help you in developing your solution.

  1. Java RMI Tutorial (from Sun Microsystems Inc)
  2. Java RMI Tutorial (from JGuru)
  3. Notes on RMI Callbacks (from David Grimshaw)
  4. Notes on using variable arguments in Java (from java.net)
  5. CORBA and RMI comparison (from Gopalan Suresh Raj)

Deadline

You need to submit your solution until November the 30th, 17:59 o'clock.

Closing words

Have fun, without sockets ;-)


Last Modified: Fre Nov 10 11:43:50 CET 2006


Distributed Systems Group, Technical University of Vienna, Argentinierstrasse 8 / 184-1, 1040 Vienna, Austria, www.infosys.tuwien.ac.at