You are located in service: RWTHmoodle

Automatic solution evaluation with multiple files

Automatic solution evaluation with multiple files

Kurzinformation

VPL activities can also use multiple submission files and test cases. This tutorial illustrates this using the example of a VPL task that evaluates a Java project with two classes, which receives its input from a data file. The project is tested with 4 different data files. One file is provided by the student, three by the lecturer.

 Detailinformation

Content

1. Create VPL submissions
2. Create test case
3. Set execution options
4. Define requested files
5. Define execution files
6. Define files top keep
7. Test automatic evaluation with a sample solution
 

1. Create VPL submission

First create a new VPL submission. Make the following settings:

  • Name - Java: multi-file test
  • Short description - This program checks the output with four different input files, one of which must be provided by you in data.txt.
  • Full Description - Write a Java program that evaluates min and max values of a given file.
  • Submission period  - Activate "Available from" and "Due date". Select today's day under "Available from". Select a date in the future as "Due date".
  • Submission restriction - Select 3 for "Maximum number of files".
  • Grade - Select "Point" as the rating type and 100 as the "Maximum Points".

Click on "Save and display". Your VPL activity is now created. You will then be automatically taken to the VPL activity overview page.

 

2. Create test case

In the next step you create a test case.

  1. Click on the gear inside the white box in the opened VPL assignment to open the VPL Administration.
  2. Select the "Test Cases" option.
  3. Enter the following Test Case:

    case = Test 1

    input = data.txt

    output = "File name? min = -1

    max = 110

    "


    case = Test 2

    input = data1.txt

    output = "File name? min = 1

    max = 1

    "

    case = Test 3

    input = data2.txt

    output = "File name? Empty array

    min undefined

    max undefined

    "

    case = Test 4

    input = data3.txt

    output = "File name? min = 10

    max = 10

    "

  4. Finally, click on the disk to save the test case.
  5. Return to the overview page of the activity by clicking on the submission title "Java: Multi-file test" in the breadcrumb navigation.

 

3. Set execution options

  1. On the VPL activity overview page, click on the gear inside the white box to open the VPL Administration.
  2. Select "Execution Options".
  3. Set the following options:
    • Execute (of the program on the VPL Execution Server): Yes
    • Debugging: No
    • Evaluate: Yes
    • Evaluate only on delivery: No
    • Automatic evaluation: Yes
  4. Finally, click on "Save settings".
  5. Return to the task overview page by clicking on the task title "Java: Multi-file test" in the breadcrumb navigation.

 

4. Define requested files
  1. On the VPL activity overview page, click on the gear inside the white box to open the VPL Administration.
  2. Select "Requested files".
  3. Type "data.txt" as the file name, and then click OK.
  4. In the editor, you can provide your students with a code construct. We use here for example:

        3
        4
        5
        1
        10
        11
        11
        -1
        100
        3
        9
        110
  5. Save the file in the editor by clicking on the disk symbol.
  6. Additionally add the file "Main.java" as well as "SortedList.java" by clicking on the plus sign. Save the files by clicking on the disk symbol.
  7. Return to the submission overview page by clicking on "Java: Multi-file test" in the breadcrumb navigation.
 

5. Define execution files

  1. On the VPL activity overview page, click on the gear inside the white box to open the VPL Administration.
  2. In the "Advanced Settings" tab, select the "Execution Files" option.
  3. Add the following for "vpl_run.sh" in the editor:

    #! /bin/bash
     
    cat > vpl_execution << 'EOF'
    #! /bin/bash
    javac -J-Xmx128m *.java
    java Main
     
    EOF
     
    chmod +x vpl_execution

  4. Add the files "data1.txt", "data2.txt" and "data3.tx" by clicking on the plus symbol
  5. Add the following for "data1.txt":

    1
    1
    1
    1
    1
  6. Do not add text for "data2.txt".
  7. Add the following for "data3.txt":

    10
  8. Return to the activity overview page by clicking on "Java: Multi-file test" in the breadcrumb navigation.
 

6. Define files to keep

  1. On the VPL task overview page, click on the gear inside the white box to open the VPL Administration.
  2. On the Advanced Settings tab, select the option "Files to keep when running".
  3. Select all files here and click "Save options".
  4. Return to the activity overview page by clicking on "Java: Multi-file test" in the breadcrumb navigation.

The automatic evaluation is now created.

 

7. Test the automatic evaluation with a sample solution

  1. To test a sample solution, click on the "Test activity" tab in the opened task and then on "Edit".
  2. Now enter the following into the editor:
  3. For Main.java:
     

    // Main.java
    import java.io.BufferedReader;
    import java.io.File;
    import java.io.FileReader;
    import java.io.IOException;
    import java.util.ArrayList;
    import java.util.Collections;

    import java.util.Scanner;

    public class Main {
        
        
        public static void main(String[] args) {
            BufferedReader br = null;
            String sCurrentLine = null;
            SortedList array = new SortedList();
            
            Scanner keyboard = new Scanner( System.in );
            System.out.print("File name? ");
            String fileName  = keyboard.next().trim();

            try {
                br = new BufferedReader( new FileReader( fileName ) );
                while ((sCurrentLine = br.readLine()) != null) {
                    int x = Integer.parseInt( sCurrentLine.trim() );
                    array.add( x );
                }
                
            } catch (IOException e) {
                //e.printStackTrace();
                System.err.println( "Could not find file" );
                return;
                
            } finally {
                try {
                    if (br != null)
                        br.close();
                } catch (IOException ex) {
                    //ex.printStackTrace();
                    return;
                }
            }
            
            if ( array.isEmpty() ) {
                System.out.println( "Empty array" );
                System.out.println( "min undefined" );
                System.out.println( "max undefined" );
                return;
            }
            int max = array.getLast();
            int min = array.getFirst();
            System.out.println( "min = " + min );
            System.out.println( "max = " + max );
        }
    }

  4. For SortedList.java
     

    import java.util.ArrayList;
    import java.util.Collections;


    public class SortedList {
        
        ArrayList<Integer> array = null;
        
        SortedList() {
            array = new ArrayList<Integer>();
        }
        
        public void add( int x ) {
            array.add( x );
            Collections.sort( array );
        }
        
        public boolean isEmpty() {
            return array.isEmpty();
        }
        
        public int getFirst( ) {
            return array.get( 0 );
        }
        
        public int getLast( ) {
            return array.get( array.size() - 1 );
        }
    }

  5. Click on the disk to save. By clicking on the rocket you run your program.
  6. By clicking on the hook with the adjacent 0 you execute the automatic test. You should now find the following output on the right side::
    • |  4  test run/ 4  test passed |
 

 Zusatzinformation

Please read about this: