Session 08: Menu Programs

At the end of this session participants will be able to:

Understand how to design the screens of a menu program
Use variables of type pff to launch one CSPro application from another
Create a simple menu program to launch the main data entry program
Create dynamic menus with dynamic value sets

What is a Menu Program?

A menu program is a CSPro data entry application that is used to manage the data entry workflow. A menu program is not used for capturing any interview data itself. Instead, it launches other data entry programs for interviews. Menu programs generally have some or all of following functions:

Launches other CSPro applications to do data collection (often pre-filling ID items)
Show reports on progress, summary statistics
Manage user access through usernames/passwords
Manage household listing and interview assignments
Launch synchronization

Designing the Screens and Flow

Once you decide on which functions you want your menu program to perform, the next step is to design the screens and how they link together. This is most easily done as a diagram like the one below for a fairly simple menu program.

Creating the Dictionary and Forms

Once you have the screens, the next step is to create a new data entry application for your menu program. Let's name ours Menu and put it in the folder Popstan2020/Menu.

Each menu screen will be a different variable in the dictionary. The value set for the variable will show the available menu choices to the enumerator on that screen. In the menu example above we will have three variables:

LOGIN (value set: Interviewer - 1, Supervisor - 2)
INTERVIEWER_MAIN_MENU (value set: List Households - 1, Interview Household - 2, Logout - 9)
SUPERVISOR_MAIN_MENU (value set: Completion report - 1, Logout - 9).

We will keep the default ID item that CSPro creates for us. The menu program dictionary doesn't really need an ID item since we do not need to save our menu choices to the data file. However, CSPro requires that we have at least one ID item so we will keep it.

Create the items in the dictionary and then create a form and drop the items onto the form. Do not drop the ID item onto the form. We can leave it off the form since we will never actually write out a completed case from the menu program. Unlike typical data entry applications, menu programs tend not to have a linear flow. As a result, the order of the variables on the form is less important. We will use skips and reenters to move from one menu to another.

Menu Program Logic

The logic for processing the menu choice for each screen goes in the postproc of the variable for the menu. For example, to process the login menu field in our example we would have the following logic:

PROC LOGIN

// Go to the appropriate menu for the role chosen
if $ = 1 then
skip to INTERVIEWER_MAIN_MENU;
else
skip to SUPERVISOR_MAIN_MENU;
endif;

If the user selects to login as an interviewer we skip to the interviewer main menu field to show the interviewer menu, otherwise we skip to the supervisor main menu field to show the supervisor menu.

Handling the interviewer menu is similar. Here we will create user-defined functions to launch the listing and household programs.

PROC INTERVIEWER_MAIN_MENU

postproc
// Handle the menu choice
if $ = 1 then
// List households
launchHouseholdListing();
elseif $ = 2 then
// Household questionnaire
launchHouseholdDataEntry();
elseif $ = 9 then
// Logout
stop(1);
endif;

// Show interviewer menu again
reenter;

It is important to make sure that after the postproc of the menu field we do not let CSEntry continue to the next field, otherwise after the interviewer launches the household listing they would end up in the next field, which, in this case is the supervisor menu. To prevent this, we put a reenter at the end of the postproc so that we go back into the same menu field again.

It looks kind of strange that when we go back into a menu, the previous choice is still selected. We can prevent this by clearing the choice in the onfocus of the field.

onfocus
// Clear previous choice
$ = notappl;

The supervisor menu is similar to the interviewer menu:

PROC SUPERVISOR_MAIN_MENU

onfocus
// Clear previous choice
$ = notappl;

postproc
// Handle the menu choice
if $ = 1 then
// Show report
showCompletionReport();
elseif $ = 9 then
// Logout
stop(1);
endif;

// Show supervisor menu again
reenter;

Launching one CSPro Application from Another

Let's fill in the function to launch the household data entry program. We can launch other CSPro programs from within our data entry program using a variable of type Pff. A Pff variable represents a pff file. You can load the contents of an existing pff file into a Pff variable using the load() function and passing it the path to the file. You can then run the associated CSPro application by calling the exec() function. This will start the application using the parameters in the pff file.

The following logic will launch a data entry application using the pff file Popstan2020.pff in the sibling directory to the menu program directory.

function launchHouseholdDataEntry()

Pff householdPff;
householdPff.load("../Household/Popstan2020.pff");
householdPff.exec();
end;

This will start the data entry application and immediately exit the menu. The ".." in the path to Popstan2020.pff tells CSEntry to go one directory up, i.e. to go to the parent of the Menu directory and from there to Household/Popstan2020.pff.

Making the Entry Program Return to the Menu Program

When we exit the household application, we want to return to the menu but the menu program stopped when we launched the household application. To get it to restart we add a parameter to the pff file to tell it to start the menu program when it exits. Right click on the Popstan2020.pff and choose "Edit". This will bring up the pff file editor. From the Add menu add a new On Exit PFF entry. Use the browse button in that entry to locate the Menu.pff which will fill in the path to it.

Deploying multiple applications on Mobile

When you copy your application to your phone or tablet it is important that you preserve the same folder structure as you have on the PC. In our case we must create a separate menu folder to contain the pen and pff files for the menu and this menu folder must be in the same parent directory as the Household folder so that when we use "../Household/" from the menu application it points to the folder containing the entry application we are launching. When you have multiple applications like we do, it can be time consuming to create the .pen files for each application and then copy only the .pen and .pff files to the mobile device.

To simplify deployment use the Deploy Application tool from the Tools menu. This tool automates the process of publishing and deployment. Drag the whole Popstan2020 folder to the Files window. The tool will add the entry applications (.ent files) and .pff files found in the folder and its subfolders to the list of files to deploy. By default it does not add data files since data files are often just used for testing. However, for our application we need to add the lookup files for the districts and possession limits. We can do this by dragging the individual files onto the files window.

Once the files have been added, enter the name and an optional description for the application. Under Deploy To choose Local Folder and click on "..." to choose the folder to deploy to. Click the Deploy button. This will automatically create .pen files for each data entry application and copy them, along with any other files from the Files window, into the deployment folder chosen. The deployment folder can then be copied directly to the mobile device. The folder structure of the deployment folder will match the folder structure we are using for development and testing on the desktop so there will be no problems with paths.

Once you have set up the deployment the first time you can save the deployment specification file to simplify the process for future deployments.

In the next session we will learn how to use this tool to deploy applications to a server for deployment over the Internet.

Tidying up the Menu Program

Currently the menu program shows up in the applications list on Android as "Menu" instead of something like "Popstan 2020 Census". In addition, when we tap on Menu we have to then tap "Start New Case" which doesn't make sense for a menu. We can fix both of these problems by modifying the pff file for the menu. Right click on the Menu.pff and choose "Edit with pff editor". Change "Start mode" to "Add" so that we won't have to tap "Start New Case". Enter "Popstan 2020 Census" for the description to change what shows in the application listing. While we are at it, open the Menu program in the CSPro designer and in data entry options turn off the display of the case tree since the case tree is not useful for menu programs. Finally, in the data entry options, check "Automatically advance on selection". This way the interviewer will not have to tap next to move from one field to another in the menu program.

Now that we have the menu program to launch the household data entry program, we don't want the household application to appear in the list of applications on the mobile device. By default, CSPro creates the list of applications from all of the pff files in the CSEntry directory on the device. To prevent an application from being listed you can edit the pff file in the pff editor and choose "Hidden by Default" or "Never" for "Show in Application Listing".

Creating a Dynamic Menu

When the interviewer launches the data entry application, they can currently enter any ID items but we would like to restrict them to only interviewing households that have already been listed. We can do this by displaying the households from the listing file in a dynamic value set and having the interviewer choose which one to interview.

In order to read the listing file, we need to add the listing dictionary as an external dictionary in the menu application. We will also need to add a new menu item to list the households. This will be a new variable in the dictionary named CHOOSE_HOUSEHOLD. We will make it 10 digits long to hold the full set of ID items (province, district, EA, area type and household number). It will be an alpha field as this will be easier to work with later on. The interviewer menu will need to be modified to skip to this new field instead of calling launchHouseholdDataEntry().

PROC INTERVIEWER_MAIN_MENU
postproc

// Handle the menu choice
if $ = 1 then
// List households
launchHouseholdListing();
elseif $ = 2 then
// Household questionnaire
skip to CHOOSE_HOUSEHOLD;
elseif $ = 9 then
// Logout
stop(1);
endif;

// Show interviewer menu again
reenter;

In the onfocus proc for CHOOSE_HOUSEHOLD we need to loop through every case in the listing file. We can use the forcase statement to loop through the cases in the listing file and build the value set:

PROC CHOOSE_HOUSEHOLD
onfocus

ValueSet string householdVSet;

// Loop through all cases in listing file
// to build dynamic value set.
forcase LISTING_DICT do
// Values are household ids concatenated together
string code = maketext("%v%v%v%v",
LI_PROVINCE, LI_DISTRICT, LI_ENUMERATION_AREA,
LI_HOUSEHOLD_NUMBER);

// Labels have househould number and name of head
string label = maketext("%v: (%s)", LI_HOUSEHOLD_NUMBER,
strip(LI_NAME_OF_HEAD_OF_HOUSEHOLD));

householdVSet.add(label, code);
endfor;

setvalueset($, householdVSet);

When we run this we should see a list of the households in the listing file as the value set for CHOOSE_HOUSEHOLD.

While the dynamic value set works correctly when there are households that have been listed, if the listing file is empty we get an empty value set. Instead we should show an error message and return to the main menu. We can determine if the value set is empty by checking the length of the list of codes in the value set. You can access the lists of codes and labels of a value set using the variable name followed by a dot (".") and "codes" or "labels". The codes and labels properties of the value set are CSPro lists so you can retreive their sizes using the length() function.

PROC CHOOSE_HOUSEHOLD
onfocus

ValueSet string householdVSet;
string code, label;

// Loop through all cases in listing file
// to build dynamic value set.
forcase LISTING_DICT do
// Values are household ids concatenated together
code = maketext("v%v%v%v%v",
LI_PROVINCE, LI_DISTRICT, LI_ENUMERATION_AREA,
LI_AREA_TYPE, LI_HOUSEHOLD_NUMBER);

// Labels have househould number and name of head
label = maketext("%v: (%v)", LI_HOUSEHOLD_NUMBER,
strip(LI_NAME_OF_HEAD_OF_HOUSEHOLD));

householdVSet.add(label, code);
endfor;

if length(householdVSet.codes) = 0 then
errmsg("No households have been listed. Please list households first before interviewing.");
reenter INTERVIEWER_MAIN_MENU;
endif;

setvalueset($, householdVSet);

Setting the "Key" attribute in the pff

The next step is to handle the menu choice in the postproc to launch the household data entry program using the ID items for the household that was chosen. In order to do this, we will need to pass the ID items of the selected household to household application as the "Key" attribute of the pff file. The household data entry program will then automatically read the value from the pff and prefill the ID items once it is launched. To set the "Key" attribute in the pff file call the setproperty() function of the pff variable and pass it the attribute name and attribute value. The value for the key should be the entire caseid string picked in the field CHOOSE_HOUSEHOLD.

// Launch household questionnaire data entry application
function launchHouseholdDataEntry()
Pff householdPff;
householdPff.load("../Household/Popstan2020.pff");
householdPff.setProperty("Key", CHOOSE_HOUSEHOLD);
householdPff.exec();
end;

CSEntry prefills the ID items using the string passed in the "Key" attribute. Let's also make the field protected if it is prefilled. We can do this using protect.

PROC PROVINCE
preproc

// protect the field if it is prefilled
if visualvalue(PROVINCE) <> notappl then

// protect field so the interviewer cannot modify it
protect(PROVINCE, true);

endif;

The logic for DISTRICT, ENUMERATION_AREA, and HOUSEHOLD_NUMBER is similar.

Using a Lookup File for Login

Rather than have the user choose their role, lets assign each interviewer and supervisor a staff code and then create a lookup file containing the staff codes along with the role and the district/enumeration area assigned to the interviewer/supervisor. Here is an example staff file:

Staff code	Name	Role (1=interviewer, 2=supervisor)	Province	District	EA
001	Shemika Rothenberger	2	1	1
002	Andrew Benninger	1	1	1	1
003	Angelica Swenson	1	1	1	2
004	Zelma Hawke	1	1	1	3
005	Willis Catron	1	1	1	4

Note that for the supervisor we leave the enumeration area blank since they are assigned to an entire district and not to an enumeration area. The supervisor will supervise all of the enumerators in their district.

Let's create a new external dictionary in the menu program for this file and use Excel to CSPro to convert the spreadsheet to a data file named staff.dat.

Let's modify the login field so that the user enters the staff code instead of picking the role. We will need to increase the size of the field and delete value set. In the postproc we now need to look up the staff code in the staff file, validate it, and then skip to the appropriate menu based on the role.

PROC LOGIN

// Verify staff code using lookup file
if loadcase(STAFF_DICT, LOGIN) = 0 then
errmsg("Invalid staff code. Try again.");
reenter;
endif;

// Go to the appropriate menu for the role chosen
if STAFF_ROLE = 1 then
skip to INTERVIEWER_MAIN_MENU;
else
skip to SUPERVISOR_MAIN_MENU;
endif;

When we deploy our application we need to remember to add the staff data file to our deployment specification, otherwise we will not be able to login.

Preserving Login when Returning to Menu

Currently when you launch the listing program and return to the menu, you have to enter the user code again. Let's save the login code so that you only have to enter it once. We can do this using the commands savesetting() and loadsetting(). These commands store and retrieve values in persistent storage. Values saved in the settings are available even after CSEntry is closed and restarted, and they are available in all CSPro applications on the same device. We will save the login code in the login postproc after validating it:

PROC LOGIN
postproc
// Verify staff code using lookup file
if loadcase(STAFF_DICT, LOGIN) = 0 then
errmsg("Invalid staff code. Try again.");
reenter;
endif;

// Save login so we do not have to enter it again
savesetting("login", maketext("%v", LOGIN));

// Go to the appropriate menu for the role chosen
if STAFF_ROLE = 1 then
skip to INTERVIEWER_MENU;
else
skip to SUPERVISOR_MENU;
endif;

Settings are always stored as alphanumeric so we need to use maketext to convert the numeric LOGIN code to a string.

In the preproc we will try to retrieve the login code from the settings and if it is not empty we will use it instead of asking the user to enter the code.

PROC LOGIN
preproc
// Check to see if there is an existing login code
// use that
if loadsetting("login") <> "" then
LOGIN = tonumber(loadsetting("login"));
noinput;
endif;

Finally, we need to clear the setting on logout:

PROC INTERVIEWER_MAIN_MENU
postproc

// Handle the menu choice
if $ = 1 then
// List households
launchHouseholdListing();
elseif $ = 2 then
// Household questionnaire
launchHouseholdDataEntry();
elseif $ = 9 then
// Logout
// Clear login from settings
savesetting("login", "");
stop(1);
endif;

// Show interviewer menu again
reenter;

PFF Parameters

The parameter section in the pff file allows you to pass any value to your data entry program. Let's use this to pass the interviewer code chosen in the menu program to the household entry program.

The next step is to modify the household data entry application to prefill the interviewer code using the parameter in the pff file. This can be done using the sysparm() command which retrieves a parameter by name from the pff file. We can do this in the preproc of the INTERVIEWER_CODE in the household application. sysparm always returns the result as a string so we need to convert it to a number.

PROC INTERVIEWER_CODE
// Prefill with parameter from pff
preproc
if sysparm("INTERVIEWER_CODE") <> "" then
$ = tonumber(sysparm("INTERVIEWER_CODE"));

// make field protected
protect($, true);
endif;

The Completion Report

Let's add the report to the supervisor menu. We will create a report that shows the total number of households by interview status. Here is an example:

Completion Report
-----------------
Province: 01 District: 01

Interview Status
----------------
Completed: 10
Non-contact: 1
Vacant: 2
Refused: 1
Partially complete: 5
Total: 19

To generate this report, we need to loop through all the cases in the household dictionary and count the number of cases with each interview status. We can use forcase to do this. To compute the totals for each category we create a local variable for each status and increment the appropriate variable every time we encounter a household with that status.

// Display the completion report that shows total interview status
// for all cases in the supervisor's district.
function showCompletionReport()
string reportFilename = maketext("%sreport.txt", pathname(Application));
File tempFile;
setfile(tempFile, reportFilename, create);

filewrite(tempFile, "Completion Report");
filewrite(tempFile, "-----------------");
filewrite(tempFile, "");
filewrite(tempFile, "Province %v District %v",
STAFF_PROVINCE,
STAFF_DISTRICT);
filewrite(tempFile, "");
filewrite(tempFile, "Interview Status");
filewrite(tempFile, "----------------");

numeric complete = 0;
numeric nonContact = 0;
numeric vacant = 0;
numeric refused = 0;
numeric partial = 0;

forcase POPSTAN2020_DICT do
if INTERVIEW_STATUS = 1 then
complete = complete + 1;
elseif INTERVIEW_STATUS = 2 then
nonContact = nonContact + 1;
elseif INTERVIEW_STATUS = 3 then
vacant = vacant + 1;
elseif INTERVIEW_STATUS = 4 then
refused = refused + 1;
else
partial = partial + 1;
endif;
endfor;

filewrite(tempFile, "Completed: %d", complete);
filewrite(tempFile, "Non-contact: %d", nonContact);
filewrite(tempFile, "Vacant: %d", vacant);
filewrite(tempFile, "Refused: %d", refused);
filewrite(tempFile, "Partially complete: %d", partial);
filewrite(tempFile, "Total: %d", complete + nonContact +
vacant + refused + partial);

close(tempFile);
view(reportFilename);
end;

Exercises

Implement the function launchHouseholdListing() in the menu program. It should write out and launch a pff file to run the listing program. Set the "Key" attribute in the pff file to the province, district, and enumeration area. Set the staff area type and the code from the staff file as a parameter in the pff file. Prefill the staff code in the listing program. Make sure that the listing program returns to the menu program on exit.
Modify the logic in the onfocus and postproc of CHOOSE_HOUSEHOLD to add an extra option to the end of the list labeled "Back" that will move back to the interviewer main menu.
Modify the dynamic value set we create in the onfocus proc of CHOOSE_HOUSEHOLD to show the interview status (A10 in the household questionnaire) in addition to the household number and the head's name. To do this you will need to use loadcase on the household questionnaire dictionary to find the case from the household data file. You will also need to handle the case where the household does not yet exist in the household data file.
Create a new menu called "Summary Reports" that has options for the completion report (the one we implemented already) and a new "Total Population Report" that you will implement. This new menu should also have an option to go back to the main menu. The "Summary Reports" menu should be accessed from the Supervisor Main Menu. The new "Total Population Report" should look like:

Total Population Report
-----------------------
Province: 01 District: 02
Male: 1020
Female: 1025
Total: 2045