Happy 25th Birthday, CSPro!


The first version of CSPro, version 2.0, was released 25 years ago today. Only one member of the current development team was present for the first release, and some on the team were infants at the time! However, CSPro's legacy is not connected to any particular developer but instead belongs to the global community of users who have collected and processed data with CSPro over the years.

The first release, on May 5, 2000, occurred on a day that some had prophesied as the end of the world. On that day:

...all five naked-eye planets (Mercury, Venus, Mars, Jupiter, Saturn) plus the Moon and the Sun will at least approximately line-up. As viewed from planet Earth, they will be clustered within about 26 degrees, the closest alignment for all these celestial bodies since February 1962, when there was a solar eclipse!

NASA reassured the public not to worry:

On May 5, 2000 the planets Mercury, Venus, Earth, Mars, Jupiter, and Saturn will be more or less positioned in a line with the Sun. Additionally, the Moon will be almost lined up between the Earth and Sun. Although this has led to many dire predictions of global catastrophes such as melting ice caps, floods, hurricanes, earthquakes, etc. there is absolutely no scientific basis for these claims. The distance to the planets is too great for their gravity, magnetic fields, radiation, etc. to have any discernible effect on Earth. In fact, we won't even be able to see this alignment, as all the planets will be on the opposite side of the Sun from the Earth.

The world obviously did not end 25 years ago. Instead, that day was an auspicious beginning of a long run of success for CSPro. The team involved in the first release, heralding from the U.S. Census Bureau, Macro International, and Serpro S.A., could not have anticipated exactly how data collection would evolve over the decades, from keying paper forms to scanning to CAPI to CAWI, but they laid a strong foundation that has allowed CSPro to adapt to these changes, growing in a myriad of ways and finally becoming open source earlier this year.

To celebrate CSPro's 25th birthday, here are a few images from the first release:

v2-splash-screen
The splash screen used in CSPro version 2.0.

Computers are so fast these days, but 25 years ago, software often took a while to start, so many programs used splash screens to inform users that the program was still loading.

v2-new-file
The dialog to create a new file in CSPro version 2.0.

The first release did not contain a batch editing module. Data entry applications were also created with a few different files than are used now. There used to be a "skip code file" with the extension .skp, and the data entry application framework, now part of the .ent file, was instead an .apl (application) file.

v2-tools
The Designer's Tools menu in CSPro version 2.0.

CSPro's first release had only five tools. In contrast, CSPro 8.0 now includes 23 tools, reflecting how much the software has grown to support its users.

On May 5, 2000, no one could have predicted that CSPro would still be in use in 2025, and while we cannot say with certainty what the next 25 years will bring, CSPro has already built a lasting legacy. The software has been used to process billions of records, to collect and disseminate data for hundreds of censuses, and has been the software of choice for thousands of developers, all while remaining free thanks to generous support from USAID.

Happy 25th Birthday, CSPro! Or rather:

errmsg("Happy %d%s Birthday, CSPro!"25"th");

cupcakes-25
Cupcakes inscribed with 25 and CS to celebrate the big day!

Restoring Old Blog Posts


This upcoming Monday is the 25th anniversary of CSPro's first release. In anticipation of celebrating CSPro's longevity, I took a look at the second CSPro Users website, which was designed in WordPress:

2025-05-01-old-website

(The first website was designed with Drupal. The current version—the third iteration you are viewing now—is powered by Jekyll.)

We added a blog for the WordPress website but the posts did not make the transition when we moved to this new site. For this #ThrowbackThursday, I restored the 47 blog posts from the old site, originally published between December 2011 and March 2016.

If the blog looks different than you remember, that is because it is now generated using CSDocument. This tool, which also creates CSPro's documentation, colorizes code and automatically links functions and keywords to their help pages.

Although the blog has seen little activity over the past decade, we hope to post more frequently in the future.

CSPro 8.0 is Out!


The latest iteration of CSPro, version 8.0, has been released! Dive into the newest features and enhancements packed into CSPro 8.0.

What's new with CSPro 8.0

  • Many specification files are now saved in JSON format, which facilitates working with them in other programming languages or when using other tools.
  • In addition to being able to declare numeric and alphanumeric data types in the dictionary, you can now specify four new data types: Audio, Document, Geometry, and Image.
  • A new framework, the Action Invoker, provides a standard way of running actions from CSPro logic, embedded JavaScript, JavaScript executed on web pages, and in JSON format.
  • A new view, questionnaire view, displays the contents of a case and its components (dictionary, forms, question text) in a read-only mode that facilitates reviewing or printing.
  • The introduction of a new version of logic that enhances the ability to work with string literals and newline characters, and positions the syntax of CSPro logic closer to other languages.
  • A new tool, CSCode, is a text editor with support for tabbed editing of documents. The tool also facilitates running JavaScript, editing and validating JSON, designing and testing HTML dialogs, validating specification files, and more.

To get started with CSPro 8.0, be sure to update CSPro, CEntry Android, and CSWeb to the latest version.

A New CSEntry Path on a More Secure Android OS


When installing CSEntry 7.5+ you may notice the CSEntry folder has moved. The location of the CSEntry folder will depend on whether your installation is a new install or an upgrade of an existing installation. This was necessary to support more stringent security requirements on the Android OS.

New Installations of CSEntry 7.5+

With a new installation of CSEntry 7.5+ (i.e., CSEntry is not installed on the device) the CSEntry folder will be found at /Android/data/gov.census.cspro.csentry/files/csentry.

Upgrading Installations to CSEntry 7.5 – 7.6.1

Upgrading an earlier version of CSEntry to CSEntry 7.5 - 7.6.1 the location of the CSEntry folder will continue to be found at /csentry. For example, an upgrade from CSEntry 7.4 to CSEntry 7.5 will continue to use the CSEntry folder at /csentry. Upgrading to CSEntry 7.6 will again not change the location of the CSEntry folder.

Android 11+

On Android 11+, when upgrading a version of CSEntry which runs applications from /csentry to CSEntry 7.6.2+ it is necessary to complete a new installation. This can be done by uninstalling the current installation of CSEntry and then installing the new version. Note that this is necessary for all versions less than 7.5. For CSEntry 7.5 – 7.6.1 this will be necessary if it is running CSPro applications from the folder at /csentry due to an upgrade installation.

Also, with Android 11+, the only application automatically granted access to manage the contents of /Android/data is the AOSP Files application. You can access the AOSP Files application by going to Settings > Storage > tap on Files. You will be prompted to select the default application. Choose the AOSP Files application.

Caution: Do Not Delete Your Data

A consequence of this change is that uninstalling CSEntry will now remove the CSEntry folder, if it is located at /Android/data/gov.census.cspro.csentry/files/csentry. This will delete your applications and more importantly delete your data!

New and Improved Recode


CSPro 7.4 has introduced a new and improved recode statement that will allow you to get more done in a single recode. To see the differences between the old and new recode you will compare implementations of a typical education edit. Note that the old recode has been deprecated.

Specification

The goal of the two recode implementations will be to verify that students currently attending school are within the allowable age range for their grade. The specfication below defines the valid combination of grades and ages.

current-grade-attending-specification

Old Recode Implementation

Using the old recode syntax notice that you will need two recode statements. One for the minimum age and another for the maximum age.

PROC P10_GRADE_NOW_ATTENDING

preproc

   
ask if P09_ATTEND = 1 and P04_AGE >= 3// Currently attending school

postproc

   
numeric minAgeForGrade, maxAgeForGrade;

   
recode P10_GRADE_NOW_ATTENDING => minAgeForGrade;
                                 
0 => 3// Preschool and kindergarten
                                 1 => 5;
                                 
2 => 6;
                                 
3 => 7;
                                 
4 => 8;
                                 
5 => 9;
                                 
6 => 10;
                                 
7 => 11;
                                 
8 => 12;
                                 
9 => 13;
                               
10 => 14;
                               
11 => 15;
                               
12 => 16;
                               
13 => 16// University but not graduate school
                                14 => 18// Graduate school
                                15 => 15// Trade or technical school
    endrecode;

   
recode P10_GRADE_NOW_ATTENDING => maxAgeForGrade;
                                 
0 => 6// Preschool and kindergarten
                                 1 => 8;
                                 
2 => 9;
                                 
3 => 10;
                                 
4 => 11;
                                 
5 => 12;
                                 
6 => 13;
                                 
7 => 14;
                                 
8 => 15;
                                 
9 => 18;
                               
10 => 20;
                               
11 => 21;
                               
12 => 22;
                               
13 => 95// University but not graduate school
                                14 => 95// Graduate school
                                15 => 95// Trade or technical school
    endrecode;

   
if P04_AGE in minAgeForGrade:maxAgeForGrade then
       
errmsg("P10_GRADE_NOW_ATTENDING(=%d) is valid for age(=%d)", $, P04_AGE);
   
else
       
errmsg("P10_GRADE_NOW_ATTENDING(=%d) NOT valid for age(=%d)", $, P04_AGE);
       
reenter;
   
endif;

Syntax Change Between Old and New Recode

To make use of the new recode statement you will have to write your recodes with a slighly modified syntax. The table below documents these changes.

OperatorOld Recode SyntaxNew Recode Syntax
Assignment=>->
And:::
Range-:

New Recode Implementation

With this new recode statement you can determine the minimum and maximum ages within a single recode. Making your logic easier to understand and maintain.

PROC P10_GRADE_NOW_ATTENDING

preproc

   
ask if P09_ATTEND = 1 and P04_AGE >= 3// Currently attending school

postproc

   
numeric minAgeForGrade, maxAgeForGrade;

   
recode P10_GRADE_NOW_ATTENDING -> minAgeForGrade :: maxAgeForGrade;
                                 
0 ->              3 :: 6// Preschool and kindergarten
                                 1 ->              5 :: 8;
                                 
2 ->              6 :: 9;
                                 
3 ->              7 :: 10;
                                 
4 ->              8 :: 11;
                                 
5 ->              9 :: 12;
                                 
6 ->             10 :: 13;
                                 
7 ->             11 :: 14;
                                 
8 ->             12 :: 15;
                                 
9 ->             13 :: 18;
                               
10 ->             14 :: 20;
                               
11 ->             15 :: 21;
                               
12 ->             16 :: 22;
                               
13 ->             16 :: 95// University but not graduate school
                                14 ->             18 :: 95// Graduate school
                                15 ->             15 :: 95// Trade or technical school
    endrecode;

   
if P04_AGE in minAgeForGrade:maxAgeForGrade then
       
errmsg("P10_GRADE_NOW_ATTENDING(=%d) is valid for age(=%d)", $, P04_AGE);
   
else
       
errmsg("P10_GRADE_NOW_ATTENDING(=%d) NOT valid for age(=%d)", $, P04_AGE);
       
reenter;
   
endif;

Flag Implementation

Another approach is to create a test flag. In the logic below, grade_is_valid is used to show whether or not the combination of grades and ages are valid. This can further increase the readability of your logic.

PROC P10_GRADE_NOW_ATTENDING

preproc

   
ask if P09_ATTEND = 1 and P04_AGE >= 3// Currently attending school

postproc

   
numeric grade_is_valid = false;

   
recode P10_GRADE_NOW_ATTENDING :: P04_AGE -> grade_is_valid;
                                 
0 ::  3:6    -> true// Preschool and kindergarten
                                 1 ::  5:8    -> true;
                                 
2 ::  6:9    -> true;
                                 
3 ::  7:10   -> true;
                                 
4 ::  8 11   -> true;
                                 
5 ::  9:12   -> true;
                                 
6 :: 10:13   -> true;
                                 
7 :: 11:14   -> true;
                                 
8 :: 12:15   -> true;
                                 
9 :: 13:18   -> true;
                               
10 :: 14:20   -> true;
                               
11 :: 15:21   -> true;
                               
12 :: 16:22   -> true;
                               
13 :: 16:95   -> true// University but not graduate school
                                14 :: 18:95   -> true// Graduate school
                                15 :: 15:95   -> true// Trade or technical school
    endrecode;

   
if grade_is_valid then
       
errmsg("P10_GRADE_NOW_ATTENDING(=%d) is valid for age(=%d)", $, P04_AGE);
   
else
       
errmsg("P10_GRADE_NOW_ATTENDING(=%d) NOT valid for age(=%d)", $, P04_AGE);
       
reenter;
   
endif;