CSSOrt silently omit data when a record have more instances than the allowed maximum

Discussions about tools to complement CSPro data processing
Forum rules
New release: CSPro 8.0
pleon
Posts: 7
Joined: March 17th, 2017, 5:05 pm

CSSOrt silently omit data when a record have more instances than the allowed maximum

Post by pleon »

Our dictionary have a record of people/population with a maximimum of 30 person per home.

The data file (text), built with a custom non-CSPRO application, contained homes with more than 30 people.

When sorted the people above the 30 mark where omitted without warning.

We thinnk that there should be a warning in those cases.

regards
- Pablo
josh
Posts: 2399
Joined: May 5th, 2014, 12:49 pm
Location: Washington DC

Re: CSSOrt silently omit data when a record have more instances than the allowed maximum

Post by josh »

Normally there should be a warning written to the listing file. Is this with version 6.3 or version 7.0?
pleon
Posts: 7
Joined: March 17th, 2017, 5:05 pm

Re: CSSOrt silently omit data when a record have more instances than the allowed maximum

Post by pleon »

Oh, sorry!

it's Version 7.0.2 (29 june 2017)


I've not seen a listingfile window, just a dialog box sayoing that sort was completed.


regards
- Pablo
pleon
Posts: 7
Joined: March 17th, 2017, 5:05 pm

Re: CSSOrt silently omit data when a record have more instances than the allowed maximum

Post by pleon »

The CSFreqRun do show a message like:

Process Messages

*** Case [XXXXXXXXXXX] has 5 messages (0 E / 5 W / 0U)
W 10006 Too many occurrences for Record or Group POP_R (Case:
'XXXXXXXXXXX)
W 10006 Too many occurrences for Record or Group POP_R (Case:
'XXXXXXXXXXX')
W 10006 Too many occurrences for Record or Group POP_R (Case:
'XXXXXXXXXXX')
W 10006 Too many occurrences for Record or Group POP_R (Case:
'XXXXXXXXXXX')
W 10006 Too many occurrences for Record or Group POP_R (Case:
'XXXXXXXXXXX')


- P
josh
Posts: 2399
Joined: May 5th, 2014, 12:49 pm
Location: Washington DC

Re: CSSOrt silently omit data when a record have more instances than the allowed maximum

Post by josh »

You should see the same errors in the file CSSort.lst that is generated in the same folder as the dictionary. We should probably display this file if it has warnings in it like we do with Frequencies.
pleon
Posts: 7
Joined: March 17th, 2017, 5:05 pm

Re: CSSOrt silently omit data when a record have more instances than the allowed maximum

Post by pleon »

Yep, showing the file or at least advice to look the log file on closing dialog box.

regards
- Pablo
josh
Posts: 2399
Joined: May 5th, 2014, 12:49 pm
Location: Washington DC

Re: CSSOrt silently omit data when a record have more instances than the allowed maximum

Post by josh »

Added to our todo list.
josh
Posts: 2399
Joined: May 5th, 2014, 12:49 pm
Location: Washington DC

Re: CSSOrt silently omit data when a record have more instances than the allowed maximum

Post by josh »

This has been fixed for version 7.6 which will be released soon.
tcroft
Posts: 2
Joined: December 16th, 2020, 5:03 pm

Re: CSSOrt silently omit data when a record have more instances than the allowed maximum

Post by tcroft »

Hey Josh, I think we are seeing the same problem when we create recode files (special output files) and have more than one case at the second level in a 2-level dictionary. The output data file looks ok (not certain about this yet, but it looks ok), but we are getting many many warning messages like this, even using CSPro 7.5.1.

Do you have a date for release of 7.6? Otherwise we have to revert back to CSPro 6.3

Thanks. Trevor
josh
Posts: 2399
Joined: May 5th, 2014, 12:49 pm
Location: Washington DC

Re: CSSOrt silently omit data when a record have more instances than the allowed maximum

Post by josh »

Trevor, I think this is a different issue. The fix for this in 7.6 is simply to show the message at the end of the cssort run if there were warnings. In this case the warnings are legit since the number of record occs in the data file is greater than the max in the dictionary. So the fix doesn't remove the warnings or anything - it just shows the message file so that they are more noticeable.

If you can create a test app/test data file that shows the issue you are having and send it to us we will take a look.
Post Reply