Encoding problems in CSPro 5

Discussions about CSEntry
Forum rules
New release: CSPro 8.0
Post Reply
Anne
Posts: 104
Joined: January 11th, 2012, 12:55 am

Encoding problems in CSPro 5

Post by Anne »

I and my russian speaking colleagues have been developing an application together - me on my western computer, and the other two on computer with russian windows operating system.

The whole application has always had a "garblegarble" look like this: "Îáðàçîâàíèå ÷ëåíîâ äîìîõîçÿéñòâà" on my computer where it should have been cyrillic, so I have been hoping that CSPro 5 could deal with it. (I always managed to get the field fonts, text fonts and CAPI questions font to be in cyrillic, but not the dictionary (the labels and the value set labels))

Instead of fixing the problem, CSPro 5 only made it worse: Now there's garblegarble on the russian computers too. This is what I've tried to fix the problem so far:

- using the converter included in CSPro 5
- using text editors (ultraedit and foxe) to convert the dictionary to unicode.

The first method didn't make any difference to my dictionary, while ultraedit converted my file to an encoding that looks fine in notepad, but in CSPro it now looks like this: " по письменному ". Definitly another kind of garblegarble, but I don't want garblegarble.

Does anybody know how to deal with this?
Gregory Martin
Posts: 1777
Joined: December 5th, 2011, 11:27 pm
Location: Washington, DC

Re: Encoding problems in CSPro 5

Post by Gregory Martin »

I share your desire in not wanting "garblegarble," so let's try to figure out what's wrong.

In versions of CSPro prior to 5, the installer program asked for an installation language. This setting was saved in the registry and used for some options, like the labels in the dictionary. The labels would be rendered in Courier New, using the Russian character set flag, which is Windows-1251.

So what you need to do is convert from Windows-1251 to Unicode. Here's how you can do this in Notepad++.

1) Open the various files (dictionary, form, etc.).

2) From the Encoding menu, select Character sets -> Cyrillic -> Windows-1251.

3) Then select Encoding -> Convert to UTF-8.

4) Save the file, and now, hopefully, CSPro will render all of the text correctly.
Anne
Posts: 104
Joined: January 11th, 2012, 12:55 am

Re: Encoding problems in CSPro 5

Post by Anne »

Tried this, but maybe I do something wrong? The files did'nt change at all.

Attachingone of my files to let anybody else try ;)

(The file extention .fmf was not allowed so I added .txt too..)

Anne
Attachments
F6.fmf.txt
(88.77 KiB) Downloaded 391 times
Gregory Martin
Posts: 1777
Joined: December 5th, 2011, 11:27 pm
Location: Washington, DC

Re: Encoding problems in CSPro 5

Post by Gregory Martin »

The file changes for me, but I'm not sure the output is correct because there still seem to be strange characters, like this:

ÆÈËÈÙÍÎ-ÊÎÌÌÓÍÀËÜÍÛÅ ÐÀÑÕÎÄÛ

Is that correct?

See attached.
Attachments
F6.zip
(13.14 KiB) Downloaded 384 times
Anne
Posts: 104
Joined: January 11th, 2012, 12:55 am

Re: Encoding problems in CSPro 5

Post by Anne »

Thank you very much for trying, but no: it's still not correct.

..and now I'm starting to worry that we have to change the texts manually.. My russian speaking colleagues are so not going to like this :cry:

Anne
Gregory Martin
Posts: 1777
Joined: December 5th, 2011, 11:27 pm
Location: Washington, DC

Re: Encoding problems in CSPro 5

Post by Gregory Martin »

Can you send me the original application (not just the form file)? Let me take a look and see if there's anything else that can be done. You shouldn't have to manually edit the text!
Anne
Posts: 104
Joined: January 11th, 2012, 12:55 am

Re: Encoding problems in CSPro 5

Post by Anne »

Here it is!

Anne
Attachments
F6.zip
(69.3 KiB) Downloaded 396 times
Gregory Martin
Posts: 1777
Joined: December 5th, 2011, 11:27 pm
Location: Washington, DC

Re: Encoding problems in CSPro 5

Post by Gregory Martin »

I'm not sure what I was doing previously, because I was now able to successfully convert the files using the Notepad++ trick I detailed above. See attached.
Attachments
F6.zip
(36.64 KiB) Downloaded 419 times
Anne
Posts: 104
Joined: January 11th, 2012, 12:55 am

Re: Encoding problems in CSPro 5

Post by Anne »

Finally found a solution working on my computers too:

I had no problems using your method to convert non-CSPro files with weird charsets. It just didn't work for the CSPro files, and now I know why:

the default for the non-CSPro files was that they were opened "encoded in ANSI" when using Notepad++. Not so for the CSPro files (at least not on my computers), so by doing the following it works here too:

1) open the file
2) Choose Encoding - convert to ANSI
3) choose Enconding - Character sets cyrillic - windows-1251
4) Choose encoding - convert to UTF-8

Very happy that we finally managed! Thank you very much!

Anne
lestcape
Posts: 37
Joined: August 27th, 2016, 1:11 pm

Re: Encoding problems in CSPro 5

Post by lestcape »

Maybe to late for this post, but probably useful. The CSPro dictionary is saving the positions base on an encode and then will be helpfully set the encode that was used to estimate the positions of the items as a dictionary property. In that way, will be possible always return back the data file to the original encode where the dictionary was create and read the information of the data file using the dictionary positions. Otherwise, any other procedure that attemp to read the data file that come from another different encode will fail, and if we have not the property information of the original encode restore original encode, will be really difficult. CSPro is an international software then will be nice to have support for translate data from different languages (encodes).
Post Reply