RESTful API calls to csweb server

[tercio]

Hi all,
I intend to develop a real-time dashboard that presents survey data on demand, therefore, we'd like to know if there is a way of making RESTful API calls to csweb endpoints, to retrieve data stored in the csweb mysql database. I want to create a frontend application that will make requests to the server (Using PHP or Angular, or other framework), and displays the data on the app.

Alternatively, I could take the data directly from the csweb database, but I could not understand the data in the mysql table... I could not see the survey variables in the database.

Please advise the best way to achieve this.

Regards,
Tércio

[josh]

CSWeb has a REST API. It is documented in csweb/src/api/app/swagger.json. That file is best using the tools here https://swagger.io/

The challenging part of what you want to do it is extracting the data from the CSPro text format that is stored in the database. You need to use the start positions and lengths of elements in the data dictionary to take substrings from the data and convert them to numbers. There are a couple of open source projects that have already done this that you should look at:

https://github.com/IstatCooperation/CSProDashboard
https://medium.com/@nahomt/working-with ... 6161b84ffa

[htuser]

Dear Tercio and Josh,
I intend also to use best in class dashboard https://superset.apache.org/ for live reporting of:
a) Census/Survey data;
b) Geo-enableb monitoring and evaluation system;
c) or whatever information system implemented using CSPro;

But, always waiting automatic data parsing on CSWeb (relational schema support as requested long ago, and also by Tercio);
I already tested CSpro2SQL, but it's limited...
I've just ask Aaron : viewforum.php?f=1

Please let me know!

[tercio]

CSWeb has a REST API. It is documented in csweb/src/api/app/swagger.json. That file is best using the tools here https://swagger.io/

The challenging part of what you want to do it is extracting the data from the CSPro text format that is stored in the database. You need to use the start positions and lengths of elements in the data dictionary to take substrings from the data and convert them to numbers. There are a couple of open source projects that have already done this that you should look at:

https://github.com/IstatCooperation/CSProDashboard
https://medium.com/@nahomt/working-with ... 6161b84ffa

Dear Josh,

Thank you for your promt reply, and sorry for the late reply from my side.
I really appreciate for the details you provided. I successfully found the CSWEB REST API documentation, however I will create the dashboard form the data straight from the csweb database. Many of the fields I need for the dashboard are ID Items, like to see the total number of interviews, number of interviews per province, number of interviews by area of residence (Rural, Urban), etc. As per the substrings, I am thinking of implementing a real time search engine, and for that I have already tested the use of substrings from the 'questionnaire' field in csweb, to return the specific fields I need. I am using php for now, with a database connection file, instead of using API calls, because I struggled with API authentication/authorisation.

My next goal is to find inconsistencies with the syncronized data and provide a report to subject matter specialists.

Kind regards,
Tércio

[tercio]

Dear Tercio and Josh,
I intend also to use best in class dashboard https://superset.apache.org/ for live reporting of:
a) Census/Survey data;
b) Geo-enableb monitoring and evaluation system;
c) or whatever information system implemented using CSPro;

But, always waiting automatic data parsing on CSWeb (relational schema support as requested long ago, and also by Tercio);
I already tested CSpro2SQL, but it's limited...
I've just ask Aaron : viewforum.php?f=1

Please let me know!

Hi htsuer,

Thank you for your feedback, which I really appreciate. I saw the link, however the requirement is a dashboard that is created by the local team, and this will be used for all the surveys and censuses using Cspro.

Regards,
Tércio

[Daffa Khairy]

CSWeb has a REST API. It is documented in csweb/src/api/app/swagger.json. That file is best using the tools here https://swagger.io/

The challenging part of what you want to do it is extracting the data from the CSPro text format that is stored in the database. You need to use the start positions and lengths of elements in the data dictionary to take substrings from the data and convert them to numbers. There are a couple of open source projects that have already done this that you should look at:

https://github.com/IstatCooperation/CSProDashboard
https://medium.com/@nahomt/working-with ... 6161b84ffa

Dear Josh,

Thank you for your promt reply, and sorry for the late reply from my side.
I really appreciate for the details you provided. I successfully found the CSWEB REST API documentation, however I will create the dashboard form the data straight from the csweb database. Many of the fields I need for the dashboard are ID Items, like to see the total number of interviews, number of interviews per province, number of interviews by area of residence (Rural, Urban), etc. As per the substrings, I am thinking of implementing a real time search engine, and for that I have already tested the use of substrings from the 'questionnaire' field in csweb, to return the specific fields I need. I am using php for now, with a database connection file, instead of using API calls, because I struggled with API authentication/authorisation.

My next goal is to find inconsistencies with the syncronized data and provide a report to subject matter specialists.

Kind regards,
Tércio

Hi tercio,

Could you tell me how exactly do you establish a direct database connection to the CSWeb database using PHP? I'm looking for the most reliable method to query the data for my dashboard as i'm having trouble with using API calls too

[htuser]

Hi Daffa Khairy ,
Since this post, CSPro support relational schema on the server, so you can directly connect any MySQL RDBMS Compatible dashboard with the parsed MySQL data (https://www.csprousers.org/help/CSWeb/d ... tings.html). I tested this with Apache Superset a few years ago.
For using REST API call, only the CSPro Developer Team can help you because, I directly connected the database who store questionnaire data into relational tables to the Dashboard.
Best

[Abdalla]

Hello Everyone, I just got this error about creating and setup about csweb.
Failed to contact API server http://localhost/csweb/api/ : Server error: `POST http://localhost/csweb/api/token` resulted in a `500 Internal Server Error` response: PHP Fatal error: Uncaught Error: Class "AppBundle\AppBundle" not found in C:\inetpub\wwwroot\csweb\api\ApiKernel.php:16 (truncated...)

[savy]

Were you able to resolve this issue? For setup using IIS, see the helps https://www.csprousers.org/help/CSWeb/i ... setup.html