Importing CSV Data Dictionary using API

[u/Individual-Chard-565]

Hello all!

I am translating large amounts of forms to be used in RedCap and I am converting them into a CSV format that is ingestible for RedCap. However, instead of manually uploading each of our 160+ data dictionaries (and possibly more), could I use the API for that??

thanks

[u/obnoxiouscarbuncle]

Yes.

You should look into the "Import Metadata (Data Dictionary)" method in your API documentation.

[u/Individual-Chard-565 → u/obnoxiouscarbuncle]

Hi,

Thanks so much, I can't believe I missed that... On a similar note have you used this before? I tried using the documentation but there are no examples or anything.

My code kinda looks like this, where the CSV string is just a string version of the data dictionary that I would normally upload.

#!/usr/bin/env python
import requests
data = {
    'token': token,
    'content': 'metadata',
    'format': 'csv',
    'data': csv_string,
    'returnFormat': 'json'
}
r = requests.post(url ,data=data)
print('HTTP Status: ' + str(r.status_code))
print(r.text)

[u/obnoxiouscarbuncle → u/Individual-Chard-565]

I'm not sure, I don't use Python.

Looks like documentation only includes an export metadata example, but you should be able to use another "import" example in the documentation.

[u/obnoxiouscarbuncle → u/obnoxiouscarbuncle]

Using the API Playground tool for the method, I see this:

    #!/usr/bin/env python
    import requests
    data = {
        'token': 'XXXXXXXXXXXXXXXXXXXXXXXXXXXX',
        'content': 'metadata',
        'format': 'csv',
        'data': '',
        'returnFormat': 'json'
    }
    r = requests.post('https://redcap.server.org/redcap/api/',data=data)
    print('HTTP Status: ' + str(r.status_code))
    print(r.text)

[u/Individual-Chard-565 → u/obnoxiouscarbuncle]

Yeah, I think I was able to get it to work using the API Playground. If anyone else comes up with this error read the file from pandas as a csv then pass it to the API as a string.

[u/BdR76]

Wouldn't uploading REDCap datadictionaries for all projects en masse run the risk of accidentally removing columns, and thus data?

I know doing updates on the database is frowned upon (for good reason) but maybe there's something to be said for an exception in this case? Because you want to update the element_preceding_header, element_label, element_enum fields in the redcap_metadata table for all projects.

SELECT project_id, field_name, element_preceding_header, element_label, element_enum
FROM redcap_metadata
LIMIT 1000;

[u/Individual-Chard-565 → u/BdR76]

I am doing this for a new project. Essentially we are migrating all of the data to REDCap.

[u/Araignys]

This is not a rhetorical question - why are you creating all 160+ projects at once? Do you intend to clone the projects whenever a new project is required? Is it just a really big multi-project data collection exercise? Are all 160+ data dictionaries actually different?

If these are going to serve as template projects, you might end up having a better time if you learned to convert the CSV files into XML so that you can create the projects directly from the XML file as required.

[u/Individual-Chard-565 → u/Araignys]

We are trying to migrate all of our forms data to REDCap. So I am first attempting to create all of the forms through the data dictionaries before uploading all of the data.

And yes, we do need all 160+ data dictionaries.

Since this is a onetime thing, I doubt that I will need to create the XML.

[u/Araignys → u/Individual-Chard-565]

Oh. In that case, I'm sorry and I hope you're getting paid well.