Update Existing Resource File (Make Revision)

API Endpoint: resource_update_api

Updates existing resource file and/or resource metadata [Private Workspace Only]

EDX resources have a revision history. An update to the resource’s revision history can be a change in the resource’s metadata and/or a new file upload that becomes the most recent file at the “top of the revision history stack”.

Attention

Due to EDX review workflow governance policies for approval of public submissions, this feature is only available to EDX workspaces (private side) at this time.

HTML resource formats are NOT supported at this time. Uploaded File resources are the ONLY supported format at this time.

Important

An API Key is required for ALL private workspace activities.

The API Key name component can be added to your header using any of the following ways, “X-CKAN-API-Key”, “EDX-API-Key”, or “Authorization”.

Updating Revision Scenarios

  1. Upload file to resource to become latest revision

  2. Update resource metadata

  3. Upload file to become latest revision and update resource metadata

Resource Revision Update Clarifications

  • HTML Link Resources - Attempting to upload a file to an existing HTML link resource will result in the API returning a message noting that this action is not allowed.

  • Only Updating Metadata - When only updating resource metadata, you do NOT need to specify a file path or resource_name.

  • Only Updating File - When only updating resource file, you ONLY need to specify a file path, resource_id, and resource_name.

EDX License Parameters

These parameters are used when specifying a license type.

Note

You may use either the exact License Value Name or the exact Description text when defining license in your data dictionary

Important

Only “Open” license types can be utilized when pushing data to the public side of EDX

License Value Name

Description

License Type

bsd3

BSD-3 Clause

Open

cc-by

Creative Commons Attribution

Open

cc-by-sa

Creative Commons Attribution Share-Alike

Open

cc-zero

Creative Commons CCZero

Open

cc-nc

Creative Commons Non-Commercial (Any)

Open

cui

Controlled Unclassified Information

Closed

gfdl

GNU Free Documentation License

Open

notspecified

License Not Specified

Open

non-disclosure-agreement

Non-Disclosure Agreement

Closed

no-license-restriction

No License Restrictions

Open

ouo

Official Use Only

Closed

odc-by

Open Data Commons Attribution License

Open

odc-odbl

Open Data Commons Open Database License (ODbL)

Open

odc-pddl

Open Data Commons Public Domain Dedication and License (PDDL)

Open

other-at

Other (Attribution)

Closed

other-nc

Other (Non-Commercial)

Open

other-closed

Other (Not Open)

Closed

other-open

Other (Open)

Open

other-pd

Other (Public Domain)

Open

prop-comm

Proprietary (Commercial)

Closed

uk-ogl

UK Open Government License (OGL)

Open

Parameter Definitions

Parameter name

Description

Required Field

resource_id

The ID of resource that you wish to revise

Required

resource_name

The name that you want to give to your uploaded resource. Be sure to include file extension.

Required

description

The description of the resource you are revising

Optional

name

The human readable name of resource if you wish to not use resource’s file name by default

Optional

license_type

License type for resource (must use EDX license types [see license table above])

Optional

chunk

Boolean (True, False) to send file to server in chunks

Optional

original_size

Total size in bytes of file being transferred via API

Required if using chunk

Example 1: Resource Revision [File Only]

Attention

  • Add the "User-Agent": parameter within the headers of the API request

  • Set "User-Agent": to the value "EDX-USER"

  • python
  • output
import os
import requests

headers = {
    "EDX-API-Key": 'YOUR-API-KEY-HERE',
    "User-Agent": "EDX-USER"
}

# Path to your file
path = '/home/user/test.png'

data = {
    "resource_id": '99f513c5-e309-4e15-8db2-b1a37395c3be',
    "resource_name": 'test.png',
}

if os.path.exists(path):
    with open(path, "rb") as f:
        url = 'https://edx.netl.doe.gov/api/3/action/resource_update_api'

        r = requests.post(
            url, # URL to API endpoint
            headers=headers, # Headers dictionary
            files={'file': f},  # byte stream of chunked data
            data=data, # Dictionary of data params
        )

        print (r.json())

else:
    print('The path does not exist.')

{'help': '/api/3/action/help_show?name=resource_update_api', 'success': True,
'result': {'id': '99f513c5-e309-4e15-8db2-b1a37395c3be',
'key': '2024-08-05T12:11:38.532891/8b0ebdbb-dbce-4f66-9811-a3558d5c18dd/test.png', 'progress': '100%',
'created': '2024-08-05T12:11:39.041170', 'resource_name': 'test.png'}}

Example 2: Resource Revision [Metadata Only]

Attention

  • Add the "User-Agent": parameter within the headers of the API request

  • Set "User-Agent": to the value "EDX-USER"

  • python
  • output
import requests

headers = {
    "EDX-API-Key": 'YOUR-API-KEY-HERE',
    "User-Agent": "EDX-USER"
}

data = {
    "resource_id": '99f513c5-e309-4e15-8db2-b1a37395c3be',
    "description": 'This is the description of the resource.',
    "name": 'Final Report Document from John Smith',
    "license_type": 'Creative Commons CCZero',
}


url = 'https://edx.netl.doe.gov/api/3/action/resource_update_api'

r = requests.post(
    url, # URL to API endpoint
    headers=headers, # Headers dictionary
    data=data, # Dictionary of data params
)

print (r.json())

{'help': '/api/3/action/help_show?name=resource_update_api', 'success': True,
'result': {'id': '99f513c5-e309-4e15-8db2-b1a37395c3be',
'key': '2024-08-05T12:11:38.532891/8b0ebdbb-dbce-4f66-9811-a3558d5c18dd/test.png', 'progress': '100%',
'created': '2024-08-05T12:11:39.041170', 'resource_name': 'test.png'}}

Example 3: Resource Revision [File Upload & Metadata Update]

Attention

  • Add the "User-Agent": parameter within the headers of the API request

  • Set "User-Agent": to the value "EDX-USER"

  • python
  • output
import os
import requests

headers = {
    "EDX-API-Key": 'YOUR-API-KEY-HERE',
    "User-Agent": "EDX-USER"
}

# Path to your file
path = '/home/user/test.png'

data = {
    "resource_id": '99f513c5-e309-4e15-8db2-b1a37395c3be',
    "description": 'This is the description of the resource.',
    "resource_name": 'test.png',
    "name": 'resource name',
    "license_type": 'cc-by',
    "chunk": True, # Set to True when sending file in chunks. Server logic uses this to know when to expect additional file data.
    "original_size": os.path.getsize(path), # Determine your file size. Server logic uses this to determine when a file is complete.
}

# Set chunk size. In bytes.
chunk_size = 10000000 # 10 MB chunks

if os.path.exists(path):
    with open(path, "rb") as f:
        url = 'https://edx.netl.doe.gov/api/3/action/resource_update_api'
        while True:
            chunked_stream = f.read(chunk_size)
            if not chunked_stream:
                break # Break out of loop and stop uploading when no more bytes to stream
            r = requests.post(
                url, # URL to API endpoint
                headers=headers, # Headers dictionary
                files={'file': chunked_stream}, # byte stream of chunked data
                data=data, # Dictionary of data params
            )

            r_json = r.json()

            print (r.json())
            # IMPORTANT!!! Add key parameter from response object to request data when chunking.
            # Server logic will require key data in order to determine which file to append additional file data.
            data['key'] = r_json['result']['key'] # ex. of response data: u'key': u'2020-10-27T14:18:29.946Z/c9a8f82e-4bf0-4563-b931-d97b27a3f2a1/test.png'

            print (r.json()['result']['progress']) # Print out current progress of upload.
        print(r.json())
else:
    print('The path does not exist.')

{'help': '/api/3/action/help_show?name=resource_update_api', 'success': True,
'result': {'id': '99f513c5-e309-4e15-8db2-b1a37395c3be',
'key': '2024-08-05T12:11:38.532891/8b0ebdbb-dbce-4f66-9811-a3558d5c18dd/test.png', 'progress': '100%',
'created': '2024-08-05T12:11:39.041170', 'resource_name': 'test.png'}}

Example 4: Resource Revision [Chunked Revision Update]

Attention

  • Add the "User-Agent": parameter within the headers of the API request

  • Set "User-Agent": to the value "EDX-USER"

Note

If your chunk size is larger than your file size, then the whole file will be uploaded in one request. It is highly recommended to upload large files in smaller chunks (e.g. 5 GB file - Chunked at chunk_size = 1024 * 1024 * 10 #10 MB chunks, value must be multiple of 1024 ). Returned output will provide the status of uploading file

  • python
  • output
import os
import requests

headers = {
    "EDX-API-Key": 'YOUR-API-KEY-HERE',
    "User-Agent": "EDX-USER"
}

# Path to your file
path = '/home/user/test.png'
chunk_size = 1024 * 1024 * 10 # 10 MB chunks, value must be multiple of 1024

data = {
    "resource_id": 'fef2106d-b726-4314-84e2-e7551fa34f0e',
    "resource_name": 'test.png',
    "original_size": os.path.getsize(path), # Determine your file size. Server logic uses this to determine when a file is complete.
    "chunk_size": chunk_size
}

if os.path.exists(path):
    with open(path, "rb") as f:
        url = 'https://edx.netl.doe.gov/api/3/action/resource_update_api'
        while True:
            chunked_stream = f.read(chunk_size)
            if not chunked_stream:
                break # Break out of loop and stop uploading when no more bytes to stream
            r = requests.post(
                url, # URL to API endpoint
                headers=headers, # Headers dictionary
                files={'file': chunked_stream}, # byte stream of chunked data
                data=data, # Dictionary of data params
            )

            print (r.json())

else:
    print('The path does not exist.')

{'help': '/api/3/action/help_show?name=resource_update_api', 'success': True,
'result': {'id': 'fef2106d-b726-4314-84e2-e7551fa34f0e',
'key': '2024-08-05T12:11:38.532891/8b0ebdbb-dbce-4f66-9811-a3558d5c18dd/test.png', 'progress': '100%',
'created': '2024-08-05T12:11:39.041170', 'resource_name': 'test.png'}}