Update Existing Resource File (Make Revision) ============================================= .. admonition:: API Endpoint: resource_update_api Updates existing resource file and/or resource metadata [Private Workspace Only] EDX resources have a revision history. An update to the resource's revision history can be a change in the resource's metadata and/or a new file upload that becomes the most recent file at the "top of the revision history stack". .. attention:: Due to EDX review workflow governance policies for approval of public submissions, this feature is only available to EDX workspaces (private side) at this time. HTML resource formats are **NOT** supported at this time. Uploaded File resources are the ONLY supported format at this time. .. important:: An API Key is required for ALL private workspace activities. The API Key name component can be added to your header using any of the following ways, "X-CKAN-API-Key", "EDX-API-Key", or "Authorization". Updating Revision Scenarios --------------------------- 1. Upload file to resource to become latest revision 2. Update resource metadata 3. Upload file to become latest revision and update resource metadata Resource Revision Update Clarifications --------------------------------------- * **HTML Link Resources** - Attempting to upload a file to an existing HTML link resource will result in the API returning a message noting that this action is not allowed. * **Only Updating Metadata** - When only updating resource metadata, you do NOT need to specify a file path or ``resource_name``. * **Only Updating File** - When only updating resource file, you ONLY need to specify a file path, ``resource_id``, and ``resource_name``. EDX License Parameters ---------------------- These parameters are used when specifying a license type. .. note:: You may use either the exact License Value Name or the exact Description text when defining license in your data dictionary .. important:: Only "Open" license types can be utilized when pushing data to the public side of EDX .. list-table:: :header-rows: 1 * - License Value Name - Description - License Type * - ``bsd3`` - BSD-3 Clause - Open * - ``cc-by`` - Creative Commons Attribution - Open * - ``cc-by-sa`` - Creative Commons Attribution Share-Alike - Open * - ``cc-zero`` - Creative Commons CCZero - Open * - ``cc-nc`` - Creative Commons Non-Commercial (Any) - Open * - ``cui`` - Controlled Unclassified Information - Closed * - ``gfdl`` - GNU Free Documentation License - Open * - ``notspecified`` - License Not Specified - Open * - ``non-disclosure-agreement`` - Non-Disclosure Agreement - Closed * - ``no-license-restriction`` - No License Restrictions - Open * - ``ouo`` - Official Use Only - Closed * - ``odc-by`` - Open Data Commons Attribution License - Open * - ``odc-odbl`` - Open Data Commons Open Database License (ODbL) - Open * - ``odc-pddl`` - Open Data Commons Public Domain Dedication and License (PDDL) - Open * - ``other-at`` - Other (Attribution) - Closed * - ``other-nc`` - Other (Non-Commercial) - Open * - ``other-closed`` - Other (Not Open) - Closed * - ``other-open`` - Other (Open) - Open * - ``other-pd`` - Other (Public Domain) - Open * - ``prop-comm`` - Proprietary (Commercial) - Closed * - ``uk-ogl`` - UK Open Government License (OGL) - Open Parameter Definitions --------------------- .. list-table:: :header-rows: 1 * - Parameter name - Description - Required Field * - ``resource_id`` - The ID of resource that you wish to revise - **Required** * - ``resource_name`` - The name that you want to give to your uploaded resource. *Be sure to include file extension.* - **Required** * - ``description`` - The description of the resource you are revising - Optional * - ``name`` - The human readable name of resource *if you wish to not use resource's file name by default* - Optional * - ``license_type`` - License type for resource (must use EDX license types [see license table above]) - Optional * - ``chunk`` - Boolean (True, False) to send file to server in chunks - Optional * - ``original_size`` - Total size in bytes of file being transferred via API - **Required** if using ``chunk`` Example 1: Resource Revision [File Only] ---------------------------------------- .. attention:: * Add the ``"User-Agent":`` parameter within the ``headers`` of the API request * Set ``"User-Agent":`` to the value ``"EDX-USER"`` .. tabs:: lang .. code-tab:: python import os import requests headers = { "EDX-API-Key": 'YOUR-API-KEY-HERE', "User-Agent": "EDX-USER" } # Path to your file path = '/home/user/test.png' data = { "resource_id": '99f513c5-e309-4e15-8db2-b1a37395c3be', "resource_name": 'test.png', } if os.path.exists(path): with open(path, "rb") as f: url = 'https://edx.netl.doe.gov/api/3/action/resource_update_api' r = requests.post( url, # URL to API endpoint headers=headers, # Headers dictionary files={'file': f}, # byte stream of chunked data data=data, # Dictionary of data params ) print (r.json()) else: print('The path does not exist.') .. code-tab:: output {'help': '/api/3/action/help_show?name=resource_update_api', 'success': True, 'result': {'id': '99f513c5-e309-4e15-8db2-b1a37395c3be', 'key': '2024-08-05T12:11:38.532891/8b0ebdbb-dbce-4f66-9811-a3558d5c18dd/test.png', 'progress': '100%', 'created': '2024-08-05T12:11:39.041170', 'resource_name': 'test.png'}} Example 2: Resource Revision [Metadata Only] -------------------------------------------- .. attention:: * Add the ``"User-Agent":`` parameter within the ``headers`` of the API request * Set ``"User-Agent":`` to the value ``"EDX-USER"`` .. tabs:: lang1 .. code-tab:: python import requests headers = { "EDX-API-Key": 'YOUR-API-KEY-HERE', "User-Agent": "EDX-USER" } data = { "resource_id": '99f513c5-e309-4e15-8db2-b1a37395c3be', "description": 'This is the description of the resource.', "name": 'Final Report Document from John Smith', "license_type": 'Creative Commons CCZero', } url = 'https://edx.netl.doe.gov/api/3/action/resource_update_api' r = requests.post( url, # URL to API endpoint headers=headers, # Headers dictionary data=data, # Dictionary of data params ) print (r.json()) .. code-tab:: output {'help': '/api/3/action/help_show?name=resource_update_api', 'success': True, 'result': {'id': '99f513c5-e309-4e15-8db2-b1a37395c3be', 'key': '2024-08-05T12:11:38.532891/8b0ebdbb-dbce-4f66-9811-a3558d5c18dd/test.png', 'progress': '100%', 'created': '2024-08-05T12:11:39.041170', 'resource_name': 'test.png'}} Example 3: Resource Revision [File Upload & Metadata Update] ------------------------------------------------------------ .. attention:: * Add the ``"User-Agent":`` parameter within the ``headers`` of the API request * Set ``"User-Agent":`` to the value ``"EDX-USER"`` .. tabs:: lang2 .. code-tab:: python import os import requests headers = { "EDX-API-Key": 'YOUR-API-KEY-HERE', "User-Agent": "EDX-USER" } # Path to your file path = '/home/user/test.png' data = { "resource_id": '99f513c5-e309-4e15-8db2-b1a37395c3be', "description": 'This is the description of the resource.', "resource_name": 'test.png', "name": 'resource name', "license_type": 'cc-by', "chunk": True, # Set to True when sending file in chunks. Server logic uses this to know when to expect additional file data. "original_size": os.path.getsize(path), # Determine your file size. Server logic uses this to determine when a file is complete. } # Set chunk size. In bytes. chunk_size = 10000000 # 10 MB chunks if os.path.exists(path): with open(path, "rb") as f: url = 'https://edx.netl.doe.gov/api/3/action/resource_update_api' while True: chunked_stream = f.read(chunk_size) if not chunked_stream: break # Break out of loop and stop uploading when no more bytes to stream r = requests.post( url, # URL to API endpoint headers=headers, # Headers dictionary files={'file': chunked_stream}, # byte stream of chunked data data=data, # Dictionary of data params ) r_json = r.json() print (r.json()) # IMPORTANT!!! Add key parameter from response object to request data when chunking. # Server logic will require key data in order to determine which file to append additional file data. data['key'] = r_json['result']['key'] # ex. of response data: u'key': u'2020-10-27T14:18:29.946Z/c9a8f82e-4bf0-4563-b931-d97b27a3f2a1/test.png' print (r.json()['result']['progress']) # Print out current progress of upload. print(r.json()) else: print('The path does not exist.') .. code-tab:: output {'help': '/api/3/action/help_show?name=resource_update_api', 'success': True, 'result': {'id': '99f513c5-e309-4e15-8db2-b1a37395c3be', 'key': '2024-08-05T12:11:38.532891/8b0ebdbb-dbce-4f66-9811-a3558d5c18dd/test.png', 'progress': '100%', 'created': '2024-08-05T12:11:39.041170', 'resource_name': 'test.png'}} Example 4: Resource Revision [Chunked Revision Update] ------------------------------------------------------------ .. attention:: * Add the ``"User-Agent":`` parameter within the ``headers`` of the API request * Set ``"User-Agent":`` to the value ``"EDX-USER"`` .. note:: If your chunk size is larger than your file size, then the whole file will be uploaded in one request. It is highly recommended to upload large files in smaller chunks (e.g. 5 GB file - Chunked at chunk_size = 1024 * 1024 * 10 **#10 MB chunks, value must be multiple of 1024** ). Returned output will provide the status of uploading file .. tabs:: lang2 .. code-tab:: python import os import requests headers = { "EDX-API-Key": 'YOUR-API-KEY-HERE', "User-Agent": "EDX-USER" } # Path to your file path = '/home/user/test.png' chunk_size = 1024 * 1024 * 10 # 10 MB chunks, value must be multiple of 1024 data = { "resource_id": 'fef2106d-b726-4314-84e2-e7551fa34f0e', "resource_name": 'test.png', "original_size": os.path.getsize(path), # Determine your file size. Server logic uses this to determine when a file is complete. "chunk_size": chunk_size } if os.path.exists(path): with open(path, "rb") as f: url = 'https://edx.netl.doe.gov/api/3/action/resource_update_api' while True: chunked_stream = f.read(chunk_size) if not chunked_stream: break # Break out of loop and stop uploading when no more bytes to stream r = requests.post( url, # URL to API endpoint headers=headers, # Headers dictionary files={'file': chunked_stream}, # byte stream of chunked data data=data, # Dictionary of data params ) print (r.json()) else: print('The path does not exist.') .. code-tab:: output {'help': '/api/3/action/help_show?name=resource_update_api', 'success': True, 'result': {'id': 'fef2106d-b726-4314-84e2-e7551fa34f0e', 'key': '2024-08-05T12:11:38.532891/8b0ebdbb-dbce-4f66-9811-a3558d5c18dd/test.png', 'progress': '100%', 'created': '2024-08-05T12:11:39.041170', 'resource_name': 'test.png'}}