Upload records on the OSSR

To upload records on the OSSR, you may use the Zenodo client.

The client is also used by the continuous integration to automatise project’s uploads from GitLab to Zenodo check the EOSSR scripts here.

However, to upload records to the OSSR, you will need to use your Zenodo token (create one).

import os

token = os.getenv("SANDBOX_ZENODO_TOKEN")  # Replace with your own token

if not token:
    raise SystemExit("You need to use a token for that notebook!")

import json
from eossr.api.zenodo import ZenodoAPI, Record

# Please note that for this demo we are using Zenodo Sandbox.
zen = ZenodoAPI(access_token=token, sandbox=True)

# Test token
user_deposits_response = zen.query_user_deposits()

Create a new deposit

First we would need to create a new deposit

new_deposit_response = zen.create_new_deposit()

Zenodo HTTP status

The function create_new_deposit would have raised an error in case of failure. But if you can its status code meaning using our ZenodoHTTPStatus class

from eossr.api.zenodo.http_status import ZenodoHTTPStatus

status_new_deposit = ZenodoHTTPStatus(new_deposit_response)
print(status_new_deposit)

HTTP Status Code: 201 - Created.
Request succeeded. Response included. Usually sent for POST requests.

Add files to the deposit

Then we upload to the new deposit a file (let’s say a codemeta.json metadata file)

deposit_id = new_deposit_response.json()["id"]
upload_file_response = zen.upload_file_deposit(
    deposit_id, name_file="codemeta.json", path_file="../../codemeta.json"
)

status_upload = ZenodoHTTPStatus(upload_file_response)
print(status_upload)

HTTP Status Code: 201 - Created.
Request succeeded. Response included. Usually sent for POST requests.

Add metadata to the deposit

We add metadata to the Zenodo deposit using the EOSSR codemeta2zenodo conversor

from eossr.metadata.codemeta2zenodo import parse_codemeta_and_write_zenodo_metadata_file

parse_codemeta_and_write_zenodo_metadata_file(codemeta_filename='../../codemeta.json',
                                              outdir='.',
                                              add_escape2020=True,
                                              overwrite=True,
                                              )

/builds/escape-ossr/eossr/eossr/metadata/codemeta.py:190: CodemetaRecommendedWarning: Missing 5 recommended keys: ['memoryRequirements' 'processorRequirements' 'storageRequirements'
 'copyrightHolder' 'identifier']
  warn(

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
File /builds/escape-ossr/eossr/eossr/metadata/codemeta2zenodo/converters.py:461, in DateConverter.check_type(self)
    460 try:
--> 461     datetime.datetime.fromisoformat(self.value)
    462 except ValueError as exc:

ValueError: Invalid isoformat string: ''

The above exception was the direct cause of the following exception:

ValueError                                Traceback (most recent call last)
Cell In[9], line 1
----> 1 parse_codemeta_and_write_zenodo_metadata_file(codemeta_filename='../../codemeta.json',
      2                                               outdir='.',
      3                                               add_escape2020=True,
      4                                               overwrite=True,
      5                                               )

File /builds/escape-ossr/eossr/eossr/metadata/codemeta2zenodo/core.py:217, in parse_codemeta_and_write_zenodo_metadata_file(codemeta_filename, outdir, add_escape2020, overwrite)
    203 """
    204 Reads the codemeta.json file and creates a new `.zenodo.json` file in outdir.
    205 This file contains the same information that in the codemeta.json file but following the zenodo metadata schema.
   (...)
    214     overwrite existing `.zendoo.json` file in `outdir`
    215 """
    216 meta_converter = CodeMeta2ZenodoController.from_file(codemeta_filename)
--> 217 meta_converter.convert()
    218 if add_escape2020:
    219     meta_converter.add_escape2020_community()

File /builds/escape-ossr/eossr/eossr/metadata/codemeta2zenodo/core.py:149, in CodeMeta2ZenodoController.convert(self, validate)
    147 def convert(self, validate=True):
    148     """Convert data over to zenodo format"""
--> 149     self.zenodo_data = codemeta2zenodo(self.codemeta_data)
    150     if validate:
    151         self.validate_zenodo()

File /builds/escape-ossr/eossr/eossr/metadata/codemeta2zenodo/core.py:101, in codemeta2zenodo(codemeta_dict, zenodo_access_right)
     99         sub_zenodo_dict[zenodo_key].append(converted_value)
    100 else:
--> 101     converted_value = MasterConverter(
    102         codemeta_value,
    103         codemeta_type,
    104         codemeta_key=codemeta_key,
    105         zenodo_contributor_type=zenodo_contributor_type,
    106     )
    107     if converted_value is None:
    108         pass

File /builds/escape-ossr/eossr/eossr/metadata/codemeta2zenodo/converters.py:100, in MasterConverter(codemeta_value, codemeta_type, codemeta_key, zenodo_contributor_type)
     97         raise ValueError(f"Unknown @type for {codemeta_value}")
     99 if codemeta_type in __CONVERTER_MAPPING__:
--> 100     return __CONVERTER_MAPPING__[codemeta_type](codemeta_value)
    101 else:
    102     raise ValueError(f"Unknown key type {codemeta_type}")

File /builds/escape-ossr/eossr/eossr/metadata/codemeta2zenodo/converters.py:33, in <lambda>(x)
     17 CODEMETA_CONTRIBUTORS_ROLES = __filtered_table__.loc[__filtered_table__[
     18     'Zenodo'] == 'contributors', 'Property'].values
     19 CODEMETA_CREATORS_ROLES = __filtered_table__.loc[__filtered_table__[
     20     'Zenodo'] == 'creators', 'Property'].values
     23 __CONVERTER_MAPPING__ = {
     24     "Text or URL": lambda x: TextOrUrlConverter(x).convert(),
     25     "Number or Text": lambda x: NumberOrTextConverter(x).convert(),
     26     "Text": lambda x: TextConverter(x).convert(),
     27     "URL": lambda x: URLConverter(x).convert(),
     28     "Person": lambda x: PersonConverter(x).convert(),
     29     "Organization": lambda x: OrganizationConverter(x).convert(),
     30     "Boolean": lambda x: BooleanConverter(x).convert(),
     31     "PropertyValue or URL": lambda x: URLConverter(x).convert(),
     32     "Number": lambda x: NumberConverter(x).convert(),
---> 33     "Date": lambda x: DateConverter(x).convert(),
     34     "ComputerLanguage": lambda x: ComputerLanguageConverter(x).convert(),
     35     "SoftwareSourceCode": lambda x: SoftwareSourceCodeConverter(x).convert(),
     36     "CreativeWork": lambda x: CreativeWorkConverter(x).convert(),
     37     "DataFeed": lambda x: DataFeedConverter(x).convert(),
     38     "ScholarlyArticle": lambda x: ScholarlyArticleConverter(x).convert(),
     39     "MediaObject": lambda x: MediaObjectConverter(x).convert(),
     40     "Review": lambda x: TextConverter(x).convert(),
     41     "CodeRepository": lambda x: CodeRepositoryConverter(x).convert(),
     42     "Readme": lambda x: ReadmeConverter(x).convert(),
     43     "Integer or Text": lambda x: NumberOrTextConverter(x).convert(),
     44     "CreativeWork or URL": lambda x: URLConverter(x).convert(),
     45 }
     48 def MasterConverter(codemeta_value: Any, codemeta_type: str, codemeta_key=None, zenodo_contributor_type=None):
     49     """
     50     Converts a given value to a Zenodo-compatible format based on the provided codemeta info.
     51
   (...)
     61         ValueError: If the key type is unknown or if the @type for an Organization or Person is unknown.
     62     """

File /builds/escape-ossr/eossr/eossr/metadata/codemeta2zenodo/converters.py:152, in BaseConverter.__init__(self, value)
    150 def __init__(self, value):
    151     self.value = value
--> 152     self.check_type()

File /builds/escape-ossr/eossr/eossr/metadata/codemeta2zenodo/converters.py:463, in DateConverter.check_type(self)
    461     datetime.datetime.fromisoformat(self.value)
    462 except ValueError as exc:
--> 463     raise ValueError(
    464         f"Incorrect date format, should be ISO 8601 format, but is {self.value}") from exc

ValueError: Incorrect date format, should be ISO 8601 format, but is

# Let's have a look to the output to the .zenodo.json file
! cat .zenodo.json

cat: .zenodo.json: No such file or directory

with open('.zenodo.json') as f:
    zenodo_metadata = json.load(f)

update_metadata_info_response = zen.set_deposit_metadata(
    deposit_id, zenodo_metadata)
status_metadata = ZenodoHTTPStatus(update_metadata_info_response)
print(status_metadata)

---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
Cell In[11], line 1
----> 1 with open('.zenodo.json') as f:
      2     zenodo_metadata = json.load(f)
      4 update_metadata_info_response = zen.set_deposit_metadata(
      5     deposit_id, zenodo_metadata)

File /usr/local/lib/python3.9/site-packages/IPython/core/interactiveshell.py:310, in _modified_open(file, *args, **kwargs)
    303 if file in {0, 1, 2}:
    304     raise ValueError(
    305         f"IPython won't let you open fd={file} by default "
    306         "as it is likely to crash IPython. If you know what you are doing, "
    307         "you can use builtins' open."
    308     )
--> 310 return io_open(file, *args, **kwargs)

FileNotFoundError: [Errno 2] No such file or directory: '.zenodo.json'

Check the content of the new deposit

The request answers contain all the information of the current upload.

One can check the metadata in the JSON dict:

update_metadata_info_response.json()['metadata']

---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
Cell In[12], line 1
----> 1 update_metadata_info_response.json()['metadata']

NameError: name 'update_metadata_info_response' is not defined

---

We can check the files that we have already uploaded

update_metadata_info_response.json()["files"]

---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
Cell In[13], line 1
----> 1 update_metadata_info_response.json()["files"]

NameError: name 'update_metadata_info_response' is not defined

Publish the deposit - or not.

Finally we can publish the new deposit. However in this case, because this is just a demo, we will just erase the deposit.

# In case you would like to publish the deposit, just uncomment the following lines.

# # publish_response = z.publish_deposit(deposit_id)
# # status = ZenodoHTTPStatus(publish_response)

from eossr.api.zenodo.http_status import HTTPStatusError

erase_deposit = zen.erase_deposit(deposit_id)

The deposit has been deleted