Merge branch 'nono-zenodo' into 'v3'

Remove zenodo scripts and job

See merge request openflexure/openflexure-microscope-server!336

.gitlab-ci.yml

@@ -275,29 +275,6 @@ server_integration_tests:
 #     - tags
 #     - web
 
-# zenodo:
-#   stage: deploy
-#   image: ubuntu:20.04
-#   before_script:
-#     - apt-get update -qq
-#     - apt-get -y -qq install git python3-pip
-#     - pip3 install -r scripts/zenodo/requirements.txt
-#   variables:
-#     ZENODO_USE_SANDBOX: "false"
-#   script:
-#     - git archive "${CI_COMMIT_REF_NAME}" --format=zip --output="${CI_PROJECT_NAME}-${CI_COMMIT_REF_NAME}-source.zip"
-#     - python3 scripts/zenodo/upload_to_zenodo.py "${CI_PROJECT_NAME}-${CI_COMMIT_REF_NAME}-source.zip" dist/*
-#   artifacts:
-#     # this is only a small html link, let's just keep it forever
-#     # gitlab doesn't understand "expire_in: never" yet though
-#     expire_in: 100000 years
-#     name: zenodo-${CI_PROJECT_NAME}-${CI_COMMIT_REF_NAME}-link.html
-#     paths:
-#       - zenodo-link.html
-#   only:
-#     - tags
-#     - web
-
 pages:
   needs:
     - job: build-docs

scripts/zenodo/README.md

@@ -1,45 +0,0 @@
-# Zenodo archiving
-This folder contains some scripts put together by Kaspar Bumke, and reused by Richard, to upload the contents of the repository, together with any CI build artefacts, to Zenodo.  The result is a link, taking you to a pre-populated upload on Zenodo that you can manually correct and upload.  In order to customise it for a new project, there are a few steps you need, outlined below.
-
-## Setting up archival
-* Copy this folder to `scripts/zenodo` in your repository.
-* Customise `metadata.yaml` (in this folder) with the metadata you want to be attached to your uploads.  Zenodo provide [documentation](https://developers.zenodo.org/#deposit-metadata) on the contents of this file, but hopefully the one here will be a useful start.
-* Add a CI job to `.gitlab-ci.yml` at the top level of your repository.  The one in this repository should serve as an example.  This requires you to add a couple of places:
-  - Ensure you have a "stage" called "deploy" that runs after you've built anything you will want to archive.  This is the last stage by default, but if you have a custom list of stages, it is usually found at the top of the file.  For example:
-    ```yaml
-    stages:
-    - analysis
-    - testing
-    - build
-    - package
-    - deploy
-    ```
-  - Copy and paste the block below, to build and upload the source and built artefacts.
-    ```yaml
-    zenodo:
-    stage: deploy
-    image: ubuntu:20.04
-    before_script:
-        - apt-get update -qq
-        - apt-get -y -qq install git python3-pip
-        - pip3 install -r scripts/zenodo/requirements.txt
-    variables:
-        ZENODO_USE_SANDBOX: "true"
-    script:
-        - git archive "${CI_COMMIT_REF_NAME}" --format=zip --output="${CI_PROJECT_NAME}-${CI_COMMIT_REF_NAME}-source.zip"
-        - python3 scripts/zenodo/upload_to_zenodo.py "${CI_PROJECT_NAME}-${CI_COMMIT_REF_NAME}-source.zip" dist/*
-    artifacts:
-        # this is only a small html link, let's just keep it forever
-        # gitlab doesn't understand "expire_in: never" yet though
-        expire_in: 100000 years
-        name: zenodo-${CI_PROJECT_NAME}-${CI_COMMIT_REF_NAME}-link.html
-        paths:
-        - zenodo-link.html
-    only:
-        - tags
-        - web
-    ```
-  - Customise the lines that create archives or specify files for upload.  As a minimum, you will need to customise the list of files to upload.  The block above will upload everything in the `dist/` folder, which is populated with archives produced in previous CI jobs.  If you don't have any other CI jobs, you can just delete the `dist/*` from the end of the relevant line, and the script will upload an archive of the source code only.
-  - **Note: LFS files are not included in the source archive.** If you use LFS, you may want to make the source archive manually in CI, so that LFS files are downloaded and included.  If you don't know what LFS is, you can probably ignore this.  If you look at your upload, and there are lots of files that should be large, and are actually less than 200 bytes, you probably do have LFS enabled.
-* Generate a Zenodo access token.  There are instructions in the [Zenodo documentation](https://developers.zenodo.org/#authentication), but in brief you must first log in to Zenodo and [generate an access token](https://zenodo.org/account/settings/applications/tokens/new/).  If you want to try it out using the "sandbox" before uploading to Zenodo proper, you can do the same thing with the [sandbox site](https://sandbox.zenodo.org/account/settings/applications/tokens/new/).  You need to tick `deposit:write` on the token's permissions.
-* Paste your Zenodo access token into a [GitLab CI variable](https://docs.gitlab.com/ee/ci/variables/#add-a-cicd-variable-to-a-project) called `ZENODO_API_KEY_REAL`, and paste the sandbox access token into another variable called `ZENODO_API_KEY_SANDBOX`.

scripts/zenodo/metadata.yaml

@@ -1,20 +0,0 @@
-# see https://developers.zenodo.org/#deposit-metadata
-title: OpenFlexure Microscope Software
-upload_type: other
-description: >
-  Software to run an OpenFlexure Microscope.  This consists of a Python 
-  server, and a web application client.
-creators:
-  - name: 'Collins, Joel T.'
-    affiliation: 'University of Bath'
-    orcid: '0000-0002-0167-4969'
-  - name: 'Stirling, Julian'
-    affiliation: 'University of Bath'
-    orcid: '0000-0002-8270-9237'
-  - name: 'Bumke, Kaspar'
-    affiliation: 'University of Bath'
-  - name: 'Bowman, Richard'
-    affiliation: 'University of Bath'
-grants:
-  - id: "10.13039/501100000690::EP/R011443/1"
-  - id: "10.13039/501100000690::EP/R013969/1"

scripts/zenodo/requirements.txt

@@ -1,2 +0,0 @@
-PyYAML==5.3.1
-requests==2.23.0

scripts/zenodo/upload_to_zenodo.py

@@ -1,66 +0,0 @@
-"""A script for uploading data to Zenodo."""
-
-import os
-from argparse import ArgumentParser, Namespace
-from zenodo import Zenodo
-import yaml
-
-# You have to explicitly set ZENODO_USE_SANDBOX=false to not use the
-# sandbox, any other value or unset variable means this script will use
-# the sandbox zenodo site
-if "ZENODO_USE_SANDBOX" in os.environ:
-    USE_SANDBOX = not os.environ["ZENODO_USE_SANDBOX"] == "false"
-else:
-    USE_SANDBOX = True
-
-
-if USE_SANDBOX:
-    API_KEY = os.environ["ZENODO_API_KEY_SANDBOX"]
-else:
-    API_KEY = os.environ["ZENODO_API_KEY_REAL"]
-
-
-def parse_arguments() -> Namespace:
-    """Parse the commandline arguments and return a namespace containing the result."""
-    p = ArgumentParser(description="Upload data to Zenodo")
-    p.add_argument("paths", help="Directories and files to upload to Zenodo", nargs="*")
-    return p.parse_args()
-
-
-def script_directory(path):
-    """Return path to directory of the current script."""
-    return os.path.join(os.path.dirname(os.path.realpath(__file__)), path)
-
-
-def get_meta():
-    """Return the metadata in the script directory as a dictionary."""
-    with open(script_directory("metadata.yaml"), encoding="utf-8") as f:
-        metadata = f.read()
-
-    return yaml.safe_load(metadata)
-
-
-def main():
-    """Create the zenodo deposit and upload it to Zenodo.
-
-    This is the main function called when the full script is run.
-    """
-    args = parse_arguments()
-
-    metadata = get_meta()
-
-    zenodo = Zenodo(API_KEY, USE_SANDBOX)
-    deposit = zenodo.create_new_deposit()
-
-    zenodo.set_metadata(deposit["id"], metadata)
-
-    for path in args.paths:
-        zenodo.upload_file(deposit["id"], path)
-
-    link = deposit["links"]["latest_draft_html"]
-    with open("zenodo-link.html", "w", encoding="utf-8") as f:
-        f.write(f'<a href="{link}">{link}</a>')
-
-
-if __name__ == "__main__":
-    main()

scripts/zenodo/zenodo.py

@@ -1,106 +0,0 @@
-"""A script for packaging data for Zenodo.
-
-Copyright (c) 2020 Bath Open Instrumentation Group
-Adapted from: https://gitlab.com/schlauch/zenodo-api-test/
-Copyright (c) 2018 German Aerospace Center (DLR). All rights reserved.
-SPDX-License-Identifier: MIT-DLR
-"""
-
-import json
-import os
-
-import requests
-
-
-class Zenodo:
-    """A class that packages data for Zenodo."""
-
-    def __init__(self, api_token, use_sandbox=True):
-        """Initialise the zenodo packager with the API token.
-
-        :param api_token: Your Zenodo API key.
-        :param use_sandbox: True to pushing to ``sandbox.zenodo.org`` rather than
-            ``zenodo.org`` for testing.
-        """
-        self._api_token = api_token
-        self._use_sandbox = use_sandbox
-        if use_sandbox:
-            self.zenodo_url = "https://sandbox.zenodo.org/api/deposit/depositions"
-        else:
-            self.zenodo_url = "https://zenodo.org/api/deposit/depositions"
-
-    def create_new_deposit(self):
-        """Create a new (unpublished) Zenodo deposit and return its deposition ID."""
-        headers = {"Content-Type": "application/json"}
-        r = requests.post(
-            self.zenodo_url,
-            params={"access_token": self._api_token},
-            json={},
-            headers=headers,
-        )
-        print(r.status_code)
-        print(r.json())
-        return r.json()
-
-    def set_metadata(self, deposition_id, metadata):
-        """Set the given metadata for the specified deposit."""
-        headers = {"Content-Type": "application/json"}
-        r = requests.put(
-            self.zenodo_url + "/{}".format(deposition_id),
-            params={"access_token": self._api_token},
-            data=json.dumps({"metadata": metadata}),
-            headers=headers,
-        )
-        print(r.status_code)
-        print(r.json())
-
-    def upload_file(self, deposition_id, file_path):
-        """Upload a new file for the given deposit."""
-        file_name = os.path.basename(file_path)
-        data = {"filename": file_name}
-        files = {"file": open(file_path, "rb")}
-        r = requests.post(
-            self.zenodo_url + "/{}/files".format(deposition_id),
-            params={"access_token": self._api_token},
-            data=data,
-            files=files,
-        )
-        print(r.status_code)
-        print(r.json())
-
-    def publish_deposit(self, deposition_id):
-        """Publish the given deposit. BEWARE: It is now visible to all!!!"""
-        r = requests.post(
-            self.zenodo_url + "/{}/actions/publish".format(deposition_id),
-            params={"access_token": self._api_token},
-        )
-        print(r.status_code)
-        print(r.json())
-
-    def create_new_version(self, deposition_id):
-        """Create a new version of an already published deposit."""
-        r = requests.post(
-            self.zenodo_url + "/{}/actions/newversion".format(deposition_id),
-            params={"access_token": self._api_token},
-        )
-        print(r.status_code)
-        print(r.json())
-        return os.path.basename(r.json()["links"]["latest_draft"])
-
-    def remove_all_files(self, deposition_id):
-        """Remove all uploaded files of a unpublished deposit."""
-        r = requests.get(
-            self.zenodo_url + "/{}/files".format(deposition_id),
-            params={"access_token": self._api_token},
-        )
-        print(r.status_code)
-        print(r.json())
-        for file_entry in r.json():
-            print("Remove file entry: " + file_entry["id"])
-            print(
-                requests.delete(
-                    self.zenodo_url
-                    + "/{}/files/{}".format(deposition_id, file_entry["id"]),
-                    params={"access_token": self._api_token},
-                )
-            )