Registry Aspect API Usage

[BrandonFeldhaus]

Hi,

I I'm currently developing with MAGDA and having an issue using the Registry Aspect API on my instance. What I am trying to do is modify an Aspect by ID, specifically the project-open-data dataset and distribution references but I receive an "Unauthorized" error message with a 401 error code.

The MAGDA instance was set up using the Magda-config repository which has a MAGDA helm chart version of '0.0.59'.

I receive this error when preforming a PUT and JSON PATCH, here is an example of my PUT request where I include all necessary headers. The JWT was created using the acs-cmd tool.

var XMLHttpRequest = require("xmlhttprequest").XMLHttpRequest;

var url = "http://dn.datacube.online/api/v0/registry/aspects/project-open-data-dataset";

var xhr = new XMLHttpRequest();
xhr.open("PUT", url);

xhr.setRequestHeader("X-Magda-Session", "<JWT>");
xhr.setRequestHeader("X-Magda-API-Key-Id", "<API-KEY-ID>");
xhr.setRequestHeader("X-Magda-Tenant-Id", 0);
xhr.setRequestHeader("X-Magda-API-Key", "<API-KEY>");
xhr.setRequestHeader("Content-Type", "application/json");

xhr.onreadystatechange = function () {
   if (xhr.readyState === 4) {
      console.log(xhr.status);
      console.log(xhr.responseText);
   }};

var data = `{
    "$schema": "http://json-schema.org/schema#",
    "title": "Aspect capturing a complete representation of a dataset according to Project Open Data (data.json).",
    "type": "object",
    "allOf": [
        {
            "$ref": "https://raw.githubusercontent.com/BrandonFeldhaus/project-open-data-dashboard/main/schema/non-federal-v1.1/dataset-non-federal.json"
        }
    ]
}`;

xhr.send(data);

My initial thinking is this error could be related to the limited opa policies and auth policies in place for this specific version of MAGDA and that I should migrate to a newer version of the release but not to sure the best way to go about this using the Magda-config repository.

Any recommendations or help greatly appreciated!

[t83714]

Hi @BrandonFeldhaus,
Noticed you also attached the X-Magda-Session (JWT token) header.
In fact, if we use the API key already, we don't need the X-Magda-Session header.
The API key option is actually a better option if we talk to Magda externally (outside the cluster) and X-Magda-Session is mainly used internally (inside the cluster behind the Magda gateway module).
Could you please drop the X-Magda-Session header and try again?
By the way, not sure whether you come across our Magda tutorial repo. It might cover a similar topic.
https://github.com/magda-io/magda-brown-bag

[BrandonFeldhaus]

Hi @t83714

I have dropped the X-Magda-Session header and still receive the same error of "Unathorized" with 401 error code.

After reviewing the documentation on the https://github.com/magda-io/magda-brown-bag repository, I performed the JSON-PATCH via curl, without the X-Magda-Session header and still receive the same response.

Request

curl --location --request PUT 'http://dn.datacube.online/api/v0/registry/aspects/project-open-data-dataset' \
--header 'Content-Type: application/json' \
--header 'X-Magda-API-Key-Id:<API KEY ID>' \
--header 'X-Magda-API-Key: <API KEY> \
--data-raw '{
    "$schema": "http://json-schema.org/schema#",
    "title": "Aspect capturing a complete representation of a dataset according to Project Open Data (data.json).",
    "type": "object",
    "allOf": [
        {
            "$ref": "https://raw.githubusercontent.com/BrandonFeldhaus/project-open-data-dashboard/main/schema/non-federal-v1.1/dataset-non-federal.json"
        }
    ]
}'

Response

{"isError":true,"errorCode":401,"errorMessage":"Unauthorized"}

My current plan/thinking is to create a another set of API Keys to re-verify the issue. If that does not resolve the issue, I will test with the Magda Brown Bag repository as it seems the helm chart version for Magda is 0.0.60-rc.2 while for Magda-config it is 0.0.59.

Will continue to update this comment with progress throughout the coming weeks.

[t83714]

Hi @BrandonFeldhaus,
All api keys are tied to a user. You probably can check whether the user has the key by:

yarn create-api-key -u [user id] -l

And also make sure the user has admin role:

yarn acs-cmd list users

If not, set a user as admin by:

yarn acs-cmd admin set [user id]

[BrandonFeldhaus]

Apologies for the late reply, this issue got backlogged due to other priorities.

I was able to finally resolve issues with the create-api-key script (these issues were M1 Mac specific on npm library Bcrypt, this stack overflow comment resolved my issue) and create another set of API Keys that would not return an Unauthorized error.

Once I passed the Unauthorized error, I found errors in the request I was making above. I was improperly formatting the data to send. Correctly formatted data is as follows:

var XMLHttpRequest = require("xmlhttprequest").XMLHttpRequest;

var url = "http://dn.datacube.online/api/v0/registry/aspects/project-open-data-dataset";

var xhr = new XMLHttpRequest();
xhr.open("PUT", url);

xhr.setRequestHeader("X-Magda-API-Key-Id", "<API-KEY-ID>");
xhr.setRequestHeader("X-Magda-Tenant-Id", 0);
xhr.setRequestHeader("X-Magda-API-Key", "<API-KEY>");
xhr.setRequestHeader("Content-Type", "application/json");

xhr.onreadystatechange = function () {
   if (xhr.readyState === 4) {
      console.log(xhr.status);
      console.log(xhr.responseText);
   }};

var data = `{
    "id":"project-open-data-dataset",
    "name": "Project Open Data (data.json) Dataset",
    "jsonSchema":{
        "$schema": "http://json-schema.org/schema#",
        "title": "Aspect capturing a complete representation of a distribution according to Project Open Data (data.json).",
        "type": "object",
        "allOf": [
            {
                "$ref": "https://raw.githubusercontent.com/BrandonFeldhaus/project-open-data-dashboard/main/schema/non-federal-v1.1/dataset.json"
            }
        ]
    }

}`;

xhr.send(data);

[t83714]

Hi @BrandonFeldhaus Thanks for sharing your progress.
If you are on M1, many npm packages (e.g. node-sass) would require binary build as there won't be the pre-built versions available.
For the binary build, node-gyp often requires python2 & 3.
On Mac, I often use homebrew to install pyenv and then install multiple versions of python.
By the way, if you request outside the cluster with API keys, X-Magda-Tenant-Id is not compulsory and will be auto-generated by the gateway before forwarding to other microservices.

[BrandonFeldhaus]

That is very helpful information, the node-gyp package did require some work arounds to finally get it functioning.

Thank you for additionally information and help!