feat(marketplace-cli): Create marketplace-cli subcommand to generate CSV files from YAML files #470
Conversation
|
Important This PR includes changes that affect public-facing API. Please ensure you are adding/updating documentation for new features or behavior. Changed Packages
|
|
WIP PR to implement the CSV subcommand. @christoph-jerolimov am I on the right track/reading from the right files? |
logonoff
force-pushed
the
csv
branch
5 times, most recently
from
cb2e11c to
cc183dd
Compare
Yeah that's a really good direction. 👍 Please extend this so that you read all YAMLs from theplugins and packages folder and see if we can have a combination of both. We don't know the exact data we want export, but it would be great if you can show for each plugin (contains the most metadata) show the related frontend and backend package. Incl. the npm package name and version. These are linked via their backstage catalog entity names. |
logonoff
force-pushed
the
csv
branch
2 times, most recently
from
52c0a29 to
c6cf831
Compare
logonoff
requested review from
christoph-jerolimov,
ciiay,
debsmita1,
divyanshiGupta,
gashcrumb,
its-mitesh-kumar and
karthikjeeyar
as code owners
…CSV files from YAML files implements https://issues.redhat.com/browse/RHIDP-6231
|
There was a problem hiding this comment.
Hi @logonoff, good work so far. I hope the comments below helps:
| */ | ||
| key: string; | ||
| /** The getter function for each cell. The return value must be castable to a string */ | ||
| getter: (plugin: T) => any; |
There was a problem hiding this comment.
tiny: if this CSVGenerator is generic, try to avoid "plugin" code or variables here.
I would also recommend to avoid any. Can it maybe return a string | number?
| const plugins = fileContent | ||
| .split('\n---\n') | ||
| .filter(Boolean) | ||
| .map((plugin: string) => { | ||
| return YAML.parse(plugin); | ||
| }) | ||
| .filter(isEntity); | ||
| return plugins; |
There was a problem hiding this comment.
Hey, the YAML library supports parsing multiple docs via YAML.parseAllDocuments. Each document should have something like toJSON.
| const plugins = fileContent | |
| .split('\n---\n') | |
| .filter(Boolean) | |
| .map((plugin: string) => { | |
| return YAML.parse(plugin); | |
| }) | |
| .filter(isEntity); | |
| return plugins; | |
| const plugins = YAML.parseAllDocuments(fileContent) | |
| .map((doc => doc.toJSON()) | |
| .filter(isMarketplacePlugin) | |
| return plugins; |
| * Fetches a `package.json` from a given package from the npm registry. | ||
| * @param packageName The name of the package to fetch | ||
| * @returns The package.json object | ||
| */ | ||
| const fetchNpmPackageJson = async (packageName: string, version: string) => { | ||
| const response = await fetch( | ||
| `https://registry.npmjs.org/${packageName}/${version}`, | ||
| ); | ||
|
|
||
| return await response.json(); | ||
| }; |
There was a problem hiding this comment.
This is a "nice idea" but there is no need to fetch external information. All information should be part of the Plugin or Package YAML.
Esp. the descriptions and version strings. You can drop that code.
| type CombinedPackage = { | ||
| yaml: MarketplacePackage; | ||
| pkg: PackageJson; | ||
| }; |
There was a problem hiding this comment.
The idea is good, but could be something like this:
| type CombinedPackage = { | |
| yaml: MarketplacePackage; | |
| pkg: PackageJson; | |
| }; | |
| type CombinedPackage = { | |
| plugin: MarketplacePlugin; | |
| frontendPackage?: MarketplacePackage; | |
| backendPackage?: MarketplacePackage; | |
| otherPackages?: MarketplcaePackage[]; | |
| }; |
The plugin and packages YAMLs contains many information and you should parse them, find the relationships and render "grouped information". The main entity == one row should be the plugin.
But a plugin can link zero, one or more packages.
For example:
https://github.com/redhat-developer/rhdh/blob/main/catalog-entities/marketplace/plugins/kubernetes.yaml#L91-L93
https://github.com/redhat-developer/rhdh/blob/main/catalog-entities/marketplace/packages/backstage-plugin-kubernetes.yaml#L4
https://github.com/redhat-developer/rhdh/blob/main/catalog-entities/marketplace/packages/backstage-plugin-kubernetes-backend.yaml#L4
So the CLI should load all plugins and all packages. At the end, you should iterate over all plugins and try to find related Packages.
The spec.backstage.role identifies if the Package is a a frontend or backend plugin:
https://github.com/redhat-developer/rhdh/blob/main/catalog-entities/marketplace/packages/backstage-plugin-kubernetes.yaml#L22-L23
https://github.com/redhat-developer/rhdh/blob/main/catalog-entities/marketplace/packages/backstage-plugin-kubernetes-backend.yaml#L22-L23
And then you can display in the CSV the Plugin name, description (and so one), and the frontend and backend package version.
Does this make sense to you?
| 'name', | ||
| 'title', | ||
| 'description', | ||
| 'developer', |
There was a problem hiding this comment.
We used author and developer in some YAMLs. The final version uses "author"
| 'developer', | |
| 'author', |
| 'developer', | ||
| 'categories', | ||
| 'lifecycle', | ||
| 'icon', |
There was a problem hiding this comment.
We used "icon" with an URL while developing the plugin. In the current version we use inline base64 icons. I guess we should skip this information until we have another soliution.
| 'icon', |
implements https://issues.redhat.com/browse/RHIDP-6231
Hey, I just made a Pull Request!
✔️ Checklist