Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: azure byoc #11

Merged
merged 2 commits into from
Oct 25, 2024
Merged

feat: azure byoc #11

merged 2 commits into from
Oct 25, 2024

Conversation

prajjwaldimri
Copy link
Collaborator

No description provided.

@prajjwaldimri prajjwaldimri added the enhancement New feature or request label Oct 23, 2024
@prajjwaldimri prajjwaldimri self-assigned this Oct 23, 2024
Comment on lines +450 to +464
cacheKey = await cacheHttpClient.saveCache(
'azure_blob',
key,
cacheVersion,
archivePath,
// S3 Params are undefined for GCS
undefined,
undefined,
undefined,
undefined,
// GCS Params are undefined for Azure Blob
undefined,
undefined,
undefined,
reserveCacheResponse?.result?.azure_blob?.pre_signed_url ?? ''
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this not an object?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not important to change now but this looks like a low hanging fruit to improve safety.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup. This needs to be fixed.

core.debug('Uploading cache')
const blockBlobClient = new BlockBlobClient(AzureSASURL)
await blockBlobClient.uploadFile(archivePath, {
maxSingleShotSize: 10 * 1024 * 1024 // 10 MB
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is 10 MB optimal?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we increase or leave the default?

If the size of the data is less than or equal to this value, it's uploaded in a single put rather than broken up into chunks. If the data is uploaded in a single shot, the block size is ignored. Default value is 256 MiB.

From https://learn.microsoft.com/en-us/azure/storage/blobs/storage-blob-upload-javascript

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Default value for parallel uploads is way higher than what we use across other providers. That's why it is overridden here.

Copy link
Member

@abhijit-hota abhijit-hota left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Few questions/comments. No specific changes. LGTM.

@prajjwaldimri prajjwaldimri merged commit 2409210 into main Oct 25, 2024
9 of 12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants