Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Blob copy operation succeed without correct permission on source blob. #2317

Closed
arnabfromjec05 opened this issue Dec 8, 2023 · 7 comments
Closed

Blob copy operation succeed without correct permission on source blob. #2317

arnabfromjec05 opened this issue Dec 8, 2023 · 7 comments
Assignees
@EmmaZhu
Labels
blob-storage bug Something isn't working

Comments

@arnabfromjec05
Copy link

arnabfromjec05 commented Dec 8, 2023 •
edited
Loading

Which service(blob, file, queue, table) does this issue concern?

blob

Which version of the Azurite was used?

3.28.0

Where do you get Azurite? (npm, DockerHub, NuGet, Visual Studio Code Extension)

npm

What's the Node.js version?

v16.19.0

What problem was encountered?

Writing an UT where blob copy operation throws error due to source blob sas uri doesn't have valid permission. Provided "Create" and "Delete" permission on the source blob turn by turn with a given value of destination blob sas uri, which is expected to fail the blob copy operation.
It is throwing error correctly in case of any production blobs, storage emulator but is silently passing in case of azurite.

Storage emulator and Production blobs throwing error (expected):
This request is not authorized to perform this operation using this permission. RequestId: <redacted> Time:2023-12-08T14:08:08.1345700Z Status: 403 (This request is not authorized to perform this operation using this permission.) ErrorCode: CannotVerifyCopySource.

Azurite (not-expected):
Succeding silently.

Steps to reproduce the issue?

If possible, please provide the debug log using the -d parameter, replacing <pathtodebuglog> with an appropriate path for your OS, or review the instructions for docker containers:

-d "<pathtodebuglog>"

Please be sure to remove any PII or sensitive information before sharing!
The debug log will log raw request headers and bodies, so that we can replay these against Azurite using REST and create tests to validate resolution.

Logs:

2023-12-11T10:26:08.092Z bd46ae63-23c1-4b7d-b19e-58c800ed7ee6 info: BlobStorageContextMiddleware: RequestMethod=HEAD RequestURL=http://127.0.0.1/devstoreaccount1/f8cbdc53-77cc-4e21-a4ef-cc45ccc2b05a/destination.txt RequestHeaders:{"host":"127.0.0.1:10000","x-ms-version":"2020-04-08","x-ms-client-request-id":"751da309-6e84-46c3-8c34-85787dc9569e","x-ms-return-client-request-id":"true","user-agent":"azsdk-net-Storage.Blobs/12.8.3 (.NET 6.0.25; Microsoft Windows 10.0.22621)","x-ms-date":"Mon, 11 Dec 2023 10:26:08 GMT","authorization":"SharedKey devstoreaccount1:{redacted}"} ClientIP=127.0.0.1 Protocol=http HTTPVersion=1.1
2023-12-11T10:26:08.092Z bd46ae63-23c1-4b7d-b19e-58c800ed7ee6 info: BlobStorageContextMiddleware: Account=devstoreaccount1 Container=f8cbdc53-77cc-4e21-a4ef-cc45ccc2b05a Blob=destination.txt
2023-12-11T10:26:08.092Z bd46ae63-23c1-4b7d-b19e-58c800ed7ee6 verbose: DispatchMiddleware: Dispatching request...
2023-12-11T10:26:08.093Z bd46ae63-23c1-4b7d-b19e-58c800ed7ee6 info: DispatchMiddleware: Operation=Blob_GetProperties
2023-12-11T10:26:08.093Z bd46ae63-23c1-4b7d-b19e-58c800ed7ee6 verbose: AuthenticationMiddlewareFactory:createAuthenticationMiddleware() Validating authentications.
2023-12-11T10:26:08.093Z bd46ae63-23c1-4b7d-b19e-58c800ed7ee6 info: PublicAccessAuthenticator:validate() Start validation against public access.
2023-12-11T10:26:08.093Z bd46ae63-23c1-4b7d-b19e-58c800ed7ee6 debug: PublicAccessAuthenticator:validate() Getting account properties...
2023-12-11T10:26:08.093Z bd46ae63-23c1-4b7d-b19e-58c800ed7ee6 debug: PublicAccessAuthenticator:validate() Retrieved account name from context: devstoreaccount1, container: f8cbdc53-77cc-4e21-a4ef-cc45ccc2b05a, blob: destination.txt
2023-12-11T10:26:08.094Z bd46ae63-23c1-4b7d-b19e-58c800ed7ee6 debug: PublicAccessAuthenticator:validate() Skip public access authentication. Cannot get public access type for container f8cbdc53-77cc-4e21-a4ef-cc45ccc2b05a
2023-12-11T10:26:08.094Z bd46ae63-23c1-4b7d-b19e-58c800ed7ee6 info: BlobSharedKeyAuthenticator:validate() Start validation against account shared key authentication.
2023-12-11T10:26:08.094Z bd46ae63-23c1-4b7d-b19e-58c800ed7ee6 info: BlobSharedKeyAuthenticator:validate() [STRING TO SIGN]:"HEAD\n\n\n\n\n\n\n\n\n\n\n\nx-ms-client-request-id:751da309-6e84-46c3-8c34-85787dc9569e\nx-ms-date:Mon, 11 Dec 2023 10:26:08 GMT\nx-ms-return-client-request-id:true\nx-ms-version:2020-04-08\n/devstoreaccount1/devstoreaccount1/f8cbdc53-77cc-4e21-a4ef-cc45ccc2b05a/destination.txt"
2023-12-11T10:26:08.094Z bd46ae63-23c1-4b7d-b19e-58c800ed7ee6 info: BlobSharedKeyAuthenticator:validate() Calculated authentication header based on key1: SharedKey devstoreaccount1:{redacted}
2023-12-11T10:26:08.094Z bd46ae63-23c1-4b7d-b19e-58c800ed7ee6 info: BlobSharedKeyAuthenticator:validate() Signature 1 matched.
2023-12-11T10:26:08.094Z bd46ae63-23c1-4b7d-b19e-58c800ed7ee6 verbose: DeserializerMiddleware: Start deserializing...
2023-12-11T10:26:08.094Z bd46ae63-23c1-4b7d-b19e-58c800ed7ee6 info: HandlerMiddleware: DeserializedParameters={"options":{"requestId":"751da309-6e84-46c3-8c34-85787dc9569e","leaseAccessConditions":{},"cpkInfo":{},"modifiedAccessConditions":{}},"version":"2020-04-08"}
2023-12-11T10:26:08.095Z bd46ae63-23c1-4b7d-b19e-58c800ed7ee6 verbose: SerializerMiddleware: Start serializing...
2023-12-11T10:26:08.095Z bd46ae63-23c1-4b7d-b19e-58c800ed7ee6 info: EndMiddleware: End response. TotalTimeInMS=3 StatusCode=200 StatusMessage=undefined Headers={"server":"Azurite-Blob/3.28.0","last-modified":"Mon, 11 Dec 2023 10:25:15 GMT","x-ms-creation-time":"Mon, 11 Dec 2023 10:25:15 GMT","x-ms-blob-type":"BlockBlob","x-ms-copy-completion-time":"Mon, 11 Dec 2023 10:25:15 GMT","x-ms-copy-id":"b525ecb1-2f5a-4c87-ad14-a49689d67ff7","x-ms-copy-progress":"26/26","x-ms-copy-source":"http://127.0.0.1:10000/devstoreaccount1/f8cbdc53-77cc-4e21-a4ef-cc45ccc2b05a/source.txt?sv=2020-04-08&se=2023-12-12T10%3A21%3A46Z&sr=c&sp=c&sig={redacted}","x-ms-copy-status":"success","x-ms-incremental-copy":"false","x-ms-lease-state":"available","x-ms-lease-status":"unlocked","content-length":"26","content-type":"application/octet-stream","etag":"\"0x236C0B7C3E01540\"","content-md5":"w/zT12GS5AB9+0lsymfhOw==","x-ms-client-request-id":"751da309-6e84-46c3-8c34-85787dc9569e","x-ms-request-id":"bd46ae63-23c1-4b7d-b19e-58c800ed7ee6","x-ms-version":"2023-11-03","date":"Mon, 11 Dec 2023 10:26:08 GMT","accept-ranges":"bytes","x-ms-server-encrypted":"true","x-ms-access-tier":"Hot","x-ms-access-tier-inferred":"true"}
2023-12-11T10:26:09.377Z 14874c5d-3e63-4ee0-a06b-6e55141aa985 info: BlobStorageContextMiddleware: RequestMethod=DELETE RequestURL=http://127.0.0.1/devstoreaccount1/f8cbdc53-77cc-4e21-a4ef-cc45ccc2b05a/destination.txt RequestHeaders:{"host":"127.0.0.1:10000","x-ms-version":"2020-04-08","x-ms-client-request-id":"b0859b23-75da-45a2-84ff-f02b5d8db74b","x-ms-return-client-request-id":"true","user-agent":"azsdk-net-Storage.Blobs/12.8.3 (.NET 6.0.25; Microsoft Windows 10.0.22621)","x-ms-date":"Mon, 11 Dec 2023 10:26:09 GMT","authorization":"SharedKey devstoreaccount1:{redacted}"} ClientIP=127.0.0.1 Protocol=http HTTPVersion=1.1
2023-12-11T10:26:09.377Z 14874c5d-3e63-4ee0-a06b-6e55141aa985 info: BlobStorageContextMiddleware: Account=devstoreaccount1 Container=f8cbdc53-77cc-4e21-a4ef-cc45ccc2b05a Blob=destination.txt
2023-12-11T10:26:09.377Z 14874c5d-3e63-4ee0-a06b-6e55141aa985 verbose: DispatchMiddleware: Dispatching request...
2023-12-11T10:26:09.377Z 14874c5d-3e63-4ee0-a06b-6e55141aa985 info: DispatchMiddleware: Operation=Blob_Delete
2023-12-11T10:26:09.378Z 14874c5d-3e63-4ee0-a06b-6e55141aa985 verbose: AuthenticationMiddlewareFactory:createAuthenticationMiddleware() Validating authentications.
2023-12-11T10:26:09.378Z 14874c5d-3e63-4ee0-a06b-6e55141aa985 info: PublicAccessAuthenticator:validate() Start validation against public access.
2023-12-11T10:26:09.378Z 14874c5d-3e63-4ee0-a06b-6e55141aa985 debug: PublicAccessAuthenticator:validate() Getting account properties...
2023-12-11T10:26:09.378Z 14874c5d-3e63-4ee0-a06b-6e55141aa985 debug: PublicAccessAuthenticator:validate() Retrieved account name from context: devstoreaccount1, container: f8cbdc53-77cc-4e21-a4ef-cc45ccc2b05a, blob: destination.txt
2023-12-11T10:26:09.378Z 14874c5d-3e63-4ee0-a06b-6e55141aa985 debug: PublicAccessAuthenticator:validate() Skip public access authentication. Cannot get public access type for container f8cbdc53-77cc-4e21-a4ef-cc45ccc2b05a
2023-12-11T10:26:09.378Z 14874c5d-3e63-4ee0-a06b-6e55141aa985 info: BlobSharedKeyAuthenticator:validate() Start validation against account shared key authentication.
2023-12-11T10:26:09.378Z 14874c5d-3e63-4ee0-a06b-6e55141aa985 info: BlobSharedKeyAuthenticator:validate() [STRING TO SIGN]:"DELETE\n\n\n\n\n\n\n\n\n\n\n\nx-ms-client-request-id:b0859b23-75da-45a2-84ff-f02b5d8db74b\nx-ms-date:Mon, 11 Dec 2023 10:26:09 GMT\nx-ms-return-client-request-id:true\nx-ms-version:2020-04-08\n/devstoreaccount1/devstoreaccount1/f8cbdc53-77cc-4e21-a4ef-cc45ccc2b05a/destination.txt"
2023-12-11T10:26:09.378Z 14874c5d-3e63-4ee0-a06b-6e55141aa985 info: BlobSharedKeyAuthenticator:validate() Calculated authentication header based on key1: SharedKey devstoreaccount1:{redacted}
2023-12-11T10:26:09.378Z 14874c5d-3e63-4ee0-a06b-6e55141aa985 info: BlobSharedKeyAuthenticator:validate() Signature 1 matched.
2023-12-11T10:26:09.379Z 14874c5d-3e63-4ee0-a06b-6e55141aa985 verbose: DeserializerMiddleware: Start deserializing...
2023-12-11T10:26:09.379Z 14874c5d-3e63-4ee0-a06b-6e55141aa985 info: HandlerMiddleware: DeserializedParameters={"options":{"requestId":"b0859b23-75da-45a2-84ff-f02b5d8db74b","leaseAccessConditions":{},"modifiedAccessConditions":{}},"version":"2020-04-08"}
2023-12-11T10:26:09.380Z 14874c5d-3e63-4ee0-a06b-6e55141aa985 verbose: SerializerMiddleware: Start serializing...
2023-12-11T10:26:09.381Z 14874c5d-3e63-4ee0-a06b-6e55141aa985 info: EndMiddleware: End response. TotalTimeInMS=4 StatusCode=202 StatusMessage=undefined Headers={"server":"Azurite-Blob/3.28.0","x-ms-client-request-id":"b0859b23-75da-45a2-84ff-f02b5d8db74b","x-ms-request-id":"14874c5d-3e63-4ee0-a06b-6e55141aa985","x-ms-version":"2023-11-03","date":"Mon, 11 Dec 2023 10:26:09 GMT","x-ms-delete-type-permanent":"true"}
2023-12-11T10:26:10.940Z d47732c8-cdca-42d5-aee2-18ebd4cd96a6 info: BlobStorageContextMiddleware: RequestMethod=HEAD RequestURL=http://127.0.0.1/devstoreaccount1/f8cbdc53-77cc-4e21-a4ef-cc45ccc2b05a/destination.txt?sv=2020-04-08&se=2023-12-12T10%3A21%3A45Z&sr=c&sp=rc&sig={redacted} RequestHeaders:{"host":"127.0.0.1:10000","x-ms-version":"2020-04-08","x-ms-client-request-id":"aaf2e119-5fee-4292-8081-9892c19d31f6","x-ms-return-client-request-id":"true","user-agent":"azsdk-net-Storage.Blobs/12.8.3 (.NET 6.0.25; Microsoft Windows 10.0.22621)"} ClientIP=127.0.0.1 Protocol=http HTTPVersion=1.1
2023-12-11T10:26:10.941Z d47732c8-cdca-42d5-aee2-18ebd4cd96a6 info: BlobStorageContextMiddleware: Account=devstoreaccount1 Container=f8cbdc53-77cc-4e21-a4ef-cc45ccc2b05a Blob=destination.txt
2023-12-11T10:26:10.941Z d47732c8-cdca-42d5-aee2-18ebd4cd96a6 verbose: DispatchMiddleware: Dispatching request...
2023-12-11T10:26:10.941Z d47732c8-cdca-42d5-aee2-18ebd4cd96a6 info: DispatchMiddleware: Operation=Blob_GetProperties
2023-12-11T10:26:10.942Z d47732c8-cdca-42d5-aee2-18ebd4cd96a6 verbose: AuthenticationMiddlewareFactory:createAuthenticationMiddleware() Validating authentications.
2023-12-11T10:26:10.942Z d47732c8-cdca-42d5-aee2-18ebd4cd96a6 info: PublicAccessAuthenticator:validate() Start validation against public access.
2023-12-11T10:26:10.942Z d47732c8-cdca-42d5-aee2-18ebd4cd96a6 debug: PublicAccessAuthenticator:validate() Getting account properties...
2023-12-11T10:26:10.942Z d47732c8-cdca-42d5-aee2-18ebd4cd96a6 debug: PublicAccessAuthenticator:validate() Retrieved account name from context: devstoreaccount1, container: f8cbdc53-77cc-4e21-a4ef-cc45ccc2b05a, blob: destination.txt
2023-12-11T10:26:10.942Z d47732c8-cdca-42d5-aee2-18ebd4cd96a6 debug: PublicAccessAuthenticator:validate() Skip public access authentication. Cannot get public access type for container f8cbdc53-77cc-4e21-a4ef-cc45ccc2b05a
2023-12-11T10:26:10.942Z d47732c8-cdca-42d5-aee2-18ebd4cd96a6 info: BlobSharedKeyAuthenticator:validate() Start validation against account shared key authentication.
2023-12-11T10:26:10.942Z d47732c8-cdca-42d5-aee2-18ebd4cd96a6 info: BlobSharedKeyAuthenticator:validate() Request doesn't include valid authentication header. Skip shared key authentication.
2023-12-11T10:26:10.943Z d47732c8-cdca-42d5-aee2-18ebd4cd96a6 info: AccountSASAuthenticator:validate() Start validation against account Shared Access Signature pattern.
2023-12-11T10:26:10.943Z d47732c8-cdca-42d5-aee2-18ebd4cd96a6 debug: AccountSASAuthenticator:validate() Getting account properties...
2023-12-11T10:26:10.943Z d47732c8-cdca-42d5-aee2-18ebd4cd96a6 debug: AccountSASAuthenticator:validate() Retrieved account name from context: devstoreaccount1, container: f8cbdc53-77cc-4e21-a4ef-cc45ccc2b05a, blob: destination.txt
2023-12-11T10:26:10.943Z d47732c8-cdca-42d5-aee2-18ebd4cd96a6 debug: AccountSASAuthenticator:validate() Got account properties successfully.
2023-12-11T10:26:10.943Z d47732c8-cdca-42d5-aee2-18ebd4cd96a6 debug: AccountSASAuthenticator:validate() Retrieved signature from URL parameter sig: {redacted}
2023-12-11T10:26:10.943Z d47732c8-cdca-42d5-aee2-18ebd4cd96a6 info: AccountSASAuthenticator:validate() Failed to get valid account SAS values from request.
2023-12-11T10:26:10.943Z d47732c8-cdca-42d5-aee2-18ebd4cd96a6 info: BlobSASAuthenticator:validate() Start validation against blob service Shared Access Signature pattern.
2023-12-11T10:26:10.943Z d47732c8-cdca-42d5-aee2-18ebd4cd96a6 debug: BlobSASAuthenticator:validate() Getting account properties...
2023-12-11T10:26:10.943Z d47732c8-cdca-42d5-aee2-18ebd4cd96a6 debug: BlobSASAuthenticator:validate() Retrieved account name from context: devstoreaccount1, container: f8cbdc53-77cc-4e21-a4ef-cc45ccc2b05a, blob: destination.txt
2023-12-11T10:26:10.943Z d47732c8-cdca-42d5-aee2-18ebd4cd96a6 debug: BlobSASAuthenticator:validate() Got account properties successfully.
2023-12-11T10:26:10.943Z d47732c8-cdca-42d5-aee2-18ebd4cd96a6 debug: BlobSASAuthenticator:validate() Retrieved signature from URL parameter sig: {redacted}
2023-12-11T10:26:10.943Z d47732c8-cdca-42d5-aee2-18ebd4cd96a6 debug: BlobSASAuthenticator:validate() Signed resource type is c.
2023-12-11T10:26:10.943Z d47732c8-cdca-42d5-aee2-18ebd4cd96a6 debug: BlobSASAuthenticator:validate() Successfully got valid blob service SAS values from request. {"version":"2020-04-08","expiryTime":"2023-12-12T10:21:45Z","permissions":"rc","containerName":"f8cbdc53-77cc-4e21-a4ef-cc45ccc2b05a","blobName":"destination.txt","signedResource":"c"}
2023-12-11T10:26:10.943Z d47732c8-cdca-42d5-aee2-18ebd4cd96a6 info: BlobSASAuthenticator:validate() Validate signature based account key1.
2023-12-11T10:26:10.944Z d47732c8-cdca-42d5-aee2-18ebd4cd96a6 debug: BlobSASAuthenticator:validate() String to sign is: "rc\n\n2023-12-12T10:21:45Z\n/blob/devstoreaccount1/f8cbdc53-77cc-4e21-a4ef-cc45ccc2b05a\n\n\n\n2020-04-08\nc\n\n\n\n\n\n"
2023-12-11T10:26:10.944Z d47732c8-cdca-42d5-aee2-18ebd4cd96a6 debug: BlobSASAuthenticator:validate() Calculated signature is: {redacted}
2023-12-11T10:26:10.944Z d47732c8-cdca-42d5-aee2-18ebd4cd96a6 info: BlobSASAuthenticator:validate() Signature based on key1 validation passed.
2023-12-11T10:26:10.944Z d47732c8-cdca-42d5-aee2-18ebd4cd96a6 info: BlobSASAuthenticator:validate() Validate start and expiry time.
2023-12-11T10:26:10.944Z d47732c8-cdca-42d5-aee2-18ebd4cd96a6 info: BlobSASAuthenticator:validate() Validate IP range.
2023-12-11T10:26:10.945Z d47732c8-cdca-42d5-aee2-18ebd4cd96a6 info: BlobSASAuthenticator:validate() Validate request protocol.
2023-12-11T10:26:10.946Z d47732c8-cdca-42d5-aee2-18ebd4cd96a6 debug: BlobSASAuthenticator:validate() Got permission requirements for operation Blob_GetProperties - {"permission":"r"}
2023-12-11T10:26:10.946Z d47732c8-cdca-42d5-aee2-18ebd4cd96a6 info: BlobSASAuthenticator:validate() Blob service SAS validation successfully.
2023-12-11T10:26:10.946Z d47732c8-cdca-42d5-aee2-18ebd4cd96a6 verbose: DeserializerMiddleware: Start deserializing...
2023-12-11T10:26:10.946Z d47732c8-cdca-42d5-aee2-18ebd4cd96a6 info: HandlerMiddleware: DeserializedParameters={"options":{"requestId":"aaf2e119-5fee-4292-8081-9892c19d31f6","leaseAccessConditions":{},"cpkInfo":{},"modifiedAccessConditions":{}},"version":"2020-04-08"}
2023-12-11T10:26:10.947Z d47732c8-cdca-42d5-aee2-18ebd4cd96a6 error: ErrorMiddleware: Received a MiddlewareError, fill error information to HTTP response
2023-12-11T10:26:10.948Z d47732c8-cdca-42d5-aee2-18ebd4cd96a6 error: ErrorMiddleware: ErrorName=StorageError ErrorMessage=The specified blob does not exist. ErrorHTTPStatusCode=404 ErrorHTTPStatusMessage=The specified blob does not exist. ErrorHTTPHeaders={"x-ms-error-code":"BlobNotFound","x-ms-request-id":"d47732c8-cdca-42d5-aee2-18ebd4cd96a6"} ErrorHTTPBody="\n\n BlobNotFound\n The specified blob does not exist.\nRequestId:d47732c8-cdca-42d5-aee2-18ebd4cd96a6\nTime:2023-12-11T10:26:10.946Z\n" ErrorStack="StorageError: The specified blob does not exist.\n at Function.getBlobNotFound (C:\Users\arnabjana\AppData\Roaming\nvm\v16.19.0\node_modules\azurite\dist\src\blob\errors\StorageErrorFactory.js:29:16)\n at LokiBlobMetadataStore.getBlobProperties (C:\Users\arnabjana\AppData\Roaming\nvm\v16.19.0\node_modules\azurite\dist\src\blob\persistence\LokiBlobMetadataStore.js:810:49)\n at processTicksAndRejections (node:internal/process/task_queues:96:5)\n at async BlobHandler.getProperties (C:\Users\arnabjana\AppData\Roaming\nvm\v16.19.0\node_modules\azurite\dist\src\blob\handlers\BlobHandler.js:71:21)"
2023-12-11T10:26:10.948Z d47732c8-cdca-42d5-aee2-18ebd4cd96a6 error: ErrorMiddleware: Set HTTP code: 404
2023-12-11T10:26:10.948Z d47732c8-cdca-42d5-aee2-18ebd4cd96a6 error: ErrorMiddleware: Set HTTP status message: The specified blob does not exist.
2023-12-11T10:26:10.948Z d47732c8-cdca-42d5-aee2-18ebd4cd96a6 error: ErrorMiddleware: Set HTTP Header: x-ms-error-code=BlobNotFound
2023-12-11T10:26:10.948Z d47732c8-cdca-42d5-aee2-18ebd4cd96a6 error: ErrorMiddleware: Set HTTP Header: x-ms-request-id=d47732c8-cdca-42d5-aee2-18ebd4cd96a6
2023-12-11T10:26:10.948Z d47732c8-cdca-42d5-aee2-18ebd4cd96a6 error: ErrorMiddleware: Set HTTP body: "\n\n BlobNotFound\n The specified blob does not exist.\nRequestId:d47732c8-cdca-42d5-aee2-18ebd4cd96a6\nTime:2023-12-11T10:26:10.946Z\n"
2023-12-11T10:26:10.948Z d47732c8-cdca-42d5-aee2-18ebd4cd96a6 info: EndMiddleware: End response. TotalTimeInMS=8 StatusCode=404 StatusMessage=The specified blob does not exist. Headers={"server":"Azurite-Blob/3.28.0","x-ms-error-code":"BlobNotFound","x-ms-request-id":"d47732c8-cdca-42d5-aee2-18ebd4cd96a6"}
2023-12-11T10:26:14.246Z bcc3c56d-8e5d-4046-b599-4da938c54e36 info: BlobStorageContextMiddleware: RequestMethod=PUT RequestURL=http://127.0.0.1/devstoreaccount1/f8cbdc53-77cc-4e21-a4ef-cc45ccc2b05a/destination.txt?sv=2020-04-08&se=2023-12-12T10%3A21%3A45Z&sr=c&sp=rc&sig={redacted} RequestHeaders:{"host":"127.0.0.1:10000","x-ms-copy-source":"http://127.0.0.1:10000/devstoreaccount1/f8cbdc53-77cc-4e21-a4ef-cc45ccc2b05a/source.txt?sv=2020-04-08&se=2023-12-12T10%3A26%3A09Z&sr=c&sp=d&sig={redacted}","x-ms-version":"2020-04-08","x-ms-client-request-id":"abeef25f-8698-4b94-8749-23241d5d2768","x-ms-return-client-request-id":"true","user-agent":"azsdk-net-Storage.Blobs/12.8.3 (.NET 6.0.25; Microsoft Windows 10.0.22621)","content-length":"0"} ClientIP=127.0.0.1 Protocol=http HTTPVersion=1.1
2023-12-11T10:26:14.246Z bcc3c56d-8e5d-4046-b599-4da938c54e36 info: BlobStorageContextMiddleware: Account=devstoreaccount1 Container=f8cbdc53-77cc-4e21-a4ef-cc45ccc2b05a Blob=destination.txt
2023-12-11T10:26:14.246Z bcc3c56d-8e5d-4046-b599-4da938c54e36 verbose: DispatchMiddleware: Dispatching request...
2023-12-11T10:26:14.248Z bcc3c56d-8e5d-4046-b599-4da938c54e36 info: DispatchMiddleware: Operation=Blob_StartCopyFromURL
2023-12-11T10:26:14.249Z bcc3c56d-8e5d-4046-b599-4da938c54e36 verbose: AuthenticationMiddlewareFactory:createAuthenticationMiddleware() Validating authentications.
2023-12-11T10:26:14.249Z bcc3c56d-8e5d-4046-b599-4da938c54e36 info: PublicAccessAuthenticator:validate() Start validation against public access.
2023-12-11T10:26:14.249Z bcc3c56d-8e5d-4046-b599-4da938c54e36 debug: PublicAccessAuthenticator:validate() Getting account properties...
2023-12-11T10:26:14.249Z bcc3c56d-8e5d-4046-b599-4da938c54e36 debug: PublicAccessAuthenticator:validate() Retrieved account name from context: devstoreaccount1, container: f8cbdc53-77cc-4e21-a4ef-cc45ccc2b05a, blob: destination.txt
2023-12-11T10:26:14.249Z bcc3c56d-8e5d-4046-b599-4da938c54e36 debug: PublicAccessAuthenticator:validate() Skip public access authentication. Cannot get public access type for container f8cbdc53-77cc-4e21-a4ef-cc45ccc2b05a
2023-12-11T10:26:14.249Z bcc3c56d-8e5d-4046-b599-4da938c54e36 info: BlobSharedKeyAuthenticator:validate() Start validation against account shared key authentication.
2023-12-11T10:26:14.250Z bcc3c56d-8e5d-4046-b599-4da938c54e36 info: BlobSharedKeyAuthenticator:validate() Request doesn't include valid authentication header. Skip shared key authentication.
2023-12-11T10:26:14.250Z bcc3c56d-8e5d-4046-b599-4da938c54e36 info: AccountSASAuthenticator:validate() Start validation against account Shared Access Signature pattern.
2023-12-11T10:26:14.250Z bcc3c56d-8e5d-4046-b599-4da938c54e36 debug: AccountSASAuthenticator:validate() Getting account properties...
2023-12-11T10:26:14.250Z bcc3c56d-8e5d-4046-b599-4da938c54e36 debug: AccountSASAuthenticator:validate() Retrieved account name from context: devstoreaccount1, container: f8cbdc53-77cc-4e21-a4ef-cc45ccc2b05a, blob: destination.txt
2023-12-11T10:26:14.250Z bcc3c56d-8e5d-4046-b599-4da938c54e36 debug: AccountSASAuthenticator:validate() Got account properties successfully.
2023-12-11T10:26:14.250Z bcc3c56d-8e5d-4046-b599-4da938c54e36 debug: AccountSASAuthenticator:validate() Retrieved signature from URL parameter sig: {redacted}
2023-12-11T10:26:14.250Z bcc3c56d-8e5d-4046-b599-4da938c54e36 info: AccountSASAuthenticator:validate() Failed to get valid account SAS values from request.
2023-12-11T10:26:14.250Z bcc3c56d-8e5d-4046-b599-4da938c54e36 info: BlobSASAuthenticator:validate() Start validation against blob service Shared Access Signature pattern.
2023-12-11T10:26:14.250Z bcc3c56d-8e5d-4046-b599-4da938c54e36 debug: BlobSASAuthenticator:validate() Getting account properties...
2023-12-11T10:26:14.250Z bcc3c56d-8e5d-4046-b599-4da938c54e36 debug: BlobSASAuthenticator:validate() Retrieved account name from context: devstoreaccount1, container: f8cbdc53-77cc-4e21-a4ef-cc45ccc2b05a, blob: destination.txt
2023-12-11T10:26:14.250Z bcc3c56d-8e5d-4046-b599-4da938c54e36 debug: BlobSASAuthenticator:validate() Got account properties successfully.
2023-12-11T10:26:14.250Z bcc3c56d-8e5d-4046-b599-4da938c54e36 debug: BlobSASAuthenticator:validate() Retrieved signature from URL parameter sig: {redacted}
2023-12-11T10:26:14.250Z bcc3c56d-8e5d-4046-b599-4da938c54e36 debug: BlobSASAuthenticator:validate() Signed resource type is c.
2023-12-11T10:26:14.250Z bcc3c56d-8e5d-4046-b599-4da938c54e36 debug: BlobSASAuthenticator:validate() Successfully got valid blob service SAS values from request. {"version":"2020-04-08","expiryTime":"2023-12-12T10:21:45Z","permissions":"rc","containerName":"f8cbdc53-77cc-4e21-a4ef-cc45ccc2b05a","blobName":"destination.txt","signedResource":"c"}
2023-12-11T10:26:14.251Z bcc3c56d-8e5d-4046-b599-4da938c54e36 info: BlobSASAuthenticator:validate() Validate signature based account key1.
2023-12-11T10:26:14.251Z bcc3c56d-8e5d-4046-b599-4da938c54e36 debug: BlobSASAuthenticator:validate() String to sign is: "rc\n\n2023-12-12T10:21:45Z\n/blob/devstoreaccount1/f8cbdc53-77cc-4e21-a4ef-cc45ccc2b05a\n\n\n\n2020-04-08\nc\n\n\n\n\n\n"
2023-12-11T10:26:14.251Z bcc3c56d-8e5d-4046-b599-4da938c54e36 debug: BlobSASAuthenticator:validate() Calculated signature is: {redacted}
2023-12-11T10:26:14.251Z bcc3c56d-8e5d-4046-b599-4da938c54e36 info: BlobSASAuthenticator:validate() Signature based on key1 validation passed.
2023-12-11T10:26:14.251Z bcc3c56d-8e5d-4046-b599-4da938c54e36 info: BlobSASAuthenticator:validate() Validate start and expiry time.
2023-12-11T10:26:14.251Z bcc3c56d-8e5d-4046-b599-4da938c54e36 info: BlobSASAuthenticator:validate() Validate IP range.
2023-12-11T10:26:14.251Z bcc3c56d-8e5d-4046-b599-4da938c54e36 info: BlobSASAuthenticator:validate() Validate request protocol.
2023-12-11T10:26:14.251Z bcc3c56d-8e5d-4046-b599-4da938c54e36 debug: BlobSASAuthenticator:validate() Got permission requirements for operation Blob_StartCopyFromURL - {"permission":"wc"}
2023-12-11T10:26:14.251Z bcc3c56d-8e5d-4046-b599-4da938c54e36 info: BlobSASAuthenticator:validate() For Blob_StartCopyFromURL, if blob exists, the permission must be Write.
2023-12-11T10:26:14.251Z bcc3c56d-8e5d-4046-b599-4da938c54e36 info: BlobSASAuthenticator:validate() Blob service SAS validation successfully.
2023-12-11T10:26:14.252Z bcc3c56d-8e5d-4046-b599-4da938c54e36 verbose: DeserializerMiddleware: Start deserializing...
2023-12-11T10:26:14.252Z bcc3c56d-8e5d-4046-b599-4da938c54e36 info: HandlerMiddleware: DeserializedParameters={"options":{"metadata":{},"requestId":"abeef25f-8698-4b94-8749-23241d5d2768","sourceModifiedAccessConditions":{},"modifiedAccessConditions":{},"leaseAccessConditions":{}},"copySource":"http://127.0.0.1:10000/devstoreaccount1/f8cbdc53-77cc-4e21-a4ef-cc45ccc2b05a/source.txt?sv=2020-04-08&se=2023-12-12T10%3A26%3A09Z&sr=c&sp=d&sig={redacted}","version":"2020-04-08"}
2023-12-11T10:26:14.253Z bcc3c56d-8e5d-4046-b599-4da938c54e36 verbose: SerializerMiddleware: Start serializing...
2023-12-11T10:26:14.254Z bcc3c56d-8e5d-4046-b599-4da938c54e36 info: EndMiddleware: End response. TotalTimeInMS=9 StatusCode=202 StatusMessage=undefined Headers={"server":"Azurite-Blob/3.28.0","etag":""0x1BB60F3A2CEED50"","last-modified":"Mon, 11 Dec 2023 10:26:14 GMT","x-ms-client-request-id":"abeef25f-8698-4b94-8749-23241d5d2768","x-ms-request-id":"bcc3c56d-8e5d-4046-b599-4da938c54e36","x-ms-version":"2023-11-03","date":"Mon, 11 Dec 2023 10:26:14 GMT","x-ms-copy-id":"c83d367b-ab74-431f-a0e5-28755f3d90ab","x-ms-copy-status":"success"}

Have you found a mitigation/solution?

Found a closed issue similar to this: #1824 which is on the destination blob invalid permissions.

Tasks
Preview Give feedback
No tasks being tracked yet.
@blueww
Copy link
Member

blueww commented Dec 11, 2023

@arnabfromjec05

Would you please attach Azurite debug log for this issue?
So we can get more details to investigate this issue.

@blueww blueww mentioned this issue Dec 11, 2023
Closed
@arnabfromjec05
Copy link
Author

arnabfromjec05 commented Dec 11, 2023 •
edited
Loading

@blueww

Added debug logs in the description.
from logs you can verify:

  • for source.txt sas permission has 'c' access
  • for destination.txt 'rc' access

and uri validation succeeds as per log line:- BlobSASAuthenticator:validate() Blob service SAS validation successfully.

@blueww blueww added blob-storage bug Something isn't working labels Dec 12, 2023
@blueww
Copy link
Member

blueww commented Dec 12, 2023

In Azurite, blob copy still not check the source blob permission.
@EmmaZhu , as we discussed, would you please help to fix this?

@arnabfromjec05
Copy link
Author

Hi @EmmaZhu
wanted to understand the solution approach and the priority.
Have pinged you offline on this.

@blueww
Copy link
Member

blueww commented Jan 8, 2024

Fix PR is merged.
#2330

@arnabfromjec05
Copy link
Author

Thanks for addressing this issue!

@blueww
Copy link
Member

blueww commented Apr 29, 2024

Close as the fix is already released.

@blueww blueww closed this as completed Apr 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@EmmaZhu EmmaZhu

Labels
blob-storage bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@EmmaZhu @blueww @arnabfromjec05