From 9a351b9985395b1eb662883707871f9a0cb0adbd Mon Sep 17 00:00:00 2001 From: Tamer Sherif <69483382+tasherif-msft@users.noreply.github.com> Date: Wed, 19 Jul 2023 10:30:52 -0700 Subject: [PATCH] [AzDatalake] File Client Support (#21141) * Enable gocritic during linting (#20715) Enabled gocritic's evalOrder to catch dependencies on undefined behavior on return statements. Updated to latest version of golangci-lint. Fixed issue in azblob flagged by latest linter. * Cosmos DB: Enable merge support (#20716) * Adding header and value * Wiring and tests * format * Fixing value * change log * [azservicebus, azeventhubs] Stress test and logging improvement (#20710) Logging improvements: * Updating the logging to print more tracing information (per-link) in prep for the bigger release coming up. * Trimming out some of the verbose logging, seeing if I can get it a bit more reasonable. Stress tests: * Add a timestamp to the log name we generate and also default to append, not overwrite. * Use 0.5 cores, 0.5GB as our baseline. Some pods use more and I'll tune them more later. * update proxy version (#20712) Co-authored-by: Scott Beddall * Return an error when you try to send a message that's too large. (#20721) This now works just like the message batch - you'll get an ErrMessageTooLarge if you attempt to send a message that's too large for the link's configured size. NOTE: there's a patch to `internal/go-amqp/Sender.go` to match what's in go-amqp's main so it returns a programmatically useful error when the message is too large. Fixes #20647 * Changes in test that is failing in pipeline (#20693) * [azservicebus, azeventhubs] Treat 'entity full' as a fatal error (#20722) When the remote entity is full we get a resource-limit-exceeded condition. This isn't something we should keep retrying on and it's best to just abort and let the user know immediately, rather than hoping it might eventually clear out. This affected both Event Hubs and Service Bus. Fixes #20647 * [azservicebus/azeventhubs] Redirect stderr and stdout to tee (#20726) * Update changelog with latest features (#20730) * Update changelog with latest features Prepare for upcoming release. * bump minor version * pass along the artifact name so we can override it later (#20732) Co-authored-by: scbedd <45376673+scbedd@users.noreply.github.com> * [azeventhubs] Fixing checkpoint store race condition (#20727) The checkpoint store wasn't guarding against multiple owners claiming for the first time - fixing this by using IfNoneMatch Fixes #20717 * Fix azidentity troubleshooting guide link (#20736) * [Release] sdk/resourcemanager/paloaltonetworksngfw/armpanngfw/0.1.0 (#20437) * [Release] sdk/resourcemanager/paloaltonetworksngfw/armpanngfw/0.1.0 generation from spec commit: 85fb4ac6f8bfefd179e6c2632976a154b5c9ff04 * client factory * fix * fix * update * add sdk/resourcemanager/postgresql/armpostgresql live test (#20685) * add sdk/resourcemanager/postgresql/armpostgresql live test * update assets.json * set subscriptionId default value * format * add sdk/resourcemanager/eventhub/armeventhub live test (#20686) * add sdk/resourcemanager/eventhub/armeventhub live test * update assets * add sdk/resourcemanager/compute/armcompute live test (#20048) * add sdk/resourcemanager/compute/armcompute live test * skus filter * fix subscriptionId default value * fix * gofmt * update recording * sdk/resourcemanager/network/armnetwork live test (#20331) * sdk/resourcemanager/network/armnetwork live test * update subscriptionId default value * update recording * add sdk/resourcemanager/cosmos/armcosmos live test (#20705) * add sdk/resourcemanager/cosmos/armcosmos live test * update assets.json * update assets.json * update assets.json * update assets.json * Increment package version after release of azcore (#20740) * [azeventhubs] Improperly resetting etag in the checkpoint store (#20737) We shouldn't be resetting the etag to nil - it's what we use to enforce a "single winner" when doing ownership claims. The bug here was two-fold: I had bad logic in my previous claim ownership, which I fixed in a previous PR, but we need to reflect that same constraint properly in our in-memory checkpoint store for these tests. * Eng workflows sync and branch cleanup additions (#20743) Co-authored-by: James Suplizio * [azeventhubs] Latest start position can also be inclusive (ie, get the latest message) (#20744) * Update GitHubEventProcessor version and remove pull_request_review procesing (#20751) Co-authored-by: James Suplizio * Rename DisableAuthorityValidationAndInstanceDiscovery (#20746) * fix (#20707) * AzFile (#20739) * azfile: Fixing connection string parsing logic (#20798) * Fixing connection string parse logic * Update README * [azadmin] fix flaky test (#20758) * fix flaky test * charles suggestion * Prepare azidentity v1.3.0 for release (#20756) * Fix broken podman link (#20801) Co-authored-by: Wes Haggard * [azquery] update doc comments (#20755) * update doc comments * update statistics and visualization generation * prep-for-release * Fixed contribution section (#20752) Co-authored-by: Bob Tabor * [azeventhubs,azservicebus] Some API cleanup, renames (#20754) * Adding options to UpdateCheckpoint(), just for future potential expansion * Make Offset an int64, not a *int64 (it's not optional, it'll always come back with ReceivedEvents) * Adding more logging into the checkpoint store. * Point all imports at the production go-amqp * Add supporting features to enable distributed tracing (#20301) (#20708) * Add supporting features to enable distributed tracing This includes new internal pipeline policies and other supporting types. See the changelog for a full description. Added some missing doc comments. * fix linter issue * add net.peer.name trace attribute sequence custom HTTP header policy before logging policy. sequence logging policy after HTTP trace policy. keep body download policy at the end. * add span for iterating over pages * Restore ARM CAE support for azcore beta (#20657) This reverts commit 902097226ff3fe2fc6c3e7fc50d3478350253614. * Upgrade to stable azcore (#20808) * Increment package version after release of data/azcosmos (#20807) * Updating changelog (#20810) * Add fake package to azcore (#20711) * Add fake package to azcore This is the supporting infrastructure for the generated SDK fakes. * fix doc comment * Updating CHANGELOG.md (#20809) * changelog (#20811) * Increment package version after release of storage/azfile (#20813) * Update changelog (azblob) (#20815) * Updating CHANGELOG.md * Update the changelog with correct version * [azquery] migration guide (#20742) * migration guide * Charles feedback * Richard feedback --------- Co-authored-by: Charles Lowell <10964656+chlowell@users.noreply.github.com> * Increment package version after release of monitor/azquery (#20820) * [keyvault] prep for release (#20819) * prep for release * perf tests * update date * added apis and models * cleanup * cleanup + block blob client * added tests * added tests * added tests * added more tests * added more tests * dir block client * sas test * sas test --------- Co-authored-by: Joel Hendrix Co-authored-by: Matias Quaranta Co-authored-by: Richard Park <51494936+richardpark-msft@users.noreply.github.com> Co-authored-by: Azure SDK Bot <53356347+azure-sdk@users.noreply.github.com> Co-authored-by: Scott Beddall Co-authored-by: siminsavani-msft <77068571+siminsavani-msft@users.noreply.github.com> Co-authored-by: scbedd <45376673+scbedd@users.noreply.github.com> Co-authored-by: Charles Lowell <10964656+chlowell@users.noreply.github.com> Co-authored-by: Peng Jiahui <46921893+Alancere@users.noreply.github.com> Co-authored-by: James Suplizio Co-authored-by: Sourav Gupta <98318303+souravgupta-msft@users.noreply.github.com> Co-authored-by: gracewilcox <43627800+gracewilcox@users.noreply.github.com> Co-authored-by: Wes Haggard Co-authored-by: Bob Tabor Co-authored-by: Bob Tabor --- sdk/storage/azdatalake/assets.json | 2 +- .../azdatalake/datalakeerror/error_codes.go | 3 + sdk/storage/azdatalake/directory/client.go | 28 +- sdk/storage/azdatalake/file/client.go | 200 +- sdk/storage/azdatalake/file/client_test.go | 2312 +++++++++++++++++ sdk/storage/azdatalake/file/constants.go | 51 +- sdk/storage/azdatalake/file/models.go | 355 ++- sdk/storage/azdatalake/file/responses.go | 54 +- .../azdatalake/filesystem/client_test.go | 24 +- sdk/storage/azdatalake/filesystem/models.go | 15 + .../azdatalake/internal/base/clients.go | 10 +- .../internal/exported/access_conditions.go | 9 + .../internal/exported/set_expiry.go | 38 +- .../internal/testcommon/clients_auth.go | 40 + .../azdatalake/internal/testcommon/common.go | 6 +- sdk/storage/azdatalake/service/client_test.go | 10 +- 16 files changed, 2869 insertions(+), 288 deletions(-) create mode 100644 sdk/storage/azdatalake/file/client_test.go diff --git a/sdk/storage/azdatalake/assets.json b/sdk/storage/azdatalake/assets.json index cc6130dbb776..e712ffb1d71d 100644 --- a/sdk/storage/azdatalake/assets.json +++ b/sdk/storage/azdatalake/assets.json @@ -2,5 +2,5 @@ "AssetsRepo": "Azure/azure-sdk-assets", "AssetsRepoPrefixPath": "go", "TagPrefix": "go/storage/azdatalake", - "Tag": "go/storage/azdatalake_e05ec93d89" + "Tag": "go/storage/azdatalake_820b86faa9" } \ No newline at end of file diff --git a/sdk/storage/azdatalake/datalakeerror/error_codes.go b/sdk/storage/azdatalake/datalakeerror/error_codes.go index 2acc2e79fdc6..a40e54cebe37 100644 --- a/sdk/storage/azdatalake/datalakeerror/error_codes.go +++ b/sdk/storage/azdatalake/datalakeerror/error_codes.go @@ -178,4 +178,7 @@ const ( var ( // MissingSharedKeyCredential - Error is returned when SAS URL is being created without SharedKeyCredential. MissingSharedKeyCredential = bloberror.MissingSharedKeyCredential + + // MissingParameters - Error is returned when at least one parameter should be set for any API. + MissingParameters = errors.New("at least one parameter should be set for SetAccessControl API") ) diff --git a/sdk/storage/azdatalake/directory/client.go b/sdk/storage/azdatalake/directory/client.go index 749dfbc86724..b48045888290 100644 --- a/sdk/storage/azdatalake/directory/client.go +++ b/sdk/storage/azdatalake/directory/client.go @@ -11,7 +11,7 @@ import ( "github.com/Azure/azure-sdk-for-go/sdk/azcore" "github.com/Azure/azure-sdk-for-go/sdk/azcore/policy" "github.com/Azure/azure-sdk-for-go/sdk/azcore/runtime" - "github.com/Azure/azure-sdk-for-go/sdk/storage/azblob/blob" + "github.com/Azure/azure-sdk-for-go/sdk/storage/azblob/blockblob" "github.com/Azure/azure-sdk-for-go/sdk/storage/azdatalake/internal/base" "github.com/Azure/azure-sdk-for-go/sdk/storage/azdatalake/internal/exported" "github.com/Azure/azure-sdk-for-go/sdk/storage/azdatalake/internal/generated" @@ -22,7 +22,7 @@ import ( type ClientOptions base.ClientOptions // Client represents a URL to the Azure Datalake Storage service. -type Client base.CompositeClient[generated.PathClient, generated.PathClient, blob.Client] +type Client base.CompositeClient[generated.PathClient, generated.PathClient, blockblob.Client] // NewClient creates an instance of Client with the specified values. // - directoryURL - the URL of the directory e.g. https://.dfs.core.windows.net/fs/dir @@ -46,10 +46,10 @@ func NewClient(directoryURL string, cred azcore.TokenCredential, options *Client if options == nil { options = &ClientOptions{} } - blobClientOpts := blob.ClientOptions{ + blobClientOpts := blockblob.ClientOptions{ ClientOptions: options.ClientOptions, } - blobClient, _ := blob.NewClient(blobURL, cred, &blobClientOpts) + blobClient, _ := blockblob.NewClient(blobURL, cred, &blobClientOpts) dirClient := base.NewPathClient(directoryURL, blobURL, blobClient, azClient, nil, (*base.ClientOptions)(conOptions)) return (*Client)(dirClient), nil @@ -74,10 +74,10 @@ func NewClientWithNoCredential(directoryURL string, options *ClientOptions) (*Cl if options == nil { options = &ClientOptions{} } - blobClientOpts := blob.ClientOptions{ + blobClientOpts := blockblob.ClientOptions{ ClientOptions: options.ClientOptions, } - blobClient, _ := blob.NewClientWithNoCredential(blobURL, &blobClientOpts) + blobClient, _ := blockblob.NewClientWithNoCredential(blobURL, &blobClientOpts) dirClient := base.NewPathClient(directoryURL, blobURL, blobClient, azClient, nil, (*base.ClientOptions)(conOptions)) return (*Client)(dirClient), nil @@ -105,15 +105,15 @@ func NewClientWithSharedKeyCredential(directoryURL string, cred *SharedKeyCreden if options == nil { options = &ClientOptions{} } - blobClientOpts := blob.ClientOptions{ + blobClientOpts := blockblob.ClientOptions{ ClientOptions: options.ClientOptions, } blobSharedKey, err := cred.ConvertToBlobSharedKey() if err != nil { return nil, err } - blobClient, _ := blob.NewClientWithSharedKeyCredential(blobURL, blobSharedKey, &blobClientOpts) - dirClient := base.NewPathClient(directoryURL, blobURL, blobClient, azClient, nil, (*base.ClientOptions)(conOptions)) + blobClient, _ := blockblob.NewClientWithSharedKeyCredential(blobURL, blobSharedKey, &blobClientOpts) + dirClient := base.NewPathClient(directoryURL, blobURL, blobClient, azClient, cred, (*base.ClientOptions)(conOptions)) return (*Client)(dirClient), nil } @@ -140,22 +140,22 @@ func NewClientFromConnectionString(connectionString string, options *ClientOptio func (d *Client) generatedDirClientWithDFS() *generated.PathClient { //base.SharedKeyComposite((*base.CompositeClient[generated.BlobClient, generated.BlockBlobClient])(bb)) - dirClientWithDFS, _, _ := base.InnerClients((*base.CompositeClient[generated.PathClient, generated.PathClient, blob.Client])(d)) + dirClientWithDFS, _, _ := base.InnerClients((*base.CompositeClient[generated.PathClient, generated.PathClient, blockblob.Client])(d)) return dirClientWithDFS } func (d *Client) generatedDirClientWithBlob() *generated.PathClient { - _, dirClientWithBlob, _ := base.InnerClients((*base.CompositeClient[generated.PathClient, generated.PathClient, blob.Client])(d)) + _, dirClientWithBlob, _ := base.InnerClients((*base.CompositeClient[generated.PathClient, generated.PathClient, blockblob.Client])(d)) return dirClientWithBlob } -func (d *Client) blobClient() *blob.Client { - _, _, blobClient := base.InnerClients((*base.CompositeClient[generated.PathClient, generated.PathClient, blob.Client])(d)) +func (d *Client) blobClient() *blockblob.Client { + _, _, blobClient := base.InnerClients((*base.CompositeClient[generated.PathClient, generated.PathClient, blockblob.Client])(d)) return blobClient } func (d *Client) sharedKey() *exported.SharedKeyCredential { - return base.SharedKeyComposite((*base.CompositeClient[generated.PathClient, generated.PathClient, blob.Client])(d)) + return base.SharedKeyComposite((*base.CompositeClient[generated.PathClient, generated.PathClient, blockblob.Client])(d)) } // DFSURL returns the URL endpoint used by the Client object. diff --git a/sdk/storage/azdatalake/file/client.go b/sdk/storage/azdatalake/file/client.go index c7355f3e0916..9d3e9da012ff 100644 --- a/sdk/storage/azdatalake/file/client.go +++ b/sdk/storage/azdatalake/file/client.go @@ -11,18 +11,21 @@ import ( "github.com/Azure/azure-sdk-for-go/sdk/azcore" "github.com/Azure/azure-sdk-for-go/sdk/azcore/policy" "github.com/Azure/azure-sdk-for-go/sdk/azcore/runtime" - "github.com/Azure/azure-sdk-for-go/sdk/storage/azblob/blob" + "github.com/Azure/azure-sdk-for-go/sdk/storage/azblob/blockblob" + "github.com/Azure/azure-sdk-for-go/sdk/storage/azdatalake/datalakeerror" "github.com/Azure/azure-sdk-for-go/sdk/storage/azdatalake/internal/base" "github.com/Azure/azure-sdk-for-go/sdk/storage/azdatalake/internal/exported" "github.com/Azure/azure-sdk-for-go/sdk/storage/azdatalake/internal/generated" "github.com/Azure/azure-sdk-for-go/sdk/storage/azdatalake/internal/shared" + "github.com/Azure/azure-sdk-for-go/sdk/storage/azdatalake/sas" + "time" ) // ClientOptions contains the optional parameters when creating a Client. type ClientOptions base.ClientOptions // Client represents a URL to the Azure Datalake Storage service. -type Client base.CompositeClient[generated.PathClient, generated.PathClient, blob.Client] +type Client base.CompositeClient[generated.PathClient, generated.PathClient, blockblob.Client] // NewClient creates an instance of Client with the specified values. // - fileURL - the URL of the blob e.g. https://.dfs.core.windows.net/fs/file.txt @@ -45,10 +48,10 @@ func NewClient(fileURL string, cred azcore.TokenCredential, options *ClientOptio if options == nil { options = &ClientOptions{} } - blobClientOpts := blob.ClientOptions{ + blobClientOpts := blockblob.ClientOptions{ ClientOptions: options.ClientOptions, } - blobClient, _ := blob.NewClient(blobURL, cred, &blobClientOpts) + blobClient, _ := blockblob.NewClient(blobURL, cred, &blobClientOpts) fileClient := base.NewPathClient(fileURL, blobURL, blobClient, azClient, nil, (*base.ClientOptions)(conOptions)) return (*Client)(fileClient), nil @@ -73,10 +76,10 @@ func NewClientWithNoCredential(fileURL string, options *ClientOptions) (*Client, if options == nil { options = &ClientOptions{} } - blobClientOpts := blob.ClientOptions{ + blobClientOpts := blockblob.ClientOptions{ ClientOptions: options.ClientOptions, } - blobClient, _ := blob.NewClientWithNoCredential(blobURL, &blobClientOpts) + blobClient, _ := blockblob.NewClientWithNoCredential(blobURL, &blobClientOpts) fileClient := base.NewPathClient(fileURL, blobURL, blobClient, azClient, nil, (*base.ClientOptions)(conOptions)) return (*Client)(fileClient), nil @@ -104,15 +107,15 @@ func NewClientWithSharedKeyCredential(fileURL string, cred *SharedKeyCredential, if options == nil { options = &ClientOptions{} } - blobClientOpts := blob.ClientOptions{ + blobClientOpts := blockblob.ClientOptions{ ClientOptions: options.ClientOptions, } blobSharedKey, err := cred.ConvertToBlobSharedKey() if err != nil { return nil, err } - blobClient, _ := blob.NewClientWithSharedKeyCredential(blobURL, blobSharedKey, &blobClientOpts) - fileClient := base.NewPathClient(fileURL, blobURL, blobClient, azClient, nil, (*base.ClientOptions)(conOptions)) + blobClient, _ := blockblob.NewClientWithSharedKeyCredential(blobURL, blobSharedKey, &blobClientOpts) + fileClient := base.NewPathClient(fileURL, blobURL, blobClient, azClient, cred, (*base.ClientOptions)(conOptions)) return (*Client)(fileClient), nil } @@ -139,22 +142,26 @@ func NewClientFromConnectionString(connectionString string, options *ClientOptio func (f *Client) generatedFileClientWithDFS() *generated.PathClient { //base.SharedKeyComposite((*base.CompositeClient[generated.BlobClient, generated.BlockBlobClient])(bb)) - dirClientWithDFS, _, _ := base.InnerClients((*base.CompositeClient[generated.PathClient, generated.PathClient, blob.Client])(f)) + dirClientWithDFS, _, _ := base.InnerClients((*base.CompositeClient[generated.PathClient, generated.PathClient, blockblob.Client])(f)) return dirClientWithDFS } func (f *Client) generatedFileClientWithBlob() *generated.PathClient { - _, dirClientWithBlob, _ := base.InnerClients((*base.CompositeClient[generated.PathClient, generated.PathClient, blob.Client])(f)) + _, dirClientWithBlob, _ := base.InnerClients((*base.CompositeClient[generated.PathClient, generated.PathClient, blockblob.Client])(f)) return dirClientWithBlob } -func (f *Client) blobClient() *blob.Client { - _, _, blobClient := base.InnerClients((*base.CompositeClient[generated.PathClient, generated.PathClient, blob.Client])(f)) +func (f *Client) blobClient() *blockblob.Client { + _, _, blobClient := base.InnerClients((*base.CompositeClient[generated.PathClient, generated.PathClient, blockblob.Client])(f)) return blobClient } func (f *Client) sharedKey() *exported.SharedKeyCredential { - return base.SharedKeyComposite((*base.CompositeClient[generated.PathClient, generated.PathClient, blob.Client])(f)) + return base.SharedKeyComposite((*base.CompositeClient[generated.PathClient, generated.PathClient, blockblob.Client])(f)) +} + +func (f *Client) getClientOptions() *base.ClientOptions { + return base.GetCompositeClientOptions((*base.CompositeClient[generated.PathClient, generated.PathClient, blockblob.Client])(f)) } // DFSURL returns the URL endpoint used by the Client object. @@ -169,96 +176,143 @@ func (f *Client) BlobURL() string { // Create creates a new file (dfs1). func (f *Client) Create(ctx context.Context, options *CreateOptions) (CreateResponse, error) { - // TODO: format for options should be able to handle the access conditions parameter correctly - return CreateResponse{}, nil + lac, mac, httpHeaders, createOpts, cpkOpts := options.format() + return f.generatedFileClientWithDFS().Create(ctx, createOpts, httpHeaders, lac, mac, nil, cpkOpts) } // Delete deletes a file (dfs1). func (f *Client) Delete(ctx context.Context, options *DeleteOptions) (DeleteResponse, error) { - // TODO: recursive set to false when calling generated code - return DeleteResponse{}, nil + lac, mac, deleteOpts := options.format() + return f.generatedFileClientWithDFS().Delete(ctx, deleteOpts, lac, mac) } // GetProperties gets the properties of a file (blob3) func (f *Client) GetProperties(ctx context.Context, options *GetPropertiesOptions) (GetPropertiesResponse, error) { - // TODO: format blob response to path response - return GetPropertiesResponse{}, nil -} - -// Rename renames a file (dfs1). -func (f *Client) Rename(ctx context.Context, newName string, options *RenameOptions) (RenameResponse, error) { - return RenameResponse{}, nil -} - -// SetExpiry operation sets an expiry time on an existing file. -func (f *Client) SetExpiry(ctx context.Context, expiryType ExpiryType, o *SetExpiryOptions) (SetExpiryResponse, error) { - // TODO: consider using the blob client set expiry - // TODO: call methods in set_expiry.go - return SetExpiryResponse{}, nil -} - -// Upload uploads data to a file. -func (f *Client) Upload(ctx context.Context) { - -} - -// Append appends data to a file. -func (f *Client) Append(ctx context.Context) { - + opts := options.format() + // TODO: format response + add acls, owner, group, permissions to it + return f.blobClient().GetProperties(ctx, opts) } -// Flush flushes previous uploaded data to a file. -func (f *Client) Flush(ctx context.Context) { - +// TODO: implement below +//// Rename renames a file (dfs1). TODO: look into returning a new client possibly or changing the url +//func (f *Client) Rename(ctx context.Context, newName string, options *RenameOptions) (RenameResponse, error) { +// path, err := url.Parse(f.DFSURL()) +// if err != nil { +// return RenameResponse{}, err +// } +// lac, mac, smac, createOpts := options.format(path.Path) +// fileURL := runtime.JoinPaths(f.generatedFileClientWithDFS().Endpoint(), newName) +// // TODO: remove new azcore.Client creation after the API for shallow copying with new client name is implemented +// clOpts := f.getClientOptions() +// azClient, err := azcore.NewClient(shared.FileClient, exported.ModuleVersion, *(base.GetPipelineOptions(clOpts)), &(clOpts.ClientOptions)) +// if err != nil { +// if log.Should(exported.EventError) { +// log.Writef(exported.EventError, err.Error()) +// } +// return RenameResponse{}, err +// } +// blobURL, fileURL := shared.GetURLs(fileURL) +// tempFileClient := (*Client)(base.NewPathClient(fileURL, blobURL, nil, azClient, f.sharedKey(), clOpts)) +// // this tempClient does not have a blobClient +// return tempFileClient.generatedFileClientWithDFS().Create(ctx, createOpts, nil, lac, mac, smac, nil) +//} + +// SetExpiry operation sets an expiry time on an existing file (blob2). +func (f *Client) SetExpiry(ctx context.Context, expiryType SetExpiryType, o *SetExpiryOptions) (SetExpiryResponse, error) { + expMode, opts := expiryType.Format(o) + return f.generatedFileClientWithBlob().SetExpiry(ctx, expMode, opts) } -// Download downloads data from a file. -func (f *Client) Download(ctx context.Context) { - -} +//// Upload uploads data to a file. +//func (f *Client) Upload(ctx context.Context) { +// +//} +// +//// Append appends data to a file. +//func (f *Client) Append(ctx context.Context) { +// +//} +// +//// Flush flushes previous uploaded data to a file. +//func (f *Client) Flush(ctx context.Context) { +// +//} +// +//// Download downloads data from a file. +//func (f *Client) Download(ctx context.Context) { +// +//} // SetAccessControl sets the owner, owning group, and permissions for a file or directory (dfs1). func (f *Client) SetAccessControl(ctx context.Context, options *SetAccessControlOptions) (SetAccessControlResponse, error) { - return SetAccessControlResponse{}, nil -} - -// SetAccessControlRecursive sets the owner, owning group, and permissions for a file or directory (dfs1). -func (f *Client) SetAccessControlRecursive(ctx context.Context, options *SetAccessControlRecursiveOptions) (SetAccessControlRecursiveResponse, error) { - // TODO explicitly pass SetAccessControlRecursiveMode - return SetAccessControlRecursiveResponse{}, nil + opts, lac, mac, err := options.format() + if err != nil { + return SetAccessControlResponse{}, err + } + return f.generatedFileClientWithDFS().SetAccessControl(ctx, opts, lac, mac) } -// UpdateAccessControlRecursive updates the owner, owning group, and permissions for a file or directory (dfs1). -func (f *Client) UpdateAccessControlRecursive(ctx context.Context, options *UpdateAccessControlRecursiveOptions) (UpdateAccessControlRecursiveResponse, error) { - // TODO explicitly pass SetAccessControlRecursiveMode - return SetAccessControlRecursiveResponse{}, nil +// UpdateAccessControl updates the owner, owning group, and permissions for a file or directory (dfs1). +func (f *Client) UpdateAccessControl(ctx context.Context, ACL string, options *UpdateAccessControlOptions) (UpdateAccessControlResponse, error) { + opts, mode := options.format(ACL) + return f.generatedFileClientWithDFS().SetAccessControlRecursive(ctx, mode, opts) } // GetAccessControl gets the owner, owning group, and permissions for a file or directory (dfs1). func (f *Client) GetAccessControl(ctx context.Context, options *GetAccessControlOptions) (GetAccessControlResponse, error) { - return GetAccessControlResponse{}, nil + opts, lac, mac := options.format() + return f.generatedFileClientWithDFS().GetProperties(ctx, opts, lac, mac) } -// RemoveAccessControlRecursive removes the owner, owning group, and permissions for a file or directory (dfs1). -func (f *Client) RemoveAccessControlRecursive(ctx context.Context, options *RemoveAccessControlRecursiveOptions) (RemoveAccessControlRecursiveResponse, error) { - // TODO explicitly pass SetAccessControlRecursiveMode - return SetAccessControlRecursiveResponse{}, nil +// RemoveAccessControl removes the owner, owning group, and permissions for a file or directory (dfs1). +func (f *Client) RemoveAccessControl(ctx context.Context, ACL string, options *RemoveAccessControlOptions) (RemoveAccessControlResponse, error) { + opts, mode := options.format(ACL) + return f.generatedFileClientWithDFS().SetAccessControlRecursive(ctx, mode, opts) } // SetMetadata sets the metadata for a file or directory (blob3). func (f *Client) SetMetadata(ctx context.Context, options *SetMetadataOptions) (SetMetadataResponse, error) { - // TODO: call directly into blob - return SetMetadataResponse{}, nil + opts, metadata := options.format() + return f.blobClient().SetMetadata(ctx, metadata, opts) } // SetHTTPHeaders sets the HTTP headers for a file or directory (blob3). func (f *Client) SetHTTPHeaders(ctx context.Context, httpHeaders HTTPHeaders, options *SetHTTPHeadersOptions) (SetHTTPHeadersResponse, error) { - // TODO: call formatBlobHTTPHeaders() since we want to add the blob prefix to our options before calling into blob - // TODO: call into blob - return SetHTTPHeadersResponse{}, nil + opts, blobHTTPHeaders := options.format(httpHeaders) + resp, err := f.blobClient().SetHTTPHeaders(ctx, blobHTTPHeaders, opts) + newResp := SetHTTPHeadersResponse{} + formatSetHTTPHeadersResponse(&newResp, &resp) + return newResp, err } -// UndeletePath restores the specified path that was previously deleted. (dfs op/blob2). -func (f *Client) UndeletePath(ctx context.Context, path string, options *UndeletePathOptions) (UndeletePathResponse, error) { - return UndeletePathResponse{}, nil +// GetSASURL is a convenience method for generating a SAS token for the currently pointed at blob. +// It can only be used if the credential supplied during creation was a SharedKeyCredential. +func (f *Client) GetSASURL(permissions sas.FilePermissions, expiry time.Time, o *GetSASURLOptions) (string, error) { + if f.sharedKey() == nil { + return "", datalakeerror.MissingSharedKeyCredential + } + + urlParts, err := sas.ParseURL(f.BlobURL()) + if err != nil { + return "", err + } + + st := o.format() + + qps, err := sas.DatalakeSignatureValues{ + FilePath: urlParts.PathName, + FilesystemName: urlParts.FilesystemName, + Version: sas.Version, + Permissions: permissions.String(), + StartTime: st, + ExpiryTime: expiry.UTC(), + }.SignWithSharedKey(f.sharedKey()) + + if err != nil { + return "", err + } + + endpoint := f.BlobURL() + "?" + qps.Encode() + + return endpoint, nil } diff --git a/sdk/storage/azdatalake/file/client_test.go b/sdk/storage/azdatalake/file/client_test.go new file mode 100644 index 000000000000..cd786f2a08e3 --- /dev/null +++ b/sdk/storage/azdatalake/file/client_test.go @@ -0,0 +1,2312 @@ +//go:build go1.18 +// +build go1.18 + +// Copyright (c) Microsoft Corporation. All rights reserved. +// Licensed under the MIT License. See License.txt in the project root for license information. + +package file_test + +import ( + "context" + "github.com/Azure/azure-sdk-for-go/sdk/azcore/to" + "github.com/Azure/azure-sdk-for-go/sdk/internal/recording" + "github.com/Azure/azure-sdk-for-go/sdk/storage/azdatalake/datalakeerror" + "github.com/Azure/azure-sdk-for-go/sdk/storage/azdatalake/file" + "github.com/Azure/azure-sdk-for-go/sdk/storage/azdatalake/internal/testcommon" + "github.com/Azure/azure-sdk-for-go/sdk/storage/azdatalake/sas" + "github.com/stretchr/testify/require" + "github.com/stretchr/testify/suite" + "net/http" + "testing" + "time" +) + +var proposedLeaseIDs = []*string{to.Ptr("c820a799-76d7-4ee2-6e15-546f19325c2c"), to.Ptr("326cc5e1-746e-4af8-4811-a50e6629a8ca")} + +func Test(t *testing.T) { + recordMode := recording.GetRecordMode() + t.Logf("Running datalake Tests in %s mode\n", recordMode) + if recordMode == recording.LiveMode { + suite.Run(t, &RecordedTestSuite{}) + suite.Run(t, &UnrecordedTestSuite{}) + } else if recordMode == recording.PlaybackMode { + suite.Run(t, &RecordedTestSuite{}) + } else if recordMode == recording.RecordingMode { + suite.Run(t, &RecordedTestSuite{}) + } +} + +func (s *RecordedTestSuite) BeforeTest(suite string, test string) { + testcommon.BeforeTest(s.T(), suite, test) +} + +func (s *RecordedTestSuite) AfterTest(suite string, test string) { + testcommon.AfterTest(s.T(), suite, test) +} + +func (s *UnrecordedTestSuite) BeforeTest(suite string, test string) { + +} + +func (s *UnrecordedTestSuite) AfterTest(suite string, test string) { + +} + +type RecordedTestSuite struct { + suite.Suite +} + +type UnrecordedTestSuite struct { + suite.Suite +} + +func validateFileDeleted(_require *require.Assertions, fileClient *file.Client) { + _, err := fileClient.GetAccessControl(context.Background(), nil) + _require.NotNil(err) + + testcommon.ValidateErrorCode(_require, err, datalakeerror.BlobNotFound) +} + +func (s *RecordedTestSuite) TestCreateFileAndDelete() { + _require := require.New(s.T()) + testName := s.T().Name() + + filesystemName := testcommon.GenerateFilesystemName(testName) + fsClient, err := testcommon.GetFilesystemClient(filesystemName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + defer testcommon.DeleteFilesystem(context.Background(), _require, fsClient) + + _, err = fsClient.Create(context.Background(), nil) + _require.Nil(err) + + fileName := testcommon.GenerateFileName(testName) + fClient, err := testcommon.GetFileClient(filesystemName, fileName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + + defer testcommon.DeleteFile(context.Background(), _require, fClient) + + resp, err := fClient.Create(context.Background(), nil) + _require.Nil(err) + _require.NotNil(resp) +} + +func (s *RecordedTestSuite) TestCreateFileWithNilAccessConditions() { + _require := require.New(s.T()) + testName := s.T().Name() + + filesystemName := testcommon.GenerateFilesystemName(testName) + fsClient, err := testcommon.GetFilesystemClient(filesystemName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + defer testcommon.DeleteFilesystem(context.Background(), _require, fsClient) + + _, err = fsClient.Create(context.Background(), nil) + _require.Nil(err) + + fileName := testcommon.GenerateFileName(testName) + fClient, err := testcommon.GetFileClient(filesystemName, fileName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + + defer testcommon.DeleteFile(context.Background(), _require, fClient) + + resp, err := fClient.Create(context.Background(), nil) + _require.Nil(err) + _require.NotNil(resp) + + createFileOpts := &file.CreateOptions{ + AccessConditions: nil, + } + + resp, err = fClient.Create(context.Background(), createFileOpts) + _require.Nil(err) + _require.NotNil(resp) +} + +func (s *RecordedTestSuite) TestCreateFileIfModifiedSinceTrue() { + _require := require.New(s.T()) + testName := s.T().Name() + + filesystemName := testcommon.GenerateFilesystemName(testName) + fsClient, err := testcommon.GetFilesystemClient(filesystemName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + defer testcommon.DeleteFilesystem(context.Background(), _require, fsClient) + + _, err = fsClient.Create(context.Background(), nil) + _require.Nil(err) + + fileName := testcommon.GenerateFileName(testName) + fClient, err := testcommon.GetFileClient(filesystemName, fileName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + + defer testcommon.DeleteFile(context.Background(), _require, fClient) + + resp, err := fClient.Create(context.Background(), nil) + _require.Nil(err) + _require.NotNil(resp) + + currentTime := testcommon.GetRelativeTimeFromAnchor(resp.Date, -10) + + createFileOpts := &file.CreateOptions{ + AccessConditions: &file.AccessConditions{ + ModifiedAccessConditions: &file.ModifiedAccessConditions{ + IfModifiedSince: ¤tTime, + }, + }, + } + + resp, err = fClient.Create(context.Background(), createFileOpts) + _require.Nil(err) + _require.NotNil(resp) +} + +func (s *RecordedTestSuite) TestCreateFileIfModifiedSinceFalse() { + _require := require.New(s.T()) + testName := s.T().Name() + + filesystemName := testcommon.GenerateFilesystemName(testName) + fsClient, err := testcommon.GetFilesystemClient(filesystemName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + defer testcommon.DeleteFilesystem(context.Background(), _require, fsClient) + + _, err = fsClient.Create(context.Background(), nil) + _require.Nil(err) + + fileName := testcommon.GenerateFileName(testName) + fClient, err := testcommon.GetFileClient(filesystemName, fileName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + + defer testcommon.DeleteFile(context.Background(), _require, fClient) + + resp, err := fClient.Create(context.Background(), nil) + _require.Nil(err) + _require.NotNil(resp) + + currentTime := testcommon.GetRelativeTimeFromAnchor(resp.Date, 10) + + createFileOpts := &file.CreateOptions{ + AccessConditions: &file.AccessConditions{ + ModifiedAccessConditions: &file.ModifiedAccessConditions{ + IfModifiedSince: ¤tTime, + }, + }, + } + + resp, err = fClient.Create(context.Background(), createFileOpts) + _require.NotNil(err) + testcommon.ValidateErrorCode(_require, err, datalakeerror.ConditionNotMet) +} + +func (s *RecordedTestSuite) TestCreateFileIfUnmodifiedSinceTrue() { + _require := require.New(s.T()) + testName := s.T().Name() + + filesystemName := testcommon.GenerateFilesystemName(testName) + fsClient, err := testcommon.GetFilesystemClient(filesystemName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + defer testcommon.DeleteFilesystem(context.Background(), _require, fsClient) + + _, err = fsClient.Create(context.Background(), nil) + _require.Nil(err) + + fileName := testcommon.GenerateFileName(testName) + fClient, err := testcommon.GetFileClient(filesystemName, fileName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + + defer testcommon.DeleteFile(context.Background(), _require, fClient) + + resp, err := fClient.Create(context.Background(), nil) + _require.Nil(err) + _require.NotNil(resp) + + currentTime := testcommon.GetRelativeTimeFromAnchor(resp.Date, 10) + + createFileOpts := &file.CreateOptions{ + AccessConditions: &file.AccessConditions{ + ModifiedAccessConditions: &file.ModifiedAccessConditions{ + IfUnmodifiedSince: ¤tTime, + }, + }, + } + + resp, err = fClient.Create(context.Background(), createFileOpts) + _require.Nil(err) + _require.NotNil(resp) +} + +func (s *RecordedTestSuite) TestCreateFileIfUnmodifiedSinceFalse() { + _require := require.New(s.T()) + testName := s.T().Name() + + filesystemName := testcommon.GenerateFilesystemName(testName) + fsClient, err := testcommon.GetFilesystemClient(filesystemName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + defer testcommon.DeleteFilesystem(context.Background(), _require, fsClient) + + _, err = fsClient.Create(context.Background(), nil) + _require.Nil(err) + + fileName := testcommon.GenerateFileName(testName) + fClient, err := testcommon.GetFileClient(filesystemName, fileName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + + defer testcommon.DeleteFile(context.Background(), _require, fClient) + + resp, err := fClient.Create(context.Background(), nil) + _require.Nil(err) + _require.NotNil(resp) + + currentTime := testcommon.GetRelativeTimeFromAnchor(resp.Date, -10) + + createFileOpts := &file.CreateOptions{ + AccessConditions: &file.AccessConditions{ + ModifiedAccessConditions: &file.ModifiedAccessConditions{ + IfUnmodifiedSince: ¤tTime, + }, + }, + } + + resp, err = fClient.Create(context.Background(), createFileOpts) + _require.NotNil(err) + + testcommon.ValidateErrorCode(_require, err, datalakeerror.ConditionNotMet) +} + +func (s *RecordedTestSuite) TestCreateFileIfETagMatch() { + _require := require.New(s.T()) + testName := s.T().Name() + + filesystemName := testcommon.GenerateFilesystemName(testName) + fsClient, err := testcommon.GetFilesystemClient(filesystemName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + defer testcommon.DeleteFilesystem(context.Background(), _require, fsClient) + + _, err = fsClient.Create(context.Background(), nil) + _require.Nil(err) + + fileName := testcommon.GenerateFileName(testName) + fClient, err := testcommon.GetFileClient(filesystemName, fileName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + + defer testcommon.DeleteFile(context.Background(), _require, fClient) + + resp, err := fClient.Create(context.Background(), nil) + _require.Nil(err) + _require.NotNil(resp) + + etag := resp.ETag + + createFileOpts := &file.CreateOptions{ + AccessConditions: &file.AccessConditions{ + ModifiedAccessConditions: &file.ModifiedAccessConditions{ + IfMatch: etag, + }, + }, + } + + resp, err = fClient.Create(context.Background(), createFileOpts) + _require.Nil(err) + _require.NotNil(resp) +} + +func (s *RecordedTestSuite) TestCreateFileIfETagMatchFalse() { + _require := require.New(s.T()) + testName := s.T().Name() + + filesystemName := testcommon.GenerateFilesystemName(testName) + fsClient, err := testcommon.GetFilesystemClient(filesystemName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + defer testcommon.DeleteFilesystem(context.Background(), _require, fsClient) + + _, err = fsClient.Create(context.Background(), nil) + _require.Nil(err) + + fileName := testcommon.GenerateFileName(testName) + fClient, err := testcommon.GetFileClient(filesystemName, fileName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + + defer testcommon.DeleteFile(context.Background(), _require, fClient) + + resp, err := fClient.Create(context.Background(), nil) + _require.Nil(err) + _require.NotNil(resp) + + etag := resp.ETag + + createFileOpts := &file.CreateOptions{ + AccessConditions: &file.AccessConditions{ + ModifiedAccessConditions: &file.ModifiedAccessConditions{ + IfNoneMatch: etag, + }, + }, + } + + resp, err = fClient.Create(context.Background(), createFileOpts) + _require.NotNil(err) + + testcommon.ValidateErrorCode(_require, err, datalakeerror.ConditionNotMet) +} + +func (s *RecordedTestSuite) TestCreateFileWithMetadataNotNil() { + _require := require.New(s.T()) + testName := s.T().Name() + + filesystemName := testcommon.GenerateFilesystemName(testName) + fsClient, err := testcommon.GetFilesystemClient(filesystemName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + defer testcommon.DeleteFilesystem(context.Background(), _require, fsClient) + + createFileOpts := &file.CreateOptions{ + Metadata: testcommon.BasicMetadata, + } + + _, err = fsClient.Create(context.Background(), nil) + _require.Nil(err) + + fileName := testcommon.GenerateFileName(testName) + fClient, err := testcommon.GetFileClient(filesystemName, fileName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + + defer testcommon.DeleteFile(context.Background(), _require, fClient) + + resp, err := fClient.Create(context.Background(), createFileOpts) + _require.Nil(err) + _require.NotNil(resp) +} + +func (s *RecordedTestSuite) TestCreateFileWithEmptyMetadata() { + _require := require.New(s.T()) + testName := s.T().Name() + + filesystemName := testcommon.GenerateFilesystemName(testName) + fsClient, err := testcommon.GetFilesystemClient(filesystemName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + defer testcommon.DeleteFilesystem(context.Background(), _require, fsClient) + + createFileOpts := &file.CreateOptions{ + Metadata: nil, + } + + _, err = fsClient.Create(context.Background(), nil) + _require.Nil(err) + + fileName := testcommon.GenerateFileName(testName) + fClient, err := testcommon.GetFileClient(filesystemName, fileName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + + defer testcommon.DeleteFile(context.Background(), _require, fClient) + + resp, err := fClient.Create(context.Background(), createFileOpts) + _require.Nil(err) + _require.NotNil(resp) +} + +func (s *RecordedTestSuite) TestCreateFileWithNilHTTPHeaders() { + _require := require.New(s.T()) + testName := s.T().Name() + + filesystemName := testcommon.GenerateFilesystemName(testName) + fsClient, err := testcommon.GetFilesystemClient(filesystemName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + defer testcommon.DeleteFilesystem(context.Background(), _require, fsClient) + + createFileOpts := &file.CreateOptions{ + HTTPHeaders: nil, + } + + _, err = fsClient.Create(context.Background(), nil) + _require.Nil(err) + + fileName := testcommon.GenerateFileName(testName) + fClient, err := testcommon.GetFileClient(filesystemName, fileName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + + defer testcommon.DeleteFile(context.Background(), _require, fClient) + + resp, err := fClient.Create(context.Background(), createFileOpts) + _require.Nil(err) + _require.NotNil(resp) +} + +func (s *RecordedTestSuite) TestCreateFileWithHTTPHeaders() { + _require := require.New(s.T()) + testName := s.T().Name() + + filesystemName := testcommon.GenerateFilesystemName(testName) + fsClient, err := testcommon.GetFilesystemClient(filesystemName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + defer testcommon.DeleteFilesystem(context.Background(), _require, fsClient) + + createFileOpts := &file.CreateOptions{ + HTTPHeaders: &testcommon.BasicHeaders, + } + + _, err = fsClient.Create(context.Background(), nil) + _require.Nil(err) + + fileName := testcommon.GenerateFileName(testName) + fClient, err := testcommon.GetFileClient(filesystemName, fileName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + + defer testcommon.DeleteFile(context.Background(), _require, fClient) + + resp, err := fClient.Create(context.Background(), createFileOpts) + _require.Nil(err) + _require.NotNil(resp) +} + +func (s *RecordedTestSuite) TestCreateFileWithExpiryAbsolute() { + _require := require.New(s.T()) + testName := s.T().Name() + + filesystemName := testcommon.GenerateFilesystemName(testName) + fsClient, err := testcommon.GetFilesystemClient(filesystemName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + defer testcommon.DeleteFilesystem(context.Background(), _require, fsClient) + + expiryTimeAbsolute := time.Now().Add(8 * time.Second) + expiry := file.CreationExpiryTypeAbsolute(expiryTimeAbsolute) + createFileOpts := &file.CreateOptions{ + Expiry: expiry, + } + + _, err = fsClient.Create(context.Background(), nil) + _require.Nil(err) + + fileName := testcommon.GenerateFileName(testName) + fClient, err := testcommon.GetFileClient(filesystemName, fileName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + + defer testcommon.DeleteFile(context.Background(), _require, fClient) + + resp, err := fClient.Create(context.Background(), createFileOpts) + _require.Nil(err) + _require.NotNil(resp) + + resp1, err := fClient.GetProperties(context.Background(), nil) + _require.Nil(err) + _require.NotNil(resp1.ExpiresOn) + _require.Equal(expiryTimeAbsolute.UTC().Format(http.TimeFormat), (*resp1.ExpiresOn).UTC().Format(http.TimeFormat)) +} + +func (s *RecordedTestSuite) TestCreateFileWithExpiryRelativeToNow() { + _require := require.New(s.T()) + testName := s.T().Name() + + filesystemName := testcommon.GenerateFilesystemName(testName) + fsClient, err := testcommon.GetFilesystemClient(filesystemName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + defer testcommon.DeleteFilesystem(context.Background(), _require, fsClient) + + expiry := file.CreationExpiryTypeRelativeToNow(8 * time.Second) + createFileOpts := &file.CreateOptions{ + Expiry: expiry, + } + + _, err = fsClient.Create(context.Background(), nil) + _require.Nil(err) + + fileName := testcommon.GenerateFileName(testName) + fClient, err := testcommon.GetFileClient(filesystemName, fileName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + + resp, err := fClient.Create(context.Background(), createFileOpts) + _require.Nil(err) + _require.NotNil(resp) + + resp1, err := fClient.GetProperties(context.Background(), nil) + _require.Nil(err) + _require.NotNil(resp1.ExpiresOn) + + time.Sleep(time.Second * 10) + _, err = fClient.GetProperties(context.Background(), nil) + testcommon.ValidateErrorCode(_require, err, datalakeerror.BlobNotFound) +} + +func (s *RecordedTestSuite) TestCreateFileWithNeverExpire() { + _require := require.New(s.T()) + testName := s.T().Name() + + filesystemName := testcommon.GenerateFilesystemName(testName) + fsClient, err := testcommon.GetFilesystemClient(filesystemName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + defer testcommon.DeleteFilesystem(context.Background(), _require, fsClient) + + createFileOpts := &file.CreateOptions{ + Expiry: file.CreationExpiryTypeNever{}, + } + + _, err = fsClient.Create(context.Background(), nil) + _require.Nil(err) + + fileName := testcommon.GenerateFileName(testName) + fClient, err := testcommon.GetFileClient(filesystemName, fileName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + + resp, err := fClient.Create(context.Background(), createFileOpts) + _require.Nil(err) + _require.NotNil(resp) + + resp1, err := fClient.GetProperties(context.Background(), nil) + _require.Nil(err) + _require.Nil(resp1.ExpiresOn) +} + +func (s *RecordedTestSuite) TestCreateFileWithLease() { + _require := require.New(s.T()) + testName := s.T().Name() + + filesystemName := testcommon.GenerateFilesystemName(testName) + fsClient, err := testcommon.GetFilesystemClient(filesystemName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + defer testcommon.DeleteFilesystem(context.Background(), _require, fsClient) + + createFileOpts := &file.CreateOptions{ + ProposedLeaseID: proposedLeaseIDs[0], + LeaseDuration: to.Ptr(int64(15)), + } + + _, err = fsClient.Create(context.Background(), nil) + _require.Nil(err) + + fileName := testcommon.GenerateFileName(testName) + fClient, err := testcommon.GetFileClient(filesystemName, fileName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + + resp, err := fClient.Create(context.Background(), createFileOpts) + _require.Nil(err) + _require.NotNil(resp) + + // should fail since leased + _, err = fClient.Create(context.Background(), createFileOpts) + _require.NotNil(err) + + time.Sleep(time.Second * 15) + resp, err = fClient.Create(context.Background(), createFileOpts) + _require.Nil(err) + _require.NotNil(resp) +} + +func (s *RecordedTestSuite) TestCreateFileWithPermissions() { + _require := require.New(s.T()) + testName := s.T().Name() + + filesystemName := testcommon.GenerateFilesystemName(testName) + fsClient, err := testcommon.GetFilesystemClient(filesystemName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + perms := "0777" + defer testcommon.DeleteFilesystem(context.Background(), _require, fsClient) + + createFileOpts := &file.CreateOptions{ + Permissions: &perms, + } + + _, err = fsClient.Create(context.Background(), nil) + _require.Nil(err) + + fileName := testcommon.GenerateFileName(testName) + fClient, err := testcommon.GetFileClient(filesystemName, fileName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + + resp, err := fClient.Create(context.Background(), createFileOpts) + _require.Nil(err) + _require.NotNil(resp) + + //TODO: GetProperties() when you figured out how to add permissions into response +} + +func (s *RecordedTestSuite) TestCreateFileWithOwnerGroupACLUmask() { + _require := require.New(s.T()) + testName := s.T().Name() + + filesystemName := testcommon.GenerateFilesystemName(testName) + fsClient, err := testcommon.GetFilesystemClient(filesystemName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + owner := "4cf4e284-f6a8-4540-b53e-c3469af032dc" + group := owner + acl := "user::rwx,group::r-x,other::rwx" + umask := "0000" + defer testcommon.DeleteFilesystem(context.Background(), _require, fsClient) + + createFileOpts := &file.CreateOptions{ + Owner: &owner, + Group: &group, + ACL: &acl, + Umask: &umask, + } + + _, err = fsClient.Create(context.Background(), nil) + _require.Nil(err) + + fileName := testcommon.GenerateFileName(testName) + fClient, err := testcommon.GetFileClient(filesystemName, fileName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + + resp, err := fClient.Create(context.Background(), createFileOpts) + _require.Nil(err) + _require.NotNil(resp) + + //TODO: GetProperties() when you figured out how to add o,g, ACL into response +} + +func (s *RecordedTestSuite) TestDeleteFileWithNilAccessConditions() { + _require := require.New(s.T()) + testName := s.T().Name() + + filesystemName := testcommon.GenerateFilesystemName(testName) + fsClient, err := testcommon.GetFilesystemClient(filesystemName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + defer testcommon.DeleteFilesystem(context.Background(), _require, fsClient) + + _, err = fsClient.Create(context.Background(), nil) + _require.Nil(err) + + fileName := testcommon.GenerateFileName(testName) + fClient, err := testcommon.GetFileClient(filesystemName, fileName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + + _, err = fClient.Create(context.Background(), nil) + _require.Nil(err) + + deleteOpts := &file.DeleteOptions{ + AccessConditions: nil, + } + + resp, err := fClient.Delete(context.Background(), deleteOpts) + _require.Nil(err) + _require.NotNil(resp) +} + +func (s *RecordedTestSuite) TestDeleteFileIfModifiedSinceTrue() { + _require := require.New(s.T()) + testName := s.T().Name() + + filesystemName := testcommon.GenerateFilesystemName(testName) + fsClient, err := testcommon.GetFilesystemClient(filesystemName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + defer testcommon.DeleteFilesystem(context.Background(), _require, fsClient) + + _, err = fsClient.Create(context.Background(), nil) + _require.Nil(err) + + fileName := testcommon.GenerateFileName(testName) + fClient, err := testcommon.GetFileClient(filesystemName, fileName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + + resp, err := fClient.Create(context.Background(), nil) + _require.Nil(err) + + currentTime := testcommon.GetRelativeTimeFromAnchor(resp.Date, -10) + + deleteOpts := &file.DeleteOptions{ + AccessConditions: &file.AccessConditions{ + ModifiedAccessConditions: &file.ModifiedAccessConditions{ + IfModifiedSince: ¤tTime, + }, + }, + } + + resp1, err := fClient.Delete(context.Background(), deleteOpts) + _require.Nil(err) + _require.NotNil(resp1) +} + +func (s *RecordedTestSuite) TestDeleteFileIfModifiedSinceFalse() { + _require := require.New(s.T()) + testName := s.T().Name() + + filesystemName := testcommon.GenerateFilesystemName(testName) + fsClient, err := testcommon.GetFilesystemClient(filesystemName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + defer testcommon.DeleteFilesystem(context.Background(), _require, fsClient) + + _, err = fsClient.Create(context.Background(), nil) + _require.Nil(err) + + fileName := testcommon.GenerateFileName(testName) + fClient, err := testcommon.GetFileClient(filesystemName, fileName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + + resp, err := fClient.Create(context.Background(), nil) + _require.Nil(err) + _require.NotNil(resp) + + currentTime := testcommon.GetRelativeTimeFromAnchor(resp.Date, 10) + + deleteOpts := &file.DeleteOptions{ + AccessConditions: &file.AccessConditions{ + ModifiedAccessConditions: &file.ModifiedAccessConditions{ + IfModifiedSince: ¤tTime, + }, + }, + } + + _, err = fClient.Delete(context.Background(), deleteOpts) + _require.NotNil(err) + testcommon.ValidateErrorCode(_require, err, datalakeerror.ConditionNotMet) +} + +func (s *RecordedTestSuite) TestDeleteFileIfUnmodifiedSinceTrue() { + _require := require.New(s.T()) + testName := s.T().Name() + + filesystemName := testcommon.GenerateFilesystemName(testName) + fsClient, err := testcommon.GetFilesystemClient(filesystemName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + defer testcommon.DeleteFilesystem(context.Background(), _require, fsClient) + + _, err = fsClient.Create(context.Background(), nil) + _require.Nil(err) + + fileName := testcommon.GenerateFileName(testName) + fClient, err := testcommon.GetFileClient(filesystemName, fileName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + + resp, err := fClient.Create(context.Background(), nil) + _require.Nil(err) + _require.NotNil(resp) + + currentTime := testcommon.GetRelativeTimeFromAnchor(resp.Date, 10) + + deleteOpts := &file.DeleteOptions{ + AccessConditions: &file.AccessConditions{ + ModifiedAccessConditions: &file.ModifiedAccessConditions{ + IfUnmodifiedSince: ¤tTime, + }, + }, + } + + _, err = fClient.Delete(context.Background(), deleteOpts) + _require.Nil(err) + _require.NotNil(resp) +} + +func (s *RecordedTestSuite) TestDeleteFileIfUnmodifiedSinceFalse() { + _require := require.New(s.T()) + testName := s.T().Name() + + filesystemName := testcommon.GenerateFilesystemName(testName) + fsClient, err := testcommon.GetFilesystemClient(filesystemName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + defer testcommon.DeleteFilesystem(context.Background(), _require, fsClient) + + _, err = fsClient.Create(context.Background(), nil) + _require.Nil(err) + + fileName := testcommon.GenerateFileName(testName) + fClient, err := testcommon.GetFileClient(filesystemName, fileName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + + resp, err := fClient.Create(context.Background(), nil) + _require.Nil(err) + _require.NotNil(resp) + + currentTime := testcommon.GetRelativeTimeFromAnchor(resp.Date, -10) + + deleteOpts := &file.DeleteOptions{ + AccessConditions: &file.AccessConditions{ + ModifiedAccessConditions: &file.ModifiedAccessConditions{ + IfUnmodifiedSince: ¤tTime, + }, + }, + } + + _, err = fClient.Delete(context.Background(), deleteOpts) + _require.NotNil(err) + + testcommon.ValidateErrorCode(_require, err, datalakeerror.ConditionNotMet) +} + +func (s *RecordedTestSuite) TestDeleteFileIfETagMatch() { + _require := require.New(s.T()) + testName := s.T().Name() + + filesystemName := testcommon.GenerateFilesystemName(testName) + fsClient, err := testcommon.GetFilesystemClient(filesystemName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + defer testcommon.DeleteFilesystem(context.Background(), _require, fsClient) + + _, err = fsClient.Create(context.Background(), nil) + _require.Nil(err) + + fileName := testcommon.GenerateFileName(testName) + fClient, err := testcommon.GetFileClient(filesystemName, fileName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + + resp, err := fClient.Create(context.Background(), nil) + _require.Nil(err) + _require.NotNil(resp) + + etag := resp.ETag + + deleteOpts := &file.DeleteOptions{ + AccessConditions: &file.AccessConditions{ + ModifiedAccessConditions: &file.ModifiedAccessConditions{ + IfMatch: etag, + }, + }, + } + + _, err = fClient.Delete(context.Background(), deleteOpts) + _require.Nil(err) + _require.NotNil(resp) +} + +func (s *RecordedTestSuite) TestDeleteFileIfETagMatchFalse() { + _require := require.New(s.T()) + testName := s.T().Name() + + filesystemName := testcommon.GenerateFilesystemName(testName) + fsClient, err := testcommon.GetFilesystemClient(filesystemName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + defer testcommon.DeleteFilesystem(context.Background(), _require, fsClient) + + _, err = fsClient.Create(context.Background(), nil) + _require.Nil(err) + + fileName := testcommon.GenerateFileName(testName) + fClient, err := testcommon.GetFileClient(filesystemName, fileName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + + resp, err := fClient.Create(context.Background(), nil) + _require.Nil(err) + _require.NotNil(resp) + + etag := resp.ETag + + deleteOpts := &file.DeleteOptions{ + AccessConditions: &file.AccessConditions{ + ModifiedAccessConditions: &file.ModifiedAccessConditions{ + IfNoneMatch: etag, + }, + }, + } + + _, err = fClient.Delete(context.Background(), deleteOpts) + _require.NotNil(err) + + testcommon.ValidateErrorCode(_require, err, datalakeerror.ConditionNotMet) +} + +func (s *RecordedTestSuite) TestSetAccessControlNil() { + _require := require.New(s.T()) + testName := s.T().Name() + + filesystemName := testcommon.GenerateFilesystemName(testName) + fsClient, err := testcommon.GetFilesystemClient(filesystemName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + defer testcommon.DeleteFilesystem(context.Background(), _require, fsClient) + + _, err = fsClient.Create(context.Background(), nil) + _require.Nil(err) + + fileName := testcommon.GenerateFileName(testName) + fClient, err := testcommon.GetFileClient(filesystemName, fileName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + + resp, err := fClient.Create(context.Background(), nil) + _require.Nil(err) + _require.NotNil(resp) + + _, err = fClient.SetAccessControl(context.Background(), nil) + _require.NotNil(err) + + _require.Equal(err, datalakeerror.MissingParameters) +} + +// TODO: write test that fails if you provide permissions and acls +func (s *RecordedTestSuite) TestSetAccessControl() { + _require := require.New(s.T()) + testName := s.T().Name() + + filesystemName := testcommon.GenerateFilesystemName(testName) + fsClient, err := testcommon.GetFilesystemClient(filesystemName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + owner := "4cf4e284-f6a8-4540-b53e-c3469af032dc" + group := owner + acl := "user::rwx,group::r-x,other::rwx" + defer testcommon.DeleteFilesystem(context.Background(), _require, fsClient) + + opts := &file.SetAccessControlOptions{ + Owner: &owner, + Group: &group, + ACL: &acl, + } + + _, err = fsClient.Create(context.Background(), nil) + _require.Nil(err) + + fileName := testcommon.GenerateFileName(testName) + fClient, err := testcommon.GetFileClient(filesystemName, fileName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + + resp, err := fClient.Create(context.Background(), nil) + _require.Nil(err) + _require.NotNil(resp) + + _, err = fClient.SetAccessControl(context.Background(), opts) + _require.Nil(err) +} + +func (s *RecordedTestSuite) TestSetAccessControlWithNilAccessConditions() { + _require := require.New(s.T()) + testName := s.T().Name() + + filesystemName := testcommon.GenerateFilesystemName(testName) + fsClient, err := testcommon.GetFilesystemClient(filesystemName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + owner := "4cf4e284-f6a8-4540-b53e-c3469af032dc" + group := owner + acl := "user::rwx,group::r-x,other::rwx" + defer testcommon.DeleteFilesystem(context.Background(), _require, fsClient) + + opts := &file.SetAccessControlOptions{ + Owner: &owner, + Group: &group, + ACL: &acl, + AccessConditions: nil, + } + + _, err = fsClient.Create(context.Background(), nil) + _require.Nil(err) + + fileName := testcommon.GenerateFileName(testName) + fClient, err := testcommon.GetFileClient(filesystemName, fileName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + + resp, err := fClient.Create(context.Background(), nil) + _require.Nil(err) + _require.NotNil(resp) + + _, err = fClient.SetAccessControl(context.Background(), opts) + _require.Nil(err) +} + +func (s *RecordedTestSuite) TestSetAccessControlIfModifiedSinceTrue() { + _require := require.New(s.T()) + testName := s.T().Name() + + filesystemName := testcommon.GenerateFilesystemName(testName) + fsClient, err := testcommon.GetFilesystemClient(filesystemName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + owner := "4cf4e284-f6a8-4540-b53e-c3469af032dc" + group := owner + acl := "user::rwx,group::r-x,other::rwx" + defer testcommon.DeleteFilesystem(context.Background(), _require, fsClient) + + _, err = fsClient.Create(context.Background(), nil) + _require.Nil(err) + + fileName := testcommon.GenerateFileName(testName) + fClient, err := testcommon.GetFileClient(filesystemName, fileName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + + resp, err := fClient.Create(context.Background(), nil) + _require.Nil(err) + _require.NotNil(resp) + + currentTime := testcommon.GetRelativeTimeFromAnchor(resp.Date, -10) + opts := &file.SetAccessControlOptions{ + Owner: &owner, + Group: &group, + ACL: &acl, + AccessConditions: &file.AccessConditions{ + ModifiedAccessConditions: &file.ModifiedAccessConditions{ + IfModifiedSince: ¤tTime, + }, + }, + } + + _, err = fClient.SetAccessControl(context.Background(), opts) + _require.Nil(err) +} + +func (s *RecordedTestSuite) TestSetAccessControlIfModifiedSinceFalse() { + _require := require.New(s.T()) + testName := s.T().Name() + + filesystemName := testcommon.GenerateFilesystemName(testName) + fsClient, err := testcommon.GetFilesystemClient(filesystemName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + owner := "4cf4e284-f6a8-4540-b53e-c3469af032dc" + group := owner + acl := "user::rwx,group::r-x,other::rwx" + defer testcommon.DeleteFilesystem(context.Background(), _require, fsClient) + + _, err = fsClient.Create(context.Background(), nil) + _require.Nil(err) + + fileName := testcommon.GenerateFileName(testName) + fClient, err := testcommon.GetFileClient(filesystemName, fileName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + + resp, err := fClient.Create(context.Background(), nil) + _require.Nil(err) + _require.NotNil(resp) + currentTime := testcommon.GetRelativeTimeFromAnchor(resp.Date, 10) + + opts := &file.SetAccessControlOptions{ + Owner: &owner, + Group: &group, + ACL: &acl, + AccessConditions: &file.AccessConditions{ + ModifiedAccessConditions: &file.ModifiedAccessConditions{ + IfModifiedSince: ¤tTime, + }, + }, + } + + _, err = fClient.SetAccessControl(context.Background(), opts) + _require.NotNil(err) + testcommon.ValidateErrorCode(_require, err, datalakeerror.ConditionNotMet) +} + +func (s *RecordedTestSuite) TestSetAccessControlIfUnmodifiedSinceTrue() { + _require := require.New(s.T()) + testName := s.T().Name() + + filesystemName := testcommon.GenerateFilesystemName(testName) + fsClient, err := testcommon.GetFilesystemClient(filesystemName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + owner := "4cf4e284-f6a8-4540-b53e-c3469af032dc" + group := owner + acl := "user::rwx,group::r-x,other::rwx" + defer testcommon.DeleteFilesystem(context.Background(), _require, fsClient) + + _, err = fsClient.Create(context.Background(), nil) + _require.Nil(err) + + fileName := testcommon.GenerateFileName(testName) + fClient, err := testcommon.GetFileClient(filesystemName, fileName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + + resp, err := fClient.Create(context.Background(), nil) + _require.Nil(err) + _require.NotNil(resp) + + currentTime := testcommon.GetRelativeTimeFromAnchor(resp.Date, 10) + opts := &file.SetAccessControlOptions{ + Owner: &owner, + Group: &group, + ACL: &acl, + AccessConditions: &file.AccessConditions{ + ModifiedAccessConditions: &file.ModifiedAccessConditions{ + IfUnmodifiedSince: ¤tTime, + }, + }} + + _, err = fClient.SetAccessControl(context.Background(), opts) + _require.Nil(err) +} + +func (s *RecordedTestSuite) TestSetAccessControlIfUnmodifiedSinceFalse() { + _require := require.New(s.T()) + testName := s.T().Name() + + filesystemName := testcommon.GenerateFilesystemName(testName) + fsClient, err := testcommon.GetFilesystemClient(filesystemName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + owner := "4cf4e284-f6a8-4540-b53e-c3469af032dc" + group := owner + acl := "user::rwx,group::r-x,other::rwx" + defer testcommon.DeleteFilesystem(context.Background(), _require, fsClient) + + _, err = fsClient.Create(context.Background(), nil) + _require.Nil(err) + + fileName := testcommon.GenerateFileName(testName) + fClient, err := testcommon.GetFileClient(filesystemName, fileName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + + resp, err := fClient.Create(context.Background(), nil) + _require.Nil(err) + _require.NotNil(resp) + + currentTime := testcommon.GetRelativeTimeFromAnchor(resp.Date, -10) + + opts := &file.SetAccessControlOptions{ + Owner: &owner, + Group: &group, + ACL: &acl, + AccessConditions: &file.AccessConditions{ + ModifiedAccessConditions: &file.ModifiedAccessConditions{ + IfUnmodifiedSince: ¤tTime, + }, + }, + } + + _, err = fClient.SetAccessControl(context.Background(), opts) + _require.NotNil(err) + + testcommon.ValidateErrorCode(_require, err, datalakeerror.ConditionNotMet) +} + +func (s *RecordedTestSuite) TestSetAccessControlIfETagMatch() { + _require := require.New(s.T()) + testName := s.T().Name() + + filesystemName := testcommon.GenerateFilesystemName(testName) + fsClient, err := testcommon.GetFilesystemClient(filesystemName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + owner := "4cf4e284-f6a8-4540-b53e-c3469af032dc" + group := owner + acl := "user::rwx,group::r-x,other::rwx" + defer testcommon.DeleteFilesystem(context.Background(), _require, fsClient) + + _, err = fsClient.Create(context.Background(), nil) + _require.Nil(err) + + fileName := testcommon.GenerateFileName(testName) + fClient, err := testcommon.GetFileClient(filesystemName, fileName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + + resp, err := fClient.Create(context.Background(), nil) + _require.Nil(err) + _require.NotNil(resp) + etag := resp.ETag + + opts := &file.SetAccessControlOptions{ + Owner: &owner, + Group: &group, + ACL: &acl, + AccessConditions: &file.AccessConditions{ + ModifiedAccessConditions: &file.ModifiedAccessConditions{ + IfMatch: etag, + }, + }, + } + + _, err = fClient.SetAccessControl(context.Background(), opts) + _require.Nil(err) +} + +func (s *RecordedTestSuite) TestSetAccessControlIfETagMatchFalse() { + _require := require.New(s.T()) + testName := s.T().Name() + + filesystemName := testcommon.GenerateFilesystemName(testName) + fsClient, err := testcommon.GetFilesystemClient(filesystemName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + owner := "4cf4e284-f6a8-4540-b53e-c3469af032dc" + group := owner + acl := "user::rwx,group::r-x,other::rwx" + defer testcommon.DeleteFilesystem(context.Background(), _require, fsClient) + + _, err = fsClient.Create(context.Background(), nil) + _require.Nil(err) + + fileName := testcommon.GenerateFileName(testName) + fClient, err := testcommon.GetFileClient(filesystemName, fileName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + + resp, err := fClient.Create(context.Background(), nil) + _require.Nil(err) + _require.NotNil(resp) + + etag := resp.ETag + opts := &file.SetAccessControlOptions{ + Owner: &owner, + Group: &group, + ACL: &acl, + AccessConditions: &file.AccessConditions{ + ModifiedAccessConditions: &file.ModifiedAccessConditions{ + IfNoneMatch: etag, + }, + }} + + _, err = fClient.SetAccessControl(context.Background(), opts) + _require.NotNil(err) + testcommon.ValidateErrorCode(_require, err, datalakeerror.ConditionNotMet) +} + +func (s *RecordedTestSuite) TestGetAccessControl() { + _require := require.New(s.T()) + testName := s.T().Name() + + filesystemName := testcommon.GenerateFilesystemName(testName) + fsClient, err := testcommon.GetFilesystemClient(filesystemName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + acl := "user::rwx,group::r-x,other::rwx" + defer testcommon.DeleteFilesystem(context.Background(), _require, fsClient) + + createOpts := &file.CreateOptions{ + ACL: &acl, + } + _, err = fsClient.Create(context.Background(), nil) + _require.Nil(err) + + fileName := testcommon.GenerateFileName(testName) + fClient, err := testcommon.GetFileClient(filesystemName, fileName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + + resp, err := fClient.Create(context.Background(), createOpts) + _require.Nil(err) + _require.NotNil(resp) + + getACLResp, err := fClient.GetAccessControl(context.Background(), nil) + _require.Nil(err) + _require.Equal(acl, *getACLResp.ACL) +} + +func (s *RecordedTestSuite) TestGetAccessControlWithSAS() { + _require := require.New(s.T()) + testName := s.T().Name() + + filesystemName := testcommon.GenerateFilesystemName(testName) + fsClient, err := testcommon.GetFilesystemClient(filesystemName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + acl := "user::rwx,group::r-x,other::rwx" + defer testcommon.DeleteFilesystem(context.Background(), _require, fsClient) + + createOpts := &file.CreateOptions{ + ACL: &acl, + } + _, err = fsClient.Create(context.Background(), nil) + _require.Nil(err) + + fileName := testcommon.GenerateFileName(testName) + fClient, err := testcommon.GetFileClient(filesystemName, fileName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + + resp, err := fClient.Create(context.Background(), createOpts) + _require.Nil(err) + _require.NotNil(resp) + + // Adding SAS and options + permissions := sas.FilePermissions{ + Read: true, + Add: true, + Write: true, + Create: true, + Delete: true, + } + expiry := time.Now().Add(time.Hour) + + sasURL, err := fClient.GetSASURL(permissions, expiry, nil) + _require.Nil(err) + + fClient2, _ := file.NewClientWithNoCredential(sasURL, nil) + + getACLResp, err := fClient2.GetAccessControl(context.Background(), nil) + _require.Nil(err) + _require.Equal(acl, *getACLResp.ACL) +} + +func (s *RecordedTestSuite) TestDeleteWithSAS() { + _require := require.New(s.T()) + testName := s.T().Name() + + filesystemName := testcommon.GenerateFilesystemName(testName) + fsClient, err := testcommon.GetFilesystemClient(filesystemName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + defer testcommon.DeleteFilesystem(context.Background(), _require, fsClient) + + _, err = fsClient.Create(context.Background(), nil) + _require.Nil(err) + + fileName := testcommon.GenerateFileName(testName) + fClient, err := testcommon.GetFileClient(filesystemName, fileName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + + resp, err := fClient.Create(context.Background(), nil) + _require.Nil(err) + _require.NotNil(resp) + + // Adding SAS and options + permissions := sas.FilePermissions{ + Read: true, + Add: true, + Write: true, + Create: true, + Delete: true, + } + expiry := time.Now().Add(time.Hour) + + sasURL, err := fClient.GetSASURL(permissions, expiry, nil) + _require.Nil(err) + + fClient2, _ := file.NewClientWithNoCredential(sasURL, nil) + + _, err = fClient2.Delete(context.Background(), nil) + _require.Nil(err) +} + +func (s *RecordedTestSuite) TestGetAccessControlWithNilAccessConditions() { + _require := require.New(s.T()) + testName := s.T().Name() + + filesystemName := testcommon.GenerateFilesystemName(testName) + fsClient, err := testcommon.GetFilesystemClient(filesystemName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + acl := "user::rwx,group::r-x,other::rwx" + defer testcommon.DeleteFilesystem(context.Background(), _require, fsClient) + + createOpts := &file.CreateOptions{ + ACL: &acl, + } + _, err = fsClient.Create(context.Background(), nil) + _require.Nil(err) + + fileName := testcommon.GenerateFileName(testName) + fClient, err := testcommon.GetFileClient(filesystemName, fileName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + + resp, err := fClient.Create(context.Background(), createOpts) + _require.Nil(err) + _require.NotNil(resp) + + opts := &file.GetAccessControlOptions{ + AccessConditions: nil, + } + + getACLResp, err := fClient.GetAccessControl(context.Background(), opts) + _require.Nil(err) + _require.Equal(acl, *getACLResp.ACL) +} + +func (s *RecordedTestSuite) TestGetAccessControlIfModifiedSinceTrue() { + _require := require.New(s.T()) + testName := s.T().Name() + + filesystemName := testcommon.GenerateFilesystemName(testName) + fsClient, err := testcommon.GetFilesystemClient(filesystemName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + acl := "user::rwx,group::r-x,other::rwx" + defer testcommon.DeleteFilesystem(context.Background(), _require, fsClient) + + createOpts := &file.CreateOptions{ + ACL: &acl, + } + _, err = fsClient.Create(context.Background(), nil) + _require.Nil(err) + + fileName := testcommon.GenerateFileName(testName) + fClient, err := testcommon.GetFileClient(filesystemName, fileName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + + resp, err := fClient.Create(context.Background(), createOpts) + _require.Nil(err) + _require.NotNil(resp) + + currentTime := testcommon.GetRelativeTimeFromAnchor(resp.Date, -10) + opts := &file.GetAccessControlOptions{ + AccessConditions: &file.AccessConditions{ + ModifiedAccessConditions: &file.ModifiedAccessConditions{ + IfModifiedSince: ¤tTime, + }, + }, + } + + getACLResp, err := fClient.GetAccessControl(context.Background(), opts) + _require.Nil(err) + _require.Equal(acl, *getACLResp.ACL) +} + +func (s *RecordedTestSuite) TestGetAccessControlIfModifiedSinceFalse() { + _require := require.New(s.T()) + testName := s.T().Name() + + filesystemName := testcommon.GenerateFilesystemName(testName) + fsClient, err := testcommon.GetFilesystemClient(filesystemName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + acl := "user::rwx,group::r-x,other::rwx" + defer testcommon.DeleteFilesystem(context.Background(), _require, fsClient) + + createOpts := &file.CreateOptions{ + ACL: &acl, + } + _, err = fsClient.Create(context.Background(), nil) + _require.Nil(err) + + fileName := testcommon.GenerateFileName(testName) + fClient, err := testcommon.GetFileClient(filesystemName, fileName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + + resp, err := fClient.Create(context.Background(), createOpts) + _require.Nil(err) + _require.NotNil(resp) + currentTime := testcommon.GetRelativeTimeFromAnchor(resp.Date, 10) + + opts := &file.GetAccessControlOptions{ + AccessConditions: &file.AccessConditions{ + ModifiedAccessConditions: &file.ModifiedAccessConditions{ + IfModifiedSince: ¤tTime, + }, + }, + } + + _, err = fClient.GetAccessControl(context.Background(), opts) + _require.NotNil(err) + testcommon.ValidateErrorCode(_require, err, datalakeerror.ConditionNotMet) +} + +func (s *RecordedTestSuite) TestGetAccessControlIfUnmodifiedSinceTrue() { + _require := require.New(s.T()) + testName := s.T().Name() + + filesystemName := testcommon.GenerateFilesystemName(testName) + fsClient, err := testcommon.GetFilesystemClient(filesystemName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + acl := "user::rwx,group::r-x,other::rwx" + defer testcommon.DeleteFilesystem(context.Background(), _require, fsClient) + + createOpts := &file.CreateOptions{ + ACL: &acl, + } + _, err = fsClient.Create(context.Background(), nil) + _require.Nil(err) + + fileName := testcommon.GenerateFileName(testName) + fClient, err := testcommon.GetFileClient(filesystemName, fileName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + + resp, err := fClient.Create(context.Background(), createOpts) + _require.Nil(err) + _require.NotNil(resp) + + currentTime := testcommon.GetRelativeTimeFromAnchor(resp.Date, 10) + opts := &file.GetAccessControlOptions{ + AccessConditions: &file.AccessConditions{ + ModifiedAccessConditions: &file.ModifiedAccessConditions{ + IfUnmodifiedSince: ¤tTime, + }, + }} + + getACLResp, err := fClient.GetAccessControl(context.Background(), opts) + _require.Nil(err) + _require.Equal(acl, *getACLResp.ACL) +} + +func (s *RecordedTestSuite) TestGetAccessControlIfUnmodifiedSinceFalse() { + _require := require.New(s.T()) + testName := s.T().Name() + + filesystemName := testcommon.GenerateFilesystemName(testName) + fsClient, err := testcommon.GetFilesystemClient(filesystemName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + acl := "user::rwx,group::r-x,other::rwx" + defer testcommon.DeleteFilesystem(context.Background(), _require, fsClient) + + createOpts := &file.CreateOptions{ + ACL: &acl, + } + _, err = fsClient.Create(context.Background(), nil) + _require.Nil(err) + + fileName := testcommon.GenerateFileName(testName) + fClient, err := testcommon.GetFileClient(filesystemName, fileName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + + resp, err := fClient.Create(context.Background(), createOpts) + _require.Nil(err) + _require.NotNil(resp) + + currentTime := testcommon.GetRelativeTimeFromAnchor(resp.Date, -10) + + opts := &file.GetAccessControlOptions{ + AccessConditions: &file.AccessConditions{ + ModifiedAccessConditions: &file.ModifiedAccessConditions{ + IfUnmodifiedSince: ¤tTime, + }, + }, + } + + _, err = fClient.GetAccessControl(context.Background(), opts) + _require.NotNil(err) + + testcommon.ValidateErrorCode(_require, err, datalakeerror.ConditionNotMet) +} + +func (s *RecordedTestSuite) TestGetAccessControlIfETagMatch() { + _require := require.New(s.T()) + testName := s.T().Name() + + filesystemName := testcommon.GenerateFilesystemName(testName) + fsClient, err := testcommon.GetFilesystemClient(filesystemName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + acl := "user::rwx,group::r-x,other::rwx" + defer testcommon.DeleteFilesystem(context.Background(), _require, fsClient) + + createOpts := &file.CreateOptions{ + ACL: &acl, + } + _, err = fsClient.Create(context.Background(), nil) + _require.Nil(err) + + fileName := testcommon.GenerateFileName(testName) + fClient, err := testcommon.GetFileClient(filesystemName, fileName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + + resp, err := fClient.Create(context.Background(), createOpts) + _require.Nil(err) + _require.NotNil(resp) + etag := resp.ETag + + opts := &file.GetAccessControlOptions{ + AccessConditions: &file.AccessConditions{ + ModifiedAccessConditions: &file.ModifiedAccessConditions{ + IfMatch: etag, + }, + }, + } + + getACLResp, err := fClient.GetAccessControl(context.Background(), opts) + _require.Nil(err) + _require.Equal(acl, *getACLResp.ACL) +} + +func (s *RecordedTestSuite) TestGetAccessControlIfETagMatchFalse() { + _require := require.New(s.T()) + testName := s.T().Name() + + filesystemName := testcommon.GenerateFilesystemName(testName) + fsClient, err := testcommon.GetFilesystemClient(filesystemName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + acl := "user::rwx,group::r-x,other::rwx" + defer testcommon.DeleteFilesystem(context.Background(), _require, fsClient) + + createOpts := &file.CreateOptions{ + ACL: &acl, + } + _, err = fsClient.Create(context.Background(), nil) + _require.Nil(err) + + fileName := testcommon.GenerateFileName(testName) + fClient, err := testcommon.GetFileClient(filesystemName, fileName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + + resp, err := fClient.Create(context.Background(), createOpts) + _require.Nil(err) + _require.NotNil(resp) + + etag := resp.ETag + opts := &file.GetAccessControlOptions{ + AccessConditions: &file.AccessConditions{ + ModifiedAccessConditions: &file.ModifiedAccessConditions{ + IfNoneMatch: etag, + }, + }} + + _, err = fClient.GetAccessControl(context.Background(), opts) + _require.NotNil(err) + testcommon.ValidateErrorCode(_require, err, datalakeerror.ConditionNotMet) +} + +func (s *RecordedTestSuite) TestUpdateAccessControl() { + _require := require.New(s.T()) + testName := s.T().Name() + + filesystemName := testcommon.GenerateFilesystemName(testName) + fsClient, err := testcommon.GetFilesystemClient(filesystemName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + acl := "user::rwx,group::r-x,other::rwx" + acl1 := "user::rwx,group::r--,other::r--" + defer testcommon.DeleteFilesystem(context.Background(), _require, fsClient) + + createOpts := &file.CreateOptions{ + ACL: &acl, + } + _, err = fsClient.Create(context.Background(), nil) + _require.Nil(err) + + fileName := testcommon.GenerateFileName(testName) + fClient, err := testcommon.GetFileClient(filesystemName, fileName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + + resp, err := fClient.Create(context.Background(), createOpts) + _require.Nil(err) + _require.NotNil(resp) + + _, err = fClient.UpdateAccessControl(context.Background(), acl1, nil) + _require.Nil(err) + + getACLResp, err := fClient.GetAccessControl(context.Background(), nil) + _require.Nil(err) + _require.Equal(acl1, *getACLResp.ACL) +} + +func (s *RecordedTestSuite) TestRemoveAccessControl() { + _require := require.New(s.T()) + testName := s.T().Name() + + filesystemName := testcommon.GenerateFilesystemName(testName) + fsClient, err := testcommon.GetFilesystemClient(filesystemName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + acl := "mask," + "default:user,default:group," + + "user:ec3595d6-2c17-4696-8caa-7e139758d24a,group:ec3595d6-2c17-4696-8caa-7e139758d24a," + + "default:user:ec3595d6-2c17-4696-8caa-7e139758d24a,default:group:ec3595d6-2c17-4696-8caa-7e139758d24a" + defer testcommon.DeleteFilesystem(context.Background(), _require, fsClient) + + _, err = fsClient.Create(context.Background(), nil) + _require.Nil(err) + + fileName := testcommon.GenerateFileName(testName) + fClient, err := testcommon.GetFileClient(filesystemName, fileName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + + resp, err := fClient.Create(context.Background(), nil) + _require.Nil(err) + _require.NotNil(resp) + + _, err = fClient.RemoveAccessControl(context.Background(), acl, nil) + _require.Nil(err) +} + +func (s *RecordedTestSuite) TestSetMetadataNil() { + _require := require.New(s.T()) + testName := s.T().Name() + + filesystemName := testcommon.GenerateFilesystemName(testName) + fsClient, err := testcommon.GetFilesystemClient(filesystemName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + defer testcommon.DeleteFilesystem(context.Background(), _require, fsClient) + + _, err = fsClient.Create(context.Background(), nil) + _require.Nil(err) + + fileName := testcommon.GenerateFileName(testName) + fClient, err := testcommon.GetFileClient(filesystemName, fileName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + + defer testcommon.DeleteFile(context.Background(), _require, fClient) + + resp, err := fClient.Create(context.Background(), nil) + _require.Nil(err) + _require.NotNil(resp) + + _, err = fClient.SetMetadata(context.Background(), nil) + _require.Nil(err) +} + +func (s *RecordedTestSuite) TestSetMetadataWithEmptyOpts() { + _require := require.New(s.T()) + testName := s.T().Name() + + filesystemName := testcommon.GenerateFilesystemName(testName) + fsClient, err := testcommon.GetFilesystemClient(filesystemName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + defer testcommon.DeleteFilesystem(context.Background(), _require, fsClient) + + _, err = fsClient.Create(context.Background(), nil) + _require.Nil(err) + + fileName := testcommon.GenerateFileName(testName) + fClient, err := testcommon.GetFileClient(filesystemName, fileName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + + defer testcommon.DeleteFile(context.Background(), _require, fClient) + + resp, err := fClient.Create(context.Background(), nil) + _require.Nil(err) + _require.NotNil(resp) + + opts := &file.SetMetadataOptions{ + Metadata: nil, + } + _, err = fClient.SetMetadata(context.Background(), opts) + _require.Nil(err) +} + +func (s *RecordedTestSuite) TestSetMetadataWithBasicMetadata() { + _require := require.New(s.T()) + testName := s.T().Name() + + filesystemName := testcommon.GenerateFilesystemName(testName) + fsClient, err := testcommon.GetFilesystemClient(filesystemName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + defer testcommon.DeleteFilesystem(context.Background(), _require, fsClient) + + _, err = fsClient.Create(context.Background(), nil) + _require.Nil(err) + + fileName := testcommon.GenerateFileName(testName) + fClient, err := testcommon.GetFileClient(filesystemName, fileName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + + defer testcommon.DeleteFile(context.Background(), _require, fClient) + + resp, err := fClient.Create(context.Background(), nil) + _require.Nil(err) + _require.NotNil(resp) + + opts := &file.SetMetadataOptions{ + Metadata: testcommon.BasicMetadata, + } + _, err = fClient.SetMetadata(context.Background(), opts) + _require.Nil(err) +} + +func (s *RecordedTestSuite) TestSetMetadataWithAccessConditions() { + _require := require.New(s.T()) + testName := s.T().Name() + + filesystemName := testcommon.GenerateFilesystemName(testName) + fsClient, err := testcommon.GetFilesystemClient(filesystemName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + defer testcommon.DeleteFilesystem(context.Background(), _require, fsClient) + + _, err = fsClient.Create(context.Background(), nil) + _require.Nil(err) + + fileName := testcommon.GenerateFileName(testName) + fClient, err := testcommon.GetFileClient(filesystemName, fileName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + + defer testcommon.DeleteFile(context.Background(), _require, fClient) + + resp, err := fClient.Create(context.Background(), nil) + _require.Nil(err) + _require.NotNil(resp) + + currentTime := testcommon.GetRelativeTimeFromAnchor(resp.Date, -10) + + opts := &file.SetMetadataOptions{ + Metadata: testcommon.BasicMetadata, + AccessConditions: &file.AccessConditions{ + ModifiedAccessConditions: &file.ModifiedAccessConditions{ + IfModifiedSince: ¤tTime, + }, + }, + } + _, err = fClient.SetMetadata(context.Background(), opts) + _require.Nil(err) +} + +func validatePropertiesSet(_require *require.Assertions, fileClient *file.Client, disposition string) { + resp, err := fileClient.GetProperties(context.Background(), nil) + _require.Nil(err) + _require.Equal(*resp.ContentDisposition, disposition) +} + +func (s *RecordedTestSuite) TestSetHTTPHeaders() { + _require := require.New(s.T()) + testName := s.T().Name() + + filesystemName := testcommon.GenerateFilesystemName(testName) + fsClient, err := testcommon.GetFilesystemClient(filesystemName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + defer testcommon.DeleteFilesystem(context.Background(), _require, fsClient) + + _, err = fsClient.Create(context.Background(), nil) + _require.Nil(err) + + fileName := testcommon.GenerateFileName(testName) + fClient, err := testcommon.GetFileClient(filesystemName, fileName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + + resp, err := fClient.Create(context.Background(), nil) + _require.Nil(err) + _require.NotNil(resp) + + _, err = fClient.SetHTTPHeaders(context.Background(), testcommon.BasicHeaders, nil) + _require.Nil(err) + validatePropertiesSet(_require, fClient, *testcommon.BasicHeaders.ContentDisposition) +} + +func (s *RecordedTestSuite) TestSetHTTPHeadersWithNilAccessConditions() { + _require := require.New(s.T()) + testName := s.T().Name() + + filesystemName := testcommon.GenerateFilesystemName(testName) + fsClient, err := testcommon.GetFilesystemClient(filesystemName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + defer testcommon.DeleteFilesystem(context.Background(), _require, fsClient) + + _, err = fsClient.Create(context.Background(), nil) + _require.Nil(err) + + fileName := testcommon.GenerateFileName(testName) + fClient, err := testcommon.GetFileClient(filesystemName, fileName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + + resp, err := fClient.Create(context.Background(), nil) + _require.Nil(err) + _require.NotNil(resp) + + opts := &file.SetHTTPHeadersOptions{ + AccessConditions: nil, + } + + _, err = fClient.SetHTTPHeaders(context.Background(), testcommon.BasicHeaders, opts) + _require.Nil(err) + validatePropertiesSet(_require, fClient, *testcommon.BasicHeaders.ContentDisposition) +} + +func (s *RecordedTestSuite) TestSetHTTPHeadersIfModifiedSinceTrue() { + _require := require.New(s.T()) + testName := s.T().Name() + + filesystemName := testcommon.GenerateFilesystemName(testName) + fsClient, err := testcommon.GetFilesystemClient(filesystemName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + defer testcommon.DeleteFilesystem(context.Background(), _require, fsClient) + + _, err = fsClient.Create(context.Background(), nil) + _require.Nil(err) + + fileName := testcommon.GenerateFileName(testName) + fClient, err := testcommon.GetFileClient(filesystemName, fileName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + + resp, err := fClient.Create(context.Background(), nil) + _require.Nil(err) + _require.NotNil(resp) + + currentTime := testcommon.GetRelativeTimeFromAnchor(resp.Date, -10) + + opts := &file.SetHTTPHeadersOptions{ + AccessConditions: &file.AccessConditions{ + ModifiedAccessConditions: &file.ModifiedAccessConditions{ + IfModifiedSince: ¤tTime, + }, + }, + } + _, err = fClient.SetHTTPHeaders(context.Background(), testcommon.BasicHeaders, opts) + _require.Nil(err) + validatePropertiesSet(_require, fClient, *testcommon.BasicHeaders.ContentDisposition) +} + +func (s *RecordedTestSuite) TestSetHTTPHeadersIfModifiedSinceFalse() { + _require := require.New(s.T()) + testName := s.T().Name() + + filesystemName := testcommon.GenerateFilesystemName(testName) + fsClient, err := testcommon.GetFilesystemClient(filesystemName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + defer testcommon.DeleteFilesystem(context.Background(), _require, fsClient) + + _, err = fsClient.Create(context.Background(), nil) + _require.Nil(err) + + fileName := testcommon.GenerateFileName(testName) + fClient, err := testcommon.GetFileClient(filesystemName, fileName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + + resp, err := fClient.Create(context.Background(), nil) + _require.Nil(err) + _require.NotNil(resp) + + currentTime := testcommon.GetRelativeTimeFromAnchor(resp.Date, 10) + + opts := &file.SetHTTPHeadersOptions{ + AccessConditions: &file.AccessConditions{ + ModifiedAccessConditions: &file.ModifiedAccessConditions{ + IfModifiedSince: ¤tTime, + }, + }, + } + _, err = fClient.SetHTTPHeaders(context.Background(), testcommon.BasicHeaders, opts) + _require.NotNil(err) + testcommon.ValidateErrorCode(_require, err, datalakeerror.ConditionNotMet) +} + +func (s *RecordedTestSuite) TestSetHTTPHeadersIfUnmodifiedSinceTrue() { + _require := require.New(s.T()) + testName := s.T().Name() + + filesystemName := testcommon.GenerateFilesystemName(testName) + fsClient, err := testcommon.GetFilesystemClient(filesystemName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + defer testcommon.DeleteFilesystem(context.Background(), _require, fsClient) + + _, err = fsClient.Create(context.Background(), nil) + _require.Nil(err) + + fileName := testcommon.GenerateFileName(testName) + fClient, err := testcommon.GetFileClient(filesystemName, fileName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + + resp, err := fClient.Create(context.Background(), nil) + _require.Nil(err) + _require.NotNil(resp) + + currentTime := testcommon.GetRelativeTimeFromAnchor(resp.Date, 10) + + opts := &file.SetHTTPHeadersOptions{ + AccessConditions: &file.AccessConditions{ + ModifiedAccessConditions: &file.ModifiedAccessConditions{ + IfUnmodifiedSince: ¤tTime, + }, + }, + } + _, err = fClient.SetHTTPHeaders(context.Background(), testcommon.BasicHeaders, opts) + _require.Nil(err) + validatePropertiesSet(_require, fClient, *testcommon.BasicHeaders.ContentDisposition) +} + +func (s *RecordedTestSuite) TestSetHTTPHeadersIfUnmodifiedSinceFalse() { + _require := require.New(s.T()) + testName := s.T().Name() + + filesystemName := testcommon.GenerateFilesystemName(testName) + fsClient, err := testcommon.GetFilesystemClient(filesystemName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + defer testcommon.DeleteFilesystem(context.Background(), _require, fsClient) + + _, err = fsClient.Create(context.Background(), nil) + _require.Nil(err) + + fileName := testcommon.GenerateFileName(testName) + fClient, err := testcommon.GetFileClient(filesystemName, fileName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + + resp, err := fClient.Create(context.Background(), nil) + _require.Nil(err) + _require.NotNil(resp) + + currentTime := testcommon.GetRelativeTimeFromAnchor(resp.Date, -10) + + opts := &file.SetHTTPHeadersOptions{ + AccessConditions: &file.AccessConditions{ + ModifiedAccessConditions: &file.ModifiedAccessConditions{ + IfUnmodifiedSince: ¤tTime, + }, + }, + } + _, err = fClient.SetHTTPHeaders(context.Background(), testcommon.BasicHeaders, opts) + _require.NotNil(err) + testcommon.ValidateErrorCode(_require, err, datalakeerror.ConditionNotMet) +} + +func (s *RecordedTestSuite) TestSetHTTPHeadersIfETagMatch() { + _require := require.New(s.T()) + testName := s.T().Name() + + filesystemName := testcommon.GenerateFilesystemName(testName) + fsClient, err := testcommon.GetFilesystemClient(filesystemName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + defer testcommon.DeleteFilesystem(context.Background(), _require, fsClient) + + _, err = fsClient.Create(context.Background(), nil) + _require.Nil(err) + + fileName := testcommon.GenerateFileName(testName) + fClient, err := testcommon.GetFileClient(filesystemName, fileName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + + resp, err := fClient.Create(context.Background(), nil) + _require.Nil(err) + _require.NotNil(resp) + + etag := resp.ETag + + opts := &file.SetHTTPHeadersOptions{ + AccessConditions: &file.AccessConditions{ + ModifiedAccessConditions: &file.ModifiedAccessConditions{ + IfMatch: etag, + }, + }} + _, err = fClient.SetHTTPHeaders(context.Background(), testcommon.BasicHeaders, opts) + _require.Nil(err) + validatePropertiesSet(_require, fClient, *testcommon.BasicHeaders.ContentDisposition) +} + +func (s *RecordedTestSuite) TestSetHTTPHeadersIfETagMatchFalse() { + _require := require.New(s.T()) + testName := s.T().Name() + + filesystemName := testcommon.GenerateFilesystemName(testName) + fsClient, err := testcommon.GetFilesystemClient(filesystemName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + defer testcommon.DeleteFilesystem(context.Background(), _require, fsClient) + + _, err = fsClient.Create(context.Background(), nil) + _require.Nil(err) + + fileName := testcommon.GenerateFileName(testName) + fClient, err := testcommon.GetFileClient(filesystemName, fileName, s.T(), testcommon.TestAccountDatalake, nil) + _require.NoError(err) + + resp, err := fClient.Create(context.Background(), nil) + _require.Nil(err) + _require.NotNil(resp) + + etag := resp.ETag + + opts := &file.SetHTTPHeadersOptions{ + AccessConditions: &file.AccessConditions{ + ModifiedAccessConditions: &file.ModifiedAccessConditions{ + IfNoneMatch: etag, + }, + }, + } + _, err = fClient.SetHTTPHeaders(context.Background(), testcommon.BasicHeaders, opts) + _require.NotNil(err) + testcommon.ValidateErrorCode(_require, err, datalakeerror.ConditionNotMet) +} + +//func (s *RecordedTestSuite) TestRenameNoOptions() { +// _require := require.New(s.T()) +// testName := s.T().Name() +// +// filesystemName := testcommon.GenerateFilesystemName(testName) +// fsClient, err := testcommon.GetFilesystemClient(filesystemName, s.T(), testcommon.TestAccountDatalake, nil) +// _require.NoError(err) +// defer testcommon.DeleteFilesystem(context.Background(), _require, fsClient) +// +// _, err = fsClient.Create(context.Background(), nil) +// _require.Nil(err) +// +// fileName := testcommon.GenerateFileName(testName) +// fClient, err := testcommon.GetFileClient(filesystemName, fileName, s.T(), testcommon.TestAccountDatalake, nil) +// _require.NoError(err) +// +// resp, err := fClient.Create(context.Background(), nil) +// _require.Nil(err) +// _require.NotNil(resp) +// +// resp, err = fClient.Rename(context.Background(), "newName", nil) +// _require.Nil(err) +// _require.NotNil(resp) +//} +// +//func (s *RecordedTestSuite) TestRenameFileWithNilAccessConditions() { +// _require := require.New(s.T()) +// testName := s.T().Name() +// +// filesystemName := testcommon.GenerateFilesystemName(testName) +// fsClient, err := testcommon.GetFilesystemClient(filesystemName, s.T(), testcommon.TestAccountDatalake, nil) +// _require.NoError(err) +// defer testcommon.DeleteFilesystem(context.Background(), _require, fsClient) +// +// _, err = fsClient.Create(context.Background(), nil) +// _require.Nil(err) +// +// fileName := testcommon.GenerateFileName(testName) +// fClient, err := testcommon.GetFileClient(filesystemName, fileName, s.T(), testcommon.TestAccountDatalake, nil) +// _require.NoError(err) +// +// resp, err := fClient.Create(context.Background(), nil) +// _require.Nil(err) +// _require.NotNil(resp) +// +// renameFileOpts := &file.RenameOptions{ +// AccessConditions: nil, +// } +// +// resp, err = fClient.Rename(context.Background(), "new"+fileName, renameFileOpts) +// _require.Nil(err) +// _require.NotNil(resp) +//} +// +//func (s *RecordedTestSuite) TestRenameFileIfModifiedSinceTrue() { +// _require := require.New(s.T()) +// testName := s.T().Name() +// +// filesystemName := testcommon.GenerateFilesystemName(testName) +// fsClient, err := testcommon.GetFilesystemClient(filesystemName, s.T(), testcommon.TestAccountDatalake, nil) +// _require.NoError(err) +// defer testcommon.DeleteFilesystem(context.Background(), _require, fsClient) +// +// _, err = fsClient.Create(context.Background(), nil) +// _require.Nil(err) +// +// fileName := testcommon.GenerateFileName(testName) +// fClient, err := testcommon.GetFileClient(filesystemName, fileName, s.T(), testcommon.TestAccountDatalake, nil) +// _require.NoError(err) +// +// defer testcommon.DeleteFile(context.Background(), _require, fClient) +// +// resp, err := fClient.Create(context.Background(), nil) +// _require.Nil(err) +// _require.NotNil(resp) +// +// currentTime := testcommon.GetRelativeTimeFromAnchor(resp.Date, -10) +// +// createFileOpts := &file.CreateOptions{ +// AccessConditions: &file.AccessConditions{ +// ModifiedAccessConditions: &file.ModifiedAccessConditions{ +// IfModifiedSince: ¤tTime, +// }, +// }, +// } +// +// resp, err = fClient.Create(context.Background(), createFileOpts) +// _require.Nil(err) +// _require.NotNil(resp) +//} +// +//func (s *RecordedTestSuite) TestRenameFileIfModifiedSinceFalse() { +// _require := require.New(s.T()) +// testName := s.T().Name() +// +// filesystemName := testcommon.GenerateFilesystemName(testName) +// fsClient, err := testcommon.GetFilesystemClient(filesystemName, s.T(), testcommon.TestAccountDatalake, nil) +// _require.NoError(err) +// defer testcommon.DeleteFilesystem(context.Background(), _require, fsClient) +// +// _, err = fsClient.Create(context.Background(), nil) +// _require.Nil(err) +// +// fileName := testcommon.GenerateFileName(testName) +// fClient, err := testcommon.GetFileClient(filesystemName, fileName, s.T(), testcommon.TestAccountDatalake, nil) +// _require.NoError(err) +// +// defer testcommon.DeleteFile(context.Background(), _require, fClient) +// +// resp, err := fClient.Create(context.Background(), nil) +// _require.Nil(err) +// _require.NotNil(resp) +// +// currentTime := testcommon.GetRelativeTimeFromAnchor(resp.Date, 10) +// +// createFileOpts := &file.CreateOptions{ +// AccessConditions: &file.AccessConditions{ +// ModifiedAccessConditions: &file.ModifiedAccessConditions{ +// IfModifiedSince: ¤tTime, +// }, +// }, +// } +// +// resp, err = fClient.Create(context.Background(), createFileOpts) +// _require.NotNil(err) +// testcommon.ValidateErrorCode(_require, err, datalakeerror.ConditionNotMet) +//} +// +//func (s *RecordedTestSuite) TestRenameFileIfUnmodifiedSinceTrue() { +// _require := require.New(s.T()) +// testName := s.T().Name() +// +// filesystemName := testcommon.GenerateFilesystemName(testName) +// fsClient, err := testcommon.GetFilesystemClient(filesystemName, s.T(), testcommon.TestAccountDatalake, nil) +// _require.NoError(err) +// defer testcommon.DeleteFilesystem(context.Background(), _require, fsClient) +// +// _, err = fsClient.Create(context.Background(), nil) +// _require.Nil(err) +// +// fileName := testcommon.GenerateFileName(testName) +// fClient, err := testcommon.GetFileClient(filesystemName, fileName, s.T(), testcommon.TestAccountDatalake, nil) +// _require.NoError(err) +// +// defer testcommon.DeleteFile(context.Background(), _require, fClient) +// +// resp, err := fClient.Create(context.Background(), nil) +// _require.Nil(err) +// _require.NotNil(resp) +// +// currentTime := testcommon.GetRelativeTimeFromAnchor(resp.Date, 10) +// +// createFileOpts := &file.CreateOptions{ +// AccessConditions: &file.AccessConditions{ +// ModifiedAccessConditions: &file.ModifiedAccessConditions{ +// IfUnmodifiedSince: ¤tTime, +// }, +// }, +// } +// +// resp, err = fClient.Create(context.Background(), createFileOpts) +// _require.Nil(err) +// _require.NotNil(resp) +//} +// +//func (s *RecordedTestSuite) TestRenameFileIfUnmodifiedSinceFalse() { +// _require := require.New(s.T()) +// testName := s.T().Name() +// +// filesystemName := testcommon.GenerateFilesystemName(testName) +// fsClient, err := testcommon.GetFilesystemClient(filesystemName, s.T(), testcommon.TestAccountDatalake, nil) +// _require.NoError(err) +// defer testcommon.DeleteFilesystem(context.Background(), _require, fsClient) +// +// _, err = fsClient.Create(context.Background(), nil) +// _require.Nil(err) +// +// fileName := testcommon.GenerateFileName(testName) +// fClient, err := testcommon.GetFileClient(filesystemName, fileName, s.T(), testcommon.TestAccountDatalake, nil) +// _require.NoError(err) +// +// defer testcommon.DeleteFile(context.Background(), _require, fClient) +// +// resp, err := fClient.Create(context.Background(), nil) +// _require.Nil(err) +// _require.NotNil(resp) +// +// currentTime := testcommon.GetRelativeTimeFromAnchor(resp.Date, -10) +// +// createFileOpts := &file.CreateOptions{ +// AccessConditions: &file.AccessConditions{ +// ModifiedAccessConditions: &file.ModifiedAccessConditions{ +// IfUnmodifiedSince: ¤tTime, +// }, +// }, +// } +// +// resp, err = fClient.Create(context.Background(), createFileOpts) +// _require.NotNil(err) +// +// testcommon.ValidateErrorCode(_require, err, datalakeerror.ConditionNotMet) +//} +// +//func (s *RecordedTestSuite) TestRenameFileIfETagMatch() { +// _require := require.New(s.T()) +// testName := s.T().Name() +// +// filesystemName := testcommon.GenerateFilesystemName(testName) +// fsClient, err := testcommon.GetFilesystemClient(filesystemName, s.T(), testcommon.TestAccountDatalake, nil) +// _require.NoError(err) +// defer testcommon.DeleteFilesystem(context.Background(), _require, fsClient) +// +// _, err = fsClient.Create(context.Background(), nil) +// _require.Nil(err) +// +// fileName := testcommon.GenerateFileName(testName) +// fClient, err := testcommon.GetFileClient(filesystemName, fileName, s.T(), testcommon.TestAccountDatalake, nil) +// _require.NoError(err) +// +// defer testcommon.DeleteFile(context.Background(), _require, fClient) +// +// resp, err := fClient.Create(context.Background(), nil) +// _require.Nil(err) +// _require.NotNil(resp) +// +// etag := resp.ETag +// +// createFileOpts := &file.CreateOptions{ +// AccessConditions: &file.AccessConditions{ +// ModifiedAccessConditions: &file.ModifiedAccessConditions{ +// IfMatch: etag, +// }, +// }, +// } +// +// resp, err = fClient.Create(context.Background(), createFileOpts) +// _require.Nil(err) +// _require.NotNil(resp) +//} +// +//func (s *RecordedTestSuite) TestRenameFileIfETagMatchFalse() { +// _require := require.New(s.T()) +// testName := s.T().Name() +// +// filesystemName := testcommon.GenerateFilesystemName(testName) +// fsClient, err := testcommon.GetFilesystemClient(filesystemName, s.T(), testcommon.TestAccountDatalake, nil) +// _require.NoError(err) +// defer testcommon.DeleteFilesystem(context.Background(), _require, fsClient) +// +// _, err = fsClient.Create(context.Background(), nil) +// _require.Nil(err) +// +// fileName := testcommon.GenerateFileName(testName) +// fClient, err := testcommon.GetFileClient(filesystemName, fileName, s.T(), testcommon.TestAccountDatalake, nil) +// _require.NoError(err) +// +// defer testcommon.DeleteFile(context.Background(), _require, fClient) +// +// resp, err := fClient.Create(context.Background(), nil) +// _require.Nil(err) +// _require.NotNil(resp) +// +// etag := resp.ETag +// +// createFileOpts := &file.CreateOptions{ +// AccessConditions: &file.AccessConditions{ +// ModifiedAccessConditions: &file.ModifiedAccessConditions{ +// IfNoneMatch: etag, +// }, +// }, +// } +// +// resp, err = fClient.Create(context.Background(), createFileOpts) +// _require.NotNil(err) +// +// testcommon.ValidateErrorCode(_require, err, datalakeerror.ConditionNotMet) +//} diff --git a/sdk/storage/azdatalake/file/constants.go b/sdk/storage/azdatalake/file/constants.go index 60eabcfcce37..c536d01673cf 100644 --- a/sdk/storage/azdatalake/file/constants.go +++ b/sdk/storage/azdatalake/file/constants.go @@ -8,36 +8,55 @@ package file import ( "github.com/Azure/azure-sdk-for-go/sdk/storage/azblob/blob" - "github.com/Azure/azure-sdk-for-go/sdk/storage/azdatalake/internal/generated" + "github.com/Azure/azure-sdk-for-go/sdk/storage/azblob/lease" ) -type ResourceType = generated.PathResourceType +type EncryptionAlgorithmType = blob.EncryptionAlgorithmType -// TODO: consider the possibility of not exposing this and just pass it under the hood const ( - ResourceTypeFile ResourceType = generated.PathResourceTypeFile - ResourceTypeDirectory ResourceType = generated.PathResourceTypeDirectory + EncryptionAlgorithmTypeNone EncryptionAlgorithmType = blob.EncryptionAlgorithmTypeNone + EncryptionAlgorithmTypeAES256 EncryptionAlgorithmType = blob.EncryptionAlgorithmTypeAES256 ) -type RenameMode = generated.PathRenameMode +// responses models: + +type ImmutabilityPolicyMode = blob.ImmutabilityPolicyMode -// TODO: consider the possibility of not exposing this and just pass it under the hood const ( - RenameModeLegacy RenameMode = generated.PathRenameModeLegacy - RenameModePosix RenameMode = generated.PathRenameModePosix + ImmutabilityPolicyModeMutable ImmutabilityPolicyMode = blob.ImmutabilityPolicyModeMutable + ImmutabilityPolicyModeUnlocked ImmutabilityPolicyMode = blob.ImmutabilityPolicyModeUnlocked + ImmutabilityPolicyModeLocked ImmutabilityPolicyMode = blob.ImmutabilityPolicyModeLocked ) -type SetAccessControlRecursiveMode = generated.PathSetAccessControlRecursiveMode +type CopyStatusType = blob.CopyStatusType const ( - SetAccessControlRecursiveModeSet SetAccessControlRecursiveMode = generated.PathSetAccessControlRecursiveModeSet - SetAccessControlRecursiveModeModify SetAccessControlRecursiveMode = generated.PathSetAccessControlRecursiveModeModify - SetAccessControlRecursiveModeRemove SetAccessControlRecursiveMode = generated.PathSetAccessControlRecursiveModeRemove + CopyStatusTypePending CopyStatusType = blob.CopyStatusTypePending + CopyStatusTypeSuccess CopyStatusType = blob.CopyStatusTypeSuccess + CopyStatusTypeAborted CopyStatusType = blob.CopyStatusTypeAborted + CopyStatusTypeFailed CopyStatusType = blob.CopyStatusTypeFailed ) -type EncryptionAlgorithmType = blob.EncryptionAlgorithmType +type LeaseDurationType = lease.DurationType const ( - EncryptionAlgorithmTypeNone EncryptionAlgorithmType = blob.EncryptionAlgorithmTypeNone - EncryptionAlgorithmTypeAES256 EncryptionAlgorithmType = blob.EncryptionAlgorithmTypeAES256 + LeaseDurationTypeInfinite LeaseDurationType = lease.DurationTypeInfinite + LeaseDurationTypeFixed LeaseDurationType = lease.DurationTypeFixed +) + +type LeaseStateType = lease.StateType + +const ( + LeaseStateTypeAvailable LeaseStateType = lease.StateTypeAvailable + LeaseStateTypeLeased LeaseStateType = lease.StateTypeLeased + LeaseStateTypeExpired LeaseStateType = lease.StateTypeExpired + LeaseStateTypeBreaking LeaseStateType = lease.StateTypeBreaking + LeaseStateTypeBroken LeaseStateType = lease.StateTypeBroken +) + +type LeaseStatusType = lease.StatusType + +const ( + LeaseStatusTypeLocked LeaseStatusType = lease.StatusTypeLocked + LeaseStatusTypeUnlocked LeaseStatusType = lease.StatusTypeUnlocked ) diff --git a/sdk/storage/azdatalake/file/models.go b/sdk/storage/azdatalake/file/models.go index d05d0953d430..af88f45bd326 100644 --- a/sdk/storage/azdatalake/file/models.go +++ b/sdk/storage/azdatalake/file/models.go @@ -7,27 +7,33 @@ package file import ( + "errors" + "github.com/Azure/azure-sdk-for-go/sdk/azcore/to" "github.com/Azure/azure-sdk-for-go/sdk/storage/azblob/blob" + "github.com/Azure/azure-sdk-for-go/sdk/storage/azdatalake/datalakeerror" "github.com/Azure/azure-sdk-for-go/sdk/storage/azdatalake/internal/exported" "github.com/Azure/azure-sdk-for-go/sdk/storage/azdatalake/internal/generated" + "net/http" + "strconv" "time" ) -// CreateOptions contains the optional parameters when calling the Create operation. dfs endpoint. TODO: Design formatter +// CreateOptions contains the optional parameters when calling the Create operation. dfs endpoint. type CreateOptions struct { // AccessConditions contains parameters for accessing the file. AccessConditions *AccessConditions // Metadata is a map of name-value pairs to associate with the file storage object. Metadata map[string]*string // CPKInfo contains a group of parameters for client provided encryption key. - CPKInfo CPKInfo + CPKInfo *CPKInfo // HTTPHeaders contains the HTTP headers for path operations. - HTTPHeaders HTTPHeaders - //PathExpiryOptions *ExpiryOptions - // ExpiresOn specifies the time that the file will expire. - ExpiresOn *time.Time - // LeaseDuration specifies the duration of the lease. - LeaseDuration *time.Duration + HTTPHeaders *HTTPHeaders + // Expiry specifies the type and time of expiry for the file. + Expiry CreationExpiryType + // LeaseDuration specifies the duration of the lease, in seconds, or negative one + // (-1) for a lease that never expires. A non-infinite lease can be + // between 15 and 60 seconds. + LeaseDuration *int64 // ProposedLeaseID specifies the proposed lease ID for the file. ProposedLeaseID *string // Permissions is the octal representation of the permissions for user, group and mask. @@ -42,19 +48,45 @@ type CreateOptions struct { ACL *string } -func (o *CreateOptions) format() (*generated.LeaseAccessConditions, *generated.ModifiedAccessConditions, *generated.PathHTTPHeaders, error) { - // TODO: add all other required options for the create operation, we don't need sourceModAccCond since this is not rename +func (o *CreateOptions) format() (*generated.LeaseAccessConditions, *generated.ModifiedAccessConditions, *generated.PathHTTPHeaders, *generated.PathClientCreateOptions, *generated.CPKInfo) { + resource := generated.PathResourceTypeFile + createOpts := &generated.PathClientCreateOptions{ + Resource: &resource, + } + if o == nil { + return nil, nil, nil, createOpts, nil + } leaseAccessConditions, modifiedAccessConditions := exported.FormatPathAccessConditions(o.AccessConditions) - httpHeaders := &generated.PathHTTPHeaders{ - CacheControl: o.HTTPHeaders.CacheControl, - ContentDisposition: o.HTTPHeaders.ContentDisposition, - ContentEncoding: o.HTTPHeaders.ContentEncoding, - ContentLanguage: o.HTTPHeaders.ContentLanguage, - ContentMD5: o.HTTPHeaders.ContentMD5, - ContentType: o.HTTPHeaders.ContentType, - TransactionalContentHash: o.HTTPHeaders.ContentMD5, + if o.Expiry == nil { + createOpts.ExpiryOptions = nil + createOpts.ExpiresOn = nil + } else { + expOpts, expiresOn := o.Expiry.Format() + createOpts.ExpiryOptions = (*generated.PathExpiryOptions)(&expOpts) + createOpts.ExpiresOn = expiresOn + } + createOpts.ACL = o.ACL + createOpts.Group = o.Group + createOpts.Owner = o.Owner + createOpts.Umask = o.Umask + createOpts.Permissions = o.Permissions + createOpts.ProposedLeaseID = o.ProposedLeaseID + createOpts.LeaseDuration = o.LeaseDuration + + var httpHeaders *generated.PathHTTPHeaders + var cpkOpts *generated.CPKInfo + + if o.HTTPHeaders != nil { + httpHeaders = o.HTTPHeaders.formatPathHTTPHeaders() + } + if o.CPKInfo != nil { + cpkOpts = &generated.CPKInfo{ + EncryptionAlgorithm: (*generated.EncryptionAlgorithmType)(o.CPKInfo.EncryptionAlgorithm), + EncryptionKey: o.CPKInfo.EncryptionKey, + EncryptionKeySHA256: o.CPKInfo.EncryptionKeySHA256, + } } - return leaseAccessConditions, modifiedAccessConditions, httpHeaders, nil + return leaseAccessConditions, modifiedAccessConditions, httpHeaders, createOpts, cpkOpts } // DeleteOptions contains the optional parameters when calling the Delete operation. dfs endpoint @@ -63,19 +95,52 @@ type DeleteOptions struct { AccessConditions *AccessConditions } -func (o *DeleteOptions) format() (*generated.LeaseAccessConditions, *generated.ModifiedAccessConditions, error) { +func (o *DeleteOptions) format() (*generated.LeaseAccessConditions, *generated.ModifiedAccessConditions, *generated.PathClientDeleteOptions) { + recursive := false + deleteOpts := &generated.PathClientDeleteOptions{ + Recursive: &recursive, + } + if o == nil { + return nil, nil, deleteOpts + } leaseAccessConditions, modifiedAccessConditions := exported.FormatPathAccessConditions(o.AccessConditions) - return leaseAccessConditions, modifiedAccessConditions, nil + return leaseAccessConditions, modifiedAccessConditions, deleteOpts } -// RenameOptions contains the optional parameters when calling the Rename operation. TODO: Design formatter +// RenameOptions contains the optional parameters when calling the Rename operation. type RenameOptions struct { - // SourceModifiedAccessConditions identifies the source path access conditions. - SourceModifiedAccessConditions *SourceModifiedAccessConditions + // SourceAccessConditions identifies the source path access conditions. + SourceAccessConditions *SourceAccessConditions // AccessConditions contains parameters for accessing the file. AccessConditions *AccessConditions } +func (o *RenameOptions) format(path string) (*generated.LeaseAccessConditions, *generated.ModifiedAccessConditions, *generated.SourceModifiedAccessConditions, *generated.PathClientCreateOptions) { + // we don't need sourceModAccCond since this is not rename + mode := generated.PathRenameModeLegacy + createOpts := &generated.PathClientCreateOptions{ + Mode: &mode, + RenameSource: &path, + } + if o == nil { + return nil, nil, nil, createOpts + } + leaseAccessConditions, modifiedAccessConditions := exported.FormatPathAccessConditions(o.AccessConditions) + if o.SourceAccessConditions != nil { + if o.SourceAccessConditions.SourceModifiedAccessConditions != nil { + sourceModifiedAccessConditions := &generated.SourceModifiedAccessConditions{ + SourceIfMatch: o.SourceAccessConditions.SourceModifiedAccessConditions.SourceIfMatch, + SourceIfModifiedSince: o.SourceAccessConditions.SourceModifiedAccessConditions.SourceIfModifiedSince, + SourceIfNoneMatch: o.SourceAccessConditions.SourceModifiedAccessConditions.SourceIfNoneMatch, + SourceIfUnmodifiedSince: o.SourceAccessConditions.SourceModifiedAccessConditions.SourceIfUnmodifiedSince, + } + createOpts.SourceLeaseID = o.SourceAccessConditions.SourceLeaseAccessConditions.LeaseID + return leaseAccessConditions, modifiedAccessConditions, sourceModifiedAccessConditions, createOpts + } + } + return leaseAccessConditions, modifiedAccessConditions, nil, createOpts +} + // GetPropertiesOptions contains the optional parameters for the Client.GetProperties method type GetPropertiesOptions struct { AccessConditions *AccessConditions @@ -98,6 +163,7 @@ func (o *GetPropertiesOptions) format() *blob.GetPropertiesOptions { } // ===================================== PATH IMPORTS =========================================== + // SetAccessControlOptions contains the optional parameters when calling the SetAccessControl operation. dfs endpoint type SetAccessControlOptions struct { // Owner is the owner of the path. @@ -114,10 +180,13 @@ type SetAccessControlOptions struct { func (o *SetAccessControlOptions) format() (*generated.PathClientSetAccessControlOptions, *generated.LeaseAccessConditions, *generated.ModifiedAccessConditions, error) { if o == nil { - return nil, nil, nil, nil + return nil, nil, nil, datalakeerror.MissingParameters } // call path formatter since we're hitting dfs in this operation leaseAccessConditions, modifiedAccessConditions := exported.FormatPathAccessConditions(o.AccessConditions) + if o.Owner == nil && o.Group == nil && o.ACL == nil && o.Permissions == nil { + return nil, nil, nil, errors.New("at least one parameter should be set for SetAccessControl API") + } return &generated.PathClientSetAccessControlOptions{ Owner: o.Owner, Group: o.Group, @@ -134,76 +203,43 @@ type GetAccessControlOptions struct { AccessConditions *AccessConditions } -func (o *GetAccessControlOptions) format() (*generated.PathClientGetPropertiesOptions, *generated.LeaseAccessConditions, *generated.ModifiedAccessConditions, error) { +func (o *GetAccessControlOptions) format() (*generated.PathClientGetPropertiesOptions, *generated.LeaseAccessConditions, *generated.ModifiedAccessConditions) { action := generated.PathGetPropertiesActionGetAccessControl if o == nil { return &generated.PathClientGetPropertiesOptions{ Action: &action, - }, nil, nil, nil + }, nil, nil } // call path formatter since we're hitting dfs in this operation leaseAccessConditions, modifiedAccessConditions := exported.FormatPathAccessConditions(o.AccessConditions) return &generated.PathClientGetPropertiesOptions{ Upn: o.UPN, Action: &action, - }, leaseAccessConditions, modifiedAccessConditions, nil -} - -// SetAccessControlRecursiveOptions contains the optional parameters when calling the SetAccessControlRecursive operation. TODO: Design formatter -type SetAccessControlRecursiveOptions struct { - // ACL is the access control list for the path. - ACL *string - // BatchSize is the number of paths to set access control recursively in a single call. - BatchSize *int32 - // MaxBatches is the maximum number of batches to perform the operation on. - MaxBatches *int32 - // ContinueOnFailure indicates whether to continue on failure when the operation encounters an error. - ContinueOnFailure *bool - // Marker is the continuation token to use when continuing the operation. - Marker *string + }, leaseAccessConditions, modifiedAccessConditions } -func (o *SetAccessControlRecursiveOptions) format() (*generated.PathClientSetAccessControlRecursiveOptions, error) { - // TODO: design formatter - return nil, nil +// UpdateAccessControlOptions contains the optional parameters when calling the UpdateAccessControlRecursive operation. +type UpdateAccessControlOptions struct { + //placeholder } -// UpdateAccessControlRecursiveOptions contains the optional parameters when calling the UpdateAccessControlRecursive operation. TODO: Design formatter -type UpdateAccessControlRecursiveOptions struct { - // ACL is the access control list for the path. - ACL *string - // BatchSize is the number of paths to set access control recursively in a single call. - BatchSize *int32 - // MaxBatches is the maximum number of batches to perform the operation on. - MaxBatches *int32 - // ContinueOnFailure indicates whether to continue on failure when the operation encounters an error. - ContinueOnFailure *bool - // Marker is the continuation token to use when continuing the operation. - Marker *string +func (o *UpdateAccessControlOptions) format(ACL string) (*generated.PathClientSetAccessControlRecursiveOptions, generated.PathSetAccessControlRecursiveMode) { + mode := generated.PathSetAccessControlRecursiveModeModify + return &generated.PathClientSetAccessControlRecursiveOptions{ + ACL: &ACL, + }, mode } -func (o *UpdateAccessControlRecursiveOptions) format() (*generated.PathClientSetAccessControlRecursiveOptions, error) { - // TODO: design formatter - similar to SetAccessControlRecursiveOptions - return nil, nil +// RemoveAccessControlOptions contains the optional parameters when calling the RemoveAccessControlRecursive operation. +type RemoveAccessControlOptions struct { + //placeholder } -// RemoveAccessControlRecursiveOptions contains the optional parameters when calling the RemoveAccessControlRecursive operation. TODO: Design formatter -type RemoveAccessControlRecursiveOptions struct { - // ACL is the access control list for the path. - ACL *string - // BatchSize is the number of paths to set access control recursively in a single call. - BatchSize *int32 - // MaxBatches is the maximum number of batches to perform the operation on. - MaxBatches *int32 - // ContinueOnFailure indicates whether to continue on failure when the operation encounters an error. - ContinueOnFailure *bool - // Marker is the continuation token to use when continuing the operation. - Marker *string -} - -func (o *RemoveAccessControlRecursiveOptions) format() (*generated.PathClientSetAccessControlRecursiveOptions, error) { - // TODO: design formatter - similar to SetAccessControlRecursiveOptions - return nil, nil +func (o *RemoveAccessControlOptions) format(ACL string) (*generated.PathClientSetAccessControlRecursiveOptions, generated.PathSetAccessControlRecursiveMode) { + mode := generated.PathSetAccessControlRecursiveModeRemove + return &generated.PathClientSetAccessControlRecursiveOptions{ + ACL: &ACL, + }, mode } // SetHTTPHeadersOptions contains the optional parameters for the Client.SetHTTPHeaders method. @@ -211,14 +247,22 @@ type SetHTTPHeadersOptions struct { AccessConditions *AccessConditions } -func (o *SetHTTPHeadersOptions) format() *blob.SetHTTPHeadersOptions { +func (o *SetHTTPHeadersOptions) format(httpHeaders HTTPHeaders) (*blob.SetHTTPHeadersOptions, blob.HTTPHeaders) { + httpHeaderOpts := blob.HTTPHeaders{ + BlobCacheControl: httpHeaders.CacheControl, + BlobContentDisposition: httpHeaders.ContentDisposition, + BlobContentEncoding: httpHeaders.ContentEncoding, + BlobContentLanguage: httpHeaders.ContentLanguage, + BlobContentMD5: httpHeaders.ContentMD5, + BlobContentType: httpHeaders.ContentType, + } if o == nil { - return nil + return nil, httpHeaderOpts } accessConditions := exported.FormatBlobAccessConditions(o.AccessConditions) return &blob.SetHTTPHeadersOptions{ AccessConditions: accessConditions, - } + }, httpHeaderOpts } // HTTPHeaders contains the HTTP headers for path operations. @@ -239,25 +283,24 @@ type HTTPHeaders struct { ContentType *string } -func (o *HTTPHeaders) formatBlobHTTPHeaders() (*blob.HTTPHeaders, error) { - if o == nil { - return nil, nil - } - opts := blob.HTTPHeaders{ - BlobCacheControl: o.CacheControl, - BlobContentDisposition: o.ContentDisposition, - BlobContentEncoding: o.ContentEncoding, - BlobContentLanguage: o.ContentLanguage, - BlobContentMD5: o.ContentMD5, - BlobContentType: o.ContentType, - } - return &opts, nil -} - -func (o *HTTPHeaders) formatPathHTTPHeaders() (*generated.PathHTTPHeaders, error) { +// +//func (o HTTPHeaders) formatBlobHTTPHeaders() blob.HTTPHeaders { +// +// opts := blob.HTTPHeaders{ +// BlobCacheControl: o.CacheControl, +// BlobContentDisposition: o.ContentDisposition, +// BlobContentEncoding: o.ContentEncoding, +// BlobContentLanguage: o.ContentLanguage, +// BlobContentMD5: o.ContentMD5, +// BlobContentType: o.ContentType, +// } +// return opts +//} + +func (o *HTTPHeaders) formatPathHTTPHeaders() *generated.PathHTTPHeaders { // TODO: will be used for file related ops, like append if o == nil { - return nil, nil + return nil } opts := generated.PathHTTPHeaders{ CacheControl: o.CacheControl, @@ -268,32 +311,36 @@ func (o *HTTPHeaders) formatPathHTTPHeaders() (*generated.PathHTTPHeaders, error ContentType: o.ContentType, TransactionalContentHash: o.ContentMD5, } - return &opts, nil + return &opts } // SetMetadataOptions provides set of configurations for Set Metadata on path operation type SetMetadataOptions struct { + Metadata map[string]*string AccessConditions *AccessConditions CPKInfo *CPKInfo CPKScopeInfo *CPKScopeInfo } -func (o *SetMetadataOptions) format() *blob.SetMetadataOptions { +func (o *SetMetadataOptions) format() (*blob.SetMetadataOptions, map[string]*string) { if o == nil { - return nil + return nil, nil } accessConditions := exported.FormatBlobAccessConditions(o.AccessConditions) - return &blob.SetMetadataOptions{ + opts := &blob.SetMetadataOptions{ AccessConditions: accessConditions, - CPKInfo: &blob.CPKInfo{ + } + if o.CPKInfo != nil { + opts.CPKInfo = &blob.CPKInfo{ EncryptionKey: o.CPKInfo.EncryptionKey, EncryptionAlgorithm: o.CPKInfo.EncryptionAlgorithm, EncryptionKeySHA256: o.CPKInfo.EncryptionKeySHA256, - }, - CPKScopeInfo: &blob.CPKScopeInfo{ - EncryptionScope: o.CPKScopeInfo.EncryptionScope, - }, + } + } + if o.CPKScopeInfo != nil { + opts.CPKScopeInfo = (*blob.CPKScopeInfo)(o.CPKScopeInfo) } + return opts, o.Metadata } // CPKInfo contains a group of parameters for the PathClient.Download method. @@ -303,43 +350,87 @@ type CPKInfo struct { EncryptionKeySHA256 *string } -// CPKScopeInfo contains a group of parameters for the PathClient.SetMetadata method. -type CPKScopeInfo struct { - EncryptionScope *string -} - -// UndeletePathOptions contains the optional parameters for the Filesystem.UndeletePath operation. -type UndeletePathOptions struct { - // placeholder +// GetSASURLOptions contains the optional parameters for the Client.GetSASURL method. +type GetSASURLOptions struct { + StartTime *time.Time } -func (o *UndeletePathOptions) format() *UndeletePathOptions { +func (o *GetSASURLOptions) format() time.Time { if o == nil { - return nil + return time.Time{} } - return &UndeletePathOptions{} + + var st time.Time + if o.StartTime != nil { + st = o.StartTime.UTC() + } else { + st = time.Time{} + } + return st +} + +// CreationExpiryType defines values for Create() ExpiryType +type CreationExpiryType interface { + Format() (generated.ExpiryOptions, *string) + notPubliclyImplementable() } -// SourceModifiedAccessConditions identifies the source path access conditions. -type SourceModifiedAccessConditions = generated.SourceModifiedAccessConditions +// CreationExpiryTypeAbsolute defines the absolute time for the blob expiry +type CreationExpiryTypeAbsolute time.Time + +// CreationExpiryTypeRelativeToNow defines the duration relative to now for the blob expiry +type CreationExpiryTypeRelativeToNow time.Duration + +// CreationExpiryTypeNever defines that the blob will be set to never expire +type CreationExpiryTypeNever struct { + // empty struct since NeverExpire expiry type does not require expiry time +} + +func (e CreationExpiryTypeAbsolute) Format() (generated.ExpiryOptions, *string) { + return generated.ExpiryOptionsAbsolute, to.Ptr(time.Time(e).UTC().Format(http.TimeFormat)) + +} + +func (e CreationExpiryTypeAbsolute) notPubliclyImplementable() {} + +func (e CreationExpiryTypeRelativeToNow) Format() (generated.ExpiryOptions, *string) { + return generated.ExpiryOptionsRelativeToNow, to.Ptr(strconv.FormatInt(time.Duration(e).Milliseconds(), 10)) +} + +func (e CreationExpiryTypeRelativeToNow) notPubliclyImplementable() {} + +func (e CreationExpiryTypeNever) Format() (generated.ExpiryOptions, *string) { + return generated.ExpiryOptionsNeverExpire, nil +} + +func (e CreationExpiryTypeNever) notPubliclyImplementable() {} + +// ACLFailedEntry contains the failed ACL entry (response model). +type ACLFailedEntry = generated.ACLFailedEntry + +// SetAccessControlRecursiveResponse contains part of the response data returned by the []OP_AccessControl operations. +type SetAccessControlRecursiveResponse = generated.SetAccessControlRecursiveResponse + +// CPKScopeInfo contains a group of parameters for the PathClient.SetMetadata method. +type CPKScopeInfo blob.CPKScopeInfo // SharedKeyCredential contains an account's name and its primary or secondary key. type SharedKeyCredential = exported.SharedKeyCredential -// ExpiryType defines values for ExpiryType. -type ExpiryType = exported.ExpiryType +// SetExpiryType defines values for ExpiryType. +type SetExpiryType = exported.SetExpiryType -// ExpiryTypeAbsolute defines the absolute time for the expiry. -type ExpiryTypeAbsolute = exported.ExpiryTypeAbsolute +// SetExpiryTypeAbsolute defines the absolute time for the expiry. +type SetExpiryTypeAbsolute = exported.SetExpiryTypeAbsolute -// ExpiryTypeRelativeToNow defines the duration relative to now for the expiry. -type ExpiryTypeRelativeToNow = exported.ExpiryTypeRelativeToNow +// SetExpiryTypeRelativeToNow defines the duration relative to now for the expiry. +type SetExpiryTypeRelativeToNow = exported.SetExpiryTypeRelativeToNow -// ExpiryTypeRelativeToCreation defines the duration relative to creation for the expiry. -type ExpiryTypeRelativeToCreation = exported.ExpiryTypeRelativeToCreation +// SetExpiryTypeRelativeToCreation defines the duration relative to creation for the expiry. +type SetExpiryTypeRelativeToCreation = exported.SetExpiryTypeRelativeToCreation -// ExpiryTypeNever defines that will be set to never expire. -type ExpiryTypeNever = exported.ExpiryTypeNever +// SetExpiryTypeNever defines that will be set to never expire. +type SetExpiryTypeNever = exported.SetExpiryTypeNever // SetExpiryOptions contains the optional parameters for the Client.SetExpiry method. type SetExpiryOptions = exported.SetExpiryOptions @@ -347,8 +438,14 @@ type SetExpiryOptions = exported.SetExpiryOptions // AccessConditions identifies blob-specific access conditions which you optionally set. type AccessConditions = exported.AccessConditions +// SourceAccessConditions identifies blob-specific access conditions which you optionally set. +type SourceAccessConditions = exported.SourceAccessConditions + // LeaseAccessConditions contains optional parameters to access leased entity. type LeaseAccessConditions = exported.LeaseAccessConditions // ModifiedAccessConditions contains a group of parameters for specifying access conditions. type ModifiedAccessConditions = exported.ModifiedAccessConditions + +// SourceModifiedAccessConditions contains a group of parameters for specifying access conditions. +type SourceModifiedAccessConditions = exported.SourceModifiedAccessConditions diff --git a/sdk/storage/azdatalake/file/responses.go b/sdk/storage/azdatalake/file/responses.go index 9222c5c042c3..2e6a3cecee2e 100644 --- a/sdk/storage/azdatalake/file/responses.go +++ b/sdk/storage/azdatalake/file/responses.go @@ -7,8 +7,10 @@ package file import ( + "github.com/Azure/azure-sdk-for-go/sdk/azcore" "github.com/Azure/azure-sdk-for-go/sdk/storage/azblob/blob" "github.com/Azure/azure-sdk-for-go/sdk/storage/azdatalake/internal/generated" + "time" ) // SetExpiryResponse contains the response fields for the SetExpiry operation. @@ -23,29 +25,55 @@ type DeleteResponse = generated.PathClientDeleteResponse // SetAccessControlResponse contains the response fields for the SetAccessControl operation. type SetAccessControlResponse = generated.PathClientSetAccessControlResponse -// SetAccessControlRecursiveResponse contains the response fields for the SetAccessControlRecursive operation. -type SetAccessControlRecursiveResponse = generated.PathClientSetAccessControlRecursiveResponse +// UpdateAccessControlResponse contains the response fields for the UpdateAccessControlRecursive operation. +type UpdateAccessControlResponse = generated.PathClientSetAccessControlRecursiveResponse -// UpdateAccessControlRecursiveResponse contains the response fields for the UpdateAccessControlRecursive operation. -type UpdateAccessControlRecursiveResponse = generated.PathClientSetAccessControlRecursiveResponse - -// RemoveAccessControlRecursiveResponse contains the response fields for the RemoveAccessControlRecursive operation. -type RemoveAccessControlRecursiveResponse = generated.PathClientSetAccessControlRecursiveResponse +// RemoveAccessControlResponse contains the response fields for the RemoveAccessControlRecursive operation. +type RemoveAccessControlResponse = generated.PathClientSetAccessControlRecursiveResponse // GetAccessControlResponse contains the response fields for the GetAccessControl operation. type GetAccessControlResponse = generated.PathClientGetPropertiesResponse // GetPropertiesResponse contains the response fields for the GetProperties operation. -type GetPropertiesResponse = generated.PathClientGetPropertiesResponse +type GetPropertiesResponse = blob.GetPropertiesResponse // SetMetadataResponse contains the response fields for the SetMetadata operation. type SetMetadataResponse = blob.SetMetadataResponse -// SetHTTPHeadersResponse contains the response fields for the SetHTTPHeaders operation. -type SetHTTPHeadersResponse = blob.SetHTTPHeadersResponse - // RenameResponse contains the response fields for the Create operation. type RenameResponse = generated.PathClientCreateResponse -// UndeletePathResponse contains the response from method FilesystemClient.UndeletePath. -type UndeletePathResponse = generated.PathClientUndeleteResponse +//// SetHTTPHeadersResponse contains the response fields for the SetHTTPHeaders operation. +//type SetHTTPHeadersResponse = blob.SetHTTPHeadersResponse + +// we need to remove the blob sequence number from the response + +// SetHTTPHeadersResponse contains the response from method Client.SetHTTPHeaders. +type SetHTTPHeadersResponse struct { + // ClientRequestID contains the information returned from the x-ms-client-request-id header response. + ClientRequestID *string + + // Date contains the information returned from the Date header response. + Date *time.Time + + // ETag contains the information returned from the ETag header response. + ETag *azcore.ETag + + // LastModified contains the information returned from the Last-Modified header response. + LastModified *time.Time + + // RequestID contains the information returned from the x-ms-request-id header response. + RequestID *string + + // Version contains the information returned from the x-ms-version header response. + Version *string +} + +func formatSetHTTPHeadersResponse(r *SetHTTPHeadersResponse, blobResp *blob.SetHTTPHeadersResponse) { + r.ClientRequestID = blobResp.ClientRequestID + r.Date = blobResp.Date + r.ETag = blobResp.ETag + r.LastModified = blobResp.LastModified + r.RequestID = blobResp.RequestID + r.Version = blobResp.Version +} diff --git a/sdk/storage/azdatalake/filesystem/client_test.go b/sdk/storage/azdatalake/filesystem/client_test.go index fa78bef404ad..3615506e870e 100644 --- a/sdk/storage/azdatalake/filesystem/client_test.go +++ b/sdk/storage/azdatalake/filesystem/client_test.go @@ -63,7 +63,7 @@ func validateFilesystemDeleted(_require *require.Assertions, filesystemClient *f _, err := filesystemClient.GetAccessPolicy(context.Background(), nil) _require.NotNil(err) - testcommon.ValidateBlobErrorCode(_require, err, datalakeerror.ContainerNotFound) + testcommon.ValidateErrorCode(_require, err, datalakeerror.ContainerNotFound) } func (s *RecordedTestSuite) TestCreateFilesystem() { @@ -158,7 +158,7 @@ func (s *RecordedTestSuite) TestFilesystemCreateInvalidName() { _, err = fsClient.Create(context.Background(), nil) _require.NotNil(err) - testcommon.ValidateBlobErrorCode(_require, err, datalakeerror.InvalidResourceName) + testcommon.ValidateErrorCode(_require, err, datalakeerror.InvalidResourceName) } func (s *RecordedTestSuite) TestFilesystemCreateNameCollision() { @@ -175,7 +175,7 @@ func (s *RecordedTestSuite) TestFilesystemCreateNameCollision() { _, err = fsClient.Create(context.Background(), nil) _require.NotNil(err) - testcommon.ValidateBlobErrorCode(_require, err, datalakeerror.FilesystemAlreadyExists) + testcommon.ValidateErrorCode(_require, err, datalakeerror.FilesystemAlreadyExists) } func (s *RecordedTestSuite) TestFilesystemGetProperties() { @@ -224,7 +224,7 @@ func (s *RecordedTestSuite) TestFilesystemDeleteNonExistent() { _, err = fsClient.Delete(context.Background(), nil) _require.NotNil(err) - testcommon.ValidateBlobErrorCode(_require, err, datalakeerror.ContainerNotFound) + testcommon.ValidateErrorCode(_require, err, datalakeerror.ContainerNotFound) } func (s *RecordedTestSuite) TestFilesystemDeleteIfModifiedSinceTrue() { @@ -275,7 +275,7 @@ func (s *RecordedTestSuite) TestFilesystemDeleteIfModifiedSinceFalse() { } _, err = fsClient.Delete(context.Background(), &deleteFilesystemOptions) _require.NotNil(err) - testcommon.ValidateBlobErrorCode(_require, err, datalakeerror.ConditionNotMet) + testcommon.ValidateErrorCode(_require, err, datalakeerror.ConditionNotMet) } func (s *RecordedTestSuite) TestFilesystemDeleteIfUnModifiedSinceTrue() { @@ -328,7 +328,7 @@ func (s *RecordedTestSuite) TestFilesystemDeleteIfUnModifiedSinceFalse() { _, err = fsClient.Delete(context.Background(), &deleteFilesystemOptions) _require.NotNil(err) - testcommon.ValidateBlobErrorCode(_require, err, datalakeerror.ConditionNotMet) + testcommon.ValidateErrorCode(_require, err, datalakeerror.ConditionNotMet) } func (s *RecordedTestSuite) TestFilesystemSetMetadataNonEmpty() { @@ -432,7 +432,7 @@ func (s *RecordedTestSuite) TestFilesystemSetMetadataNonExistent() { _, err = fsClient.SetMetadata(context.Background(), nil) _require.NotNil(err) - testcommon.ValidateBlobErrorCode(_require, err, datalakeerror.ContainerNotFound) + testcommon.ValidateErrorCode(_require, err, datalakeerror.ContainerNotFound) } func (s *RecordedTestSuite) TestSetEmptyAccessPolicy() { @@ -731,7 +731,7 @@ func (s *RecordedTestSuite) TestFilesystemSetPermissionsACLMoreThanFive() { _, err = fsClient.SetAccessPolicy(context.Background(), &setAccessPolicyOptions) _require.NotNil(err) - testcommon.ValidateBlobErrorCode(_require, err, datalakeerror.InvalidXMLDocument) + testcommon.ValidateErrorCode(_require, err, datalakeerror.InvalidXMLDocument) } func (s *RecordedTestSuite) TestFilesystemSetPermissionsDeleteAndModifyACL() { @@ -940,7 +940,7 @@ func (s *RecordedTestSuite) TestFilesystemSetPermissionsSignedIdentifierTooLong( _, err = fsClient.SetAccessPolicy(context.Background(), &setAccessPolicyOptions) _require.NotNil(err) - testcommon.ValidateBlobErrorCode(_require, err, datalakeerror.InvalidXMLDocument) + testcommon.ValidateErrorCode(_require, err, datalakeerror.InvalidXMLDocument) } func (s *RecordedTestSuite) TestFilesystemSetPermissionsIfModifiedSinceTrue() { @@ -992,7 +992,7 @@ func (s *RecordedTestSuite) TestFilesystemSetPermissionsIfModifiedSinceFalse() { _, err = fsClient.SetAccessPolicy(context.Background(), &setAccessPolicyOptions) _require.NotNil(err) - testcommon.ValidateBlobErrorCode(_require, err, datalakeerror.ConditionNotMet) + testcommon.ValidateErrorCode(_require, err, datalakeerror.ConditionNotMet) } func (s *RecordedTestSuite) TestFilesystemSetPermissionsIfUnModifiedSinceTrue() { @@ -1044,7 +1044,7 @@ func (s *RecordedTestSuite) TestFilesystemSetPermissionsIfUnModifiedSinceFalse() _, err = fsClient.SetAccessPolicy(context.Background(), &setAccessPolicyOptions) _require.NotNil(err) - testcommon.ValidateBlobErrorCode(_require, err, datalakeerror.ConditionNotMet) + testcommon.ValidateErrorCode(_require, err, datalakeerror.ConditionNotMet) } func (s *RecordedTestSuite) TestSetAccessPoliciesInDifferentTimeFormats() { @@ -1134,7 +1134,7 @@ func (s *RecordedTestSuite) TestSetAccessPolicyWithNullId() { options := filesystem.SetAccessPolicyOptions{FilesystemACL: signedIdentifiers} _, err = fsClient.SetAccessPolicy(context.Background(), &options) _require.NotNil(err) - testcommon.ValidateBlobErrorCode(_require, err, datalakeerror.InvalidXMLDocument) + testcommon.ValidateErrorCode(_require, err, datalakeerror.InvalidXMLDocument) resp1, err := fsClient.GetAccessPolicy(context.Background(), nil) _require.Nil(err) diff --git a/sdk/storage/azdatalake/filesystem/models.go b/sdk/storage/azdatalake/filesystem/models.go index c36a5acf1eb8..e0d6f6ce0b86 100644 --- a/sdk/storage/azdatalake/filesystem/models.go +++ b/sdk/storage/azdatalake/filesystem/models.go @@ -199,6 +199,18 @@ func (o *GetSASURLOptions) format() time.Time { return st } +// UndeletePathOptions contains the optional parameters for the Filesystem.UndeletePath operation. +type UndeletePathOptions struct { + // placeholder +} + +func (o *UndeletePathOptions) format() *UndeletePathOptions { + if o == nil { + return nil + } + return &UndeletePathOptions{} +} + // SharedKeyCredential contains an account's name and its primary or secondary key. type SharedKeyCredential = exported.SharedKeyCredential @@ -216,3 +228,6 @@ type PathList = generated.PathList // Path contains the path properties type Path = generated.Path + +// UndeletePathResponse contains the response from method FilesystemClient.UndeletePath. +type UndeletePathResponse = generated.PathClientUndeleteResponse diff --git a/sdk/storage/azdatalake/internal/base/clients.go b/sdk/storage/azdatalake/internal/base/clients.go index 3ebb6fc7ee69..c80152dc4b83 100644 --- a/sdk/storage/azdatalake/internal/base/clients.go +++ b/sdk/storage/azdatalake/internal/base/clients.go @@ -9,7 +9,7 @@ package base import ( "github.com/Azure/azure-sdk-for-go/sdk/azcore" "github.com/Azure/azure-sdk-for-go/sdk/azcore/runtime" - "github.com/Azure/azure-sdk-for-go/sdk/storage/azblob/blob" + "github.com/Azure/azure-sdk-for-go/sdk/storage/azblob/blockblob" "github.com/Azure/azure-sdk-for-go/sdk/storage/azblob/container" "github.com/Azure/azure-sdk-for-go/sdk/storage/azblob/service" "github.com/Azure/azure-sdk-for-go/sdk/storage/azdatalake/internal/exported" @@ -69,10 +69,10 @@ func NewServiceClient(serviceURL string, serviceURLWithBlobEndpoint string, clie } } -func NewPathClient(dirURL string, dirURLWithBlobEndpoint string, client *blob.Client, azClient *azcore.Client, sharedKey *exported.SharedKeyCredential, options *ClientOptions) *CompositeClient[generated.PathClient, generated.PathClient, blob.Client] { - return &CompositeClient[generated.PathClient, generated.PathClient, blob.Client]{ - innerT: generated.NewPathClient(dirURL, azClient), - innerK: generated.NewPathClient(dirURLWithBlobEndpoint, azClient), +func NewPathClient(pathURL string, pathURLWithBlobEndpoint string, client *blockblob.Client, azClient *azcore.Client, sharedKey *exported.SharedKeyCredential, options *ClientOptions) *CompositeClient[generated.PathClient, generated.PathClient, blockblob.Client] { + return &CompositeClient[generated.PathClient, generated.PathClient, blockblob.Client]{ + innerT: generated.NewPathClient(pathURL, azClient), + innerK: generated.NewPathClient(pathURLWithBlobEndpoint, azClient), sharedKey: sharedKey, innerU: client, options: options, diff --git a/sdk/storage/azdatalake/internal/exported/access_conditions.go b/sdk/storage/azdatalake/internal/exported/access_conditions.go index 974cb1ed628e..9e6fb3850bbd 100644 --- a/sdk/storage/azdatalake/internal/exported/access_conditions.go +++ b/sdk/storage/azdatalake/internal/exported/access_conditions.go @@ -12,12 +12,21 @@ type AccessConditions struct { LeaseAccessConditions *LeaseAccessConditions } +// SourceAccessConditions identifies container-specific access conditions which you optionally set. +type SourceAccessConditions struct { + SourceModifiedAccessConditions *SourceModifiedAccessConditions + SourceLeaseAccessConditions *LeaseAccessConditions +} + // LeaseAccessConditions contains optional parameters to access leased entity. type LeaseAccessConditions = generated.LeaseAccessConditions // ModifiedAccessConditions contains a group of parameters for specifying access conditions. type ModifiedAccessConditions = generated.ModifiedAccessConditions +// SourceModifiedAccessConditions contains a group of parameters for specifying access conditions of a source. +type SourceModifiedAccessConditions = generated.SourceModifiedAccessConditions + // FormatContainerAccessConditions formats FilesystemAccessConditions into container's LeaseAccessConditions and ModifiedAccessConditions. func FormatContainerAccessConditions(b *AccessConditions) *container.AccessConditions { if b == nil { diff --git a/sdk/storage/azdatalake/internal/exported/set_expiry.go b/sdk/storage/azdatalake/internal/exported/set_expiry.go index 041dd8ba60c0..2b8e156c315f 100644 --- a/sdk/storage/azdatalake/internal/exported/set_expiry.go +++ b/sdk/storage/azdatalake/internal/exported/set_expiry.go @@ -15,23 +15,23 @@ import ( "github.com/Azure/azure-sdk-for-go/sdk/storage/azdatalake/internal/generated" ) -// ExpiryType defines values for ExpiryType -type ExpiryType interface { +// SetExpiryType defines values for ExpiryType +type SetExpiryType interface { Format(o *SetExpiryOptions) (generated.ExpiryOptions, *generated.PathClientSetExpiryOptions) notPubliclyImplementable() } -// ExpiryTypeAbsolute defines the absolute time for the blob expiry -type ExpiryTypeAbsolute time.Time +// SetExpiryTypeAbsolute defines the absolute time for the blob expiry +type SetExpiryTypeAbsolute time.Time -// ExpiryTypeRelativeToNow defines the duration relative to now for the blob expiry -type ExpiryTypeRelativeToNow time.Duration +// SetExpiryTypeRelativeToNow defines the duration relative to now for the blob expiry +type SetExpiryTypeRelativeToNow time.Duration -// ExpiryTypeRelativeToCreation defines the duration relative to creation for the blob expiry -type ExpiryTypeRelativeToCreation time.Duration +// SetExpiryTypeRelativeToCreation defines the duration relative to creation for the blob expiry +type SetExpiryTypeRelativeToCreation time.Duration -// ExpiryTypeNever defines that the blob will be set to never expire -type ExpiryTypeNever struct { +// SetExpiryTypeNever defines that the blob will be set to never expire +type SetExpiryTypeNever struct { // empty struct since NeverExpire expiry type does not require expiry time } @@ -40,32 +40,32 @@ type SetExpiryOptions struct { // placeholder for future options } -func (e ExpiryTypeAbsolute) Format(o *SetExpiryOptions) (generated.ExpiryOptions, *generated.PathClientSetExpiryOptions) { +func (e SetExpiryTypeAbsolute) Format(o *SetExpiryOptions) (generated.ExpiryOptions, *generated.PathClientSetExpiryOptions) { return generated.ExpiryOptionsAbsolute, &generated.PathClientSetExpiryOptions{ ExpiresOn: to.Ptr(time.Time(e).UTC().Format(http.TimeFormat)), } } -func (e ExpiryTypeAbsolute) notPubliclyImplementable() {} +func (e SetExpiryTypeAbsolute) notPubliclyImplementable() {} -func (e ExpiryTypeRelativeToNow) Format(o *SetExpiryOptions) (generated.ExpiryOptions, *generated.PathClientSetExpiryOptions) { +func (e SetExpiryTypeRelativeToNow) Format(o *SetExpiryOptions) (generated.ExpiryOptions, *generated.PathClientSetExpiryOptions) { return generated.ExpiryOptionsRelativeToNow, &generated.PathClientSetExpiryOptions{ ExpiresOn: to.Ptr(strconv.FormatInt(time.Duration(e).Milliseconds(), 10)), } } -func (e ExpiryTypeRelativeToNow) notPubliclyImplementable() {} +func (e SetExpiryTypeRelativeToNow) notPubliclyImplementable() {} -func (e ExpiryTypeRelativeToCreation) Format(o *SetExpiryOptions) (generated.ExpiryOptions, *generated.PathClientSetExpiryOptions) { +func (e SetExpiryTypeRelativeToCreation) Format(o *SetExpiryOptions) (generated.ExpiryOptions, *generated.PathClientSetExpiryOptions) { return generated.ExpiryOptionsRelativeToCreation, &generated.PathClientSetExpiryOptions{ ExpiresOn: to.Ptr(strconv.FormatInt(time.Duration(e).Milliseconds(), 10)), } } -func (e ExpiryTypeRelativeToCreation) notPubliclyImplementable() {} +func (e SetExpiryTypeRelativeToCreation) notPubliclyImplementable() {} -func (e ExpiryTypeNever) Format(o *SetExpiryOptions) (generated.ExpiryOptions, *generated.PathClientSetExpiryOptions) { - return generated.ExpiryOptionsNeverExpire, nil +func (e SetExpiryTypeNever) Format(o *SetExpiryOptions) (generated.ExpiryOptions, *generated.PathClientSetExpiryOptions) { + return generated.ExpiryOptionsNeverExpire, &generated.PathClientSetExpiryOptions{} } -func (e ExpiryTypeNever) notPubliclyImplementable() {} +func (e SetExpiryTypeNever) notPubliclyImplementable() {} diff --git a/sdk/storage/azdatalake/internal/testcommon/clients_auth.go b/sdk/storage/azdatalake/internal/testcommon/clients_auth.go index 5988e217e87e..bd1607b2fec4 100644 --- a/sdk/storage/azdatalake/internal/testcommon/clients_auth.go +++ b/sdk/storage/azdatalake/internal/testcommon/clients_auth.go @@ -8,6 +8,7 @@ import ( "github.com/Azure/azure-sdk-for-go/sdk/azcore/to" "github.com/Azure/azure-sdk-for-go/sdk/internal/recording" "github.com/Azure/azure-sdk-for-go/sdk/storage/azdatalake" + "github.com/Azure/azure-sdk-for-go/sdk/storage/azdatalake/file" "github.com/Azure/azure-sdk-for-go/sdk/storage/azdatalake/filesystem" "github.com/Azure/azure-sdk-for-go/sdk/storage/azdatalake/service" "github.com/stretchr/testify/require" @@ -33,6 +34,23 @@ const ( var BasicMetadata = map[string]*string{"Foo": to.Ptr("bar")} +var ( + DatalakeContentType = "my_type" + DatalakeContentDisposition = "my_disposition" + DatalakeCacheControl = "control" + DatalakeContentLanguage = "my_language" + DatalakeContentEncoding = "my_encoding" +) + +var BasicHeaders = file.HTTPHeaders{ + ContentType: &DatalakeContentType, + ContentDisposition: &DatalakeContentDisposition, + CacheControl: &DatalakeCacheControl, + ContentMD5: nil, + ContentLanguage: &DatalakeContentLanguage, + ContentEncoding: &DatalakeContentEncoding, +} + type TestAccountType string const ( @@ -105,6 +123,23 @@ func GetFilesystemClient(fsName string, t *testing.T, accountType TestAccountTyp return filesystemClient, err } +func GetFileClient(fsName, fName string, t *testing.T, accountType TestAccountType, options *file.ClientOptions) (*file.Client, error) { + if options == nil { + options = &file.ClientOptions{} + } + + SetClientOptions(t, &options.ClientOptions) + + cred, err := GetGenericSharedKeyCredential(accountType) + if err != nil { + return nil, err + } + + fileClient, err := file.NewClientWithSharedKeyCredential("https://"+cred.AccountName()+".dfs.core.windows.net/"+fsName+"/"+fName, cred, options) + + return fileClient, err +} + func ServiceGetFilesystemClient(filesystemName string, s *service.Client) *filesystem.Client { return s.NewFilesystemClient(filesystemName) } @@ -114,6 +149,11 @@ func DeleteFilesystem(ctx context.Context, _require *require.Assertions, filesys _require.Nil(err) } +func DeleteFile(ctx context.Context, _require *require.Assertions, fileClient *file.Client) { + _, err := fileClient.Delete(ctx, nil) + _require.Nil(err) +} + func GetGenericConnectionString(accountType TestAccountType) (*string, error) { accountName, accountKey := GetGenericAccountInfo(accountType) if accountName == "" || accountKey == "" { diff --git a/sdk/storage/azdatalake/internal/testcommon/common.go b/sdk/storage/azdatalake/internal/testcommon/common.go index 71534c98af95..fcb537b98c52 100644 --- a/sdk/storage/azdatalake/internal/testcommon/common.go +++ b/sdk/storage/azdatalake/internal/testcommon/common.go @@ -23,6 +23,10 @@ func GenerateFilesystemName(testName string) string { return FilesystemPrefix + GenerateEntityName(testName) } +func GenerateFileName(testName string) string { + return FilePrefix + GenerateEntityName(testName) +} + func GenerateEntityName(testName string) string { return strings.ReplaceAll(strings.ReplaceAll(strings.ToLower(testName), "/", ""), "test", "") } @@ -62,7 +66,7 @@ func GetRequiredEnv(name string) (string, error) { } } -func ValidateBlobErrorCode(_require *require.Assertions, err error, code bloberror.Code) { +func ValidateErrorCode(_require *require.Assertions, err error, code bloberror.Code) { _require.NotNil(err) var responseErr *azcore.ResponseError errors.As(err, &responseErr) diff --git a/sdk/storage/azdatalake/service/client_test.go b/sdk/storage/azdatalake/service/client_test.go index c222bac865d8..a9541d4356a4 100644 --- a/sdk/storage/azdatalake/service/client_test.go +++ b/sdk/storage/azdatalake/service/client_test.go @@ -307,7 +307,7 @@ func (s *ServiceRecordedTestsSuite) TestAccountDeleteRetentionPolicyDaysTooLarge _, err = svcClient.SetProperties(context.Background(), &service.SetPropertiesOptions{DeleteRetentionPolicy: &service.RetentionPolicy{Enabled: &enabled, Days: &days}}) _require.NotNil(err) - testcommon.ValidateBlobErrorCode(_require, err, datalakeerror.InvalidXMLDocument) + testcommon.ValidateErrorCode(_require, err, datalakeerror.InvalidXMLDocument) } } @@ -321,7 +321,7 @@ func (s *ServiceRecordedTestsSuite) TestAccountDeleteRetentionPolicyDaysOmitted( _, err = svcClient.SetProperties(context.Background(), &service.SetPropertiesOptions{DeleteRetentionPolicy: &service.RetentionPolicy{Enabled: &enabled}}) _require.NotNil(err) - testcommon.ValidateBlobErrorCode(_require, err, datalakeerror.InvalidXMLDocument) + testcommon.ValidateErrorCode(_require, err, datalakeerror.InvalidXMLDocument) } func (s *ServiceRecordedTestsSuite) TestSASServiceClient() { @@ -495,7 +495,7 @@ func (s *ServiceRecordedTestsSuite) TestSASFilesystemClient() { _, err = fsClient2.Create(context.Background(), &filesystem.CreateOptions{Metadata: testcommon.BasicMetadata}) _require.NotNil(err) - testcommon.ValidateBlobErrorCode(_require, err, datalakeerror.AuthorizationFailure) + testcommon.ValidateErrorCode(_require, err, datalakeerror.AuthorizationFailure) } func (s *ServiceRecordedTestsSuite) TestSASFilesystem2() { @@ -526,7 +526,7 @@ func (s *ServiceRecordedTestsSuite) TestSASFilesystem2() { // filesystem metadata and properties can't be read or written with SAS auth _, err = fsClient1.GetProperties(context.Background(), nil) _require.Error(err) - testcommon.ValidateBlobErrorCode(_require, err, datalakeerror.AuthorizationFailure) + testcommon.ValidateErrorCode(_require, err, datalakeerror.AuthorizationFailure) start = time.Now().Add(-5 * time.Minute).UTC() opts = filesystem.GetSASURLOptions{StartTime: &start} @@ -540,7 +540,7 @@ func (s *ServiceRecordedTestsSuite) TestSASFilesystem2() { // filesystems can't be created, deleted, or listed with SAS auth _, err = fsClient2.Create(context.Background(), nil) _require.Error(err) - testcommon.ValidateBlobErrorCode(_require, err, datalakeerror.AuthorizationFailure) + testcommon.ValidateErrorCode(_require, err, datalakeerror.AuthorizationFailure) } func (s *ServiceRecordedTestsSuite) TestListFilesystemsBasic() {