S3-API Related Operations¶
Prior Knowledge¶
When choosing Amazon S3 API1 as LeoGateway's protocol, you need LeoFS' S3-API related operations including Endpoint, User, and Bucket operations.
You can access a LeoFS system whose LeoGateway's protocol is S3-API by following the flow.
- Create an endpoint to be able to access a LeoFS system's resources.
- Create a user to separately manage the user's buckets and objects.
- After creating a bucket, If you need to change ACL of a bucket, you can set it as an administrator of a LeoFS system. The current version, v1.3 of LeoFS does not support
Operations¶
Endpoint¶
You can continue to use a LeoFS system's DNS name to access its resources after you've set up an endpoint, which has a policy that controls the use of the endpoint to access the resources.
Create an endpoint, add-endpoint¶
Creates an endpoint so users can access a LeoFS system's resources.
1 2 3 4 5 | $ leofs-adm add-endpoint <endpoint>
## Example
$ leofs-adm add-endpoint leo-project.net
OK
|
Remove an endpoint, delete-endpoint¶
Removes an endpoint if a LeoFS system's system does not need it.
1 2 3 4 5 | $ leofs-adm delete-endpoint <endpoint>
## Example
$ leofs-adm delete-endpoint leo-project.net
OK
|
Retrieve a list of endpoints, get-endpoints¶
Retrieves a list of the endpoints if you need to know the existing endpoints.
1 2 3 4 5 6 7 8 9 | $ leofs-adm get-endpoints ## Example $ leofs-adm get-endpoints endpoint | created at -----------------+--------------------------- leo-project.net | 2017-04-07 10:07:31 +0900 localhost | 2017-04-07 10:06:18 +0900 s3.amazonaws.com | 2017-04-07 10:06:18 +0900 |
User¶
To access a LeoFS system, its system's users need to create an account and retrieve AccessKeyID and SecretAccessKey.
Create a user, create-user¶
Creates a user's account so its user can access a LeoFS system.
1 2 3 4 5 6 | $ leofs-adm create-user <user-id> <password>
## Example
$ leofs-adm create-user leofs-user-1 PASSWORD
access-key-id: b5f0413d45855fcc055e
secret-access-key: b74f9ae91dd0de81f71e19f8d455a7c081f34c57
|
Remove a user, delete-user¶
Removes a user account if a LeoFS system does not need it.
1 2 3 4 5 | $ leofs-adm delete-user <user-id>
## Example
$ leofs-adm delete-user leofs-user-1
OK
|
Retrieve a list of users, get-users¶
Retrieves a list of the users if you need to know the existing users.
1 2 3 4 5 6 7 8 | $ leofs-adm get-users ## Example $ leofs-adm get-users user_id | role_id | access_key_id | created_at -------------+---------+------------------------+--------------------------- _test_leofs | 9 | 05236 | 2017-04-10 09:52:39 +0900 leofs-user-2 | 1 | b5f0413d45855fcc055e | 2017-04-10 09:54:35 +0900 |
Change role of a user, update-user-role¶
Updates a user's role if you need to change the role of a LeoFS system. You can choose a role from Administrator or General user.
| Role Id | Description |
|---|---|
| 1 | Administrator |
| 9 | General User |
1 2 3 4 5 6 7 8 9 10 11 12 13 | $ leofs-adm update-user-role <user-id> <role-id> ## Example $ leofs-adm update-user-role leofs-user-2 9 OK ## Confirm the latest list of the users $ leofs-adm get-users user_id | role_id | access_key_id | created_at -------------+---------+------------------------+--------------------------- _test_leofs | 9 | 05236 | 2017-04-10 09:52:39 +0900 leofs-user | 1 | 3c1d889cdd2088fb9482 | 2017-04-10 09:53:33 +0900 leofs-user-2 | 9 | b5f0413d45855fcc055e | 2017-04-10 09:54:35 +0900 |
Bucket¶
To manage objects under a bucket, you need to create a bucket with leofs-adm or any client for Amazon S3 such as s3cmd2 and DragonDisk3.
Create a bucket, add-bucket¶
Creates a bucket so a LeoFS system' users can manage resources under its bucket.
1 2 3 4 5 | $ leofs-adm add-bucket <bucket-name> <access-key-id> ## Example $ leofs-adm add-bucket backup 05236 OK |
Note: Creating bucket with the same name as one being removed
It's not possible to create bucket with the same name as one currently being removed. Deletion of all objects belonging to former bucket must complete before bucket with the same name can be created again.
Remove a bucket, delete-bucket¶
Removes a bucket if its user does not need it. The bucket name is removed right away, while objects from that bucket will be deleted in asynchronous fashion after that.
1 2 3 4 5 | $ leofs-adm delete-bucket <bucket-name> <access-key-id> ## Example $ leofs-adm delete-bucket backup 05236 OK |
Note: How many delete-bucket requests can be proceeded at once
8 delete-bucket requests can be proceeded at once and if there are over 8 ones then exceeded ones are pushed on a queue and will be proceeded later once the number of processing delete-bucket go down under 8.
Retrieve a list of the ongoing delete-buckets, delete-bucket-stats¶
Retrieves a list of the ongoing delete-buckets if you need to know the progress of the ongoing delete-buckets.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 | $ leofs-adm delete-bucket-stats <bucket-name> ## Example $ leofs-adm delete-bucket-stats - Bucket: test_2 node | state | date -----------------------------+------------------+----------------------------- [email protected] | pending | 2017-07-27 09:20:47 +0900 [email protected] | monitoring | 2017-07-27 09:21:21 +0900 [email protected] | monitoring | 2017-07-27 09:21:21 +0900 [email protected] | monitoring | 2017-07-27 09:21:22 +0900 - Bucket: test_4 node | state | date -----------------------------+------------------+----------------------------- [email protected] | finished | 2017-07-27 09:21:20 +0900 [email protected] | finished | 2017-07-27 09:21:18 +0900 [email protected] | finished | 2017-07-27 09:21:21 +0900 [email protected] | monitoring | 2017-07-27 09:21:10 +0900 ## Example (No ongoing delete-buckets) $ leofs-adm delete-bucket-stats [ERROR] Delete-bucket's stats not found |
The below table explains what each state means.
| State | Description |
|---|---|
| pending | Waiting on the queue. |
| enqueuing | Iterating all objects and enqueuing object identifiers into the queue if those belong to the bucket deleted. |
| monitoring | Iterating all items in the queue and deleting objects one by one. |
| finished | Finished delete-bucket and waiting for other storage nodes. |
Note: How long the delete-bucket records exist
Once the state of every storage node transit into finished, the delete-bucket records get deleted within several tens of seconds.
Reset a delete-bucket-stats record, reset-delete-bucket-stats¶
Reset a delete-bucket-stats record if you face some trouble that inconsistency between manager nodes and storage nodes happened and only way to solve this problem is to reset(delete) a delete-bucket-stats record.
1 2 3 4 5 | $ leofs-adm reset-delete-bucket-stats <bucket-name>
## Example
$ leofs-adm reset-delete-bucket-stats bucket
OK
|
Note: Removing inaccessible objects after reset-delete-bucket-stats
Since this operation aborts deletion of objects belonging to (already deleted) bucket, it might be a good idea to create and delete bucket with same name again after reset-delete-bucket-stats, to make sure that bucket deletion completes properly and no stale objects remain.
Retrieve a list of buckets, get-buckets¶
Retrieves a list of the buckets if you need to know the existing buckets.
1 2 3 4 5 6 7 | $ leofs-adm get-buckets ## Example $ leofs-adm get-buckets cluster id | bucket | owner | permissions | redundancy method | created at -------------+----------+-------------+------------------+------------------------------+--------------------------- leofs_1 | test | _test_leofs | Me(full_control) | copy, {n:1, w:1, r:1, d:1} | 2017-04-10 10:57:29 +0900 |
Retrieve a bucket, get-bucket¶
Retrieves a bucket if you need to know its bucket's information.
1 2 3 4 5 6 7 8 9 10 | $ leofs-adm get-bucket <access-key-id> ## Example $ leofs-adm get-bucket 05236 bucket | permissions | created at ---------+----------------------------------------+--------------------------- backup | Me(full_control), Everyone(read) | 2017-04-10 10:39:01 +0900 docs | Me(full_control), Everyone(read) | 2017-04-10 10:39:25 +0900 logs | Me(full_control), Everyone(read,write) | 2017-04-10 10:39:38 +0900 movie | Me(full_control) | 2017-04-10 10:39:45 +0900 |
Change an owner of a bucket, chown-bucket¶
Updates an owner of a bucket if you need to change its owner to someone else.
1 2 3 4 5 6 7 8 9 10 11 | $ leofs-adm chown-bucket <bucket> <access-key-id> ## Example $ leofs-adm chown-bucket test b5f0413d45855fcc055e OK ## Confirm the latest list of the buckets $ leofs-adm get-buckets cluster id | bucket | owner | permissions | redundancy method | created at -------------+----------+--------------+------------------+------------------------------+--------------------------- leofs_1 | test | leofs-user-2 | Me(full_control) | copy, {n:1, w:1, r:1, d:1} | 2017-04-10 10:57:29 +0900 |
Update ACL of a bucket, update-acl¶
Updates ACL of a bucket if you need to change it.
1 2 3 4 5 6 7 8 9 | $ leofs-adm update-acl <bucket> <access-key-id> <canned-ACL> ## Example $ leofs-adm update-acl test 05236 private ok $ leofs-adm update-acl test 05236 public-read ok $ leofs-adm update-acl test 05236 public-read-write ok |
Canned ACL¶
When using S3-API, LeoFS supports a set of predefined grants, known as canned ACLs. Each canned ACL has a predefined a set of grantees and permissions. The following table lists the set of canned ACLs and the associated predefined grants.
| Canned ACL | Applies To | Permissions Added To ACL |
|---|---|---|
| private | Bucket and object | [default] Owner gets FULL_CONTROL. No one else has access rights. |
| public-read | Bucket and object | Owner gets FULL_CONTROL. The AllUsers group gets READ access. |
| public-read-write | Bucket and object | Owner gets FULL_CONTROL. The AllUsers group gets READ and WRITE access. Granting this on a bucket is generally not recommended. |