Our S3 storage supports lifecycle policy.
Lifecycle configuration is an XML file with a set of rules which define files expiration time. Once the files expire, they are removed from the storage.
There are different ways to set lifecycle configuration on a bucket. Read more on them here.
Lifecycle configuration elements
Lifecycle configuration is an XML file:
Inside the <Rule>,</Rules> tags the following elements should be specified:
- Metadata: ID and status,
- Prefix (leave empty since the policy is applied to the content of the bucket),
- Action (file expiration time).
Each rule must have a unique identifier. ID length is limited to 255 characters. Numbers, Latin letter and underscores are allowed.
The status value can be either Enabled or Disabled. If a rule is Disabled, no actions specified in the rule will be performed.
The policy is applied to the content of the entire bucket so you can leave the Prefix tag empty.
Specify the number of days the file should be kept in your bucket.
Example: Adding lifecycle configuration with aws cli
Create an XML file:
- ID — one_day,
- the rule is applied to the entire bucket (<Prefix/>),
- the rule is active (status Enabled),
- the file expiration time is 1 day.
If you are using aws cli to add configuration to your bucket, you won't be able to upload an XML file so you should convert it to JSON first (save it as lifecycle.json):
Important! The following header's value ("Prefix": "") applies a lifecycle configuration to a whole bucket. If you need to apply the policy to a specific folder in a bucket, specify the folder. For example, if files are in deleteme / [files ...] folder, the value will be: "Prefix": "deleteme /"
Start the aws cli from a directory with the lifecycle.json file and run the command below. Don't forget to replace my_bucket name in the example with your bucket name. To choose the correct value for the --endpoint-url parameter use this article.
$aws s3api put-bucket-lifecycle --bucket my_bucket --lifecycle-configuration file://lifecycle.json --endpoint-url=https://s-ed1.cloud.gcore.lu
To check if the policy is uploaded correctly use the following command:
$aws s3api get-bucket-lifecycle-configuration --bucket my_bucket --endpoint-url=https://s-ed1.cloud.gcore.lu
In the response you should get the uploaded JSON file:
To remove the policy from the bucket use this command:
aws s3api delete-bucket-lifecycle --bucket my-bucket --endpoint-url=https://s-ed1.cloud.gcore.lu
Lifecycle policy logic
The file removal process starts around midnight (UTC timezone).
Lifecycle policy is set after the files are uploaded
If you upload a file on January, 1st (any time), then set lifecycle policy with a file expiration time of 1 day, the file will be deleted on January, 3rd at around midnight.
Lifecycle policy is set before the files are uploaded
If the lifecycle policy is already set for 1 day and you upload your file on January, 2nd, the file will be removed on January, 4th at around midnight.