Storing Files with CarrierX

CarrierX provides means to store files within CarrierX infrastructure. This is done to more easily save the recordings of your conferences, voicemails, and FlexML-related recordings and files.

Storage Structure

All the files within CarrierX infrastructure are stored inside containers. We provide free 1 GB storage per container to each partner when the partner is created.

tip

By default, CarrierX allows the partners to have only two containers per account. Please contact our support team at support@carrierx.com if you need more storage containers.

General Structure

The typical storage structure looks like this:

Here, each partner has a container associated with their account. The partner uses this container for storing files (either created by endpoints or uploaded). Some endpoints can also create and store files in this container, e.g., Voicemail, FlexML, or Conference. Refer to the sections below for more information on how to use CarrierX storage with each of the endpoint types.

tip

You can extend the limits of the CarrierX storage size if you use one of the external storage services. Refer to the section below to learn more about this.

Storage Objects Explained

API objects used with CarrierX storage include Container and File objects. The Container object is associated with the partner account via the partner_sid attribute (used in the Partner and Container objects), and the File object is connected to the parent Container object via the container_sid attribute:

The Endpoint object is also associated with the partner account via the partner_sid attribute.

You can also use specific containers with some of the endpoints. To do that, you need to additionally specify the container_sid attribute for the endpoint of your choice in its properties attribute. Once you do this, the endpoint will have access to the container you specify and will be able to save files to it, or access files by their secure ID (the file_sid attribute of the File object). If you do not do that, the endpoint will save files to the default container that your account has associated with it.

note

In FlexML endpoints, you can also use the containers and files with the FlexML applications. Refer to the FlexML Endpoint section below for more information on the relations between FlexML applications and files.

Each of the supported endpoints (FlexML, Conference, Conference Playback, and Voicemail) has some specifics when you associate it with the containers. Refer to the sections below for more details.

FlexML Endpoint

Normally, FlexML endpoints use the containers to store the files recorded using the FlexML Record verb. You need to either associate the container secure ID (the container_sid attribute) with the FlexML endpoint, or specify it using the containerSid attribute of the Record verb (in this case, you must do this every time you use the Record verb in your FlexML application syntax). Either way, the files will be stored to the container when the FlexML application records the call.

tip

Set the container_sid to the FlexML endpoint properties.container_sid attribute.

FlexML also allows you to use the files stored with CarrierX with the FlexML Play verb. You can either use the links to the files stored in containers directly inside the Play verb (the publish_uri attribute of the File object), or use the file secure ID (the file_sid attribute of the File object) with the File noun nested inside the Play verb.

note

If you prefer to use the publish_uri attribute of the File object, do not forget to set the publish attribute of the File object to either file_access_name or file_sid, otherwise the file will be inaccessible to your FlexML application.

The FlexML application will playback the audio file you specify to the caller.

Conference and Conference Playback Endpoints

If you use a standalone Conference endpoint, it follows the general storage structure to store the files and recordings: the system associates the Partner object with the containers and files, and the same Partner object is associated with the Conference endpoint. The endpoint properties contain the reference to the container secure ID and it allows saving the endpoint-related files to the associated container.

But when it comes to the Conference Playback endpoint, you need some additional steps to make sure that the Conference Playback endpoint has the access to the files which contain the recordings of the Conference endpoint:

The Conference Playback endpoint does not have a container_sid attribute in its properties. Therefore, you cannot directly associate a Conference Playback endpoint with a container so that it could get access to the recordings files.

That is why the Conference Playback endpoint properties attribute contains the reference to the Conference endpoint (the conference_endpoint_sid attribute) to which files it must have access. Only the referenced Conference endpoint has direct access to the container with the recordings.

Voicemail Endpoint

A Voicemail endpoint also follows the general storage structure to store the recordings: the system associates the Partner object with the containers and files, and the same Partner object is associated with the Voicemail endpoint. The endpoint properties contain the reference to the container secure ID and it allows saving the endpoint-related files to the associated container.

You can additionally use the container to store not only the voicemail recordings, but also greeting files which you can use to greet the callers. To link the greeting, use the publish_uri attribute of the File object and specify it in the greeting attribute of the Voicemail endpoint properties.

note

Do not forget to set the publish attribute of the File object to either file_access_name or file_sid, otherwise the file will be inaccessible to your Voicemail endpoint.

The Voicemail endpoint will playback the audio file you specify to the callers.

Uploading and Downloading Files

CarrierX storage server allows the users both to upload and download files using the API requests.

If you need additional control of the file, which might include range requests, content or cache control, etc., access the file in the container using either the file secure ID—the file_sid attribute (FlexML applications have such an ability, refer to the section above for more details on this) or the publicly accessible publish_uri link.

Uploading Files to Container

To upload a file to CarrierX storage, you need the file itself and the secure ID of the container where you are going to store the file.

Create Container

If you do not have a container, you can create it using the following POST request:

curl -X POST \
'https://api.carrierx.com/core/v2/storage/containers' \
-H 'Content-Type: application/json' \
--data-binary '{}' \
-H 'Authorization: Bearer 5ebc03d6-8b2b-44ad-bf65-72d4f1491dda'

note

You do not need to specify any container attributes, and you can send an empty Container object.

A successful request will return a 200 status code with the serialized copy of the created Container object.

Upload Files

CarrierX supports two ways to upload a file to a container using a POST request:

1. Using multipart/form-data content type.

   If you use this method, you need to pass the File object parameters as the file_object form argument. The only required parameter here is container_sid. It defines the secure ID of the container to which the file must be uploaded.

   You need to pass the file itself as the data form argument. It must contain the valid path to the file to be uploaded.

   curl -X POST \
   'https://api.carrierx.com/core/v2/storage/files' \
   -H 'Content-Type: multipart/form-data' \
   -F 'file_object={"container_sid":"cb05c1f4-2398-432c-8314-cc306156281c"}' \
   -F 'data=@my_audio.mp3' \
   -H 'Authorization: Bearer 5ebc03d6-8b2b-44ad-bf65-72d4f1491dda'

2. Using application/octet-stream content type.

   If you prefer to use this method, you need to pass the File object parameters as query arguments. The only required parameter here is also container_sid. It defines the secure ID of the container to which the file must be uploaded.

   You need to pass the file itself as raw data. It must contain the valid path to the file to be uploaded.

   curl -X POST \
   'https://api.carrierx.com/core/v2/admin/storage/files?container_sid=cb05c1f4-2398-432c-8314-cc306156281c&name=my_audio.mp3' \
   -H 'Content-Type: application/octet-stream' \
   --data-binary '@my_audio.mp3' \
   -H 'Authorization: Bearer 5ebc03d6-8b2b-44ad-bf65-72d4f1491dda'

note

If you do not specify the file name, it will be uploaded without any extension (file type) and with a N/A file name. The MIME type will also be set to application/octet-stream. If you want to set it to one of the multimedia types, be sure to either specify the appropriate mime_type or name attribute.

A successful request for any of the above methods will return a 201 status code with the serialized copy of the created File object.

After that, you can access the file using either the file_sid or the publish_uri attribute, depending on your need or the application settings. Refer to the sections above for more information on how different endpoints access the files.

note

If you want the file to be publicly inaccessible via the internet, set the File object publish attribute value to no.

Downloading Files

There are several ways to download the files from CarrierX storage.

1. You can get either information about the file, or the download the file data using API requests.

   • To get the detailed information about the file, use the following GET request, which will return a 200 response together with a serialized copy of the File object:

   curl -X GET \
   'https://api.carrierx.com/core/v2/storage/files/f6070be2-9abd-4726-870c-c86c9aac5c7e' \
   -H 'Authorization: Bearer 5ebc03d6-8b2b-44ad-bf65-72d4f1491dda'
   • To download the file data, use the following GET request, which will return a 200 response and a download of the file data:

   curl -X GET \
   'https://api.carrierx.com/core/v2/storage/files/f6070be2-9abd-4726-870c-c86c9aac5c7e/data' \
   -H 'Authorization: Bearer 5ebc03d6-8b2b-44ad-bf65-72d4f1491dda'

2. Neither of the above requests to the API supports range requests. Use the publicly available link to the file (the file publish_uri attribute) when you need to request only a portion of it:

   curl -X GET \
   'https://storage.carrierx.com/f/6e930ad8-c3d0-49db-801e-86d64fedd2c2' \
   -H 'Range: bytes=0-499'

   Refer to the section below for more information about range requests to files in CarrierX storage server.

Range Requests

Range requests are partial requests which allow you to request and playback only a portion of the file from any particular place (e.g., from the fourth to twenty-first second, etc.) This might be useful when searching a specific place in the media file, as you can playback the file from a certain place. If range requests are not supported, the software will download the whole file and the playback will always start from the beginning of the file.

When you store a file in one format but an application requests automatic transcoding during download, and tries to do range requests at the same time, it might cause issues. If the system does not have a full cached copy of this file locally already transcoded, it will return the entire file with a 200 response code and the playback will start from the beginning of the file, not from the place the application requires.

Range requests are supported for the so called "streaming" formats. Such formats do not contain any metadata required for the file playback at the end of the file so the applications do not need to completely download them before they start the playback and they can start the playback from any position in the file after they download only the required portion of the file. Unlike them, the so called "downloadable" file formats contain some metadata required for the correct file playback at the end of the file.

When you need range requests to receive only a portion of a file, use the direct request to the CarrierX storage server instead of the API requests. You can do this via the publicly available link to the file (the file publish_uri attribute).

curl -X GET \
'https://storage.carrierx.com/f/6e930ad8-c3d0-49db-801e-86d64fedd2c2' \
-H 'Range: bytes=20-499'

To confirm that CarrierX storage server supports range request, it returns the following Accept-Ranges header to the client software:

Accept-Ranges: bytes

In its turn, the application that you use to download the portions of the files must specify which part of the file it wants to receive. The application can do this with the help of the Range header:

Range: bytes=20-499

The response from the server will include a similar Content-Range header in the case of the successful range request.

The complete response from the server for a range request to the file will look like the following:

HTTP/1.1 206 PARTIAL CONTENT
Accept-Ranges: bytes
Access-Control-Allow-Methods: GET
Access-Control-Allow-Origin: *
Access-Control-Expose-Headers: Access-Control-Allow-Methods,Access-Control-Allow-Origin,Content-Disposition
Cache-Control: None
Connection: close
Content-Length: 480
Content-Range: bytes 20-499/189952
Content-Type: audio/G722
Content-Disposition: inline; filename*=UTF-8''recording.g722
Date: Tue, 01 Jun 2021 14:21:07 GMT
Last-Modified: Tue, 25 May 2021 13:12:44 GMT
Server: nginx

• The HTTP/1.1 206 PARTIAL CONTENT status code confirms that only the requested portion of the file was downloaded (the complete file download would return a 200 status code).
• The Accept-Ranges: bytes header informs the client that the server accepts range requests.
• The Content-Length: 480 header shows the downloaded content length in bytes.
• The Content-Range: bytes 20-499/189952 header shows the bytes range which was downloaded and the complete file size in bytes (after the slash).
• The Access-Control-Allow-Origin: * header shows that CarrierX storage server supports the access from any origin.

You can additionally use the If-Range client-side header together with the Range header to make sure that the file was not modified between the requests. This header will compare its value against the server Last-Modified file header value.

curl -X GET \
'https://storage.carrierx.com/f/6e930ad8-c3d0-49db-801e-86d64fedd2c2' \
-H 'Range: bytes=20-499' \
-H 'If-Range: Tue, 25 May 2021 13:12:44 GMT'

If the values are equal, the server will accept the range request and will return the 206 response together with the partial content download:

HTTP/1.1 206 PARTIAL CONTENT
Accept-Ranges: bytes
Content-Range: bytes 20-499/189952
Last-Modified: Tue, 25 May 2021 13:12:44 GMT

Otherwise, if the Last-Modified value changes between the requests, the server will return the 200 response together with the complete file download. No Content-Range header will be present in the server response in this case:

HTTP/1.1 200 OK
Accept-Ranges: bytes
Last-Modified: Mon, 05 Apr 2021 14:51:53 GMT

Additional Parameters

To control your files while downloading, you can use additional parameters for your requests. CarrierX web storage supports request parameters to control the response server-side content-related headers (Content-Disposition, Content-Type) and cache-related headers (Cache-Control).

Content Headers

You can set the content headers on the server side and the server will respond with the disposition type and file name you specify in the download request.

disposition and disposition_name Parameters

The disposition and disposition_name parameters set the server Content-Disposition header to the desired value.

They are provided as request query parameters:

curl -X GET \
'https://storage.carrierx.com/f/6e930ad8-c3d0-49db-801e-86d64fedd2c2?disposition=attachment&disposition_name=my_file.wav'

If the request is successful, the server will return the modified Content-Disposition header in the response:

Content-Disposition: attachment; filename*=UTF-8''my_file.wav

Values accepted for the disposition parameter are:

• inline if you want the server to return the file inline (e.g., if your application requires the file to be displayed as a part of a webpage).
• attachment if you want the server to return the file as an attachment that your application can download and save locally.

The default value for the disposition parameter is inline.

content_type Parameter

The content_type parameter sets the server Content-Type header to the desired value. The Content-Type header is normally used to tell the application what the content type of the returned file actually is.

It is provided as a request query parameter:

curl -X GET \
'https://storage.carrierx.com/f/6e930ad8-c3d0-49db-801e-86d64fedd2c2?content_type=audio/G722'

If the request is successful, the server will return the modified Content-Type header in the response:

Content-Type: audio/G722

Values accepted for the content_type parameter for various formats are:

Format	Content-Type
al, alaw	audio/basic
flac	audio/flac
g722	audio/G722
mp3	audio/mpeg
mp4	video/mp4
ogg	audio/ogg
ul, ulaw	audio/basic
wav	audio/x-wav

Cache Headers

CarrierX storage server uses cache headers to control the file caching mechanism. The client application can change the server caching policies and define its own caching policies.

By default, when you use none of the cache parameters in the request, the server will return the Cache-Control header with the value set to None, meaning that it uses no cache control:

Cache-Control: None

cache_headers Parameter

The cache_headers parameter sets the server Cache-Control header to the desired value.

It is provided as a request query parameter:

curl -X GET \
'https://storage.carrierx.com/f/6e930ad8-c3d0-49db-801e-86d64fedd2c2?cache_headers=3600'

If the request is successful, the server will return the modified Cache-Control header in the response:

Cache-Control: public, max-age=3600

Values accepted for the cache_headers parameter are:

• no to set the Cache-Control header to no-cache, meaning that the file will not be cached without first checking if there is an updated version of it on the server.
• an integer value for the number of seconds to set the Cache-Control header to public and define the cache maximum amount of time during which the file is considered fresh and is not requested anew (the max-age value).

cache_control Parameter

The cache_control parameter also sets the server Cache-Control header to the desired value.

It is provided as a request query parameter:

curl -X GET \
'https://storage.carrierx.com/f/6e930ad8-c3d0-49db-801e-86d64fedd2c2?cache_control=min-fresh=3600'

If the request is successful, the server will return the modified Cache-Control header in the response:

Cache-Control: min-fresh=3600

Values accepted for the cache_control parameter are:

• max-age=<seconds> to indicate the maximum amount of time in seconds during which the file is considered fresh and is not requested anew.
• max-stale or max-stale=<seconds> to indicate that the client will accept a stale response. An optional value in seconds indicates the upper limit of staleness the client will accept.
• min-fresh=<seconds> to indicate that the client wants a response that will still be fresh for at least the specified number of seconds.
• no-cache to indicate that the file will not be cached without first checking if there is an updated version of it on the server.
• no-store to indicate that the client is not allowed to cache the file and must check for it on the server each time.
• no-transform to indicate that an intermediate cache or proxy cannot edit the response body, Content-Encoding, Content-Range, or Content-Type headers.
• only-if-cached to indicate that the cache must only respond using a stored response.

note

If you use both cache_headers and cache_control parameters in the request simultaneously, the cache_headers parameter will have higher priority and will override the values of the cache_control parameter.

Transcoding Parameters

CarrierX web storage allows its users to send requests with special parameters which will instruct the server to transcode the files into either a different format or to change its bitrate on-the-fly during downloading.

These parameters include the f (format) and br (bitrate) parameters. Refer to the Transcoding Files During Download section for the detailed information on this.

Transcoding Files

CarrierX allows uploading and downloading the files not in the file original format, but in a format which the system supports for conversion or transcoding. It might be useful when you need to store the file in the container in a format different from that in which the file was originally created. You can also use on-the-fly transcoding when you download a file from the container but your application needs to receive it in some other format.

CarrierX supports automatic transcoding from and to most popular media formats used with telephony and conferencing during upload or during download. The table below shows the codecs that the system can transcode the files from and to:

Format	Transcode from Codec	Transcode to Codec
al, alaw	yes	yes
flac	yes	yes
g722	yes	yes
mp3	yes	yes
mp4	no	yes
ogg	yes	yes
ul, ulaw	yes	yes
wav	yes	yes

Transcoding Files During Upload

When you upload a file to the container and if you do not specify any additional parameters, the file will be uploaded without any format changes. In most cases it is okay, but there can be situations (especially with the media files), when we recommend that you store the file in the format that is going to be served by the application.

Example

For example, if you plan to broadcast the file to a conference, please store the file in the g722 format.

In most cases, it is not necessary to reformat or transcode media files prior to uploading them. With media files, you can select the format in which the file will be stored with CarrierX storage. In this case, it will be transcoded to the selected format during the upload, on the fly.

note

To avoid issues when using range requests, we recommend that you store the file in the format you (or the application you use) will want to retrieve it in. In this case, range requests will always be supported, even on the first request of a file. Refer to the Range Requests section for more information on this.

You can transcode the file during upload using the File object desired_format attribute like this:

curl -X POST \
'https://api.carrierx.com/core/v2/storage/files' \
-F 'file_object={"container_sid":"cb05c1f4-2398-432c-8314-cc306156281c","type":"audio","desired_format":"g722"}' \
-F 'data=@my_audio.mp3' \
-H 'Authorization: Bearer 5ebc03d6-8b2b-44ad-bf65-72d4f1491dda'

warning

Please pay attention to the "type":"audio" parameter sent in the request when transcoding the file to another format. If you do not specify it, the file will not be transcoded and will be uploaded in the original format.

Transcoding from one format to another is useful not only for the sake of compatibility, but also when you want to reduce the file size. For example, files in the wav format can take considerable storage space when stored and the network bandwidth when accessed via the internet. It might be not too good on slower connections. Transcoding them to, for example, the mp3 format will dramatically reduce the file size and network bandwidth.

To further reduce the file size when uploading mp3 files and not transcoding them to any other format, you can set the File object desired_bitrate to a smaller value. You can set this field to one of these values: 8, 16, 24, 32, 40, 48, 64, 80, 96, 112, 128, 160, 192, 224, 256, 320.

note

Setting a too small desired_bitrate value can result in sound deterioration.

You can change the file bitrate during upload like this (do not forget to add the "type":"audio" parameter when sending the POST request):

curl -X POST \
'https://api.carrierx.com/core/v2/storage/files' \
-F 'file_object={"container_sid":"cb05c1f4-2398-432c-8314-cc306156281c","type":"audio","desired_bitrate":"32"}' \
-F 'data=@my_audio.mp3' \
-H 'Authorization: Bearer 5ebc03d6-8b2b-44ad-bf65-72d4f1491dda'

Transcoding Files During Download

When you download the file from CarrierX storage service, you can also transcode it into a necessary format and change its bitrate.

You can transcode the files into another format using the f query parameter:

curl -X GET \
'https://storage.carrierx.com/f/6e930ad8-c3d0-49db-801e-86d64fedd2c2?f=mp3'

This parameter accepts the values from the Transcode to Codec column of the table here (al, alaw, flac, g722, mp3, mp4, ogg, ul, ulaw, and wav).

To change the file bitrate, use the br query parameter when downloading the file:

curl -X GET \
'https://storage.carrierx.com/f/6e930ad8-c3d0-49db-801e-86d64fedd2c2?br=64

You can set the br parameter to one of these values: 8, 16, 24, 32, 40, 48, 64, 80, 96, 112, 128, 160, 192, 224, 256, 320.

note

The bitrate value you provide with the br query parameter must be lower than that of the file in the storage, otherwise it will be ignored.

File Classifications

Partners (and sometimes their clients) can upload different types of files to the containers. To protect the containers from the harmful or unwanted file types, CarrierX introduced the file classifications option. It serves to block certain file types or allow only files of certain types.

The file classification works the following way:

• The Container objects have two attributes: allowed_classifications and blocked_classifications. Both can contain lists of available file classifications values (archive for *.zip, *.tar, and other archive files, document for *.doc, *.docx, and other document files, executable for *.exe and other executable or binary files, media for audio, video, and image files, and unknown for the rest of the files).

• You can allow or block any types of the files (e.g., only allow media file classifications and block all the rest—"allowed_classifications": ["media"], or allow most of the file classifications and block only executable files—"blocked_classifications": ["executable"]).

• The File object has a content_classification attribute. This attribute can have one of the values from the above list.

• When you upload a new file, you can either specify the file classification manually, or the system will try and guess the file classification based on its extension, mime type, and some other properties.

  • If the file classification is present in the list of the container blocked classifications or absent from the list of the container allowed classifications, it will cause the 422 error response and will not be uploaded.

  • Otherwise, if you set no restrictions on the container or the file is in the container allowed classifications list (not in the blocked classifications list), it will be uploaded to the container.

Unique Files and Containers

If for some reason you want to make sure that you store unique files in the containers, you can use the unique attribute of the File object. If you set it to true, the file with such an attribute must have the unique key set.

The key set is the total of the values of the following six attributes:

• integer_key_1
• integer_key_2
• string_key_1
• string_key_2
• string_key_3
• string_key_4

This key set must be unique for each file that has the unique attribute set to true (i.e., at least one of the mentioned key values for two unique files must differ and be unique):

If you try to upload a file with the unique attribute set to true and the same file key set, which has already been used with some other file, a 422 (object validation error) error will be returned and the file will not be uploaded:

{
    "message":"object validation error",
    "errors":[
        {
            "field":"unique",
            "message":"file with the same set of keys already exists",
            "reference_sid":"f6070be2-9abd-4726-870c-c86c9aac5c7e"
        }
    ]
}

Refer to the File Object Specific Errors section for more information about the error structure and your possible actions to avoid it.

The same rule applies to the unique containers: to create a unique container all the other containers with the unique attribute set to true must have a different key set (the container key set also consists of the integer_key_1, integer_key_2, string_key_1, string_key_2, string_key_3, and string_key_4 attributes).

warning

Please set the unique attribute value when the Container or File object is initially created to avoid errors.

File Management

CarrierX allows its partners to set the file upload threshold and automatically manage the uploaded files when this threshold is reached.

The following three options are available:

1. Set no threshold.

2. Set a threshold and reject uploading files when the set threshold is exceeded.

3. Set a threshold and run cleanup jobs when the set threshold is exceeded.

Set No Threshold

When you do not set a threshold, you can upload files to the container until either the quota_files or quota_bytes value is reached. After that the container object will return errors if you try to upload new files.

note

The same behavior is when you specify a threshold (either threshold_bytes or threshold_files) but set the threshold_action attribute to the no_action value. The container will continue to accept files until it is full, ignoring the threshold.

This is the default option, you do not need to set any threshold attributes for it.

Set Threshold and Reject Exceeding Files

You can set a file upload threshold (either the number of files or their size in bytes). Once this threshold is exceeded, no new files will be uploaded until you delete some of the previously uploaded files.

To enable this option, do the following:

• Set the container threshold_action attribute to reject.
• Set the value for either threshold_bytes or threshold_files. If you choose to set both attributes, the threshold will be considered exceeded when either or the values is reached.

note

Please note, that the threshold_bytes and threshold_files attributes values must be less or equal than those of the quota_bytes/ quota_files for the correct work.

• Set the value for the threshold_grace_period attribute measured in seconds. This will allow uploading files to the container during the time specified when the threshold is exceeded. It might be useful when the system transcodes the file uploaded to the container on the fly, and both the original file and its transcoded copy occupy space in the container. Once the transcoding is finished, the system deletes the original file freeing the storage space.
• Set the threshold_include attribute of the files you upload to the container to true. Only these files will be counted as included into the total size or number of the files which affect the container threshold.

To set the threshold with the reject action for an existing container, send the following PATCH request:

curl -X PATCH \
'https://api.carrierx.com/core/v2/storage/containers/a291fe99-43d6-446f-a63e-91f93030dc7e' \
-H 'Content-Type: application/json' \
--data-binary '{"threshold_action":"reject","threshold_files":90,"threshold_grace_period":300}' \
-H 'Authorization: Bearer 5ebc03d6-8b2b-44ad-bf65-72d4f1491dda'

This will update the existing container and set the threshold action to reject. The grace period for the uploaded files will be equal to 300 seconds (5 minutes) during which the system will allow uploading files to the container even when the threshold of 90 files is exceeded.

This means that the number of files that you will be able to upload to the container might be 91 or 92 or even more if they all are uploaded during the 5 minutes following the moment when the threshold is reached. You can delete these files manually later if you need to.

After the grace period is also exceeded, all the files will be rejected from uploading to the container.

You can control the current threshold status using the Container object threshold_total_bytes/ threshold_total_files or the threshold_available_bytes_percent/ threshold_available_files_percent attribute values. These attributes will show you the current size or number of the files which affect the threshold, or the percentage of the bytes/files remaining until the threshold is exceeded.

Set Threshold and Delete Files Automatically

If you periodically upload files to the container and do not need them after some time, or you want to automatically get rid of the files which are no longer accessed and used, you can use the storage cleanup jobs to do that for you.

tip

A cleanup job is a cron job which runs every several minutes and checks if there are files in the the containers which must be deleted.

Set a threshold and, once it is exceeded, the cleanup job will detect and delete some of the least recently used (LRU) files freeing the storage space for new files.

To enable this option, do the following:

• Set the container threshold_action attribute to delete_lru.
• Set the value for either threshold_bytes or threshold_files. If you choose to set both attributes, the threshold will be considered exceeded when either or the values is reached.
• Set the value for threshold_bytes_target/ threshold_files_target which will determine the size or number of files which the cleanup must target.
• Set the threshold_include attribute of the files you upload to the container to true. Only these files will be counted as included into the total size or number of the files which affect the container threshold.

To set the threshold with the delete_lru action for an existing container, send the following PATCH request:

curl -X PATCH \
'https://api.carrierx.com/core/v2/storage/containers/a291fe99-43d6-446f-a63e-91f93030dc7e' \
-H 'Content-Type: application/json' \
--data-binary '{"threshold_action":"delete_lru","threshold_files":90,"threshold_files_target":70}' \
-H 'Authorization: Bearer 5ebc03d6-8b2b-44ad-bf65-72d4f1491dda'

This will update the existing container and set the threshold action to delete_lru. When the number of files in the container reaches 90, the storage cleanup job deletes the least recently used files (the ones with the oldest date_last_accessed attribute value) until their number is equal to 70.

warning

The system updates the date_last_accessed attribute value only using the Download Data by File SID method. The date_last_accessed attribute of the files accessed or downloaded using the direct link (the file publish_uri attribute) will not be updated. If you want to keep these files, set their threshold_include attribute to false to exclude them from cleanup.

If you want, you can also control the current threshold status using the Container object threshold_total_bytes/ threshold_total_files or the threshold_available_bytes_percent/ threshold_available_files_percent attribute values. These attributes will show you the current size or number of the files which affect the threshold, or the percentage of the bytes/files remaining until the threshold is exceeded.

In practice, this is not necessary as all the management is performed automatically in this case.

Using External Storage Services

If you prefer to store the files with one of the cloud services (AWS S3, Google Cloud Platform, or Microsoft Azure), CarrierX allows you to do this instead of using the standard containers hosted with CarrierX. You can also choose to use the external storage services if you want to avoid the CarrierX limitation to 1 GB of storage per container.

warning

When you specify one of the external storage services as a container, the following limitations are applied:

• The system will not check for the external container limits (containers quantity limit, total space limit).
• You cannot upload files to the external containers using CarrierX API or portal.

As is mentioned above, the following external storage services are supported for containers:

• Amazon Simple Storage Service (AWS S3)
• Google Cloud Platform (GCP)
• Microsoft Azure Blob storage

You will need to provide the credentials and specify the storage location for each of the external storage used. You can use only one external storage at a time for a single container.

For this, set the container type to either aws (for Amazon Simple Storage Service), or azure (for Microsoft Azure Blob storage), or gcp (for Google Cloud Platform).

Refer to the Supported External Storage section of the Core API documentation for more information on the parameters you need to specify for each storage type.

Further Reading

Container and File Objects API Reference

Refer to the Container and File objects API reference to get the complete list of their attributes and methods used with them:

Container Object

• Container Object
• Create Container
• Get Containers
• Update Container
• Delete Container
• Supported External Storage

File Object

• File Object
• Upload File
• Get Files
• Download File Data
• Update File
• Delete File