Skip to content

Zip Files Service Tutorial¤

    In this tutorial, we will explore the usage of a file service for training and inference with file-related models. This service is specifically designed to handle large-scale files datasets, containing over 300 million filess. The files service operates in the backend and offers an HTTP API that allows Users & GPU processes to access file bytes by using a file ID.

Zip Files Service

Configuration files¤

Single zip folder¤

[core/cli]
zip_folder = zip_folder/
service_name = core/service/zip_files

[core/service/zip_files]
zip_folder = ${core/cli:zip_folder}
zip_extension = .zip

Multiple zip folders¤

[core/cli]
zip_folder1 = zip_folder1/
zip_folder2 = zip_folder2/
zip_folder3 = zip_folder3/
service_name = core/service/zip_files

[core/service/zip_files]
zip_folder = [
    "${core/cli:zip_folder1}",
    "${core/cli:zip_folder2}",
    "${core/cli:zip_folder3}"
  ]
zip_extension = .zip

Start Service¤

unitorch-service start path/to/zip/files/service.ini \
    --zip_folder path/to/zip/folder

Stop Service¤

unitorch-service stop path/to/zip/files/service.ini \
    --zip_folder path/to/zip/folder

Restart Service¤

unitorch-service restart path/to/zip/files/service.ini \
    --zip_folder path/to/zip/folder