Zip Files Service Tutorial¤
In this tutorial, we will explore the usage of a file service for training and inference with file-related models. This service is specifically designed to handle large-scale files datasets, containing over 300 million filess. The files service operates in the backend and offers an HTTP API that allows Users & GPU processes to access file bytes by using a file ID.
Configuration files¤
Single zip folder¤
[core/cli]
zip_folder = zip_folder/
service_name = core/service/zip_files
[core/service/zip_files]
zip_folder = ${core/cli:zip_folder}
zip_extension = .zip
Multiple zip folders¤
[core/cli]
zip_folder1 = zip_folder1/
zip_folder2 = zip_folder2/
zip_folder3 = zip_folder3/
service_name = core/service/zip_files
[core/service/zip_files]
zip_folder = [
"${core/cli:zip_folder1}",
"${core/cli:zip_folder2}",
"${core/cli:zip_folder3}"
]
zip_extension = .zip
Start Service¤
unitorch-service start path/to/zip/files/service.ini \
--zip_folder path/to/zip/folder
Stop Service¤
unitorch-service stop path/to/zip/files/service.ini \
--zip_folder path/to/zip/folder
Restart Service¤
unitorch-service restart path/to/zip/files/service.ini \
--zip_folder path/to/zip/folder