Azure container instance failing due some issue while pulling the image

Question

I am trying to run an image on ACI, whenever it is trying to pull image as this pulling image "mysql@sha256:asakjvnankvaknaklfvkabjaoenla" the container is failing, whereas in command i have given image as mysql:latest.

az container create \
    --resource-group $ACI_PERS_RESOURCE_GROUP \
    --name xxxxxxx \
    --location eastus \
    --image mysql:latest \
    --dns-name-label xxxxxxxx \
    --environment-variables MYSQL_ROOT_PASSWORD=password@123 \
    --ports 3306 33060 \
    --azure-file-volume-account-name $ACI_PERS_STORAGE_ACCOUNT_NAME \
    --azure-file-volume-account-key $STORAGE_KEY \
    --azure-file-volume-share-name $ACI_PERS_SHARE_NAME2 \
    --azure-file-volume-mount-path /var/lib/mysql

The issue is when it try to pull image as mysql@sha256:asakjvnankvaknaklfvkabjaoenla, the continer fails to start while if the image pulled is pulling image "mysql:latest" than container works.

Attaching the pic for refrenece.

Not sure why this issue is happening

Command to run three containers

ACI_PERS_RESOURCE_GROUP=myresourcegroup
ACI_PERS_STORAGE_ACCOUNT_NAME=mydatabasest2501
ACI_PERS_LOCATION=eastus
ACI_PERS_SHARE_NAME=mysqlshare1
ACI_PERS_SHARE_NAME2=mysqlshare2
ACI_PERS_SHARE_NAME3=mysqlshare3


#I already have storage account so only creating fileshare


# Create the file share
az storage share create \
  --name $ACI_PERS_SHARE_NAME \
  --account-name $ACI_PERS_STORAGE_ACCOUNT_NAME
  

# Create the file share
az storage share create \
  --name $ACI_PERS_SHARE_NAME2 \
  --account-name $ACI_PERS_STORAGE_ACCOUNT_NAME


# Create the file share
az storage share create \
  --name $ACI_PERS_SHARE_NAME3 \
  --account-name $ACI_PERS_STORAGE_ACCOUNT_NAME  
 

 
  
  
  
echo $ACI_PERS_STORAGE_ACCOUNT_NAME

STORAGE_KEY=$(az storage account keys list --resource-group $ACI_PERS_RESOURCE_GROUP --account-name $ACI_PERS_STORAGE_ACCOUNT_NAME --query "[0].value" --output tsv)
echo $STORAGE_KEY  


az container create \
    --resource-group $ACI_PERS_RESOURCE_GROUP \
    --name contaier1 \
    --location eastus \
    --image mysql:latest \
    --dns-name-label uniqueddns1 \
    --environment-variables MYSQL_ROOT_PASSWORD=password@123 \
    --ports 3306 33060 \
    --azure-file-volume-account-name $ACI_PERS_STORAGE_ACCOUNT_NAME \
    --azure-file-volume-account-key $STORAGE_KEY \
    --azure-file-volume-share-name $ACI_PERS_SHARE_NAME \
    --azure-file-volume-mount-path /var/lib/mysql
    
    

az container create \
    --resource-group $ACI_PERS_RESOURCE_GROUP \
    --name contaier2 \
    --location eastus \
    --image mysql:latest \
    --dns-name-label uniqueddns2 \
    --environment-variables MYSQL_ROOT_PASSWORD=password@123 \
    --ports 3306 33060 \
    --azure-file-volume-account-name $ACI_PERS_STORAGE_ACCOUNT_NAME \
    --azure-file-volume-account-key $STORAGE_KEY \
    --azure-file-volume-share-name $ACI_PERS_SHARE_NAME2 \
    --azure-file-volume-mount-path /var/lib/mysql


az container create \
    --resource-group $ACI_PERS_RESOURCE_GROUP \
    --name contaier3 \
    --location eastus \
    --image mysql:latest \
    --dns-name-label uniqueddns3 \
    --environment-variables MYSQL_ROOT_PASSWORD=password@123 \
    --ports 3306 33060 \
    --azure-file-volume-account-name $ACI_PERS_STORAGE_ACCOUNT_NAME \
    --azure-file-volume-account-key $STORAGE_KEY \
    --azure-file-volume-share-name $ACI_PERS_SHARE_NAME3 \
    --azure-file-volume-mount-path /var/lib/mysql

well, look at the logs? image is pulled successfully, so why do you think it has something to do with that — 4c74356b41, Dec 11 '20 at 12:10
did you ever figure this out? I have the EXACT same problem. the cause is definitely related to the simultaneous creation of multiple containers using the **same image**. i create 7 container groups simultaneously, and 1-2 of them always pulls the container using a digest (`container@shah256:...`) vs the image tag (`container:0.0.5`). no idea why this occurs or why the container gets killed. fyi I have no file shares, and my container is in the ACR. I think maybe some sort of weird caching going on behind the scenes by ACI when the same image is pulled — riptusk331, Sep 22 '21 at 02:02

Charles Xu · Accepted Answer · 2020-12-14T09:21:45.103

1

The problem does not due to the reason that you think, it's a problem with the volume that mounts the Azure File Share to the existing folder which is the dependency of the MySQL. When you mount the File Share, it will Occlude the existing folder then all the files in this folder will not be accessible. Therefore MySQL will not work fine anymore by losing the dependencies. See more details here.

Mounting an Azure Files share to a container instance is similar to a Docker bind mount. Be aware that if you mount a share into a container directory in which files or directories exist, these files or directories are obscured by the mount and are not accessible while the container runs.

There are three ways to solve it. The first way is that not mount the File Share, but then you cannot persist the data of the database. The second way is to mount the File Share to a new folder that does not exist in the image and then copy the data to the new folder. The third way is that copy the dependencies into the File Share and then mount the File Share to the container, it will also work.

Update:

A possible solution is that you can push the MySQL image to an ACR and then use it to create ACI, in this way, all the things are the same, then maybe all ACI succeeds or fails.

edited Dec 14 '20 at 09:21

answered Dec 14 '20 at 02:08

Charles Xu

29,862
2
22
39

If the problem is volume mount than it should always fails right, isn't working sometimes and failing sometimes is an ambiguous behavior. The images i have attached are both with volume mount, the one is working and other isn't and file share for them is different also, i tried to deploy 3 container(all with volume mount) and two are working and only one is failing. – yatharth meena Dec 14 '20 at 04:45
@yatharthmeena Three containers? How do you configure them? Can you share more details? And what do you mean by sometimes? The same command sometimes works without error and sometimes does not work? – Charles Xu Dec 14 '20 at 06:12
I have added in question, there continer named container1 and container3 started normally and container2 i am facing the issue – yatharth meena Dec 14 '20 at 06:22
@yatharthmeena The only thing I find different when you run the image in local is that the ownership of the folder `mysql`. But others seem no problem. What does the log of the failure container show? – Charles Xu Dec 14 '20 at 07:11
@yatharthmeena It's wired. Can you share all words of the ACI killing event? – Charles Xu Dec 14 '20 at 08:06
Killing container with id . There is some 64 character alpha numeric string after id. – yatharth meena Dec 14 '20 at 08:17
@yatharthmeena If you do not mount the file share, then how about it? Also the same error sometimes? – Charles Xu Dec 14 '20 at 08:22
haven't tried that multiple times but till now not an issue like that – yatharth meena Dec 14 '20 at 08:23
with volume mount, what i observed is this issue only come when image pulled is something like this :: pulling image "mysql@sha256:alcalkdicakhulalcliuqckahcac" – yatharth meena Dec 14 '20 at 08:27
@yatharthmeena Where do you observe it? It always works fine on my side. – Charles Xu Dec 14 '20 at 08:31
Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/225921/discussion-between-yatharth-meena-and-charles-xu). – yatharth meena Dec 14 '20 at 08:43
@yatharthmeena Any more updates on this question? Does it solve your problem? If yes, please accept it. – Charles Xu Dec 15 '20 at 08:07

Azure container instance failing due some issue while pulling the image

1 Answers1

Linked